Repository navigation
[Draft] AVRO module - #208
Draft
GoodforGod wants to merge 7 commits into
Draft
GoodforGod wants to merge 7 commits into
GoodforGod wants to merge 7 commits into
Conversation
Test Results691 tests 687 ✅ 27m 0s ⏱️ Results for commit 7123165. ♻️ This comment has been updated with latest results. |
GoodforGod
force-pushed
the
feature/avro
branch
from
January 9, 2025 06:12
d77333a to
8a74bfc
Compare
GoodforGod
marked this pull request as draft
January 18, 2025 11:16
Dependency Update ReportUpdate level: Found
|
- Restore Avro integration around a single @avro binary format and generate AvroReader/AvroWriter implementations automatically from AP and KSP extensions. - Move Kafka Avro serializer/deserializer support into kafka module, wire it to shared generated Avro mappers, and add Confluent Schema Registry wire format handling with a shared SchemaRegistryClient provided by optional kafka-schema-registry module. - Remove legacy Avro JSON/Binary split, drop obsolete avro-module, centralize new dependencies in the version catalog, and update tests for the new @avro model.
GoodforGod
force-pushed
the
feature/avro
branch
from
August 18, 2026 14:29
7123165 to
d18b1cd
Compare
… serdes and correct schema resolution. Added a dedicated kafka-avro module providing Confluent Schema Registry serializers and deserializers for both SpecificRecord and GenericRecord, and moved all Kafka+Avro code out of the base kafka module, which no longer depends on Avro or Confluent. Deserialization now resolves the writer schema from the registry by the schema id embedded in the payload and performs Avro writer-to-reader resolution, so decoding stays correct across schema evolution. Avro serdes, registry configuration, and telemetry are wired through the new KafkaAvroModule. - Fixed(kafka-avro): the deserializer resolves the writer schema from Schema Registry by id and applies writer-to-reader schema resolution; previously the id was ignored and payloads were decoded with the reader schema, producing wrong results once the producer schema evolved. - Added(kafka-avro): SpecificRecord and GenericRecord serializers/deserializers in the Confluent wire format with per-subject schema-id caching, configurable subject-name strategy (topic, record, topic-record) and auto-register via KafkaAvroSerializerConfig. - Added(kafka-schema-registry): typed basic-auth and bearer-auth configuration for the Schema Registry client, merged into the client properties. - Added(avro): standard logical-type conversions (decimal, uuid, date, time, timestamp, local-timestamp) on the generic reader/writer path, so logical types round-trip as their Java representations. - Added(kafka-avro): Avro registry telemetry for lookup latency, schema-cache hit/miss, and serde errors, backed by Micrometer through a factory/impl telemetry structure and falling back to no-op when no MeterRegistry is present. - Improved(avro): Breaking: AvroReader exposes only unchecked read methods throwing UncheckedIOException, adds getSchema() and read(Schema writerSchema, ...), and caches a datum reader per writer schema. - Improved(kafka-ap-ksp): the generated Kafka publisher now calls Serializer.configure() with the driver properties and the key/value flag, so subject naming and auto-register settings take effect. - Removed(kafka-avro): the @avro tag requirement on AvroReader and AvroWriter injection points, since the type is already self-describing; @avro is kept on the ambiguous Serializer and Deserializer beans. The change restores a coherent Avro integration around the Confluent wire format, isolates the Kafka+Avro concern in its own module, and closes the schema-evolution correctness gap that made the previous serdes unsafe against a Schema Registry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AVRO module with readers/writers added