Skip to content

[Draft] AVRO module - #208

Draft
GoodforGod wants to merge 7 commits into
masterfrom
feature/avro
Draft

GoodforGod wants to merge 7 commits into
masterfrom
feature/avro

Conversation

@GoodforGod

Copy link
Copy Markdown
Contributor

AVRO module with readers/writers added

@GoodforGod GoodforGod added the new module Feature where new module is added label Dec 16, 2024
@github-actions

github-actions Bot commented Dec 16, 2024 •

Copy link
Copy Markdown

Test Results

691 tests   687 ✅  27m 0s ⏱️
 98 suites    4 💤
 98 files      0 ❌

Results for commit 7123165.

♻️ This comment has been updated with latest results.

@GoodforGod
GoodforGod requested a review from Squiry December 16, 2024 16:26
@GoodforGod
GoodforGod marked this pull request as draft January 18, 2025 11:16
@GoodforGod GoodforGod changed the title AVRO module [Draft] AVRO module Jan 18, 2025
@GoodforGod GoodforGod added this to the v2.0.0 milestone Aug 13, 2025
@github-actions

github-actions Bot commented Jul 16, 2026 •

Copy link
Copy Markdown

Dependency Update Report

Update level: patch

Found 10 dependency updates.

gradle/libs.versions.toml

  • jackson: 3.2.1 -> 3.2.2
  • byte-buddy: 1.18.11 -> 1.18.12
  • avro: 1.12.0 -> 1.12.2
  • zeebe: 8.9.15 -> 8.9.16
  • logback: 1.6.2 -> 1.6.3
  • jackson2-coreline: 2.22.1 -> 2.22.2
  • swagger-coreline: 2.2.53 -> 2.2.54
  • swagger-parser: 2.1.46 -> 2.1.47
  • classgraph (io.github.classgraph:classgraph, inline:174): 4.8.191 -> 4.8.192

buildSrc/build.gradle

  • line 7: tools.jackson.core:jackson-databind (jackson-databind) 3.2.1 -> 3.2.2

- Restore Avro integration around a single @avro binary format and generate
AvroReader/AvroWriter implementations automatically from AP and KSP extensions.
- Move Kafka Avro serializer/deserializer support into kafka module, wire it to
shared generated Avro mappers, and add Confluent Schema Registry wire format
handling with a shared SchemaRegistryClient provided by optional
kafka-schema-registry module.
- Remove legacy Avro JSON/Binary split, drop obsolete avro-module, centralize
new dependencies in the version catalog, and update tests for the new @avro
model.
… serdes and correct schema resolution.

Added a dedicated kafka-avro module providing Confluent Schema Registry serializers and deserializers for both SpecificRecord and GenericRecord, and moved all Kafka+Avro code out of the base kafka module, which no longer depends on Avro or Confluent. Deserialization now resolves the writer schema from the registry by the schema id embedded in the payload and performs Avro writer-to-reader resolution, so decoding stays correct across schema evolution. Avro serdes, registry configuration, and telemetry are wired through the new KafkaAvroModule.

- Fixed(kafka-avro): the deserializer resolves the writer schema from Schema Registry by id and applies writer-to-reader schema resolution; previously the id was ignored and payloads were decoded with the reader schema, producing wrong results once the producer schema evolved.
- Added(kafka-avro): SpecificRecord and GenericRecord serializers/deserializers in the Confluent wire format with per-subject schema-id caching, configurable subject-name strategy (topic, record, topic-record) and auto-register via KafkaAvroSerializerConfig.
- Added(kafka-schema-registry): typed basic-auth and bearer-auth configuration for the Schema Registry client, merged into the client properties.
- Added(avro): standard logical-type conversions (decimal, uuid, date, time, timestamp, local-timestamp) on the generic reader/writer path, so logical types round-trip as their Java representations.
- Added(kafka-avro): Avro registry telemetry for lookup latency, schema-cache hit/miss, and serde errors, backed by Micrometer through a factory/impl telemetry structure and falling back to no-op when no MeterRegistry is present.
- Improved(avro): Breaking: AvroReader exposes only unchecked read methods throwing UncheckedIOException, adds getSchema() and read(Schema writerSchema, ...), and caches a datum reader per writer schema.
- Improved(kafka-ap-ksp): the generated Kafka publisher now calls Serializer.configure() with the driver properties and the key/value flag, so subject naming and auto-register settings take effect.
- Removed(kafka-avro): the @avro tag requirement on AvroReader and AvroWriter injection points, since the type is already self-describing; @avro is kept on the ambiguous Serializer and Deserializer beans.

The change restores a coherent Avro integration around the Confluent wire format, isolates the Kafka+Avro concern in its own module, and closes the schema-evolution correctness gap that made the previous serdes unsafe against a Schema Registry.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new module Feature where new module is added

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant