Commit c9b2dd3

committed

adapters: kafka: support AWS MSK IAM SASL

Uses the `OAUTHBEARER` `sasl.mechanism` to support IAM based authentication and SASL / SSL. There are some todos like inserting timeouts while trying to get the bearer token, unsure what a good timeout period would be. Before fetching the metadata, it is important to call `poll` to set the bearer token, this is currently done in a hacky way outside the `count_partitions_in_topic` method. This should be improved. Also, the code to fetch the bearer token across different contexts is repeated and identical. We should extract this out in one place. More importantly, it might cause a regression in end to end performance with Kafka. ``` "sasl.mechanism": "OAUTHBEARER", "security.protocol": "SASL_SSL", ``` Signed-off-by: Abhinav Gyawali <22275402+abhizer@users.noreply.github.com>

1 parent a6443b2 commit c9b2dd3Copy full SHA for c9b2dd3

9 files changed

.gitignore
Cargo.lock
crates/adapters
- Cargo.toml
- src/transport/kafka
  - ft
  - nonft
    - output.rs
docs/connectors
- sinks
  - kafka.md
- sources
  - kafka.md

`‎.gitignore‎`

-Original file line number
+Diff line change
 .DS_Store
 sql-to-dbsp-compiler/SQL-compiler/pom.xml-E
 +crates/dbsp-enterprise
++
 **/benches/galen_data/
 **/benches/gdelt-data/
 **/benches/ldbc-graphalytics-data/

Comments

(0)