Skip to content

Commit 6c25344

Browse files
ryzhykryzhyk
committed
[adapters] Support ingesting raw data with metadata.
Fixes #5184 With this fix it's now possible to ingest raw data into a table with multiple columns as long as all other columns are nullable or have default values. The other columns can be used to store connector metadata. Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
1 parent b7165be commit 6c25344

11 files changed

Lines changed: 551 additions & 127 deletions

File tree

‎crates/adapterlib/src/catalog.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ pub enum RecordFormat {
4242
Parquet(SqlSerdeConfig),
4343
#[cfg(feature = "with-avro")]
4444
Avro,
45-
Raw,
45+
Raw(String),
4646
}
4747

4848
/// An input handle that deserializes and buffers records.

‎crates/adapters/src/format/json/input.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -541,7 +541,7 @@ impl JsonSplitter {
541541
mod test {
542542
use crate::{
543543
format::{InputBuffer, Parser},
544-
test::{init_test_logger, kafka::TestStructMetadata, mock_parser_pipeline, MockUpdate},
544+
test::{data::TestStructMetadata, init_test_logger, mock_parser_pipeline, MockUpdate},
545545
transport::InputConsumer,
546546
FormatConfig, ParseError,
547547
};

0 commit comments

Comments
 (0)