Skip to content

Commit ac1dfbd

Browse files
swanandxswanandx
committed
[adapters]: parallel delta output encoder
Use SplitCursor to split the batch and distribute it across tasks, each task retries encoding and writing to delta lake and then returns Add actions which main task retries to commit to delta lake Signed-off-by: Swanand Mulay <73115739+swanandx@users.noreply.github.com>
1 parent 91c6514 commit ac1dfbd

4 files changed

Lines changed: 913 additions & 338 deletions

File tree

0 commit comments

Comments
 (0)