Skip to content

Commit 103c5e9

Browse files
authored
fix: Support arro3 table schema with newer deltalake packages (#5799)
* fix: Support arro3 table schema with newer deltalake packages Signed-off-by: Anton Shtarev <a.o.shtarev@gmail.com> * Fix lint Signed-off-by: Anton Shtarev <a.o.shtarev@gmail.com> --------- Signed-off-by: Anton Shtarev <a.o.shtarev@gmail.com>
1 parent e484c12 commit 103c5e9

File tree

1 file changed

+12
-5
lines changed

1 file changed

+12
-5
lines changed

sdk/python/feast/infra/offline_stores/file_source.py

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -202,11 +202,18 @@ def get_table_column_names_and_types(
202202
"AWS_ENDPOINT_URL": str(self.s3_endpoint_override),
203203
}
204204

205-
schema = (
206-
DeltaTable(self.path, storage_options=storage_options)
207-
.schema()
208-
.to_pyarrow()
209-
)
205+
delta_schema = DeltaTable(
206+
self.path, storage_options=storage_options
207+
).schema()
208+
if hasattr(delta_schema, "to_arrow"):
209+
# deltalake >= 0.10.0
210+
arro3_schema = delta_schema.to_arrow()
211+
schema = pyarrow.schema(arro3_schema)
212+
elif hasattr(delta_schema, "to_pyarrow"):
213+
# deltalake < 0.10.0
214+
schema = delta_schema.to_pyarrow()
215+
else:
216+
raise Exception("Unknown DeltaTable package version")
210217
else:
211218
raise Exception(f"Unknown FileFormat -> {self.file_format}")
212219

0 commit comments

Comments
 (0)