Skip to content

Commit b77a740

Browse files
committed
fix: build Spark DataFrame from Arrow with schema and empty handling(#5594)
Signed-off-by: Jacob Weinhold <29459386+jfw-ppi@users.noreply.github.com>
1 parent f507b7c commit b77a740

File tree

1 file changed

+1
-1
lines changed
  • sdk/python/feast/infra/compute_engines/spark

1 file changed

+1
-1
lines changed

sdk/python/feast/infra/compute_engines/spark/nodes.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,6 @@ def execute(self, context: ExecutionContext) -> DAGValue:
9090
else:
9191
spark_df = self.spark_session.createDataFrame(arrow_table.to_pandas())
9292

93-
9493
return DAGValue(
9594
data=spark_df,
9695
format=DAGFormat.SPARK,
@@ -103,6 +102,7 @@ def execute(self, context: ExecutionContext) -> DAGValue:
103102
},
104103
)
105104

105+
106106
class SparkAggregationNode(DAGNode):
107107
def __init__(
108108
self,

0 commit comments

Comments
 (0)