Current there's a to_dataframe() method that returns a pandas DataFrame from a query. DataFrames don't efficiently support array and struct values, but pyarrow provides efficient support for them:
In [11]: import pyarrow as pa
In [13]: a = pa.array(
...: [
...: {'a': [1, 2, 3]}
...: ],
...: pa.struct([pa.field('a', pa.list_(pa.int64()))])
...: )
In [14]: a
Out[14]:
<pyarrow.lib.StructArray object at 0x7f55f8764b88>
[
{'a': [1, 2, 3]}
]
A to_arrow() method will make it easier to efficiently support more complex types going forward in downstream libraries like ibis.
Current there's a
to_dataframe()method that returns a pandasDataFramefrom a query.DataFrames don't efficiently support array and struct values, butpyarrowprovides efficient support for them:A
to_arrow()method will make it easier to efficiently support more complex types going forward in downstream libraries like ibis.