Follow-up to #7552.
We now support displaying a progress bar in to_dataframe when the tabledata.list API is used, but not when the bqstorage_client is used. The BigQuery Storage API is faster, but not instantaneous, so it would still be useful to display a progress bar.
There are a couple of complications with this.
- When the BigQuery Storage API is used, a threadpool is used to download query results. The progress bar should only be updated from a single thread, so the threadsafe queue should probably be used from the worker threads to send updates to the main thread that updates the progress bar.
to_dataframe in the BigQuery Storage API client only supports downloading a whole stream at once. The reader needs to be updated to provide a pages property to allow converting individual blocks to a pandas DataFrame. That way incremental updates can be sent.
Follow-up to #7552.
We now support displaying a progress bar in
to_dataframewhen the tabledata.list API is used, but not when thebqstorage_clientis used. The BigQuery Storage API is faster, but not instantaneous, so it would still be useful to display a progress bar.There are a couple of complications with this.
to_dataframein the BigQuery Storage API client only supports downloading a whole stream at once. The reader needs to be updated to provide apagesproperty to allow converting individual blocks to a pandas DataFrame. That way incremental updates can be sent.