Skip to content

BigQuery: order not preserved when downloading ORDER BY query results to dataframe with BQ Storage API #7759

Closed

Description

@tswast

Since data is downloaded in parallel, order might not be preserved. I believe if you explicitly request only a single stream, order is preserved. (Internal bug 130887748 to confirm this).

Activity

  1. added
    api: bigqueryIssues related to the BigQuery API.
    api: bigquerystorageIssues related to the BigQuery Storage API.
    type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.
    on Apr 19, 2019
  2. self-assigned this
    on Apr 19, 2019
  3. tswast commented on Apr 19, 2019

    @tswast
    ContributorAuthor

    Idea: Add a _preserve_order property to RowIterator. If true, only use one stream. Defualt to false. Set to true in QueryJob.result() if the query string contains "ORDER BY" (case insensitive, any amount of whitespace between ORDER and BY).

  4. added
    priority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.
    and removed
    triage meI really want to be triaged.
    on Apr 22, 2019
  5. added
    🚨This issue needs some love.
    and removed
    🚨This issue needs some love.
    on Apr 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the BigQuery API.api: bigquerystorageIssues related to the BigQuery Storage API.priority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions