Skip to content

fix(bigquery): BQ batch size #5684

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yevgenypats merged 4 commits into from
Dec 16, 2022
Merged

fix(bigquery): BQ batch size #5684

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a61a3df
add batch size
bbernays Dec 15, 2022
4afa073
Merge branch 'main' into bq-batch-size
yevgenypats Dec 15, 2022
d48f096
Update client.go
bbernays Dec 15, 2022
d757bc8
Merge branch 'main' into bq-batch-size
bbernays Dec 15, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion plugins/destination/bigquery/client/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ type Client struct {
logger zerolog.Logger
spec specs.Destination
metrics plugins.DestinationMetrics
batchSize int
Copy link
Copy Markdown
Contributor

@candiduslynx candiduslynx Dec 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see only initialization of this field.
Where is the propagation to the write?

Copy link
Copy Markdown
Collaborator Author

@bbernays bbernays Dec 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#5681

pluginSpec Spec
}

Expand All @@ -37,7 +38,7 @@ func New(ctx context.Context, logger zerolog.Logger, destSpec specs.Destination)
}

c.pluginSpec = spec

c.batchSize = spec.BatchSize
// create a client to test that we can do it, but new clients will also be instantiated
// for queries so that we can use a new context there.
client, err := c.bqClient(ctx)
Expand Down