docs: add Pub/Sub to ClickHouse Dataflow template docs#87
Open
mintlify[bot] wants to merge 1 commit into
Open
Conversation
Contributor
Author
|
Preview deployment for your docs. Learn more about Mintlify Previews.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports ClickHouse/clickhouse-docs#6295, adding documentation for the new Pub/Sub to ClickHouse Google Dataflow template.
Changes
integrations/connectors/data-ingestion/GCP/google-dataflow/templates/pubsub-to-clickhouse.mdx(slug preserved:/integrations/google-dataflow/templates/pubsub-to-clickhouse)pubsub-inqueue-job.pngimagetemplates.mdxto link to the new template and remove the "coming soon" Pub/Sub entrydataflow.mdxto mention streaming ingestion from Pub/Subbigquery-to-clickhouse.mdxtemplate source code section to list both the upstream GCP repo and the ClickHouse forkintegrations/connectors/navigation.jsonNotes / ambiguities
:::importantand:::noteadmonitions plus a<TOCInline>import; these were converted to Mintlify<Warning>/<Note>components, and the TOC import was dropped (Mintlify renders the page TOC automatically).@theme/IdealImage) were replaced with the local<Image>snippet used elsewhere in this section.{/* ... */}) and left in place — the source PR also leaves these as TODOs.../../../sql-reference/data-types/*were rewritten to absolute Mintlify slugs under/core/reference/data-types/*to match the convention used in the sibling BigQuery template page.Note
Low Risk
Documentation-only changes with no application, auth, or data-plane code.
Overview
Adds a full Pub/Sub to ClickHouse Dataflow template guide and wires it into the connectors docs so streaming ingestion is documented alongside the existing BigQuery template.
The new page documents JSON-to-table mapping (case-sensitive column names, type coercion, startup schema fetch), batching/windowing (
windowSeconds/batchRowCount), dead-letter handling (ClickHouse table schema or Pub/Sub topic attributes), console run steps (with screenshot placeholders), PubSubToClickHouse metrics, and troubleshooting (memory 241, DLQ-only traffic, no rows). The overview and templates index now mention streaming from Pub/Sub; the templates list links the new page and drops the “coming soon” Pub/Sub entry. BigQuery template source links now point to both upstream GoogleCloudPlatform/DataflowTemplates and the ClickHouse fork. navigation.json registers the new page under Dataflow templates.Reviewed by Cursor Bugbot for commit c79246f. Bugbot is set up for automated code reviews on this repo. Configure here.