fix(brightdata): align integration with live API docs, add markdown/mode/error-report support#5470
Conversation
…ode/error-report support
- fix Discover numResults docs (max 20, not 1000) and contentFormat value ("md" not "markdown", which the API always rejected)
- add Discover mode param (standard/deep/fast/zeroRanking)
- add markdown output support to Web Unlocker scrape_url via data_format
- add include_errors support to scrape_dataset, matching sync_scrape
- add .trim() on datasetId in scrape_dataset/sync_scrape URLs
- add wandConfig on complex fields, tighten block output descriptions
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
PR SummaryLow Risk Overview Scrape URL gains an advanced Convert To option that maps to Web Unlocker The workflow UI adds Reviewed by Cursor Bugbot for commit aef2da2. Configure here. |
Greptile SummaryThis PR aligns the Bright Data integration with the live API specs, fixing two values that caused 400 errors and adding several new optional parameters. All changes are backwards-compatible since the corrected values were never valid API inputs.
Confidence Score: 5/5Safe to merge — all changes are verified bug fixes and additive optional parameters with no breaking changes. The corrected API values ("md" for Discover format, data_format: "markdown" for Web Unlocker) are confirmed against Bright Data's live OpenAPI spec. The new optional params (mode, dataFormat, includeErrors) are all properly guarded and only forwarded when explicitly set. The previous mode: "standard" default issue flagged in the earlier review thread was resolved in the head commit. No files require special attention. Important Files Changed
Sequence Diagram%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant User
participant Block as BrightData Block
participant Tool as Tool Layer
participant API as Bright Data API
User->>Block: Set operation + params
Note over Block: transform() maps block params<br/>to tool params
alt scrape_url
Block->>Tool: "{zone, url, format, country, dataFormat?}"
Tool->>API: POST /request with data_format: "markdown" (if set)
end
alt discover
Block->>Tool: "{query, numResults?, mode?, intent?, format?}"
Note over Tool: mode only forwarded when non-empty
Tool->>API: POST /discover with format: "md"
API-->>Tool: "{task_id}"
loop Poll until done/failed/timeout
Tool->>API: "GET /discover?task_id=..."
API-->>Tool: "{status, results}"
end
end
alt sync_scrape / scrape_dataset
Block->>Tool: "{datasetId.trim(), urls, format, includeErrors?}"
Tool->>API: POST /datasets/v3/scrape or /trigger
end
API-->>Tool: Response
Tool-->>Block: Transformed output
Block-->>User: results / content / data
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant User
participant Block as BrightData Block
participant Tool as Tool Layer
participant API as Bright Data API
User->>Block: Set operation + params
Note over Block: transform() maps block params<br/>to tool params
alt scrape_url
Block->>Tool: "{zone, url, format, country, dataFormat?}"
Tool->>API: POST /request with data_format: "markdown" (if set)
end
alt discover
Block->>Tool: "{query, numResults?, mode?, intent?, format?}"
Note over Tool: mode only forwarded when non-empty
Tool->>API: POST /discover with format: "md"
API-->>Tool: "{task_id}"
loop Poll until done/failed/timeout
Tool->>API: "GET /discover?task_id=..."
API-->>Tool: "{status, results}"
end
end
alt sync_scrape / scrape_dataset
Block->>Tool: "{datasetId.trim(), urls, format, includeErrors?}"
Tool->>API: POST /datasets/v3/scrape or /trigger
end
API-->>Tool: Response
Tool-->>Block: Transformed output
Block-->>User: results / content / data
Reviews (2): Last reviewed commit: "fix(brightdata): default mode field to e..." | Re-trigger Greptile |
…r advanced params
Greptile flagged that mode shipped with a non-empty default ('standard'),
causing it to be sent on every Discover call unlike the sibling dataFormat
field which uses an empty/None default and only sends when the user opts in.
Aligns mode with that same pattern.
|
@greptile review |
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit aef2da2. Configure here.
Summary
modeparam (standard/deep/fast/zeroRanking) for search depth/ranking controlscrape_urlviadata_formatinclude_errorssupport toscrape_dataset, matching existingsync_scrapebehavior.trim()ondatasetIdin scrape_dataset/sync_scrape URLs to guard against copy-paste whitespacewandConfigon complex fields, tightened block output descriptionsFull re-validation against Bright Data's live OpenAPI specs (Web Unlocker, SERP, Discover, Datasets v3) confirms every tool is aligned field-for-field. No backward-incompatible changes — the two corrected values were never valid API inputs, so no working workflow could have depended on them.
Type of Change
Testing
Tested manually — validated every tool against Bright Data's live docs/OpenAPI specs and re-verified with 5 independent verification passes (web-unlocker/serp, discover, datasets-v3, block wiring/registry, lint/typecheck/diff review).
bunx biome checkclean,bunx tsc --noEmitclean (exit 0).Checklist