Skip to content

feat(integrations): add Azure Data Explorer - #6701

Merged
waleedlatif1 merged 9 commits into
stagingfrom
feat/azure-data-explorer
Aug 14, 2026
Merged

feat(integrations): add Azure Data Explorer#6701
waleedlatif1 merged 9 commits into
stagingfrom
feat/azure-data-explorer

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Add Azure Data Explorer (Kusto) integration with 14 operations: run KQL queries, list databases/tables/functions, read table and database schemas, read table details, create and drop tables, ingest rows inline or from a query, list ingestion failures, check operations, and run arbitrary management commands
  • Authenticate with a Microsoft Entra service principal (client credentials) through an internal proxy route, since the Kusto token audience is per-cluster and cannot use a static-scope OAuth provider
  • Pin the proxy to the documented Azure Data Explorer and Fabric Eventhouse service domains, and restrict entity names to Kusto's documented identifier character set so nothing user-supplied can extend a command string
  • Bound result payloads: cap the response body and project at most 10,000 rows, reporting rowCount, totalRowCount, and truncated instead of silently returning a short answer
  • Detect partial query failures — Kusto answers 200 as soon as it starts processing and reports later failures through a QueryStatus table

Type of Change

  • New feature

Testing

35 unit tests covering the proxy route and command builders. Verified every guard fails when removed. Type-check, lint, tool-metadata, integration-catalog, and API-validation gates pass.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Add a 14-operation Azure Data Explorer (Kusto) integration covering KQL
queries, schema and metadata discovery, table management, inline and
query-sourced ingestion, ingestion-failure triage, and arbitrary
management commands.

Authentication uses a Microsoft Entra service principal through an
internal proxy route, since the Kusto token audience is per-cluster and
cannot be expressed as a static-scope OAuth provider.
@vercel

vercel Bot commented Aug 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview Aug 14, 2026 7:26pm

Request Review

@cursor

cursor Bot commented Aug 14, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
New server proxy handles client secrets and can run destructive Kusto management (ingest, drop table); mitigations include host allowlists, identifier validation, read-only queries, and payload caps, but misconfigured principals or agent-generated KQL still warrant careful review.

Overview
Adds a full Azure Data Explorer (Kusto) integration so workflows and agents can query telemetry and manage clusters via Microsoft Entra service principal credentials (no interactive OAuth).

A new authenticated proxy exchanges client credentials for cluster-scoped tokens, calls Kusto query / mgmt REST endpoints, and hardens outbound access: cluster URIs are limited to documented ADX/Fabric domains (with sovereign-cloud Entra authorities), entity names are validated, responses are size-capped, and results are projected with a 10,000-row ceiling plus truncated / totalRowCount. Query answers use the response table-of-contents for the primary result and treat QueryStatus severity ≤2 as failure even on HTTP 200; optional x-ms-readonly is supported for KQL.

14 operations are exposed as tools and a canvas block (KQL query, schema discovery, inline/query ingest, create/drop table, ingestion failures, operations, and advanced management commands), with shared CSL/command builders to prevent injection into management strings. Docs, catalog metadata, icons, templates/skills, and proxy + utils unit tests are included.

Docs for the Sim Table integration Query output also document nextCursor for paging when a page ends at the byte budget.

Reviewed by Cursor Bugbot for commit 3b0cac9. Configure here.

…plorer

# Conflicts:
#	apps/sim/tools/generated/tool-outputs.ts
@greptile-apps

greptile-apps Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds an Azure Data Explorer integration with service-principal authentication, bounded Kusto result projection, partial-query-failure handling, management and ingestion operations, catalog registration, tests, and documentation.

  • Adds a validated internal proxy for Azure Data Explorer and Fabric Eventhouse endpoints.
  • Adds fourteen query, discovery, schema, ingestion, table-management, and operations tools.
  • Registers the integration across blocks, tool metadata, icons, contracts, and documentation.
  • Adds regression coverage ensuring ordinary result columns named Severity and StatusDescription are not interpreted as query status.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
apps/sim/app/api/tools/azure_data_explorer/proxy/route.ts Adds the authenticated proxy, endpoint validation, token handling, bounded response projection, and table-of-contents-based query-status parsing; the previously reported false positive is fixed.
apps/sim/app/api/tools/azure_data_explorer/proxy/route.test.ts Covers authentication, endpoint restrictions, cloud authorities, result selection, partial failures, response limits, and the prior Severity-column regression.
apps/sim/lib/api/contracts/tools/azure_data_explorer.ts Defines the proxy request and response boundary validation for Azure Data Explorer operations.
apps/sim/blocks/blocks/azure_data_explorer.ts Adds the integration block configuration and maps its operation choices to the new tools.
apps/sim/tools/azure_data_explorer/utils.ts Provides shared command construction, identifier validation, and proxy request behavior for the integration.
apps/sim/tools/azure_data_explorer/management.ts Adds arbitrary Kusto management-command execution as an explicitly documented privileged operation.
apps/docs/content/docs/en/integrations/azure_data_explorer.mdx Documents authentication, operations, result limits, ingestion semantics, and destructive-command considerations.

Sequence Diagram

sequenceDiagram
  participant Workflow as Workflow/Agent
  participant Tool as Azure Data Explorer Tool
  participant Proxy as Internal ADX Proxy
  participant Entra as Microsoft Entra
  participant Kusto as ADX/Eventhouse Cluster
  Workflow->>Tool: Invoke operation
  Tool->>Proxy: Credentials, cluster, database, KQL/command
  Proxy->>Proxy: Authenticate and validate request
  Proxy->>Entra: Request cluster-scoped access token
  Entra-->>Proxy: Bearer token
  Proxy->>Kusto: Query or management request
  Kusto-->>Proxy: Kusto tables and table of contents
  Proxy->>Proxy: Detect QueryStatus failure and bound rows
  Proxy-->>Tool: Structured result
  Tool-->>Workflow: Operation output
Loading

Reviews (8): Last reviewed commit: "fix(azure-data-explorer): handle commas ..." | Re-trigger Greptile

Comment thread apps/sim/app/api/tools/azure_data_explorer/proxy/route.ts Outdated
…ueryStatus table

Scanning every returned table for Severity and StatusDescription columns
misread an ordinary query as a failed request whenever the user's own
result selected columns of those names — a common shape for a log table.

Failure detection now consults only the table the response's table of
contents names as QueryStatus, and primary-result selection reuses the
same index instead of re-reading it.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/app/api/tools/azure_data_explorer/proxy/route.ts
Comment thread apps/sim/tools/azure_data_explorer/utils.ts
Comment thread apps/sim/app/api/tools/azure_data_explorer/proxy/route.ts Outdated
…ails cards from painting empty

check:canvas-sentences flagged the Show Operations sentence: it anchored
`core` on operationId, which is an advanced-mode optional field, so an
untouched card resolved to nothing and painted empty. Show Table Details
had the same shape in milder form — table is optional there, since
omitting it describes every table, leaving a dangling preposition.

Both now lead with literal copy and treat their field as an optional
refinement. Also simplifies the primary-table condition to a single
`!= null` check.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

1 issue from previous review remains unresolved.

Fix All in Cursor

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 0d1502a. Configure here.

…ir own Entra authority

The cluster allowlist accepted Azure China and US Government hosts, but
every token request went to login.microsoftonline.com. Those clouds are
isolated instances with their own Entra endpoints, so a sovereign cluster
passed URI validation and then could never obtain a token.

Each Kusto service domain is now declared alongside the authority that
issues tokens for it, so the two cannot drift apart, and the authority is
part of the token cache key.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 3a04d0a. Configure here.

… columns by position

Kusto aligns an ingested query result to the target table on column type
and order, never on column name, so a query projecting the right columns
in the wrong order lands data in the wrong columns without erroring.

Surfaces that in the tool description and param the model reads, in the
wand prompt that generates the query, in the rollup skill's steps, and in
the docs. Also verifies the target schema first rather than after.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 2559fb4. Configure here.

… cluster allowlist

Every other entry traces to a Microsoft reference — the Kusto
connection-string doc, the national-cloud endpoint tables, and the Fabric
KQL-database REST reference. kustomfa.windows.net does not, and the
connection-string doc states the trust boundary as hostnames ending in
kusto.windows.net.

An allowlist should only hold hosts we can justify, so this drops it and
records the sourcing standard for anything added later.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit c7f22be. Configure here.

…plorer

# Conflicts:
#	apps/sim/tools/generated/tool-ids.ts
#	apps/sim/tools/generated/tool-metadata.ts
#	apps/sim/tools/generated/tool-outputs.ts
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/tools/azure_data_explorer/utils.ts
Comment thread apps/sim/tools/azure_data_explorer/utils.ts
…empty extent IDs

Two defects in the shared command helpers:

buildWithClause split the property list on every comma before validating,
so a value that legally contains one — a docstring sentence, or a tags
array with more than one entry — was torn in half and rejected. Splitting
is now quote-aware, and an unterminated quote is rejected outright rather
than swallowing the rest of the clause.

transformColumnListResponse dropped empty strings, but `.ingest inline`
reports "no data shards were generated" as a single record carrying an
empty extent ID. A no-op load therefore looked like a missing column
instead of an empty result. Only non-strings are skipped now.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 3b0cac9. Configure here.

@waleedlatif1
waleedlatif1 merged commit a7115e8 into staging Aug 14, 2026
30 checks passed
@waleedlatif1
waleedlatif1 deleted the feat/azure-data-explorer branch August 14, 2026 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant