Skip to content

feat: add a way to see the script that produced the result#128

Merged
constantinius merged 2 commits intomainfrom
constantinius/feat/inline-scripts
Apr 1, 2026
Merged

feat: add a way to see the script that produced the result#128
constantinius merged 2 commits intomainfrom
constantinius/feat/inline-scripts

Conversation

@constantinius
Copy link
Copy Markdown
Collaborator

@constantinius constantinius commented Mar 30, 2026

@linear-code
Copy link
Copy Markdown

linear-code Bot commented Mar 30, 2026

TET-2163 Add an easy way to look into the script that produced the outcome

@constantinius constantinius requested a review from a team March 30, 2026 14:54
sentry[bot]
sentry Bot reviewed Mar 30, 2026
View reviewed changes
Comment thread src/generate-html-report.ts Outdated
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 30, 2026 •
edited
Loading

🟡 AI SDK Integration Test Results

Status: 130 tests failing (no regressions)

Summary

Metric main PR Change
Total Tests 547 547 —
Passed 400 400 —
Failed 130 130 —

Test Matrix

Agent Tests

SDK Basic Agent Test Conversation ID Agent Test Long Input Agent Test Tool Call Agent Test Tool Error Agent Test Vision Agent Test
browser/langgraph ❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain ❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain ❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain ❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain ❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain ❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain
cloudflare/langgraph ❌ ❌ ❌ ❌ ❌ ❌
cloudflare/vercel ✅ ✅ ✅ ✅ ✅ ✅
nextjs/mastra ❌ ❌ — ❌ ❌ ❌
nextjs/vercel ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str
node/langgraph ❌ ❌ ❌ ❌ ❌ ❌
node/manual ✅ ✅ ✅ ✅ ✅ ✅
node/mastra ❌ ❌ ❌ ❌ ❌ ❌
node/vercel ✅ ✅ ✅ ✅ ✅ ✅
python/langgraph ❌a ❌s ❌a ❌s ❌a ❌s ❌a ❌s ❌a ❌s ❌a ❌s
python/manual ✅a ✅s ✅a ✅s ✅a ✅s ✅a ✅s ✅a ✅s ✅a ✅s
python/openai-agents ✅ ✅ ✅ ✅ ✅ ❌
python/pydantic-ai ✅a, fallback ✅a, single ✅a, fallback ✅a, single ✅a, fallback ✅a, single ✅a, fallback ✅a, single ✅a, fallback ✅a, single ✅a, fallback ✅a, single

Embedding Tests

SDK Basic Embeddings Test
browser/google-genai ✅
browser/langchain ❌
browser/openai ✅
cloudflare/google-genai ✅
cloudflare/langchain ❌
cloudflare/openai ✅
cloudflare/vercel ✅
nextjs/google-genai ✅
nextjs/langchain ❌
nextjs/openai ✅
nextjs/vercel ✅
node/google-genai ✅
node/langchain ❌
node/openai ✅
node/vercel ✅
python/google-genai ✅a, blk ✅s, blk
python/langchain ❌a, blk ❌s, blk
python/litellm ❌a, blk ✅s, blk
python/manual ✅a, blk ✅s, blk
python/openai ✅a, blk ✅s, blk

LLM Tests

SDK Basic Error LLM Test Basic LLM Test Conversation ID LLM Test Long Input LLM Test Multi-Turn LLM Test Vision LLM Test
browser/anthropic ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str
browser/google-genai ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str
browser/langchain ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ❌blk ❌str ✅blk ✅str
browser/openai ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ❌blk ❌str ✅blk ✅str
cloudflare/anthropic ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str
cloudflare/google-genai ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str
cloudflare/langchain ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str
cloudflare/openai ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str
nextjs/anthropic ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str
nextjs/google-genai ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str
nextjs/langchain ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ❌blk ❌str ✅blk ✅str
nextjs/openai ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ❌blk ❌str ✅blk ✅str
node/anthropic ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str
node/google-genai ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str
node/langchain ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str
node/manual — ✅ ✅ ✅ ✅ ✅
node/openai ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str ✅blk ✅str
python/anthropic ✅a, blk ❌a, str ✅s, blk ❌s, str ✅a, blk ✅a, str ✅s, blk ✅s, str ✅a, blk ✅a, str ✅s, blk ✅s, str ✅a, blk ✅a, str ✅s, blk ✅s, str ✅a, blk ✅a, str ✅s, blk ✅s, str ✅a, blk ✅a, str ✅s, blk ✅s, str
python/google-genai ✅a, blk ✅a, str ✅s, blk ✅s, str ✅a, blk ✅a, str ✅s, blk ✅s, str ✅a, blk ✅a, str ✅s, blk ✅s, str ✅a, blk ✅a, str ✅s, blk ✅s, str ❌a, blk ❌a, str ❌s, blk ❌s, str ❌a, blk ❌a, str ❌s, blk ❌s, str
python/langchain ✅a, blk ✅a, str ✅s, blk ✅s, str ✅a, blk ✅a, str ✅s, blk ✅s, str ✅a, blk ✅a, str ✅s, blk ✅s, str ✅a, blk ✅a, str ✅s, blk ✅s, str ✅a, blk ✅a, str ✅s, blk ✅s, str ✅a, blk ✅a, str ✅s, blk ✅s, str
python/litellm ❌a, blk ❌a, str ❌s, blk ❌s, str ❌a, blk ❌a, str ✅s, blk ✅s, str ❌a, blk ❌a, str ✅s, blk ✅s, str ❌a, blk ❌a, str ✅s, blk ✅s, str ❌a, blk ❌a, str ✅s, blk ✅s, str ❌a, blk ❌a, str ✅s, blk ✅s, str
python/manual — ✅a, blk ✅s, blk ✅a, blk ✅s, blk ✅a, blk ✅s, blk ✅a, blk ✅s, blk ✅a, blk ✅s, blk
python/openai ✅a, blk ✅a, str ✅s, blk ✅s, str ✅a, blk ❌a, str ✅s, blk ❌s, str ✅a, blk ❌a, str ✅s, blk ❌s, str ✅a, blk ❌a, str ✅s, blk ❌s, str ✅a, blk ❌a, str ✅s, blk ❌s, str ❌a, blk ❌a, str ❌s, blk ❌s, str

MCP Tests

SDK Basic MCP Tool Call Test MCP Multiple Tool Calls Test MCP Prompt Get Test MCP Resource Read Test MCP Tool Error Test
node/mcp ✅sse ✅io ✅sse ✅io ✅sse ✅io ✅sse ✅io ✅sse ✅io
python/fastmcp ✅a, blk, sse ✅a, blk, io ✅a, blk, sse ✅a, blk, io ✅a, blk, sse ✅a, blk, io ✅a, blk, sse ✅a, blk, io ✅a, blk, sse ✅a, blk, io
python/mcp ✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo ✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo ✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo ✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo ✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo

Legend: ✅ Pass | ❌ Fail | ✅🔧 Fixed | ❌📉 Regressed | ✅🆕 New (pass) | ❌🆕 New (fail) | 🗑️ Removed | str=streaming blk=blocking a=async s=sync io=stdio sse=sse hi=highlevel lo=lowlevel

Generated by AI SDK Integration Tests

@constantinius constantinius merged commit f725ff0 into main Apr 1, 2026
11 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sentry sentry[bot] sentry[bot] left review comments

@obostjancic obostjancic obostjancic approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@constantinius @obostjancic