improvement(agent, file-block): files in agent block, file block v4#4610
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
Greptile SummaryThis PR adds file attachment support to the agent block (upload or wire files into any agent's latest user message) and introduces a new
Confidence Score: 3/5The file attachment pipeline works correctly end-to-end for all supported providers, but the early guard in the agent handler silently passes for three declared-unsupported providers, causing wasted file hydration and a broken test. The early rejection check in apps/sim/executor/handlers/agent/agent-handler.ts and apps/sim/executor/handlers/agent/agent-handler.test.ts need the most attention. Important Files Changed
Sequence DiagramsequenceDiagram
participant UI as Agent Block UI
participant AH as AgentBlockHandler
participant AT as attachments.ts
participant HY as hydrateUserFilesWithBase64
participant PR as Provider (openai/anthropic/…)
UI->>AH: execute(inputs.files)
AH->>AH: normalizeFileInput(filesInput)
AH->>AH: getAttachmentProvider(providerId) — guard
Note over AH: xai/deepseek/cerebras pass guard (non-null) ⚠️
AH->>HY: hydrateUserFilesWithBase64(userFiles)
HY-->>AH: UserFile[] with base64
AH->>AH: attach files to last user message
AH->>PR: executeProviderRequest(messages with files)
PR->>AT: buildXxxMessageContent / formatMessagesForProvider
AT->>AT: prepareProviderAttachments
Note over AT: Throws for xai/deepseek/cerebras here
AT-->>PR: formatted content parts
PR-->>AH: ProviderResponse
Reviews (1): Last reviewed commit: "Clean up attachments" | Re-trigger Greptile |
…#4619) * improvement(providers): align attachment dispatch to vendor SDK types Post-merge audit of #4610 surfaced three follow-ups: 1. xAI Grok vision was blocked. Grok runs through the OpenAI-compatible chat-completions endpoint, so removing xAI from UNSUPPORTED_FILE_PROVIDERS and routing it through the image-only branch restores image attachments on vision models. 2. Azure OpenAI chat-completions deployments blocked any file attachment. Added a per-message image_url parts path; documents still require the Responses API endpoint and throw a clear, actionable error. 3. Wire shapes were loosely typed (Record<string, unknown> arrays). Replaced with `satisfies` clauses against each vendor SDK union at every push site: OpenAI Responses/Chat, Anthropic ContentBlockParam, Gemini Part, Bedrock ContentBlock members. AnthropicImageMediaType now derives from Base64ImageSource['media_type'] so it tracks SDK updates. Also collapsed the validation cascade into an exhaustive switch with `never` enforcement, and dropped the redundant per-provider formatMessagesForProvider call from xai/index.ts (providers/index.ts already runs the dispatcher centrally). * fix(providers): restore getProviderAttachmentMaxBytes export and xAI message dispatch - Restore `getProviderAttachmentMaxBytes` — still consumed by agent-handler.ts for per-provider attachment size limits in file hydration - Restore `formatMessagesForProvider(allMessages, 'xai')` — providers/index.ts does NOT dispatch centrally on this branch; each OpenAI-compat provider formats its own messages. Without it, xAI Grok vision drops image attachments * fix(providers): tighten ResponsesInputItem content type to SDK ResponseInputContent Build fix: buildOpenAIMessageContent returns ResponseInputContent[] which isn't assignable to Record<string, unknown>[] (ResponseInputText lacks an index signature). Align the type to the SDK shape.
Summary
Add files to agent block, update file block
Type of Change
Testing
How has this been tested? What should reviewers focus on?
Checklist