Skip to content

fix: avoid repeated numpy checks for embeddings - #3757

Merged
jbeckwith-oai merged 1 commit into
mainfrom
codex/fix-embeddings-numpy-check
Aug 28, 2026
Merged

fix: avoid repeated numpy checks for embeddings#3757
jbeckwith-oai merged 1 commit into
mainfrom
codex/fix-embeddings-numpy-check

Conversation

@jbeckwith-oai

Copy link
Copy Markdown
Contributor
  • I understand that this repository is auto-generated and my pull request may not be merged

Changes being requested

Avoid calling has_numpy() once per returned embedding when the default embeddings decoder handles a bulk base64 response. The parser now checks whether any encoded vectors need decoding, resolves NumPy availability once per response, and reuses that result for every encoded vector.

Add focused regressions that verify multiple encoded vectors trigger one availability check and responses containing only already-decoded vectors do not inspect NumPy.

Additional context & links

Closes #3753

Parser-only benchmark for 1,500 vectors x 1,536 dimensions with NumPy unavailable: 86.0 ms -> 20.5 ms median across 30 alternating runs, a 76.1% reduction and 4.18x speedup. With NumPy installed, performance was effectively unchanged/slightly improved: 24.0 ms -> 23.5 ms.

Validation:

  • 79 focused embeddings tests passed under Pydantic v2
  • 79 focused embeddings tests passed under Pydantic v1
  • 11,232 full-suite tests passed; 144 skipped
  • Ruff format/check passed
  • Pyright passed
  • mypy passed
  • Three adversarial review rounds completed; the final two consecutive rounds were clean

@jbeckwith-oai
jbeckwith-oai requested a review from a team as a code owner August 28, 2026 17:11
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-28T17:12:56.378471Z 7138d3e PR opened
🔒 Security Review Completed 2026-08-28T17:13:26.711866Z 7138d3e PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@openai-sdks

openai-sdks Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

OkTest Summary

237/237 SDK tests passed in 9.11s for Python SDK PR #3757.

Test results — 42 files
Test Result Time
tests/chat-completions-complex-body.test.ts ✅ Passed 152ms
tests/chat-completions-create.test.ts ✅ Passed 176ms
tests/chat-completions-stream.test.ts ✅ Passed 95ms
tests/files-content-binary.test.ts ✅ Passed 179ms
tests/files-create-multipart.test.ts ✅ Passed 208ms
tests/files-list-pagination.test.ts ✅ Passed 116ms
tests/initialize-config.test.ts ✅ Passed 121ms
tests/instance-isolation.test.ts ✅ Passed 127ms
tests/models-list.test.ts ✅ Passed 201ms
tests/responses-background-lifecycle.test.ts ✅ Passed 152ms
tests/responses-body-method-errors.test.ts ✅ Passed 250ms
tests/responses-cancel-timeout.test.ts ✅ Passed 191ms
tests/responses-cancel.test.ts ✅ Passed 144ms
tests/responses-compact-retries.test.ts ✅ Passed 210ms
tests/responses-compact.test.ts ✅ Passed 247ms
tests/responses-create-advanced-stream.test.ts ✅ Passed 88ms
tests/responses-create-advanced.test.ts ✅ Passed 158ms
tests/responses-create-disconnect.test.ts ✅ Passed 1.069s
tests/responses-create-errors.test.ts ✅ Passed 159ms
tests/responses-create-malformed-api-responses.test.ts ✅ Passed 98ms
tests/responses-create-retries.test.ts ✅ Passed 226ms
tests/responses-create-stream-failures.test.ts ✅ Passed 98ms
tests/responses-create-stream-timeout.test.ts ✅ Passed 183ms
tests/responses-create-stream-wire.test.ts ✅ Passed 2.889s
tests/responses-create-stream.test.ts ✅ Passed 199ms
tests/responses-create-terminal-states.test.ts ✅ Passed 243ms
tests/responses-create-timeout.test.ts ✅ Passed 214ms
tests/responses-create.test.ts ✅ Passed 444ms
tests/responses-delete.test.ts ✅ Passed 246ms
tests/responses-input-items-errors.test.ts ✅ Passed 311ms
tests/responses-input-items-list.test.ts ✅ Passed 239ms
tests/responses-input-items-options.test.ts ✅ Passed 234ms
tests/responses-input-tokens-count-timeout.test.ts ✅ Passed 203ms
tests/responses-input-tokens-count.test.ts ✅ Passed 167ms
tests/responses-malformed-inputs.test.ts ✅ Passed 2.088s
tests/responses-not-found-errors.test.ts ✅ Passed 201ms
tests/responses-parse.test.ts ✅ Passed 312ms
tests/responses-retrieve-retries.test.ts ✅ Passed 190ms
tests/responses-retrieve.test.ts ✅ Passed 170ms
tests/responses-stored-method-errors.test.ts ✅ Passed 478ms
tests/retry-behavior.test.ts ✅ Passed 3.017s
tests/sdk-error-shape.test.ts ✅ Passed 251ms

View OkTest run #33193548587

SDK merge (dba16520a181) · head (7138d3e5aa60) · base (9917c6e28e66) · OkTest (2b1bdfd25e98)

@github-actions

Copy link
Copy Markdown
Contributor

Castiron custom code

✅ No new custom-code files detected.

34 mixed files remain; 0 existing customizations changed.

Compared 9917c6e28e667138d3e5aa60. Generated baselines verified.

34 existing customizations unchanged
  • api.md
  • scripts/castiron/README.md
  • scripts/castiron/custom_code_report.py
  • scripts/castiron/test_custom_code_report.py
  • src/openai/init.py
  • src/openai/_client.py
  • src/openai/resources/audio/transcriptions.py
  • src/openai/resources/audio/translations.py
  • src/openai/resources/beta/beta.py
  • src/openai/resources/beta/responses/responses.py
  • src/openai/resources/beta/threads/runs/runs.py
  • src/openai/resources/beta/threads/threads.py
  • src/openai/resources/chat/completions/completions.py
  • src/openai/resources/embeddings.py
  • src/openai/resources/files.py
  • src/openai/resources/realtime/realtime.py
  • src/openai/resources/responses/responses.py
  • src/openai/resources/uploads/uploads.py
  • src/openai/resources/vector_stores/file_batches.py
  • src/openai/resources/vector_stores/files.py
  • src/openai/resources/videos.py
  • src/openai/resources/webhooks/init.py
  • src/openai/resources/webhooks/webhooks.py
  • src/openai/types/chat/init.py
  • src/openai/types/chat/chat_completion_message_tool_call.py
  • src/openai/types/fine_tuning/fine_tuning_job_integration.py
  • src/openai/types/responses/init.py
  • src/openai/types/responses/response.py
  • src/openai/types/responses/response_function_web_search.py
  • src/openai/types/responses/response_function_web_search_param.py
  • src/openai/types/responses/tool.py
  • src/openai/types/responses/tool_param.py
  • src/openai/types/webhooks/init.py
  • tests/api_resources/test_videos.py

A changed generated baseline means this report cannot reliably identify which handwritten lines changed.

Inspect the custom-code diff

Download the exact patch produced by this run (requires repository access):

gh run download 33193573074 --repo openai/openai-python \
  --name castiron-custom-code-33193573074-1 --dir /tmp/castiron-custom-code-33193573074-1
git apply --stat /tmp/castiron-custom-code-33193573074-1/custom-code.patch
cat /tmp/castiron-custom-code-33193573074-1/custom-code.patch

Or reproduce it from an SDK checkout containing the vendored reporter:

git fetch --no-tags origin 9917c6e28e66e90e1227b3d223c06a8c5441515a 7138d3e5aa609d385c7a4594d55cfdb7b690867e
python3 scripts/castiron/custom_code_report.py report \
  --base 9917c6e28e66e90e1227b3d223c06a8c5441515a \
  --head 7138d3e5aa609d385c7a4594d55cfdb7b690867e --fetch --require-head-hash --public \
  --out /tmp/castiron-custom-code-7138d3e5aa60
cat /tmp/castiron-custom-code-7138d3e5aa60/custom-code.patch

This is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR.

Full report and patch

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the exact current head. NumPy availability is now resolved once only when encoded vectors are present, while mixed/already-decoded vectors retain the existing behavior and focused tests cover both paths. No blocking findings. (GitHub does not permit this account to formally approve its own PR.)

@HAYDEN-OAI HAYDEN-OAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 7138d3e5aa609d385c7a4594d55cfdb7b690867e; no substantive issues found. NumPy availability is resolved once per response only when encoded vectors are present, while explicit formats, mixed/already-decoded vectors, decoder error behavior, and the shared sync/async response paths retain their existing semantics.

@jbeckwith-oai
jbeckwith-oai added this pull request to the merge queue Aug 28, 2026
Merged via the queue into main with commit b19c216 Aug 28, 2026
26 checks passed
@jbeckwith-oai
jbeckwith-oai deleted the codex/fix-embeddings-numpy-check branch August 28, 2026 22:38
@openai-sdks openai-sdks Bot mentioned this pull request Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bulk embeddings very slow

2 participants