Skip to content

tests: resolve TOCTOU port race conditions for streamable-HTTP/SSE tests#2705

Open
Ar-maan05 wants to merge 1 commit into
modelcontextprotocol:mainfrom
Ar-maan05:deterministic-test-ports
Open

tests: resolve TOCTOU port race conditions for streamable-HTTP/SSE tests#2705
Ar-maan05 wants to merge 1 commit into
modelcontextprotocol:mainfrom
Ar-maan05:deterministic-test-ports

Conversation

@Ar-maan05
Copy link
Copy Markdown

This PR resolves intermittent test failures and flakiness in the CI matrix (specifically under pytest -n auto parallelism) caused by a time-of-check/time-of-use (TOCTOU) port race when allocating ports for test servers.

Closes #2704

Root Cause

Previously, test server port fixtures bound to 127.0.0.1:0 to find a free port, then closed the socket immediately to return the port number. Between this allocation and the server starting in a separate multiprocessing.Process to bind that port, another parallel worker could be assigned the same port. This caused two primary issues:

  1. OSError: [Errno 98] Address already in use (failing to start the server)
  2. Crossed/interleaved connections where a client connects to the wrong test server instance.

Solution

I converted the server processes to bind their socket ephemerally (127.0.0.1:0) and called listen() inside the child process itself. The socket remains open and in a listening state throughout the process lifecycle. The child process reports its dynamically allocated port back to the parent process using a unidirectional multiprocessing.Pipe.

• This fully closes the TOCTOU gap because the socket is never closed between check and use.
• It removes the need for wait_for_server polling (since the socket is already listening before the fixture yields).
• It remains fully cross-platform compatible as it does not require inheriting or pickling socket descriptors across process boundaries (only the pipe connection and integers are sent across process boundaries).

Affected Files Migrated

• tests/shared/test_streamable_http.py
• tests/shared/test_sse.py
• tests/server/test_sse_security.py
• tests/server/test_streamable_http_security.py
• tests/client/test_http_unicode.py

Other Cleanups

• Added a reusable running_server context manager in tests/test_helpers.py.
• Removed the unused wait_for_server helper from tests/test_helpers.py to maintain 100% test coverage.
• Updated affected test cases to consume these fixtures directly as URL strings, avoiding unnecessary server startups.

Verification

Automated Tests

uv run --frozen --no-default-groups --group dev pytest

• Result: 1271 passed (1172 passed, 98 skipped, 1 xfailed).

Linting & Formatting

uv run --frozen --no-default-groups --group dev ruff check .
uv run --frozen --no-default-groups --group dev pyright

• Result: 0 errors, 0 warnings.

Test Coverage

uv run --frozen --no-default-groups --group dev coverage report

• Result: 100.00% Coverage (19,675 statements, 0 missed).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky streamable-HTTP/SSE tests: TOCTOU port race under pytest -n auto

1 participant