Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 122 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -899,6 +899,128 @@ You can also customize the client on a per-request basis by using `with_options(
client.with_options(http_client=DefaultHttpxClient(...))
```

#### Mutual TLS

Before configuring a client, review the
[OpenAI Mutual TLS Beta Program](https://help.openai.com/en/articles/10876024-openai-mutual-tls-beta-program)
for enrollment, currently supported endpoints, and certificate requirements.

For API-key authenticated HTTP requests that require mutual TLS (mTLS), configure
a native [`ssl.SSLContext`](https://docs.python.org/3/library/ssl.html#ssl.SSLContext)
and pass it through the custom HTTP client:

```python
import os
import ssl

from openai import OpenAI, DefaultHttpxClient

# Server trust is configured independently. Without `cafile`, this uses the
# operating system's normal trusted certificate authorities.
ssl_context = ssl.create_default_context(
cafile=os.environ.get("OPENAI_MTLS_CA_BUNDLE"),
)
ssl_context.load_cert_chain(
# This PEM must contain the leaf certificate first, followed by every
# intermediate certificate needed to reach the server's trust anchor.
certfile=os.environ["OPENAI_MTLS_CERTIFICATE_CHAIN"],
keyfile=os.environ["OPENAI_MTLS_PRIVATE_KEY"],
password=os.environ.get("OPENAI_MTLS_PRIVATE_KEY_PASSWORD"),
)

client = OpenAI(
api_key=os.environ["OPENAI_API_KEY"],
# A custom HTTP client does not tell the SDK that mTLS is configured, so
# select the mTLS endpoint explicitly. Preserve an EU or custom override.
base_url=os.environ.get(
"OPENAI_BASE_URL",
"https://mtls.api.openai.com/v1",
),
# A client certificate belongs to the HTTP client, not the base URL.
# Disable redirects so it cannot follow a response to another origin.
http_client=DefaultHttpxClient(
verify=ssl_context,
follow_redirects=False,
),
)
```

The async configuration is equivalent:

```python
import os
import ssl

from openai import AsyncOpenAI, DefaultAsyncHttpxClient

ssl_context = ssl.create_default_context(
cafile=os.environ.get("OPENAI_MTLS_CA_BUNDLE"),
)
ssl_context.load_cert_chain(
certfile=os.environ["OPENAI_MTLS_CERTIFICATE_CHAIN"],
keyfile=os.environ["OPENAI_MTLS_PRIVATE_KEY"],
password=os.environ.get("OPENAI_MTLS_PRIVATE_KEY_PASSWORD"),
)

client = AsyncOpenAI(
api_key=os.environ["OPENAI_API_KEY"],
base_url=os.environ.get(
"OPENAI_BASE_URL",
"https://mtls.api.openai.com/v1",
),
http_client=DefaultAsyncHttpxClient(
verify=ssl_context,
follow_redirects=False,
),
)
```

Experimental HTTPX2 uses the same native `SSLContext`. Install the optional
extra with `pip install 'openai[httpx2]'`, then use `DefaultHttpx2Client` or
`DefaultAsyncHttpx2Client` in place of the corresponding HTTPX client above:

```python
from openai import OpenAI, DefaultHttpx2Client

client = OpenAI(
api_key=os.environ["OPENAI_API_KEY"],
base_url=os.environ.get(
"OPENAI_BASE_URL",
"https://mtls.api.openai.com/v1",
),
http_client=DefaultHttpx2Client(
verify=ssl_context,
follow_redirects=False,
),
)
```

See the complete [sync HTTPX2](examples/mtls_httpx2.py) and
[async HTTPX2](examples/mtls_httpx2_async.py) examples.

The certificate-bearing HTTP client is transport-wide. Dedicate it to the
selected mTLS origin; do not reuse it for other services or pass it through
`with_options()` with a different `base_url`. If redirects are required, add an
HTTPX request hook that rejects requests whose scheme, host, or port differs
from the configured mTLS origin before enabling `follow_redirects`.

`SSLContext.load_cert_chain()` raises during setup for unreadable or malformed
files and for a private key that does not match the leaf certificate. Certificate
expiry, key usage, extended key usage, SAN, and trust policy remain TLS server
decisions. OpenAI does not fetch missing intermediates through AIA, so provide a
complete, leaf-first client-chain PEM. Intermediate-chain support is currently
enabled by request. Until it is enabled for your organization, use a client leaf
certificate directly signed by the uploaded CA.

For certificate rotation, build a new `SSLContext`, HTTP client, and `OpenAI` or
`AsyncOpenAI` client. This creates a fresh connection pool; close the old SDK
client after its in-flight requests finish. Do not assume existing TLS
connections will renegotiate.

This recipe applies to ordinary API-key HTTP traffic. It does not implement
certificate-only X.509 workload identity, token exchange, or Realtime WebSocket
mTLS.

### Managing HTTP resources

By default the library closes underlying HTTP connections whenever the client is [garbage collected](https://docs.python.org/3/reference/datamodel.html#object.__del__). You can manually close the client using the `.close()` method if desired, or with a context manager that closes when exiting.
Expand Down
30 changes: 30 additions & 0 deletions examples/mtls_httpx.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#!/usr/bin/env -S rye run python

import os
import ssl

from openai import OpenAI, DefaultHttpxClient

ssl_context = ssl.create_default_context(
cafile=os.environ.get("OPENAI_MTLS_CA_BUNDLE"),
)
ssl_context.load_cert_chain(
# Leaf certificate first; if intermediate-chain support is enabled, follow
# it with all required intermediates.
certfile=os.environ["OPENAI_MTLS_CERTIFICATE_CHAIN"],
keyfile=os.environ["OPENAI_MTLS_PRIVATE_KEY"],
password=os.environ.get("OPENAI_MTLS_PRIVATE_KEY_PASSWORD"),
)

with OpenAI(
api_key=os.environ["OPENAI_API_KEY"],
base_url=os.environ.get(
"OPENAI_BASE_URL",
"https://mtls.api.openai.com/v1",
),
http_client=DefaultHttpxClient(
verify=ssl_context,
follow_redirects=False,
),
) as client:
print(client.files.list())
30 changes: 30 additions & 0 deletions examples/mtls_httpx2.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#!/usr/bin/env -S rye run python

import os
import ssl

from openai import OpenAI, DefaultHttpx2Client

ssl_context = ssl.create_default_context(
cafile=os.environ.get("OPENAI_MTLS_CA_BUNDLE"),
)
ssl_context.load_cert_chain(
# Leaf certificate first; if intermediate-chain support is enabled, follow
# it with all required intermediates.
certfile=os.environ["OPENAI_MTLS_CERTIFICATE_CHAIN"],
keyfile=os.environ["OPENAI_MTLS_PRIVATE_KEY"],
password=os.environ.get("OPENAI_MTLS_PRIVATE_KEY_PASSWORD"),
)

with OpenAI(
api_key=os.environ["OPENAI_API_KEY"],
base_url=os.environ.get(
"OPENAI_BASE_URL",
"https://mtls.api.openai.com/v1",
),
http_client=DefaultHttpx2Client(
verify=ssl_context,
follow_redirects=False,
),
) as client:
print(client.files.list())
36 changes: 36 additions & 0 deletions examples/mtls_httpx2_async.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
#!/usr/bin/env -S rye run python

import os
import ssl
import asyncio

from openai import AsyncOpenAI, DefaultAsyncHttpx2Client


async def main() -> None:
ssl_context = ssl.create_default_context(
cafile=os.environ.get("OPENAI_MTLS_CA_BUNDLE"),
)
ssl_context.load_cert_chain(
# Leaf certificate first; if intermediate-chain support is enabled,
# follow it with all required intermediates.
certfile=os.environ["OPENAI_MTLS_CERTIFICATE_CHAIN"],
keyfile=os.environ["OPENAI_MTLS_PRIVATE_KEY"],
password=os.environ.get("OPENAI_MTLS_PRIVATE_KEY_PASSWORD"),
)

async with AsyncOpenAI(
api_key=os.environ["OPENAI_API_KEY"],
base_url=os.environ.get(
"OPENAI_BASE_URL",
"https://mtls.api.openai.com/v1",
),
http_client=DefaultAsyncHttpx2Client(
verify=ssl_context,
follow_redirects=False,
),
) as client:
print(await client.files.list())


asyncio.run(main())
36 changes: 36 additions & 0 deletions examples/mtls_httpx_async.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
#!/usr/bin/env -S rye run python

import os
import ssl
import asyncio

from openai import AsyncOpenAI, DefaultAsyncHttpxClient


async def main() -> None:
ssl_context = ssl.create_default_context(
cafile=os.environ.get("OPENAI_MTLS_CA_BUNDLE"),
)
ssl_context.load_cert_chain(
# Leaf certificate first; if intermediate-chain support is enabled,
# follow it with all required intermediates.
certfile=os.environ["OPENAI_MTLS_CERTIFICATE_CHAIN"],
keyfile=os.environ["OPENAI_MTLS_PRIVATE_KEY"],
password=os.environ.get("OPENAI_MTLS_PRIVATE_KEY_PASSWORD"),
)

async with AsyncOpenAI(
api_key=os.environ["OPENAI_API_KEY"],
base_url=os.environ.get(
"OPENAI_BASE_URL",
"https://mtls.api.openai.com/v1",
),
http_client=DefaultAsyncHttpxClient(
verify=ssl_context,
follow_redirects=False,
),
) as client:
print(await client.files.list())


asyncio.run(main())
25 changes: 20 additions & 5 deletions src/openai/_base_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import sys
import json
import math
import time
import uuid
import email
Expand Down Expand Up @@ -86,6 +87,7 @@
DEFAULT_MAX_RETRIES,
INITIAL_RETRY_DELAY,
RAW_RESPONSE_HEADER,
MAX_RETRY_AFTER_DELAY,
OVERRIDE_CAST_TO_HEADER,
DEFAULT_CONNECTION_LIMITS,
)
Expand Down Expand Up @@ -781,11 +783,15 @@ def _parse_retry_after_header(self, response_headers: Optional[httpx.Headers] =
pass

# Last, try parsing `retry-after` as a date.
retry_date_tuple = email.utils.parsedate_tz(retry_header)
if retry_date_tuple is None:
try:
retry_date_tuple = email.utils.parsedate_tz(retry_header)
if retry_date_tuple is None:
return None

retry_date = email.utils.mktime_tz(retry_date_tuple)
except (TypeError, ValueError, OverflowError, OSError):
return None

retry_date = email.utils.mktime_tz(retry_date_tuple)
return float(retry_date - time.time())

def _calculate_retry_timeout(
Expand All @@ -796,9 +802,9 @@ def _calculate_retry_timeout(
) -> float:
max_retries = options.get_max_retries(self.max_retries)

# If the API asks us to wait a certain amount of time (and it's a reasonable amount), just do what it says.
# Honor server-directed delays up to two minutes.
retry_after = self._parse_retry_after_header(response_headers)
if retry_after is not None and 0 < retry_after <= 60:
if retry_after is not None and math.isfinite(retry_after) and 0 < retry_after <= MAX_RETRY_AFTER_DELAY:
return retry_after

# Also cap retry count to 1000 to avoid any potential overflows with `pow`
Expand All @@ -813,6 +819,15 @@ def _calculate_retry_timeout(
return timeout if timeout >= 0 else 0

def _should_retry(self, response: httpx.Response) -> bool:
retry_after = self._parse_retry_after_header(response.headers)
if retry_after is not None and math.isfinite(retry_after) and retry_after > MAX_RETRY_AFTER_DELAY:
log.debug(
"Not retrying because `Retry-After` of %s seconds exceeds the maximum of %s seconds",
retry_after,
MAX_RETRY_AFTER_DELAY,
)
return False

# Note: this is not a standard header
should_retry_header = response.headers.get("x-should-retry")

Expand Down
1 change: 1 addition & 0 deletions src/openai/_constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,4 @@

INITIAL_RETRY_DELAY = 0.5
MAX_RETRY_DELAY = 8.0
MAX_RETRY_AFTER_DELAY = 2 * 60
41 changes: 41 additions & 0 deletions tests/fixtures/mtls/client-chain.pem
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
-----BEGIN CERTIFICATE-----
MIIDdzCCAl+gAwIBAgIUYfX+FG1H1GIF9JGcTGENkRmmO0AwDQYJKoZIhvcNAQEL
BQAwLzEtMCsGA1UEAwwkb3BlbmFpLXB5dGhvbi1tdGxzLXRlc3QtaW50ZXJtZWRp
YXRlMCAXDTI2MDcyODE5Mjc1MloYDzIxMjYwNzA0MTkyNzUyWjApMScwJQYDVQQD
DB5vcGVuYWktcHl0aG9uLW10bHMtdGVzdC1jbGllbnQwggEiMA0GCSqGSIb3DQEB
AQUAA4IBDwAwggEKAoIBAQDKfqXwVVGGA+o6tk1pJi8yyYpApRadLwTO643ETasj
7x41w9MH1jznEaZB+UZkm8IO/tcRF11+YiKqY2gFS0bthafjDCwB4+Ts6v3/GLfz
kCi6FgMFoOhNs5QvQiM5hpSfUOyHNf0yPsMxxqb8K0+YF0yMmryc+cVTwD31ZyKw
gQ24zWNR9HOsHAqHCo0+SDzcBufo6Ps+BstJvxkp1DDRFTvvSqhVZ7w5WrP/CfzP
qvKXTbuv0fGrFCakhNeyRN9y7ygP0YOODglnduIV9al4N9k/8axAZsQjjnxMV+5l
T/FLlqSQTmH9HHsjTQoOKyqOhHk8JSP+QctAotLWYK+dAgMBAAGjgY4wgYswDAYD
VR0TAQH/BAIwADAOBgNVHQ8BAf8EBAMCBaAwEwYDVR0lBAwwCgYIKwYBBQUHAwIw
FgYDVR0RBA8wDYILY2xpZW50LnRlc3QwHQYDVR0OBBYEFJELELi4PoIeS0pnxF9s
EoToL3etMB8GA1UdIwQYMBaAFLZ4Gtv+mnYi/DpIUXkfprFU6HbEMA0GCSqGSIb3
DQEBCwUAA4IBAQAp7S4SWpxs6GPmBBT8Nu6bXmlSdjtLNZ2C4sG9BBY3uACseN2B
6G149VLxLMaWPHd/L46SYAhAkN/zj7LnIrygiFW5eDZCxvYCBEk75U5zTpk9SQL5
bQqsR0RH9NxJkoKIUkQjrnxD+u7C2RBF3sE5KZmTGWYbvJXIOVrDHHWrb+z07bzQ
T8XRa4qdBGuV8OYQBzYR6EIb/DkEgBmfC/uTu7XFpQbmfw7Dnh0JcoJ9Dx9St7Xl
0HBb5gY0thL2cCrwe+PTSLV7S0nZLxp/azz/ROuzYWFSsSWMC4/hqdsoaa7PoVC6
Lr66LVF6hzL/TtACRCS4X7nu/7GzXNwE1BTa
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIDTDCCAjSgAwIBAgIUDYU6Ki4xOPXM8RawT32qDO00934wDQYJKoZIhvcNAQEL
BQAwJzElMCMGA1UEAwwcb3BlbmFpLXB5dGhvbi1tdGxzLXRlc3Qtcm9vdDAgFw0y
NjA3MjgxOTI3NTJaGA8yMTI2MDcwNDE5Mjc1MlowLzEtMCsGA1UEAwwkb3BlbmFp
LXB5dGhvbi1tdGxzLXRlc3QtaW50ZXJtZWRpYXRlMIIBIjANBgkqhkiG9w0BAQEF
AAOCAQ8AMIIBCgKCAQEAmye/0UvrNwfMZRx53QYM1W47rTtCQlp4Yo7L6EmceAzE
9Njf7IW6NPDqfH35zFOtHlD4rFaU6elkwNO0dyG37u3haaRUcUGhpDerI0RebZkH
XFwXVB9nbDI/5+7wtpqdKM10Mn88UtHG/akTpoqjZnlBfYBoNaTzb8UgVCQxsXuH
qxoFzxr1zjBl+wCm0WidzhsLCCBogH3h9RZ/kKrWTWkoVx6Wky7gUut04g3CEQtW
6XGV3edVHEBUAnXsx32ktpFiZwL1MKo68Lebb4PrXrAHuVyjYBFGXwSXluxSDKsH
44q82YMtwge1uhwdFg9Tiz7U5LeEcTTSp48aCeymnwIDAQABo2YwZDASBgNVHRMB
Af8ECDAGAQH/AgEAMA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUtnga2/6adiL8
OkhReR+msVTodsQwHwYDVR0jBBgwFoAUHDFyfiCXhqFiwcl9ZAWApe40+m0wDQYJ
KoZIhvcNAQELBQADggEBACaN1i97TDdZUB0I2p1yBOnbGDVNdmJYwkrWU+CR3StT
dz84/QAWztvKSwtPzuzk4RRpE6EzOPbYmEqTxRdKB08t1aBMZTasdkKjl8WmRoAE
XOYtPE/d53dnC+T3qfyJhq882H95kHBBsvPG8kWEhX5BN30qET2FP6xi6cTAXtR0
py3AV9qZqx9gxWIC2O1sh9nHmkvhk7WKxlhjkn5EEpDcLX9QfWy8iGrhoN+5RMR7
1/T6JyP58q1DR8SZ5RaFBO7e5QPYNj8mFf2/wMJZN8kMZZgrXQmX3W9GkIq48evk
ACDggWyBKx+I44bcCRuSOfDTiGLRY6AmEGx7uQBHDT4=
-----END CERTIFICATE-----
Loading
Loading