Skip to content

AIORateLimiter: a RetryAfter halt is released early by any concurrent request finishing #5338

Description

@0xSoftBoi

Steps to Reproduce

AIORateLimiter documents that a RetryAfter "will halt all requests for retry_after + 0.1 seconds". It does not, if any other request happens to finish during that window.

process_request ends each attempt with:

            except RetryAfter as exc:
                ...
                self._retry_after_event.clear()
                await asyncio.sleep(sleep)
            finally:
                # Allow other requests to be processed
                self._retry_after_event.set()

_retry_after_event is shared by the whole limiter, but the finally runs for every request, including ones that never cleared it. So a request that was already past inner()'s await self._retry_after_event.wait() when the halt began will set() it on completion and release a halt it did not establish.

Script (no network, patches nothing):

import asyncio, time
from telegram.error import RetryAfter
from telegram.ext import AIORateLimiter

async def main():
    rl = AIORateLimiter(overall_max_rate=0, group_max_rate=0, max_retries=1)

    async def flooded(*a, **kw):
        if not flooded.hit:
            flooded.hit = True
            raise RetryAfter(2)
        return True
    flooded.hit = False

    async def slow(*a, **kw):
        await asyncio.sleep(0.3)   # an ordinary in-flight request
        return True

    async def fast(*a, **kw):
        return True

    async def request(cb, cid):
        return await rl.process_request(callback=cb, args=(), kwargs={},
            endpoint="sendMessage", data={"chat_id": cid}, rate_limit_args=None)

    t0 = time.monotonic()
    b = asyncio.create_task(request(slow, 2))     # in flight first
    await asyncio.sleep(0.05)
    a = asyncio.create_task(request(flooded, 1))  # floods, halts the bot for 2s
    await asyncio.sleep(0.05)
    await b
    await request(fast, 3)                        # should wait out the halt
    print(f"third request sent {time.monotonic() - t0:.2f}s into a 2s halt")
    await a

asyncio.run(main())

Expected behaviour

The third request waits out the halt (~2.1s).

Actual behaviour

third request sent 0.30s into a 2s halt

It is sent as soon as the unrelated in-flight request completes.

The same root cause has a second effect: when two requests are backing off concurrently, the shorter backoff expiring releases the longer one's halt.

Practical impact is that the halt is least effective exactly when it matters most — under load there is almost always another request in flight, so a 429 storm keeps being fed instead of backing off.

Operating System

Linux

Version of Python, python-telegram-bot & dependencies

python-telegram-bot 22.8 (also present on master), aiolimiter 1.2.1, Python 3.14

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Effort

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions