Repository navigation
Using @sentry_sdk.trace with concurrent futures messes up span hierarchy #4241
Description
How do you use Sentry?
Sentry Saas (sentry.io)
Version
2.25.1
Steps to Reproduce
I am starting (calling) a bunch of futures, before I actually await them concurrently using asyncio.gather.
The code looks like this:
import asyncio
import sentry_sdk
@sentry_sdk.trace
async def foo(a):
await asyncio.sleep(1)
return a
@sentry_sdk.trace
async def real_main():
futures = [foo(a) for a in range(5)]
results = await asyncio.gather(*futures)
print(results)
async def main():
sentry_sdk.init(
dsn="…",
debug=True,
traces_sample_rate=1.0,
)
with sentry_sdk.start_transaction(op="task", name="main"):
await real_main()
if __name__ == "__main__":
asyncio.run(main())I might be able to work around this in this toy example by using explicit isolation_scope myself somehow.
However, my real usecase happens deep within the graphql library, somewhere here:
https://github.com/graphql-python/graphql-core/blob/416247c1d511350445c23096f9491fbef424b69b/src/graphql/execution/execute.py#L431-L470
Expected Result
I expect the futures to create sibling spans, as they are being "started" in the same caller function.
Actual Result
Instead of the calls ending up with sibling spans, they end up as parent->child spans instead.
As the first async fn has not "finished" yet when I "start" the next one, it is on the top of the span hierarchy.
Example trace: https://sentry-sdks.sentry.io/discover/trace/9f5c02338db64a91901d28c3d90541b7
Activity
Swatinem commentedon Apr 3, 2025
Actually https://sentry-sdks.sentry.io/discover/trace/a375c59ca7764110aa43973bcade293d is matching the example code, the other trace exhibits the same problem, but with a slightly different code, where I was initializing the SDK within the sync module. Doesn’t really matter, the result is the same unfortunately.
szokeasaurusrex commentedon May 20, 2025
Investigation notes
This appears to be a more fundamental problem with how scopes are isolated between async contexts. The problem is not unique to using
@sentry_sdk.trace; I can also reproduce it by manually starting spans inside the concurrent tasksfix(tracing): Fix span nesting concurrency issues
szokeasaurusrex commentedon May 21, 2025
@Swatinem Apparently, this bug is fixed in the
potel_basebranch, which will be released as version 3.0 of the Python SDK.Unless we see large demand to fix this bug in version 2.x of the Python SDK, we think fixing in 3.0 should be sufficient (especially because the fix would be non-trivial)
Swatinem commentedon May 21, 2025
that sounds reasonable. I would still keep this bug report open though until the version with the fix is actually released?
szokeasaurusrex commentedon May 21, 2025
In the Python SDK our workflow typically is that we close issues once all the work is completed (i.e. PR is merged), even if the changes are not released yet. The reasoning here is that it is typically possible to auto-close issues once PRs are merged, but we don't have a workflow which would auto-close issues upon release. It also keeps our issue list clean, since we ensure that open issues are always actionable from our side.
Therefore, I would prefer to keep this issue closed to stay consistent with our existing workflow.
Also, technically, the fix here should already be available in the
3.0.0a1alpha pre-release.