Persist the numba compile of the built-in UDF binops to disk - #611
Open
eriknw wants to merge 1 commit into
Open
Conversation
eriknw
marked this pull request as ready for review
August 4, 2026 16:07
eriknw
force-pushed
the
31-numba-cache-builtin-udfs
branch
from
August 4, 2026 16:12
4130be1 to
e848303
Compare
eriknw
force-pushed
the
31-numba-cache-builtin-udfs
branch
from
August 5, 2026 00:06
e848303 to
408900a
Compare
eriknw
force-pushed
the
31-numba-cache-builtin-udfs
branch
from
August 5, 2026 03:18
408900a to
e4f6717
Compare
eriknw
force-pushed
the
31-numba-cache-builtin-udfs
branch
from
August 5, 2026 17:44
e4f6717 to
6aecc12
Compare
eriknw
force-pushed
the
31-numba-cache-builtin-udfs
branch
from
August 5, 2026 18:03
6aecc12 to
a934b2d
Compare
eriknw
force-pushed
the
31-numba-cache-builtin-udfs
branch
from
August 5, 2026 18:05
a934b2d to
f8c1826
Compare
eriknw
force-pushed
the
31-numba-cache-builtin-udfs
branch
from
August 6, 2026 07:59
f8c1826 to
188b387
Compare
eriknw
force-pushed
the
31-numba-cache-builtin-udfs
branch
from
August 6, 2026 15:39
188b387 to
d57105f
Compare
eriknw
force-pushed
the
31-numba-cache-builtin-udfs
branch
2 times, most recently
from
August 6, 2026 20:36
2265b21 to
c4a03f3
Compare
eriknw
force-pushed
the
31-numba-cache-builtin-udfs
branch
from
August 6, 2026 20:42
c4a03f3 to
5fc2928
Compare
eriknw
force-pushed
the
31-numba-cache-builtin-udfs
branch
from
August 7, 2026 02:49
5fc2928 to
e2a01e7
Compare
floordiv, rfloordiv, absfirst, abssecond, and rpow are UDFs, so every process that touched one paid its numba compile again. They are plain module-level functions, which is exactly the case numba can cache, so that cost belongs once per machine rather than once per process. Thread the flag through as a private keyword-only `_cache` on register_new into _build, where it reaches numba.njit alongside the existing error_model. Keeping it private avoids widening the public API for something no caller outside this module should set. Ops registered by users stay uncached: numba keys a cache entry on a stable on-disk source, which lambdas and interactively defined functions do not have. Measured here, fresh process, medians of 3, load average ~4 on a shared machine so read these as indicative. First-touch floordiv alone: 434 ms uncached, 347 ms warm (-20%). First-touch of all five: 1337 ms uncached, 733 ms warm (-45%). Writing a cold cache costs about 107 ms once for all five (1444 ms on the run that populates it). What stays: the remaining ~733 ms is numba/LLVM startup and per-process object linking, which no on-disk cache removes. In-process dispatch once an op is built is untouched. The first process on a machine is slightly slower, not faster. Read-only installs are safe, verified rather than assumed. With graphblas/core/operator and its __pycache__ chmod'd a-w, the run wrote zero files in-tree and 66 .nbi/.nbc files under ~/Library/Caches/numba, with empty stderr under `python -W always`. The NumbaWarning in numba/core/caching.py is a source-content check, not a filesystem one. docs/user_guide/udf.rst gains a "Compilation caching" section covering the cache location, the read-only fallback, and why user ops are excluded.
eriknw
force-pushed
the
31-numba-cache-builtin-udfs
branch
from
August 7, 2026 05:09
e2a01e7 to
ae91f10
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
floordiv, rfloordiv, absfirst, abssecond, and rpow are UDFs, so every
process that touched one paid its numba compile again. They are plain
module-level functions, which is exactly the case numba can cache, so
that cost belongs once per machine rather than once per process.
Thread the flag through as a private keyword-only
_cacheonregister_new into _build, where it reaches numba.njit alongside the
existing error_model. Keeping it private avoids widening the public API
for something no caller outside this module should set. Ops registered by
users stay uncached: numba keys a cache entry on a stable on-disk source,
which lambdas and interactively defined functions do not have.
Measured here, fresh process, medians of 3, load average ~4 on a shared
machine so read these as indicative. First-touch floordiv alone: 434 ms
uncached, 347 ms warm (-20%). First-touch of all five: 1337 ms uncached,
733 ms warm (-45%). Writing a cold cache costs about 107 ms once for all
five (1444 ms on the run that populates it).
What stays: the remaining ~733 ms is numba/LLVM startup and per-process
object linking, which no on-disk cache removes. In-process dispatch once
an op is built is untouched. The first process on a machine is slightly
slower, not faster.
Read-only installs are safe, verified rather than assumed. With
graphblas/core/operator and its pycache chmod'd a-w, the run wrote
zero files in-tree and 66 .nbi/.nbc files under ~/Library/Caches/numba,
with empty stderr under
python -W always. The NumbaWarning innumba/core/caching.py is a source-content check, not a filesystem one.
docs/user_guide/udf.rst gains a "Compilation caching" section covering the
cache location, the read-only fallback, and why user ops are excluded.
Stack created with GitHub Stacks CLI • Give Feedback 💬