Skip to content

Persist the numba compile of the built-in UDF binops to disk - #611

Open
eriknw wants to merge 1 commit into
30-udf-import-costfrom
31-numba-cache-builtin-udfs
Open

Persist the numba compile of the built-in UDF binops to disk#611
eriknw wants to merge 1 commit into
30-udf-import-costfrom
31-numba-cache-builtin-udfs

Conversation

@eriknw

@eriknw eriknw commented Aug 4, 2026

Copy link
Copy Markdown
Member

floordiv, rfloordiv, absfirst, abssecond, and rpow are UDFs, so every
process that touched one paid its numba compile again. They are plain
module-level functions, which is exactly the case numba can cache, so
that cost belongs once per machine rather than once per process.

Thread the flag through as a private keyword-only _cache on
register_new into _build, where it reaches numba.njit alongside the
existing error_model. Keeping it private avoids widening the public API
for something no caller outside this module should set. Ops registered by
users stay uncached: numba keys a cache entry on a stable on-disk source,
which lambdas and interactively defined functions do not have.

Measured here, fresh process, medians of 3, load average ~4 on a shared
machine so read these as indicative. First-touch floordiv alone: 434 ms
uncached, 347 ms warm (-20%). First-touch of all five: 1337 ms uncached,
733 ms warm (-45%). Writing a cold cache costs about 107 ms once for all
five (1444 ms on the run that populates it).

What stays: the remaining ~733 ms is numba/LLVM startup and per-process
object linking, which no on-disk cache removes. In-process dispatch once
an op is built is untouched. The first process on a machine is slightly
slower, not faster.

Read-only installs are safe, verified rather than assumed. With
graphblas/core/operator and its pycache chmod'd a-w, the run wrote
zero files in-tree and 66 .nbi/.nbc files under ~/Library/Caches/numba,
with empty stderr under python -W always. The NumbaWarning in
numba/core/caching.py is a source-content check, not a filesystem one.

docs/user_guide/udf.rst gains a "Compilation caching" section covering the
cache location, the read-only fallback, and why user ops are excluded.


Stack created with GitHub Stacks CLIGive Feedback 💬

@eriknw
eriknw marked this pull request as ready for review August 4, 2026 16:07
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 4130be1 to e848303 Compare August 4, 2026 16:12
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from e848303 to 408900a Compare August 5, 2026 00:06
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 408900a to e4f6717 Compare August 5, 2026 03:18
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from e4f6717 to 6aecc12 Compare August 5, 2026 17:44
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 6aecc12 to a934b2d Compare August 5, 2026 18:03
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from a934b2d to f8c1826 Compare August 5, 2026 18:05
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from f8c1826 to 188b387 Compare August 6, 2026 07:59
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 188b387 to d57105f Compare August 6, 2026 15:39
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch 2 times, most recently from 2265b21 to c4a03f3 Compare August 6, 2026 20:36
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from c4a03f3 to 5fc2928 Compare August 6, 2026 20:42
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from 5fc2928 to e2a01e7 Compare August 7, 2026 02:49
floordiv, rfloordiv, absfirst, abssecond, and rpow are UDFs, so every
process that touched one paid its numba compile again. They are plain
module-level functions, which is exactly the case numba can cache, so
that cost belongs once per machine rather than once per process.

Thread the flag through as a private keyword-only `_cache` on
register_new into _build, where it reaches numba.njit alongside the
existing error_model. Keeping it private avoids widening the public API
for something no caller outside this module should set. Ops registered by
users stay uncached: numba keys a cache entry on a stable on-disk source,
which lambdas and interactively defined functions do not have.

Measured here, fresh process, medians of 3, load average ~4 on a shared
machine so read these as indicative. First-touch floordiv alone: 434 ms
uncached, 347 ms warm (-20%). First-touch of all five: 1337 ms uncached,
733 ms warm (-45%). Writing a cold cache costs about 107 ms once for all
five (1444 ms on the run that populates it).

What stays: the remaining ~733 ms is numba/LLVM startup and per-process
object linking, which no on-disk cache removes. In-process dispatch once
an op is built is untouched. The first process on a machine is slightly
slower, not faster.

Read-only installs are safe, verified rather than assumed. With
graphblas/core/operator and its __pycache__ chmod'd a-w, the run wrote
zero files in-tree and 66 .nbi/.nbc files under ~/Library/Caches/numba,
with empty stderr under `python -W always`. The NumbaWarning in
numba/core/caching.py is a source-content check, not a filesystem one.

docs/user_guide/udf.rst gains a "Compilation caching" section covering the
cache location, the read-only fallback, and why user ops are excluded.
@eriknw
eriknw force-pushed the 31-numba-cache-builtin-udfs branch from e2a01e7 to ae91f10 Compare August 7, 2026 05:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant