Skip to content

Build the C builtins without C library headers - #3977

Merged
aneshlya merged 4 commits into
mainfrom
fix-3887-nolibc
Nov 9, 2026
Merged

aneshlya merged 4 commits into
mainfrom
fix-3887-nolibc

Conversation

@azwolski

@azwolski azwolski commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Description

The C builtins for print() and num_cores() previously depended on target libc headers. Missing or incompatible headers could cause compilation errors hidden by the clang | llvm-as pipeline, producing empty bitcode and leaving these functions unresolved at link time.

This PR declares the small Unix libc interface directly and selects the target's stdout symbol and _SC_NPROCESSORS_ONLN constant.

CPU and WASM builtins now emit bitcode directly. Compilation errors stop the build instead of silently producing incomplete libraries; targets with failed builtins are not automatically disabled. On Windows, the WASM build command uses call to return from the emcc batch launcher.

Unix builtin generation no longer requires target libc headers, removing the Android NDK and obsolete GNUWIN32 libc-header dependencies. Windows builds still require m4, and Windows cross-target builds still require Windows SDK and Visual C++ headers. Without those SDK paths, the Windows target is disabled on Linux and macOS. Target SDKs and libraries may still be required when linking generated programs.

macOS ARM target support no longer depends on the detected macOS SDK version.

The PlayStation builtin triple is corrected to x86_64-scei-ps4, with PS4 and PS5 regression checks for sysconf(58). Unsupported target libc configurations now produce an explicit compilation error instead of falling through to the glibc declarations. Comments document the ABI sources and Android API 23 requirement.

Related Issue

Checklist

  • Code formatted with clang-format.
  • Git history contains one commit per logical change.
  • Compiler changes covered by lit tests.
  • Runtime behavior covered by functional tests.
  • Documentation updated.

@azwolski
azwolski force-pushed the fix-3887-nolibc branch 3 times, most recently from 1a65d4b to 0d93d36 Compare November 2, 2026 13:24
@azwolski
azwolski marked this pull request as ready for review November 2, 2026 15:22
@azwolski
azwolski requested a review from aneshlya as a code owner November 2, 2026 15:22
@azwolski
azwolski force-pushed the fix-3887-nolibc branch 2 times, most recently from 382b2d5 to 86c4ff4 Compare November 2, 2026 15:50

@aneshlya aneshlya left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 86c4ff4fb988f27f016e96ff987f417b6f29897d.

The overall approach looks sound and is a maintainability improvement: the small, stable libc interface is easier to maintain than the previous host/target header-selection matrix. Emitting bitcode directly also fixes the hidden compiler-error problem.

I would address the PlayStation ABI selection before merging. In get_target_flags(), the PlayStation triple is ${arch}-scei-ps. LLVM does not recognize ps as a PlayStation OS, so the new declarations fall through to the glibc branch and generate sysconf(84).

I compiled this PR's builtin source with LLVM 20, 21, 22, and 23 and consistently obtained:

Triple stdout symbol sysconf argument
x86_64-scei-ps (current) stdout 84
x86_64-scei-ps4 __stdoutp 58
x86_64-sie-ps5 __stdoutp 58

Public PS4 ABI definitions identify 84 as _SC_THREAD_CPUTIME, while 58 is _SC_NPROCESSORS_ONLN. The current code therefore selects CPU-time support instead of the processor count. ISPC also aliases the PS4 builtin library for PS5.

The smallest correction is set(triple ${arch}-scei-ps4), with a lit regression test for both --target-os=ps4 and ps5 checking sysconf(58). The malformed triple predates this PR, but this seems an appropriate place to fix it because the change explicitly selects the target libc ABI.

The stdout spelling alone is not evidence of a link failure: PS4 export records contain both stdout and __stdoutp. I have not validated against Sony's SDK or hardware; the finding combines reproduced compiler output with public ABI evidence.

Additional maintainability suggestions:

  • Replace the unconditional glibc fallback with an explicit supported Linux branch and an error for unsupported targets. This would catch malformed triples instead of silently assigning them an ABI.
  • Record the ABI sources and minimum versions beside the declarations/constants, especially Android API 23.
  • Add a build-level negative test for compiler-error propagation. The new lit tests validate successful builtin generation, but would still pass if the old pipeline were restored while compilation succeeds.
  • Finish removing the obsolete header-package dependency from Windows CI: install-build-deps.ps1 still downloads and extracts libgw32c-0.4-lib.zip, and fails the job if downloading fails. The separate gnuwin32-m4 tool is still needed.

Validation: built the PR with LLVM 23 and cross-target, macOS, ARM32, RISC-V, and PowerPC support; checked all 17 generated Unix builtin variants; ran the lit suite (706 passed, 6 expected failures, 105 unsupported, no unexpected failures); and ran the new runtime test on Linux x86 and x86-64. Nine CMake configuration cases behaved as documented. An injected compiler error reproduced the original bug: the base build succeeds and creates empty bitcode, while this PR fails and creates none.

@dbabokin dbabokin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

The reason we didn't do that earlier is that we were not comfortable with relying on the specific constants and stability of these values. But the tests and safeguards that this PR introduces look good. I think the reduction in the complexity of cross-builds outweighs the risks.

@aneshlya

aneshlya commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

Thanks, this looks good. There are several little things worth cleaning up before merging:

1. The m4 builtins still pipe into llvm-as (cmake/GenerateBuiltins.cmake:94-95)

COMMAND ${M4_EXECUTABLE} -I${include} -DBUILD_OS=${OS_UP} -DRUNTIME=${bit} ${input}
    | \"${LLVM_AS_EXECUTABLE}\" -o ${bc}

This is the same issue the PR fixes for the C builtins. The pipe drops m4's exit status, and llvm-as exits 0 on empty input, writing a valid empty module. svml.m4, util.m4 and util-xe.m4 stop on purpose with errprint + m4exit(1), so those errors are hidden too, as long as the output up to that point still assembles. Could m4 write to an intermediate .ll first, then llvm-as ${ll} -o ${bc}? Similarly, cat ${input} | llvm-as -o ${bc} in builtin_xe_to_cpp (line 342) could just be llvm-as ${input} -o ${bc}.

2. arm_macos_cross.ispc needs MACOS_HOST too (tests/lit-tests/arm_macos_cross.ispc:6)

The PR adds MACOS_HOST to arm64_macos.ispc and arm64_macos_cross.ispc, but not to the ARM32 version. Its first RUN line uses --target-os=linux, so on a Linux ARM host lGetARMDeviceType() sees no cross compilation and detects the host CPU (e.g. cortex-a53), while line 9 expects cortex-a35. Since MACOS_ARM_ENABLED no longer depends on the SDK version, -DISPC_MACOS_TARGET=ON on a Linux ARM host is enough to hit this:

// REQUIRES: MACOS_HOST && ARM32_ENABLED && MACOS_ARM_ENABLED && LINUX_ENABLED

Nits

  • Dead MinGW branch. builtins/builtins-c-cpu.cpp:178: the __MINGW32__ half of #if defined(_MSC_VER) || defined(__MINGW32__) can no longer compile, because MinGW without _MSC_VER hits the #error at line 63 first. No configured triple uses MinGW, so __MINGW32__ can be dropped there (or a MinGW branch added if it is meant to be supported).
  • MACOS_ARM_ENABLED. CMakeLists.txt:229: ISPC_MACOS_ARM_TARGET is now always equal to ISPC_MACOS_TARGET, so the MACOS_ARM_ENABLED lit feature no longer says anything separate. What actually decides whether macOS ARM builtins are built is ISPC_MACOS_TARGET AND ARM_ENABLED. Defining it as that condition (or removing it) would also make tests like cpu_arm.ispc and ftz_daz_flags_aarch64.ispc (!MACOS_HOST || MACOS_ARM_ENABLED) correct with ARM_ENABLED=OFF.
  • Termux ARM32_ENABLED=OFF. .github/workflows/scripts/build-ispc-termux.sh:23 keeps the flag, but the comment explaining it was removed, along with the one for ARM32_ENABLED in CMakeLists.txt. The original reason, missing 32-bit ARM headers, no longer applies. Either enable ARM32 so Termux CI covers the header-independent ARM32 builtins, or say why it stays off.
  • Leftover GNUWIN32 settings.
    • .github/workflows/scripts/init-env.ps1:18-20 still sets CROSS_TOOLS_GNUWIN32.
    • reusable.ispc.build.yml:209,219, pre-release-artifacts.yml:101 and rk-superbuild.yml:64 still pass -DGNUWIN32=%CROSS_TOOLS_GNUWIN32%.
    • Superbuild now only uses GNUWIN32 for LLVM's LIT_TOOLS_DIR (superbuild/CMakeLists.txt:205), and CI no longer populates that directory.
  • Linux cross packages. .github/workflows/scripts/install-build-deps.sh:43,46 still installs libc6-dev-armhf-cross, gcc-*-cross-base and similar packages, which were mainly needed for the builtin headers (including the RISC-V --sysroot). Some may still be needed to link or run tests, so it's worth checking which ones can go.

Declare the small Unix libc interface directly and select the target stdout symbol and sysconf constant. Preserve fputs and stdout-only flushing. Emit CPU and WASM bitcode directly so compiler errors stop the build instead of producing incomplete libraries.

Remove obsolete target-header dependencies from CMake, CI and superbuild while retaining Windows SDK requirements and the existing Termux and Windows-cross target selections. Add libc, architecture, processor-count and stdout-flushing regression coverage for #3887. Correct the PlayStation builtin triple to match ISPC target selection, and fix the iOS lit feature and its ARM backend prerequisite.
Record ABI references and reject unsupported target OSes. Exercise production builtin generation with an intentional compiler error from check-all, keeping check-one focused.

Call the Windows emcc batch launcher so subsequent builtin commands execute after direct bitcode generation.
@aneshlya
aneshlya merged commit 4269f01 into main Nov 9, 2026
167 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Build silently continues when builtins-c-cpu.cpp fails to compile for certain targets

3 participants