-
Notifications
You must be signed in to change notification settings - Fork 345
ISPC Development Guide
You may follow the steps below or use docker files (available for Linux only) as a reference on how to build LLVM and ISPC.
- CMake 3.23 or later - use your package manager to install or to go http://www.cmake.org/cmake/resources/software.html
- Python 3.8 or later - use your package manager to install or to go http://www.python.org/download/
- Bison 3.0 or later - use your package manager to install. Note that the default version that comes with macOS is too old (2.3), using HomeBrew to get a newer one is recommended.
- Flex 2.6 or later - use your package manager to install.
- m4 1.4 or later - use your package manager to install.
- git client to check out LLVM and ISPC sources accordingly. Make sure that git has access to the Internet, i.e. all proxy settings are in place.
- Clang/LLVM
Additionally, on Windows you need:
- Visual Studio 2019/Visual Studio 2019 Build Tools or later
- GnuWin32 - http://gnuwin32.sourceforge.net/
- libc library for all target platforms - use your package manager to install (e.g. `g++-multilib` on Debian/Ubuntu).
These section explains how to build ISPC using prebuilt Clang/LLVM.
Ubuntu 22.04/24.04:
apt-get install git curl cmake xz-utils m4 flex bison python3 python3-dev libtbb-dev g++-multilib
Arch Linux:
pacman -S git cmake make gcc-libs glibc lib32-glibc gcc m4 flex bison python onetbb
choco install winflexbison3 choco install gnuwin32-m4
Build ISPC with the following commands:
git clone --recursive https://github.com/ispc/ispc ./ispc/scripts/quick-start-build.py
Build ISPC with the following commands:
git clone --recursive https://github.com/ispc/ispc python ispc\scripts\quick-start-build.py
On Windows, the quick-start-build.py script requires py7zr to unpack downloaded archives.
pip install py7zr
Note: Incremental builds can be performed by simply re-running the quick-start-build.py script.
There is a script named quick-start-build.py that automatically downloads Clang/LLVM and builds ISPC.
You can provide the required LLVM version as a positional argument. By default, this script downloads the current version LLVM used for development.
./ispc/scripts/quick-start-build.py 20
You can also specify the directory where LLVM/Clang will be unpacked using the LLVM_HOME environment variable.
bash:
LLVM_HOME=/path/to/llvm python quick-start-build.py
cmd (Windows):
set LLVM_HOME=C:\path\to\llvm && python quick-start-build.py
PowerShell:
$env:LLVM_HOME = "C:\path\to\llvm"; python quick-start-build.py
Additionally, there is an environment variable ISPC_HOME to specify where this script should create the build directory for the ISPC build. Under this directory, this script creates a build-<version_number> directory, where you can find the built ISPC executable, rerun the build, or inspect it. It also safe to rerun the entire quick-start-build.py script to rebuild ISPC after making changes to the codebase.
When running quick-start-build.py, the script downloads LLVM built on Ubuntu 22.04, which has glibc 2.35. So for quick-start-build.py to work correctly, your system must have glibc 2.35 or later. Below are the default glibc versions for different Ubuntu releases:
| Ubuntu Version | Default glibc Version |
|---|---|
| Ubuntu 18.04 | glibc 2.27 |
| Ubuntu 20.04 | glibc 2.31 |
| Ubuntu 22.04 | glibc 2.35 |
Here is a sample output from running quick-start-build.py with a Ubuntu 20.04 as operating system:
It fails with the following output:
Change Dir: /home/user/workspace/ispc/build-20/CMakeFiles/CMakeTmp
Run Build Command(s):/usr/bin/make cmTC_8dbd9/fast && /usr/bin/make -f CMakeFiles/cmTC_8dbd9.dir/build.make CMakeFiles/cmTC_8dbd9.dir/build
make[1]: Entering directory '/home/user/workspace/ispc/build-20/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_8dbd9.dir/testCCompiler.c.o
/home/user/workspace/llvm-20/bin/clang -o CMakeFiles/cmTC_8dbd9.dir/testCCompiler.c.o -c /home/user/workspace/ispc/build-20/CMakeFiles/CMakeTmp/testCCompiler.c
/home/user/workspace/llvm-20/bin/clang: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /home/user/workspace/llvm-20/bin/clang)
/home/user/workspace/llvm-20/bin/clang: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /home/user/workspace/llvm-20/bin/clang)
/home/user/workspace/llvm-20/bin/clang: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /home/user/workspace/llvm-20/bin/clang)
/home/user/workspace/llvm-20/bin/clang: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/user/workspace/llvm-20/bin/clang)
/home/user/workspace/llvm-20/bin/clang: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /home/user/workspace/llvm-20/bin/clang)
/home/user/workspace/llvm-20/bin/clang: /lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.13' not found (required by /home/user/workspace/llvm-20/bin/clang)
make[1]: *** [CMakeFiles/cmTC_8dbd9.dir/build.make:66: CMakeFiles/cmTC_8dbd9.dir/testCCompiler.c.o] Error 1
make[1]: Leaving directory '/home/user/workspace/ispc/build-20/CMakeFiles/CMakeTmp'
make: *** [Makefile:121: cmTC_8dbd9/fast] Error 2
What to do if your Ubuntu system has an older LLVM or glibc version? If your system has an older version that does not meet the minimum requirement, you have two options:
-
Upgrade or build LLVM manually to version
18.1.0or higher. ISPC can be built with LLVM installed from the system package manager if the version is supported by ISPC. However, some lit-tests may fail with the system LLVM. For example, on Ubuntu 24.04, you can install clang and llvm with the following command:
apt install llvm clang libclang-dev
- Then configure ISPC with:
-
PATH=/usr/lib/llvm-18/bin:$PATH cmake <ispc-source-dir>
- If you need help upgrading or building LLVM, refer to the official LLVM documentation.
- Build a Docker image. Dockerfiles are provided as part of the repository in the docker directory.
First, build the LLVM headers and libraries and the clang compiler on your system. For successful ISPC build LLVM must be built with proper flags. The recommended way to build LLVM is to use superbuild CMake configuration. It passes all required flags depending on LLVM version and applies required patches.
Note: all examples below are using LLVM 20.1 version but you can use any supported LLVM version. They also suppose that ISPC_HOME environment variable points to the path of ISPC repository and HOME_DIR points to the directory where an user starts to run command listed below.
git clone --recursive https://github.com/ispc/ispc cmake -B build-llvm ispc/superbuild --preset os -DLLVM_VERSION=20.1 -DCMAKE_INSTALL_PREFIX=llvm-20.1 -DBUILD_STAGE2_TOOLCHAIN_ONLY=ON cmake --build build-llvm
You may want to speed up your build with -jN switch, where N is a number of parallel jobs to run.
Once cmake build is completed, the llvm-20.1 directory contains clang and LLVM with required libraries and headers ready to be used to build ISPC.
Make sure to add bin folder under this directory to the system PATH environment variable.
For Linux/Mac OS:
export PATH=$HOME_DIR/llvm-20.1/bin:$PATH
For Windows PowerShell:
$env:PATH = "$env:HOME_DIR\llvm-20.1\bin;" + $env:PATH
For Windows cmd:
set "PATH=%HOME_DIR\llvm-20.1\bin;%PATH%"
You're now ready to build ISPC. Run CMake with the following options:
cmake -B build-ispc ispc cmake --build build-ispc
ISPC executable is placed under bin directory in the build directory. You may use it or provide CMAKE_INSTALL_PRERIX and run install target to install ISPC in the desired location.
ISPC-specific variables
- ARM_ENABLED - build ISPC with ARM support
- ISPC_INCLUDE_EXAMPLES - generate build targets for the ISPC examples
- ISPC_INCLUDE_TESTS - generate build targets for the ISPC tests
- ISPC_INCLUDE_UTILS - generate build targets for the utils
- ISPC_PREPARE_PACKAGE - generate build targets for ISPC package. It also disables generation of build targets for examples and sets static linking for Linux packages.
- BISON_EXECUTABLE - absolute path to Bison executable
- FLEX_EXECUTABLE - absolute path to Flex executable
- M4_EXECUTABLE - absolute path to m4 executable
You can build ISPC with cross-OS compilation support by passing -DISPC_CROSS=ON on Linux/macOS and -DISPC_CROSS=ON -DISPC_GNUWIN32_PATH=/absolute/path/to/GnuWin32/bin on Windows to CMake command described in 2.1. Depending on your host system ISPC may be built for Windows, Linux, FreeBSD, macOS, Android, iOS, and PS targets. You can disable any target system by passing -DISPC_[WINDOWS|LINUX|FREEBSD|MACOS|ANDROID|IOS|PS]_TARGET=OFF to CMake.
By default the following combinations are supported:
- Windows host - Windows, Linux, FreeBSD, Android, PS targets
- Linux host - Linux, FreeBSD, Android targets
- macOS host - macOS, Android, Linux, iOS targets
- ISPC_GNUWIN32_PATH - path to the root of [GnuWin32](http://gnuwin32.sourceforge.net/)
- ISPC_MACOS_SDK_PATH - path to the root of Mac OS SDK (is available on macOS systems)
- ISPC_ANDROID_NDK_PATH - path to the root of [Android](https://developer.android.com/ndk)
- ISPC_IOS_SDK_PATH - path to the root of iOS SDK (is available on macOS systems)
Xe-enabled build is supported on Windows and Linux, and it has three additional dependencies:
Please use exact commits SHA as used in osPresets and the same LLVM as you plan to build ISPC. CMake commands used in Dockerfile are applicable for both Windows and Linux. Install build artifacts from SPIRV-LLVM-Translator and vc-intrinsics to one folder (XE_DEPS) and artifacts from level-zero build to another folder (L0_ROOT_DIR).
Now you're ready to build ISPC with Xe support, just provide extra flags to CMake commands from 2.1: -DXE_ENABLED=ON -DXE_DEPS_DIR=$XE_DEPS -DLEVEL_ZERO_ROOT=$L0_ROOT_DIR
If you want to build a complete ISPC package with cross-compilation and Xe support, the easiest way to sue superbuild CMake using prebuilt LLVM from the stage 1. Run the following CMake commands:
On Linux:
cmake superbuild -B build-xe-cross-ispc --preset os -DISPC_CROSS=ON -DINSTALL_WITH_XE_DEPS=ON -DPREBUILT_STAGE2_PATH="${HOME_DIR}/llvm-20.1" -DCMAKE_INSTALL_PREFIX="${HOME_DIR}"/ispc-xe-cross
On Windows cmd:
cmake superbuild -B build --preset os -G "NMake Makefiles" -DINSTALL_WITH_XE_DEPS=ON -DPREBUILT_STAGE2_PATH=%HOME_DIR%/llvm-20.1 -DCMAKE_INSTALL_PREFIX=%HOME_DIR%\ispc-xe-cross -DGNUWIN32=%CROSS_TOOLS_GNUWIN32% -DISPC_CROSS=ON
The system consists of three main scripts:
- scripts/alloy.py - the main part of test system. This script executes other scripts and gathers their results.
- scripts/run_tests.py - stability part of test system. The script reports the names of failing tests for a given target.
- scripts/perf.py - performance part of test system. The script runs performance tests for CPU targets and reports out results for a given compiler.
- tests/fail_db.txt - file with list of known fails.
- tests/test_static.cpp - C++ file, which executes ISPC functions in stability testing for CPU targets.
- tests/test_static_l0.cpp - C++ file, which executes ISPC functions in stability testing for Xe targets.
- tests/test_static.isph - ISPC header containing CPU "export" wrappers for "task" functions used in tests.
- scripts/perf.ini - configuration file for performance testing.
- scripts/common.py - file with common functions of test system.
- examples - performance tests. Each test in its folder
- tests/func-tests - stability tests. Executes through test_static.cpp/test_static_l0.cpp
- tests/lit-tests - LLVM LIT regression tests
If you want to check stability of ISPC compiler after your changes you should use run_tests.py or alloy.py -r --only=stability. Note that you must set LLVM_HOME and ISPC_HOME for alloy.py
Run_tests.py will execute all tests from “tests” and “tests_errors” directories for a given target, arch and opt_level. Then it will report which failed tests (runfail or compfail). Also run_tests.py will check fail_db.txt file with known fails and will report if the fails are new or known.
If you want to have more general view you should use alloy.py -r --only=stability. This will run run_tests.py for all/selected supported targets, archs, opt_levels and LLVM versions (Note that generic targets will run only with x86-64 arch, this is by design). Then you will have full report about fails, new_fails, passes and new_passes. If you don’t have appropriate LLVM version alloy will silently download and build it. You can select combinations for your test runs by using option --only=”” and select targets by using --only-targets=””. Note that alloy will automatically detect targets of your system and SDE (if you set SDE_HOME environment variable).
If you want to change fail_db.txt file by adding or deleting fails of current run you should use option --update-errors=F (update fails) or --update-errors=FP (update fails and passes) both in alloy.py and run_tests.py.
Each test in tests folder is in a self-contained ispc source file checking particular functionality; it must define two functions:
- result(uniform float[]), which returns the result that the test function should return
- a test function, named one of f_v, f_f, f_fu, f_di, f_du, or f_duf. These various names encode the signature of the test function.
result() match the values returned from the call to the test function; if they don't the differing values are printed along with an error message.
To make this concrete, here is a example of a test (a cleaned-up version of tests/bool-float-typeconv.ispc). It does a quick sanity check of bool to float type conversion.
#include "../test_static.isph"
task void f_f(uniform float RET[], uniform float aUniform[]) {
float a = aUniform[programIndex];
RET[programIndex] = a < 3.;
}
task void result(uniform float RET[]) {
RET[programIndex] = 0; RET[0] = RET[1] = 1;
}
The test starts from #include "../test_static.isph". It contains CPU wrappers for task functions used in tests. It is motivated by different entry points of ISPC program on CPU and Xe - on CPU it is an export function, on Xe it is a task. To have unified set of tests for CPU and Xe we use task modifier in test functions and wrap them into export functions with a launch task inside for CPU using test_static.isph. For example, the wrapper for the f_f task will be:
task void f_f(uniform float res[], uniform float vfloat[]);
export void f_f_cpu_entry_point(uniform float res[], uniform float vfloat[]) { launch[1] f_f(res, vfloat); }
Now let's look into ISPC functions themselves.
First, notice that the test function defined here is f_f. In addition to the array in which to store the result values computed by the function, the RET parameter, functions with the name f_f are also passed an array of floats, with values {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}. The test function converts this to a varying value a by directly indexing into this array with programIndex, giving a the value one in the first program instance and so forth. By inspection, we can see that the boolean test in the last line of f_f should evaluate to true for the first two program instances running, and false for all of the rest, and that the conversion of those bools to floats should put 1 in the first two program instances result values and zero in the rest. These, in turn, are the values that result() reports are expected.
Here are the types and values passed as parameters by the test driver for functions with the various signatures listed above:
task void f_v(uniform float RET[]); // i.e. no parameters passed other than the output array
// a[] = { 1, 2, 3, ... };
task void f_f(uniform float RET[], uniform float a[]);
// a[] = { 1, 2, 3, ... };
// b = 5;
task void f_fu(uniform float RET[], uniform float a[], float b);
// a[] = { 1, 2, 3, ... };
// b[] = { 2, 4, 6, ... };
task void f_fi(uniform float RET[], uniform float a[], int b[]);
// a[] = { 1, 2, 3, ... };
// b[] = { 5, 6, 7, ... };
task void f_di(uniform float RET[], uniform double a[], int b[]);
// a[] = { 1, 2, 3, ... };
// b = 5;
task void f_du(uniform float RET[], uniform double a[], double b);
// a[] = { 1, 2, 3, ... };
// b = 5;
task void f_duf(uniform float RET[], uniform double a[], float b);
There is a slightly different group of tests related to print (e.g. print_uf, print_f, print_fuf). Both testing function and result prints output to stdout and run_tests.py checks that that output is correct.
New functionality should have targeted tests that exercise the set of features that the functionality introduces. If the functionality is in any way dependent on the mask, it's important to exercise a few cases like 'mask all on', 'mask all off', and 'mixed mask'.
To run the tests, run the run_tests.py python script in the top-level ispc source directory.
If successful, the test script will print output like:
% ./scripts/run_tests.py Executed 1517 / 1543 (26 skipped) PASSRATE (1517/1517) = 100% 1517 / 1543 tests PASSED 0 / 1543 tests FAILED compilation 0 / 1543 tests FAILED execution 26 / 1543 tests SKIPPED <List of skipped tests> No new fails %
If some tests fail, the test system will generate an additional output indicating which test failed and how it failed. The exit code is equal to the number of tests that failed. Thus, if all pass, it generates a regular exit code of 0.
Useful commands:
- run_tests.py will run all tests and report about fails, passes, new fails, new passes
- run_tests.py -a xe64 -t gen9-x16 --platform=skl will run all tests for gen9-x16 target and skl device, and report about fails, passes, new fails, new passes
- run_tests.py --target= avx1-i32x16 --arch=x86 --no-opt will run all tests with selected target, arch and opt level
- run_tests.py --target=avx2-i32x16 --wrap-exe=”sde -hsw -- ” will run target through sde
- run_tests.py --verify will verify file fail_db.txt
- To add skip rules – add STRING to .ispc file of test: STRING: “// rule: [skip,] on” + [arch=archname,]*
- alloy.py -r --only=stability will run tests with (all targets); -O2; (x86, x86-64); (LLVM 3.3, trunk). Each with each.
- alloy.py -r --only=stability --notify=mail@mail.com will send results to your e-mail
- alloy.py -r --only=stability --update-errors=F will add new fails to fail_db.txt
When running on Windows, failing tests may cause a dialog window suggesting to find a solution on the web or debug a problem. This may be annoying. To turn it off, you need to do two things:
- Turn debugging off by adding registry DWORD key "DontShowUI" equal to 1 in "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting". More info is here:
https://docs.microsoft.com/en-us/windows/win32/wer/wer-settings
- Turn problem reporting off. Start->Search for "Choose how to report problems"->Choose "Never check for solutions (not recommended)".
In addition to functional tests described in previous sections, ISPC has a set of lit tests in tests/lit-tests based on LLVM test infrastructure. They are mainly used to check compiler code generation or to check compiler diagnostics. To write ISPC lit tests, follow the same rules as for LLVM FileCheck.
ISPC lit tests can be run using make check-all on Linux or Tests/check-all target in Visual Studio on Windows.
There is a set of features that you can use in lit tests if you need to set up execution rules:
- WINDOWS_HOST, LINUX_HOST, MACOS_HOST - will run only on particular host
- X86_ENABLED, ARM_ENABLED, WASM_ENABLED, XE_ENABLED - will execute test if ISPC was built with support of the particular target
- LLVM_12_0+, LLVM_13_0+, LLVM_14_0+ - will execute test if ISPC was built with particular version of LLVM.
tests/lit-tests/lit.cfg
Example of ISPC lit test:
// The test checks that cpu definitions (including all synonyms) are successfully consumed by compiler.
// RUN: %{ispc} %s -o %t.o --nostdlib --target=sse2-i32x4 --cpu=znver3
// RUN: %{ispc} %s -o %t.o --nostdlib --target=sse2-i32x4 --cpu=alderlake
// RUN: %{ispc} %s -o %t.o --nostdlib --target=sse2-i32x4 --cpu=adl
// RUN: %{ispc} %s -o %t.o --nostdlib --target=sse2-i32x4 --cpu=sapphirerapids
// RUN: %{ispc} %s -o %t.o --nostdlib --target=sse2-i32x4 --cpu=spr
// REQUIRES: X86_ENABLED
// REQUIRES: LLVM_12_0+
uniform int i;
void foo() {}
All compiler changes should be covered by lit tests.
If you want to validate how your changes influence ISPC performance you should use perf.py or alloy -r --only=performance. Note that you must set LLVM_HOME and ISPC_HOME for alloy.py
If you want to measure performance of your changes use perf.py. Perf.py will build and run all tests listed in perf.ini from “examples/cpu” directory and report performance numbers. If you want to compare performance of two ISPC compilers you should use perf.py --ref=reference_compiler. This will generate a comparison report between two compiler versions.
If you want to compare two branches of ISPC (For example branch with your changes and master) you should use alloy -r --only=performance. This will build the newest LLVM if needed (Note that LLVM will be built silently. If you want selfbuild or source from tar you should use alloy -b first), build your ISPC compiler, switched to “master” branch, build reference compiler and then execute perf.py. Logs will be in alloy_results[date] directory. Option --compare-with=name_of_chekout_or_branch will change reference branch.
If you get suspicious results of runs you can increase the number of runs using -nX ( the switch is available in both alloy.py and perf.py).
- perf.py will run each test from perf.ini three times and prints results
- perf.py -n10 will run each test ten times. Use if results have big difference.
- perf.py -o to_excel.txt will write output file in machine-readable format.
- perf.py --ref=ispc_ref will compare current ISPC with other ISPC called ispc_ref.
- To skip tests – comment them in perf.ini
- alloy.py -r --only=performance will build test and ref compilers and run perf.py
- alloy.py -r --only=performance -n10 will run each test ten times
- alloy.py -r --only=performance --notify=mail@mail.com will send results to your e-mail
- alloy.py -r --only=performance --compare-with=”old_version” will compare to ISPC from “old_version” branch
With the introduction of generic targets in ISPC, adding a new target has become straightforward. Follow the steps below to integrate a new target seamlessly. You can use https://github.com/ispc/ispc/pull/3188 as a reference:
- Add the new target to the
ISPCTargetclass in target_enums.h and target_enums.cpp (a870459).
- Include the new target in the appropriate group, such as x86, Neon, or Wasm (e.g.
ISPCTargetIsNeon) (52337f5).
- Configure the target's defaults in ispc.cpp, following the pattern of existing targets (445e6e0). Key properties to define include:
ISAnativeVectorWidthnativeVectorAlignmentdataTypeWidthvectorWidthmaskBitCount
- Descriptions of these properties can be found in ispc.h.
- Create a new target file
target-<your_target>.llin the<root>/builtinsfolder using the appropriate template. - Add the target to the corresponding list in CMakeLists.txt (8af7dcf).
- Add the target to the
targetParentMapin builtins.cpp: determine the parent target, which provides standard library function implementations. If custom implementations are required, place them inbuiltins/target-<your_target>.ll. If you want to learn more about hierarchical ISPC targets, read this (030dc9e).
- Build ISPC and run the following command to confirm the target appears in the supported list:
ispc --support-matrix
- You can also run initial tests using:
scripts/run_tests.py --target=<your-target>
- Example:
scripts/run_tests.py --target=neon-i16x16
- Review the list of supported hardware features in ispc.h (features prefixed with
m_has, e.g.,m_hasHalfConverts). - For x86 targets, features are statically defined in ispc.cpp.
- For ARM targets, some features are dynamically detected at runtime (grep
lIsARMFeatureSupportedfunction to follow the logic). - Implement target-specific feature optimizations where necessary to avoid falling back to less efficient parent implementations (52b053b).
- Run all tests, add new ones if needed, and ensure they pass with different optimization levels:
scripts/run_tests.py --target=<your-target> -o [O0|O1|O2]
- Add the new target to the official ISPC documentation to provide visibility and guidance for the users (4fe913c).
- Add the new target to regular testing in ispc-ci.yml to ensure ongoing validation and stability (d05bc62).
Bisecting is often necessary to find the faulty change or the change that fixes an issue. This can be accomplished by dissecting ISPC and/or LLVM history with git bisect.
Important Note: Bisect works well when something changes monotonously in history (e.g., good → good → good → bad → bad → bad). However, if there are multiple points where the behavior swaps back and forth, bisect may find the wrong change.
Before starting bisect, you need:
- A reproducible test case that you can compile and run (or just compile)
-
Two known revisions:
- One revision where the behavior is "bad" (usually newer)
- One revision where the behavior is "good" (usually older)
old/new aliases instead of bad/good. You can also override these names with user-provided ones (consult the git manual for details).
It's highly recommended to minimize your test case before bisecting:
- Manually write a small reproducer
- Use minimization tools like
deltaor similar - Small programs are easier to debug and reason about
Sometimes the wrong behavior can be isolated to LLVM only. In this case:
- Create a direct test using
opt/llcon minimal LLVM IR - Use
bugpointto minimize LLVM IR
-
Start bisect:
git bisect start - Rebuild the project
-
Test your case and mark the revision as
good/bad(orold/new) accordingly - Repeat until the guilty commit is found
If the faulty behavior can be checked automatically, create a script and run:
git bisect run <your-test-script>
Bisecting is only possible on linear history. Use git merge-base to find start and end points of linear history when the project has forked release branches.
- Manual ifdef reversions: When bisecting ISPC history with a particular LLVM version, manual reversion of some ifdefs may be required (usually straightforward)
- Unbuildable states: If some state is impossible to build or check, it can be skipped with:
git bisect skip
This example demonstrates bisecting LLVM history when an ISPC test starts failing due to an LLVM update.
- Test
tests/func-tests/bug.ispcstarted to fail at some point - Known working state: LLVM version K with ISPC 1.N
- Known failing state: LLVM version K+1 with ISPC 1.N
# Known failing state PATH=/path-to-llvm-K+1/bin:$PATH ./scripts/run_tests.py ./tests/func-tests/bug.ispc # → FAIL # Known working state PATH=/path-to-llvm-K/bin:$PATH ./scripts/run_tests.py ./tests/func-tests/bug.ispc # → OK
# Checkout known good LLVM version
[/llvm-project] $ git checkout K
# Configure LLVM (build only what you need for faster iteration)
[/build-llvm] $ cmake /llvm-project/llvm -G Ninja \
-DLLVM_TARGETS_TO_BUILD="X86" \
-DLLVM_ENABLE_PROJECTS="clang" \
-DCMAKE_INSTALL_PREFIX=/dir-llvm-bin \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_ZLIB=OFF \
-DLLVM_ENABLE_ZSTD=OFF \
-DLLVM_INSTALL_UTILS=ON
# Build and install
[/build-llvm] $ ninja && ninja install
# Checkout ISPC version
[/ispc] $ git checkout 1.N
# Configure ISPC
[/build-ispc] $ PATH=/dir-llvm-bin/bin:$PATH cmake /ispc -G Ninja \
-DISPC_SLIM_BINARY=ON \
-DARM_ENABLED=OFF
# Build ISPC
[/build-ispc] $ ninja
[/ispc] $ PATH=/build-ispc/bin:$PATH ./scripts/run_tests.py ./tests/func-tests/bug.ispc # Should return: OK
# Initialize bisect [/llvm-project] $ git bisect start # Mark current (known good) commit as good [/llvm-project] $ git bisect good # Checkout known bad commit [/llvm-project] $ git checkout K+1 # Rebuild and install LLVM [/build-llvm] $ ninja && ninja install # Rebuild ISPC [/build-ispc] $ ninja clean && ninja # Test the behavior (should fail) [/ispc] $ PATH=/build-ispc/bin:$PATH ./scripts/run_tests.py ./tests/func-tests/bug.ispc # Should return: FAIL # Mark as bad [/llvm-project] $ git bisect bad
At this point, git will change HEAD to an intermediate commit. Repeat the following process:
-
Rebuild LLVM:
ninja && ninja install -
Rebuild ISPC:
ninja clean && ninja -
Test:
PATH=/build-ispc/bin:$PATH ./scripts/run_tests.py ./tests/func-tests/bug.ispc -
Mark result:
git bisect goodorgit bisect bad
# When done, clean up bisect state [/llvm-project] $ git bisect reset
- Clean builds: Always do clean rebuilds of ISPC to avoid stale build artifacts
- Consistent environment: Keep your PATH and build environment consistent throughout the bisect process
- Document your process: Keep notes of the commands and results for future reference
-
Automate when possible: If you can script the test, use
git bisect runfor faster results - Minimal configuration: Build only what you need to speed up the bisect process
-
Build failures: Use
git bisect skipfor commits that don't build - Inconsistent results: Verify your test case is deterministic
- Long bisect sessions: Consider using faster build configurations