You can build and install oneDAL using few simple command with conda environment manager. It automatically creates temporal environments for building and testing of oneDAL and outputs portable conda packages with shared and static libraries, library headers and development files.

Follow next steps to build and install oneDAL:

1. Install conda environment manager. Recommended installer is Miniforge.

2. Install conda-build tool:

    conda install conda-build
   

3. Clone oneDAL repository and go to its root:

    git clone https://github.com/uxlfoundation/oneDAL.git && cd oneDAL
   

4. Build oneDAL conda packages:

    conda build .
   

5. Install built packages from local conda index where they are stored:

    conda install -c local dal-devel dal-static
   

conda-build produces next oneDAL conda packages:

Required Software:

• C/C++ Compiler

• DPC++ Compiler and oneMKL if building with SYCL support

• BLAS and LAPACK libraries - both provided by oneMKL

• oneTBB library (repository contains script to download it)

• oneDPL library

• Microsoft Visual Studio* (Windows* only)

• MSYS2 (Windows* only)

• make; which can be installed using MSYS2 on Windows* as follows:

    pacman -S msys/make
  

For details, see System Requirements for oneDAL.

Note: the Intel(R) oneAPI components listed here can be installed together through the oneAPI Base Toolkit bundle:

https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit.html

All of these dependencies can alternatively be installed through the conda software, but doing so will require a few additional setup steps - see Conda Development Environment Setup for details.

Docker file with the oneDAL development environment is available as an alternative to the manual setup.

1. Clone the sources from GitHub* as follows:

    git clone https://github.com/uxlfoundation/oneDAL.git
   

2. Set the PATH environment variable to the MSYS2* bin directory (Windows* only). For example:

    set PATH=C:\msys64\usr\bin;%PATH%
   

3. Set the environment variables for one of the supported C/C++ compilers, such as Intel(R)'s DPC++ compiler. For example:

   • Microsoft Visual Studio* 2022:

       call "C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Auxiliary\Build\vcvarsall.bat" x64
     

   • Intel(R) oneAPI DPC++/C++ Compiler 2023.2 (Linux*):

       source /opt/intel/oneapi/compiler/latest/env/vars.sh
     

   • Intel(R) oneAPI DPC++/C++ Compiler 2023.2 (Windows*):

       call "C:\Program Files (x86)\Intel\oneAPI\compiler\latest\env\vars.bat"
     

   Note: if the Intel compilers were installed as part of a bundle such as oneAPI Base Toolkit, it's also possible to set the environment variables at once for all oneAPI components used here (Compilers, oneMKL, oneTBB) through the more general script that they provide - for Linux:

        source /opt/intel/oneapi/setvars.sh
   

4. Set up MKL:

   Note: if you used the general oneAPI setvars script from a Base Toolkit installation, this step will not be necessary as oneMKL will already have been set up.

   Download and install Intel(R) oneMKL. Set the environment variables for for Intel(R) oneMKL. For example:

   • Windows*:

       call "C:\Program Files (x86)\Intel\oneAPI\mkl\latest\env\vars.bat" intel64
     

   • Linux*:

       source /opt/intel/oneapi/mkl/latest/env/vars.sh
     

5. Set up oneAPI Threading Building Blocks (oneTBB):

   Note: if you used the general oneAPI setvars script from a Base Toolkit installation, this step will not be necessary as oneTBB will already have been set up.

   Download and install oneTBB. Set the environment variables for for oneTBB. For example:

   • oneTBB (Windows*):

       call "C:\Program Files (x86)\Intel\oneAPI\tbb\latest\env\vars.bat" intel64
     

   • oneTBB (Linux*):

       source /opt/intel/oneapi/tbb/latest/env/vars.sh intel64
     

   Alternatively, you can use scripts to do this for you (Linux*):

        ./dev/download_tbb.sh
   

6. Set up oneDPL Note: if you used the general oneAPI setvars script from a Base Toolkit installation, this step will not be necessary as oneDPL will already have been set up.

   Download and install Intel(R) oneDPL. Set the environment variables for for Intel(R) oneDPL. For example:

   • oneDPL (Windows*):

       call "C:\Program Files (x86)\Intel\oneAPI\dpl\latest\env\vars.bat" intel64
     

   • oneDPL (Linux*):

       source /opt/intel/oneapi/dpl/latest/env/vars.sh intel64
     

7. Build oneDAL via command-line interface. Choose the appropriate commands based on the interface, platform, compiler, linker, memory allocator, and the optimization level you use. Interface and platform are required arguments of makefile while others are optional. Below you can find the set of examples for building oneDAL. You may use a combination of them to get the desired build configuration:

   • DAAL interfaces on Linux* using Intel(R) C++ Compiler:

       make -f makefile daal PLAT=lnx32e
     

   • DAAL interfaces on Linux* using GNU Compiler Collection*:

       make -f makefile daal PLAT=lnx32e COMPILER=gnu OPTFLAG=O0 LINKER=bfd
     

   • DAAL interfaces on Linux* using Clang*:

       make -f makefile daal PLAT=lnx32e COMPILER=clang OPTFLAG=O1 LINKER=gold
     

   • oneAPI C++/DPC++ interfaces on Windows* using Intel(R) DPC++ compiler:

       make -f makefile oneapi PLAT=win32e LINKER=llvm-lib
     

   • oneAPI C++ interfaces on Windows* using Microsoft Visual* C++ Compiler:

       make -f makefile oneapi_c PLAT=win32e COMPILER=vc OPTFLAG=O2
     

   • DAAL and oneAPI C++ interfaces on Linux* using GNU Compiler Collection*:

       make -f makefile daal oneapi_c PLAT=lnx32e COMPILER=gnu OPTFLAG=O3 LINKER=lld
     

It is possible to build oneDAL libraries with selected set of algorithms and/or CPU optimizations. CORE.ALGORITHMS.CUSTOM and REQCPUS makefile defines are used for it.

• To build oneDAL with Linear Regression and Support Vector Machine algorithms, run:

        make -f makefile daal PLAT=win32e CORE.ALGORITHMS.CUSTOM="linear_regression svm" -j16
  

• To build oneDAL with AVX2 and AVX512 CPU optimizations, run:

        make -f makefile daal PLAT=win32e REQCPU="avx2 avx512" -j16
  

• To build oneDAL with Moments of Low Order algorithm and AVX2 CPU optimizations, run:

        make -f makefile daal PLAT=win32e CORE.ALGORITHMS.CUSTOM=low_order_moments REQCPU=avx2 -j16
  

On Linux* it is possible to build debug version of oneDAL or the version that allows to do kernel profiling using <ittnotify.h>.

• To build debug version of oneDAL (including debug symbols and asserts), run:

        make -f makefile daal oneapi_c PLAT=lnx32e REQDBG=yes
  

• To build oneDAL to include only debug symbols, run:

        make -f makefile daal oneapi_c PLAT=lnx32e REQDBG=symbols
  

By default, on x86 platforms, oneDAL uses highly optimized aligned memory allocators from oneMLK, but for easier debugging, it's also possible to make it use only allocators from the standard library. For this, add option STDALLOC=yes to the make call - but note that, when using the ICX compiler (default), adding this flag makes it statically-link the GNU stdlibc++ library.:

        make -f makefile daal oneapi_c PLAT=lnx32e REQDBG=symbols STDALLOC=yes

It is possible to integrate various sanitizers by specifying the REQSAN flag, available sanitizers are dependent on the compiler.

• To integrate AddressSanitizer in a debug oneDAL build (recommended), run:

  Note: Windows support of REQSAN in oneDAL is experimental, static AddressSanitizer can be set with value: static

        make -f makefile daal oneapi_c PLAT=lnx32e REQSAN=address REQDBG=yes
  

• To integrate MemorySanitizer in a debug oneDAL build, run:

  Note: Clang and Clang-derived compilers (including the Intel DPC++ compiler) support additional sanitizers such MSan, TSan, and UBSan

        make -f makefile daal oneapi_c PLAT=lnx32e REQSAN=memory REQDBG=yes
  

• To build oneDAL with gcov code coverage tool integration, run:

  Note: Only available when building with the Intel DPC++ compiler on Linux operating systems

        make -f makefile daal oneapi_c PLAT=lnx32e CODE_COVERAGE=yes
  

• To build oneDAL with kernel profiling information (REQPROFILE=yes):

  Note: if you used the general oneAPI setvars script from a Base Toolkit installation, those steps will not be necessary as Intel(R) VTune(TM) Profiler will already have been set up.

  • Download and install Intel(R) VTune(TM) Profiler.

  • Set the environment variables for Intel(R) VTune(TM) Profiler. For example:

      source /opt/intel/oneapi/vtune/latest/vtune-vars.sh
    

  • Run make to build oneDAL:

      make -f makefile daal oneapi_c PLAT=lnx32e REQPROFILE=yes
    

NOTE: Built libraries are located in the __release_{os_name}[_{compiler_name}]/daal directory.

After having built the library, if one wishes to use it for building scikit-learn-intelex or for executing the usage examples or samples, one can set the required environment variables to point to the generated build by sourcing the script that it creates under the env folder. The script will be located under __release_{os_name}[_{compiler_name}]/daal/latest/env/vars.sh and can be sourced with a POSIX-compliant shell such as bash, by executing something like the following from inside the __release* folder:

cd daal/latest
source env/vars.sh

The provided unit tests for the library can be executed through the Bazel system - see the Bazel docs for more information.

Examples of library usage for both the DAAL and oneAPI interfaces will also be auto-generated as part of the build, under paths daal/latest/examples/daal/cpp/source and daal/latest/examples/oneapi/cpp/source. These can be built through CMake - assuming one starts from the release path __release_{os_name}[_{compiler_name}], the following would do:

• DAAL examples:

cd daal/latest/examples/daal/cpp
mkdir -p build
cd build
cmake ..
make -j$(nproc)

• oneAPI examples:

cd daal/latest/examples/oneapi/cpp
mkdir -p build
cd build
cmake ..
make -j$(nproc)

This will generate executables under paths daal/latest/examples/daal/cpp/_cmake_results/{platform_name} and daal/latest/examples/oneapi/cpp/_cmake_results/{platform_name}. They can be executed as follows (note that they require access to the data files under daal/latest/examples/daal/data and daal/latest/examples/oneapi/data), assuming that one starts from inside the build folder (as at the end of the previous steps):

cd ..
./_cmake_results/{platform_name}/{example}

For example, in a Linux platform, assuming one wishes to execute the adaboost_dense_batch example:

./_cmake_results/intel_intel64_so/adaboost_dense_batch

DPC++ examples (running on devices supported by SYCL, such as GPU) from oneAPI are also auto-generated within these folders when oneDAL is built with DPC++ support (target oneapi in the Makefile), but be aware that it requires a DPC++ compiler such as ICX, and executing the examples requires the DPC++ runtime as well as the GPGPU drivers. The DPC++ examples can be found under examples/oneapi/dpc. These examples work only with dynamic linking and do not support static linking.

oneDAL samples are also auto-generated in daal/latest/samples/oneapi/cpp/(Multi-CPU) and daal/latest/samples/oneapi/dpc/(Multi-GPU) when oneDAL is built with DPC++ support (target oneapi in the Makefile). Note that building and running these samples requires a DPC++ compiler for GPU (such as ICX), as well as MPI/CCL for CPU/GPU execution.

• oneAPI samples:

cd daal/latest/samples/oneapi/cpp/mpi
mkdir -p build
cd build
cmake ..
make -j$(nproc)

Once built, the samples can be run with mpirun specifying the number of processes:

• On Linux:

mpirun -n {num_processes} ./_cmake_results/{platform_name}/{example}

• On Windows:

mpiexec -n {num_processes} ./_cmake_results/{platform_name}/{example}

When building oneDAL with ASAN (flags REQSAN=address, typically combined with REQDBG=yes), building and executing the generated examples requires additional steps - assuming a Linux system (ASAN on Windows has not been tested):

• Configure CMake to build the examples with the same compiler as was one for oneDAL (ICX by default) and with dynamic linkage to ASAN - e.g. by setting these flags:

  export CC=icx
  export CXX=icpx
  export CXXFLAGS="-fsanitize=address"
  export LDFLAGS="-shared-libasan"

• Create a symlink to the ASAN runtime in the same folder from where the examples are executed. ICX uses the same ASAN runtime as CLANG, so something like this should do:

  ln -s $(clang -print-file-name=libclang_rt.asan-x86_64.so) libclang_rt.asan.so

• Use the verbose mode in oneDAL when executing examples (otherwise ASAN won't produce prints):

  export ONEDAL_VERBOSE=1

Putting it all together, the earlier snippets for executing the examples but with ASAN enabled should look like this:

cd daal/latest/examples/daal/cpp
mkdir -p build
cd build
CC=icx CXX=icpx CXXFLAGS="-fsanitize=address" LDFLAGS="-shared-libasan" cmake ..
make -j$(nproc)
ln -s $(clang -print-file-name=libclang_rt.asan-x86_64.so) libclang_rt.asan.so
ONEDAL_VERBOSE=1 ./_cmake_results/intel_intel64_so/adaboost_dense_batch

Be aware that ASAN is known to generate many false-positive reports of memory leaks when used with oneDAL.

The previous instructions assumed system-wide installs of the necessary dependencies. On Linux*, these can also be installed at a user-level through the conda or mamba ecosystems.

First, create a conda environment for building oneDAL, after conda has been installed:

conda create -y -n onedal_env
conda activate onedal_env

Then, install the necessary dependencies with conda - note that these are not available in the Anaconda main channel, but can be installed from either conda-forge or from Intel's conda channel (https://software.repos.intel.com/python/conda/):

conda install -c conda-forge \
    make `# used by the build system` \
    dpcpp-cpp-rt dpcpp_linux-64 intel-sycl-rt `# Intel compiler packages` \
    tbb tbb-devel `# required TBB packages` \
    onedpl-devel `# required oneDPL package` \
    mkl mkl-devel mkl-static mkl-dpcpp mkl-devel-dpcpp `# required MKL packages` \
    cmake `# required to build the examples only` \
    impi-devel impi_rt `# required to build the samples only`

Then modify the relevant environment variables to point to the conda-installed libraries:

export MKLROOT="${CONDA_PREFIX}"
export TBBROOT="${CONDA_PREFIX}"
export DPL_ROOT="${CONDA_PREFIX}"
export LD_LIBRARY_PATH="${CONDA_PREFIX}/lib:${LD_LIBRARY_PATH}"
export LIBRARY_PATH="${CONDA_PREFIX}/lib:${LIBRARY_PATH}"
export CPLUS_INCLUDE_PATH="${CONDA_PREFIX}/include:${CPLUS_INCLUDE_PATH}"
export PKG_CONFIG_PATH="${CONDA_PREFIX}/lib/pkgconfig:${PKG_CONFIG_PATH}"
export CMAKE_PREFIX_PATH="${CONDA_PREFIX}/lib/cmake:${CMAKE_PREFIX_PATH}"

Note: variable $PATH is also required to contain ${CONDA_PREFIX}/bin, but that should have been handled automatically by conda activate.

After that, it should be possible to build oneDAL and run the examples using the ICX compiler and the oneMKL libraries as per the instructions.

For other setups in Linux*, such as building for platforms like aarch64 that are not supported by Intel's toolkits or using non-default options offered by the Makefile, other software can be installed as follows:

• GCC compilers (option COMPILER=gnu):

conda install -y -c conda-forge \
    gcc gxx c-compiler cxx-compiler

(no environment variables are needed for COMPILER=gnu)

• Reference (non-tuned) computational backends, and BLAS/LAPACK backends from OpenBLAS (both through option BACKEND_CONFIG=ref):

conda install -y -c conda-forge \
    blas=*=*openblas* openblas

• Optionally, if one wishes to install the OpenMP variant of OpenBLAS instead of the pthreads one, or to use the ILP64 variant:

conda install -y -c conda-forge \
    blas=*=*openblas* openblas-ilp64=*=*openmp*

Then set environment variables as needed:

export OPENBLASROOT=${CONDA_PREFIX}

(note that other variables such as TBBROOT and CMAKE_PREFIX_PATH are still required)