Skip to content

externpro/externpro

Repository files navigation

externpro

A CMake build platform and dependency provider with reusable CI (continuous integration) pipelines.

Contents

Why externpro?

externpro exists to make builds reproducible and reusable across projects: same toolchain, same dependencies, same CI - across developers, machines, and projects - so teams can move faster with fewer "works on my machine" surprises.

What pain does it remove?

  • Toolchain drift and "works on my machine" failures across developers and CI.
  • Dependency sprawl: each repo reinventing how third-party libraries are fetched, built, patched, and packaged.
  • CI duplication: repeated build/test/release logic that slowly diverges across projects.
  • Onboarding time: long time-to-first-successful-build for new contributors.
  • Supply-chain blind spots: difficulty answering "what did we build?" and "what is in this artifact?".

What you get instead:

  • A repeatable development and CI baseline that produces consistent results across environments.
  • Dependencies that can be shared and consumed across projects (rather than rebuilt ad hoc in every repo).
  • Software supply-chain metadata generated by CI so outputs are traceable and auditable.
  • Contributors can validate changes across supported platforms without reproducing every environment locally.

What makes externpro different?

  • A CMake-native dependency provider model (via CMAKE_PROJECT_TOP_LEVEL_INCLUDES injected through CMakePresets) that standardizes how dependencies are introduced to builds.
  • Prebuilt toolchain environments for Linux (buildpro images) that keep compilers, build tools, and system dependencies consistent.
  • Reusable CI pipelines that standardize build/test/release across repos.
  • Dependency reports produced (graph, table including versions and licenses) -- for example, curl at https://github.com/externpro/curl/blob/xpro/xprodeps.md

Why not just use a package manager? externpro is complementary, but it optimizes for a different workflow: cross-repo reuse + patch-friendly packages + CI-generated provenance.

A strength of CMake is that it has a low adoption cost: you can start with "just a compiler" and grow from there. externpro aims for the same "low ceremony" adoption model for builds and dependencies: you can add externpro as a submodule and start benefiting from consistent toolchains, dependency packaging, and CI workflows without first onboarding to a separate package-manager toolchain.

externpro is compatible with package managers (like vcpkg and Conan) — you can use them together.

How externpro differs from vcpkg/Conan/other package managers:

  • Adoption is repo-native: add externpro as a submodule; no separate bootstrap step is required just to participate in the externpro workflow.
  • Dependencies are delivered as build artifacts/packages that can be shared across projects and CI runs.
  • Patch-friendly dependency workflows: if you need to carry a change (static analysis fixes, compiler compatibility, backports, bug fixes or features not yet upstream), externpro makes it straightforward to build and publish a patched variant as part of the normal pipeline.

Who is externpro for?

externpro is for:

  • Projects that want to provide consistent, reliable builds across multiple platforms and environments (not just open source projects -- but also internal, proprietary projects)
  • Teams that need to manage dependencies across multiple projects
  • Developers who want to focus on writing code instead of setting up build systems
  • Organizations that want to standardize their build and dependency management processes

Best fit:

  • CMake-based C/C++ projects (including Node.js addons). However, the foundations are general and can be built upon for other languages and build systems.
  • Repos hosted on public GitHub or GitHub Enterprise Server.
  • Projects that build dependencies or want externally provided deps.

What externpro provides

externpro is intentionally "small surface area, big leverage". When you add externpro as a submodule to a git repository you get:

The goal is simple: make "develop -> build -> test -> release" fast and consistent across multiple platforms. Also make dependency sharing, tracking, and attestation easy across projects.

CMake Build Platform

  • Multiple OS: Linux, macOS, Windows
    • Rocky Linux 8, Rocky Linux 9, Rocky Linux 10, and Ubuntu 24.04 Docker base images with developer tools (CMake, Ninja, compilers) and package dependencies required to build a plethora of open source projects
  • Multiple CPU architectures: x86_64, arm64
  • Multiple toolchains (compiler versions) across platforms:
    • Linux: GCC 9/13/15
    • macOS: Clang (AppleClang)
    • Windows: MSVC (Visual Studio 2022/2026)
  • Architected to be easily extensible
  • CI pipelines run on GitHub-hosted runners, which provide the macOS/Windows toolchains (Linux uses buildpro containers for the toolchain)
  • CMake toolkit (functions/macros) for dependency provisioning, packaging (CPack, including installers like msi/rpm/deb when applicable), dependency reporting, and optional classified/offline source integration; see cmake/docs/

Leverage Docker buildpro images (Linux)

  • Pre-built toolchains
    • buildpro images from ghcr.io/externpro/buildpro/... give you a consistent compiler/tooling baseline without rebuilding a dev image from scratch.
  • One-command container lifecycle
    • compose.*.sh wrappers + shared shell functions (funcs.sh) drive Docker Compose with sane defaults.
    • ./docker-compose.sh -h displays a help message showing usage and options.
  • X11 forwarding
    • If you're running ./docker-compose.sh on a remote system you've connected to via ssh -X or ssh -Y, the denv.sh script should automatically detect this case and will do additional configuration and populate environment variables so that X display from the running container will (hopefully) work as expected.
    • If you get a "can't open display" error trying to run X applications, you may need to change the X11UseLocalhost option in /etc/ssh/sshd_config to no (and restart sshd).
  • Dev Containers support
    • devcontainer.json is set up to use your repo’s docker-compose.yml and attach to a named service.
  • Environment bootstrap
    • denv.sh generates a .env file used by compose for user IDs, host naming, X11 wiring, timezone, and optional certificate injection.
  • Project portability
    • Designed to be vendored as a .devcontainer submodule so every project can share the same foundation.
  • How it works (high level)
    • The compose.*.sh scripts specify a buildpro image (example: compose.bld.sh uses rocky-mdv).
    • denv.sh computes values (tag, user/group IDs, host name, timezone, X11 env) and writes .env.
    • compose.*.yml consumes .env to mount the workspace, forward SSH + gitconfig, and keep file ownership sane.
    • The result is a consistent, reproducible build environment that can be shared across projects and team members. It still feels "local": your editor can run on the host (or inside the container), your files are your files, and builds happen in a known toolchain.
    • buildpro images are designed to be easily extensible and can be customized to fit your needs.

Dependency Provider

  • Instead of CMake (or your "consuming" project) fetching/building a dependency itself, externpro "provides" that dependency to the build as an already-available package/target — so the consuming project can just link to it. See the CMake documentation overview on dependency providers.
  • For details on how externpro wires this into CMake, see Dependency provider (xproinc).
  • Leverage externpro as a dependency provider by using the provided CMakePresets, which automatically inject CMAKE_PROJECT_TOP_LEVEL_INCLUDES to point to .devcontainer/cmake/xproinc.cmake. The CMakePresets handle the injection automatically, so no manual set() call is needed in your CMakeLists.txt.
  • See cmake/README.md for externpro projects — each one is an example of vendoring externpro/externpro as a submodule and using it as the CMake build platform and dependency provider, with reusable CI pipelines, to create an xpro package (a release artifact plus its manifest metadata and generated CMake "use" config) that can be consumed by other "downstream" projects.

Provide your project as a dependency for others

  • Quickly and easily make your project consumable by other downstream projects by calling externpro-provided cmake functions to create an xpro package
  • For details on generating the xpro package, manifest metadata, and the generated use script, see Extern package.
  • For example, this is what "creating an xpro package" looks like in practice: spdlog calls xpExternPackage() in its CMakeLists.txt to create an xpro package, which includes metadata about the project (its dependency on fmt is determined automatically by externpro)
    xpExternPackage(EXPORT spdlog
      TARGETS_FILE spdlogConfigTargets
      LIBRARIES spdlog spdlog_header_only
      DEFAULT_TARGETS spdlog
      BASE v${SPDLOG_VERSION} XPDIFF "patch"
      WEB "https://github.com/gabime/spdlog/wiki"
      UPSTREAM "github.com/gabime/spdlog"
      DESC "Fast C++ logging library"
      LICENSE "[MIT](https://github.com/gabime/spdlog?tab=License-1-ov-file 'MIT License')"
      )

Reusable CI Pipelines

  • The caller workflows (xpinit, xpupdate, xpbuild, xptag, xprelease) are vendored into project repos under .github/workflows/.
  • They call reusable workflows provided by externpro (build, update, tag, release).
  • See GitHub Actions docs index.

GitHub-hosted Runners

The CI pipelines are designed to run on GitHub-hosted runners, providing a consistent and reliable build environment. https://docs.github.com/en/actions/concepts/runners/github-hosted-runners and https://github.com/actions/runner-images

Documentation

Getting Started

  • For adopting externpro into a project, see How to adopt externpro.
  • For workflow prerequisites before running xpInit, see xpInit preconditions.
  • For a hands-on example repo that walks through forking a project, adding externpro as a submodule, running xpInit, and producing a release that downstream projects can consume, see externpro/tutorial.

Optimally, externpro is added to any project as a submodule at the path .devcontainer:

git submodule add https://github.com/externpro/externpro .devcontainer

It is possible to add externpro as a submodule at a different path, but the workflows, templates, and documentation assume .devcontainer in many places, so using a different path requires updating those references.

History

There is a legacy externpro project at smanders/externpro that was continued at externpro/exdlpro (to avoid the naming conflict with this repo) that created a bundled tar.xz package of several third-party projects -- see releases -- this "legacy" externpro/exdlpro has now been phased out and archived as all projects now build standalone and host their packages as github release assets

  • The first commits of externpro (in a private repo) were made on 2012.02.22: "Initial commit, boost and patch" and "Make things more modular"
  • The initial commits of a public repo externpro were made on 2015.11.19

The legacy externpro made heavy use of cmake's ExternalProject module -- see Building External Projects with CMake 2.8 for a good overview of the module when it was first introduced -- and was the origin of the "externpro" name