| <!-- --> | <!-- --> |

|----------------|----------|

| **Status**: | DRAFT |

| **Date**: | 2022-01-18 |

| **Author(s)**: | Max Maass <max.maass@iteratec.com> |

Software version numbers should serve as an indicator of compatibility between different versions, both during operation and while updating to a newer release.

At the moment, the _secureCodeBox_ is following the [Semantic Versioning (semver)][semver] approach.

According to semver, version numbers are of the format MAJOR.MINOR.PATCH, and the different places are incremented as follows:

However, the architecture of _secureCodeBox_, with its operator and many individual scanners that can be installed and used separately, makes this seemingly simple versioning system more difficult.

For example, we need to answer the following questions:

1. Are breaking changes in the parameterization of a scanner (e.g. nmap, Gitleaks) a breaking change for the entire _secureCodeBox_ project?

2. Are changes to the output format of a single scanner a breaking change for the entire project?

3. In an environment where operator and scanners aren't necessarily using the same version number because the operator is controlled by a different team than the scanners (a setup that we want to support), how do we indicate compatibility between different versions of operator and scanner?

Depending on how these questions are answered, different versioning schemes are possible.

The basic premise for this versioning scheme would be:

This manual action may include making changes to scan definitions, or to systems that are ingesting data from the SCB findings of a scanner.

It sees the entire _secureCodeBox_ as **one large piece of software with many components** that are all equally important to the overall compatibility, and where all components are (usually) updated in lockstep.

As illustration, here are a few examples and what kind of release they would require:

| Action |Version |

|------------------------------------------------------------------------------------|--------|

| A scanner changes how it is parameterized | Major |

| A scanner changes what data it returns | Major |

| The SCB makes changes to the findings format of one scanner (e.g., renaming a key) | Major |

| The SCB makes breaking changes to the CRDs (renaming or removing fields) | Major |

| The SCB makes backwards-compatible changes to the CRDs (adding new fields) | Minor |

| The SCB fixes a small bug in the operator or a scanner | Patch |

1. **MAJOR versions indicate that manual action may be required to keep existing workflows running** and that the users should read the changelog. However, it may turn out that the breaking change does not apply to the users' environment (e.g., because they are not using a specific scanner), in which case no manual action may be required.

2. **MINOR versions can be installed with the expectation that no manual action will be required to keep existing workflows working.** However, manual action _may_ be required to benefit from new features (due to changes to the CRDs that need to be manually installed).

3. **Frequent MAJOR releases lower the inhibition to make larger changes.** At the moment, many proposed larger changes are pushed back to "the next major version", without actively planning towards releasing such a version. Making major releases more common makes it easier to include smaller breaking changes that improve the consistency of the system (e.g., in the case of the output format).

4. **It is compliant with the expectations of users that expect SCB to be versioned like a single monolithic piece of software.**

1. **The MAJOR version no longer indicates compatibility between operator and scanner versions.** In environments where operator and scanners are updated separately by different groups, this increases complexity greatly (we know that such environments exist, and want to support them). It also raises the question how this compatibility will be documented instead.

2. **The versioning scheme does not distinguish between changes that are breaking to a small subset of users, and breaking to all users.** This makes it harder for users to distinguish based on the version number alone if an update is going to take 5 minutes (because all the breakage is in a component that they do not use) or 5 hours (because there are large changes to the CRDs, like there were between SCBv2 and SCBv3).

The basic premise for this versioning scheme would be:

This approach sees the _secureCodeBox_ as **a platform with independent components**, more akin to an operating system or kernel than a monolithic piece of software.

The MAJOR version number indicates compatibility between the operator and scanners, while MINOR version changes can still be breaking to some users (in which case this will be denoted prominently at the top of the release notes).

As illustration, here are a few examples and what kind of release they would require:

| Action |Version |

|------------------------------------------------------------------------------------|--------|

| A scanner changes how it is parameterized | Minor |

| A scanner changes what data it returns | Minor |

| The SCB makes changes to the findings format of one scanner (e.g., renaming a key) | Minor |

| The SCB makes breaking changes to the CRDs (renaming or removing fields) | Major |

| The SCB makes backwards-compatible changes to the CRDs (adding new fields) | Minor |

| The SCB fixes a small bug in the operator or a scanner | Patch |

1. **MAJOR versions indicate that a joint upgrade of operator and scanners is required,** which is highly relevant in environments where operator and scanners are maintained by different teams (we know that these environments exist, and want to support them). This obviates the need for a detailed compatibility matrix between scanner and operator versions.

2. **It is compliant to the expectations of users that expect SCB to be versioned like a platform or operating system.**

1. **MINOR version changes can be breaking to some users,** forcing everyone to read the changelogs of all intermediate minor version changes when upgrading a scanner.

A different variant of the previous option would be to prefix the version number with an ARCHITECTURE (ARCH) number, so the final versioning would be ARCH.MAJOR.MINOR.PATCH. In that case, the table would look like this:

| Action |Version |

|------------------------------------------------------------------------------------|--------|

| A scanner changes how it is parameterized | Major |

| A scanner changes what data it returns | Major |

| The SCB makes changes to the findings format of one scanner (e.g., renaming a key) | Major |

| The SCB makes breaking changes to the CRDs (renaming or removing fields) | Arch |

| The SCB makes backwards-compatible changes to the CRDs (adding new fields) | Minor |

| The SCB fixes a small bug in the operator or a scanner | Patch |

This could be considered "playing tricks" (by just adding a digit and relabeling the meaning of the positions) and would deviate from semver, but would allow us to use the ARCH number to denote operator compatibility, while the MAJOR version denotes scanner compatibility, the MINOR indicates feature additions, and the PATCH indicates bugfixes.

1. **ARCH versions indicate that a joint upgrade of operator and scanners is required,** which is highly relevant in environments where operator and scanners are maintained by different teams. This obviates the need for a detailed compatibility matrix between scanner and operator versions.

2. **MAJOR versions indicate that there have been breaking changes in a component (a scanner, a hook, ...), but compability with other components remains.** The exact breakage and how to address it is communicated prominently in the changelog.

3. **MINOR and PATCH versions can be installed without worrying about compatibility for existing scans.**

1. **The proposal deviates from the SemVer standard** and is thus unexpected for people who do not know about it.

For completeness sake, we also include this option, in which each component is versioned separately, so that a breaking change in one component only changes the verion number of that component.

We previous discarded this idea because it would require complex documentation of which scanner versions work with which operator versions.

We will thus not consider this proposal in greater detail here.

TBD

- Create a page in the documentation explaining the versioning scheme (as part of the yet-to-be-written "how to upgrade" page).

- Check the different files in the repo where the versioning and upgrading process are discussed, and make sure they are consistent with the new policy.

- Depending on the decision, potentially write a blog post explaining it.

- Depending on the decision, consider if automated compatibility tests that combine the most up-to-date operator and older scanners (plus, potentially, the reverse) would be a useful thing to have, to have an automated assurance that interoperability is actually maintained as advertised.

[semver]: https://semver.org/