Software Supply Chains

In-House software development often follows the same workflow, independently of the programming environment and from the organization itself:

Developer do not write everything from scratch, they reuse plenty of external tools: operating systems, language compilers, database management systems, web servers, external libraries, etc.

These tools are specified in the build system configuration scripts and stored in the version control system. There are several build systems available, for instance Maven and Gradle for Java and Kotlin; npm for Javascript ans Typescript, Cargo for , etc. These build systems will retrieve all pieces of software needed in the build.

The introduction of dependency manager software transformed the development lifecycle. The final software product no longer depends on some local developers and libraries, but a complex chain of suppliers, spread all over the world, using different developers, tools, delivery cycles, from active or inactive projects.

The local development lifecycle remains almost the same, the only difference is that before building deployable software artifacts, the build system retrieves other software artifacts and integrates them to the deployable artifact.

Thus, each time a build is triggered, the build system checks for updates on the imported artifacts and fetch the new artifacts. While it is not mandatory, the fetched artifacts are often (always) open source.

It is important to that the software product that we build and distribute has been built, configured, and controlled by our software development team as well as by external teams teams on which we have absolutely no control. There is nothing the developers can do about it: software systems are too complex nowadays and they cannot build everything from scratch, they must rely on external artifacts.

The build system integrates local artifacts, but it also fetches libraries from external sources of code, called package managers. When the developer think about reusing an external artifact, the build system will fetch the binary of this external library from a package manager, which is outside the scope of the code that is managed and controlled by the local (in-house) software developers. Thus, those package managers are typically databases that contain binary code for external artifacts.

For instance, your favorite NPM package or Maven artifact. It is important to understand that these binaries are themselves pieces of code, which have been built and developed by software development teams, which have put the code in the version control system, this code has been built, and then eventually pushed to the package manager.

As you would expected, this process is more complex than this, the build system not only depends on the local source code and one external package manager, but it can depend from many external package managers the developers will include in the build script.

Among all possible risk, the security ones are the most alarming. For instance, in March 2022, versions 10.1.1 and 10.1.2 of popular npm module node-ipc contained the code deleting file system content of IPs geo-located in Belarus or Russia.^[1]

Malicious code added in Git [3], but history got re-written. No external attackers, but politicized and disgruntled open-source maintainers.

In November 2018, the NPM package event-stream was attacked. This package has more than 1.5 million downloads per week and has 1,600 dependent packages.

A malicious user (right9control) asked the original maintainer to give him ownership and succeeded. Then, the new owner added malicious dependency flatmap-stream to the build script.

The new package containing the malicious code only was published to the NPM package the. Malware and decryption only ran in the context of a release build of the bitcoin wallet copay

This Malware was discovered only by accident, the use of deprecated command resulting in a warning

Another way to provide a robust software supply chains is to secure the development pipeline.

As mentioned earlier, the risk starts from one developer making a change in the code somewhere and then this change is passed to a version control system, and then it passes through build system to create a deliverable binary, which is pushed onto a package manager or a deployment platform.

All along the line, the question is how are we sure that the binary we get has not been tempered with, in other words, how do we ensure that the delivered binary contains no more and no less than the code that has been pushed by the developers.

Russ Cox, who is the main developer of the package manager for the Go language, identifies several challenges and proposes 6 ways to take car of software dependencies [Cox:2019]:

The first is to inspect the dependencies. This is related to the metrics previously mentioned. Given one external library, we analyze how often is it updated, how often is it maintained, how many people maintain it.

The second one is to test it. It is not because someone reuses a piece of code that we should assume it is correct.

Another proposal is to add an abstraction layer between your code and this external library to reduce the coupling. When you need to change this dependency, you will not need to change your code too much. Abstraction is related to the idea of isolating the dependencies.

A fifth way is to avoid external libraries. Today, it is very easy to reuse external code. This is a good thing, which has been promoted by the community. However, there is also a counterpart, developers tend to integrate in their products more libraries than he actually needs, so his advice is to avoid using libraries you do not need.

The last one is upgrade, which is a very generic guideline in terms of security. It is valid for software dependencies, as well as for any product that you have, it is always a good idea get the latest version.

There are today bots that automatically watch dependency and vulnerability databases, which are programs who are in charge of looking at your code and external libraries and check whether your external dependencies are up to date and then send you a warning, for instance, in the form of a pull request.

This kind of bots are available and running on the GitHub forge, for instance. In 2019, GitHub acquired Dependabot, which scans Java and JavaScript dependencies, compares with a vulnerability database, and proposes pull requests. GitHub claims to have one million Dependabot pull requests merged.

Renovate is a similar bot, which is open source, multi-language, has its own vulnerability database, and proposes automatic pull requests

Reusing external dependencies is not only necessary, but is also a good practice. However, developers should not depend on more dependencies he actually needs.

Depclean is an open-source tool for Java/Maven products. Depclean automatically generates a debloated build script for your projects. It detects dependencies that are declared but not necessary.

A BOM (for Bill of Materials) is an exhaustive list of all the components of a product, rom a single screw to electronic components to the parts that make up the packaging. It’s your shopping list for creating your product.

A software bill of materials, or SBOM, is a list of all the components and dependencies (proprietary and open source) that an application uses. SBOMs often also include information on the version of each component and dependency. This is important because security problems often only affect specific versions of the software.

SBOMs are generally formatted in a specific way, and can usually be represented as XML or JSON files. That said, the exact nature of an SBOM varies according to the SBOM standard - such as CycloneDX, SPDX or SWID - that teams choose to store and structure SBOM information. However, despite minor differences, all SBOM formats focus on making it easy to determine at a glance what “goes into” each application used by a company.

SBOMs are the only way to systematically track an application’s components and dependencies. By extension, they provide the information needed to determine whether components or dependencies are exposed to safety risks.

SBOMs are also important because, in some cases, they may be required by regulators or customers. For example, in 2021, the US federal government announced its intention to require SBOMs from suppliers who do business with government agencies, as part of an effort to ensure that agencies have some visibility into the software on which they depend.

What’s more, even if SBOMs aren’t strictly required by your customers, partners or regulators, generating them is an excellent way of showing that you take software supply chain security seriously. SBOMs offer external stakeholders the peace of mind of having greater visibility over software components and dependencies, and the ability to determine whether third-party software exposes them to risk.

The most efficient way to generate an SBOM is to take advantage of SBOM tools, which automatically track each application’s components and dependencies, as well as their versions, and then report this information in a standardized SBOM format, such as CyloneDX, SPDX or SWID. Using SBOM tools, you can make SBOM generation an automated task in your CI/CD process. SBOM tools can automatically analyze each new application version or package to determine whether application components have changed, ensuring that your SBOMs are continually updated.

The software supply chain is a complex mechanism, where an institution thats builds software products distributes code that was developed locally, as well as code written externally, by different institutions.

This mechanism is necessary, but this is also a risk for dependability and security.

There are many different solutions going from analyzing licenses and the social technical network of people who build those external libraries to more tool-oriented technical solutions that will run automatic dependency analysis and help you reduce the surface of the software supply chain.