Supply Chain Risk: Log4j, SolarWinds, What’s Next?

Software ‘ate the world’ years ago, and now supply chain risk is eating firms alive, finds Phil Muncaster, as he investigates how software dependencies are becoming a popular backdoor to compromise

In 2011, venture capitalist guru Marc Andreesen wrote an article entitled: Why Software Is Eating the World. If it was true over a decade ago, it is beyond doubt today. Every business is a software business, even those that aren’t. If an organisation doesn’t directly offer customers software-based products and services, it will at least rely on back-end code to run the business. But as developers seek faster ways to keep up with insatiable demand for new applications and features, they unwittingly expose the organisation to Pandora’s Box of new cyber risks.

Getting a handle on these often-hidden risks will require more embedding of security into the software development lifecycle (SDLC) and greater visibility into a vast and increasingly complex supply chain.

Why is software risky?

Increasingly, it is open source components that power today’s software. Open source refers to software developed collaboratively and made publicly available for anyone to modify, use, or distribute. One report claims that developers made 3.1 trillion requests for these open source components last year alone.

The advantage?  Developers can speed up time to market by piecing together pre-built packages. That’s particularly useful for teams responding rapidly to changing market requirements. These components are present across proprietary and open source software: it’s claimed a massive 96% of codebases contain open source today.

The problem? These components, or ‘dependencies’, often contain vulnerabilities that could sneak into finished products if not scanned for properly. One estimate from the Linux Foundation claims there are 49 vulnerabilities across 80 direct dependencies in the average application development project. While these are relatively easy to detect, track and fix, the same is not true of indirect or transitive dependencies. These are effectively dependencies of dependencies: the libraries which direct dependencies call. As such, they may remain hidden from many developer teams until it’s too late.

The case of Log4j

This is precisely what happened with Log4j, a widely used logging tool featured in tens of thousands of Java software packages. Organisations may have tracked their direct Java package dependency but not the transitive Log4j dependency linked to it. So when a critical (CVSS 10.0) vulnerability was found in Log4j, global organisations were plunged into a race against the clock (and the hackers) to find and fix all Log4j dependencies in their environment. Many lost that race.

“In the case of Log4J, we discovered that open source software often comes bundled with numerous components and dependencies, both explicit and concealed”

“In the case of Log4J, we discovered that open source software often comes bundled with numerous components and dependencies, both explicit and concealed,” Vercara field CTO Michael Smith tells Assured Intelligence. “Uncovering these hidden software elements at scale poses a formidable challenge and leads to inherited vulnerabilities that remain concealed.”

Sonatype director of developer advocacy, Steve Poole, argues that five risks are associated with these complex supply chains. First, there is “insufficient due diligence” by developers, which lets buggy components slip through. Second, poor vetting of open source producers. Third, the use of underpowered software composition analysis (SCA) tools, which results in false negatives. Fourth, “keeping the developer out of the security loop”, which can mean assets are left untracked and unremediated. And five, sophisticated supply chain attacks. Sonatype claimed last year there had been a 742% increase in such attacks—where hackers plant malware in upstream components—over the previous three years.

Proprietary software is also a challenge

Yet this isn’t the only software supply chain threat facing organisations. As mentioned, commercial proprietary software also potentially contains open source components and tools like Log4j. Because these building blocks are usually hidden, it can give developer teams a false sense of security.

“The proprietary software’s source code is not open for public review, and this lack of transparency makes it harder for organisations to assess the software’s security and identify potential vulnerabilities,” IEEE senior member Kevin Curran tells Assured Intelligence.

“Without access to the source code, organisations must rely on the vendor’s claims about security practices. Additionally, there may be delayed patching as some vendors may not release security patches in a timely manner, leaving organisations susceptible to known vulnerabilities. This could occur due to a lack of resources or prioritisation on the vendor’s part, putting the organisation’s data and systems at risk.”

A further risk is of hidden malware in the proprietary code, he adds. This was the case in the infamous SolarWinds campaign, where state-backed threat actors compromised the eponymous software vendor’s IT environment and inserted malware in legitimate updates. Multiple US government agencies were compromised as a result.

The fightback starts with visibility

So, how can organisations manage these risks effectively? Let’s take proprietary software first. For the record, proprietary software is the opposite of open source.

“The consumer must examine the contract for the specific terms on software updates and security patches. Additionally, they should insist on a Software Bill of Materials (SBOM), whether in formal machine-readable form or not. This information will help assess the likelihood of existing vulnerabilities,” explains Sonatype’s Poole.

“It’s important to recognise that the SBOM dominantly addresses the ‘how’ rather than the ‘what’” Michael Smith

“Finally, assess the provider’s security posture—what do they say or claim about their processes?  Do they, for instance, have a published vulnerability reporting process? How often do they update the software?”

Samuel Barfoot, security team leader at Checkmarx, argues that for open source risk management, the key is implementing mature policies that prioritise selecting well-established and thoroughly tested components.

“Organisations should also incorporate open-source components into secure development pipelines, conduct comprehensive risk assessments, and proactively monitor transitive dependencies, including fourth-party assessments, to prevent potential vulnerabilities and code hijacking,” he tells Assured Intelligence.

For Vercara’s Smith, detecting and mitigating bugs in open source should be more straightforward than for proprietary code due to its relative transparency.

“A commit log provides a record of code additions to the project, enabling the analysis of the source code and understanding how vulnerabilities work,” he says. “By examining the changes, one can assess the impact on software built on top of the OSS project. However, this process does require time and information security professionals with a development background.”

DevSecOps

For many, the overarching answer to the software supply chain challenge will be DevSecOps, or integrating security into every stage of the software development and operations lifecycle, rather than leaving it to the final testing phase. Makes sense, right? But even then, teams must have the right tools to hand. Sonatype’s pool argues that more focus should be placed on evaluating SCA tools for their effectiveness and choosing those that can provide “qualitative insight into the security posture of the related projects.”

As with proprietary risk, a software bill of materials is widely regarded as a ‘must’ to manage open source supply chain threats.

“The concept has gained significant popularity recently, and it was included in an Executive Order from the White House in 2021. Like a food ingredients label, it provides crucial insight into the composition of the software and services you use,” explains Vercara’s Smith. “However, it’s important to recognise that the SBOM dominantly addresses the ‘how’ rather than the ‘what’. Its main value lies in enabling rational, data-driven decisions when responding to vulnerabilities or vendor compromises in open source or other software.”

The good news is that it seems to be taking off. Separate Sonatype research in the UK and US finds that 76% of enterprises have adopted an SBOM since the EO’s introduction, while another 16% plan to do so within the next year. Of those with SBOMs in place, only 4% adopted them over three years ago.

There’s a long way to go, but the tools and processes are at least there to help organisations shore up risk in this fast-expanding attack surface.