Codecov Informs of a Supply Chain Hack That Goes 2.5 Months Back

Last updated April 17, 2021

Written by:

Bill Toulas
Cybersecurity Journalist

Codecov suffered a catastrophic supply chain attack that lasted for at least 2.5 months.
Someone took advantage of a misconfiguration and stole credentials required to tamper with a product’s code.
Several high-profile companies use the compromised tool, potentially suffering deeper infiltration.

Codecov has released a security notice to inform about the unauthorized access of a malicious actor onto their Bash Uploader script and its modification without their permission. Apparently, the firm’s developers had made a security-related error in the image creation process of Codecov’s Docker image, making it possible for the actor to extract the credentials required to modify the project’s code.

This was reported to the company by a customer on April 1, 2021, and a subsequent investigation has revealed that the unauthorized access started as far back as January 31, 2021. Immediately, the company informed the law enforcement authorities, engaged with forensic experts to conduct an investigation, and sent out notifications of a breach to all users who were confirmed to have been affected by the incident. Simultaneously, all employee credentials were rotated, an in-depth auditing action was launched, and network monitoring tools were deployed.

The Bash Uploader is used by a range of Codecov products, providing a framework and language-agnostic method for sending coverage reports. Users are often integrating it with Github Actions, Circle Orb, and Bitrise Step as well. While Codecov didn’t disclose how many clients were impacted by this supply chain attack that lasted for so long, the list is bound to be big. Also, the firm lists some high-profile customers on its website, like Atlassian, Tile, RBC, The Washington Post, Mozilla, Elastic, Kubernetes, Python, GoDaddy, and P&G.

Setu Kulkarni, Vice President, Strategy at WhiteHat Security tells us:

While the Bash Uploader is touted as an alternative for sending coverage reports to Codecov, its use is more mainstream than it being an alternative given its simplicity and convenience for the DevOps team that wants to automate as much as that can be. The likelihood of this script being embedded in 1000s of devops pipeline is very high, as illustrated by the fact that the script is also embedded within key integrations with Github, CircleCI & Bitrise, each of whom has 1000s of enterprise and individual users.

Why this particular incident is more unfortunate than others is because Codecov is a code coverage analysis product that basically lets the development team know whether their unit and functional tests indeed test the development team’s software. And importantly, Codecov informs development teams about the percentage of code that was not tested by the tests that were developed by the dev/devtest teams.

Those who have used the Bash Uploader script laced by the threat actor have run the risk of having their credentials, tokens, and keys that were passed through their CI runner stolen. Any services, datastores, and application code that required these credentials for access may have been compromised too. And finally, git remote information (URL of the origin repository) of repos using the Bash Uploader may have been exposed.

The advisory suggests that all credentials, tokens, and keys should now be re-rolled and also includes details about what specific checks need to be carried out now. In summary, though, the damage may have been done now as the actor had 2.5 months to exploit the high-profile firms that trust Codecov. And if it wasn’t for that diligent admin who ran a checksum validation of the downloaded software package and discovered the discrepancy, the actors could continue for much longer.