When securing your CI/CD pipeline, it is not only about producing a secure product – it is also about securing the pipeline itself. Here are 7 major risks for your CI/CD pipeline and a few ideas on how to threatmodel your own CI/CD pipeline.

1. What is a CI/CD pipeline?

A CI/CD pipeline is a combination of systems that help you to build, test and even deploy a software in a mostly automated way – CI/CD means continuous integration and delivery or deployment. In the best case, software is built, tested and deployed in a completely automated way when a commit is promoted to the release or production branches.

When talking about CI/CD security we usually talk about integrating automated security testing of your software product in the pipeline. But looking at the pipeline and involved systems, there is a bit more to it.

2. Perspectives on CI/CD security

Changing the viewpoint on what to secure in your CI/CD pipeline may help to identify relevant threats. Here are some ideas when we talk about CI/CD security we can talk about:

• Security of the pipeline: Is the pipeline secure and are the involved systems appropriately secured?

• Security of the product: Is the final product that the pipeline produces free from vulnerabilities and malicious code?

• Security of the process: How can the process ensure that issues in the product are discovered in time, that a secure product is deployed, and intentional or accidental harmful interference with the process itself is limited as far as possible?

3. 7 Risks to Threatmodel your CI/CD pipeline

While there are a lot more risks and circumstances to consider when threatmodeling your CI/CD pipeline, here are major 7 risks we commonly see when analyzing CI/CD pipelines:

1) Malicious or vulnerable code in the product

Your product should not contain vulnerabilities or malicious code. To ensure that, your product should pass multiple checks and tests during the pipeline process – from static codescanning over manual code reviews, dynamic or interactive security scans on the build or even malware scans and fuzzing. There are plenty options on how and when to implement these checks. Make sure that sufficient tests are in place to discover as many relevant security issues as possible without hindering the deployment of relevant business features. Furthermore, ensure only authorized developers can commit changes to the source code and all stage transitions – from development to test to production – contain clearly defined quality gates that are enforced reliably.

2) Unprotected CI/CD systems and developer workstations

While testing the software is one part of your product’s security, all those checks can be circumvented or even disabled when the systems involved in the pipeline are compromised – if the workstation of your lead developer gets infected and malicious code is introduced or internal API keys abused, automated checks might not protect your product anymore. If your CI/CD systems itself are attacked and infected, the security checks of your product can no longer be relied on. Make sure that developer workstations and pipeline systems are adequately protected and user accounts follow the principle of least privilege to limit any potentially malicious behavior. This also applies to cloud-based components of your pipeline, where authentication settings, access control and user permissions need to be managed with appropriate care.

3) Vulnerabilities in 3rd party dependencies and images

While your developers might have control over their own code, they rely heavily on code that they have not written – open or closed source components, dependencies, libraries, or underlying operating system code as in container deployments. Make sure that foreign code has been checked for vulnerabilities, is kept up to date, stems from trusted sources, checksums and certificates have been validated so that no tampering during transfer or before use of the software components can happen - and, if it still happens, is detected in time. This also is important for systems which have already been deployed and running in production for a while - checks for vulnerabilities do not only need to be conducted during development but also after deployment (e.g. for running containers and container images that are not regularly updated) as the situation might change and new issues are discovered.

4) Secrets and credentials not well managed

Your CI/CD pipeline should be focused on producing secure and reliable code however, handling secrets, credentials, confidential API keys etc. requires sufficient care as well. Credentials should not be exposed in your code repository and be separated between your development, test and production environments. You should make appropriate use of a credential management system, to ensure that systems are transferred, stored and managed in a secure way, be it a vendor-based cloud system as provided in AWS or Azure or a cross-cloud system that fulfills your requirements. Also keep in mind that credentials might need to be changed quickly when they have been leaked, so the system you use should provide the needed flexibility to react quickly in case of an incident.

5) Deployment of insecure cloud and container infrastructures

Securing your applications is one thing, securing your cloud and container-based infrastructure that you deploy as code is another thing. This infrastructure should not only be free from OS level vulnerabilities in images, but also be configured in a secure way – no unprotected S3 buckets, no open SSH ports, no too generous IAM policies. Infrastructure as code deployments need to undergo sufficient policy validation and comply to the companies’ requirements for operational infrastructure, even when managed by DevOps teams directly and not a central operations team.

6) Deployment of insecure default configurations and extensive application attack surfaces

A typical mistake when deploying applications is that code and dependencies have been thoroughly tested – but the systems are still inappropriately configured for the environment. Make sure that production configuration does not contain default credentials, that sensitive resources (as your MongoDB) are protected by authentication and authorization, disable extensive debug logging and minimize your attack surface in general by disabling features that are neither needed nor secured - e.g. by disabling your REST APIs XML capabilities to resolve remote references or by restricting javascript sources with the help of a content security policy as far as possible.

7) Missing security logging and monitoring

As the speed of modern deployment pipelines does not allow for endless checks – and as all these checks will never be 100% - monitoring and logging becomes even more important in identifying and quickly resolving security issues – be it in your pipeline or in the final product. Detecting malicious code commits, hijacked developer workstations or failed updates of local vulnerability databases to check your libraries becomes just as important as detecting that nearly successful attack on your product, so ensure that security monitoring and security logging is not only working for your product, but also for your pipeline and processes to detect any unwanted activity or failure in time.

While these risks will give you an idea of what to look at when securing your CI/CD pipeline and product, the list is by no means complete. A threatmodel should be adapted to your setup, whether it’s a mix of cloud based and on-premise systems, programming languages, team sizes etc. to provide a more accurate picture. As a CI/CD pipeline quickly becomes a central technical component of your business, it deserves appropriate care - not only in terms of speed and reliability but also in terms of security of the involved systems, the product, and the process.