To the best of our knowledge, we are the first in the industry to discuss how to build such a compromise-resilient CI/CD system: that is, we protect the authenticity and integrity of Agent integrations, from the moment that our developers commit source code, to the point that our end-users install them as packages.
What exactly is your CI/CD system? You discussed to technologies in use by your CI/CD system, but not the exact system itself.
There was a post here on lobste.rs just a few weeks ago about compromised CI/CD systems and how to make them better. You don’t have to believe me, but I have been a part of many discussions on my own team, since 2017, discussing the security of our CI/CD system and making it more secure.
Essentially it sounds like you are using two mechanisms that both borrow from existing package-management tools as a part of your CI/CD pipeline. But the usage doesn’t guarantee a “compromise-resilient” CI/CD pipeline - just like attempting to integrate a build artifact signing and verification step into an existing Jenkins pipeline doesn’t guarantee a “compromise-resilient” Jenkins instance.
What exactly is your CI/CD system? You discussed to technologies in use by your CI/CD system, but not the exact system itself.
The point is that the exact CI/CD system we use — whether Jenkins, CircleCI, Travis, GitLab, Bamboo — is not important. We could replace what we use with any other CI/CD system, and we would still get exactly the same security guarantees.
There was a post here on lobste.rs just a few weeks ago about compromised CI/CD systems and how to make them better. You don’t have to believe me, but I have been a part of many discussions on my own team, since 2017, discussing the security of our CI/CD system and making it more secure.
I’m not aware of this post, and while I’m sure that others have worked on trying to secure their CI/CD systems, I’m not aware of any other compromise-resilient one.
Essentially it sounds like you are using two mechanisms that both borrow from existing package-management tools as a part of your CI/CD pipeline. But the usage doesn’t guarantee a “compromise-resilient” CI/CD pipeline - just like attempting to integrate a build artifact signing and verification step into an existing Jenkins pipeline doesn’t guarantee a “compromise-resilient” Jenkins instance.
The two mechanisms, TUF and in-toto, that we use do not borrow from existing package-management tools — in fact, it’s the other way around: many existing package-management tools borrow techniques now in TUF.
When used correctly, combining both TUF and in-toto does guarantee a compromise-resilient pipeline, because attacking any part of it between developers and end-users does not result in the total failure of the system: you cannot forge developer signatures.
I hope this makes sense. If it is unclear, please see our Kubecon talk and USENIX paper linked in the blog post. Thanks!
@sangy and I are happy to answer questions you have. Thanks!
This is an interesting bundle of statements:
What exactly is your CI/CD system? You discussed to technologies in use by your CI/CD system, but not the exact system itself.
There was a post here on lobste.rs just a few weeks ago about compromised CI/CD systems and how to make them better. You don’t have to believe me, but I have been a part of many discussions on my own team, since 2017, discussing the security of our CI/CD system and making it more secure.
Essentially it sounds like you are using two mechanisms that both borrow from existing package-management tools as a part of your CI/CD pipeline. But the usage doesn’t guarantee a “compromise-resilient” CI/CD pipeline - just like attempting to integrate a build artifact signing and verification step into an existing Jenkins pipeline doesn’t guarantee a “compromise-resilient” Jenkins instance.
Hi @crunxi,
The point is that the exact CI/CD system we use — whether Jenkins, CircleCI, Travis, GitLab, Bamboo — is not important. We could replace what we use with any other CI/CD system, and we would still get exactly the same security guarantees.
I’m not aware of this post, and while I’m sure that others have worked on trying to secure their CI/CD systems, I’m not aware of any other compromise-resilient one.
The two mechanisms, TUF and in-toto, that we use do not borrow from existing package-management tools — in fact, it’s the other way around: many existing package-management tools borrow techniques now in TUF.
When used correctly, combining both TUF and in-toto does guarantee a compromise-resilient pipeline, because attacking any part of it between developers and end-users does not result in the total failure of the system: you cannot forge developer signatures.
I hope this makes sense. If it is unclear, please see our Kubecon talk and USENIX paper linked in the blog post. Thanks!