DevSecOps Essentials: An Overview
TL;DR / Summary at the end of the post. The information shared in this series is the distilled knowledge gained from my experience building the DevSecOps program at Thermo Fisher Scientific (a global Fortune 100 laboratory sciences company).
Full Disclosure up-front: I am employed as a Code Scanning Architect at GitHub at the time of publishing this blog post.
Background
At Thermo Fisher Scientific - a global Fortune 100 laboratory sciences company - I learned the hard way the challenges that come with building a scaleable DevSecOps program. When I joined the company in 2018, I remember my boss (and friend) Bryan saying it would take me 3 years to build the program - I saw it as a challenge, and set out to prove him wrong.
Thankfully I had leadership’s confidence when it came to trying new things. I built out the program from a team of one (myself) to a team of less than 10 people covering over 3,500 developers and more than a thousand applications in less than 18 months. This series of posts on DevSecOps Essentials contains the distilled knowledge I gained from that experience.
The DevSecOps Essentials
When it comes to applying DevSecOps principles at scale, there are several practices (and corresponding technologies) that are foundational to building a scaleable, long-lasting program. I break these practices (and technologies) into several investment layers as follows:
I intend to write additional posts for each of these practices (and their respective technologies), but for the sake of providing an overview I’ll share a brief summary as to what they are and why I consider them essential.
What you control before your code is in production:
01: Software Asset Management (Version Control System): To put it simply - you can’t secure software you don’t know about; This is the reason that “asset management” is listed as the #1 priority on any security standard worth your time. Centralizing on a single technology and set of practices here makes developers more productive, and allows security teams to scale their impact by orders-of-magnitude.
02: Code Security (SAST / Secrets Management / SCA): There are probably a few additional acronyms I could throw in here for “code security” technologies that correspond with this practice, but the point here is to focus on addressing issues within your codebase before it becomes technical debt. Once security vulnerabilities become technical debt, your company will be paying it back with interest until it eventually needs to be addressed - and let me tell you, when the debt collector shows up it can be really disruptive to the business.
03: Mitigations (Application Firewall): Some people might call this Runtime Application Security Protection (RASP) or In-Application Security Tooling (IAST), and I’m sure there are many other capabilities that go along with this practice. Anyway, the point here is to mitigate against risks your company has decided to accept. These are the first interest payments you’ll make until you decide to pay down the technical debt.
That being said, mitigations should never be treated as a panacea - what happens if they break? Or worse, what happens if the vendor jacks up their pricing and you decide not to renew - or have to use an inferior product because you can’t afford the tool anymore? The investments made here are the cost of doing business based on the risks that have been accepted. When that calculus changes, you’ll want to encourage the business to consider the cost of continued mitigation vs. the cost of remediating.
How to support this content: If you find this post useful, enjoyable, or influential (and have the coin to spare) - you can support the content I create via Patreon.️ Thank you to those who already support this blog! 😊 And now, back to that content you were enjoying! 🎉
What you can’t control once the code is in production:
04: Dynamic Testing (DAST): While you certainly can - and probably should - invest in your own Dynamic Application Security Testing tools in order to validate your mitigations, there’s nothing stopping attackers from using these same tools to test your application. When it comes to building a DevSecOps program, dynamic testing practices are used for reassurance/validation and risk assessment - but they do little to prevent vulnerabilities from making it into production.
05: Manual Testing (Hacking): There are regulations that require businesses to perform annual security assessments either via Penetration Testing or through Bug Bounty programs. That being said, investing here is a mistake unless you’ve made investments in the other four DevSecOps essentials first. The maturity required from both development and security organizations to make effective use of the information that comes out of this practice is fairly high. That being said, a bug bounty gone poorly can help unlock budget for everything else you might need.
In the interest of full disclosure here, I was a Trust & Security Engineer at Bugcrowd from 2017 to 2018. I was also a Top 100 security Researcher on the platform in 2018. Don’t make the same mistakes my targets made by investing in manual testing without the maturity necessary to effectively work with external security researchers.
Making an impact across every category:
00: Talent In writing this series I realized there is something missing from the investment categories I’ve drawn out. The one element that cuts across all categories and has the potential to make every investment produce an exponential impact is Talent 🧑💻 It comes in many forms, and with the right mix you can create repeatable, scaleable processes to address nearly any challenge companies might experience.
TL;DR / Summary
When building a DevSecOps program, starting with the essentials is the fastest and most impactful way to achieve DevSecOps speed at scale. By focusing on the things you control before code is in production (software asset management, code security, and mitigations) you reduce the cost for the interest payments you’ll need to make on technical debt - which will eventually need to be repaid.
Once you’ve built a solid foundation, expanding your program to include practices that attackers will use against your production application offers both validation for your mitigations, and helps in calculating the impact of the risks your company is willing to accept. When this calculus changes, it’s important that you encourage the business to consider the cost of continued mitigation vs. the cost of remediating the issue.
Anyway, I think that’s enough for this overview post on DevSecOps Essentials. I’ll be writing additional posts for each practice and their respective technologies throughout the next few weeks. Thanks again for stopping by, and until next time remember to git commit && stay classy!
Cheers,
Keith // securingdev
If you found this post useful or interesting, I invite you to support my content through Patreon 😊 and thanks once again to those who already support this content!😊