DevSecOps Essentials: Code Security
TL;DR / Summary at the end of the post. The information shared in this series is the distilled knowledge gained from my experience building the DevSecOps program at Thermo Fisher Scientific (a global Fortune 100 laboratory sciences company).
Full Disclosure up-front: I am employed as a Code Scanning Architect at GitHub at the time of publishing this article.
A recap of the DevSecOps Essentials
In the first post in this series I provided an overview of what I consider to be the “DevSecOps Essentials”, in terms of both practices and their corresponding technologies. In the most recent post on Software Asset Management I covered many of the reasons you should implement a corporate version control system in order to scale your DevSecOps efforts - along with the business value proposition for doing so.
In this post I’ll be talking about Layer 02- Code Security, where I discuss the various practices and technologies that are worth investing your time and resources in so that you achieve maximum coverage during early stages of development. Before we jump in, here’s a recap of the diagram for how I categorize where companies should make investments in order to maximize the scale and impact of their DevSecOps program:
Background
Given the limited budget I had when starting out at Thermo Fisher Scientific, my boss (and friend - Bryan) told me I had enough money for either a Static Analysis tool or an Application Firewall 😅 When faced with this decision, I will admit that I went for the Application Firewall. At the time, I needed to protect the attack surface I knew about and could make an impact on.
After some successes there, further investments were made in the DevSecOps program and I found myself with enough resources (time, money, and people) to switch from being tactical to solving bigger problems in a strategic way. That’s when I went after consolidating software assets on a corporate version control system, along with my initial investments into code security.
Primary Investments
While there are many investments one could make in code security, there are a few Pareto Principle investments when it comes to hardening your company’s software assets. The 20% of your code security investments that can achieve an 80% impact should be in the three areas outlined below.
Be warned: if you haven’t started working toward consolidating on a corporate version control system, your investments in code security will have a significantly lower impact when it comes to driving remediation - and will be much more difficult to scale.
Static Analysis
I’ve written extensively on this topic, so I’ll try keep this section brief. There are a lot of tools that can perform Static Analysis, and they all pretty-much boil down to one of three methods. My advice here is to make investments in Static Analysis technologies that focus on developer experience, don’t require a lot of customization to have an impact, and can provide you (and your developers) with a data flow that shows where the source of your exploitable finding comes from.
Bonus points if you can acquire a technology that focuses on vulnerabilities stemming from remote sources, as this will eliminate a lot of findings that developers typically flag as false positives.
Software Composition Analysis (SCA)
Simply put, if your recipe is good but your ingredients are 💩 then you’re going to end up with a pretty awful meal. The same can be said for software - even after hardening the in-house code, the myriad of dependencies we import can lead to a business compromise.
To address this challenge, it’s important to (once again) focus on the developer experience and ease of implementation. People can’t address problems they can’t see, so it’s important to get your findings in front of developers where they work (*ahem* in the version control system). It’s just as important that the information you’re presenting to developers is brief, to the point, and easily understood.
Likewise, it’s worth investing in technologies that zero-in on findings where both the version in use has an exploitable vulnerability, and the tool can point to the use of that exploitable functionality. We’re starting to reach a point in this technology space where we are going from “a stack of hay that might have some needles” to “here’s a stack of needles”. If you can eliminate noise for your developers, you will benefit substantially when it comes to driving remediation.
Dealing with Secrets in your code
People make mistakes - and sometimes that mistake is committing a highly privileged AWS key to the company’s version control system. If the repository it was committed to is internally facing, then there’s not too much reason for alarm - just roll the credential, clean up the commit history, check the AWS logs, and make sure the developer learns the right way to go about importing secrets in their code.
On the other hand, if it’s in an open source repository then hopefully your company is using a centralized version control system that can hep you catch the issue before the key gets stolen. Even better, if your version control system can prevent the secret from being committed in the first place - now that’s winning the game!
Thankfully there are a lot of advancements being made in this space - but it’s important to remember the developer experience. As security professionals, we need to go beyond finding the problem and offer realistic solutions. If your team hasn’t provided a secrets management platform for your developers - and documented how to use it - then you can only blame yourself when the problem repeats itself.
How to support this content: If you find this post useful, enjoyable, or influential (and have the coin to spare) - you can support the content I create via Patreon.️ Thank you to those who already support this blog! 😊 And now, back to that content you were enjoying! 🎉
Secondary Investments
While there are a lot of other tools vendors want you to shove into the “Code Security” layer, I’m going to be a bit controversial here and share which technologies you can probably wait on (and why). If you’re still building out your DevSecOps program and have limited resources - money, time, or people - then below are the technologies that will have a 20% impact based on my experience.
Semantic Code Analysis(a.k.a Static Code Analysis)
I’m sure you’re thinking “wait, didn’t you just say Static Analysis should be a primary investment?” and the answer is yes - I did - but not all Static Analysis technologies are equally good at solving this problem at scale. Likewise, there are a lot of fun Open Source tools to play with in this space, but the need to write custom rules and/or sift through false positives will act as a drag on your resources.
If you’re going to invest in a Static Analysis technology, invest in something that does the hard work for you so that you can focus on driving remediation. And while it can be a fun activity for a hackathon, you should probably save the security research tasks for when you have enough time and people to make this investment; most companies just aren’t there yet.
On the other hand, if you’ve made investments in the other DevSecOps Essentials and you have resources to spare then look for a Semantic Code Analysis tool that makes writing new rules trivial and can produce results quickly. Such technologies can act as a force-multipler for internal security research teams, but will require broad access to the company’s source code to make good use of your investment.
Interactive Application Security Testing (IAST)
By the time your development teams are at the point where this technology becomes useful for detection, it’s probably too late to stop them from moving their code to production. Given how this technology works, it often gets treated as a panacea because it tends to offer built-in “blocking” capabilities along with detection functionality. What this means is teams will opt to move code to production in spite of existing vulnerabilities, making you dependent on the mitigations IAST offers - rather than remediating risks.
Likewise, this technology has been challenging to bring into DevSecOps programs because it often takes significant time and resources to scale - and there generally isn’t a line in the company’s budget for it. You can either get Static Analysis or IAST at this layer, but usually not both (due to cost). That being said, I think there is a place in the “Mitigations” layer where companies can (and should) find a place for it - which I’ll be talking about in my next post.
Infrastructure-as-Code Scanning (IACS)
Let’s face it, if your application is trivially exploitable at Layer 7 - then it hardly matters how secure your underlying Docker container is. Likewise, if your application is rock solid then the likelihood of being exploited at the infrastructure layer is going to be much lower.
That being said, I think it’s important to make security investments here eventually - and there are a number of Open Source tools out there that offer decent coverage in this space without a ton of investment. If you really need to address this problem, start with the Open Source tools and prepare templates for developers to pull from in your DevSecOps repository (located in that corporate version control system). After you’ve done that, re-assess how much more investment you really need to make here.
TL;DR / Summary
While there are a lot of ways to invest in code security, you only need three technologies to turn a 20% investment into an 80% impact; those technologies are Static Analysis, Software Composition Analysis, and Secrets Management. That being said - even within these investment areas - not all technologies are of equal value, or have an equal impact. A good example of this is the Static Analysis space. It has been over two decades since some of the earliest advancements were made, and not much has changed in that time.
Whatever investments you choose to make, the greatest impacts will come from focusing on developer experience, improving the signal-to-noise ratio of your findings, and providing templates (and documentation!) for developers to work from. In security we often forget that it is our job to go beyond finding problems. Whether you build, buy, or git pull your technologies to solve these challenges - invest in making it really easy for developers to get value out of them.
Most importantly, focus on people first. I haven’t included a specific section on that topic in the DevSecOps Essentials because it is a constant throughout the investments you’ll make as a business. If you prioritize people first, process second, and technology third you will find the investments you make will have an impact that grows by leaps and bounds. Good people will help you build a great process, which in turn can make (most) technologies useful and scaleable in your organization.
You know, that turned out to be a lot more content than I expected to write on this topic 😅 Anyway, stay tuned for what will hopefully be a more succinct post on Layer 03 - Mitigations. Your company’s code is about to hit production, and it pays to be prepared for when it does.
As always, thanks again for stopping by! You can git checkout other (usually off-topic) content I’m reading over at Instapaper - and until next time, remember to git commit && stay classy!
Cheers,
Keith // securingdev
If you found this post useful or interesting, I invite you to support my content through Patreon 😊 and thanks once again to those who already support this content!😊