Scanning in the IDE: A Bad IDE(A) for Developers
The information shared in this series is derived from my experience building the DevSecOps program at Thermo Fisher Scientific (a global Fortune 100 laboratory sciences company).
Full Disclosure up-front: I am employed as a Principal Security Specialist at GitHub at the time of publishing this blog post.
There’s more to Security than “Shifting Left”
As security professionals we regularly use the refrain that we need to “shift left” - by which we mean introduce security practices earlier in the software development lifecycle. The problem with this (overused) term is that it has become a mindless refrain security vendors make without considering the consequences. In the era of smart phones we’ve become all too familiar with how easy it is to become distracted by our technologies - and performing security scans in an Integrated Development Environment (IDE) is no exception.
What IDE scanners are doing to a developer’s flow:
Image credit: XKCD (https://xkcd.com/356/)
IDE Scanners make for Distracted Developers
Have you ever found yourself having a conversation with someone, only to be suddenly interrupted by something (or someone) in your surrounding environment? Maybe it was someone suddenly interrupting you when you were speaking (I’m usually guilty of this 😅). Remember how challenging it is to get back to the point you were trying to make? That’s what an IDE Scanner is doing to developers every time it interrupts their focus to point out a security vulnerability in what they’re working on.
And as much as we like to think of software development as an engineering practice, the act of creating something when writing software is also a form of art. The cost of losing one’s train of thought is often measured and calculated in terms of lost time due to “context switching” - and this is what happens every time a notification pops up in the IDE alerting developers to a new vulnerability in the code they just wrote. These moments of context switching waste a great deal of money in lost business efficiency every year.
How much money, you ask? Well - we can calculate that! Using Stack Overflow’s 2022 Developer Survey we find that the average company cost for a development resource is approximately $100k USD annually for each region (with the exception of India). Assuming a mixed team of resources, I think this business cost is fairy representative. With that information in hand, we can calculate an hourly rate of approximately $48 per hour of lost productivity. Assuming our developer loses just ten percent of their work efficiency due to context switching from the IDE Scanner each, implementing such a tool is costing the business $192 per week in lost productivity. That’s over $9,000 USD annually!
The math is actually much simpler - just take the percentage of developer time lost to context switching as a percentage of $100k; in this case the proposed math comes out to $10,000 annually. Anyway - I really just wanted to use that GIF, and I think you get the point.
Before investing in an IDE Scanner, ask yourself - can the business afford to absorb this context switching cost when introducing such technologies? Can the business afford the cost of losing just 2 hours per week (5% of a developer’s time) from context switching driven by an IDE Scanner?
Even if IDE Scanners weren’t a distraction - they will be challenging to implement at scale
Unless the company has modernized and consolidated the development experience on a cloud-based IDE in the browser, assuming that the software engineering teams will use a consistent IDE is rather bold. Not only will you need to contend with IDE versioning problems within the same operating system and architecture, but add-in the fact that developers often use non-standard work systems personalized to meet their needs and, well, all I can say is good luck with that.
Assuming for a moment that you overcome this challenge by shipping the same device hardware, consistently patch and mirror the operating system(s) for all of these devices, and get each of these developers using the same IDE for writing and testing their code - you’ll still end up having to teach each of these developers to consistently perform scans. This effort would be akin to teaching a group of cats a dance routine. At best it will be an arduous, time consuming, and painful endeavor for the security team. At worst it will be an incredible waste of time and money with a trivial reduction in security risk to show for it.
How to support this content: If you find this post useful, enjoyable, or influential (and have the coin to spare) - you can support the content I create via Patreon.️ Thank you to those who already support this blog! 😊 And now, back to that content you were enjoying! 🎉
You still need to scan outside of the IDE for audits and ongoing security validation
The first reason you need to perform static analysis after developers commit their code comes from the fact that IDE Scanners lack visibility into complete data flows within the application. While scanning for vulnerabilities in the IDE is a neat trick, all current forms of static code analysis suffer from the same lack of deep, comprehensive data flow mapping for their analysis. This is doubly true for IDE Scanners in that limited system resources (namely RAM) prevent the tool from producing a comprehensive data flow. This limitation leads to an abundance of false positives (and false negatives) which consume system resources and cause developers to simply turn off the IDE Scanner altogether.
The second reason you still need to perform static analysis after developers commit their code is due to the fact that the default code branch could suffer from newly discovered vulnerabilities, and will benefit from newly introduced security checks that the static analysis vendor provides. Thankfully such static analysis scans will usually be more thorough than the IDE Scanner - but unless you’re doing comprehensive static analysis, you’ll still end up with a lot of findings pointing to local sources in addition to remote sources. This is a recipe for producing hundreds or thousands of findings, and a whole lot of security theatre. We can do better than this.
What developers really want: Results in the IDE
When it comes to “shifting left”, what developers really benefit from is providing scan results natively in their IDE. While it is still beneficial to aggregate this data in the Version Control System, developer happiness is highly correlated to the amount of time they spend in their editor of choice. If you can provide findings back to developers once a comprehensive static analysis scan has completed, not only will you produce more accurate and exploitable findings - you’ll also break the cycle of security alerts that run right over developer’s flow.
And if you don’t believe me - just ask the developers. As security teams, we do ourselves (and the businesses we support) a great disservice by failing to communicate, coordinate, and collaborate with our colleagues in the development side of the house. If we want to “shift left”, we should start by inviting developers to test the security tools we want them to utilize - and to trust them when they give us valuable feedback. Otherwise, what are we even doing here? <!–
TL;DR / Summary
As security professionals we often use the refrain that we need to “shift left” - by which we mean introduce security practices earlier in the development process. Unfortunate what this has lead to is the all-too-familiar notification/distraction loop from our security technologies - especially when performing security scans in an Integrated Development Environment (IDE).
As much as we like to think of software development as an engineering practice, the act of creating something when writing software is also a form of art. What we often fail to realize is that the cost of developer’s losing their train of thought - “context switching” - is what happens every time a notification pops up in the IDE alerting developers to a new vulnerability in the code they just wrote.
This context switching comes with a cost, and we can use Stack Overflow’s 2022 Developer Survey to help us calculate it. Finding that the average company cost for a development resource is approximately $100k USD annually (in all regions except India), we can calculate an hourly rate of approximately $48 per hour of lost productivity from context switching. Assuming a developer loses just ten percent of their work efficiency due to context switching from an IDE Scanner, such technologies can cost businesses $192 per week in lost productivity. That’s over $9,000 USD annually of lost productivity per developer!
If that isn’t enough reason to give you pause, then stop to think about how much effort in terms of time and resources it will cost your security team to implement and manage an IDE Scanner at scale. Unless the company has modernized and consolidated the development experience on a cloud-based IDE in the browser, it’s highly unlikely that software engineering teams use a consistent IDE.
And while scanning for vulnerabilities in the IDE is a neat trick, all current forms of static code analysis suffer from the same lack of deep, comprehensive data flow mapping for their analysis. This is doubly true for IDE Scanners in that limited system resources (namely RAM) prevent the tool from producing a comprehensive data flow. This limitation leads to an abundance of false positives (and false negatives) which consume system resources and cause developers to simply turn off the IDE Scanner altogether.
What developers really benefit from is providing scan results natively in their IDE. While it is still beneficial to aggregate this data in the Version Control System, developer happiness is highly correlated to the amount of time they spend in their editor of choice. If you can provide findings back to developers once a comprehensive static analysis scan has completed, not only will you produce more accurate and exploitable findings - you’ll also break the cycle of security alerts that run right over developer’s flow.
If you don’t believe me - just ask the developers. As security teams, we do ourselves and the businesses we support a great disservice by failing to communicate, coordinate, and collaborate with our colleagues on the development side of things. If we want to “shift left”, we should start by inviting developers to test the security tools we want them to use - and to trust them when they give us valuable feedback. Otherwise, what are we even doing here? –>
As always, thanks again for stopping by to read my pseudo-random musing 😊 While I prepare my next blog post on preparing for the Offensive Security Certified Professional (OSCP) certification, you can git checkout other (usually off-topic) content I’m reading over at Instapaper - or read my series on the DevSecOps Essentials if you want to learn from my mistakes when starting your own journey into DevSecOps.
Until next time, remember to git commit && stay classy!
Cheers,
Keith // securingdev
If you found this post useful or interesting, I invite you to support my content through Patreon 😊 and thanks once again to those who already support this content!