TL;DR / Summary at the end of the post. The information shared in this series is derived from my experience building the DevSecOps program at Thermo Fisher Scientific (a global Fortune 100 laboratory sciences company).
Full Disclosure up-front: I am employed as a Code Scanning Architect at GitHub at the time of publishing this blog post.
As DevSecOps practices continue to gain momentum in large organizations, security teams are forced to grow beyond immature, silo’d modes of operating in order to meet the challenges modern software development demands of them. The maturity required to integrate seamlessly with these new processes has changed significantly over the last two decades, and so security must get up to speed quickly - or risk being left behind.
As it stands today, Security Maturity - in terms of both teams and their technologies - can be measured across four stages based on the direction they’re heading (forward ➡️ or backward ⬅️), and the outcomes they produce. I think of these four stages in terms of breadth and depth as follows:
As with all things, time will eventually wear on the efficacy of Security teams - DevSecOps or otherwise. Whether priorities shift, business investments dry up, key talent moves on to new opportunities, or leadership changes - the result is the same: the security organization becomes a shell of its former self. The depth and breadth of coverage the team is able to provide shrinks; processes start to unravel; technologies start to breakdown.
The speed with which mature teams sunset into a state of becoming geriatric largely depends on the attitude, aptitude, and engagement from the team - as well as how solid their technology automation is, and how well documented their processes are. Ultimately, the amount of progress willingly lost to time is up to leadership and the business to determine.
I was fortunate in that I never experienced my team backsliding into a geriatric state while at Thermo Fisher Scientific. I left an exceptionally talented team in the more-than capable hands of an engineering manager who understood and valued the team’s capabilities, as well as the Agile and DevSecOps processes we built together. Thus, my views on geriatric teams stem from my experience working with - and witnessing - other teams slide into this stage of maturity.
The first sign of a team fading into a geriatric state will occur when hole(s) in the team’s communication network start to form - which is usually a result of key talent or leadership departing the organization. The value created from relationships that have been developed and nurtured across the organization is lost with such departures.
This is the beginning of a domino effect, as stakeholders who used to be consulted or informed about decisions impacting their team(s) are now informed on shorter notice than before - if at all. Likewise, the information provided is often thinner than the previous details available to stakeholders, and there is less of an opportunity to shape security decisions when these stakeholders are no longer being consulted with. This state of affairs begins to generate friction between the remaining members of the security team and their business colleagues.
Avoiding this breakdown in communications will require rebuilding the network of relationships lost to attrition. It is of paramount importance that leadership undertakes this effort in order to slow a geriatric team’s regression into former adolescent or immature practices. Likewise, if you find yourself joining a geriatric team as a new hire - one of the fastest ways you can help the team rebuild their maturity is by setting a regular cadence of one on one meetings with stakeholders.
As lines of communication start breaking down, the volume of coordinated efforts shrink to being focused on a handful of key projects and initiatives that the business cannot afford to break. How many of these key initiative “spinning plates” are kept spinning without heroic effort is a measure of how solid the team’s documentation, processes, and technology automation were when key talent and/or leadership moved on to new opportunities.
Due to a lack of coordination, the security team loses visibility and influence with their stakeholders. What this usually leads to is the adoption of “Shadow IT” where either the stakeholders purchase their own (often ill-informed) solutions that supposedly solve for the problems the security team used to help with. Or worse, the once-trusted security team becomes relegated to “checking the box” while lines of business ignore their guidance.
Either way, the friction that geriatric security teams experience comes from this gradual slide toward obsolescence. Dwindling capacity for coordination means that fewer initiatives get the time and attention they require in order to meet the needs of the business at scale. And because fewer business needs are being met by the geriatric team, lines of business begin arbitrarily making security investments on their own without consulting the security team.
As with coordination, collaboration between geriatric security teams and their business stakeholders diminishes substantially from the earlier days of security maturity. At this stage, collaboration often turns inward toward other security teams at the expense of stakeholders in the business.
This usually starts with a cobbling-together of distinct security teams under new leadership as part of a re-organization. This is unfortunate in that security teams who rightfully deserve their own senior leadership and representation within the business are forced to adopt the interests of this new security leader, who’s interests may be at odds with the team’s earlier mission.
I say this is unfortunate because part of the security organization’s interests are responsible for mitigating internal business risks - such as Security Operations, Incident Response, Vulnerability management, and so forth. The other part of the security organization’s interests are responsible for empowering the business to mitigate customer facing risks - such as Security Research, DevSecOps, Product Security, Privacy and so forth.
These two distinct interests are sometimes in opposition with one another, and consolidation of these interests in order to “streamline security activities” often leans toward focusing on mitigating internal business risks. What this means for business stakeholders is that processes which were previously focused on integrating security into business practices are transformed into security practices that impose upon (and seek to change) such practices to serve the interest of internal security teams.
In short, this is a recipe for increasing friction between security teams and business stakeholders. While it’s important to have collaboration take place amongst security teams so that the interests of the business and customers can be held in balance - by stepping away from collaborative efforts focused on empowering the business, security teams erode credibility with business stakeholders.
For geriatric security teams, success is measured by how long the team can keep the proverbial plates spinning - by which I mean how long the team can sustain its current level of support for processes and technologies developed during earlier stages of security maturity. If the business quickly reinvests in key talent and/or leadership, the team should be able to course-correct and get back on track toward making progress on initiatives with their business stakeholders.
On the other hand if business investments stagnate or dry up, what usually happens is a decrease in the scope and impact of the security team’s work. This pattern usually coincides with further loss of key talent, until those that are left are the people who say things like “this is the way it has always been done”. Once you’ve reached that point, the team has arrived back at square one of the security maturity journey.
Then again, the company could also outsource the security team’s work - which is both more expensive to the business, and generally produces worse security outcomes. Going this route is often a “check-the-box” exercise meant for companies with less than 50 employees, and should not be considered with any degree of seriousness beyond ad-hoc staff augmentation for a majority of companies.
My philosophy for building mature security teams revolves around the simple mantra of “People first, Process second, Technology third”. Talented people will help you build an effective and resilient process, which can make good technologies great - and mediocre technologies “good enough”. Rekindling security maturity in geriatric teams starts with investing in great talent - which doesn’t necessarily mean hiring people outside the organization.
You might have talent with the right aptitude, attitude, and level of engagement needed to rekindle youth within the team - and if you do, start by nurturing that talent first. Often times what this kind-of talent needs is a leader who will give them reasonable challenges, training necessary to take on such challenges, and the trust needed to let them fail (or succeed) in their own way. If you can do this with a group of individuals - and they start to form bonds as a team - you’ll be surprised how quickly things can turn around.
On the other hand, sometimes it will be necessary to bring in outside talent to help jump-start the security maturity journey. Often times this talent needs to be in a leadership or senior individual contributor role in order to trail blaze and set the pace for the team. But be warned: if you hire exceptional individual contributor talent while failing to invest in the process and technology recommendations they make - they will leave, and you’ll be back at square one all-over again. This sequence of events can create friction with members of the existing team in the process - so be careful when bringing in outside talent.
As with all things over time, mature teams eventually sunset toward geriatric security maturity. Whether this happens because priorities shift, business investments dry up, key talent moves on to new opportunities, or leadership changes - the result is the same: the security program becomes a shadow of its former self.
This process often starts when key talent leaves the organization - creating a hole in the mesh of communications and relationships that previously existed between the security team and their business stakeholders. In order to stave off decline into a geriatric state, it is critically important for leadership to step in and reconnect that social network with as-close-to the same level of completeness as possible.
Without these connections, previously informed stakeholders who might have weighed in on security decisions are cut off from the source. This creates friction between the business and security team(s), which forms the beginning of a domino effect for how security and business teams coordinate on upcoming projects and initiatives.
Due to a reduced capacity for coordination, the security team starts losing visibility into - and influence with - the needs of their business stakeholders. As geriatric security teams begin to reduce their focus toward a handful of key projects and initiatives within their purview, business stakeholders begin to make their own (often ill-informed) decisions around processes and technology purchases in order to fill the void left behind from an absence of coordination with a mature security team.
Likewise, collaboration often turns inward toward working closely with other security teams as the formerly mature security team settles into a geriatric state. This usually happens through some form of re-organization or leadership change, and comes at the expense of the business when collaboration between business stakeholders and security teams occurs less frequently.
Often this sort of consolidation is a mistake, as there are security teams interested in reducing internal business risks - and then there are security teams focused on reducing customer facing risks. When re-organizations happen which cobble together a collection of geriatric teams, the outcome is usually a focus on internal business risks over customer facing risks. From that point onward, the security team seeks to change business processes to fit the needs of the security organization - which usually leads to the security team becoming an otherwise ignored “check-the-box” exercise for business stakeholders.
Ultimately, success for geriatric teams is keeping as many of the proverbial “spinning plates” that are their projects and initiatives “spinning” for as long as they can without heroic efforts. At the end of the day, the amount of progress and security maturity willing lost to time - and how quickly teams can recapture that maturity - is entirely up to leadership and the business to decide.
As always, thanks again for stopping by to read my pseudo-random musings 😊 While I work on the next blog post on Mature Technologies, you can
git checkout other (usually off-topic) content I’m reading over at Instapaper - or meander through my DevSecOps Essentials series to discover the worthwhile investments your DevSecOps program might benefit from.
In the mean time, remember to
git commit && stay classy!
If you found this post useful or interesting, feel free to buy me a coffee (or tea) 😊 and thank you to those who already support this content!