In the last post I covered the crucial questions from Boards and executives. Here I will cover the questions I’m asked by CIOs, CTOs and other senior IT leaders in one on one as well as group settings. These are quite consistent by geography and by sector, although there are, of course, some edge cases in certain places.
As I mentioned last time, these are not necessarily the questions I think CIOs/CTOs should be asking, what CISOs and others would like them to ask or what some might expect them to be asking, but they are what is being asked. But, I do think they’re solid questions given today’s pressures and realities, and they’re obviously colored by the fact they’re asking me given my role. I don’t doubt they’d have different questions for a line of business executive or another CIO.
1. Moving to Cloud Quicker
How do I get security, risk, compliance and audit more comfortable with an acceleration to the cloud - so I can deliver on objectives and in fact mitigate technology, security and resiliency risk more quickly?
This is a question that comes up all the time, and not just because of where I work and what I do. I hear it in other people’s discussions and at other events. Which is why, if you’ll forgive a pitch for work on my personal blog, we put so much content together like this CISO guide and this Risk and Compliance guide, but in general the answers are:
Put your oxygen mask on first. As a CIO/CTO you need to demonstrate your commitment to running a strongly governed cloud engineering and operations process. Creating a business unit IT free-for-all rush to the cloud and leaving it up to security, compliance and audit to mediate and control that is going to create concern. This will likely result in an appeal to slow down progress so those groups can instigate some control. However, if you put in place sets of standard configurations and guard rails, endorsed by security, compliance and audit that are the basis in which business unit IT teams can operate then you’re likely to be granted way more “tickets” to proceed.
I generally see (big abstraction here) three types of organizations:
Initial. Every business unit does its own thing in the cloud and security works with each to have them implement their own controls, control adherence efforts and other monitoring.
Controlled. Every business unit does their own thing but using an established configuration baseline driven by the security team, which also centrally monitors continued adherence.
Enterprise Controlled. There’s an IT / engineering function that owns cloud engineering and orchestration and provides tools for business unit development and other teams to use that orchestration layer. Security works with that engineering function to set standards and provide the tools for that team to monitor adherence with the security team providing a further independent check, sometimes with additional independent tooling to assure ongoing conformance. This begins a cycle of security-as-code to controls-as-code to continuous verification.
You should watch out for the pattern of being too perfectionist in your transition to the cloud. Sometimes organizations want to make sure everything that moves to the cloud is fully rearchitected or in other ways adjusted to take advantage of all the new control possibilities of the cloud. This is a fantastic goal, but the challenge here can be that the security team delays progress if not all of the idealized state can be met in one go. This can be a problem because there’s the missed opportunity of making some significant risk reduction even if you can’t reach your idealized state. To illustrate this, let’s say there are 20 ideal security controls you want to achieve. In your on-premise environment you’re meeting 8 of them, but in moving to the cloud you can meet 18 now, and in 12 months you can meet all 20. This could be because the cloud itself doesn’t yet have those 2 controls on the products you’re choosing to use or perhaps your application set can’t utilize those 2 controls. Some organizations hit pause until they can do the full 20. It’s easy to see why this happens, because their goal is of course to move to the cloud to hit a whole new control state. But in delaying they’re missing the benefit of the next 10 controls they could have now. Sadly, sometimes this gets relayed as a message from security up the management chain that “we can’t move to the cloud because of security reasons”, which perplexes management for obvious reasons. Of course, all this is context dependent but often the better risk move is to move now and get the benefits and roll in the incremental controls after migration.
2. Security vs. All Technology Risks
Is cybersecurity the most significant risk and how should I prioritize security vs. all my other technology risks?
A lot of CIO/CTOs, especially those who have an overall technology risk function separate from the CISO function realize quite quickly that they have a lot of technology risk and so it’s natural to ask this question. Here are some ways of thinking about that:
First, if we’re honest, cybersecurity isn’t necessarily the only or even the major technology risk in most organizations. It can be one of the potential existential risks, but there are other risks like software errors, resilience, reliability, capacity, non-security compliance, and more besides. Some of these have a higher loss history than even the cyber events and so it’s important to manage these as a portfolio of risks.
I always remember in 2015, there was a day when the New York Stock Exchange had a major outage and there was a lot of speculation that it was a cyber-attack. It was actually a bad software update. On the same day the Wall Street Journal website went down, not from a nefarious DDoS attack, but apparently because of the traffic load of people wondering if the New York Stock Exchange had been attacked. As this was unfolding, United Airlines had an East Coast ground stop because of a network outage which compounded all the suspicions of cyber attacks that day. I remember this day so well because it was when a bunch of us bank CISOs were about to briefed on something totally unrelated in The White House Situation Room, I remember thinking if the Press Corp spotted all the bank CISOs there with all the other stuff going on there really would have been a total conspiracy meltdown. Anyway, what also happened that day, mostly unreported, were many other major IT events from an oil refinery shut down to a medical system outage due to hardware issues and software errors. So it really was a day of big impact and big losses due to non-cyber technology risks. But to balance it out this was also the day before the full disclosure of the massive OPM breach.
So, it’s important to manage your set of technology risks as a portfolio of risks with relative priorities, because your biggest risk might not be cyber. Also do this because you can seek common controls and control alignment that will mitigate more than one risk. One example of this is implementing improvements in your SDLC, particularly around consolidating software repositories, increasing the amount of software that is continuously integrated and deployed, and within that pipeline improve regression testing, quality assurance and security validation. In doing that “control” you get a reduction in risk across reliability, change and security as well as likely delivering increased business agility by supporting more rapid feature releases. You can only spot these opportunities if you look at all your technology risks together.
How you prioritize all this really depends on the nature of your business or mission. You need a risk scoring method to weigh those risks by business unit but quickly move on to thinking about common control frameworks so that you can relentlessly raise the baseline by reducing the cost of control. You also need to integrate risk governance, for example, having a “Cybersecurity Committee” separate from a “Technology Risk Committee” is a cause of many problems and many missed opportunities.
3. CISO Function Alignment
How do I ensure the CISO function is integrated into IT / business processes / activities?
Contrary to some commentary that would assert that the CIO/CTO and the CISOs mission are in conflict I have yet to meet a CIO/CTO who doesn’t feel deeply accountable for cybersecurity, and more broadly technology and information risk management. They are often also held formally accountable by the Board and executive leadership. As a result, whether the CISO reports to them or not they spend a lot of time thinking how to make the CISO and their team effective. My advice is:
Whether the CISO is a formal report to the CIO/CTO the CISO should be part of the CIO/CTO’s executive leadership team to be part of the overall planning processes.
There should be a series of embedded product security and security engineering teams that are aligned into business unit and infrastructure development / engineering. If the goal is, as it should be, to have secure products not just security products then security should be in all team’s development processes.
Develop a funding model for sustaining the operational activities of the security team that doesn’t rely solely on making the case in an annual bottom’s up budgeting exercise - so that the CISO can forward plan effectively. There are a variety of funding models ranging from a straightforward percentage of IT budgets through to more incentive driven models - where business units carrying the most risk or generating the most need of CISO operational support provide the most funding. However, be careful with this though in the event of actual improvements driving this funding source below a minimum operational threshold.
Develop additional funding models for specific new mitigation projects and then allocate this funding not just to the CISO team but across IT. A key part of risk reduction is IT modernization and that relies on IT not just security having the budget to do this.
Help the CISO find a senior business line executive mentor to develop their business acumen if they’re new to the organization or come from a predominantly technical background. Similarly, if your CISO is more business oriented than technical then assign a senior technical executive mentor to develop their technology awareness.
Ask the CISO and team to regularly present their staffing and organization strategy to include succession planning, staff rotations and the formal plan for aligning their work into IT team activities.
4. Forward Planning for Security
I’d like to plan ahead but things coming from the security team seem so unpredictable?
CIO/CTOs have an immensely strategic role for organizations. The role requires significant balance to ensure a digital strategy to meet competitive or mission needs. This is in parallel with managing upgrades, efficiency improvements and developing a sustainable IT organization, embedded and distributed across businesses. This requires significant forward planning and if the security team is forever dropping seemingly random requests of varying urgency outside of the planning process then this causes frustration and inefficiency. The right way to think about this is:
First, remember the CISO and security team themselves are subject to unpredictability and have to be responsive on behalf of the organization. This could be in response to security vulnerabilities, new threats, geopolitical events, existing but previously undiscovered risks, unplanned business activities and even in some cases imminent but unexpected regulatory or legislative shifts. I remember once having to work to backhaul some global Internet connectivity into a specific country because they introduced some Internet egress rules that needed to be implemented in weeks. This met with some incredulity in IT but it was beyond everyone’s control. So, as a CIO/CTO show some empathy and don’t shoot the messenger.
But, the CISO needs to understand, respect and align with the IT planning process and assign people to work to integrate security requirements into other plans, align resources to sustainable funding models, rationalize the business case (not necessarily ROI) and constantly work to demonstrate the adjacent benefits of security controls. As discussed above, the CISO should define and integrate a technology controls roadmap to mitigate the organization’s portfolio of technology risks.
It is also useful to ask, just like any team, to review each quarter or other frequency, an overall forward 6 - 24 month roadmap of work, events, improvement plans that intersects with the wider business and IT roadmap. As a CIO/CTO recognize that these are approximations - just like your own roadmaps and help the CISO improve these through a regular dialog at IT executive leadership meetings or the relevant risk committee.
Finally, think hard - with the CISO - about what you might call architectural intercepts. In other words, there are projects happening, you have plans of what you need to do so work to make sure the CISO and team are involved in those to integrate security requirements as early as possible.
5. How Much Security is Enough?
No amount of effort ever seems enough for the security team, what should I do?
This is a common concern that usually has a root cause from the side effects of some of the issues we’ve already covered. Specifically, that the planning processes of the CIO/CTO and CISO are out of alignment such that the CISO has a forward plan but the CIO/CTO isn’t aware of it. So, things emanating from the perhaps a well-considered CISO plan appear haphazard to the CIO/CTO and their team. So the answer is:
Keep improving the alignment of the planning cycles. This is a never ending cycle of improvement and even when you think you’ve got it right you will need to adjust because the IT and business alignment will change causing other ripple effects.
The other important approach a CIO/CTO can take is to work with the security team, and likely as well as the compliance and audit teams, to shift the mind set that periodic risk assessments should always be about reducing risk. Many security (and compliance and audit) teams think doing some review or risk assessment that delivers no findings is a failure and therefore always include even some minor issues. So, after a number of years some of the findings presented to IT or business teams appear even more insignificant and a waste of effort. This contributes to the feeling that no matter what IT does, it is never enough. Having the discipline to curtail such findings, or to shift them into advisory / observations is useful - but where the CIO/CTO and CISO can partner extensively is to shift to a situation where reviews and risk assessments seek optimizations and control improvements - not just further risk reduction. It might be that the risk levels are about right but the recommendations can still drive replacements or upgrades of controls such that those new controls deliver the same risk mitigating effect but provide adjacent benefits like reduced cost, improved customer experience, improved developer productivity, or more efficiently monitored controls.
But, sadly, for everyone not just CIO/CTOs in a world of evolving threats, new technology and other rapid change it might mean that, yes, a lot of security is never enough and needs to be constantly improved to deal with those changes in the wider world. Again, this is an opportunity for a CIO/CTO to work with their CISO to build more agility across teams and more transparency so things are less of a surprise.
6. Developer Agility and Security
How do I ensure developer agility and productivity in the face of security?
This is perhaps an obvious theme from CIO/CTOs that they want to improve their ability to deliver on business or mission objectives for their organization. These objectives have various underpinnings to ensure security, reliability, resilience and other risk mitigation. So anything that unnecessarily impinges on that is a problem. Think of it this way, if the CIO/CTO has just invested massively in refactoring their software delivery pipeline to move feature release cycles from quarterly to weekly or daily and the security team’s unadapted review process puts a weeks/months review block at the end of that pipeline then that is going to be a problem, to say the least. Then, when the CIO/CTO inevitably “complains about security” the security team say “my CIO/CTO doesn’t care about security because they’re pushing back”. Instead, the CISO should take this as a need to improve the security process to meet actual business or mission goals. But the CIO/CTO can and should take a leadership position:
Have the CISO and team develop a strategy for developer productivity and agility and review that with the business unit and central IT architecture and developer tooling teams. Sponsor some projects to deliver automation and tool improvements.
Additionally, and especially if the CISO team doesn’t have the skills/depth to do this then add qualified and developer-tool aware security engineers to the developer tools team to make this happen anyway.
Establish a process, with or without the CISO, to identify common vulnerability patterns, common blocks to software release approvals from identified issues and then develop common tools, frameworks, APIs such that developers have to spend less time on developing security critical components.
Yes, by all means train software developers to understand the need for the common positive design patterns (and anti-patterns) for security but don’t place all your security bets on all developers being excellent at developing secure software. Make that a central team problem where you can focus immense security talent on those tools and frameworks that solve common security issues so that most developers don’t have to.
7. Legacy Systems
How should I deal with legacy systems and architectures that are hard to secure?
This is often a fascinating question especially because we all use the term “legacy” differently. What might be called legacy systems in the negative use of that term is confusing because those legacy systems are often the backbone of a company's operations and they’ve been around for so long because they’re actually successful systems. Now the real question is whether those systems are effectively maintained and secured. So think of it this way:
Differentiate between legacy systems that are well maintained, kept up to date and have the required enterprise controls integrated, from what you might call stagnant systems that are not maintained and are a source of security, resilience and reliability risk. These are also a constant drag on the ability to change surrounding systems. So categorize systems regularly into tiers like invest (keep investing and developing), maintain (no more strategic investments but keep it up to date) and deprecate (with an actual plan to do the deprecation).
The challenge for legacy systems is they often won’t have been built or designed to be defensible in the way some more modern architectures are (typically cloud or cloud-like on premise). For critical systems that are irredeemably insecure or progressively costly to maintain then the best bet is a deprecation approach. However, for some it makes sense to wrap them inside some additional internal control perimeter, fronted with an API gateway or some other construct to extend the defensible life of a critical legacy system.
Finally, you need to figure out ways of allocating formal funding to maintain legacy systems so they don’t become stagnant. A good way to do this is to require explicit budgeting of preventative maintenance such as 5% of a business unit's IT budget to cover this. This is similar to the way other industries formally allocate funding for physical plant preventative maintenance. It’s ultimately more cost effective to this do this than deal with the build up of maintenance debt. You can even make this preventative maintenance budget risk-adjusted so that each year it scales up or down to cover the risks. In this way, businesses are incentivized to keep a stream of regular investment such that their preventative maintenance budget can be optimized.
Bottom line: in my experience CIO/CTOs care deeply about security and other technology risks for the simple reason that if there are issues then that impacts their success and mission. They are usually held as much accountable for security as their CISOs (whether the CISO reports to them or not). So CIO/CTOs are immensely vested in assuring security but they have to balance this with organization business or mission goals. They also have to build agility into their IT organization. So they need close partnership with CISOs and security teams to ensure they are similarly invested in those goals. They need to work together to develop the right trusted relationship to partner on delivering security in an integrated, fully embedded, engineering-oriented and agile way. They want a CISO to be a tech-savvy, and business oriented peer executive.