Security operations · 9 MIN READ · YANEK KORFF · MAR 22, 2019 · TAGS: CISO / How to / Managed security / Selecting tech / Tools
I used to love my iPhone. Now, at best, it works fine when new features aren’t getting in my way. I also remember when AOL was amazing, ICQ was the best chat client and Netscape was the go-to browser.
Maybe it’s inevitable that the things we love will eventually be superseded, though hopefully not too quickly.
Let’s take a look at “security operations.” Turning logs and other forms of security signal into useful actions is an activity that’s been around for decades. Whether companies have their own internal capability or have outsourced to a managed security provider, the breach headlines have continued unabated. Okay, that’s not entirely true — they’ve accelerated. And yet, even in this morass that is the security industry, every once in a while you’ll find someone truly delighted about the products or services they’re using.
But delighted customers are the exception when it comes to managed security service providers (MSSPs).
Some will tell you that MSSPs take your money and give you nothing in return or that they’re a black stain on our industry. In fact, according to Forrester’s 2017 Global Business Technographics® Security Survey, 34 percent of responding organizations were actively evaluating alternatives or actively planning replacement of their existing MSSP. In an industry where three-year contracts are common, a third of the market was in the process of switching at the time of the survey. Math doesn’t paint a pretty picture here. In this ten billion dollar industry that’s growing nearly 10 percent each year, thousands of companies are beyond disgruntled: they’re looking to get rid of their current provider.
If you’re somewhere in that one-third of the market that’s looking to switch to another MSSP, you’re probably thinking to yourself, “I thought my provider would be better … and they were for a little while. Then it all went down the toilet.”
So, before you sign that next contract how do you determine the likelihood that the quality of the service will last? How long will you be happy with the quality of your service provider? You might be able to get a sense of this through a proof-of-concept exercise but that won’t tell you much about how you’ll feel a year (or five) from now.
Delighters will become table-stakes over time — so, to truly satisfy you, any new service will have to do more than just not deteriorate. It has to improve. Constantly.
Creating a culture that searches for quality
Why is it so essential that quality is core to your provider’s DNA? Well, because it’s already part of yours. You’ve got a limited budget and a part of your job is to get the most bang for your buck over time. So you’ll constantly be changing your investments to ensure you’re getting the most for your money. A dollar you spend a year from now should be doing more than a dollar today. This translates directly to your service provider: an hour of work your service provider does today had better do more for you a year from now than it does right this minute. This means everyone (yes, everyone) at your service provider’s organization needs to be looking at ways to improve quality constantly.
So how can you tell if an organization’s got it? Here are some key characteristics that we’ve seen that create an environment where a persistent focus on quality can emerge:
- People feel a sense of trust and psychological safety,
- People have ownership of the problems they’re trying to solve,
- People have the energy to engage in quality-seeking behaviors, and
- People can honestly self-assess throughout the process.
You’re probably thinking “that sounds pretty soft and squishy.” So how do you assess whether a company you’re talking to has built this sort of culture? Well, without further ado, here are a dozen things you can do to sniff out whether “the search for quality” exists at an organization.
1. In search of trust – look for transparency
Transparency means more than just being forthcoming. It means making the effort to be easily understood.
There’s no shortage of places you can go to find examples of an org’s transparency. Start with the website and see if you can figure out what the company does and how they do it. As you ask questions to fill in the gaps, take note of whether you can understand the answers or if they’re wrapped in marketing buzzwords or technical mumbo-jumbo.
See how deeply transparency extends into the organization. Spend some time to understand the company’s high-level goals. As you run into various employees in your evaluation process, ask them what these goals are and what they think about them. Ask what’s going well and what’s challenging. If employees can’t (or won’t) be forthcoming when they’re literally trying to sell you something, what are the chances they’ll be honest when they screw up?
2. In search of trust – look for simple execution
Trust is a fickle thing. As we approach new relationships, we come with some amount of default trust in the new partner. I like to call this the “trust bank.” If you’ve had your trust violated a little too often, you won’t be very generous when it comes to initial your initial deposit in the trust bank. If you’re a bit more optimistic you might make a huge trust deposit up front, thinking the best of people. The unfair thing about trust banks is that deposits are always small, but withdrawals are easily five times as large.
During your conversations, the service provider will promise to do many things. They’ll send you a summary. They’ll put you in touch with another customer. They’ll get you on the phone for a chat with someone with greater technical depth in an area that’s important to you. They’ll promise you a quote.
Do they follow through on those things? And do they meet the expectations they set within the timeframes they promised? It is surprisingly difficult for people to consistently meet simple obligations like doing what they said they’d do. So when you find that in an organization, it really stands out.
3. In search of trust – look for failure
It’s easy to provide examples of past successes. It’s a lot harder to admit failure.
You’re about to sign up for a long-term service. You’ve got a right to know what sort of problems there will be. How will they be identified, communicated and handled? Ask for an example, and ask for artifacts (redacted and/or anonymized presumably). Get the full story and ask a lot of questions to fill in the blanks.
An organization that knows how to handle failures and turn them into success stories is well positioned to earn (and keep) your trust.
4. In search of ownership – identify roles and responsibilities
You’ll have the opportunity to meet several people at a potential provider during the courtship process. Pick two or three different roles and get a copy of their job description (this may or may not be what’s posted on the company’s website). Ask those employees what their responsibilities are and make sure things line up.
Do employees seem to understand where their responsibilities start and end? Can they point to other teams within the org and tell you how the teams work together? Sounds pretty basic, but having a strong sense of ownership often breaks down when this foundation is missing.
5. In search of ownership – ask about projects
When you’re meeting with mid-level and senior people at the organization who aren’t part of the management team, ask about what they’re working on. Usually, technical people are more than happy to share some of the projects they have in flight. Then, ask why they’re working on those projects.
In organizations where employees feel a strong sense of ownership, they look at their work not as tasks, but as solving business problems or customer problems. They articulate their work in the context of something greater.
6. In search of energy – ask about work and life
People think about “work/life balance” differently. As you interact with people at your service provider, ask them how they view the work/life balance at the company. Does it meet their needs? Do they get vacation time? Sick leave? How much? Do people actually take vacation? Do people feel like they can disconnect?
In environments where there are lots of “single points of failure,” people tend to work hard constantly, be stressed out and make more mistakes. While this might happen from time to time due to shifts in staffing, it shouldn’t be the norm. On the other hand, where people feel like they get the space they need to bring all their enthusiasm to bear, they’ll do better work and you’ll be happier for it.
7. In search of energy – ask about celebrations and praise
One of the factors that contributes the most to quality work is recognition that individuals and teams have done well. Contrast this with environments in which “the beatings will continue until morale improves.” Yeah, you’ve been there and seen that.
Ask about the last few company events, what they were and why they happened. What were they celebrating? What about the last spot award or “kudos” someone got? Can they remember when something like that happened?
8. In search of quality-seeking behaviors – ask about conflict
There’s plenty of info out on the interwebs about the negative effects of groupthink and the need for constructive debate.
Yet “conflict” seems to be a dirty word in most office environments. Instead of having a difficult conversation we hear “let’s take it offline” which is office lingo for “let’s stop talking about this because it’s making me uncomfortable.” Ask about disagreements, technical or otherwise, and how they’re resolved within the organization. Ask for an example. You’ll quickly get a sense as to how the environment supports constructive disagreement and the extent to which “office politics” play a role.
9. In search of quality-seeking behaviors – ask about metrics
You may only get operational insight into a subset of the metrics your service provider uses to measure the quality and efficacy of what they do every day. Have someone walk you through it.
How does the org measure the effectiveness of detection logic? How do they measure the availability of technology, whether it’s their own or yours? Can someone provide an example of a metric he or she thought was useful — but turns out it wasn’t? Is there a metric the org recently added because they’ve learned something new? Look for this engine of continuous improvement within the things they count and measure.
10. In search of quality-seeking behaviors – ask about hiring
When you were hired, someone entrusted you to make good hiring decisions. When you hired a manager, you entrusted her to do the same. Maybe you provided feedback, coaching or training to help her be more effective.
As you bring on a service provider, you have the same need. Their hiring practices will directly impact the quality of the service you experience over time. How do they think about hiring? Talk to the head of HR. Do they use a structured hiring process? How do they think about evaluating experience, skills and traits? What key traits do they look for in hires throughout the organization? Any organization with rich answers around these questions (especially when these answers are consistent throughout the organization) clearly has a high hiring bar.
11. In search of self-assessment – ask about evaluations
Do employees have the opportunity to think about how they’re doing and how they’re growing? And does anyone guide them through this process? The answer here can’t be as simple as “yeah, we do annual reviews … and they’re super stressful.” A huge component of perpetually increasing quality is making sure that every employee has real, ongoing opportunities for learning and growth.
As you meet security practitioners, engineers and managers, ask what they’ve learned since they started. What technical and non-technical growth have they experienced and how has this helped them grow their careers? Who supported this growth and how much did the company do to help? Are there programs in place to encourage this development? The more a company does to invest in its employees, the more likely it is that those employees will be investing in improving the service you receive.
12. In search of self-assessment – look out for hubris
We started this blog talking about some iconic names in technology like AOL and Apple. Do you remember when AOL “bought” Time Warner? Have you seen what happens to technology companies that become so full of themselves they feel like you’re obligated to buy their stuff? That only lasts so long.
This is a difficult area to assess but an important one. If everyone you talk to is convinced they’re the best at everything they do, that’s a warning sign. If everyone is taking themselves a little too seriously, there might not be enough room for fallibility. If it’s “our way or the highway” and compromise is out of the question, then that provider probably isn’t a good fit for you. These warning signs create blinders for an organization, making it difficult for them to see when they’ve done something wrong and learn from that mistake.
What if we’re wrong about all of this?
Perhaps we’re wrong about what it takes to maintain a culture that generates quality over time. But we do know this for certain:
When you’re evaluating an MSSP, you should walk away feeling pretty confident that over the course of your working relationship you’ll both get better together.
Or maybe you’re sitting there wondering what our answers would be for some of these questions. Well, you’re welcome to ask … or maybe in the not-too-distant future, we’ll publish some of them right here.
