Alert fatigue, burnout, turnover: lather, rinse, repeat

Security operations · 7 MIN READ · GREG NOTCH · JUL 19, 2023 · TAGS: MDR

Alert fatigue is a headache. The spiral it kicks off stresses the entire organization.

According to a 1Password study:

• 84% of security professionals report feeling burned out.
• Twice as many significantly burned-out security professionals say security rules and policies “aren’t worth the hassle,” compared to those who are only somewhat burned out (44% vs. 19%).
• Nearly a third of burned-out security professionals are currently looking for new jobs or on the verge of quitting—[five] times the share of those without significant burnout (32% vs. 6 %).

Hackers are responsible for as many as several billion attacks per day, setting off a barrage of alerts in the world’s security operations centers (SOCs) and initiating a chain of events that represents one of a chief information security officer’s (CISO) biggest headaches.

The cycle looks like this:

• The sheer volume of alert activity causes alert fatigue…
• which results in missed attacks and…
• fuels burnout…
• which results in decreased motivation, errors, and the degradation of service quality, erosion of employee health (physical and mental), and…
• turnover, as employees leave the organization (and perhaps even the industry).

Alert fatigue

Alert fatigue occurs when security operations center (SOC) analysts receive an overwhelming number of alerts (often from a profusion of systems and tools), many of which may be false positives or low priority, decreasing, causing desensitization or complacency.

According to the Ponemon Institute, the average security operations center employs dozens of tools to safeguard their data and systems, and these tools generate a mind-numbing number of alerts. While estimates of precisely how many vary, a 2020 analysis from Forrester places the volume north of 11,000 alerts per day. A Trend Micro report finds that more than half of SOC teamers are “overwhelmed” by the volume of alerts. And another survey says “more than half of respondents spend more than 20% of their time deciding which alerts should be dealt with first.”

Most are time-wasters. According to IDC, investigating an alert takes about a half-hour. False positives take even longer (and in one survey ~81% of respondents said more than one-fifth of their cloud alerts were false positives).

As one large data intelligence company CISO put it (in a conspicuous fit of understatement), “It’s not feasible for a human to look at 10,000 log lines a day. And you need to overlay threat intelligence and known attack patterns if you want humans to make sense of alerts.”

Accidents will happen

The gap between the sheer number of alerts and the SOC’s ability to vet them all is simply too much. An IDC study says companies with 500-1,499 employees ignore/don’t investigate 27% of all alerts. The figure is 30% for companies with 1,500-4,999 employees and 23% for those with 5,000 or more employees.

In this environment, costly mistakes are statistically … probable. More than half of respondents to one survey say critical alerts are being missed. 41% of those said they’re being missed on a weekly basis. 22% said it happens daily.

What does “critical” look like?

A similar dynamic contributed to the recent 3CX attack.

Many users had seen their endpoint protection software incorrectly flag known, good software as malicious in the past. Since 3CX’s software was expected in their environment, they assumed the endpoint protection software was incorrect, rather than suspecting the software had been the victim of a supply chain attack.

Burnout

Burnout, a state of emotional, mental, and physical exhaustion, happens when security analysts and leaders endure long hours, heavy workloads, and the responsibility of safeguarding sensitive, high-value information. Over time, the overwhelming demands of the job can cause frustration and cynicism. It can hurt job performance and personal well-being, and can also increase the likelihood of potentially catastrophic mistakes.

Per the National Institutes of Health, burnout can exact a toll on workers’ mental well-being. Burned out workers are

more likely to experience physical and psychological health problems, including sleep disturbances, headaches, and infections, as well as higher levels of depression, suicidal ideation, anxiety, and life dissatisfaction. As a result of these health-related implications, burnout also tends to be associated with more prolonged sickness-related absences and higher turnover rates.

In the workplace, burnout is associated with cynicism, detachment, and apathy. It can erode job satisfaction and fuel professional detachment from colleagues.

(The unending proliferation of cyberattacks also means many (most?) employees are expected to put in extra hours. One report says 47% of cybersecurity pros work between 41 and 90 hours per week. Those in security leadership roles log an average of 10 extra hours per week. This, of course, has a predictable effect on burnout.)

Quality degradation and errors

One prominent CISO notes that, “Human error is one of the biggest causes of data breaches in organizations, and the risk of causing a data breach or falling for a phishing attack is only heightened when employees are stressed and burned out.” A 2020 study found that 88% of data breach incidents were caused by human error. Nearly half (47%) cited distraction as the top reason for falling for a phishing scam, while 44% blamed tiredness or stress.

The risk is exacerbated when burnout begins eating away at worker attitudes, leading them to disengage from the job. A deeper look at the findings of the report quoted in the introduction illustrates the problem.

• Twice as many significantly burned-out security professionals say security rules and policies “aren’t worth the hassle”
• 48% of burned-out security professionals…say it’s unrealistic for companies to be aware of and manage all apps and devices used by employees at work
• Security professionals were twice as likely as other workers to say that due to burnout, they are “completely checked out” and “doing the bare minimum at work” [emphasis added]

Turnover

And now, the worst-kept secret in the industry: burned-out employees leave (and are really hard to replace).

According to a 2022 Help Net Security report, top decision-makers in the U.S. revealed “an average security staff turnover rate of 20%,” and a global study by Mimecast “found that one-third are considering leaving their role in the next two years due to stress and burnout.” In our recent U.K. cybersecurity landscape study, more than half of IT decision-makers said it was “likely” or “very likely” they or members of their IT/cybersecurity team might leave the cybersecurity industry due to burnout in the next 12 months.

Worse, these departures take place in the midst of a severe security talent shortage. According to the 2022 (ISC)² Cybersecurity Workforce Study, there’s currently a global shortage of 3.4 million security professionals, representing an increase of more than 26% over 2021. (For comparison, 3.4 million is roughly the population of Utah.)

That’s a huge number, especially when a study from Trellix found 85% of organizations saying security personnel shortages were directly affecting their ability to secure and maintain their infrastructure.

Shortages, of course, drive the cost of recruiting and retaining talent through the roof and replacing workers who leave (if you can find a replacement, that is) can be brutally expensive. The direct costs of backfilling an employee can range from 150-200% their annual salary. Notch notes that at some point, short-handed orgs have to make decisions weighing the cost of staffing against the risk mitigated.

Then there’s the impact on employee morale, customer relationships, and brand equity in the marketplace…

Lather, rinse, repeat

• Huge talent shortage means high cost of replacement—if you can even find talent
• ROI: does the risk mitigated justify the increased cost?
• Fewer people means increased pressure on remaining staff

Some organizations try to use additional tools and automation to compensate for staffing deficiencies. But many places already have too many tools, and all of them require some amount of oversight to manage. The added complexity means each one added to an environment can accelerate the burnout cycle.

An IBM study confirms this, finding that those using more than 50 tools ranked themselves 8% lower in their ability to detect an attack and around 7% lower when it comes to responding to an attack. And the Forbes Technology Council reports that 43% of a major survey’s respondents say their top threat detection and remediation challenge is “an overabundance of tools.”

Breaking the burnout cycle

Managed detection and response (MDR) combines threat detection technologies, skilled analysts, and rapid incident response capabilities to safeguard organizations against cyberthreats. It provides real-time, 24/7 monitoring of network, systems, and endpoints, usinging advanced automation and expert analysis to identify and mitigate potential security breaches, malicious activities, and indicators of compromise.

Organizations looking to escape the alert fatigue burnout turnover cycle have a simple task: do everything you’re doing right now, plus the things you aren’t doing but need to be, with fewer people.

If that doesn’t sound so simple, consider the story of Pat Lefler, the Senior VP of Risk and Information Security at FIA Tech. His team was so busy it couldn’t research all of the alerts it was getting, and he wanted to focus on initiatives to drive increased revenue. Implementing MDR afforded rapid monitoring, investigation, and response, saving at least 40 work hours per week previously spent sifting through alerts—without adding new tools or headcount. MDR noticeably decreased the wear and tear of alert fatigue on his staff.

Another CISO had to monitor thousands of threat alerts daily, surface the critical ones, and decide on remediation paths. Doing so, however, would leave his team no time for product security and security infrastructure buildout initiatives.

The cybersecurity talent gap was a worry. “When you find talented people,” he explains, “you need to keep them happy. No one wants to just look at alerts all day. It isn’t interesting, challenging or meaningful work, and it leads to burnout.”

MDR breaks the cycle

MDR’s benefits are substantial. Advanced tech, such as machine learning and behavior analytics, identify potential security threats that may go unnoticed by traditional security measures. Its proactive approach to incident response combines automated detection tools with human expertise to rapidly identify, analyze, and respond to security incidents. MDR is scalable and flexible. MDR provides round-the-clock monitoring and coverage, which includes skilled security analysts who specialize in threat detection and response. These experts can analyze security incidents and provide insight into the nature and severity of threats as well as offering guidance on appropriate mitigation strategies.

And the overall ROI proposition can be tremendous—up to and exceeding 600% in our case.

In short, many organizations, perhaps including yours, are in a tight spot that’s getting worse. There are really good solutions, though. Drop us a line if you’d like to talk.