By Derek Nugent Vice President Sales, Marketing & Customer Success at Difenda
Alerts, notifications, and non-stop calls from shady telemarketers pitching extended warranties – we all get more alerts each day than we can manage. For security professionals, the flood of alerts is even worse, much worse, extending to the essential tools they need to do their jobs.
The negative impacts of this deluge of alerts are felt anytime an overworked security professional suffering from “alert fatigue” neglects to block an attacker or detect malware because the signals were ignored or simply lost amidst the noise.
What Causes Alert Fatigue?
There are five main drivers of alert fatigue:
- Security Technology Creep
- Explosion of Automated Attacks
- Ineffective Configuration and Use of Tools
- Global Threat Landscape Events
- Limited Resources to Devote to the Problem
Each new layer of security that businesses add to address evolving security risks generates its own stream of notifications, alerts, and alarms. Some are actionable, many are not. Antivirus, IPS software, and firewalls, to name only a few layers, all generate alerts that tend to be poorly correlated.
Due to the unbalanced nature of security defense vs. cyber-attackers on offense, security solutions tend to be overly sensitive by design, which makes alert fatigue inevitable. After all, attackers need only be successful once in order to severely damage a business, while the organization’s security team must ward off attacks 24x7x365 to be successful.
As a result, security analysts, who are already coping with too many responsibilities and too few resources, must constantly cope with alert fatigue, which leads to critical alerts being missed at an alarmingly high rate. Alert overload not only increases your organization’s overall cybersecurity risks, but also results in low job satisfaction and high turnover for burned out employees.
COVID-19, Digital Transformation Drive Spike in Alerts
When experts study enterprise security, they find a few troubling trends that directly cause an increase of alert overload. First, as enterprises continue to migrate applications and data to the cloud as part of digital transformation initiatives, new security protections are added, often from new vendors.
The Cloud Security Alliance’s recent report, “State of Cloud Security: Concerns, Challenges, and Incidents,” found that as remote workforces grew, so too did the reliance on additional cloud-delivered security tools and virtual firewalls. The report found that “the use of cloud providers’ additional security controls jumped from 58% in 2019 to 71% in 2021.”
The report’s authors believe that due to the current health crisis and the dramatic increase in remote work, many organizations are unable to secure their networks – which are often hybrid ones with a mix of legacy on-premises, public cloud, and private cloud infrastructure – using only traditional tools. Therefore, organizations have had no choice but to add new security controls, each of which generates new alerts.
More than 5000 Daily Security Alerts, and that Was Before COVID
Now, consider that before the pandemic hit Cisco found in its “2017 Annual Cybersecurity Report” that 44% of security operations managers were already inundated with more than 5000 security alerts per day. In other words, alert fatigue was the new normal before remote workforces exploded and digital transformation and cloud migration initiatives accelerated.
The study also found that most companies used more than five security products in their environment, and those products often came from more than five security vendors. A full 65% of enterprises surveyed used six or more security products, while more than half (55%) of those surveyed reported they had to respond to alerts from at least six different vendors.
A 2019 study by CCS Insights of 400 senior IT leaders found that in enterprises with more than 1,000 employees the thicket of tools security teams must manage is even more complicated. CCS Insights found that the average large business had more than 70 different security products from 35 different suppliers, and while most enterprises intend to consolidate security, the consolidation trend has yet to get started in any significant way.
As we start to emerge from the pandemic, security infrastructure isn’t getting any simpler and alert volumes aren’t getting any easier to manage.
According to a recent survey of nearly 400 security operations professionals (commissioned by Siemplify), 42% report that their alert volumes are higher now than before the pandemic, while 51% say investigating suspicious activities has become a much bigger challenge in remote and hybrid environments.
Unfortunately, a number of factors threaten to add to the problem of alert fatigue in the near-term. These include but are not limited to the normalization of remote and mobile workforces, the rise of state-sponsored malware and hacks related to Russia’s invasion of Ukraine, and easy access to and automation of sophisticated hacking tools. At the same time, entire classes of threats are on the rise, including critical infrastructure attacks, ransomware, cloud storage leaks, and business email compromise attacks (BECs).
Thus, the volume of alerts will continue to rise and so too will the probability that your security team will eventually miss something critical, leading to a hack, a costly data breach, or some other negative outcome.
Unfortunately, another thing on the rise in parallel to alert overload is the cost of those negative outcomes. For instance, IBM and the Ponemon Institute’s annual “Cost of a Data Breach Report” found that the cost of an average data breach rose from $3.86 million in 2020 to $4.24 million in 2021, the highest average total cost in the 17-year history of the report.
How Microsoft Tackles Alert Fatigue
One shortcut to reducing alert fatigue is through vendor consolidation. For instance, for those organizations already dependent on Microsoft productivity tools and Azure Cloud, it makes sense to consolidate on that platform.
Soon after investing heavily in its Azure cloud platform, Microsoft also saw the need to tightly integrate security into its cloud stack, rather than layering it on afterwards. Thus, in recent years, Microsoft has invested $1 billion in security development, and their investment has already earned recognition from top-tier industry analysts. For instance, research firm Gartner lists Microsoft as a “leader” in a number of its Magic Quadrant reports, including end point protection, access management, CASB, and more.
Microsoft has also mapped out a strategy to avoid alert fatigue, a strategy that can help your organization regain control over alert flows.
Microsoft recommends adopting technologies such as Artificial Intelligence (AI) and Machine Learning (ML) to help find signal in the alert noise. Organizations should automate as many error-prone, repetitive tasks as possible in their SOCs, maintain up-to-date watch lists to prioritize activities from known bad actors, and adopt cloud-native solutions for better integration.
For organizations already struggling with limited IT resources, however, there are a few other steps you can follow to mitigate alert fatigue. The seven steps outlined below will help your organization alleviate alert fatigue in a way that should align with initiatives already underway, such as digital transformation and cloud migration.
7 Ways to Mitigate Alert Fatigue
- Consolidate security tools and vendors
Managing multiple security tools from multiple vendors becomes much easier if you take a platform approach to security and then build on that platform with best-in-class tools from the same vendor and/or its vetted partners.
At my company Difenda, we decided to build our SecOps-as-a-Service around Microsoft security tools not only because so many of them are best in class, but also because we believe that a consolidated security approach is the only way to keep ahead of the problems created by an increasingly complex threat environment.
Consolidated security stacks from single vendors and their certified partners will provide you with a unified dashboard that makes it easier to correlate various alerts, while also making it less likely that interoperability will undermine your defenses.
- Integrate that which cannot be consolidated
Whatever vendor you decide to use as the foundation of your security stack – Microsoft or otherwise – should be one with robust protections against a range of threats that also integrates easily with other tools, offering your organization an easy way to pull other alerts from third-party tools into a unified dashboard. Ideally, AI or ML capabilities will then automatically correlate those alerts with those from the rest of your security stack.
Look for certified partners who have been tested for interoperability, and in the rare cases you need something from outside of that ecosystem, be sure that the security tool offers open APIs. Before adopting any new tools, it’s also a good idea to research what existing users have to say about “vendor lock” and “lack of integration” before you commit to any new security vendors.
- Embrace continuous security improvements
The core tenets of the agile software development movement apply equally well to security, especially when it comes to reducing alert fatigue: prioritize individuals over tools, iterate quickly, receive and act on real-world feedback quickly, and more.
One core tenet of agile is especially important for security: continuous improvement.
The security threat landscape and tools monitoring it will never stop evolving, so organizations will need to adopt processes that enable them to adapt quickly to stay ahead of the threat curve.
- Automate, automate, automate
Automation is another key principle for achieving agile security operations, and it’s one that Microsoft stresses in its alert fatigue mitigation plan. For most large organizations, automation is necessary to even begin to alleviate alert fatigue. In a tight labor market, there simply are not enough skilled security experts available to tackle a problem of this scale unless manual, repetitive processes are automated. For alert fatigue, automating things like basic alert correlation, checking alerts against watch lists, and automatically ingesting patches and updates are all activities that should be automated to free up security professionals to focus on other activities, such as threat hunting and remediation.
- Include compliance as part of your automation efforts
In heavily regulated industries, many security alerts may directly tie back to your regulatory obligations, but even if your business doesn’t need to comply with laws like PCI-DSS or HIPAA, new consumer privacy laws, such as the GDPR in Europe and the CCPA in California, add obligations, and thus risks, for a large swath of the economy.
As you seek to automate security tasks, be sure to investigate ways to tie compliance into the process, which will streamline the overall process and reduce risks. For instance, Microsoft’s Purview Compliance Manager helps organizations integrate compliance with security operations, ensuring that they keep up with changing regulatory requirements and shifting risks.
- Intelligently prioritize incident response
Not all alerts are created equal, and even actionable ones don’t all carry the same level of risk. Thus, it’s important to prioritize the systems and applications that pose the biggest risks if breached or otherwise damaged.
Prioritizing known attack vectors, actively watching for known high-risk behaviors like privileged access, and maintaining an active watch list of known high-risk attackers will significantly cut down response times by focusing your team on the most pressing, high-risk threats.
As you investigate how to reduce alert fatigue, be sure your security provider offers a Configuration Management Database (CMDB) to provide real-time visibility into all of your networked assets. Ideally, your CMDB should automatically track the changing state of those assets (patches, updates, etc.) and correlate them with vulnerability scans and threat hunts.
- Outsource alert management to a security provider that offers Managed Detection and Response (MDR) services
A common cause of alert fatigue can be traced back to limited resources. If your organization does not have a large enough staff to manage SOC activities, you may have a hard time recruiting and retaining staff in this tight labor market.
In late 2020, a Microsoft survey revealed that 82% of respondents planned to add security staff in the coming year, while 81% also said that they needed to lower security costs. That’s a tough combination to manage.
How does an organization add staff, while also lowering security costs?
The only way to do that today without increasing your organization’s attack surface is to outsource costly security management burdens to service providers that are positioned to take advantage of economies of scale.
Managed Detection and Response (MDR) service providers focus on one thing and one thing only – security. They will have already optimized and automated much of the alert management process, and MDR service providers will also have more resources to hunt threats, integrate alerts from third parties, and detect zero-day threats before they cause problems.
However, when outsourcing MDR, it’s probably wise to find a security service provider that will also provide complementary security services, such as managed SIEM and managed endpoint protection. Prioritizing consolidation, certified partner solutions, and tested integrations will help you not only mitigate alert fatigue, but also will help you embrace agile security as a core part of your organization’s ongoing digital transformation efforts.
Learn more about how to maximize your existing investment in Microsoft Security or qualify for your complimentary roadmap today!
About the Author
Derek Nugent is Vice President of Sales and Marketing for Difenda, a SecOps-as-a-Service company. Before joining Difenda, Derek previously served in leadership positions at Herjavec Group, Paladion, and CDW.
FAIR USE NOTICE: Under the “fair use” act, another author may make limited use of the original author’s work without asking permission. Pursuant to 17 U.S. Code § 107, certain uses of copyrighted material “for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright.” As a matter of policy, fair use is based on the belief that the public is entitled to freely use portions of copyrighted materials for purposes of commentary and criticism. The fair use privilege is perhaps the most significant limitation on a copyright owner’s exclusive rights. Cyber Defense Media Group is a news reporting company, reporting cyber news, events, information and much more at no charge at our website Cyber Defense Magazine. All images and reporting are done exclusively under the Fair Use of the US copyright act.