What is Alert Triage?

As cyber threats constantly barrage companies, security analysts must keep up with the rising volume of alerts they must tackle daily.

However, the increasing volume of threats is nearly impossible for human resources alone to keep up. As malicious actors rely more on “off-the-shelf” ransomware, reusing code to execute attacks and repeating successful attack techniques against new victims, analysts must look for new solutions to keep up with evolving threats.

With alerts going off left and right from multiple security platforms, it is challenging for analysts to identify the most pressing threats. In a recent survey, 81% of IT professionals said that 20% of their alerts are false positives. 

Many analysts dedicate hours to alerts that turn out to be false positives. They quickly get burned out from the chaos of the constant alerts, which then puts the organization at risk 

This is leading organizations to rethink their processes for alert triage. How can they improve their alert triage processes, going beyond the traditional approach for a security operations center, or an outsourced managed detection and response service?  analysts determine which threats to escalate and which to ignore? This strategy of providing deeper context helps organizations identify breaches quickly to prevent damage and limit the scope of the attack.

This process is called alert triage — a solution that lightens the load on analysts so they can do their jobs more effectively without burning out.

TOC

What is Alert Triage?

Alert triage is part of incident response – it is the process of reviewing, confirming, and prioritizing alerts generated by a security monitoring system. 

In a traditional SOC team, after acknowledging an alert, analysts triage it — which means they gather contextual information on the alert, determine whether it is legitimate, and decide whether to escalate the alert or dismiss it.

This can be a manual process, but increasingly organizations are looking for ways to automate alert triage. Alert triage is important to ensure that the most critical issues are addressed first,to avoid being overwhelmed by a large volume of less-important alerts, and to ensure that any suspicious alert is properly investigated.

Why Decision-Making is the Most Important Aspect of Alert Triage 

Tier 1 security analysts are the first decision-makers in the alert triage process. They handle the largest volume of alerts, and their job is to decide which alerts to escalate to the Tier 2 and Tier 3 analysts.

One incorrect decision could result in a data breach that costs an organization millions of dollars. A missed or ignored alert can lead to catastrophic consequences for an organization. This means Tier 1 analysts frequently escalate many alerts for further investigation, lacking the context and skills to classify them as a threat or dismiss them as a false positive.

That’s why it’s crucial to equip security teams with the tools and resources they need to effectively classify and triage alerts on a large scale.

Context is the Missing Link

Analysts struggle to gather complete contextual information, so they cannot fully triage alerts. Additionally, analysts often face a time crunch that is impossible to beat. Collecting the information they need about each threat is a time-consuming process, especially without the assistance of automation.

Key Terminology

Alert

An alert is a notification that something suspicious has happened that may require attention.

Alerts can be generated by various actors, including monitoring systems, people, and even manual processes. Many of the security alerts these days come from tools for Endpoint Detection & Response (EDR); security information and event management (SIEM); Security Orchestration, Automation and Response (SOAR); or an email security solution.

Alert Fatigue

Alert fatigue is a condition that can occur when an individual is overwhelmed by a large volume of alerts. This can lead to missed or delayed responses to critical alerts, which could indicate the early stages of an active attack.

Alert Observables

These are the aspects of the alert that provide context. They can be used to create queries to other systems to help analysts understand more about the alert.

For example, analysts can observe the time the alert-prompting activity occurred. They can review other alerts and activities around the same time to gather contextual information on the alert.

Alert Triage

This is the process an information or security analyst follows to review an alert and investigate it to see if they should escalate or ignore the alert. Alerts are usually triaged and addressed based on their severity and potential impact to the organization

False Positive

In security terminology, a false positive is an alert that incorrectly identifies a threat. It is a false alarm that distracts analysts from the real attacks that threaten their organization.

Incident

An incident is a situation that requires immediate attention. Incidents are different from alerts because they typically require some sort of intervention to resolve and remediate the threat. Incidents are usually addressed based on their urgency, based on the priority assigned during alert triage. (An incident may also resolve if further investigation uncovers no threat and determines it was a false positive.)

Indicator of Compromise (IoC)

An indicator of compromise (IoC) is an artifact that signifies a data breach. The most common indicators of compromise include:

• Unusual activity on a network
• Strange behavior within privileged user accounts
• Login attempts from suspicious locations
• An abnormally large HTML response size
• Abnormal DNS requests

Mean Time to Respond (MTTR)

The mean time to respond (MTTR) is the average time it takes an organization to effectively respond to an attack, measured from the moment the security team was notified about the attack.

Security Operation Center (SOC)

The Security Operation Center (SOC) is a unified department in an organization that monitors security threats. While it may go other names at different companies, or outsource key tasks to a managed detection and response provider, this is the team or department that is responsible for detecting and responding to threats. A large SOC may be broken up into “tiers” or levels, like having the Level 1 analysts handling initial triage, Level 2 analysts handling response, and Level 3 handling advanced tasks ( like malware analysis, reverse engineering, or threat hunting).

Threat Hunting

Threat hunting is a proactive approach to identifying undetected threats. This process empowers security teams to remediate even the most advanced threats that go unnoticed for weeks at a time, without ever triggering an alert.

Alert Triage: An Important, Early Step for Incident Responders

Alert triage is important for incident responders because it allows them to quickly identify and prioritize the most critical alerts.

This can help save time and resources by ensuring that only true threats are escalated for investigation and remediation.

Additionally, incorporating automation into the alert triage process can help avoid information overload by reducing the volume of false positive alerts that an analyst or incident responder must review.

By improving their alert triage process, incident responders can more efficiently and effectively handle incidents. Instead of risking critical threats falling through the cracks, analysts can escalate the pressing threats to handle them more quickly.

Why Manual Incident Triage Cannot Keep Up With Evolving Threats

According to a study by the Ponemon Institute, 65% of security analysts have thought about leaving their jobs due to work-related stress.

Manual incident triage is not enough anymore. CEOs, security executives, and decision-makers within organizations must invest in automation now to support their teams in their day-to-day responsibilities and secure their companies.

Best Practices for Alert Triage

Here are the most effective alert triage strategies for security analysts to implement:

Using Rules-Based Alerting to Automate the Process of Reviewing and Prioritizing Alerts

Rules-based alerting is a method of automatically reviewing and prioritizing alerts based on pre-defined criteria. This can help save time and ensure that only the most critical alerts are reviewed.

Correlating Alerts to Identify Related Issues and Prioritize them Accordingly

Correlation is the process of identifying and grouping related alerts. This can help reduce the number of alerts that need to be reviewed and make it easier to prioritize them.

Using a Dedicated Team or Individual to Review and Triage Alerts

Alerts should be reviewed by a dedicated team or individual responsible for triage. This can help ensure that alerts are addressed promptly and that critical issues are not missed.

Keeping an Up-to-Date Incident Response Plan

An incident response plan should be kept up-to-date and include procedures for alert triage. This can help ensure that incident responders are prepared to handle incidents quickly and effectively.

How to Categorize Alerts

Here are a few ways to organize alerts as they are generated:

Criticality Levels That Can be Assigned to Each Alert

A criticality level is a classification assigned to an alert that indicates its importance. This can help incident responders quickly identify and prioritize the most critical alerts.

Triage Tags that can be Used to Categorize Alerts

Triage tags are labels that can be used to categorize alerts. This can help incident responders quickly identify and review related alerts.

A Triage Queue that Organizes and Stores Alerts

A triage queue is a storage location for alerts that have been reviewed and categorized. This can help incident responders keep track of which alerts have been addressed and which still need to be reviewed.

An Alert Priorities Matrix that Outlines the Procedure for Triage

An alert priorities matrix is a table that lists the different types of alerts and their corresponding criticality levels. This can help incident responders quickly identify and prioritize the most critical alerts.

How to Prioritize Alerts

The most critical alerts should be given the highest priority and addressed first.

Four factors to consider when prioritizing alerts include:

The Severity of the Issue

How much damage will the issue cause?

The Urgency of the Issue

How quickly does the issue need to be addressed?

The Impact of the Issue

How many people or systems will be affected by the issue?

The Probability of the Issue Occurring

How likely is it that the issue will become a real problem?

4 Challenges that Alert Fatigue Causes

Tier 1 analysts face the highest volume of threats in the organization. Therefore, they are the team members who struggle the most with the effects of alert fatigue.

Here are a few of the challenges that organizations face as a result of alert fatigue:

1. High Turnover

Organizations already struggle to hire and retain quality cybersecurity talent. According to a study by Cybersecurity Ventures, the number of unfulfilled cybersecurity positions spiked by 350% over the 8-year period they surveyed (2013 to 2021).

When analyst jobs feel more like pressure cookers as a result of alert fatigue, talented professionals leave their jobs and perhaps even the industry to find positions that are less stressful. This exacerbates the shortage of cybersecurity talent in the industry.

2. Security Analyst Burnout and Overwhelm

Similarly, alert fatigue often causes burnout and overwhelm among analysts. These daily stressors make it difficult for analysts to perform well at their jobs.

3. Numerous Siloed Security Tools Wasting Time

Most organizations use an array of security tools, and analysts must switch back and forth among them. This task switching limits productivity. 

However, there is a need for security tools with advanced functionalities to keep up with the expanding attack surface. By automatically collecting and triaging alerts from different systems (like endpoint and email security) within a single investigation pipeline for incident response, teams can better manage having alerts coming from multiple sources. 

4. Threats Hiding Under an Abundance of False Positives

Most of the alerts that analysts review are false positives. These alerts distract the analysts from the real issues that must be addressed.

The frequency of false positives results from the high number of alerts and the lack of confidence in sources of information. Even a 1% false positive rate can trigger tons of alerts that waste the time of security analysts who are already overwhelmed with alerts.

According to a study by the Ponemon Institute, less than 20% of malware alerts that organizations receive are reliable. This makes it difficult for the security teams of enterprise-level organizations to sort through the false positives to find the threats that actually pose a risk.

Every false positive manually reviewed by an analyst is a waste of time and resources. Without automation, security teams will not be able to keep up with the growing volume of alerts—including the pesky false positives. 

How can Alert Fatigue be Prevented?

There are many ways to prevent alert fatigue, including:

Automating the Review and Triage of Alerts Using Rules-Based Alerting

Rules-based alerting is a method of automatically reviewing and triaging alerts using a set of predetermined rules. This can help reduce the number of alerts that need to be manually reviewed, and can ultimately help prevent alert fatigue.

Organizing Alerts Into Threat Clusters

Categorizing alerts can help reduce the overall number of alerts that need to be reviewed, and can ultimately help prevent alert fatigue. By automatically classifying threats, groups of related alerts can be responded to more efficiently.

Using a Tool to Manage Triage of Alerts

There are a variety of tools that can help with the review and triage of alerts. The best security alert triage solutions, like Intezer, are capable of collecting alerts from different security systems, extracting artifacts, and analyzing code to classify threats, identify false positives, and provide clear recommendations for response. This kind of technology for SOC teams can prevent alert fatigue, by functioning much like a Managed Detection & Response service provider (but at a fraction of the cost).

How to Identify an Incident and Determine the Level of Response Necessary

Not all incidents are created equal. Some incidents may be more serious than others and require a greater response.

When responding to an incident, it is important first to identify the severity of the issue.

This can be done by considering the potential impact of the incident and the information that has been compromised. Once the severity of the issue has been determined, the appropriate response can be initiated.

How Security Operation Centers (SOCs) Can Reduce Their Mean Time to Respond (MTTR)

It’s no secret that alert triage is one of the most detrimental inefficiencies within SOCs.

Tier 1 analysts spend too much time on each alert. Analysts can easily spend hours trying to gather information about a single alert, only to discover it was a false positive. This inefficiency damages an organization’s MTTR and leaves it vulnerable to attack.

The Information Tier 1 Analysts Need to Review Alerts Comprehensively

Surface-level details do not provide enough context for security analysts to understand the full scope of each alert. Analysts must be able to identify:

• Whether an alert is a false positive
• The risk posed to the device in question
• The priority of the risk

To gather the necessary details to review each alert, analysts must be able to quickly access:

• The activity log for the device
• Whether security patches are active
• Information on related alerts and vulnerabilities
• The level of protection each device has
• All the related software and vendors

But the key challenge analysts face is that the data they need to access is distributed throughout siloed security dashboards. 

They cannot view all the details and contextual information in one place, which makes it difficult for them to gather and organize the details that pertain to each alert.

Security teams can improve their MTTR by investing in a solution that gives them access to all the information they need in a unified interface. 

The chosen solution should provide robust contextual information as well as an organized dashboard for reviewing alerts.

How a Virtual Tier 1 Solution Can Automate and Streamline Triage to Speed Up Alert Response

Here are a few alert triage tasks that automation can improve for security teams:

1. Increased Visibility

Instead of spending time managing siloed tools, security teams need an alert triage solution that unifies all their alerts into a single dashboard view. This will save time and create a more organized process for analysts to review alerts from different sources like various endpoint and email security solutions. 

2. IOC Extraction

Instead of manually gathering contextual information, security teams need a solution that automatically analyzes and extracts IOCs, TTPs, and advanced detection content for threats. Technology can handle enriching alerts as part of an automated triage process, so teams shouldn’t have to spend time on these tasks.

3. Prioritization for Alerts

Security teams should invest in a solution that allows them to prioritize alerts for response. The prioritization process allows teams to focus on remediating confirmed threats.

4. False Positive Identification

Security teams need triage solutions that can automatically identify and verify alerts that are false positives, posing no actual threat to the organization. While false positives should be clearly labeled, they shouldn’t be hidden. Teams should retain visibility into these alerts, so they can review, verify as needed, and create rules to prevent similar false positives from triggering in the future.

5. Classifying Threats

The volume of alerts that Tier 1 analysts receive daily is impossible to navigate without the help of a categorization tool.

By applying filters to a large list of alerts, analysts can focus on specific threats and the contextual information surrounding them.

6. A Seamless Process for Escalating High-Priority Alerts

Instead of escalating alerts with incomplete information, analysts need a solution that allows them to easily compile the details of pressing alerts and escalate them to Tier 2 analysts with complete context.

How Intezer Automates the Alert Triage Process with Little Human Supervision

Intezer provides automated, algorithm-driven Tier 1 services with little oversight from information and security analysts.

Instead of worrying about siloed solutions, Intezer connects to the alert pipelines to eliminate silos and provide contextual information about alerts. You can automate alert triage and response tasks with Intezer’s solution.

Instead of Wasting Time on False Positives, Analysts Can Focus Their Attention on Pressing Issues.

Security analysts can leverage Intezer’s interface to quickly identify false positives, gain rich information on each alert to determine the real threats, .

Intezer empowers analysts to automate their “grunt work” to use their intelligence to execute tasks more effectively and efficiently.

Automatically Extract IOCs From Suspicious Links and Files.

Enterprise-level organizations receive massive numbers of emails, which makes them prime targets for phishing attacks.

Intezer allows security teams to seamlessly integrate automation into their abuse inboxes and security systems to identify threats within URLs and attachments.

Intezer Eliminates Silos Among Security Tools by Providing a Seamless Solution.

Analysts waste time switching between tools. With Intezer, security teams gain access to a private database that provides logs of every investigation across the enterprise. Stop wasting time and resources on alert triage and start improving your security ROI.

With More Time to Hunt, Intezer Helps Teams Create Customizable High-Quality Detection Rules.

There is no one-size-fits-all solution to cybersecurity. Every organization and team has its own needs based on its existing resources and the nature of the threats they face.

Intezer lets security teams set up rules to search for advanced threats proactively.

Ready to Empower Your Security Team with Automated Alert Triage?

Get started with Intezer for free.