Skip to main content

Security Operations

Important SOC KPIs and Metrics

A cybersecurity operations center (SOC) is a dedicated team or unit within an organization that is responsible for monitoring and protecting the organization’s computer systems and networks. The SOC is typically staffed by security analysts and other security professionals who use a variety of tools and techniques to detect and respond to security threats and incidents.

One of the key challenges facing SOCs is the need to effectively measure and evaluate their performance and effectiveness. This is where cybersecurity operations center KPIs (key performance indicators) and metrics come into play.

KPIs and metrics are specific, quantifiable measures that are used to evaluate the performance and effectiveness of a particular process or activity. In the context of a SOC, KPIs and metrics are used to measure and evaluate the performance of the SOC team and its various processes and activities.

Some common KPIs and metrics used by SOCs include:

  • Time to detect: This metric measures the amount of time it takes for the SOC team to detect a security threat or incident. The goal is to detect threats and incidents as quickly as possible in order to minimize their impact and prevent further damage.
  • Time to respond: This metric measures the amount of time it takes for the SOC team to respond to a detected security threat or incident. The goal is to respond to threats and incidents as quickly as possible in order to contain and remediate them.
  • Incident severity: This metric measures the severity of a security threat or incident based on factors such as the number of systems affected, the type of data compromised, and the potential impact on the organization.
  • Incident resolution: This metric measures the percentage of security threats and incidents that are successfully resolved by the SOC team. The goal is to resolve as many threats and incidents as possible in order to prevent further damage.
  • False positives: This metric measures the number of times the SOC team incorrectly identifies a non-threatening event or activity as a security threat or incident. The goal is to reduce the number of false positives in order to avoid wasting time and resources on non-threatening events.

In addition to these core KPIs and metrics, there are many other metrics that can be used to evaluate the performance and effectiveness of a SOC. For example, metrics such as staff availability, training and education, and threat intelligence can all provide valuable insights into the performance and effectiveness of a SOC.

Overall, the use of KPIs and metrics is critical for evaluating the performance and effectiveness of a cybersecurity operations center. By regularly measuring and tracking these metrics, SOC teams can identify areas for improvement and take steps to optimize their processes and activities. This can ultimately lead to more effective and efficient incident response and improved security for the organization.

The challenge of process gaps in Incident Response

Cybersecurity incident response is the process of responding to and managing a security breach or attack on an organization’s computer systems. This process is critical for mitigating the impact of an attack and preventing further damage.

However, there are often gaps in the incident response process that can hinder an organization’s ability to effectively respond to an attack. These gaps can occur at various stages of the incident response process, including the identification and assessment of an attack, the containment and remediation of the attack, and the post-incident review and analysis.

One common gap in the incident response process is a lack of clear procedures and protocols for responding to an attack. Many organizations do not have a well-defined incident response plan in place, or the plan is not regularly tested and updated. Without clear and actionable procedures, it can be difficult for an organization to quickly and effectively respond to an attack.

Another gap in the incident response process is a lack of coordination and communication among the various teams and individuals involved in responding to an attack. Different teams may have different roles and responsibilities, but they must work together in a coordinated manner to effectively respond to an attack. Without effective communication and coordination, there is a risk of confusion and duplication of effort, which can hinder the response process.

Another common gap in the incident response process is a lack of awareness and training among the individuals involved in the response. Many organizations do not provide regular training and education on incident response to their employees, which can result in a lack of knowledge and understanding of the appropriate actions to take during an attack. This lack of knowledge and training can lead to mistakes and delays in the response process.

To address these gaps in the incident response process, organizations must take a proactive approach to incident response planning and preparation. This includes regularly reviewing and updating incident response plans, providing training and education to employees on incident response procedures, and fostering effective communication and coordination among the various teams involved in the response process.

Overall, the success of an organization’s incident response efforts is heavily dependent on its ability to effectively identify and address gaps in the incident response process. By taking a proactive and holistic approach to incident response planning and preparation, organizations can improve their ability to quickly and effectively respond to a security breach or attack.

Skill gaps and limitations of scenario based training in Cybersecurity

The skills gap in cybersecurity is a growing concern for organizations around the world. As the number and sophistication of cyber threats continue to increase, the demand for skilled cybersecurity professionals is outstripping the supply. This skills gap poses a significant challenge for organizations looking to protect their networks and systems from cyber-attacks.

One of the key challenges in addressing the skills gap in cybersecurity is the limitations of scenario-based training. Scenario-based training involves simulating a real-world cyber-attack and providing participants with the opportunity to apply their knowledge and skills to respond to the threat. While this type of training can be useful for teaching specific skills and procedures, it has several limitations.

First, scenario-based training is often limited in scope and may not provide participants with a comprehensive understanding of the full range of cyber threats they may face. It is also limited in terms of its ability to prepare participants for the dynamic and rapidly changing nature of the cyber landscape.

Second, scenario-based training is often time-consuming and resource-intensive, making it difficult for organizations to provide this type of training to large numbers of employees. It is also often expensive, requiring specialized equipment and expertise to set up and run.

Third, scenario-based training can be overly focused on specific threats and may not adequately prepare participants for the wide range of cyber threats they may encounter in the real world. It can also be unrealistic and may not accurately reflect the challenges and complexities of dealing with a real cyber attack.

In conclusion, the skills gap in cybersecurity is a significant challenge for organizations. While scenario-based training can be useful for teaching specific skills and procedures, it has several limitations that make it difficult to use as a comprehensive approach to addressing the skills gap. To effectively address this challenge, organizations must adopt a more holistic approach to cybersecurity training that incorporates a range of different techniques and methods. This can include providing employees with access to ongoing professional development opportunities, as well as implementing training programs that are tailored to the specific needs of the organization and its employees.

Context in Cybersecurity: Alert Context vs Analyst Context

Context In Cybersecurity 

Let’s start simple. Context is important in the day-to-day when we’re explaining the circumstances behind a concept, situation, or product in order to be fully understood. Without context, our understanding of a situation is limited, and we can easily misinterpret information and draw false conclusions.  

Take, for example, statistics or a company’s share price in a way that makes it look appealing. But when the additional context is added around the time horizon or comparison with a benchmark, we’re told a different story.

The more context and information we have, the more we understand the matter at hand. In cybersecurity, context has been vital when it comes to triaging and responding to security alerts. Gartner defines context-aware security as:

the use of supplemental information to improve security decisions at the time they are made, resulting in more accurate security decisions capable of supporting dynamic business and IT environments

The most cited context information types are environmental, such as location and time, but let’s cover additional factors which are critical to cybersecurity and understanding security alerts.

Alert Context  

When it comes to the abundance of alerts a security operations center (SOC) receives, the more context around the alert, the better the Analysts understanding. Today’s tools are rich with alert context. Information such asIP address, devices affected, URL, application reputation, similar alerts, source information, network traffic etcThis contextual information around the alert provides insight into the circumstance of the event, and it helps determine whether it is a true incident or a false positive. It is important for Analysts to have access to this consolidated information so that they can make quick, informed decisions on how to respond to potential threats. 

To further enrich alerts, the addition of Analyst context is vital in helping Analysts paint a picture around the circumstances of the threat, how severe it is and how to address it for a particular customer. 

Analyst Context  

In addition to alert context, having Analyst context when responding to incidents helps have a more accurate analysis and, in turn, faster and more accurate remediation. In the case of Managed Security Service Providers (MSSP’s) Analyst context also includes customer context, which is information an Analyst possesses around the type of organization the alerts are coming from. Customer context is especially important since they have different clients spanning multiple industries that require unique methods of response.  

Analyst context can include recent experience solving similar problems, short-term memory, stress levels, and more. These contextual factors are critical for an Analyst to leverage effectively to perform their duties quickly and accurately. The goal is to create an environment where SOC Analysts are optimized based on their real-time mindset and skillsets to make quick, informed decisions on how to respond to potential threats.

Analyst context includes information that an Analyst possesses and their real-time state that can be leveraged to respond to unique complex incidents for a specific organization at a specific time. Thus, reinforcing the Gartner definition that the use of supplemental information to improves security decisions at the time they are made. 

For example, a phishing attack on a financial institution may require different steps than a phishing attack on an oil & gas company. These contextual decisions are why human intervention is absolutely required today and why automation will not replace people in the SOC. It is critical that organizations have a solution which monitors these subtle changes and applies the logic to its decision-making engine.  

How to include Analyst context in your SOC?  

The problem is, Analyst context is not a data point that traditional SOC tools, like SIEM and SOAR, can collect or leverage. As per the tools used today, it does not matter which Analyst is solving which problem. SOC teams are burdened with determining who is doing what task at any given moment. This often results in random or grab-bag alert assignments and Analysts spending too much time on tasks that are not leveraging their best skills.  

By capturing both alert context and Analyst context, SOCs would be able to strategically augment their teams to ensure each Analyst is focused on the most high-leverage work for their unique capability at any given moment in time. By understanding the context of a specific alert and who on the team would be best to resolve it the fastest and most accurately, SOC teams can avoid alert escalations, mistakes being made, and most importantly, stop the attack in its tracks before it gets a foothold in/of the organization.

Penfield.AI works with your existing tools to add analyst context data

Automation today is great for static processes, but incident response will continue to require human-in-the-loop feedback and analyst context due to the ever-changing nature of attacks. Therefore, the future of Cybersecurity is augmenting the Cyber-workforce with human-machine intelligence capabilities for faster and more accurate incident response.

To supercharge your SOC and Analysts, read more on how you can introduce real-time contextual coaching and on-the-job training.

To learn more about how Penfield.AI can add Analyst Context to your SOC contact us.


Next Generation of Cybersecurity Training

Current Landscape

The skills gap in cybersecurity has been a hot topic for over 10 years now. Individuals are willing to enter the cybersecurity industry, but companies rarely give those individuals seeking an entry-level position a chance. The reason for that is the presence of automation that focuses on covering the role and responsibility of entry-level positions that include many mundane, repetitive tasks and focusing on hiring individuals with more experience that can solve complex tasks. Based on the Life and Times of Cybersecurity Professionals 2021 research report presented by ESG, it has been challenging for organizations to hire and recruit mid-career professionals (4-7 years on the job experience) and senior professionals (7+ years on the job experience). This is in line with the argument that while it’s easier to hire for entry-level positions, the demand lies among the more skilled professionals.

“If we wanted the cybersecurity market to have a supply and demand ratio in line with the broader market, we effectively have to double the cybersecurity workforce overnight.” – Will Markow, VP of Applied Research at EMSI | Burning Glass

Keep reading to learn more about the cybersecurity skills gap with contributions from Will Markow, VP of Applied research at EMSI | Burning Glass, who leads the custom research and consulting team focusing on the impact of emerging technologies on the workforce and the key trends and challenges facing the cybersecurity workforce through the CyberSeek initiative.

Fast Facts about the Current Skills Shortage In the USA

We’ve covered that the skills gap has been a topic of focus for quite some time now, but how big is the issue exactly and what has been the impact?

CyberSeek 2021 Stats:

  • The global cybersecurity workforce gap stands at 3.1 million in 2020
  • The total employed cybersecurity workforce across the United States sits at approximately 956,000
  • There are currently approximately 464,000 annual cybersecurity jobs openings across the United States
  • CISSP is the most requested cybersecurity certification, being in over 100,000 opening every year but there are only 90,000 CISSP certified individuals in the entire country
  • Some of the top job titles requested by employers include
    •      Cybersecurity Analyst, Consultant, Manager
    •      Software Developer
    •      Systems Engineer
    •      Network Engineer
    •      Penetration & vulnerability Tester

“Ever since we started Cyberseek we’ve seen a continuous trend of hiring difficulty in the space and there are not many signs of that talent shortage lightening up anytime soon.” – Will Markow

ISACA 2020 State of Cybersecurity report:

  • Fewer than half of cybersecurity applicants are well qualified (according to 70% of respondents)


Total Cybersecurity Job Openings Source: Cyber Seek, 2021


Organizations have also been feeling the pain, revealing that the skills crisis has impacted 57% of the respondents. Those organizations that have been impacted shared that the top ramifications include increasing workload on personnel, new jobs remain unfilled for weeks or months, high burnout among staff and attritions and the inability to learn or use security technologies to their full potential.

“A big pain point for employers is trying to fill cybersecurity job positions. On average, they take about 21% longer to fill than other IT jobs, which are already among some of the hardest to fill jobs in the market.”- Will Markow

Many organizations also make basic mistakes when it comes to hiring and recruiting professionals in the industry. 29% said their HR departments don’t understand the skills needed for cybersecurity, so there’s a high likelihood of gaps being present in teams.

“About 85% of cybersecurity jobs are calling for a minimum of three to five years of work experience, so by not offering more entry-level jobs there is limited opportunity for employers to build their pipeline of cybersecurity workers and help grow the next generation of cyber professionals.” – Will Markow

According to cybersecurity professionals, the responsibility for taking the necessary actions to address the impact of the skills shortage lies on the CISO/CSO, executive management and VP of HR or similar positions.

The Need for Continuous Training

A simple change that organizations can make to address this industry issue is to offer cybersecurity career advancement opportunities and commit to increased cybersecurity training across the organization, which is also what 59% of professionals are asking for.

“Employers should rethink their hesitance if they currently are not training their workers or investing in their career development, because we found that it can be an effective retention tool.” – Will Markow

The cybersecurity industry is a dynamic space that is constantly evolving, given the rapid technology changes and attack vectors, so individuals tasked with protecting an organization may be operating on outdated knowledge.

Cybersecurity professionals must keep their skills fresh and seek out the latest information about security, network vulnerabilities, and the latest capabilities; otherwise, organizations they work for are at a disadvantage to the cyber adversaries. Additionally, adversaries utilize AI to create more sophisticated attacks, even applying it to information captured and posted to the dark web. Organizations need to be more open to leveraging similar technology to protect themselves better and up-skill their teams. AI and analysts can work hand in hand, rather than independently, to understand an analyst’s unique skillsets and contextual awareness and optimize their workflow automatically to enhance their performance.

Another factor that adds to the skills shortage and why training efforts haven’t advanced in the space is the conflict between the need for training and the time actually allocated to training. According to 59% of cybersecurity professionals, it’s because of the high demands of their day-to-day jobs that get in the way of proactive education.

On the Job Training

“Certificates can be an effective signal to employers that someone has experienced within cybersecurity, and they have the right skills to perform the job, but there’s a problem when that becomes the only way to demonstrate that you have competence in the cybersecurity field.” – Will Markow

Without continuous cyberlearning, professionals fall behind and are considerably less effective in as little as 3 months. It has always been taught that hands-on experience provides more value than classroom learning and 52% of cyber professionals agree that hands-on experience is more important than certifications, while 46% place equal importance on hands-on experience and certification achievements. But, since it’s difficult for them to take the time to absorb new information and complete additional training due to the job demands, there needs to be a shift towards on the job training.

Organizations can leverage technology to put data to work and get visibility into where their cybersecurity team’s skills lie and visualize unique characteristics based on the problems and incidents they resolve. Understanding how a team solves problems in almost real-time helps trainers or senior leaders understand what the best performers are doing so that knowledge is shared across the team and where there is room for improvement so it can be addressed right away. Having access to this type of data and insights into your team will help fix the issue around hiring and allowing HR professionals to know the skills they should be looking for when hiring to fill any gaps a team has. In conclusion, hiring more analysts can’t always be the answer, and organizations need to focus more on upskilling their current teams to protect themselves against cyber-attacks.


Dawson, J., & Thomson, R. (2018). The Future Cybersecurity Workforce: Going Beyond Technical Skills for Successful Cyber Performance. Frontiers in psychology9, 744.

ESG Research Report, The Life and Times of Cybersecurity Professionals 2021, Volume V, July 2021

ISC2 Cybersecurity Workforce Study 2020.

CyberSeek –

3 Key SecOps Trends to Focus On

The global shift to remote work has made it increasingly difficult for companies to stay ahead of attackers. KuppingerCole, an international independent analyst firm that contributed to HP’s report, Blurred Lines & Blindspots, noted that there has been a 238% increase in global cyberattack volume this past year.

That’s where SecOps teams come into play to protect, respond, and mitigate the overwhelming number of threats. With the shift to remote work and the increasing number of cyberattacks, there are new trends that organizations should focus on, such as automation, alert prioritization, and workforce trends.

We’ve had the pleasure to discuss those trends with Jonathan Haas, who led security operations at Carta, an equity management platform. He has been in the security operations space for several years, working for various companies with various degrees of need for security operations. Before Carta, he led in similar roles at companies like DoorDash and Snapchat. Throughout his roles, Jonathan has relied on various tools, such as automation, to help him manage the abundance of alerts that come in.


Automation has been a hot topic in the cybersecurity industry, and many companies want to automate as much as possible. In a report published by Deep Instinct, it is noted that 90% of cybersecurity professionals agree that automation allows them to free up teams to focus on higher-value and more strategic tasks.

“When most people say they want automation, what they really want is leverage. They want what you get out of using a drill versus using a screwdriver- striving for the ability to do more with less.” – Jonathan

According to a recent report from Sumologic, 93% of security teams reported that they could not address all their security alerts each day. Hiring more analysts is not always possible, so the alternative is to leverage automation to keep pace and respond to all of the alerts. Some of the tools that allow companies to incorporate automation in their organizations are AI and Machine Learning (or Intelligence Augmentation) and SOAR (security, orchestration, automation, and response).

Alert Prioritization/Management

Alert fatigue is an increasing issue as current tools are surfacing all types of abnormalities detected to be investigated by security operations teams. Sumo logic revealed that over the past five years, 70% of companies stated that the number of security alerts they receive on a daily basis has doubled, if not more. This takes a toll on teams by making them overwhelmed and stressed, reducing their confidence to prioritize and respond. This increase can be due to a combination of many vulnerabilities in an organization and detection tools creating too many false positive alerts.

Teams should focus on mitigating the noise, seeking tools that can prioritize alerts, and surface the most severe ones for mitigation and triaging intelligently. One way to manage the mountains of alerts is to balance the skills sets of the team and automation. Security operation teams should shift from tier-based incident management to skill-based, where experienced team members with the most appropriate skill sets handle a specific incident. This allows everyone to play on their strengths, resolve incidents more quickly and reduce burnout.

Workforce Trends

We’ve touched on automation above, but there’s another point worth mentioning in relation to security operations team performance. The push toward automation is also due to organizations being primarily evaluated on metrics such as MTTR and MTTD alone, without much context around accuracy or due diligence around a particular incident.

“Tracking metrics such as MTTD and MTTR alone isn’t sufficient since they incentive security operations individuals to make sure they alert on everything and closing incidents as soon as possible without doing much diligence.” – Jonathan

Those metrics have been around for so long, and even though they don’t incentive teams in the best way, the industry can’t replace them. Instead, an accuracy measure can be added to understand the teams’ processes to resolve incidents to improve performance. Jonathan brought up an interesting point during our conversation, mentioning that teams should be more focused on whether they are actually reducing the tangible amount of risk that exists in an organization.

In conclusion, leaders should focus on people, process, and how they leverage technology within the SOC. Furthermore, speaking with their team members to see where support might be needed from an automation perspective, re-evaluate how security operations performance is being measured and how processes can be streamlined.

Covid-19 Vaccine Supply Chains are Under Cyber-Attack – What can we do?

Google trends tells us that the global Google search for “Covid-19 vaccine” reached the maximum search interest around the world this week (Figure 1). This personal interest in vaccine developments is paralleled by the interest of malicious cyber criminals and nation-state actors, who seek to exploit vaccine procurement for information, disruption and financial gain. For perspective, Johnson & Johnson’s CISO has stated that healthcare organizations are seeing cyber-attacks from nation-state threat actors “every single minute of every single day”.

Read More