This article explores how effective SOC management requires balancing quantitative metrics with qualitative conversations. The article features insights from a video interview with Ben Brigida and Ray Pugh, SOC operations leaders at Expel.
The complete interview can be found here: How to measure a SOC
The most dangerous trap in SOC management involves treating data as gospel—the ultimate and only source of truth about operational effectiveness. While quantitative metrics provide valuable visibility into security operations, they never tell complete stories. Organizations that over-rely on metrics alone make decisions based on incomplete information, missing critical context that only human conversation and observation can provide.
Effective SOC leadership requires sophisticated integration of hard data with soft intelligence gathered through regular conversations with team members. This balanced approach recognizes that all metrics are proxies for what’s actually happening in the real world, and proxies always have limitations, gaps, and blind spots that must be acknowledged and addressed.
Understanding why data cannot be the only source of truth
Even organizations with highly sophisticated instrumentation and comprehensive data collection capabilities face fundamental limitations in what metrics can reveal. Numbers describe what happened but rarely explain why it happened, what it means for team health, or what actions would most effectively address emerging patterns.
Well-instrumented SOCs generate vast quantities of performance data: response times, alert volumes, investigation durations, escalation rates, and countless other measurements. This data visibility represents genuine progress compared to operating without metrics. However, the availability of data creates temptation to treat quantitative analysis as sufficient for understanding operational effectiveness.
The reality is that metrics provide partial visibility at best. They capture aspects of operations that can be easily quantified—timestamps, counts, durations—while missing equally important factors that resist simple measurement. Team morale, cognitive load, relationship dynamics, skill development, and dozens of other crucial elements exist outside what traditional metrics capture.
Organizations with long operational histories and mature measurement programs understand their data’s limitations intimately. They know which metrics reliably reflect underlying reality and which require careful interpretation. They recognize gaps in coverage where important activities occur without measurement. This institutional knowledge about data limitations proves as valuable as the data itself.
The most sophisticated approach combines confidence in data quality with appropriate skepticism about data completeness. Teams should trust their metrics when making tactical decisions about workflow optimization or resource allocation, while maintaining awareness that strategic decisions about team health and capability development require information beyond what spreadsheets provide.
Industry research consistently highlights the gap between metrics collection and meaningful insight—findings that underscore why qualitative assessment remains essential alongside quantitative data in SOC operations.
Structuring SOC management relationships to enable meaningful conversation
Effective SOC management and cross-referencing of data with reality requires organizational structures that enable managers to develop deep understanding of individual team members and collective team dynamics. This understanding comes through consistent, high-quality interactions that would be impossible with excessive spans of control.
Successful SOC programs deliberately limit the number of direct reports each manager oversees. This structural choice enables managers to spend appreciable time with every single analyst individually each week—not brief check-ins, but substantive conversations that build relationships and provide genuine insight into how analysts experience their work.
These regular one-on-one conversations serve multiple purposes beyond status updates or task management. They create space for analysts to discuss challenges they’re facing, share observations about operational patterns, express concerns about team dynamics, or seek guidance on professional development. The consistency and quality of these interactions determine how much valuable qualitative intelligence managers can gather.
Manager collaboration amplifies the value of individual conversations. When SOC managers compare notes on the conversations they’re having throughout the week, patterns emerge that might not be apparent from any single manager’s observations. Multiple managers noticing similar themes provides stronger signal than isolated observations.
Creating regular forums for this manager collaboration requires intentional program design. Scheduled meetings with clear agendas focused on synthesizing qualitative observations from team interactions ensure this cross-referencing happens systematically rather than sporadically. The topics of conversation should be carefully considered to draw out the most valuable insights.
The subjective indicators that metrics miss entirely
Quantitative metrics excel at measuring objective, quantifiable aspects of SOC operations. Subjective indicators—equally important but fundamentally different in nature—require qualitative assessment through direct observation and conversation.
Team tone represents one crucial subjective indicator. Is the team energized and engaged, or tired and stressed? Do team members interact positively with each other, or do tensions exist that might affect collaboration? These dynamics profoundly influence operational effectiveness but exist completely outside traditional performance metrics.
General sentiment provides another important dimension. How do analysts feel about their work, the organization, and their career trajectory? Are they excited about new challenges or feeling overwhelmed? Do they believe their contributions matter and are recognized? These sentiment indicators often predict retention and engagement issues long before they manifest in quantifiable performance changes.
Pain points that analysts experience in daily work frequently don’t appear in aggregate statistics. An analyst might struggle with a particular tool’s interface, find certain alert types unnecessarily time-consuming due to insufficient context, or face recurring coordination challenges with other teams. These operational friction points accumulate to reduce efficiency and satisfaction, but they’re only discoverable through conversation.
Understanding what energizes team members provides equally valuable intelligence for SOC management decisions. What aspects of the work do analysts find most engaging and fulfilling? What skills do they want to develop? What types of investigations do they find most interesting? This information helps leaders make assignment and development decisions that maintain motivation and support career growth.
The cross-referencing process involves constantly comparing these subjective indicators with objective performance data to validate whether the story you’re getting from metrics aligns with the reality you observe through direct interaction. When they align, confidence in both data and observations increases. When they conflict, investigation is needed to understand the discrepancy.
Challenging assumptions through systematic inquiry within SOC management
Even experienced SOC management leaders with strong institutional knowledge should regularly challenge their assumptions about what data means and what team conversations reveal. This systematic skepticism prevents confirmation bias and ensures that interpretations remain grounded in current reality rather than historical patterns.
The analyst examples from earlier discussions demonstrate this principle perfectly. When metrics suggested potential performance issues with analysts believed to be strong performers, the appropriate response was investigation rather than immediate acceptance of either the metrics or the pre-existing beliefs. Deep analysis revealed the metrics were misleading, but that discovery required willingness to question initial interpretations.
This same investigative approach should apply universally across the team. When metrics suggest an analyst is performing well, verify that interpretation through conversation and observation. When qualitative observations suggest challenges that aren’t reflected in metrics, investigate whether measurement gaps exist or whether the observations reflect temporary situations rather than sustained patterns.
The verification process works in both directions. Sometimes data reveals patterns that managers haven’t observed directly, prompting conversations that uncover issues requiring attention. Other times conversations reveal important context that explains apparently concerning metric patterns. Both directions of investigation improve overall understanding.
Maintaining intellectual humility about limitations of both data and human observation prevents overconfidence in either source alone. Leaders should acknowledge explicitly that they don’t have complete visibility into operations, that gaps in understanding exist, and that continuous investigation is necessary to maintain accurate assessment of team health and effectiveness.
Making data representative through continuous refinement
The goal of cross-referencing metrics with qualitative observations extends beyond validation—it drives continuous improvement in measurement approaches to make data more representative of actual operational reality.
When conversations consistently reveal important operational aspects that metrics don’t capture, that identifies opportunities to expand measurement scope or refine existing metrics. Perhaps certain investigation activities consume significant time but aren’t tracked separately from general alert handling. Perhaps mentoring contributions matter for team development but exist completely outside performance measurement.
Expanding measurement to capture previously invisible activities improves data representativeness, enabling more complete understanding from quantitative analysis alone. However, this expansion must be balanced against measurement overhead and potential for behavioral distortion when new metrics become targets.
Qualitative feedback also identifies situations where existing metrics mislead rather than inform. The analyst examples where longer response times actually indicated advanced threat detection capability, or lower alert volumes reflected high-value incident work, illustrate how metrics can be technically accurate while creating false impressions. Understanding these limitations enables better interpretation.
Continuous refinement of both what gets measured and how measurements are interpreted represents an ongoing SOC management process rather than a one-time achievement. As SOC operations evolve, as threats change, and as team capabilities mature, measurement approaches must adapt to maintain their utility for decision-making.
Integrating data and conversation in operational decision-making
The practical value of cross-referencing data with conversation becomes most apparent in how it informs actual SOC management decisions about resource allocation, process changes, training investments, and team development.
SOC management workforce planning decisions benefit significantly from combining quantitative capacity metrics with qualitative understanding of team stress levels and workload perception. Data might show teams operating at 60% of theoretical capacity—suggesting room for additional work—while conversations reveal analysts feeling overwhelmed due to particularly challenging incident patterns or knowledge gaps requiring intensive investigation. Both perspectives inform better decisions than either alone.
Process improvement initiatives should similarly integrate both data and qualitative feedback. Metrics might identify bottlenecks in specific workflow stages, while analyst conversations explain why those bottlenecks exist and what changes would actually help. The combination of quantitative problem identification with qualitative solution development produces more effective improvements.
Training and development decisions require understanding both performance metrics and individual career aspirations, learning styles, and development readiness. An analyst might have strong technical metrics while expressing interest in developing leadership skills through mentoring. Another might struggle with certain investigation types while showing strong aptitude for automation development. Data alone doesn’t surface these development opportunities.
Team composition and assignment decisions benefit from understanding skill diversity, collaboration patterns, and interpersonal dynamics that metrics cannot capture. Building effective shift teams requires considering not just individual capabilities reflected in performance data but also how team members work together, complement each other’s strengths, and create positive or negative group dynamics.
Building organizational cultures that value both data and dialogue
The organizational culture surrounding SOC management, measurement and conversation profoundly influences how effectively teams can leverage both types of information. Cultures that treat data as gospel discourage the questioning and investigation necessary for proper interpretation. Cultures that dismiss metrics entirely lose the objectivity and visibility that measurement provides.
The balanced culture that enables effective cross-referencing treats data as important but incomplete, and treats conversation as valuable but potentially biased. Leaders model this balance by regularly referencing both quantitative analysis and qualitative observations when explaining decisions, acknowledging limitations of both approaches, and demonstrating how they integrate multiple information sources.
Creating psychological safety for honest conversation proves essential. If analysts believe conversations are evaluative rather than investigative, they’ll naturally present idealized versions of their experience rather than authentic ones. This defensive posturing eliminates the value of qualitative intelligence gathering.
Transparency about how data and conversation inform decisions helps teams understand why both matter. When leaders explain how metrics revealed a pattern that prompted conversations which provided context changing interpretation, team members understand the integrated approach and can contribute more effectively to both measurement and dialogue.
Resources for effective SOC management
Organizations developing sophisticated SOC management approaches can benefit from additional resources and industry guidance:
- Performance metrics, part 1: Measuring SOC efficiency explores capacity planning, utilization measurement, and alert latency tracking for data-driven SOC leadership
- Performance metrics, part 2: Keeping things under control addresses change-point analysis and techniques for detecting operational volatility before it impacts team performance
- 7 habits of highly effective SOCs examines foundational practices for building SOC culture, managing capacity, and creating collaborative learning environments
- How to measure SOC quality discusses quality control approaches including inspection methodologies, check sheets, and balancing efficiency with thoroughness
- Stressed SOC? Data’s your best ally to justify more resources provides frameworks for using utilization metrics to communicate resource needs to leadership
- What is a security operations center (SOC)? offers comprehensive overview of SOC functions, roles, and operational challenges
- How to measure SOC efficiency features video insights from SOC operations leaders on balancing quantitative metrics with qualitative assessment
The success of SOC management ultimately depends on resisting the temptation to treat any single information source—whether quantitative metrics or qualitative observations—as sufficient for understanding operational effectiveness. Organizations that systematically cross-reference multiple perspectives, maintain appropriate humility about limitations of all data sources, and continuously refine their approaches based on integrated insights develop the most accurate understanding of their operations and the most effective leadership practices.