Evaluating an agentic MDR provider means checking five things: the autonomy model (what the AI handles without human sign-off), transparency into its reasoning, coverage across the full threat lifecycle, flexibility for custom detections, and accountability when an automated action goes wrong.
Key takeaways
- Evaluate agentic MDR providers on five criteria: autonomy model, transparency, coverage scope, custom detection flexibility, and accountability.
- Push for specific numbers, like the percentage of alerts an AI closes end-to-end, instead of general capability claims.
- A provider should be able to show you the reasoning behind an AI decision, not just the final verdict.
- Get the provider’s accountability and remediation policy in writing before you sign, not as a verbal assurance during the sales process.
- Use the 10 RFP questions below to structure evaluation conversations across all five criteria.
Once you know what agentic MDR is, evaluating a specific provider’s claim to it is harder than it sounds, since “agentic” gets used loosely in marketing. Traditional MDR evaluation criteria (SLAs, coverage, staffing) still matter, but they don’t tell you whether a provider’s AI is actually doing the work it claims.
The five evaluation criteria
The five criteria are autonomy model, transparency, coverage scope, custom detection flexibility, and accountability. Each maps to a real gap between providers who use “agentic” loosely and ones who can back the claim up with specifics.
1. Autonomy model: What the AI handles vs. what humans authorize
Ask what percentage of alerts the provider’s AI closes end-to-end without analyst review, and for what severity levels. “Our AI helps analysts investigate faster” is a different claim than “our AI closes a defined share of low-severity cases without review.” A provider that can’t give you a specific number is likely describing AI-assisted analysts, not agentic MDR.
2. Transparency: Can you see the AI’s reasoning?
You should be able to see the signal that triggered an investigation, the telemetry the agent queried, the hypothesis it formed, and why it reached its verdict, timestamped. Without that reasoning trail, agentic MDR is a black box, which creates real audit and compliance risk. Ask to see an actual investigation record, not a product screenshot from a slide deck.
3. Coverage scope: Does the AI cover your full threat surface?
Ask whether the provider’s detection library covers your specific attack surfaces (endpoint, cloud, identity, SaaS, email, network), how current that content is, and whether you can see what’s actually being monitored. Coverage that looks broad on a data sheet can still miss the specific surfaces that matter to your environment.
4. Custom detection flexibility
Most agentic MDR services run the provider’s own detection library and don’t natively incorporate a customer’s rules. If your environment has detection needs the provider’s standard library won’t catch, ask specifically how (or whether) they support custom detections before you sign, not after.
5. Accountability: Who owns it when the AI is wrong?
This is the question providers are least eager to answer directly. Ask what happens when an automated action causes unintended disruption, whether high-impact actions get human review before they execute, and what the provider’s remediation and incident-review process looks like. Get the policy in writing, not a verbal assurance.
RFP questions to ask any agentic MDR provider
- What percentage of alerts does your AI close end-to-end, and at what severity levels?
- Can you walk us through a full case, start to finish, with the AI’s role marked at each step?
- Can we see the reasoning trail behind a specific AI-driven verdict, not a summary?
- What audit and compliance evidence does the service produce for regulated environments?
- Which of our specific attack surfaces does your detection library cover today?
- How often is detection content updated, and can we see what’s currently monitored?
- Can we bring our own detection rules, and if so, how are they maintained?
- What happens when an automated response action causes unintended disruption?
- What human review happens before a high-impact action executes?
- Can you share your incident-review process and the SLA for correcting an AI error?
Expel’s take
Expel applies these same five criteria to itself.
- On autonomy: Ruxie, Expel’s AI SOC manager, is AI-powered and human-led, handling high-volume triage and investigation while Expel’s SOC analysts authorize consequential actions.
- On transparency: Every action is visible in Expel Workbench™ with the reasoning behind it.
- On coverage: Expel’s has 160+ coverage areas and counting, and we support a bring-your-own-tech (BYOT) model.
- On accountability: Expel maintains a 14-minute MTTR on high- and critical-severity incidents and a defined incident-review process.
Frequently asked questions
What should I look for in an agentic MDR provider?
Evaluate five areas: the autonomy model (what the AI resolves without sign-off), transparency into its reasoning, threat lifecycle coverage across your attack surfaces, flexibility for custom detections, and clear accountability protocols for automated actions.
How do I verify an MDR provider’s AI is truly agentic?
Ask what percentage of alerts the AI closes end-to-end without analyst intervention, and request evidence rather than estimates. Also ask to see the AI’s investigative reasoning output, not just the final verdict.
What transparency should an agentic MDR provider offer into AI decisions?
You should be able to see the triggering signal, the telemetry queried, the hypothesis formed, why the verdict was reached, and what action was taken, all timestamped. Without that trail, agentic MDR is a black box.
Who is accountable when an agentic MDR provider’s AI makes a mistake?
The provider is. Any reputable agentic MDR provider should have a clear, written policy covering what happens when an automated action causes unintended disruption, including remediation and incident review.
Can I bring my own detection rules to an agentic MDR provider?
It varies by provider. Most run their own detection library and don’t natively incorporate customer-authored rules, so confirm specifically how each provider handles custom detections before signing.