Ending the “critical” label wars: Why proof should settle finding severity debates

What severity scores actually tell you

CVSS is a useful, well-maintained standard and it should stay in your toolkit, but it’s worth being precise about what it measures: 

A CVSS Base score measures the intrinsic characteristics of a vulnerability and the potential impact if it’s exploited. It does not tell you whether your application, configured your way, with your controls in front of it, can actually be exploited today.

That distinction is vital because attackers don’t work from your score – they work from what’s reachable and what works. Every organization I speak with can point to a backlog full of “critical” findings that have never been and may never be exploitable in their environment, sitting alongside a handful of “medium” findings that turned out to matter a great deal.

When the only shared language between security and engineering is a score, the debate is inevitable. A severity score from the security team is really an opinion about potential risk. But engineers – reasonably – want evidence.

What the frameworks already tell us

The good news is that the industry’s go-to guidance has been moving in this direction for years to add a realistic risk dimension to dry numeric scores:

• NIST SP 800-40 Rev. 4 frames patching as risk-based enterprise preventive maintenance, not a race to close every ticket. Prioritization should reflect threat, exposure, and asset criticality.
• CISA’s SSVC (Stakeholder-Specific Vulnerability Categorization) replaces a single number with a decision tree built around exploitation status, exposure, and mission impact, ending in an action: Track, Track*, Attend, or Act.
• FIRST’s EPSS estimates the probability that a vulnerability will be exploited in the near term, which is a likelihood signal that CVSS was never meant to provide.
• CISA’s Known Exploited Vulnerabilities (KEV) catalog is the strongest external signal available: exploitation has been observed in the wild.
• ISO/IEC 27001:2022 (Annex A 8.8) requires organizations to evaluate exposure to technical vulnerabilities and take appropriate measures. The standard asks for judgment, not volume.

Taken together, these all point to the same principle: severity is only an input, while exploitability and context determine the decision. What all these frameworks leave open is the last mile. Which of these signals applies to this finding, in this application, running in this environment?

Where runtime validation closes the gap

Threat feeds tell you what’s being exploited somewhere. Scores tell you what could go wrong in theory. Neither can tell you whether the issue is exploitable in your own deployed application. The most direct way to get that answer is from testing the running application the way an attacker would, and capturing the evidence.

This is where taking a DAST-first approach changes the conversation. A finding that is validated at runtime comes with proof: the request, the response, the demonstrated impact. The discussion with engineering shifts from “do you believe my score?” to “here is what happened when we tried it” – and it can become a much shorter discussion. Nobody has to take anything on trust, and the developer has what they need to reproduce and fix the issue faster.

In practice, I’ve found that runtime proof does three things for a security program:

1. It removes the argument. A confirmed, reproducible exploit path doesn’t need a severity or reachability debate. It needs an owner and a remediation date.
2. It protects engineering time. Findings that can’t be exploited in context can often be deprioritized with a documented rationale, making for a defensible and auditable decision rather than an ignored ticket.
3. It confirms fixes. Re-testing after remediation shows whether the exploit path is actually closed, not just that a ticket was resolved.

A practical model for improving CVSS prioritization

If you’re looking to reduce friction between dev and security without rebuilding your entire program, a layered approach to enriching the basic CVSS score works well:

1. Start with external signals. KEV listing and high EPSS values move a finding up the queue immediately.
2. Add your context. Is the application internet-facing? Does it handle sensitive data? Who owns it? This is where ASPM-style correlation earns its keep.
3. Validate at runtime. Confirm exploitability against the running application and attach the evidence.
4. Set the SLA from the combination of signals, not from the score alone. Confirmed and exposed gets the shortest clock. Unconfirmed and unreachable gets a documented, reviewable decision.
5. Re-test and close with evidence. Closure means the exploit path no longer works.

Notice that this doesn’t discard CVSS but rather puts the score in its proper place: a starting point rather than the final verdict.

What to measure instead of severity-based remediation counts

If you want to know whether your enriched prioritization process is working, change what you report. Counting how many criticals have been opened or closed tells you less about risk and more about scanner configuration. 

More meaningful measures include:

• Time to remediate confirmed and exposed vulnerabilities
• Percentage of findings validated before being assigned to engineering
• Re-open rate (how often an issue marked as fixed turns out to still be exploitable)
• Volume of security-to-engineering tickets disputed or rejected

That last one is my favorite leading indicator of trust. When the number of disputes falls, it usually means findings are arriving with evidence.

A final reflection

The severity label wars persist because they feel like a technical disagreement that needs to be cleared up. Security asks engineering to act on a number, while engineering asks security to show the logic behind that number. Both sides are being professional and reasonable, yet the disputes keep coming back – because in reality, it’s a trust problem. 

Proof is what bridges that gap. When we hand over evidence instead of adjectives, prioritization becomes a shared decision about real risk, and the conversation moves from “how critical is it, really?” to “what do we do about it?” That’s the conversation worth having, and it’s one we can have a lot more often if only we’re willing to validate before we escalate.