The risks of doing vulnerability testing and management for compliance only

As a CISO, I’ve seen vulnerability management and security testing approached in two very different ways. One is a proactive, risk-based discipline that’s tightly woven into the development and operational fabric of the organization. The other is something far less effective, where vulnerability scanning becomes a box to check in a compliance worksheet, nothing more than a ritual performed ahead of an audit. 

If anything, going through the motions for compliance alone is worse than doing nothing because it gives you a dangerous illusion of security.

Giving the tools a chance

Nowhere is this more evident than with dynamic application security testing, or DAST. The value of DAST lies in its ability to test the entire attack surface of a running application (or the whole application environment) in real-world conditions the way an attacker would. It doesn’t need access to source code or the development environment—it sees the app as the world sees it, and that’s where many vulnerabilities live and breathe.

DAST excels at uncovering injection flaws, broken authentication, misconfigured security headers, and other exploitable issues that, left unresolved, are exactly the kinds of problems that make headlines when they result in a breach. But to address them, you need to run the tool and act on the results.

The checkbox AppSec epidemic

And here’s the problem: when DAST is only run on carefully selected assets because an audit or framework requires a vulnerability scan, its power is diminished. I’ve seen organizations launch scans just once a year, targeting non-production systems with no authentication, little coverage, and no remediation follow-through. The reports are filed, a few items are logged, and leadership breathes a sigh of relief because “we passed.” In reality, nothing of substance has changed, and all the risks remain.

To be clear, this isn’t unique to DAST. The same mindset pollutes the broader application security landscape, commonly driven by the desire to tick that box with the minimum effort and cost.

Static application security testing (SAST) is often run on codebases to tick a compliance requirement, but the alerts are ignored or filtered out to keep the noise down. Interactive testing (IAST), which combines the best of DAST and SAST, is dismissed as too complex to justify unless mandated. Even software composition analysis (SCA), which is our main defense against supply chain vulnerabilities, is all too often limited to annual scans or procurement checklists, missing newly reported vulnerabilities in third-party libraries and other real-world risks that move faster than any compliance cycle.

When compliance-as-security fails, it fails hard

I’ve also seen the longer-term consequences of treating vulnerability management as a paper-pushing exercise. When internal scans are run, the results are quietly filed away with little or no action, even if vulnerabilities were found. 

But when the same vulnerabilities are discovered by attackers instead of our own scanners, there’s a mad scramble to react. I’ve watched teams patch in production under duress, field regulator inquiries, and attempt to explain to customers why a known issue was left unaddressed. These are the moments when the façade of compliance-as-security crumbles, often with significant reputational and financial damage.

It’s critical that we evolve our thinking here. Compliance should be the minimum baseline, not the desired end result.

Bottom line: Attackers don’t care about your compliance

The way to make security real is to anchor your vulnerability management program to risk, not regulation. 

When you adopt a risk-based mindset, you stop chasing audit checkmarks and start embedding security into the software development lifecycle. You scan code and running apps continuously. You use DAST in production-equivalent environments with proper authentication and coverage. You treat SCA as an ongoing requirement, not a procurement-time activity. And most importantly, you demand accurate results that let you act on reports in a continuous cycle of triaging, fixing, validating, and retesting.

As security leaders, we have to recognize that attackers don’t care whether we’re compliant. They care whether we’re exposed. And the only way to avoid that is to make vulnerability testing and management part of our operational DNA—not just our audit narrative.