That constraint is easy to say and hard to sit with, because it points at a part of security operations that gets far less attention than it deserves. We have spent a decade getting better at detection content. Detection-as-code. Version control. Peer review. Sigma and portable rule formats. Content packs from vendors and community projects. If you built a detection program in 2019 and rebuilt it today, the mechanics of writing and shipping a rule would be unrecognizably better.

What has not improved at the same rate is our ability to answer a simpler question about the rules already running in production.

Do they work?

We measure content, not health

Ask most security teams to describe the state of their detection program and you will get content metrics. How many rules. What percentage of ATT&CK is covered. How fast new detections ship after a threat report drops.

All of that matters. None of it tells you whether the estate is healthy.

A rule counts toward your coverage number the moment it is deployed. Nothing in that number reflects whether the rule is pointed at a live data source, whether the fields it depends on are still populated, whether its logic still matches how the environment behaves, or whether it has ever produced a single alert that a human found useful.

The result is a well-documented gap between theoretical coverage and actual coverage. Your ATT&CK heatmap says a technique is covered. The rule covering it has been evaluating against an empty index since a schema change in the spring. Both statements are true at once, and only one of them will matter during an incident.

Four questions worth asking

Detection health comes down to four things.

Coverage. Are we detecting the threats and techniques that matter to us? This is the one the industry already measures, and it is the one most teams have some answer for. It is also the least interesting of the four, because it is the only one where the failure mode is visible on a dashboard.

Silence. Which detections never fire, and is that because nothing happened, or because they can’t? A rule can be pointed at the wrong data source, reading a field that stopped being populated, matching a schema that changed upstream, or written against a condition that cannot occur in your environment. In every one of those cases the rule exists, maps cleanly to a technique, and is structurally incapable of firing.

Noise. Which detections fire constantly and produce little security value? This is the inverse failure and it is just as real. A detection that generates thousands of alerts a month and has never contributed to a confirmed incident is not coverage. It is a tax on analyst attention, and it degrades the response to everything else in the queue.

Decay. Are detections keeping up as threats, data, infrastructure, and the environment change? Detections do not stay good on their own. Adversary tradecraft moves. So does your estate: new cloud accounts, a migrated identity provider, a replaced EDR, a vendor schema update, a decommissioned log source. A detection that worked six months ago may not work today, and nothing about it will look different.

Of the four, silence is the one worth dwelling on, because it is the only one that does not announce itself.

A noisy rule creates pain every single day. Analysts complain. Tickets pile up. Somebody eventually fixes it, because the cost is continuous and visible. A broken silent rule generates no complaints, no tickets, and no signal of any kind. It costs nothing until the day you need it, and then it costs everything.

That asymmetry means detection health degrades in exactly the direction nobody is watching.

Why this is hard

If the problem is this obvious, the reasonable question is why teams do not simply fix it. Three structural reasons.

The failure is invisible by construction. Detecting a broken detection requires actively going looking. There is no alert for “this rule has not fired in ninety days and its data source went dark in June.” The absence of signal looks identical to the absence of threat.

The verification work does not scale with people. Reviewing a few dozen rules is a manageable exercise. Continuously validating thousands of detections against a moving environment and a moving threat landscape is not something you can staff. Most teams that have tried do it once, as a project, and then never again, because the second pass never gets prioritized against the next incident.

Tuning has an unknown blast radius. Engineers are conservative about changing detections because they cannot see the consequences of a change before they make it. Loosening a rule to cut noise might suppress the one alert that mattered. Without a way to test a proposed change against what actually happened in your environment, the safest move is to leave it alone, and leaving it alone is how estates rot.

The loop

Detection health is not a state you reach. It is a cycle you run.

Assess. Establish what you actually have. Which rules exist, which data sources they depend on, which ones have fired and how often, which have never fired at all, and where the gaps are against the techniques that matter to you.

Test. Prove the detections work. This is the step that gets skipped, and the one that matters most.

Optimize. Fix what the testing found. Tune the noisy ones. Repair the silent ones. Retire the ones that no longer serve a purpose, which is its own form of coverage improvement.

Deploy. Promote changes with confidence, because they have been validated rather than reasoned about.

Measure. Track what changed. Alert volume, signal quality, which detections contributed to real investigations, how the estate is trending.

Repeat. Continuously, because the environment and the threat landscape will not hold still while you catch up.

What “test” actually means

Testing a detection is not reading its logic and agreeing it looks correct. That is review, and review will not catch an unpopulated field.

Testing means at least four concrete things.

Run the detection against real historical telemetry from your own environment and see what it would have matched. This tells you both whether it fires at all and what it would have cost you in noise.

Verify the data it depends on. Is the source live? Are the fields it references still populated? Has the schema changed since the rule was written?

Simulate the attack where appropriate. Generate the behavior the rule is supposed to catch and confirm that it catches it.

Then fix the logic and run it again, because a test that does not close the loop into a change is just a report.

A detection that has been through that cycle is a different asset from one that has been written and deployed. The first you can make claims about. The second you can only hope about.

Where AI changes the economics

Most of the conversation about AI in security operations is about what happens after an alert is generated. Triage it, investigate it, determine whether it is a threat, decide what to do. That work is real and it is where the immediate pain is.

But there is an equally important opportunity before the alert, in continuously improving the quality of the detection layer itself.

The reason this has not been done historically is not that nobody thought of it. It is that the economics never worked. Assessing coverage, replaying rules against months of telemetry, checking every data dependency, evaluating each new piece of threat intelligence against the entire estate, and doing all of that on a cadence fast enough to keep up with change, is a volume of work that does not fit inside a security team’s headcount.

That is the part that changes. Not the judgment about what a detection should do, which remains a human question, but the continuous, high-volume verification work underneath it.

Which leads to a governance point worth being explicit about, because the enthusiastic version of this idea is dangerous. A system that observes an environment and silently rewrites production detections will eventually suppress something real, and you will discover it during an incident. The right design produces a proposed change with the evidence attached, and a human approves anything that reduces coverage. Automation earns scope by being right repeatedly, not by being granted it at deployment.

Where to start

None of this requires buying anything to begin.

Pull a list of every detection in production and the date it last fired. The rules that have never fired are your starting inventory. For each one, answer a single question: is its data source live and are its fields populated? You will find broken rules in the first hour.

Then take your ten noisiest detections and ask how many have ever contributed to a confirmed incident. That ratio is usually uncomfortable, and it is the clearest argument you will have for treating tuning as ongoing work rather than a cleanup project.

Finally, pick one recently changed part of your environment — a migration, a new platform, a replaced tool — and check which detections referenced the old thing. That is where decay lives.

The measure that matters

Maybe the next measure of detection maturity is not how many rules you have, or even your theoretical ATT&CK coverage. Maybe it is how much of your detection estate is current, useful, and proven to work.

That number is harder to produce than a rule count, and it is worth more than one.

Because even the best AI investigation needs the right signal to investigate.

A detection isn't valuable because it exists.
It's valuable because you can prove it works.