When an AI product decides what deserves your attention

Before we switched on a daily AI monitor at ThriveAI, I had to decide what would count as a good day.

The easiest answer was the number of issues it found. That would also have been a poor measure of whether the product was useful.

The system watched product analytics and user feedback, investigated changes, and briefed product teams in Slack. It could produce a steady stream of plausible observations. Some might be wrong. Some might be correct but trivial. Others might be correct and important, yet already well known to the person receiving them.

Only a small subset changed what someone understood or did.

So the quality question we set before launch was narrower: how many important issues did the system uncover that the product owner did not already know? We paired that with false-positive and false-negative rates.

Once the system could decide when to speak, it was spending the user's attention on our behalf.

Correct can still be useless

Most evaluation starts with correctness, for good reason. A system that invents problems will lose trust quickly.

Correctness was necessary for us too, but it did not capture incremental value. Telling a product leader that yesterday's release caused a regression could be valuable. Telling her the same thing after her team had already opened an incident was mostly noise.

These cases look identical if the evaluator sees only the finding. They are very different from the user's point of view.

I found it useful to separate three outcomes:

1. The system is wrong.
2. The system is right, but the finding is already known or does not matter.
3. The system is right and changes what the recipient knows or prioritizes.

The third outcome was the product we were trying to build. The first damaged trust. The second quietly consumed attention without earning it.

Two findings, two kinds of value

In one mobile product, the weekly review found that new users had stopped completing a core booking step after a release. It narrowed the break to the part of the journey after account verification and found a corresponding rise in error events. That gave the team a specific place to investigate. It did not prove that the release caused the failure, so we kept the causal link as a hypothesis.

In another case, a batch of support feedback showed that scheduling complaints were clustering around failures in an external service and stale availability data. The individual tickets were already known to support. The useful work was connecting them into a pattern and turning that pattern into a product question: how should the experience degrade when a critical provider fails?

I would not count those findings in the same way. The first was a focused investigation lead. The second was synthesis across issues the organization had already seen. Both could be useful, but only one might qualify as something the recipient did not already know.

This distinction matters for many proactive AI products. An assistant that waits for a question can be judged largely on the answer it gives. A system that decides when to speak is also making a judgment about interruption.

Every alert spends a little attention. The product has to earn that spend.

False alarms are visible. Misses hide.

False positives were relatively easy to discuss. A product owner could look at a finding and tell us that the system had overreacted, misunderstood the data, or lacked context.

False negatives were harder. The system does not produce an artifact for the anomaly it failed to notice.

That creates a predictable evaluation trap. Teams review what the AI surfaced, improve precision, and gradually reward the system for becoming quieter. A cautious system can look excellent while missing the reason it exists.

We made missed anomalies part of the feedback contract at launch. The questions were simple:

• Was this finding helpful or significant?
• Did the AI miss an anomaly it should have caught?
• What context or detail was missing?

The second question mattered as much as the first. It created a place for absent findings in the product review, even though measuring them systematically remained difficult.

The third question served a different purpose. A thumbs up or down can grade the previous output. Missing-context feedback can improve the next judgment. It tells you whether the system lacks product knowledge, recent decisions, business context, or a signal it cannot currently observe.

The quality contract

If I were writing the contract today, I would describe it in four parts.

Significance. Is the issue large or consequential enough to deserve attention?

Novelty. Is it new to the person receiving it?

Precision. How often does the system raise something that is wrong?

Coverage. Which meaningful anomalies did it fail to surface?

There is no single clean metric across all four. Significance depends on the product and the moment. Novelty depends on what a particular person already knows. Precision can be improved by suppressing uncertain findings, sometimes at the expense of coverage. Misses require an independent view of what happened, which is often incomplete.

I would be suspicious of any single score that claimed to resolve those tradeoffs. An output count certainly could not.

It also changes product decisions upstream. A system evaluated on volume will learn to produce more. A system evaluated only on precision will learn to take fewer risks. A system evaluated on useful discovery has to get better at context, prioritization, and restraint.

What this framework does not prove

We launched the daily pipeline with this feedback contract. The record supports the decision and the launch. It does not, by itself, prove that we solved proactive AI evaluation or that the system delivered a particular long-term outcome.

Several hard questions remained.

How do you establish a reliable denominator for missed anomalies? How do you distinguish genuinely new information from something the recipient learned five minutes earlier? How should the threshold change during a major launch, when attention is already stretched but the cost of a miss is higher?

Those questions became part of operating the product. They could not be settled once in a metric definition.

The question I now ask first is simple: what makes this worth the interruption?

For a proactive AI product, that belongs in the quality bar from the start.