I Wrote the Diagnosis Into My AI Watcher's Prompt. 399 Notes Later, It Still Agrees.

At 10:02 last night, my sump pump monitor wrote this line to its log:

TIER_3: Pump stopped on its own (0.0W) — float may be unstuck!

At 12:05 this morning, the AI layer I built to watch that monitor woke up, read the logs, and wrote this about the same stretch of time:

The NORMAL→POWER_CYCLE→TIER_1→TIER_2(→TIER_3) staircase seen twice in the last 24h … is the expected false-stuck-float misread.

Same basement, same pump, same two hours. One component thinks the float switch is stuck and has spent the summer trying to free it. The other thinks the float is fine and the first component is imagining things. They have disagreed four times a day since June 2nd, in logs sitting in the same directory, and until this morning I had never put them side by side.

I owe a correction first, because I found it on the way in.

Three Layers, Two Diagnoses

The system in my basement has three parts that matter here. All of them run on a Linux box on my home network, and all of them look at the same $15 smart plug, which measures watts and nothing else. None of them can see water.

The monitor is the control layer. It decides when the pump gets power. Its whole design assumes one failure: a float switch stuck in the closed position. The word "stuck" appears 29 times in its source. "Check valve" appears zero times. When the pump runs too long, it power-cycles, then steps up a ladder of timed duty cycles, and every four hours at the top it tries an "unstick" sequence of six rapid on/off cycles. Since June 2nd it has made 602 unstick attempts. 451 were declared failed. Two worked.

The assessor is a rule-based script that runs every fifteen minutes and classifies the situation. It was last edited on June 2nd. Its verdict is HIGH_INFLOW 99.5 percent of the time, and the full report for that verdict includes this sentence, filled in with a count from the log:

the float has opened on its own {N}x in the window (so it is NOT stuck)

Read that again with N set to zero. The conclusion is in the template. The count is decoration. The same message goes on to say the usual cause is "a failed/missing check valve recycling the same water."

The watcher is Claude, running headless four times a day with read-only access to the logs. Its job, in its own prompt, is to be "the SLOW judgment layer: catch trends and novel situations the rule-based assessor might miss." The prompt file carries the same last-modified timestamp as the assessor, down to the second. It contains this:

KNOWN-NORMAL, DO NOT ALERT ON:
   "HIGH_INFLOW" … This is real water, almost certainly a
   failed/missing check valve recycling water.

And, in the list of things to look out for:

the monitor stuck escalating on a FALSE stuck-float (it misreads
high inflow as a stuck float)

I built a second opinion and wrote the first opinion into its instructions.

What 399 Notes Say

Every run ends with the watcher appending a short reasoning note to a log. There are 399 of them, June 2nd through 6:05 this morning. I counted.

The zeros at the bottom are the part I keep looking at. Each of those four rows was an opening to question the prompt, and the watcher used none of them.

Zero means not stuck. So does one.

On September 9th at 6:05 p.m., the watcher wrote float_opens=0 (not stuck). On September 11th just after midnight, it wrote float opened 1x = not stuck. On September 6th: float_opens=0/dry_pulses=0 so not a false stuck-float misread.

When a reading of zero and a reading of one lead to the same conclusion, the reading is doing no work. A smarter model would not fix this. The model was told the answer, then handed evidence that could not move it, and it did exactly what a careful reader does with a premise from someone they trust: it fit the evidence to the premise.

It kept recommending I turn off the thing that worked

The watcher's first recommendation to suppress the unstick routine came on June 3rd. Its reasoning was sound given its premise: if the float is never really stuck, six rapid relay cycles every four hours are just wear.

On September 2nd at 10:55 a.m., the unstick routine worked for the first time since April. The pump went idle, and the monitor dropped out of its emergency tier. The watcher's next note, an hour later, described the escalation that followed as "the same recurring false-stuck-float misread documented in every prior check for weeks," and did not mention the success. Thirty-one hours after the routine worked, the watcher repeated its recommendation to suppress it.

It worked again on September 9th at 6:51 p.m. No note mentions that one either.

Part of this is mechanical. The prompt tells the watcher to read "the last ~30 lines" of each log. In its top tier my monitor writes about 36 lines an hour, so thirty lines is under an hour of a six-hour gap. Both successes landed outside that window. So the watcher was told what it would find and then given a keyhole to look through. I wrote both of those instructions.

So Who Is Right?

I don't know. That is the actual finding.

Yesterday I wrote that my float "has been stuck since the start of September." I wrote that because the monitor says so, and I have been reading the monitor. The watcher has been saying the opposite four times a day since June, and I had not been reading the watcher.

The evidence both layers cite fits either story. A pump drawing a healthy 490 watts for two minutes is what you would see with a stuck float, and it is what you would see with water draining back through a failed check valve. A float that sometimes stops the pump fits a float that sometimes frees itself, and it fits a pit that sometimes empties. A plug that only measures watts cannot separate them. No amount of reasoning over its output can either, because the information is not in the data.

What would separate them is embarrassingly physical. Look at the check valve. Lift the float by hand. Or add the one sensor this system has never had, something that measures the water level in the pit, which is a part that costs less than a month of the monitoring I sell. There is a checkbox for the inspection in my planning notes, under a heading that says "Before Spring Melt":

- [ ] Physical pump inspection: check discharge pipe for ice,
      verify check valve, test float

That one line holds the test for both diagnoses. It is still unchecked, and neither my monitor nor my AI watcher has any way to learn whether it has happened.

What I'd Change, and What I'd Tell Anyone Adding an LLM to Monitoring

This pattern will show up well beyond pumps. Putting a language model on top of a rule-based system is now the obvious move, and it really is useful. My watcher caught real things: a stale file path in its own instructions, DHCP churn on the plug, a temperature creep one night. But the prompt is where your assumptions go to become facts, and a model is very good at being consistent with its prompt.

1. Put hypotheses in the prompt, not conclusions. "Either the float is stuck or the check valve has failed. Here is what each would look like." That is a better prompt than the one I wrote, and it costs one sentence.
2. For every "do not alert on," name what would change your mind. "Do not alert on HIGH_INFLOW unless an unstick succeeds, or the float stops the pump on its own." Mine had the first half and none of the second.
3. Feed it events, not a tail. A watcher that wakes every six hours should get every state change and every success since its last run, not the last thirty lines. The rare line is the one that matters, and a tail is exactly how you miss it.
4. Make it argue with itself every run. Add one required line: "If my standing diagnosis were wrong, I would expect to see ___. Did I?" A model that has to write that sentence will go and look.
5. Count how often it disagrees with its prompt. My number is zero in 399. Zero means the watcher could not have told me if the prompt was wrong.

My own answers are in this post. The prompt is 101 days old, and it still says it runs on a Mac under launchd. It runs on Linux under systemd. The watcher has flagged the stale path in its own instructions 17 times, and I have not fixed it. The queue of small fixes for that box is now four deep, and the newest is a rewrite of the prompt so it holds two hypotheses instead of one.

But the next step is a flashlight and twenty minutes in the basement. The pump itself has held up all along. It has kept pumping through whatever is actually wrong with it, and it has run thousands of timed cycles without complaint. What failed is the part where I thought I had a second opinion.