My Monitor Went Silent for Twenty-Two Hours. The Only Alarm Blamed My Router.

The float switch in my sump pit stuck at the end of July. Since then my monitoring software has been running the pump on a timer — two minutes on, ten minutes off — and emailing me about every single cycle. Yesterday morning it sent number 3,509.

It has not sent one since.

I did not notice for twenty-two hours. Nothing in my system noticed either. Everything I have built to catch exactly this was running the whole time.

The Gap

The last message went out at 13:15:20 UTC on August 20. The next thing my inbox got from the sump system was at 10:34:08 the following morning. Twenty-one hours and nineteen minutes, on a channel that had not missed a twelve-minute beat since July 22.

My first thought was that the mail had been eaten somewhere, so I checked the receiving end. The alerts go to a completely separate provider from the account that sends them, and the receiver shows the same hole:

2026-08-21T10:34:08Z  SUMP UNREACHABLE: Shelly not reachable right now
2026-08-20T13:15:20Z  Sump pump: TIER 3 cycle 3509      <-- 21h 19m
2026-08-20T13:03:19Z  Sump pump: TIER 3 cycle 3508
2026-08-20T12:51:17Z  Sump pump: TIER 3 cycle 3507

Sender and receiver agree. The messages were never sent.

The Explanation That Was Wrong

I had a good theory immediately, and I want to include it because I spent forty minutes on it and it was wrong.

Earlier that same morning, Google had bounced three of my outgoing messages with "You have reached a limit for sending mail." My monitor sends around 120 emails a day to two recipients each, on top of everything else my automation does from that account. Hitting a send quota was the obvious cause, and it fit the roughly-24-hour shape of the gap.

It does not survive contact with the timestamps. The bounces were at 10:25, 10:27 and 10:54. The monitor kept sending happily through all of them and for another two hours and twenty minutes after. And during the twenty-two-hour silence, that same account sent at least a dozen other automated messages — 14:55, 16:15, 19:13, 21:44, 23:01, then 00:01, 03:04, 03:39. Every other robot in the house was talking normally.

The quota event and the silence were three hours apart and unrelated. Two days in a row now I have caught myself building a confident story out of two numbers that happened to be adjacent.

The Pump Was Fine the Whole Time

Here is the part that reframed it for me. I queried the smart plug directly from a laptop on the same network, and then I sat and polled it every five seconds.

Then I let the poll run for eighteen minutes to see whether the timer was still keeping time. It is, to the second.

Both intervals land inside my five-second sampling resolution. The lifetime energy counter is climbing. The control loop never stopped. The pump is doing exactly what it did yesterday and the day before, on the same metronome. What died was the part that tells me about it.

That distinction matters more than it sounds. A machine that stops is a problem you find out about. A machine that keeps working while its reporting dies is a problem you find out about later, on someone else's schedule.

The One Alarm That Got Through

At 10:34, twenty-one hours into the silence, one message finally arrived. It was the first in this entire month-long saga addressed to my actual inbox rather than a logging address. It said:

Shelly not reachable right now. Could not read the Shelly on the LAN. Watchdog/guardian failsafe should cover prolonged loss; check Wi-Fi / DHCP if this persists.

Thirty-one minutes later I asked that plug how it was doing. It answered in twenty milliseconds. Its uptime was 30.48 days — it has not rebooted since before the float stuck. Its WiFi signal was −55 dBm. Same IP address it has always had, same MAC in my ARP table, no restart pending.

The device the alarm sent me to go check is the healthiest component in the entire system. A fifteen-dollar plug had outlasted every piece of software watching it, and the one alert I received in twenty-two hours told me to go look at my router.

Why It Said the Wrong Thing

The assessor that sent that message classifies the system into one state. It runs through its conditions in order and returns on the first match. The order is:

I built the check. It is right there, fourth. It has a threshold, a message, and a suggested remedy. And it is unreachable any time the plug read also happens to fail, because that check is first and the function returns.

At 10:34 both were true — one momentary read failure, and a monitor that had been dead quiet for twenty-one hours. My classifier reported the transient one and stopped looking.

It also cannot report two things at once, and the real state of my system right now is at least two of those four classes simultaneously. I wrote a function that returns a single verdict about a machine that is capable of failing in several ways at the same time.

The Watchdog Knew and Wrote It in a Log

This is the one that actually stung.

I have a separate guardian process whose entire job is to be the layer below the monitor. It runs independently, it enforces temperature cutoffs and maximum run times, and it can force the relay off on its own. Line 368 of that file computes exactly the number that would have caught this:

hb_age = monitor_heartbeat_age_s()

Here is every other place that variable appears in the file:

f"monitor_heartbeat_age_s={fmt(hb_age, '.0f')} "

It goes into a log line. That is all. There is no branch in that file that consumes it. The guardian has five different alert calls — overtemperature, hard overtemperature, maximum run, no recent run, never saw power — and not one of them is keyed on whether the process it is guarding is still alive.

My watchdog measured whether the monitor was breathing, wrote the answer to a log file on a machine I was not reading, and did nothing. It did that every few minutes for twenty-two hours.

And the Last Resort Was Switched Off

There is one more layer under the guardian: the plug itself can run a schedule on-device, with no computer and no network involved. That is the layer that survives everything else dying.

I asked it what schedules it had. It has two — one that turns the pump on at three points each hour, one that turns it off two minutes later. Both are marked "enable": false.

The Threshold That Could Never Fire

While I was in there, I found the reason none of this fired for the previous thirty days either.

The assessor decides whether the pump is actually moving water by comparing its power draw against a constant:

WET_WATTS = 350   # >= this while ON = pumping water

Yesterday I finally measured what this pump's two states really draw, using eight days of readings that had been sitting in a database on my own disk since March:

My "is it pumping water" threshold is 350 watts. The lowest reading the dry state has ever produced is 405. The threshold sits fifty-five watts below the floor of the failure it exists to detect. It cannot return false. In 3,509 cycles it never has.

Which is why my daily status email, on day twenty-nine of a stuck float, opened with the word OK and reported:

Pumping a lot but pump is healthy (8 min/hr)

Healthy. The same message helpfully adds that the float "has opened on its own" several times in the window, "so it is NOT stuck" — a count parsed out of the monitor's own log, while the monitor is the thing opening and closing that relay every twelve minutes. My assessor was reading my controller's actions and reporting them back to me as the machine's behavior.

The measured gap between wet and dry is empty between 567 and 713 watts. Exactly one reading out of 17,642 falls in it. The correct threshold is around 650. It is one number in one file, and I have been wrong about it for five months.

What I'd Tell You to Do Differently

The absence of data is data, and it needs the highest priority you have. Every alert I ship is triggered by an event. Nothing in my system is triggered by the absence of events. A heartbeat that has been perfect for 3,509 beats and then stops is the most informative thing my system has ever produced, and it produced it by doing nothing, and nothing was listening for nothing. If you are buying or building condition monitoring, ask one question first: what happens when it stops talking? Not what happens when it reports a problem — what happens when it reports nothing at all.

Writing the check is the easy ninety percent. I did not skip the monitor-the-monitor step. I built it twice. One implementation writes to a log with no consumer; the other sits fourth in a first-match-wins classifier behind a transient network read. Both are inert, and both look completely finished if you are reading the code rather than exercising it. Wire your checks to an action, then go break the thing on purpose and confirm the action happens.

A threshold you never validated against the actual machine is a comment, not a check. Mine had a confident comment on the same line — ">= this while ON = pumping water" — and nothing behind it but my intuition about what a pump ought to draw. The real numbers were in a file I owned the entire time. Every threshold in a monitoring proposal should carry the measurement that produced it or be honestly labeled a guess, and that includes the ones in mine.

What This Actually Cost

Nothing, yet. The pump is running, the pit is presumably still dry, the plug is sitting at 48°C, and yesterday's electricity was about thirty cents. Nobody flooded.

That is not the point. The failure is not that something broke. It is that for twenty-two hours I had no way to know whether anything had broken, and my system's own written opinion of itself during that window was OK.

Thirty days ago a ten-dollar float switch stuck, and I have spent every day since discovering that the expensive, clever part of my system was less trustworthy than the cheap part. The plug has 30.48 days of uptime. The software watching it has managed twenty-two hours of silence, one misdirected alarm, a watchdog that logs instead of acting, a disabled last resort, and a threshold that has been structurally incapable of firing since the day I wrote it.

None of that is an AI problem. No model on top of this stack could have helped, because the classifier returns before it reaches the branch that knows what is wrong.