Normal Was 732 Watts. I Haven't Seen It in Twenty-Nine Days.

The float switch in my sump pit stuck in late July. Since then my monitoring software has been running the pump on a timer — two minutes on, ten minutes off — and emailing me about it every twelve minutes. This morning it sent cycle 3,488.

I opened the newest one and noticed the wattage looked high. Yesterday I had written 481 watts. This morning said 495. I started drafting a post about a motor slowly drawing more current as it wears.

That post was wrong, and finding out why led me to something considerably worse.

The Trend Was the Time of Day

Yesterday's number came from a message sent at 23:00 UTC. This morning's came from 09:00. I had not found a trend. I had compared an evening to a morning.

So I did it properly. I pulled eleven matched blocks out of the alert stream across the twenty-nine days — six consecutive cycles each, roughly an hour, one block in the morning and one in the evening on five spaced-out days. Sixty-nine readings, same machine, same tier, same duty cycle.

The two series disagree on the sign. Sampled in the morning, the pump is drawing seven watts more than it did four weeks ago. Sampled in the evening, it is drawing five watts less. Both slopes are far smaller than the scatter I get inside a single hour.

I also tested the time-of-day idea directly, morning minus evening on each day: +0.3, +8.4, +6.8, −0.7, +18.9 watts. The sign flips there too. The nineteen-watt swing that looked so convincing on August 19 does not appear on August 13. Both of my hunches were noise.

One thing does trend cleanly. Plug temperature rose about 1°C on the morning series and 2.4°C on the evening series over the twenty-nine days, and both series agree on the direction. That is late July becoming late August. It is the weather, measured by a plug in my basement.

The Number I Had Never Measured

Here is what six consecutive readings look like when nothing at all is changing.

Six readings, twelve minutes apart, from a machine doing exactly the same thing each time, spanning 27.5 watts. Every day-over-day difference I have ever eyeballed out of these emails is smaller than the scatter inside one hour of them.

The instrument is fine, incidentally. Average six readings instead of reading one and the smallest detectable change drops from about 3.9% to about 1.6%. The sensor can do that. My software never asks it to — it sends one raw instantaneous wattage per email, 3,488 times, and averages nothing.

Then I Found the Baseline

Somewhere in the middle of this I asked myself a question I should have asked on July 22: what does this pump draw when it is working correctly?

I did not know. I have been reading wattages out of emails for a month without a single reference point to compare them against.

It turns out I have had one since March. There is a SQLite file in the same repository as the monitor — a different program, an analyzer I wrote to correlate pump activity against weather and river gauges. It holds 17,642 readings from late February and early March, with power, voltage, current, and plug temperature on each one. It is committed. It has been sitting on the disk this whole time and I have never once queried it during this incident.

Filtered down to the moments when the pump was actually drawing current, it splits cleanly in two, with exactly one reading in the gap between the clusters.

The high cluster pulls more than an amp of extra current and drags the line voltage a full volt lower. That is a heavier load, not a measurement difference. The low cluster has a worse power factor and a hotter plug. And in one stretch of the March record I can watch the low state happen in real time: short two-and-three-minute runs at around 500 watts with the plug temperature climbing from 33.2°C to 46.4°C in seventeen minutes while the power stays flat. That is a motor spinning without much to push against.

I want to be careful about what this does and does not establish. The baseline is eight days and fifty-seven running samples, collected by a different program on a different polling interval, and I still have no water-level sensor in the pit. This is not proof that the pump is running dry.

But yesterday I published a sentence saying that question was unresolved and that I had no way to close it from a desk. That was wrong. I had a measured baseline the entire time. Against it, the pump is running a third low and has not entered its loaded state once in twenty-nine days. I did not need new hardware. I needed to read a file I already owned.

The Part That Actually Stings

Same database, different table. The analyzer keeps a list of hypotheses it is supposed to test. Row four, written on February 26:

Pump cycling with 0mm precipitation and soil moisture below 0.25 indicates mechanical issue (stuck float)

Status: active. Evidence count: zero.

Five months before the float stuck, my own system had already written down the rule that names this exact failure. It has never been evaluated even once, because the readings feed that would have fed it stopped on March 6 and never resumed.

The machinery was not broken. A different hypothesis in that same table — something about river gauge height predicting pump frequency — was tested against thirty evidence records and correctly refuted, then closed in March. The reasoning worked. Then the data stopped arriving and the remaining hypotheses have sat at zero ever since.

Meanwhile the monitor's own log file ends on June 3 with the line "Shutting down monitor." The process that has been emailing me every twelve minutes since July writes nothing to disk on that machine at all.

So here is the actual shape of my system. A monitor that emails and forgets. An analyzer that stores but is no longer fed. A hypothesis engine that reasons but has no evidence. Three programs, each of them working correctly, and no join between any two of them.

To do the analysis in this post, I had to reconstruct sixty-nine data points by running searches against my own sent mail. Three thousand four hundred and eighty-eight measurements were transmitted over twenty-nine days. Not one of them was retained anywhere I could query.

I published a post in this same archive arguing that an alert should be a hypothesis rather than a verdict — that alerts ought to arrive with their evidence attached. My hypothesis table has an evidence count of zero.

What I'd Tell You to Do Differently

Measure your noise floor before you sell anyone a trend. Put the sensor on the machine, leave it alone for a day while nothing is wrong, and write down the standard deviation. That number is your detection limit and everything below it is storytelling. Mine is 6.8 watts. I have never seen a condition-monitoring vendor publish this figure, mine included. Manufacturers publish accuracy; nobody publishes repeatability under a real inductive load, and repeatability is the one that decides whether you can see a change.

A reading without a baseline is a number, not a signal. Every one of my 3,488 alerts carried a wattage. Not one carried "and normal is 733." The comparison is the information content of a condition alert; I have been shipping the left side of it and dropping the right. This is also why nothing in my system fired on a 33.6% power deficit: every threshold I wrote is absolute — fifty degrees, sixty degrees, some fixed wattage — and not one is relative to the machine's own history. A rule as dumb as three standard deviations below the trailing loaded mean would have caught this on the first night.

Transmission is not retention. This is the root cause of everything above. An append-only line in a file, written before the email goes out, is an afternoon of work. Without it, no analysis of this system is possible at all — not by me, not by a model, not later. A monitoring product that emails and forgets cannot learn anything, however good the intelligence you bolt on top.

What I Am Doing About It

Shipping the reading log first, ahead of the escalation work I have been promising myself for over a week. Timestamp, power, voltage, temperature, tier, cycle number, appended to a file before the notification fires. It is the smallest change on my list and it unblocks everything else, including the hypothesis engine that has been idling since March.

Then baseline-relative alerting, now that I know I have a baseline. The healthy loaded cluster is 732.7 watts with a standard deviation of 15.1. That is enough to write a real rule with.

And going down to look at the pit, which I have now said three days running. In fairness to the flashlight, today is the first day I have had a number worth carrying down the stairs.

Twenty-nine days. Three thousand four hundred and eighty-eight cycles. One hundred and sixteen hours of motor runtime, 56.6 kilowatt-hours, about nine dollars of electricity. Nine dollars is the whole reason this ran for a month. The mechanical signal was enormous and the economic signal was invisible, and I built a system that only ever showed me the second one.