The Cuts Were Never Undone. I Published That They Were.

Last night between 6:43 and 8:10 p.m., the safety backstop on my sump pump cut power to the pump five times. Each time, the main monitor saw the relay go off, logged a warning, and switched it back on in the same second. On September 17 I wrote about exactly this pattern and called it a fight that the monitor wins.

Here is the first round from last night. The two logs are merged by timestamp, because neither one tells the whole story on its own:

[18:43:28] guardian  relay OFF: pump exceeded guardian max continuous run
[18:43:28] monitor   WARNING: Plug output is OFF unexpectedly! Turning back ON.
[18:43:28] monitor   Plug turned ON
[18:43:35] monitor   Pump stopped after 4.7 min (0.0W, 50.7C)
[18:45:28] guardian  forced_rest_remaining  output=True  power=0.0W
[18:50:36] monitor   Pump started running (492.5W)

Read the fourth line carefully. Seven seconds after the relay came back on, the monitor logged that the pump "stopped after 4.7 min," and that is the same line it writes when the float switch ends a run. Two minutes later the backstop took its own reading: relay on, zero watts. The pump did not start again for seven minutes.

So the relay was restored, but the run was not. The run ended when the power dropped, and it stayed ended. It happened again at 6:55, 7:27 and 7:47. The stops came 8, 6 and 6 seconds after the cuts, and the pump then sat idle for 17, 8 and 16 minutes.

Checking the whole history

One evening proves nothing, so I went back through every backstop cut since the backstop was installed in June. There are 70. The monitor reversed 43 of them. For 30 of those 43, the backstop logged a reading about two minutes later, and that reading shows what was actually going on in the basement.

In zero of thirty measured cases was the pump running two minutes after the monitor "undid" the cut. In fifteen of them the relay was on and the pump was sitting at zero watts. The monitor was reporting that it had won.

That changes what the September 17 post was actually counting. It was right that the relay went back on. It was wrong about what that meant, because I measured the command and assumed the effect.

The part that is worse than a wrong sentence

The September 17 post also made a calibration argument. I said the backstop's four-minute limit was badly placed, because since September 1 the 90th percentile of normal, float-ended pump runs was 4.1 minutes: "the guardian's line sits right on it."

The monitor logs every float-ended run with the same line, Pump stopped after N min. I took all of those lines as normal runs. The trouble is that the monitor writes that line whenever the power reading falls to zero while the relay is on. It has no way of knowing why the power fell. When the backstop cut the pump and the monitor restored the relay a second later, the monitor saw a running pump, then a relay that was on and a pump at zero watts. From where it sits, that looks exactly like the float doing its job.

Separate the runs that ended within a minute of a backstop cut, and the distribution I used to judge the backstop falls apart:

The whole long tail came from the backstop. Remove the runs it ended, and the 90th percentile of float-ended runs drops from 4.1 minutes to 1.7. The four-minute limit is not "right on" normal behavior. It is about two and a half times the 90th percentile.

So the September 17 argument measured the backstop against a baseline that the backstop had helped create. Every cut it made added a long run to the data I then used to judge where it should cut.

What it says about the pump

For two weeks I have been working on a question I could not answer from watts: is the float switch sticking, is water falling back through a bad check valve, or is there really that much water coming in? Last night adds a piece.

Nearly no rain fell. The weather archive shows 0.4 mm for all of September 26 and zero every hour of the evening. My system classified the evening as "wet" because 10 mm is forecast for noon today, and its wet check counts the next 24 hours of forecast. So the pump ran 4 to 6 minutes at a time, repeatedly, with no rain falling. Whenever the power was interrupted for under a second, the pump stayed off for 7 to 17 minutes afterward.

If water had truly been pouring into the pit, cutting power would have left the float up and the pump would have restarted the moment the relay closed. If the check valve were leaking, stopping the pump would have sent the pipe's water back into the pit and restarted it even faster. Neither happened. What I saw fits better with something that holds the pump on while the motor runs and lets go when the power drops.

That is still an inference, and I have been wrong in public about this pump before. The monitor's own 10-second power cycle only "works" 36 times out of 120. At 8:10 last night it cycled the power and the pump came back at 501 W, and the monitor climbed its ladder back up to the top tier, where it is this morning. A sub-second interruption does not release the pump every time, and I do not know why it does when it does. That question needs someone in the basement with a flashlight watching the float during a long run while the power blinks. It does not need another query.

The rule I should have been following

Twelve days ago, in the post about the overtemp lockout, I wrote down a rule for this system: before you analyze any actuator, ask which processes can command it. I later counted ten command paths to this one relay. I applied that rule to commands and forgot it for observations.

The version I needed is the other direction. Before you call an observed event natural, check whether any actuator fired in the seconds before it. Before you trust a baseline built from those events, remove the ones an actuator caused. The check is one join on timestamps across two logs. I skipped it for ten days and kept publishing on top of the result.

The version you can run this afternoon

If anything in your building has two controllers, such as a BMS and a local thermostat, a VFD with its own protection logic, or a generator with an auto-transfer switch and a manual override, do this:

1. Pick the event your monitoring treats as normal completion: a cycle ending, a setpoint being reached, a fault clearing on its own.
2. Pull every command from every other controller that can affect that equipment.
3. For each "natural" completion, look for a command in the preceding sixty seconds.
4. Recompute whatever baseline or threshold you derived from those completions using only the events with no command in front of them.

If the baseline moves, some of your "normal" was manufactured by your own control system, and every limit you set against it is partly measuring itself. Mine moved from 4.1 minutes to 1.7.