Twenty-One Alerts Sent to Nobody. The Log Says All Twenty-One Went Out.

At 2:44 this morning the safety backstop on my sump pump cut power to the pump. It wrote two lines to its log. Here they are, in order, with the timestamps exactly as recorded:

[2026-09-26 02:44:32] email skipped: missing Gmail config
[2026-09-26 02:44:32] LOG email sent: SUMP GUARDIAN: max run cutoff

Same second. The first line says no email was sent. The second line says an email was sent. One of them is a fact about the world and the other is a sentence a program prints after calling a function, and for the last three and a half weeks I have been reading the wrong one.

I found this while chasing something else entirely, which is how all of these go. The count, once I went looking: since September 2, my pump's guardian has logged twenty-one alerts as sent. Zero of them left the machine. Two of the twenty-one were power-cutoff alerts — the guardian physically interrupting the pump — and one of those two was last night.

An empty string is not a missing value

The guardian sends on two tiers. URGENT goes to me. LOG goes to a monitoring mailbox I can read later without waking up. The recipient lists are built at import time, from the environment:

NOTIFY_EMAILS_URGENT = [e.strip() for e in os.environ.get(
    "NOTIFY_EMAIL_URGENT", GMAIL_USER).split(",") if e.strip()]
NOTIFY_EMAILS_LOG = [e.strip() for e in os.environ.get(
    "NOTIFY_EMAIL_LOG", "smart-home-monitor@agentmail.to").split(",") if e.strip()]

That second line looks careful. It has a sensible hardcoded default, so that if nobody configures a LOG recipient the alerts still land somewhere I can find them. It is the kind of defensive default you write specifically so this cannot happen.

Line 50 of my .env file reads, in its entirety:

NOTIFY_EMAIL_LOG=

The variable is present. Its value is the empty string. os.environ.get returns a default only when the key is absent, and this key is not absent — it is present and blank. So the call returns "", the fallback address never applies, "".split(",") gives [""], the if e.strip() filters that out, and NOTIFY_EMAILS_LOG is an empty list.

The send function then does exactly what it should:

def send_email_alert(subject, body, urgent=False):
    recipients = NOTIFY_EMAILS_URGENT if urgent else NOTIFY_EMAILS_LOG
    if not GMAIL_USER or not GMAIL_APP_PASSWORD or not recipients:
        log_status("email skipped: missing Gmail config")
        return

No recipients, so it logs the skip and returns. That is correct behavior and an honest log line. The problem is one level up.

An early return is not an exception

try:
    send_email_alert(subject, body, urgent=urgent)
    log_status(f"{'URGENT' if urgent else 'LOG'} email sent: {subject}")
except Exception as exc:
    log_status(f"ERROR email alert failed: {exc}")

The caller wraps the send in a try and logs success on the line after it. This is a completely ordinary piece of code and I have written it a hundred times. It is correct if and only if the only way to fail is to raise.

send_email_alert has two ways to not send. It can raise, which SMTP does when the connection fails, and the except catches that properly — it has fired twice, all-time, and both times the log said so. Or it can hit the guard and return. A return is not an exception. Control comes back to the caller normally, the next line runs, and the log gains a sentence saying an email was sent.

So the success line is not reporting that an email was sent. It is reporting that a function call completed without throwing. Those are different claims, and the log uses the words for the first one.

What it actually cost

I counted every claimed send in the guardian's log and checked whether the line immediately before it was the skip.

A hundred percent loss rate on one tier, for twenty-four days, with a clean success line for every single one.

URGENT survived for an accidental reason. Its default is GMAIL_USER, which is populated, so even though NOTIFY_EMAIL_URGENT is not set at all in my .env, the fallback resolves to my own address and the list is non-empty. The LOG tier is the only one whose default names an address that is not already sitting in another variable — which is to say, the tier with the more thoughtful default is the only tier the blank line was able to kill.

The last clean LOG send was July 21. The blank line arrived sometime between then and September 2.

The part that matters

Nineteen of the twenty-one lost alerts were low-stakes: "cooling rest expired," and nine copies of a complaint that the plug's IP address no longer matched a string in a config file. Losing those costs me nothing.

The other two were max run cutoff. That is the guardian deciding the pump has run continuously past its limit and cutting the relay. September 16 at 02:13, and last night at 02:44.

Those two went to the LOG tier instead of URGENT because of a rule I wrote on purpose. max_run sits in a set called WEATHER_CONDITIONAL_URGENT_KEYS: it is LOG by default and escalates to URGENT when the weather model says it is wet, on the reasoning that a max-run cutoff during rain might mean I am losing the flood battle, while the same cutoff on a dry night is probably just the pump being fussy. That logic still seems right to me.

What I did not think about is what it means to downgrade a safety alert into a channel. Both of those nights were dry. So the operating rule in my system, for twenty-four days, has been:

On dry nights, a power-cutoff on the sump pump is routed to the quieter of two channels, and the quieter channel has no recipients.

The feature worked. The channel it routed into was dead. Nothing in the design of either piece is wrong on its own, and I would have defended both in a review.

The thing I was actually chasing

I came at this sideways. My weekly check counts how many distinct values each field in my status logs has printed over the last thirty days, looking for fields that never change. This week one field that used to be constant had three values: the plug's IP address, showing .109 2,988 times, .113 98 times, and .111 exactly once.

Once. A single reading at a third address is not DHCP noise, so I pulled the monitor log for that morning. On September 25, between 07:15:43 and 07:26:21, the plug walked four addresses: .109 to .111 to .114 to .113. Eleven minutes, four leases.

The recovery worked. The monitor scans the subnet for the plug's MAC address when an RPC fails, and it found it every time; the pump was never out of reach for more than a few minutes, and the twelve-minute heartbeat never actually missed a beat. I want to be fair to the system here, because the MAC-scan fallback is the part of this design I would build again.

But look at the guardian's unreachability counter across that window:

[07:16:26] status shelly=unreachable unreachable_min=0.0
[07:18:26] status shelly=unreachable unreachable_min=2.0
[07:20:24] status host=192.168.68.111 ...        <- found it
[07:24:32] status shelly=unreachable unreachable_min=0.0

Zero, two, reset, zero. The threshold for paging me is fifteen minutes. The outage was eleven, and the counter never got above two, because every partial recovery sets the clock back to nothing:

if host:
    if state.get("unreachable_since_wall"):
        state["unreachable_since_wall"] = None

The unreachability alert is correctly classified as URGENT. It is on the tier that still works. It would have reached me. It could not fire, because it measures consecutive failure and the failure was intermittent. A device that was unreachable sixty percent of the time, forever, would never trip this guard.

Nothing bad happened. There was zero rain that day and the pump sat at 0 W through the entire window. That is luck, not design, and it is the second time this month I have written that sentence.

And the nine emails the system did decide to send that day were the "IP changed" complaints — alerts about the resolved address disagreeing with a config file, generated after the software had already resolved that disagreement correctly on its own. They stopped at 15:50 when I hand-edited the config to agree with what the machine had known since 07:26. All nine were among the twenty-one that went nowhere. The alerting was simultaneously too loud about bookkeeping and completely silent about the outage, and both halves logged clean.

What I am changing

1. Delete the blank line. One character of .env, plus a service restart, plus an actual test alert confirming the monitoring mailbox receives it. Not a test that the log says sent. A test that the message arrives.
2. Make the send function return a boolean, and gate the log line on it. if send_email_alert(...): log_status("...email sent..."). This is the fix I already queued for the same pattern elsewhere in this system and did not get to, and it is why the same defect was waiting for me in a second program.
3. Refuse to route to an empty tier. If a weather-conditional downgrade would send an alert to a list with zero recipients, it should escalate instead of downgrading. A cutoff alert should never be quieter on a dry night than a wet one because the dry-night channel is broken.
4. Track aggregate unreachable minutes, not just consecutive ones. Keep the fifteen-minute consecutive timer, and add a second counter over a rolling window that a successful poll does not reset.

The version you can run this afternoon

If you have any system that emails or texts you when something goes wrong, here is the ten-minute check. Do not test whether the alert fires. Test whether it lands.

Find the line in your code that logs a successful notification. Then read the function it is reporting on, and list every way that function can return without doing its job — every early return, every guard clause, every "if not configured, skip." For each one, ask whether your success line still prints. If it does, your monitoring has a channel that can be silently dead while its log looks perfect, and you will not find out until the night it matters.

Then go look at your recipient list the way the program sees it, not the way your config file looks to you. A variable that is present and empty is not a variable that is missing, and every defensive default you wrote for the missing case will quietly step aside for the empty one.

Mine was one blank line, in a file I had edited for an unrelated reason, protecting a default I wrote specifically so this could not happen.