My Plug's WiFi Was Fine Every Time It Went Dark

Hackaday wrote up a small Wi-Fi analyzer this weekend. Roni Bandini built it from a single-board Linux computer and a USB Wi-Fi adapter. It hops through the channels, counts signal strength, clients and frames per second on each one, and hands those numbers to a small model that labels each network's coverage and congestion and suggests a fix. It is a nice build, and it made me ask a question about my own basement that I had never actually checked.

The smart plug that runs my sump pump is on Wi-Fi. My monitoring software loses contact with it a few times a week. And the software has had a Wi-Fi alarm since the first day: if the plug's signal drops below −80 dBm, it warns me that "the monitor may lose connectivity to the plug" and suggests moving the router closer or adding an extender.

So, eight months in: is weak signal why the plug goes dark?

What the logs hold

Every ten minutes the monitor writes a heartbeat line that includes the plug's own signal reading. Since March 8 that is 29,359 readings. Separately, every time a request to the plug fails, it writes a warning. I grouped those warnings into dropouts, treating any failures less than fifteen minutes apart as one event: 295 dropouts since March. Then for each dropout I looked up the signal reading from the last heartbeat before it started.

The signal is healthy, and it was healthy before every dropout

(13 dropouts had no heartbeat before them to compare, mostly at startup.) For a smart plug, −57 dBm is a good signal. The readings just before a dropout look exactly like the readings at any other time. If weak signal were causing the dropouts, the right-hand column would lean weaker. It doesn't. Not one of the 282 started from a reading weaker than −72.

This morning's dropout, at 3:11, is a typical one: the last reading before it was −54 dBm, and fifteen minutes later everything was back to normal.

The alarm fired 47 times, on readings that didn't last

The alarm checks the signal on every poll and fires on the first reading below −80. It has done that 47 times. The readings that set it off were −81 to −88, and the next heartbeat, on average five minutes later, was back above −80 in 43 of the 47 cases, often in the −50s and −60s. Only 3 of the 47 warnings were followed by a dropout within half an hour.

So the alarm doesn't predict what it says it predicts. It reports one weak reading, one time, and moves on. It is the same pattern as my voltage alarm, which paged me six times in 55 minutes about one hour of high utility voltage: a threshold on a single sample, with no requirement that the condition last. A real problem lasts. A single bad sample is radio noise.

And the advice it gave would not have helped. Moving the router or adding an extender fixes weak signal. The plug didn't have weak signal.

There is a second problem, and it is the boring one. Since June, these warnings go to a logging address, not to my phone. Since September, the setting for that logging address has been blank, so 22 of the 47 warnings were written to the log as "skipped" and sent nowhere. No harm done this time, because the warnings were wrong. But that is luck, not design.

Then why does the plug go dark?

The line just before each failure says what the monitor actually got back. Across eight months:

• About 3,800 times: no response at all. The request went out and nothing came back. The plug was connected with a good signal a few minutes earlier, and usually a few minutes later.
• About 2,300 times: the plug wasn't on the network. The monitor scanned the whole subnet for the plug's hardware address and didn't find it. Most of these were one day, October 5, when the monitor computer lost its link to the plug's network for eighteen hours while the plug sat there the whole time, and a bad stretch in March.
• About 200 times: the plug refused the request. An authentication error, almost all in March.

Signal strength tells you about one hop, from the plug to the access point it hears. My failures are somewhere else: the plug's own small Wi-Fi chip not answering, the router, the mesh between the router and the computer, or the computer itself. A signal-strength number can't see any of those. That's where the Hackaday build is a useful hint: congestion and traffic per channel are a different measurement from signal, and they would actually tell me something about that path.

One thing I can't explain yet. On October 1 through 4, the plug had 45 readings weaker than −75 dBm, more than half of the 74 it has ever had. It wasn't raining. The weak spell ended a day before the computer lost the plug's network on October 5. I can't connect those two from what my logs record, and I'm not going to pretend I can.

What I'm changing

• Make the signal alarm require persistence. Warn only if the signal stays below the line for, say, three heartbeats in a row. On eight months of data that rule would have fired zero times: only two heartbeats ever read below −80, and never two in a row.
• Log the number that matters. For every poll: how long the plug took to answer, and why a failure failed. Response time drifting up is the early warning that signal strength was supposed to be.
• Fix the delivery path first. An alarm that is sent nowhere can't be right or wrong. The blank setting is a one-line fix and I've been carrying it for two months.

The test, for any wireless sensor in a building

1. Pull the signal readings from just before each failure, and compare them with all the other readings. If they look the same, signal isn't your cause, and moving the router won't help.
2. Check how long a "bad signal" reading lasts. If the next reading is fine, your alarm is reporting noise.
3. Read the failure reason, not just the failure. "No response," "not on the network" and "refused" are three different problems with three different fixes.