The Clock Doesn't Pause for My Sump Pump

A group at Stanford's Digital Economy Lab published a benchmark last month with a plain question in the title: can frontier models solve real-time interactive problems? Jack Clark picked it up in Import AI this week, which is where I read it. The setup is fifty tasks, thirty of them on live websites and twenty in games, chosen because a late action changes the outcome. A train ticket sells out. A piece lands. A timer runs down.

The clever part is that they ran every task under three different clocks.

• Wall clock. The world keeps moving while the model thinks. However long the model takes to answer, that is how much time has passed when it acts.
• Paused clock. The world freezes while the model thinks. This is how almost every agent demo you have seen was run, whether or not anyone said so.
• Token clock. Time advances with every word the model generates, so a long chain of reasoning costs time whether the server is fast or slow.

Under the paused clock, the strongest models solve roughly 70 to 80 percent of the tasks, by Import AI's reading of the results. Under the wall clock, the best model succeeds 44 percent of the time, and only 36 percent if you ask it to succeed three runs in a row. Same tasks, same models. The only thing that changed is whether the world waited.

The paper's authors put one more sentence in the abstract that I want to keep: giving the model more time to reason has "mixed effects under time pressure." Sometimes the smarter answer arrives after it stopped mattering.

The clock in my basement

I have been running a sump pump under software control for about nine months. The pump sits in a pit below my house, switched by a smart plug, and a monitor program on a small computer upstairs decides when the plug is on. If the pump runs dry it burns itself out at about 500 watts. If it does not run when the water comes, I have a flooded basement. Both of those are wall-clock problems. The water does not pause while anything thinks.

So the control loop was never given to a model. It is a few hundred lines of ordinary Python that reads the plug's power draw every 30 seconds, every 5 seconds when something is happening, and applies rules I wrote down: if the pump has been drawing power for three minutes straight, it is probably running dry, cut it. If the plug has been unreachable for 30 minutes, lock out. Every four hours, try to shake a stuck float with six quick cycles. No rule takes longer than a few milliseconds to evaluate.

There is a model in the system. It runs every six hours, at five past midnight, six, noon and six, with a hard 300-second kill and a fifty-cent budget. It reads the last hour or so of the log and writes an opinion about what the trend means. It has run 615 times. The last six runs took between 22 and 40 seconds each. And here is the point: nothing in the house waits for it. The relay is not in its hands. By the Stanford vocabulary, the model in my basement runs on a paused clock, by construction, because I put it in the only place where pausing the clock is honest.

Three minutes on October 6

I want to show what a wall-clock loop actually looks like when it is working, because it is not elegant. Early on October 6 the pump got stuck running. Two programs were watching it: the monitor, which owns the control logic, and a guardian, a second independent program whose only job is to cut the relay if the pump runs too long. Here is the next three minutes, from both logs, merged.

Four relay flips in three minutes. Every response landed within one second of the thing it was responding to. Two of those responses were the monitor undoing the guardian, because the monitor did not know a guardian existed and read a relay going off as a fault. At 01:16:23 the monitor concluded the pump could not be trusted to stop on its own and moved to a fixed schedule: two minutes on, ten minutes off, around the clock. It has been on that schedule for 28 days, 3,359 cycles as of this morning, and the float has been stuck the whole time. That is a separate story, and I have told parts of it.

What I want you to see is the speed. A one-second response, repeated, from two programs that cost nothing to run and never need a network connection to a data center. That is what the Stanford paper is measuring the absence of. Under the wall clock, the best model in the world could not have matched 01:13:20. It would still have been reading the situation.

Fast is not the same as right

I do not want to pretend that table is a victory. It shows two deterministic programs reversing each other within a second, for three minutes, on a motor that was running dry the whole time. Speed was never the problem that night. Ownership was. Two programs could write to one relay, and neither knew about the other.

The fix is a rule, and a dull one: one program owns the actuator and the other one files reports. The guardian should have logged "I would have cut this" and sent the message, and the monitor should have been the only thing touching the relay. I have known this since I read the logs in September and I have not shipped it, which goes on this week's list.

But notice that the fix is also a wall-clock decision. "Who may act, and how long may they take to decide" is the whole design. The Stanford paper's three clocks are a vocabulary for that design, and it is the same vocabulary whether the actor is a frontier model or 50 lines of Python.

Where the model goes

If you are building monitoring for a building, a pump room, a boiler, a compressor, the question to ask of every loop is: does the world keep moving while this step thinks?

• If yes, the step is deterministic code with a bounded poll and a safe default when it is late. The relay, the valve, the shutoff, the alarm that has to fire within the minute. The model never holds the lock on these. A 30-second poll is already a real-time budget; write it down as one, with the consequence of being late next to it.
• If no, the model is welcome. The six-hourly "is this trend healthy" pass. The daily digest. The post-mortem. The diagnosis that reads a month of logs and says "your risk score is parsing the wrong log format." My watcher found a real bug in my code 26 days before I did, and that was useful precisely because nothing was waiting for it. The failure in that story was that nobody read what it wrote, which is a routing problem, not a latency problem.

The paper's "mixed effects" finding is the practical warning. If you do put a model somewhere near a wall clock, cap its thinking. My watcher's 300-second kill and fifty-cent budget are the right instinct for the wrong reason: I set them to control cost. They also guarantee that the model's slowest possible day is still a paused-clock day for the pump.