When the agent works alone at night
For a few months now, an agent has been running around the clock at my place. Not a daemon I wrote myself, but an LLM that works through scheduled tasks: a morning briefing, a digest of YouTrack issues, an evening security check. The agent is my assistant – and at first I made the mistake of treating it like a cron job that just runs.
An agent is not a cron job
A classic cron job has two states: ran or didn't run. The exit code tells you whether it worked. An LLM agent has a third, more insidious state: it ran, produced an answer, and the answer is still wrong – it just looks right at first glance.
The first case that woke me up was a report that went missing every morning. No error in the log, no broken cron – the job ran through and simply produced nothing usable. The fault wasn't in the code, but in the behavior: a rate-limit message the agent simply accepted instead of trying again.
That's when I formulated the first rule: an agent needs different error handling than a script. A script may abort on an error. An agent has to distinguish between an error that resolves itself – and one that will never go away without my intervention.
Two kinds of errors
That distinction is the core of it. I split errors into two classes:
- Self-healing. Rate limits, short timeouts, a service that is briefly unreachable. The right reaction here is to retry quietly. Wait a few minutes, try again. That doesn't need me.
- Manual. Expired tokens, broken authentication, an API that changed its structure. No retry helps here. If I work around it silently, the error disappears from view for good – the job "runs", but has been delivering nothing for weeks.
The difference sounds trivial, but it isn't. An agent that reports every error loudly annoys you with rate limits until you ignore it. An agent that swallows every error silently dies without a sound, and you only notice when you actually need the result.
Concretely, that means: self-healing errors get retried automatically, without me. Manual errors reach me – as a notification, not as a silent gap.
Measure first, optimize later
The second mistake was typical for me: I let too many jobs run too often before I knew what they cost.
An LLM agent consumes tokens. Every run costs – and for an agent that checks something every fifteen minutes, it adds up faster than you'd think. When I started tracking consumption, I saw at a glance that a single job was eating almost half of all tokens. Not because it was complicated, but because it simply ran too often.
The solution wasn't technical, but tactical: question the cadence. Does this job really need a run every fifteen minutes, or is once an hour enough? In most cases, less was enough. Consumption dropped without me missing anything.
It's the same lesson as with monitoring and backups: measure first, optimize later. You can't optimize what you can't see. For me, that runs through a few small scripts that record token consumption per job – nothing elaborate, but enough to know who the biggest consumer is.
The rules I put in place
By now a small set of rules has settled in. Nothing spectacular, but it prevents most incidents before they happen:
- Every job reports. Success or failure – the result lands in a channel I actually read. A job that stays silent is a job I don't trust.
- Success has to prove itself. "Ran" is not the same as "done". Important jobs verify their own result – does the report really exist, is the entry really in the target system?
- Retries are finite. A retry that runs forever is not a retry, it's a hidden error. After a handful of attempts there's a notification instead of a silent abort.
- Rare is better than frequent. Before I let a job run more often, I ask whether the benefit justifies the cost. Usually it doesn't.
The last point matters most. It's tempting to check everything constantly – but an agent that spams you every hour with trivialities gets ignored just as surely as one that stays silent.
What I take away from it
An autonomous agent is not a tool you set up and forget. It's an employee you have to manage – with clear rules about what it may decide on its own and what it must report. The difference between a useful agent and one that quietly works away and gets it wrong is not model quality. It's supervision.
And the core message is the same as always: the best agent is not the one with the most features, but the one you actually keep an eye on. An agent you don't watch is worse than none – because it gives you the feeling that something is running while, in truth, it's only pretending to.