Twelve Months Later: What Became of Our First Automations

Of the fourteen automations we put into production twelve months ago, eight are still running. Four are switched off, two were rebuilt from scratch.

Everybody writes about go-live. Almost nobody writes about the state of things a year later. That year is exactly what decides whether the whole thing paid off.

This article is the review for our own brands. What still runs, what we buried, what maintenance costs, and which of our assumptions back then turned out to be wrong.

What still runs, unchanged

Eight automations are live. Five of them I haven't touched in over half a year.

They are the most boring ones. Hand the order to the fulfiller. Write the tracking number back. Send the shipping notification to the customer. Reconcile inventory. Post the weekly report to Slack.

The common thread is obvious: these flows hang off interfaces that rarely change, and they make no decisions. They move data from A to B. Transport barely ages.

Order handling at mate still sits at roughly 15 minutes a day instead of the previous 4 hours. That number stayed stable across the year, even as order volume grew. That was the biggest positive surprise: the effort does not grow with the volume.

What we switched off

Four automations are off. Each for a different reason.

The social media poster. It distributed product content to two channels automatically. Technically it ran without a single error for a year. We switched it off because the posts performed measurably worse than the handmade ones. It worked and still did damage.

The review follow-up. The flow was correct, the yield too small. At two percent response rate, the maintenance is not worth it.

The automatic goodwill decision. We had built a rule that approved small refunds on its own. After four months we put it back behind a manual approval, because the edge cases kept piling up. No damage was done, but the uneasy feeling stayed. Where an approval belongs is covered in Human in the loop.

A marketplace sync. That channel no longer exists for us. The automation died with the channel, through no fault of its own.

Four out of fourteen sounds like a lot. I see it differently now: a rate of zero switched-off automations would mean we never tried anything new.

What had to be rebuilt

Two automations we replaced from the ground up.

The first was support triage. The original version worked with fixed keyword rules. It was good enough until the rule list grew to 60 entries and nobody knew any more which rule did what. The new version uses a language model and a rule set of eight lines. Today it answers around 65 percent of requests automatically. "Where is my parcel?" alone accounts for about 40 percent of all requests.

The second was inventory sync. The original setup let every system write. That produced numbers we stopped trusting. Today exactly one system owns the value, everything else reads it.

Both rebuilds had the same trigger: the original design was right for the size we were then and wrong for the size we are now. That's not a construction error. It's the price of having started early.

Maintenance: the section nobody writes

Here are the numbers from twelve months of our own operations.

That averages out to half a working day per month for fourteen, later eight automations. Against 33 to 46 hours saved per week, that's a good deal. The point is that the number is not zero.

Where the failures came from, broken down across the year:

• Interfaces changed. The most common cause, a good half of all cases. A provider changes a field, our flow doesn't know about it.
• Credentials expired. Twice in the year, both times with downtime, because nobody noticed straight away.
• Our own changes. We reworked a process and forgot an automation was attached to it.
• Real bugs in the build. The rarest cause. What survives the first month usually survives the year.

The most important insight from that list: nearly all failures came from outside or from us. The code was rarely at fault. Which is why more testing helps less than you would expect, and a watchdog that reports silence helps more than you would expect.

Four assumptions from back then that were wrong

"Build it once and it runs." Automations age because their environment ages. Budget around 10 to 20 percent of the build time per flow as annual maintenance. Skip that and you are flattering the project's numbers. A breakdown of the costs is in What automation really costs.

"The most elaborate automation delivers the most." The flow with the biggest payoff across the year was the weekly report. Build time: one day. The most complex flow took four weeks and sits idle today.

"We'll notice when something breaks." We failed to notice twice, once for eleven days. Automations fail quietly. I wrote up these and other mishaps in Our biggest automation fails.

"More automation is better." Four shutdowns later I see it differently. Every automation is an object that wants care. Past a certain count you maintain more than you gain. Where that line runs is covered in When automation makes no sense.

What I would build differently today

Three things, concretely.

I would build every automation with a heartbeat from day one. A small note on every run, plus a second flow that reports when the note fails to arrive. Effort: ten minutes per workflow.

I would give every automation an expiry date. After six months the question comes up: is this still running, and is it still needed? Two of our four shutdowns would have happened half a year earlier.

And I would build less at once. We put fourteen flows live in a single quarter. Six would have been the sensible number, with monitoring and a clean handover.

On top of that, a short description per flow, right next to the workflow. Three sentences are enough: what it does, what it depends on, who to call when it stops. We wrote this retroactively for eight flows and twice discovered that nobody knew exactly which rule applied there. The writing took ten minutes per flow. At the first outage it was the best-invested time of the year.

The limits of this review

Three caveats so you read the numbers correctly.

First: these are our brands, our volume, our setup. We are two people and can repair every flow ourselves. Anyone who bought the automation in and has nobody in-house who understands it has a completely different maintenance case.

Second: twelve months is short. The second winter at high order volume is still ahead for some of these flows.

Third: I did not measure the savings continuously, only at the start and now. Part of the 33 to 46 hours per week is a solid estimate, not a stopwatch.

Conclusion

After twelve months I would do the whole thing again. Planned differently, but again.

What I take away: the simple flows hold. The complex ones die or get rebuilt. Maintenance is real and costs about half a day a month. And the most important question after go-live is: which automation should you switch off again?

If you are in the planning stage: build the monitoring first. Give every automation a date on which you review it. And write down how much time it actually saves, or in a year you will be arguing about feelings.

If you want an outside pass over your existing setup: talk to us at Flowhouse. We will also tell you which flows you are better off switching off.