There are two worlds in automation. In one, a form triggers an email and everything is done in two seconds. In the other, a nightly process walks tens of thousands of ERP records, cross-references them against a logistics system and generates documents. The first fits in any tool. The second is where most of them break. This article is about the second: what it really means for a process to "take a long time", why time limits exist, and how to design high-volume automations that don't get cut off halfway.
Tell slow apart from runaway
They look the same from outside, but they are opposite problems and the fixes are opposite too:
- Slow: few steps that take a long time (a heavy query, a slow API)
- Runaway: a huge number of steps in no time (a loop that never ends)
- A slow process is legitimate and needs room to breathe
- A runaway process needs to be cut as soon as possible
💡 Tip: if your process runs few nodes over a long period, it's slow, not broken. Giving it a short time limit only breaks it for real.
Fit the time limit to reality
The maximum time per execution exists so a badly built flow can't run forever. But a low value gets in the way of legitimate processes:
- Measure how long it really takes on a high-volume day, not a quiet one
- Leave headroom: month-end peak isn't an average Tuesday
- In Floogos the limit can be raised or removed entirely
- The brake against loops is the step budget, not the clock — removing the time limit doesn't leave you unprotected
💡 Tip: measure the real worst case before setting any limit. A number picked by feel always ends up cutting the wrong execution.
Break the volume into batches
The best way to keep a long process from being cut is for it to stop being a long process:
- Process in batches (by date, by branch, by ID range) instead of all at once
- Each batch is a short, independent execution
- If one fails you retry just that one, not the whole eight hours
- You gain visibility: you see progress instead of waiting blind
💡 Tip: batching doesn't just avoid cuts, it also lowers the memory each execution consumes. Two problems, one decision.
Attack the slow node, not the whole process
When a single step eats most of the time, the problem is there and not in the flow:
- A query without an index can take minutes where it should take seconds
- Fetching every column when you need three multiplies transfer time
- Filter at the source: better the database returns 500 rows than filtering 50,000 afterwards
- Set timeouts on the node itself, not only on the whole execution
💡 Tip: before raising any limit, look at which node eats the time. Often a properly indexed query turns an hours-long process into a minutes-long one.
Find out when something gets cut
A nightly process that fails silently is worse than one that doesn't exist, because you think it worked:
- Set up an alert flow that fires on any failure
- Include the reason and the last executed step in the notification
- A time cut and a loop cut call for different actions: tell them apart
- Review the execution history, not just the final result
💡 Tip: the valuable signal isn't "it failed", it's "it failed here and for this reason". Without that, diagnosis turns into guesswork.
Checklist for high-volume processes
Frequently asked questions
How long can an automation run in Floogos?
Up to one hour by default, and the limit can be raised or removed entirely. For high-volume nightly processes it's common to extend it or run with no cap.
Is it dangerous to remove the time limit?
No, because time isn't the main defence against loops. An infinite loop runs thousands of steps in seconds and hits the step budget long before any clock runs out. A slow but correct process runs few steps over a long period and never comes close to that budget.
Why did my process get cut at 63 minutes if the limit was 60?
Because the cut never splits a step in half. If the limit expires while a query is running, the engine waits for it to finish and cuts right after. It's deliberate: cutting mid-step would leave open transactions or half-written files.
Should long processes always be batched?
Almost always yes. Batching reduces the risk of being cut, lowers memory usage, lets you retry only the part that failed and gives you visibility of progress. The cost is a little more design up front.
The processes that move real volume are the ones that generate the most value and the ones most tools handle worst, because they assume everything finishes in seconds. Designing for volume means telling slow apart from broken, batching where you can, attacking the slow step instead of the whole process, and always finding out when something got cut. With that, an eight-hour sync stops being a risk and becomes just another process that runs at night.