Reliability, Testing & Monitoring

A demo works once. A business automation must work on the thousandth run at 3 a.m. That is when an API is slow, a customer pastes something strange and nobody is watching. Reliability is what clients really buy, and it turns a one-off build into a retainer. This lesson covers error handling, testing, monitoring and the habits that keep automations alive.

Expect things to fail

• APIs time out, limit your rate, change their fields or go down.
• Inputs arrive empty, huge, in the wrong format, in another language or with bad intent.
• Logins expire and permissions get changed.
• AI steps return output that is invalid, off-topic or of low quality.
• Platform outages and plan limits stop runs completely.

Patterns for handling errors

• Retry with longer waits for short-lived errors, such as timeouts, 429 and server errors. Retry only a small, fixed number of times.
• Do not retry permanent errors, such as bad credentials or invalid data. Send them to review and raise an alert.
• Error branches: every step that can fail has a planned fallback path.
• A dead-letter queue: failed items are stored, in a sheet, table or queue, with the input, the error and the step. Nobody has to guess, and you can replay them after the fix.
• Idempotency: use unique keys or lookups, such as does this record already exist, so replays and retries do not duplicate emails, invoices or records.
• Timeouts and size limits: protect against huge payloads and slow services.

Logging and visibility

Every run writes ONE row to a log (sheet/DB/observability tool):
run_id | started_at | workflow | version | trigger_ref | status (ok / partial / failed / needs_review)
duration_s | steps_completed | ai_calls | est_cost | error_step | error_message | output_ref (link to created record)

Alerts: any failed run; needs_review items older than 24h; zero runs when volume is normally >0
        (silent failure); error rate > 5% in an hour; AI cost per run up 3x.
Weekly report to client: runs, success rate, items sent to review, time saved estimate, issues resolved.

Testing

• Keep a test set of sample inputs, normal, messy, broken and hostile, and run it after every change.
• Test with real-looking data. Never use real client data in a test setup.
• Test the failure paths on purpose. Disconnect an account, return an error from a mock, and send an empty payload.
• For AI steps, compare the outputs on your saved set before and after a prompt change.
• Do a supervised soft launch. Run in draft or shadow mode, where actions are logged but not carried out. Then switch on gradually.

Change management

• Keep a staging copy of the workflow. Edit and test there, then promote it.
• Version your workflows and prompts. Keep a changelog with the date, the reason and who tested it.
• Prepare a rollback, so that you can switch back to the previous version quickly.
• Pin or watch model versions where the platform allows it. Behavior can change when models are updated.
• Read the changelogs of the connected apps. Retired fields and API versions are the top cause of sudden breakage.

Documentation and hand-over

For each automation, write a one-page runbook. It says what the automation does. It has the diagram, the triggers, the accounts and permissions used, and where the logs and the review queue live. It explains how to pause it, the common errors and their fixes, and who to contact. A client who can pause the workflow safely and understand an alert trusts you more. And your maintenance retainer becomes easier to justify.