Build Robomotion

0%0 of 18 items done

Saved on this device only. Tick prerequisites first, then work the phases in order · do not start one until the checks above it pass.

One automation done properly, whose hard problems are surviving a queue that delivers at least once, telling a failure worth retrying from one that will never succeed, and keeping a readable record of every run without writing a secret into it.

estimated effort weekendthe files for this build are in the project pack

RuntimeNode with a Redis-backed queue and a worker in its own processDefinitionWorkflows are declared data with typed, validated steps, never embedded codeStoragePostgreSQL holds runs and step results, Redis is only the queueSecretsReferenced by name in the definition, resolved at the moment of use, redacted in logsBrowser stepsOnly where no API exists, on sites whose terms allow it, headed by defaultSchedulesKeyed on the intended slot rather than the moment the tick woke

Before step 1

Everything below is assumed from the first step. Tick each one when you actually have it, not when you plan to.

installfree
Why Runs the scheduler, the worker and the tests.
Get it nodejs.org, or nvm install 22. open ↗
Verify node --version prints v22 or higher
installfree locally, a few pounds a month managed
Why Runs, attempts and dead letters live here, and the unique constraint on the idempotency key is part of the correctness argument.
Get it Your package manager, or a small managed instance.
Verify A duplicate idempotency key is rejected by the database
installfree locally
Why The queue, and nothing else. Its at-least-once behaviour is the reason this build is shaped the way it is.
Get it Your package manager, Docker, or a managed instance.
Verify A worker killed mid-job sees the job return to waiting
accountfree with most services
Why Exactly-once behaviour is only testable by actually calling something, and a duplicate invoice in a real account is a real invoice.
Get it A test or sandbox tenant with its own credentials, separate from anything live.
Verify A test call succeeds and the account is confirmed to be a sandbox
decidefree
Why This is the difference between a tool that saves hours and one that closes an account, and it is a reading task rather than a coding task.
Get it Read the published terms and API policy of every service the workflow touches, and write down what each permits.
Verify Each step in the definition names the permission it relies on
decidefree
Why Each is asserted by a test, and picking them after an incident means picking whatever the incident did.
Get it Write down steps per run, maximum run duration, artifact retention and the number of catch-up slots allowed.
Verify All four appear in configuration and are named in tests

Data model

Create these before the first phase that stores anything. Changing a table later is the expensive kind of change.

- workflows(id, name, version, definition_json, active, created_at) · a change is a new version row, because a run must be readable against the definition it used
- runs(id, workflow_id, workflow_version, trigger, idempotency_key, state, started_at, finished_at, error)
- unique(workflow_id, idempotency_key) · a redelivered trigger produces no second run
- step_runs(run_id, step_id, attempt, state, input_hash, output_json, external_ref, started_at, finished_at, error) · external_ref is what makes a repeat safe
- schedules(workflow_id, cron_expression, timezone, last_fired_slot, next_due_at) · a slot, not the timestamp the code happened to wake at
- artifacts(step_run_id, path, mime, bytes, delete_after) · screenshots and downloads, kept on a stated retention and then deleted
- dead_letters(run_id, step_id, at, classification, detail) · where a failure that cannot succeed goes, instead of into a backoff loop
- No table holds a secret value. Definitions name a secret and the worker resolves it at the moment of the call.

Environment variables

These go in a .env file the app reads at startup. The pack's .env.example is this table as a file · copy it, never commit the filled-in version.

Variable	Needed	Example	Where the value comes from
`DATABASE_URL`	required	`postgres://localhost:5432/automation`	Runs, attempts, dead letters. The record of what happened.
`REDIS_URL`	required	`redis://localhost:6379`	The queue only. Nothing here is a record.
`WORKER_CONCURRENCY`	required	`2`	Keep it low enough that a step never starves the event loop and stalls its own lock.
`ARTIFACT_DIR`	required	`./data/artifacts`	Screenshots and downloads, deleted at the stated retention.
`ARTIFACT_RETENTION_DAYS`	required	`7`	Short, because artifacts are the most likely place a secret is visible.
`CATCHUP_MAX_SLOTS`	required	`3`	After downtime. Without this, eight hours down becomes hundreds of runs at once.
`RUN_STEP_CEILING`	required	`200`	A run exceeding it stops and is recorded as stopped, not failed.
`SERVICE_API_KEY`secret	required	`sk-...redacted`	Referenced by name from the definition. Redacted by value in every log.

The build, in order

1
One workflow, declared and validated
A workflow is data you can read, and an invalid one is rejected when it is saved.
1. A validated boundary is what makes a failure readable later.
  Files src/steps/registry.tssrc/steps/schema.ts
2. A run is meaningless without the definition it actually used.
  Files src/workflows/versions.tsmigrations/001_workflows.sql
3. Asserted with a counter, not with a promise in the README.
  Files src/run/dryrun.tstests/dryrun.test.ts
done when · tick each as it passes
watch out
- Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made.
next phase
2
Runs that survive being run twice
A duplicate trigger creates no second run, and a repeated step causes no second side effect.
1. A unique idempotency key per workflow is the cheapest duplicate guard there is.
  Files src/run/trigger.tsmigrations/002_runs.sql
2. Then a crash leaves evidence rather than ambiguity.
  Files src/run/execute.tstests/at-least-once.test.ts
3. A step that already has one is finished, however it looks.
  Files src/run/execute.ts
done when · tick each as it passes
watch out
- A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done.
next phase
3
Retries that tell a timeout from a refusal
Only failures that could plausibly succeed later are retried.
1. The decision to retry belongs to the classifier, not the caller.
  Files src/run/classify.tstests/classify.test.ts
2. Synchronised retries are how a small outage becomes a large one.
  Files src/run/backoff.ts
3. One alert beats a week of identical failures.
  Files src/run/pause.ts
done when · tick each as it passes
watch out
- Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep.
next phase
4
Logs you can read, secrets you cannot
Every run is reconstructable from the record, and no secret appears anywhere in it.
1. Secrets end up inside strings you did not design.
  Files src/log/redact.tstests/redact.test.ts
2. Enough to prove what ran without storing what it contained.
  Files src/log/record.ts
3. A replayed run should never be mistaken for a scheduled one.
  Files src/run/replay.ts
done when · tick each as it passes
watch out
- Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL.
5
Schedules, ceilings and the credential that diesproduct builder
It runs on time, stops before it costs too much, and says so when a credential expires.
1. The last slot fired, not the last time the process woke.
  Files src/schedule/tick.tstests/dst.test.ts
2. Downtime should not become a thundering herd.
  Files src/schedule/catchup.ts
3. Paused workflows, dead letters, last success, spend against ceiling.
  Files src/ui/status.tsx
done when · tick each as it passes
watch out
- A cron computed in local time either skips the run on the day the clocks go forward or fires it twice on the day they go back, and a catch-up written as run everything missed turns eight hours of downtime into hundreds of runs starting at once. Key on the intended slot, store the last slot fired rather than the last wake time, and cap catch-up explicitly.

what this build does not replace

A connector library.
A hosted control plane with teams and roles.
A visual workflow builder.
Any step that defeats an anti-automation measure.
hundreds of maintained connectors
OAuth app verification
durable execution at scale
schema drift handling and enterprise controls

after v1, if you want it

A second workflow reusing the same step kinds, which is the real test of whether the definitions are data
A webhook trigger alongside the schedule, which the idempotency key already supports
Per-step timeouts tuned from the recorded durations, once a month of runs exists to tune against

Need the files? The project pack on the verdict page hands your agent the whole brief · more automation.