Vibecode Robomotion
track this build4 phases, 12 steps, beginner friendly0%KINDA · weekend project
The visible robotic process automation loop is buildable, but a credible replacement needs more than the first screen. Robomotion earns its keep through connectors, auth, reliability, so expect a weekend or multi-day build and a narrower personal scope.
the project pack5 files · written for this build, step by step
You are building a lean indie version of Robomotion. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Robomotion · indie build One automation done properly, whose hard problems are surviving a queue that delivers at least once, telling a failure worth retrying from one that will never succeed, and keeping a readable record of every run without writing a secret into it. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node with a Redis-backed queue and a worker in its own process | the tick and the work must not share a process, because one slow step blocks every schedule | | Definition | Workflows are declared data with typed, validated steps, never embedded code | a workflow you cannot read at three in the morning is one you cannot fix at three in the morning | | Storage | PostgreSQL holds runs and step results, Redis is only the queue | Redis is a delivery mechanism, not your record of what happened | | Secrets | Referenced by name in the definition, resolved at the moment of use, redacted in logs | definitions get exported, shared and pasted into issues | | Browser steps | Only where no API exists, on sites whose terms allow it, headed by default | an invisible browser is how you discover three weeks late that it has been failing | | Schedules | Keyed on the intended slot rather than the moment the tick woke | a catch-up after downtime should run once, not once per missed minute | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the scheduler, the worker and the tests. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **PostgreSQL 16 or newer** · free locally, a few pounds a month managed - Why: Runs, attempts and dead letters live here, and the unique constraint on the idempotency key is part of the correctness argument. - Get it: Your package manager, or a small managed instance. - Verify: A duplicate idempotency key is rejected by the database - [ ] **Redis 7 or newer** · free locally - Why: The queue, and nothing else. Its at-least-once behaviour is the reason this build is shaped the way it is. - Get it: Your package manager, Docker, or a managed instance. - Verify: A worker killed mid-job sees the job return to waiting - [ ] **A sandbox account on the service you are automating** · free with most services - Why: Exactly-once behaviour is only testable by actually calling something, and a duplicate invoice in a real account is a real invoice. - Get it: A test or sandbox tenant with its own credentials, separate from anything live. - Verify: A test call succeeds and the account is confirmed to be a sandbox - [ ] **A written decision about what you are allowed to automate** · free - Why: This is the difference between a tool that saves hours and one that closes an account, and it is a reading task rather than a coding task. - Get it: Read the published terms and API policy of every service the workflow touches, and write down what each permits. - Verify: Each step in the definition names the permission it relies on - [ ] **Your ceilings, retention and catch-up cap** · free - Why: Each is asserted by a test, and picking them after an incident means picking whatever the incident did. - Get it: Write down steps per run, maximum run duration, artifact retention and the number of catch-up slots allowed. - Verify: All four appear in configuration and are named in tests ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - A connector library. - A hosted control plane with teams and roles. - A visual workflow builder. - Any step that defeats an anti-automation measure. - hundreds of maintained connectors - OAuth app verification - durable execution at scale - schema drift handling and enterprise controls If one of those is essential to you, that is the reason to keep paying for Robomotion, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Robomotion The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a single scheduled automation that retries safely, records what it did, and can be replayed by hand to replace Robomotion. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Assume every step will run twice. The queue behind this build delivers at least once: when a worker stops renewing its lock, because the machine was busy or the process died, the job goes back to waiting and is processed again. That is documented behaviour rather than a bug, and it means a step that raises an invoice or sends an email will do both twice unless something stops it. So every step is declared either idempotent, with the key that makes it so, or unsafe to repeat, and an unsafe step halts the run and asks rather than retrying on its own. The attempt row is written before the call, and a step with an external reference already recorded is done, not due. The second rule is about what you automate at all. Services with an API, and accounts whose credentials you hold. No defeating a challenge that exists to stop automation, no driving somebody else's login on their behalf, no scraping a site whose published terms forbid it. An automation that gets an account closed has cost more than the hours it saved. ### Stack (fixed, do not substitute) - Runtime: Node with a Redis-backed queue and a worker in its own process · the tick and the work must not share a process, because one slow step blocks every schedule - Definition: Workflows are declared data with typed, validated steps, never embedded code · a workflow you cannot read at three in the morning is one you cannot fix at three in the morning - Storage: PostgreSQL holds runs and step results, Redis is only the queue · Redis is a delivery mechanism, not your record of what happened - Secrets: Referenced by name in the definition, resolved at the moment of use, redacted in logs · definitions get exported, shared and pasted into issues - Browser steps: Only where no API exists, on sites whose terms allow it, headed by default · an invisible browser is how you discover three weeks late that it has been failing - Schedules: Keyed on the intended slot rather than the moment the tick woke · a catch-up after downtime should run once, not once per missed minute ### Data model (create this before Phase 1) - workflows(id, name, version, definition_json, active, created_at) · a change is a new version row, because a run must be readable against the definition it used - runs(id, workflow_id, workflow_version, trigger, idempotency_key, state, started_at, finished_at, error) - unique(workflow_id, idempotency_key) · a redelivered trigger produces no second run - step_runs(run_id, step_id, attempt, state, input_hash, output_json, external_ref, started_at, finished_at, error) · external_ref is what makes a repeat safe - schedules(workflow_id, cron_expression, timezone, last_fired_slot, next_due_at) · a slot, not the timestamp the code happened to wake at - artifacts(step_run_id, path, mime, bytes, delete_after) · screenshots and downloads, kept on a stated retention and then deleted - dead_letters(run_id, step_id, at, classification, detail) · where a failure that cannot succeed goes, instead of into a backoff loop - No table holds a secret value. Definitions name a secret and the worker resolves it at the moment of the call. ### Phase 1 · One workflow, declared and validated Build: Define a small set of step kinds, each with a schema for its input and its output. Validate a definition on save and refuse to store an invalid one. Add a dry run that resolves every reference and prints each call it would make without making any of them. Every change to a definition writes a new version. Trap: Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. Done when: An invalid definition is refused at save time with the offending step named, tested with a missing field and a wrong type. A dry run prints every intended call and touches no external service, asserted with a counter that stays at zero. Editing a definition creates a new version and existing runs still resolve against the version they used. No step kind executes arbitrary code, verified by review. Do not build yet: No queue, no retries, no schedules, no logs. ### Phase 2 · Runs that survive being run twice Build: Derive an idempotency key from the triggering event and put a unique constraint on it. Write the attempt row before making any call. Pass a client-generated key where the service accepts one, and record the external reference the service returns. On a retry, a step that already has an external reference is treated as finished. Declare any step that cannot be made idempotent as unsafe to repeat, and halt the run instead of retrying it. Trap: A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. Done when: Triggering twice with the same event identifier creates one run, tested by replaying a webhook payload. Killing the worker between the call and the write leaves an attempt row, and the retry finds the external reference and skips the call, tested against a fake service that counts calls and asserts exactly one. A step declared unsafe to repeat halts the run and records why. Attempts are numbered and never overwrite the previous attempt's row. Do not build yet: No retry classification, logging or schedules yet. ### Phase 3 · Retries that tell a timeout from a refusal Build: Classify every failure as retryable, not retryable, or unknown. Timeouts, connection resets, 429 and 5xx are retryable with exponential backoff and jitter up to a cap; a 4xx that is not 429, a validation error and an authentication failure are not. A non-retryable failure goes to the dead letter table immediately, and an authentication failure pauses the workflow. Trap: Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. Done when: A 400 response is dead-lettered on the first attempt rather than retried, tested. A 429 is retried honouring any interval the response supplies, tested with a stubbed header. Backoff is exponential with jitter and a stated cap, asserted over a sequence of failures. An authentication failure pauses the workflow and alerts once rather than failing every scheduled run. Do not build yet: No logging, redaction, artifacts or replay yet. ### Phase 4 · Logs you can read, secrets you cannot Build: Record per-step structured logs with the input hash, the classified outcome and a redacted output. Redact by the secret's value as well as by declared field name. Store artifacts such as screenshots with a retention and delete them on it. Offer replay from any step using the recorded input, marked clearly as a replay in the run record. Trap: Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL. Done when: A run whose secret appears in a header, a body and a URL produces logs containing none of them, asserted by searching the log output for the value. Replay from a chosen step reuses the recorded input and marks the run as a replay. Artifacts are deleted at the stated retention and the deletion is recorded. Logs identify the workflow version so a run can be read against the definition it used. Do not build yet: No schedules, ceilings or export yet. ### Phase 5 · Schedules, ceilings and the credential that dies Build: Run schedules from a cron expression in a named zone, keyed on the intended slot and recorded as the last slot fired. Cap catch-up after downtime to a stated number of slots. Put a ceiling on steps per run, run duration, and spend on any metered step. Export runs and definitions as files. Show a single status page with paused workflows, dead letters and the last successful run per workflow. Trap: A cron computed in local time either skips the run on the day the clocks go forward or fires it twice on the day they go back, and a catch-up written as run everything missed turns eight hours of downtime into hundreds of runs starting at once. Key on the intended slot, store the last slot fired rather than the last wake time, and cap catch-up explicitly. Done when: Both clock change days fire the documented number of runs, asserted against fixed dates. Eight hours of simulated downtime produces at most the capped number of catch-up runs, tested. A run exceeding the step or duration ceiling stops and is recorded as stopped rather than failed. Export produces definitions and run history that reimport into a fresh instance. Do not build yet: Nothing after this. ### Out of scope (and why) - Hundreds of maintained connectors, which are the actual product being sold here and are maintenance rather than engineering. - Defeating challenges that exist to stop automation, and driving a login on behalf of somebody who did not set it up themselves. - A hosted control plane with teams, roles and shared credentials, which multiplies every problem above by a permission model. - A visual workflow builder, which is a large piece of front end work that changes nothing about whether a run is correct. ### README must contain - That the queue delivers at least once, so every step is either idempotent with a named key or declared unsafe to repeat - That a step with an external reference already recorded is treated as finished rather than due - Which failures are retried and which go straight to the dead letter table, and why - That logs are redacted by secret value as well as by field name, and how that is tested - That schedules are keyed on the intended slot and how much catch-up is allowed after downtime ===== AGENTS.md ===== # Agent instructions · Robomotion indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node with a Redis-backed queue and a worker in its own process, Workflows are declared data with typed, validated steps, never embedded code, PostgreSQL holds runs and step results, Redis is only the queue, Referenced by name in the definition, resolved at the moment of use, redacted in logs, Only where no API exists, on sites whose terms allow it, headed by default, Keyed on the intended slot rather than the moment the tick woke. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. - A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. - Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. - Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL. ===== BUILD_PLAN.md ===== # Build plan · Robomotion One automation done properly, whose hard problems are surviving a queue that delivers at least once, telling a failure worth retrying from one that will never succeed, and keeping a readable record of every run without writing a secret into it. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · One workflow, declared and validated A workflow is data you can read, and an invalid one is rejected when it is saved. ### Steps 1. Give every step kind an input and output schema A validated boundary is what makes a failure readable later. Files: `src/steps/registry.ts`, `src/steps/schema.ts` 2. Version definitions rather than editing them A run is meaningless without the definition it actually used. Files: `src/workflows/versions.ts`, `migrations/001_workflows.sql` 3. Make the dry run prove it called nothing Asserted with a counter, not with a promise in the README. Files: `src/run/dryrun.ts`, `tests/dryrun.test.ts` ### Done when - [ ] An invalid definition is refused at save time - [ ] A dry run makes zero external calls - [ ] Each edit creates a new version - [ ] No step kind executes arbitrary code ### Watch out - Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. ## Phase 2 · Runs that survive being run twice A duplicate trigger creates no second run, and a repeated step causes no second side effect. ### Steps 1. Constrain the trigger, not just the handler A unique idempotency key per workflow is the cheapest duplicate guard there is. Files: `src/run/trigger.ts`, `migrations/002_runs.sql` 2. Write the attempt before the call Then a crash leaves evidence rather than ambiguity. Files: `src/run/execute.ts`, `tests/at-least-once.test.ts` 3. Record the external reference and honour it A step that already has one is finished, however it looks. Files: `src/run/execute.ts` ### Done when - [ ] A duplicate trigger yields one run - [ ] A killed worker leads to one external call, not two - [ ] An unsafe step halts instead of retrying - [ ] Attempts are numbered and preserved ### Watch out - A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. ## Phase 3 · Retries that tell a timeout from a refusal Only failures that could plausibly succeed later are retried. ### Steps 1. Classify, then decide The decision to retry belongs to the classifier, not the caller. Files: `src/run/classify.ts`, `tests/classify.test.ts` 2. Jitter the backoff Synchronised retries are how a small outage becomes a large one. Files: `src/run/backoff.ts` 3. Pause the workflow on an auth failure One alert beats a week of identical failures. Files: `src/run/pause.ts` ### Done when - [ ] A 400 is dead-lettered on the first attempt - [ ] A 429 honours the interval it is given - [ ] Backoff is exponential with jitter and a cap - [ ] An auth failure pauses and alerts once ### Watch out - Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. ## Phase 4 · Logs you can read, secrets you cannot Every run is reconstructable from the record, and no secret appears anywhere in it. ### Steps 1. Redact by value, not only by field name Secrets end up inside strings you did not design. Files: `src/log/redact.ts`, `tests/redact.test.ts` 2. Keep the input hash rather than the input Enough to prove what ran without storing what it contained. Files: `src/log/record.ts` 3. Make replay explicit in the record A replayed run should never be mistaken for a scheduled one. Files: `src/run/replay.ts` ### Done when - [ ] No secret value appears in any log - [ ] Replay reuses the recorded input and is marked - [ ] Artifacts are deleted on retention with a record - [ ] Logs name the workflow version used ### Watch out - Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL. ## Not in this build - A connector library. - A hosted control plane with teams and roles. - A visual workflow builder. - Any step that defeats an anti-automation measure. ## After v1, if you want it - A second workflow reusing the same step kinds, which is the real test of whether the definitions are data - A webhook trigger alongside the schedule, which the idempotency key already supports - Per-step timeouts tuned from the recorded durations, once a month of runs exists to tune against ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Runs, attempts, dead letters. The record of what happened. DATABASE_URL=postgres://localhost:5432/automation # Required. The queue only. Nothing here is a record. REDIS_URL=redis://localhost:6379 # Required. Keep it low enough that a step never starves the event loop and stalls its own lock. WORKER_CONCURRENCY=2 # Required. Screenshots and downloads, deleted at the stated retention. ARTIFACT_DIR=./data/artifacts # Required. Short, because artifacts are the most likely place a secret is visible. ARTIFACT_RETENTION_DAYS=7 # Required. After downtime. Without this, eight hours down becomes hundreds of runs at once. CATCHUP_MAX_SLOTS=3 # Required. A run exceeding it stops and is recorded as stopped, not failed. RUN_STEP_CEILING=200 # Required · secret. Referenced by name from the definition. Redacted by value in every log. SERVICE_API_KEY=sk-...redacted
You are building a lean indie version of Robomotion. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Robomotion · indie build One automation done properly, whose hard problems are surviving a queue that delivers at least once, telling a failure worth retrying from one that will never succeed, and keeping a readable record of every run without writing a secret into it. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node with a Redis-backed queue and a worker in its own process | the tick and the work must not share a process, because one slow step blocks every schedule | | Definition | Workflows are declared data with typed, validated steps, never embedded code | a workflow you cannot read at three in the morning is one you cannot fix at three in the morning | | Storage | PostgreSQL holds runs and step results, Redis is only the queue | Redis is a delivery mechanism, not your record of what happened | | Secrets | Referenced by name in the definition, resolved at the moment of use, redacted in logs | definitions get exported, shared and pasted into issues | | Browser steps | Only where no API exists, on sites whose terms allow it, headed by default | an invisible browser is how you discover three weeks late that it has been failing | | Schedules | Keyed on the intended slot rather than the moment the tick woke | a catch-up after downtime should run once, not once per missed minute | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the scheduler, the worker and the tests. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **PostgreSQL 16 or newer** · free locally, a few pounds a month managed - Why: Runs, attempts and dead letters live here, and the unique constraint on the idempotency key is part of the correctness argument. - Get it: Your package manager, or a small managed instance. - Verify: A duplicate idempotency key is rejected by the database - [ ] **Redis 7 or newer** · free locally - Why: The queue, and nothing else. Its at-least-once behaviour is the reason this build is shaped the way it is. - Get it: Your package manager, Docker, or a managed instance. - Verify: A worker killed mid-job sees the job return to waiting - [ ] **A sandbox account on the service you are automating** · free with most services - Why: Exactly-once behaviour is only testable by actually calling something, and a duplicate invoice in a real account is a real invoice. - Get it: A test or sandbox tenant with its own credentials, separate from anything live. - Verify: A test call succeeds and the account is confirmed to be a sandbox - [ ] **A written decision about what you are allowed to automate** · free - Why: This is the difference between a tool that saves hours and one that closes an account, and it is a reading task rather than a coding task. - Get it: Read the published terms and API policy of every service the workflow touches, and write down what each permits. - Verify: Each step in the definition names the permission it relies on - [ ] **Your ceilings, retention and catch-up cap** · free - Why: Each is asserted by a test, and picking them after an incident means picking whatever the incident did. - Get it: Write down steps per run, maximum run duration, artifact retention and the number of catch-up slots allowed. - Verify: All four appear in configuration and are named in tests ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - A connector library. - A hosted control plane with teams and roles. - A visual workflow builder. - Any step that defeats an anti-automation measure. - hundreds of maintained connectors - OAuth app verification - durable execution at scale - schema drift handling and enterprise controls If one of those is essential to you, that is the reason to keep paying for Robomotion, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Robomotion The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a single scheduled automation that retries safely, records what it did, and can be replayed by hand to replace Robomotion. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Assume every step will run twice. The queue behind this build delivers at least once: when a worker stops renewing its lock, because the machine was busy or the process died, the job goes back to waiting and is processed again. That is documented behaviour rather than a bug, and it means a step that raises an invoice or sends an email will do both twice unless something stops it. So every step is declared either idempotent, with the key that makes it so, or unsafe to repeat, and an unsafe step halts the run and asks rather than retrying on its own. The attempt row is written before the call, and a step with an external reference already recorded is done, not due. The second rule is about what you automate at all. Services with an API, and accounts whose credentials you hold. No defeating a challenge that exists to stop automation, no driving somebody else's login on their behalf, no scraping a site whose published terms forbid it. An automation that gets an account closed has cost more than the hours it saved. ### Stack (fixed, do not substitute) - Runtime: Node with a Redis-backed queue and a worker in its own process · the tick and the work must not share a process, because one slow step blocks every schedule - Definition: Workflows are declared data with typed, validated steps, never embedded code · a workflow you cannot read at three in the morning is one you cannot fix at three in the morning - Storage: PostgreSQL holds runs and step results, Redis is only the queue · Redis is a delivery mechanism, not your record of what happened - Secrets: Referenced by name in the definition, resolved at the moment of use, redacted in logs · definitions get exported, shared and pasted into issues - Browser steps: Only where no API exists, on sites whose terms allow it, headed by default · an invisible browser is how you discover three weeks late that it has been failing - Schedules: Keyed on the intended slot rather than the moment the tick woke · a catch-up after downtime should run once, not once per missed minute ### Data model (create this before Phase 1) - workflows(id, name, version, definition_json, active, created_at) · a change is a new version row, because a run must be readable against the definition it used - runs(id, workflow_id, workflow_version, trigger, idempotency_key, state, started_at, finished_at, error) - unique(workflow_id, idempotency_key) · a redelivered trigger produces no second run - step_runs(run_id, step_id, attempt, state, input_hash, output_json, external_ref, started_at, finished_at, error) · external_ref is what makes a repeat safe - schedules(workflow_id, cron_expression, timezone, last_fired_slot, next_due_at) · a slot, not the timestamp the code happened to wake at - artifacts(step_run_id, path, mime, bytes, delete_after) · screenshots and downloads, kept on a stated retention and then deleted - dead_letters(run_id, step_id, at, classification, detail) · where a failure that cannot succeed goes, instead of into a backoff loop - No table holds a secret value. Definitions name a secret and the worker resolves it at the moment of the call. ### Phase 1 · One workflow, declared and validated Build: Define a small set of step kinds, each with a schema for its input and its output. Validate a definition on save and refuse to store an invalid one. Add a dry run that resolves every reference and prints each call it would make without making any of them. Every change to a definition writes a new version. Trap: Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. Done when: An invalid definition is refused at save time with the offending step named, tested with a missing field and a wrong type. A dry run prints every intended call and touches no external service, asserted with a counter that stays at zero. Editing a definition creates a new version and existing runs still resolve against the version they used. No step kind executes arbitrary code, verified by review. Do not build yet: No queue, no retries, no schedules, no logs. ### Phase 2 · Runs that survive being run twice Build: Derive an idempotency key from the triggering event and put a unique constraint on it. Write the attempt row before making any call. Pass a client-generated key where the service accepts one, and record the external reference the service returns. On a retry, a step that already has an external reference is treated as finished. Declare any step that cannot be made idempotent as unsafe to repeat, and halt the run instead of retrying it. Trap: A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. Done when: Triggering twice with the same event identifier creates one run, tested by replaying a webhook payload. Killing the worker between the call and the write leaves an attempt row, and the retry finds the external reference and skips the call, tested against a fake service that counts calls and asserts exactly one. A step declared unsafe to repeat halts the run and records why. Attempts are numbered and never overwrite the previous attempt's row. Do not build yet: No retry classification, logging or schedules yet. ### Phase 3 · Retries that tell a timeout from a refusal Build: Classify every failure as retryable, not retryable, or unknown. Timeouts, connection resets, 429 and 5xx are retryable with exponential backoff and jitter up to a cap; a 4xx that is not 429, a validation error and an authentication failure are not. A non-retryable failure goes to the dead letter table immediately, and an authentication failure pauses the workflow. Trap: Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. Done when: A 400 response is dead-lettered on the first attempt rather than retried, tested. A 429 is retried honouring any interval the response supplies, tested with a stubbed header. Backoff is exponential with jitter and a stated cap, asserted over a sequence of failures. An authentication failure pauses the workflow and alerts once rather than failing every scheduled run. Do not build yet: No logging, redaction, artifacts or replay yet. ### Phase 4 · Logs you can read, secrets you cannot Build: Record per-step structured logs with the input hash, the classified outcome and a redacted output. Redact by the secret's value as well as by declared field name. Store artifacts such as screenshots with a retention and delete them on it. Offer replay from any step using the recorded input, marked clearly as a replay in the run record. Trap: Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL. Done when: A run whose secret appears in a header, a body and a URL produces logs containing none of them, asserted by searching the log output for the value. Replay from a chosen step reuses the recorded input and marks the run as a replay. Artifacts are deleted at the stated retention and the deletion is recorded. Logs identify the workflow version so a run can be read against the definition it used. Do not build yet: No schedules, ceilings or export yet. ### Phase 5 · Schedules, ceilings and the credential that dies Build: Run schedules from a cron expression in a named zone, keyed on the intended slot and recorded as the last slot fired. Cap catch-up after downtime to a stated number of slots. Put a ceiling on steps per run, run duration, and spend on any metered step. Export runs and definitions as files. Show a single status page with paused workflows, dead letters and the last successful run per workflow. Trap: A cron computed in local time either skips the run on the day the clocks go forward or fires it twice on the day they go back, and a catch-up written as run everything missed turns eight hours of downtime into hundreds of runs starting at once. Key on the intended slot, store the last slot fired rather than the last wake time, and cap catch-up explicitly. Done when: Both clock change days fire the documented number of runs, asserted against fixed dates. Eight hours of simulated downtime produces at most the capped number of catch-up runs, tested. A run exceeding the step or duration ceiling stops and is recorded as stopped rather than failed. Export produces definitions and run history that reimport into a fresh instance. Do not build yet: Nothing after this. ### Out of scope (and why) - Hundreds of maintained connectors, which are the actual product being sold here and are maintenance rather than engineering. - Defeating challenges that exist to stop automation, and driving a login on behalf of somebody who did not set it up themselves. - A hosted control plane with teams, roles and shared credentials, which multiplies every problem above by a permission model. - A visual workflow builder, which is a large piece of front end work that changes nothing about whether a run is correct. ### README must contain - That the queue delivers at least once, so every step is either idempotent with a named key or declared unsafe to repeat - That a step with an external reference already recorded is treated as finished rather than due - Which failures are retried and which go straight to the dead letter table, and why - That logs are redacted by secret value as well as by field name, and how that is tested - That schedules are keyed on the intended slot and how much catch-up is allowed after downtime ===== AGENTS.md ===== # Agent instructions · Robomotion indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node with a Redis-backed queue and a worker in its own process, Workflows are declared data with typed, validated steps, never embedded code, PostgreSQL holds runs and step results, Redis is only the queue, Referenced by name in the definition, resolved at the moment of use, redacted in logs, Only where no API exists, on sites whose terms allow it, headed by default, Keyed on the intended slot rather than the moment the tick woke. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. - A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. - Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. - Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL. ===== BUILD_PLAN.md ===== # Build plan · Robomotion One automation done properly, whose hard problems are surviving a queue that delivers at least once, telling a failure worth retrying from one that will never succeed, and keeping a readable record of every run without writing a secret into it. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · One workflow, declared and validated A workflow is data you can read, and an invalid one is rejected when it is saved. ### Steps 1. Give every step kind an input and output schema A validated boundary is what makes a failure readable later. Files: `src/steps/registry.ts`, `src/steps/schema.ts` 2. Version definitions rather than editing them A run is meaningless without the definition it actually used. Files: `src/workflows/versions.ts`, `migrations/001_workflows.sql` 3. Make the dry run prove it called nothing Asserted with a counter, not with a promise in the README. Files: `src/run/dryrun.ts`, `tests/dryrun.test.ts` ### Done when - [ ] An invalid definition is refused at save time - [ ] A dry run makes zero external calls - [ ] Each edit creates a new version - [ ] No step kind executes arbitrary code ### Watch out - Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. ## Phase 2 · Runs that survive being run twice A duplicate trigger creates no second run, and a repeated step causes no second side effect. ### Steps 1. Constrain the trigger, not just the handler A unique idempotency key per workflow is the cheapest duplicate guard there is. Files: `src/run/trigger.ts`, `migrations/002_runs.sql` 2. Write the attempt before the call Then a crash leaves evidence rather than ambiguity. Files: `src/run/execute.ts`, `tests/at-least-once.test.ts` 3. Record the external reference and honour it A step that already has one is finished, however it looks. Files: `src/run/execute.ts` ### Done when - [ ] A duplicate trigger yields one run - [ ] A killed worker leads to one external call, not two - [ ] An unsafe step halts instead of retrying - [ ] Attempts are numbered and preserved ### Watch out - A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. ## Phase 3 · Retries that tell a timeout from a refusal Only failures that could plausibly succeed later are retried. ### Steps 1. Classify, then decide The decision to retry belongs to the classifier, not the caller. Files: `src/run/classify.ts`, `tests/classify.test.ts` 2. Jitter the backoff Synchronised retries are how a small outage becomes a large one. Files: `src/run/backoff.ts` 3. Pause the workflow on an auth failure One alert beats a week of identical failures. Files: `src/run/pause.ts` ### Done when - [ ] A 400 is dead-lettered on the first attempt - [ ] A 429 honours the interval it is given - [ ] Backoff is exponential with jitter and a cap - [ ] An auth failure pauses and alerts once ### Watch out - Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. ## Phase 4 · Logs you can read, secrets you cannot Every run is reconstructable from the record, and no secret appears anywhere in it. ### Steps 1. Redact by value, not only by field name Secrets end up inside strings you did not design. Files: `src/log/redact.ts`, `tests/redact.test.ts` 2. Keep the input hash rather than the input Enough to prove what ran without storing what it contained. Files: `src/log/record.ts` 3. Make replay explicit in the record A replayed run should never be mistaken for a scheduled one. Files: `src/run/replay.ts` ### Done when - [ ] No secret value appears in any log - [ ] Replay reuses the recorded input and is marked - [ ] Artifacts are deleted on retention with a record - [ ] Logs name the workflow version used ### Watch out - Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL. ## Not in this build - A connector library. - A hosted control plane with teams and roles. - A visual workflow builder. - Any step that defeats an anti-automation measure. ## After v1, if you want it - A second workflow reusing the same step kinds, which is the real test of whether the definitions are data - A webhook trigger alongside the schedule, which the idempotency key already supports - Per-step timeouts tuned from the recorded durations, once a month of runs exists to tune against ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Runs, attempts, dead letters. The record of what happened. DATABASE_URL=postgres://localhost:5432/automation # Required. The queue only. Nothing here is a record. REDIS_URL=redis://localhost:6379 # Required. Keep it low enough that a step never starves the event loop and stalls its own lock. WORKER_CONCURRENCY=2 # Required. Screenshots and downloads, deleted at the stated retention. ARTIFACT_DIR=./data/artifacts # Required. Short, because artifacts are the most likely place a secret is visible. ARTIFACT_RETENTION_DAYS=7 # Required. After downtime. Without this, eight hours down becomes hundreds of runs at once. CATCHUP_MAX_SLOTS=3 # Required. A run exceeding it stops and is recorded as stopped, not failed. RUN_STEP_CEILING=200 # Required · secret. Referenced by name from the definition. Redacted by value in every log. SERVICE_API_KEY=sk-...redacted
You are building a production product version of Robomotion. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # Robomotion · product brief ## Problem The visible robotic process automation loop is buildable, but a credible replacement needs more than the first screen. Robomotion earns its keep through connectors, auth, reliability, so expect a weekend or multi-day build and a narrower personal scope. ## Product outcome One automation that runs on its schedule, does its external work once even when the queue hands the job back, retries only what could succeed, leaves a record you can read without leaking a credential, and pauses loudly when its credential dies. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - hosted PostgreSQL and Redis - API credentials for the two chosen services - Node.js 22 ## Explicit non-goals for v1 - A connector library. - A hosted control plane with teams and roles. - A visual workflow builder. - Any step that defeats an anti-automation measure. - hundreds of maintained connectors - OAuth app verification - durable execution at scale - schema drift handling and enterprise controls ## Success criteria - A killed worker is confirmed to produce one external call, not two - A duplicate trigger is confirmed to produce one run - A 400 is confirmed dead-lettered on the first attempt - No secret value is confirmed present in any log - Both clock change days are confirmed to fire the documented count ===== BRIEF.md ===== # Build brief · Robomotion The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a single scheduled automation that retries safely, records what it did, and can be replayed by hand to replace Robomotion. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Assume every step will run twice. The queue behind this build delivers at least once: when a worker stops renewing its lock, because the machine was busy or the process died, the job goes back to waiting and is processed again. That is documented behaviour rather than a bug, and it means a step that raises an invoice or sends an email will do both twice unless something stops it. So every step is declared either idempotent, with the key that makes it so, or unsafe to repeat, and an unsafe step halts the run and asks rather than retrying on its own. The attempt row is written before the call, and a step with an external reference already recorded is done, not due. The second rule is about what you automate at all. Services with an API, and accounts whose credentials you hold. No defeating a challenge that exists to stop automation, no driving somebody else's login on their behalf, no scraping a site whose published terms forbid it. An automation that gets an account closed has cost more than the hours it saved. ### Stack (fixed, do not substitute) - Runtime: Node with a Redis-backed queue and a worker in its own process · the tick and the work must not share a process, because one slow step blocks every schedule - Definition: Workflows are declared data with typed, validated steps, never embedded code · a workflow you cannot read at three in the morning is one you cannot fix at three in the morning - Storage: PostgreSQL holds runs and step results, Redis is only the queue · Redis is a delivery mechanism, not your record of what happened - Secrets: Referenced by name in the definition, resolved at the moment of use, redacted in logs · definitions get exported, shared and pasted into issues - Browser steps: Only where no API exists, on sites whose terms allow it, headed by default · an invisible browser is how you discover three weeks late that it has been failing - Schedules: Keyed on the intended slot rather than the moment the tick woke · a catch-up after downtime should run once, not once per missed minute ### Data model (create this before Phase 1) - workflows(id, name, version, definition_json, active, created_at) · a change is a new version row, because a run must be readable against the definition it used - runs(id, workflow_id, workflow_version, trigger, idempotency_key, state, started_at, finished_at, error) - unique(workflow_id, idempotency_key) · a redelivered trigger produces no second run - step_runs(run_id, step_id, attempt, state, input_hash, output_json, external_ref, started_at, finished_at, error) · external_ref is what makes a repeat safe - schedules(workflow_id, cron_expression, timezone, last_fired_slot, next_due_at) · a slot, not the timestamp the code happened to wake at - artifacts(step_run_id, path, mime, bytes, delete_after) · screenshots and downloads, kept on a stated retention and then deleted - dead_letters(run_id, step_id, at, classification, detail) · where a failure that cannot succeed goes, instead of into a backoff loop - No table holds a secret value. Definitions name a secret and the worker resolves it at the moment of the call. ### Phase 1 · One workflow, declared and validated Build: Define a small set of step kinds, each with a schema for its input and its output. Validate a definition on save and refuse to store an invalid one. Add a dry run that resolves every reference and prints each call it would make without making any of them. Every change to a definition writes a new version. Trap: Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. Done when: An invalid definition is refused at save time with the offending step named, tested with a missing field and a wrong type. A dry run prints every intended call and touches no external service, asserted with a counter that stays at zero. Editing a definition creates a new version and existing runs still resolve against the version they used. No step kind executes arbitrary code, verified by review. Do not build yet: No queue, no retries, no schedules, no logs. ### Phase 2 · Runs that survive being run twice Build: Derive an idempotency key from the triggering event and put a unique constraint on it. Write the attempt row before making any call. Pass a client-generated key where the service accepts one, and record the external reference the service returns. On a retry, a step that already has an external reference is treated as finished. Declare any step that cannot be made idempotent as unsafe to repeat, and halt the run instead of retrying it. Trap: A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. Done when: Triggering twice with the same event identifier creates one run, tested by replaying a webhook payload. Killing the worker between the call and the write leaves an attempt row, and the retry finds the external reference and skips the call, tested against a fake service that counts calls and asserts exactly one. A step declared unsafe to repeat halts the run and records why. Attempts are numbered and never overwrite the previous attempt's row. Do not build yet: No retry classification, logging or schedules yet. ### Phase 3 · Retries that tell a timeout from a refusal Build: Classify every failure as retryable, not retryable, or unknown. Timeouts, connection resets, 429 and 5xx are retryable with exponential backoff and jitter up to a cap; a 4xx that is not 429, a validation error and an authentication failure are not. A non-retryable failure goes to the dead letter table immediately, and an authentication failure pauses the workflow. Trap: Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. Done when: A 400 response is dead-lettered on the first attempt rather than retried, tested. A 429 is retried honouring any interval the response supplies, tested with a stubbed header. Backoff is exponential with jitter and a stated cap, asserted over a sequence of failures. An authentication failure pauses the workflow and alerts once rather than failing every scheduled run. Do not build yet: No logging, redaction, artifacts or replay yet. ### Phase 4 · Logs you can read, secrets you cannot Build: Record per-step structured logs with the input hash, the classified outcome and a redacted output. Redact by the secret's value as well as by declared field name. Store artifacts such as screenshots with a retention and delete them on it. Offer replay from any step using the recorded input, marked clearly as a replay in the run record. Trap: Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL. Done when: A run whose secret appears in a header, a body and a URL produces logs containing none of them, asserted by searching the log output for the value. Replay from a chosen step reuses the recorded input and marks the run as a replay. Artifacts are deleted at the stated retention and the deletion is recorded. Logs identify the workflow version so a run can be read against the definition it used. Do not build yet: No schedules, ceilings or export yet. ### Phase 5 · Schedules, ceilings and the credential that dies Build: Run schedules from a cron expression in a named zone, keyed on the intended slot and recorded as the last slot fired. Cap catch-up after downtime to a stated number of slots. Put a ceiling on steps per run, run duration, and spend on any metered step. Export runs and definitions as files. Show a single status page with paused workflows, dead letters and the last successful run per workflow. Trap: A cron computed in local time either skips the run on the day the clocks go forward or fires it twice on the day they go back, and a catch-up written as run everything missed turns eight hours of downtime into hundreds of runs starting at once. Key on the intended slot, store the last slot fired rather than the last wake time, and cap catch-up explicitly. Done when: Both clock change days fire the documented number of runs, asserted against fixed dates. Eight hours of simulated downtime produces at most the capped number of catch-up runs, tested. A run exceeding the step or duration ceiling stops and is recorded as stopped rather than failed. Export produces definitions and run history that reimport into a fresh instance. Do not build yet: Nothing after this. ### Out of scope (and why) - Hundreds of maintained connectors, which are the actual product being sold here and are maintenance rather than engineering. - Defeating challenges that exist to stop automation, and driving a login on behalf of somebody who did not set it up themselves. - A hosted control plane with teams, roles and shared credentials, which multiplies every problem above by a permission model. - A visual workflow builder, which is a large piece of front end work that changes nothing about whether a run is correct. ### README must contain - That the queue delivers at least once, so every step is either idempotent with a named key or declared unsafe to repeat - That a step with an external reference already recorded is treated as finished rather than due - Which failures are retried and which go straight to the dead letter table, and why - That logs are redacted by secret value as well as by field name, and how that is tested - That schedules are keyed on the intended slot and how much catch-up is allowed after downtime ===== ARCHITECTURE.md ===== # Architecture · Robomotion ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node with a Redis-backed queue and a worker in its own process | the tick and the work must not share a process, because one slow step blocks every schedule | | Definition | Workflows are declared data with typed, validated steps, never embedded code | a workflow you cannot read at three in the morning is one you cannot fix at three in the morning | | Storage | PostgreSQL holds runs and step results, Redis is only the queue | Redis is a delivery mechanism, not your record of what happened | | Secrets | Referenced by name in the definition, resolved at the moment of use, redacted in logs | definitions get exported, shared and pasted into issues | | Browser steps | Only where no API exists, on sites whose terms allow it, headed by default | an invisible browser is how you discover three weeks late that it has been failing | | Schedules | Keyed on the intended slot rather than the moment the tick woke | a catch-up after downtime should run once, not once per missed minute | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Definitions | Typed steps, validation and versioning. | Any format, provided steps stay declared data rather than code. | | Executor | Attempt rows, external references and unsafe-step halts. | Not swappable for call-then-record, which is where the duplicate invoice comes from. | | Classifier | Retryable versus dead letter, backoff and jitter. | Any policy, provided a non-retryable failure never enters a backoff loop. | | Record | Redacted logs, artifacts and replay. | Any sink, provided redaction is by value as well as by field name. | | Scheduler | Slot-keyed ticks and capped catch-up. | Any cron, provided both clock change days stay tested. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DATABASE_URL` · required · Runs, attempts, dead letters. The record of what happened. - `REDIS_URL` · required · The queue only. Nothing here is a record. - `WORKER_CONCURRENCY` · required · Keep it low enough that a step never starves the event loop and stalls its own lock. - `ARTIFACT_DIR` · required · Screenshots and downloads, deleted at the stated retention. - `ARTIFACT_RETENTION_DAYS` · required · Short, because artifacts are the most likely place a secret is visible. - `CATCHUP_MAX_SLOTS` · required · After downtime. Without this, eight hours down becomes hundreds of runs at once. - `RUN_STEP_CEILING` · required · A run exceeding it stops and is recorded as stopped, not failed. - `SERVICE_API_KEY` · required, secret · Referenced by name from the definition. Redacted by value in every log. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · Robomotion product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Node with a Redis-backed queue and a worker in its own process, Workflows are declared data with typed, validated steps, never embedded code, PostgreSQL holds runs and step results, Redis is only the queue, Referenced by name in the definition, resolved at the moment of use, redacted in logs, Only where no API exists, on sites whose terms allow it, headed by default, Keyed on the intended slot rather than the moment the tick woke. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. - A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. - Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. - Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL. - A cron computed in local time either skips the run on the day the clocks go forward or fires it twice on the day they go back, and a catch-up written as run everything missed turns eight hours of downtime into hundreds of runs starting at once. Key on the intended slot, store the last slot fired rather than the last wake time, and cap catch-up explicitly. ===== MILESTONES.md ===== # Delivery milestones · Robomotion Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · One workflow, declared and validated A workflow is data you can read, and an invalid one is rejected when it is saved. ### Steps 1. Give every step kind an input and output schema A validated boundary is what makes a failure readable later. Files: `src/steps/registry.ts`, `src/steps/schema.ts` 2. Version definitions rather than editing them A run is meaningless without the definition it actually used. Files: `src/workflows/versions.ts`, `migrations/001_workflows.sql` 3. Make the dry run prove it called nothing Asserted with a counter, not with a promise in the README. Files: `src/run/dryrun.ts`, `tests/dryrun.test.ts` ### Done when - [ ] An invalid definition is refused at save time - [ ] A dry run makes zero external calls - [ ] Each edit creates a new version - [ ] No step kind executes arbitrary code ### Watch out - Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. ## M2 · Runs that survive being run twice A duplicate trigger creates no second run, and a repeated step causes no second side effect. ### Steps 1. Constrain the trigger, not just the handler A unique idempotency key per workflow is the cheapest duplicate guard there is. Files: `src/run/trigger.ts`, `migrations/002_runs.sql` 2. Write the attempt before the call Then a crash leaves evidence rather than ambiguity. Files: `src/run/execute.ts`, `tests/at-least-once.test.ts` 3. Record the external reference and honour it A step that already has one is finished, however it looks. Files: `src/run/execute.ts` ### Done when - [ ] A duplicate trigger yields one run - [ ] A killed worker leads to one external call, not two - [ ] An unsafe step halts instead of retrying - [ ] Attempts are numbered and preserved ### Watch out - A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. ## M3 · Retries that tell a timeout from a refusal Only failures that could plausibly succeed later are retried. ### Steps 1. Classify, then decide The decision to retry belongs to the classifier, not the caller. Files: `src/run/classify.ts`, `tests/classify.test.ts` 2. Jitter the backoff Synchronised retries are how a small outage becomes a large one. Files: `src/run/backoff.ts` 3. Pause the workflow on an auth failure One alert beats a week of identical failures. Files: `src/run/pause.ts` ### Done when - [ ] A 400 is dead-lettered on the first attempt - [ ] A 429 honours the interval it is given - [ ] Backoff is exponential with jitter and a cap - [ ] An auth failure pauses and alerts once ### Watch out - Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. ## M4 · Logs you can read, secrets you cannot Every run is reconstructable from the record, and no secret appears anywhere in it. ### Steps 1. Redact by value, not only by field name Secrets end up inside strings you did not design. Files: `src/log/redact.ts`, `tests/redact.test.ts` 2. Keep the input hash rather than the input Enough to prove what ran without storing what it contained. Files: `src/log/record.ts` 3. Make replay explicit in the record A replayed run should never be mistaken for a scheduled one. Files: `src/run/replay.ts` ### Done when - [ ] No secret value appears in any log - [ ] Replay reuses the recorded input and is marked - [ ] Artifacts are deleted on retention with a record - [ ] Logs name the workflow version used ### Watch out - Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL. ## M5 · Schedules, ceilings and the credential that dies (production only) It runs on time, stops before it costs too much, and says so when a credential expires. ### Steps 1. Key schedules on the slot The last slot fired, not the last time the process woke. Files: `src/schedule/tick.ts`, `tests/dst.test.ts` 2. Cap catch-up and say so Downtime should not become a thundering herd. Files: `src/schedule/catchup.ts` 3. One status page with the four numbers that matter Paused workflows, dead letters, last success, spend against ceiling. Files: `src/ui/status.tsx` ### Done when - [ ] Both clock change days fire the documented count - [ ] Catch-up after downtime respects the cap - [ ] Ceilings stop a run and record it as stopped - [ ] Export reimports into a fresh instance ### Watch out - A cron computed in local time either skips the run on the day the clocks go forward or fires it twice on the day they go back, and a catch-up written as run everything missed turns eight hours of downtime into hundreds of runs starting at once. Key on the intended slot, store the last slot fired rather than the last wake time, and cap catch-up explicitly. ===== OPERATIONS.md ===== # Operations · Robomotion ## Backup The Postgres database, which holds runs, attempts, dead letters and definitions. Redis needs no backup by design, and if that is not true of your build then Redis has become a record and should not be. ## Restore Restore and start the worker with every run left in progress moved to a state that requires reconciliation rather than retry, so nothing is repeated on the assumption that it failed. Check external references before resuming anything. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Dead letter count, paused workflows, time since the last successful run per workflow, and spend against the ceiling. A rising dead letter count with a flat failure count usually means a downstream contract changed. ## Incident checklist A duplicated side effect means the attempt row was written after the call, and the attempts table shows the two rows. A workflow that quietly stopped is nearly always a paused credential, which is why the status page leads with paused workflows. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] A killed worker is confirmed to produce one external call, not two - [ ] A duplicate trigger is confirmed to produce one run - [ ] A 400 is confirmed dead-lettered on the first attempt - [ ] No secret value is confirmed present in any log - [ ] Both clock change days are confirmed to fire the documented count ## Launch constraint Do not market omitted Robomotion capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Runs, attempts, dead letters. The record of what happened. DATABASE_URL=postgres://localhost:5432/automation # Required. The queue only. Nothing here is a record. REDIS_URL=redis://localhost:6379 # Required. Keep it low enough that a step never starves the event loop and stalls its own lock. WORKER_CONCURRENCY=2 # Required. Screenshots and downloads, deleted at the stated retention. ARTIFACT_DIR=./data/artifacts # Required. Short, because artifacts are the most likely place a secret is visible. ARTIFACT_RETENTION_DAYS=7 # Required. After downtime. Without this, eight hours down becomes hundreds of runs at once. CATCHUP_MAX_SLOTS=3 # Required. A run exceeding it stops and is recorded as stopped, not failed. RUN_STEP_CEILING=200 # Required · secret. Referenced by name from the definition. Redacted by value in every log. SERVICE_API_KEY=sk-...redacted
README.md
# Robomotion · indie build One automation done properly, whose hard problems are surviving a queue that delivers at least once, telling a failure worth retrying from one that will never succeed, and keeping a readable record of every run without writing a secret into it. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node with a Redis-backed queue and a worker in its own process | the tick and the work must not share a process, because one slow step blocks every schedule | | Definition | Workflows are declared data with typed, validated steps, never embedded code | a workflow you cannot read at three in the morning is one you cannot fix at three in the morning | | Storage | PostgreSQL holds runs and step results, Redis is only the queue | Redis is a delivery mechanism, not your record of what happened | | Secrets | Referenced by name in the definition, resolved at the moment of use, redacted in logs | definitions get exported, shared and pasted into issues | | Browser steps | Only where no API exists, on sites whose terms allow it, headed by default | an invisible browser is how you discover three weeks late that it has been failing | | Schedules | Keyed on the intended slot rather than the moment the tick woke | a catch-up after downtime should run once, not once per missed minute | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the scheduler, the worker and the tests. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **PostgreSQL 16 or newer** · free locally, a few pounds a month managed - Why: Runs, attempts and dead letters live here, and the unique constraint on the idempotency key is part of the correctness argument. - Get it: Your package manager, or a small managed instance. - Verify: A duplicate idempotency key is rejected by the database - [ ] **Redis 7 or newer** · free locally - Why: The queue, and nothing else. Its at-least-once behaviour is the reason this build is shaped the way it is. - Get it: Your package manager, Docker, or a managed instance. - Verify: A worker killed mid-job sees the job return to waiting - [ ] **A sandbox account on the service you are automating** · free with most services - Why: Exactly-once behaviour is only testable by actually calling something, and a duplicate invoice in a real account is a real invoice. - Get it: A test or sandbox tenant with its own credentials, separate from anything live. - Verify: A test call succeeds and the account is confirmed to be a sandbox - [ ] **A written decision about what you are allowed to automate** · free - Why: This is the difference between a tool that saves hours and one that closes an account, and it is a reading task rather than a coding task. - Get it: Read the published terms and API policy of every service the workflow touches, and write down what each permits. - Verify: Each step in the definition names the permission it relies on - [ ] **Your ceilings, retention and catch-up cap** · free - Why: Each is asserted by a test, and picking them after an incident means picking whatever the incident did. - Get it: Write down steps per run, maximum run duration, artifact retention and the number of catch-up slots allowed. - Verify: All four appear in configuration and are named in tests ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - A connector library. - A hosted control plane with teams and roles. - A visual workflow builder. - Any step that defeats an anti-automation measure. - hundreds of maintained connectors - OAuth app verification - durable execution at scale - schema drift handling and enterprise controls If one of those is essential to you, that is the reason to keep paying for Robomotion, and the README should say so rather than pretend.
BRIEF.md
# Build brief · Robomotion The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a single scheduled automation that retries safely, records what it did, and can be replayed by hand to replace Robomotion. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Assume every step will run twice. The queue behind this build delivers at least once: when a worker stops renewing its lock, because the machine was busy or the process died, the job goes back to waiting and is processed again. That is documented behaviour rather than a bug, and it means a step that raises an invoice or sends an email will do both twice unless something stops it. So every step is declared either idempotent, with the key that makes it so, or unsafe to repeat, and an unsafe step halts the run and asks rather than retrying on its own. The attempt row is written before the call, and a step with an external reference already recorded is done, not due. The second rule is about what you automate at all. Services with an API, and accounts whose credentials you hold. No defeating a challenge that exists to stop automation, no driving somebody else's login on their behalf, no scraping a site whose published terms forbid it. An automation that gets an account closed has cost more than the hours it saved. ### Stack (fixed, do not substitute) - Runtime: Node with a Redis-backed queue and a worker in its own process · the tick and the work must not share a process, because one slow step blocks every schedule - Definition: Workflows are declared data with typed, validated steps, never embedded code · a workflow you cannot read at three in the morning is one you cannot fix at three in the morning - Storage: PostgreSQL holds runs and step results, Redis is only the queue · Redis is a delivery mechanism, not your record of what happened - Secrets: Referenced by name in the definition, resolved at the moment of use, redacted in logs · definitions get exported, shared and pasted into issues - Browser steps: Only where no API exists, on sites whose terms allow it, headed by default · an invisible browser is how you discover three weeks late that it has been failing - Schedules: Keyed on the intended slot rather than the moment the tick woke · a catch-up after downtime should run once, not once per missed minute ### Data model (create this before Phase 1) - workflows(id, name, version, definition_json, active, created_at) · a change is a new version row, because a run must be readable against the definition it used - runs(id, workflow_id, workflow_version, trigger, idempotency_key, state, started_at, finished_at, error) - unique(workflow_id, idempotency_key) · a redelivered trigger produces no second run - step_runs(run_id, step_id, attempt, state, input_hash, output_json, external_ref, started_at, finished_at, error) · external_ref is what makes a repeat safe - schedules(workflow_id, cron_expression, timezone, last_fired_slot, next_due_at) · a slot, not the timestamp the code happened to wake at - artifacts(step_run_id, path, mime, bytes, delete_after) · screenshots and downloads, kept on a stated retention and then deleted - dead_letters(run_id, step_id, at, classification, detail) · where a failure that cannot succeed goes, instead of into a backoff loop - No table holds a secret value. Definitions name a secret and the worker resolves it at the moment of the call. ### Phase 1 · One workflow, declared and validated Build: Define a small set of step kinds, each with a schema for its input and its output. Validate a definition on save and refuse to store an invalid one. Add a dry run that resolves every reference and prints each call it would make without making any of them. Every change to a definition writes a new version. Trap: Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. Done when: An invalid definition is refused at save time with the offending step named, tested with a missing field and a wrong type. A dry run prints every intended call and touches no external service, asserted with a counter that stays at zero. Editing a definition creates a new version and existing runs still resolve against the version they used. No step kind executes arbitrary code, verified by review. Do not build yet: No queue, no retries, no schedules, no logs. ### Phase 2 · Runs that survive being run twice Build: Derive an idempotency key from the triggering event and put a unique constraint on it. Write the attempt row before making any call. Pass a client-generated key where the service accepts one, and record the external reference the service returns. On a retry, a step that already has an external reference is treated as finished. Declare any step that cannot be made idempotent as unsafe to repeat, and halt the run instead of retrying it. Trap: A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. Done when: Triggering twice with the same event identifier creates one run, tested by replaying a webhook payload. Killing the worker between the call and the write leaves an attempt row, and the retry finds the external reference and skips the call, tested against a fake service that counts calls and asserts exactly one. A step declared unsafe to repeat halts the run and records why. Attempts are numbered and never overwrite the previous attempt's row. Do not build yet: No retry classification, logging or schedules yet. ### Phase 3 · Retries that tell a timeout from a refusal Build: Classify every failure as retryable, not retryable, or unknown. Timeouts, connection resets, 429 and 5xx are retryable with exponential backoff and jitter up to a cap; a 4xx that is not 429, a validation error and an authentication failure are not. A non-retryable failure goes to the dead letter table immediately, and an authentication failure pauses the workflow. Trap: Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. Done when: A 400 response is dead-lettered on the first attempt rather than retried, tested. A 429 is retried honouring any interval the response supplies, tested with a stubbed header. Backoff is exponential with jitter and a stated cap, asserted over a sequence of failures. An authentication failure pauses the workflow and alerts once rather than failing every scheduled run. Do not build yet: No logging, redaction, artifacts or replay yet. ### Phase 4 · Logs you can read, secrets you cannot Build: Record per-step structured logs with the input hash, the classified outcome and a redacted output. Redact by the secret's value as well as by declared field name. Store artifacts such as screenshots with a retention and delete them on it. Offer replay from any step using the recorded input, marked clearly as a replay in the run record. Trap: Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL. Done when: A run whose secret appears in a header, a body and a URL produces logs containing none of them, asserted by searching the log output for the value. Replay from a chosen step reuses the recorded input and marks the run as a replay. Artifacts are deleted at the stated retention and the deletion is recorded. Logs identify the workflow version so a run can be read against the definition it used. Do not build yet: No schedules, ceilings or export yet. ### Phase 5 · Schedules, ceilings and the credential that dies Build: Run schedules from a cron expression in a named zone, keyed on the intended slot and recorded as the last slot fired. Cap catch-up after downtime to a stated number of slots. Put a ceiling on steps per run, run duration, and spend on any metered step. Export runs and definitions as files. Show a single status page with paused workflows, dead letters and the last successful run per workflow. Trap: A cron computed in local time either skips the run on the day the clocks go forward or fires it twice on the day they go back, and a catch-up written as run everything missed turns eight hours of downtime into hundreds of runs starting at once. Key on the intended slot, store the last slot fired rather than the last wake time, and cap catch-up explicitly. Done when: Both clock change days fire the documented number of runs, asserted against fixed dates. Eight hours of simulated downtime produces at most the capped number of catch-up runs, tested. A run exceeding the step or duration ceiling stops and is recorded as stopped rather than failed. Export produces definitions and run history that reimport into a fresh instance. Do not build yet: Nothing after this. ### Out of scope (and why) - Hundreds of maintained connectors, which are the actual product being sold here and are maintenance rather than engineering. - Defeating challenges that exist to stop automation, and driving a login on behalf of somebody who did not set it up themselves. - A hosted control plane with teams, roles and shared credentials, which multiplies every problem above by a permission model. - A visual workflow builder, which is a large piece of front end work that changes nothing about whether a run is correct. ### README must contain - That the queue delivers at least once, so every step is either idempotent with a named key or declared unsafe to repeat - That a step with an external reference already recorded is treated as finished rather than due - Which failures are retried and which go straight to the dead letter table, and why - That logs are redacted by secret value as well as by field name, and how that is tested - That schedules are keyed on the intended slot and how much catch-up is allowed after downtime
AGENTS.md
# Agent instructions · Robomotion indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node with a Redis-backed queue and a worker in its own process, Workflows are declared data with typed, validated steps, never embedded code, PostgreSQL holds runs and step results, Redis is only the queue, Referenced by name in the definition, resolved at the moment of use, redacted in logs, Only where no API exists, on sites whose terms allow it, headed by default, Keyed on the intended slot rather than the moment the tick woke. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. - A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. - Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. - Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL.
BUILD_PLAN.md
# Build plan · Robomotion One automation done properly, whose hard problems are surviving a queue that delivers at least once, telling a failure worth retrying from one that will never succeed, and keeping a readable record of every run without writing a secret into it. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · One workflow, declared and validated A workflow is data you can read, and an invalid one is rejected when it is saved. ### Steps 1. Give every step kind an input and output schema A validated boundary is what makes a failure readable later. Files: `src/steps/registry.ts`, `src/steps/schema.ts` 2. Version definitions rather than editing them A run is meaningless without the definition it actually used. Files: `src/workflows/versions.ts`, `migrations/001_workflows.sql` 3. Make the dry run prove it called nothing Asserted with a counter, not with a promise in the README. Files: `src/run/dryrun.ts`, `tests/dryrun.test.ts` ### Done when - [ ] An invalid definition is refused at save time - [ ] A dry run makes zero external calls - [ ] Each edit creates a new version - [ ] No step kind executes arbitrary code ### Watch out - Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. ## Phase 2 · Runs that survive being run twice A duplicate trigger creates no second run, and a repeated step causes no second side effect. ### Steps 1. Constrain the trigger, not just the handler A unique idempotency key per workflow is the cheapest duplicate guard there is. Files: `src/run/trigger.ts`, `migrations/002_runs.sql` 2. Write the attempt before the call Then a crash leaves evidence rather than ambiguity. Files: `src/run/execute.ts`, `tests/at-least-once.test.ts` 3. Record the external reference and honour it A step that already has one is finished, however it looks. Files: `src/run/execute.ts` ### Done when - [ ] A duplicate trigger yields one run - [ ] A killed worker leads to one external call, not two - [ ] An unsafe step halts instead of retrying - [ ] Attempts are numbered and preserved ### Watch out - A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. ## Phase 3 · Retries that tell a timeout from a refusal Only failures that could plausibly succeed later are retried. ### Steps 1. Classify, then decide The decision to retry belongs to the classifier, not the caller. Files: `src/run/classify.ts`, `tests/classify.test.ts` 2. Jitter the backoff Synchronised retries are how a small outage becomes a large one. Files: `src/run/backoff.ts` 3. Pause the workflow on an auth failure One alert beats a week of identical failures. Files: `src/run/pause.ts` ### Done when - [ ] A 400 is dead-lettered on the first attempt - [ ] A 429 honours the interval it is given - [ ] Backoff is exponential with jitter and a cap - [ ] An auth failure pauses and alerts once ### Watch out - Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. ## Phase 4 · Logs you can read, secrets you cannot Every run is reconstructable from the record, and no secret appears anywhere in it. ### Steps 1. Redact by value, not only by field name Secrets end up inside strings you did not design. Files: `src/log/redact.ts`, `tests/redact.test.ts` 2. Keep the input hash rather than the input Enough to prove what ran without storing what it contained. Files: `src/log/record.ts` 3. Make replay explicit in the record A replayed run should never be mistaken for a scheduled one. Files: `src/run/replay.ts` ### Done when - [ ] No secret value appears in any log - [ ] Replay reuses the recorded input and is marked - [ ] Artifacts are deleted on retention with a record - [ ] Logs name the workflow version used ### Watch out - Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL. ## Not in this build - A connector library. - A hosted control plane with teams and roles. - A visual workflow builder. - Any step that defeats an anti-automation measure. ## After v1, if you want it - A second workflow reusing the same step kinds, which is the real test of whether the definitions are data - A webhook trigger alongside the schedule, which the idempotency key already supports - Per-step timeouts tuned from the recorded durations, once a month of runs exists to tune against
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Runs, attempts, dead letters. The record of what happened. DATABASE_URL=postgres://localhost:5432/automation # Required. The queue only. Nothing here is a record. REDIS_URL=redis://localhost:6379 # Required. Keep it low enough that a step never starves the event loop and stalls its own lock. WORKER_CONCURRENCY=2 # Required. Screenshots and downloads, deleted at the stated retention. ARTIFACT_DIR=./data/artifacts # Required. Short, because artifacts are the most likely place a secret is visible. ARTIFACT_RETENTION_DAYS=7 # Required. After downtime. Without this, eight hours down becomes hundreds of runs at once. CATCHUP_MAX_SLOTS=3 # Required. A run exceeding it stops and is recorded as stopped, not failed. RUN_STEP_CEILING=200 # Required · secret. Referenced by name from the definition. Redacted by value in every log. SERVICE_API_KEY=sk-...redacted
PRODUCT.md
# Robomotion · product brief ## Problem The visible robotic process automation loop is buildable, but a credible replacement needs more than the first screen. Robomotion earns its keep through connectors, auth, reliability, so expect a weekend or multi-day build and a narrower personal scope. ## Product outcome One automation that runs on its schedule, does its external work once even when the queue hands the job back, retries only what could succeed, leaves a record you can read without leaking a credential, and pauses loudly when its credential dies. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - hosted PostgreSQL and Redis - API credentials for the two chosen services - Node.js 22 ## Explicit non-goals for v1 - A connector library. - A hosted control plane with teams and roles. - A visual workflow builder. - Any step that defeats an anti-automation measure. - hundreds of maintained connectors - OAuth app verification - durable execution at scale - schema drift handling and enterprise controls ## Success criteria - A killed worker is confirmed to produce one external call, not two - A duplicate trigger is confirmed to produce one run - A 400 is confirmed dead-lettered on the first attempt - No secret value is confirmed present in any log - Both clock change days are confirmed to fire the documented count
BRIEF.md
# Build brief · Robomotion The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a single scheduled automation that retries safely, records what it did, and can be replayed by hand to replace Robomotion. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Assume every step will run twice. The queue behind this build delivers at least once: when a worker stops renewing its lock, because the machine was busy or the process died, the job goes back to waiting and is processed again. That is documented behaviour rather than a bug, and it means a step that raises an invoice or sends an email will do both twice unless something stops it. So every step is declared either idempotent, with the key that makes it so, or unsafe to repeat, and an unsafe step halts the run and asks rather than retrying on its own. The attempt row is written before the call, and a step with an external reference already recorded is done, not due. The second rule is about what you automate at all. Services with an API, and accounts whose credentials you hold. No defeating a challenge that exists to stop automation, no driving somebody else's login on their behalf, no scraping a site whose published terms forbid it. An automation that gets an account closed has cost more than the hours it saved. ### Stack (fixed, do not substitute) - Runtime: Node with a Redis-backed queue and a worker in its own process · the tick and the work must not share a process, because one slow step blocks every schedule - Definition: Workflows are declared data with typed, validated steps, never embedded code · a workflow you cannot read at three in the morning is one you cannot fix at three in the morning - Storage: PostgreSQL holds runs and step results, Redis is only the queue · Redis is a delivery mechanism, not your record of what happened - Secrets: Referenced by name in the definition, resolved at the moment of use, redacted in logs · definitions get exported, shared and pasted into issues - Browser steps: Only where no API exists, on sites whose terms allow it, headed by default · an invisible browser is how you discover three weeks late that it has been failing - Schedules: Keyed on the intended slot rather than the moment the tick woke · a catch-up after downtime should run once, not once per missed minute ### Data model (create this before Phase 1) - workflows(id, name, version, definition_json, active, created_at) · a change is a new version row, because a run must be readable against the definition it used - runs(id, workflow_id, workflow_version, trigger, idempotency_key, state, started_at, finished_at, error) - unique(workflow_id, idempotency_key) · a redelivered trigger produces no second run - step_runs(run_id, step_id, attempt, state, input_hash, output_json, external_ref, started_at, finished_at, error) · external_ref is what makes a repeat safe - schedules(workflow_id, cron_expression, timezone, last_fired_slot, next_due_at) · a slot, not the timestamp the code happened to wake at - artifacts(step_run_id, path, mime, bytes, delete_after) · screenshots and downloads, kept on a stated retention and then deleted - dead_letters(run_id, step_id, at, classification, detail) · where a failure that cannot succeed goes, instead of into a backoff loop - No table holds a secret value. Definitions name a secret and the worker resolves it at the moment of the call. ### Phase 1 · One workflow, declared and validated Build: Define a small set of step kinds, each with a schema for its input and its output. Validate a definition on save and refuse to store an invalid one. Add a dry run that resolves every reference and prints each call it would make without making any of them. Every change to a definition writes a new version. Trap: Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. Done when: An invalid definition is refused at save time with the offending step named, tested with a missing field and a wrong type. A dry run prints every intended call and touches no external service, asserted with a counter that stays at zero. Editing a definition creates a new version and existing runs still resolve against the version they used. No step kind executes arbitrary code, verified by review. Do not build yet: No queue, no retries, no schedules, no logs. ### Phase 2 · Runs that survive being run twice Build: Derive an idempotency key from the triggering event and put a unique constraint on it. Write the attempt row before making any call. Pass a client-generated key where the service accepts one, and record the external reference the service returns. On a retry, a step that already has an external reference is treated as finished. Declare any step that cannot be made idempotent as unsafe to repeat, and halt the run instead of retrying it. Trap: A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. Done when: Triggering twice with the same event identifier creates one run, tested by replaying a webhook payload. Killing the worker between the call and the write leaves an attempt row, and the retry finds the external reference and skips the call, tested against a fake service that counts calls and asserts exactly one. A step declared unsafe to repeat halts the run and records why. Attempts are numbered and never overwrite the previous attempt's row. Do not build yet: No retry classification, logging or schedules yet. ### Phase 3 · Retries that tell a timeout from a refusal Build: Classify every failure as retryable, not retryable, or unknown. Timeouts, connection resets, 429 and 5xx are retryable with exponential backoff and jitter up to a cap; a 4xx that is not 429, a validation error and an authentication failure are not. A non-retryable failure goes to the dead letter table immediately, and an authentication failure pauses the workflow. Trap: Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. Done when: A 400 response is dead-lettered on the first attempt rather than retried, tested. A 429 is retried honouring any interval the response supplies, tested with a stubbed header. Backoff is exponential with jitter and a stated cap, asserted over a sequence of failures. An authentication failure pauses the workflow and alerts once rather than failing every scheduled run. Do not build yet: No logging, redaction, artifacts or replay yet. ### Phase 4 · Logs you can read, secrets you cannot Build: Record per-step structured logs with the input hash, the classified outcome and a redacted output. Redact by the secret's value as well as by declared field name. Store artifacts such as screenshots with a retention and delete them on it. Offer replay from any step using the recorded input, marked clearly as a replay in the run record. Trap: Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL. Done when: A run whose secret appears in a header, a body and a URL produces logs containing none of them, asserted by searching the log output for the value. Replay from a chosen step reuses the recorded input and marks the run as a replay. Artifacts are deleted at the stated retention and the deletion is recorded. Logs identify the workflow version so a run can be read against the definition it used. Do not build yet: No schedules, ceilings or export yet. ### Phase 5 · Schedules, ceilings and the credential that dies Build: Run schedules from a cron expression in a named zone, keyed on the intended slot and recorded as the last slot fired. Cap catch-up after downtime to a stated number of slots. Put a ceiling on steps per run, run duration, and spend on any metered step. Export runs and definitions as files. Show a single status page with paused workflows, dead letters and the last successful run per workflow. Trap: A cron computed in local time either skips the run on the day the clocks go forward or fires it twice on the day they go back, and a catch-up written as run everything missed turns eight hours of downtime into hundreds of runs starting at once. Key on the intended slot, store the last slot fired rather than the last wake time, and cap catch-up explicitly. Done when: Both clock change days fire the documented number of runs, asserted against fixed dates. Eight hours of simulated downtime produces at most the capped number of catch-up runs, tested. A run exceeding the step or duration ceiling stops and is recorded as stopped rather than failed. Export produces definitions and run history that reimport into a fresh instance. Do not build yet: Nothing after this. ### Out of scope (and why) - Hundreds of maintained connectors, which are the actual product being sold here and are maintenance rather than engineering. - Defeating challenges that exist to stop automation, and driving a login on behalf of somebody who did not set it up themselves. - A hosted control plane with teams, roles and shared credentials, which multiplies every problem above by a permission model. - A visual workflow builder, which is a large piece of front end work that changes nothing about whether a run is correct. ### README must contain - That the queue delivers at least once, so every step is either idempotent with a named key or declared unsafe to repeat - That a step with an external reference already recorded is treated as finished rather than due - Which failures are retried and which go straight to the dead letter table, and why - That logs are redacted by secret value as well as by field name, and how that is tested - That schedules are keyed on the intended slot and how much catch-up is allowed after downtime
ARCHITECTURE.md
# Architecture · Robomotion ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node with a Redis-backed queue and a worker in its own process | the tick and the work must not share a process, because one slow step blocks every schedule | | Definition | Workflows are declared data with typed, validated steps, never embedded code | a workflow you cannot read at three in the morning is one you cannot fix at three in the morning | | Storage | PostgreSQL holds runs and step results, Redis is only the queue | Redis is a delivery mechanism, not your record of what happened | | Secrets | Referenced by name in the definition, resolved at the moment of use, redacted in logs | definitions get exported, shared and pasted into issues | | Browser steps | Only where no API exists, on sites whose terms allow it, headed by default | an invisible browser is how you discover three weeks late that it has been failing | | Schedules | Keyed on the intended slot rather than the moment the tick woke | a catch-up after downtime should run once, not once per missed minute | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Definitions | Typed steps, validation and versioning. | Any format, provided steps stay declared data rather than code. | | Executor | Attempt rows, external references and unsafe-step halts. | Not swappable for call-then-record, which is where the duplicate invoice comes from. | | Classifier | Retryable versus dead letter, backoff and jitter. | Any policy, provided a non-retryable failure never enters a backoff loop. | | Record | Redacted logs, artifacts and replay. | Any sink, provided redaction is by value as well as by field name. | | Scheduler | Slot-keyed ticks and capped catch-up. | Any cron, provided both clock change days stay tested. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DATABASE_URL` · required · Runs, attempts, dead letters. The record of what happened. - `REDIS_URL` · required · The queue only. Nothing here is a record. - `WORKER_CONCURRENCY` · required · Keep it low enough that a step never starves the event loop and stalls its own lock. - `ARTIFACT_DIR` · required · Screenshots and downloads, deleted at the stated retention. - `ARTIFACT_RETENTION_DAYS` · required · Short, because artifacts are the most likely place a secret is visible. - `CATCHUP_MAX_SLOTS` · required · After downtime. Without this, eight hours down becomes hundreds of runs at once. - `RUN_STEP_CEILING` · required · A run exceeding it stops and is recorded as stopped, not failed. - `SERVICE_API_KEY` · required, secret · Referenced by name from the definition. Redacted by value in every log. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · Robomotion product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Node with a Redis-backed queue and a worker in its own process, Workflows are declared data with typed, validated steps, never embedded code, PostgreSQL holds runs and step results, Redis is only the queue, Referenced by name in the definition, resolved at the moment of use, redacted in logs, Only where no API exists, on sites whose terms allow it, headed by default, Keyed on the intended slot rather than the moment the tick woke. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. - A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. - Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. - Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL. - A cron computed in local time either skips the run on the day the clocks go forward or fires it twice on the day they go back, and a catch-up written as run everything missed turns eight hours of downtime into hundreds of runs starting at once. Key on the intended slot, store the last slot fired rather than the last wake time, and cap catch-up explicitly.
MILESTONES.md
# Delivery milestones · Robomotion Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · One workflow, declared and validated A workflow is data you can read, and an invalid one is rejected when it is saved. ### Steps 1. Give every step kind an input and output schema A validated boundary is what makes a failure readable later. Files: `src/steps/registry.ts`, `src/steps/schema.ts` 2. Version definitions rather than editing them A run is meaningless without the definition it actually used. Files: `src/workflows/versions.ts`, `migrations/001_workflows.sql` 3. Make the dry run prove it called nothing Asserted with a counter, not with a promise in the README. Files: `src/run/dryrun.ts`, `tests/dryrun.test.ts` ### Done when - [ ] An invalid definition is refused at save time - [ ] A dry run makes zero external calls - [ ] Each edit creates a new version - [ ] No step kind executes arbitrary code ### Watch out - Letting a workflow carry a snippet of code means the first failure at three in the morning is a stack trace in a text column with no way to see what input produced it, and the fix requires a deploy. Declare step kinds with validated schemas on both sides, reject an invalid definition at save time, and make the dry run print every call it would have made. ## M2 · Runs that survive being run twice A duplicate trigger creates no second run, and a repeated step causes no second side effect. ### Steps 1. Constrain the trigger, not just the handler A unique idempotency key per workflow is the cheapest duplicate guard there is. Files: `src/run/trigger.ts`, `migrations/002_runs.sql` 2. Write the attempt before the call Then a crash leaves evidence rather than ambiguity. Files: `src/run/execute.ts`, `tests/at-least-once.test.ts` 3. Record the external reference and honour it A step that already has one is finished, however it looks. Files: `src/run/execute.ts` ### Done when - [ ] A duplicate trigger yields one run - [ ] A killed worker leads to one external call, not two - [ ] An unsafe step halts instead of retrying - [ ] Attempts are numbered and preserved ### Watch out - A worker that stops renewing its lock has its job moved back to waiting and processed again, so a step heavy enough to starve the event loop, or a machine that swapped for thirty seconds, produces a second invoice, and the record shows one run because the second attempt overwrote the first row. Write the attempt before the call, send a client key where the service takes one, and treat a recorded external reference as proof the work is done. ## M3 · Retries that tell a timeout from a refusal Only failures that could plausibly succeed later are retried. ### Steps 1. Classify, then decide The decision to retry belongs to the classifier, not the caller. Files: `src/run/classify.ts`, `tests/classify.test.ts` 2. Jitter the backoff Synchronised retries are how a small outage becomes a large one. Files: `src/run/backoff.ts` 3. Pause the workflow on an auth failure One alert beats a week of identical failures. Files: `src/run/pause.ts` ### Done when - [ ] A 400 is dead-lettered on the first attempt - [ ] A 429 honours the interval it is given - [ ] Backoff is exponential with jitter and a cap - [ ] An auth failure pauses and alerts once ### Watch out - Retrying every failure on a tight backoff turns one rejected payload into four hundred identical rejected requests and then a rate limit block, and the original error is buried under its own retries. Classify before retrying, send a 4xx straight to the dead letter table, and add jitter so ten runs failing at the same moment do not come back in lockstep. ## M4 · Logs you can read, secrets you cannot Every run is reconstructable from the record, and no secret appears anywhere in it. ### Steps 1. Redact by value, not only by field name Secrets end up inside strings you did not design. Files: `src/log/redact.ts`, `tests/redact.test.ts` 2. Keep the input hash rather than the input Enough to prove what ran without storing what it contained. Files: `src/log/record.ts` 3. Make replay explicit in the record A replayed run should never be mistaken for a scheduled one. Files: `src/run/replay.ts` ### Done when - [ ] No secret value appears in any log - [ ] Replay reuses the recorded input and is marked - [ ] Artifacts are deleted on retention with a record - [ ] Logs name the workflow version used ### Watch out - Logging the resolved step input to make debugging easier writes an API key into a file that is later pasted into an issue, and because the key was interpolated into a query string the redaction pass that matched on field names never saw it. Redact by value as well as by name, and test with a run whose secret appears in a header, a body and a URL. ## M5 · Schedules, ceilings and the credential that dies (production only) It runs on time, stops before it costs too much, and says so when a credential expires. ### Steps 1. Key schedules on the slot The last slot fired, not the last time the process woke. Files: `src/schedule/tick.ts`, `tests/dst.test.ts` 2. Cap catch-up and say so Downtime should not become a thundering herd. Files: `src/schedule/catchup.ts` 3. One status page with the four numbers that matter Paused workflows, dead letters, last success, spend against ceiling. Files: `src/ui/status.tsx` ### Done when - [ ] Both clock change days fire the documented count - [ ] Catch-up after downtime respects the cap - [ ] Ceilings stop a run and record it as stopped - [ ] Export reimports into a fresh instance ### Watch out - A cron computed in local time either skips the run on the day the clocks go forward or fires it twice on the day they go back, and a catch-up written as run everything missed turns eight hours of downtime into hundreds of runs starting at once. Key on the intended slot, store the last slot fired rather than the last wake time, and cap catch-up explicitly.
OPERATIONS.md
# Operations · Robomotion ## Backup The Postgres database, which holds runs, attempts, dead letters and definitions. Redis needs no backup by design, and if that is not true of your build then Redis has become a record and should not be. ## Restore Restore and start the worker with every run left in progress moved to a state that requires reconciliation rather than retry, so nothing is repeated on the assumption that it failed. Check external references before resuming anything. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Dead letter count, paused workflows, time since the last successful run per workflow, and spend against the ceiling. A rising dead letter count with a flat failure count usually means a downstream contract changed. ## Incident checklist A duplicated side effect means the attempt row was written after the call, and the attempts table shows the two rows. A workflow that quietly stopped is nearly always a paused credential, which is why the status page leads with paused workflows. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] A killed worker is confirmed to produce one external call, not two - [ ] A duplicate trigger is confirmed to produce one run - [ ] A 400 is confirmed dead-lettered on the first attempt - [ ] No secret value is confirmed present in any log - [ ] Both clock change days are confirmed to fire the documented count ## Launch constraint Do not market omitted Robomotion capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Runs, attempts, dead letters. The record of what happened. DATABASE_URL=postgres://localhost:5432/automation # Required. The queue only. Nothing here is a record. REDIS_URL=redis://localhost:6379 # Required. Keep it low enough that a step never starves the event loop and stalls its own lock. WORKER_CONCURRENCY=2 # Required. Screenshots and downloads, deleted at the stated retention. ARTIFACT_DIR=./data/artifacts # Required. Short, because artifacts are the most likely place a secret is visible. ARTIFACT_RETENTION_DAYS=7 # Required. After downtime. Without this, eight hours down becomes hundreds of runs at once. CATCHUP_MAX_SLOTS=3 # Required. A run exceeding it stops and is recorded as stopped, not failed. RUN_STEP_CEILING=200 # Required · secret. Referenced by name from the definition. Redacted by value in every log. SERVICE_API_KEY=sk-...redacted
$ choose a build depth, inspect the files, then open the complete pack in your agent
Robomotion: One automation is easy. Subscribers pay for maintained integrations, credentials, retries, observability, and someone else owning breakage.
xhundreds of maintained connectors
xOAuth app verification
xdurable execution at scale
xschema drift handling and enterprise controls
share on X ↗"I just replaced Robomotion with one prompt"
Don't feel like building it? These folks already made it free.
UI.Vision RPABrowser and desktop macros with OCR and spreadsheets; cloud workers are replaced by the machine in front of you.open source↗
Power Automate for desktopWindows gets a free attended RPA designer; unattended cloud robots still report to accounting.free↗no votes, no pay-to-list · just what's real
Robomotion pricing
| plan | monthly | annual (per mo) | what you get |
|---|---|---|---|
| free | $0/workspace | $0/workspace | 50 cloud credits refreshed daily plus 250 one-time credits; 1 GB storage. |
| solo | $49/workspace | $39.20/workspace | 1 user; 1 development robot; 3 app robots; 50 flows; 2,500 AI credits; 50,000 proxy requests; 80 cloud hours; 25 GB. |
| pro | $129/workspace | $103.20/workspace | 3+ users; 3 development robots; 1 production robot; 5 app robots; 250 flows; 7,500 AI credits; 150,000 proxy requests; 240 cloud hours; 250 GB. |
| team | $399/workspace | $319.20/workspace | 5+ users; 5 development robots; 2 production robots; 15 app robots; unlimited flows; 25,000 AI credits; 500,000 proxy requests; 720 cloud hours; 1 TB. |
| self-hosted starter | $499 | — | Self-hosted control plane; current public page does not state numeric robot, user or flow caps. |
| self-hosted growth | $999 | — | Larger self-hosted deployment; current public page does not state numeric robot, user or flow caps. |
| self-hosted enterprise | custom | — | Custom self-hosted scale, security and support. |
free tier50 cloud credits per day plus 250 one-time credits and 1 GB storage; no paid top-ups on Free
billingmonthly + annual for cloud plans (annual is 20% lower); self-hosted Starter and Growth are monthly
hidden costsAdd-ons are $9/month per user, $19 per app robot, $49 per development robot and $99 per production robot. AI usage is valued at $0.01 per AI credit; higher automation, proxy and cloud-runtime use can require add-ons or an upgrade.
verified 2026-08-11 · source ↗
Vibecode Robomotion
Kinda. The core of Robomotion is buildable in a weekend with the prompt on this page, but there are real gaps: hundreds of maintained connectors, OAuth app verification. Read the honest list above before committing.
How much does Robomotion cost?
Robomotion's pricing is usage-based or varies by plan · The official pricing URL is recorded, but a stable current amount was not safely recoverable in this pass. Do not publish a number until rechecked..
What do I lose by replacing Robomotion?
Honestly: hundreds of maintained connectors; OAuth app verification; durable execution at scale; schema drift handling and enterprise controls. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to Robomotion?
Yes: UI.Vision RPA (Browser and desktop macros with OCR and spreadsheets; cloud workers are replaced by the machine in front of you.) Power Automate for desktop (Windows gets a free attended RPA designer; unattended cloud robots still report to accounting.) The prompt is for when you want it exactly your way.