Build Lindy

0%0 of 17 items done

Saved on this device only. Tick prerequisites first, then work the phases in order · do not start one until the checks above it pass.

A single-workflow automation runner whose hard problems are surviving at-least-once delivery without repeating a side effect, keeping connector credentials alive past the week they were issued, and making a failed run readable enough to replay rather than guess at.

estimated effort weekendthe files for this build are in the project pack

RuntimeA Node worker process plus a small web interface over PostgresQueueRedis-backed jobs with an explicit idempotency key written per side effectStepsTyped step definitions with a validated input and output schemaCredentialsOAuth tokens encrypted at rest with the expiry stored beside themModel stepOne call, structured output, schema-checked, never free text into the next stepReplayThe trigger payload stored verbatim so a run can be repeated without the trigger

Before step 1

Everything below is assumed from the first step. Tick each one when you actually have it, not when you plan to.

installfree
Why Runs the worker and the interface.
Get it nodejs.org or a version manager. open ↗
Verify node --version prints v22 or higher
installfree locally, a few pounds a month hosted
Why Postgres holds runs and the side-effect ledger, Redis holds the queue, and the separation is deliberate because one of them must survive the other.
Get it Docker Compose locally, or a small managed pair.
Verify A job enqueued in Redis produces a run row in Postgres and survives a Redis restart
API keyfree
Why The whole build is shaped by how those two authenticate, and finding out that one of them issues short-lived tokens after you have written the worker is the expensive order to do this in.
Get it Register an application with each and record the token lifetime from their own documentation.
Verify A token is issued and its expires_at is stored
decidefree, though verification costs time
Why A consent screen left in testing can mean a refresh token that expires within days, which turns a working automation into a weekly chore, and the alternative is a verification process with its own cost in time.
Get it Read the provider's current documentation on refresh token expiry for testing projects and choose deliberately.
Verify The choice and its consequence are written in the README
have readyfree
Why Dry runs, replay tests and the whole test suite need a real payload, and capturing one before you build is much easier than reproducing one afterwards.
Get it Point the webhook at a request bin once and save the raw body.
Verify The suite runs end to end from the fixture with no network access

Data model

Create these before the first phase that stores anything. Changing a table later is the expensive kind of change.

- workflows(id, name, version, trigger_kind, trigger_config_json, enabled, updated_at)
- runs(id, workflow_id, workflow_version, trigger_payload_json, started_at, ended_at, state, replay_of_run_id, dry_run)
- steps(id, run_id, ordinal, step_type, input_json, output_json, state, attempts, started_at, ended_at, error_json)
- side_effects(id, run_id, step_id, idempotency_key, external_ref, at) · unique on idempotency_key, and the only record that something already happened out in the world
- credentials(id, provider, account, access_token_enc, refresh_token_enc, expires_at, scopes_json, last_refreshed_at, status)
- dead_letters(id, run_id, step_id, at, reason, payload_json) · where a run goes when retrying would be guessing
- workflow_version is copied into every run. Editing a workflow must never change what an old run says it did, which is why the definition is versioned rather than updated in place.

Environment variables

These go in a .env file the app reads at startup. The pack's .env.example is this table as a file · copy it, never commit the filled-in version.

Variable	Needed	Example	Where the value comes from
`DATABASE_URL`secret	required	`postgres://localhost:5432/automation`	Runs, steps and the side-effect ledger. This is the one that must not be lost.
`REDIS_URL`secret	required	`redis://localhost:6379`	The queue. Losing it loses queued work, not history.
`CREDENTIAL_ENC_KEY`secret	required	`base64:...redacted`	Encrypts connector tokens at rest. Rotating it means re-encrypting every row.
`WEBHOOK_SIGNING_SECRET`secret	required	`whsec_...redacted`	Verified against the raw body, not the parsed object.
`WEBHOOK_TOLERANCE_SECONDS`	required	`300`	Payloads outside this window are rejected as stale.
`TOKEN_WARN_DAYS`	required	`2`	How early a dying connector token raises an alarm.
`MONTHLY_COST_CEILING`	required	`10.00`	Model spend for the month. At the ceiling the workflow pauses.
`KILL_SWITCH`	required	`off`	Set to on and the worker stops claiming jobs within seconds.

The build, in order

1
One trigger, one run, kept verbatim
A trigger creates a run row holding the exact payload that caused it, before any step executes.
1. Signature verification and replay both need the original bytes.
  Files src/triggers/webhook.tstests/signature.test.ts
2. An old run must keep describing what it actually did.
  Files src/db/schema.sqlsrc/runs/create.ts
3. A replayed webhook from last month is an attack, not a trigger.
  Files src/triggers/webhook.ts
done when · tick each as it passes
watch out
- Parsing the webhook body and storing the parsed object loses the raw bytes you need to verify the signature, and signature verification against a re-serialised body fails for reasons that take an evening to find; keep the raw body, verify against it, and store it as received so a replay months later feeds the step exactly what the step saw the first time.
next phase
2
Steps that survive being run twice
Every side effect happens exactly once even when the queue delivers the job three times.
1. Stable across retries, different across inputs.
  Files src/steps/idempotency.tstests/idempotency.test.ts
2. The order is the whole mechanism.
  Files src/steps/run-step.ts
3. You are going to be redelivered whether you planned for it or not.
  Files docs/queue-behaviour.md
done when · tick each as it passes
watch out
- A worker that loses its lock while busy has its job marked stalled, and the documented behaviour is that the job is moved back to waiting and processed again by another worker, so a step that sends an email or creates an invoice does it twice while your code never saw an error; check for the idempotency key first, write it with the result in one transaction, and confirm the default stall settings in the queue library you chose rather than assuming they give you room.
next phase
3
The model step, typed on both sides
A model call either produces schema-valid structured data or stops the run.
1. The boundary is the only place this is cheap to catch.
  Files src/steps/model.tstests/model-schema.test.ts
2. So a different answer next month has an explanation.
  Files src/db/schema.sql
3. A third attempt at the same bad shape is just money.
  Files src/steps/model.ts
done when · tick each as it passes
watch out
- Letting a model step return a paragraph and passing it to the next step means the automation fails somewhere downstream with a message about an undefined property, and because the model usually returns something reasonable it will pass your testing and break in week three; validate against the schema at the step boundary, treat a validation miss as its own failure kind, and never let unvalidated text become a step output.
next phase
4
Credentials, and the seven day cliff
A connector token's expiry is known in advance and never discovered at run time.
1. A token with an unknown expiry is a token that will surprise you.
  Files src/credentials/store.tstests/expiry.test.ts
2. One alarm beats forty identical run failures.
  Files src/credentials/watch.ts
3. Verification or weekly reauthorisation. Both are fine; drifting is not.
  Files docs/oauth-decision.md
done when · tick each as it passes
watch out
- A personal automation against a large provider usually runs on an application whose consent screen is still in testing, and that provider's own documentation says a project with an external user type and a publishing status of testing is issued a refresh token that expires in seven days unless the scopes are a subset of name, email and profile, so your automation dies every single week for reasons that look like your code; store the expiry, alarm on it, and decide deliberately whether to go through verification or to accept a weekly reauthorisation and automate the reminder.
5
Reading a run, replaying it, stopping itproduct builder
Any failed run can be understood from its timeline and safely re-run, and the whole thing can be stopped in one action.
1. The debug path must not be the destructive path.
  Files src/runs/replay.tstests/replay-dry.test.ts
2. Skipping what already happened is the point.
  Files src/runs/replay.ts
3. Both are things you will want at two in the morning.
  Files src/worker/control.tssrc/lib/cost.ts
done when · tick each as it passes
watch out
- A replay button that simply re-runs the workflow performs every side effect again, so debugging a failed run by replaying it sends the customer a second email and creates a second record; make dry run the default, reuse the original idempotency keys on a real replay so anything already done is skipped, and show a diff of what would change before anything runs.

what this build does not replace

A library of maintained connectors.
Bulk email from your own domain.
A visual workflow builder.
Multi-tenant credentials and approval chains.
schema drift handling and enterprise controls
sender reputation, deliverability, and suppression handling
hundreds of maintained connectors
OAuth app verification
durable execution at scale

after v1, if you want it

A second workflow, which is the only real test of whether your step types were general or accidental
A weekly digest of runs, failures and spend, built from the tables you already have
Recorded fixtures captured automatically from production runs, so the test suite grows from real payloads

Need the files? The project pack on the verdict page hands your agent the whole brief · more automation.