Build Activepieces Cloud

0%0 of 17 items done

Saved on this device only. Tick prerequisites first, then work the phases in order · do not start one until the checks above it pass.

A workflow automation engine whose entire difficulty is failure: a flow of six steps that fails on the fourth must be resumable without repeating the first three, and every step that touches the outside world must be safe to retry.

estimated effort multi-daythe files for this build are in the project pack

ExecutionDurable step-by-step runs with each result persistedIdempotencyA key per external effect, supplied by the engineTriggersWebhooks and polling with deduplication at the boundaryIsolationStep code runs sandboxed with declared network accessRuntimeTypeScript with PostgreSQL and a worker pool

Before step 1

Everything below is assumed from the first step. Tick each one when you actually have it, not when you plan to.

installfree
Why Runs the engine and the worker pool.
Get it nodejs.org, or nvm install 22. open ↗
Verify node --version prints v22 or higher
installfree locally
Why Durable run state, deduplication and cursors all want real transactions.
Get it Docker or your package manager. open ↗
Verify psql connects and a unique constraint enforces trigger deduplication
installfree
Why Steps are user-written code running on your infrastructure, and running them in the worker process is not something to do once.
Get it A container or isolate runtime with network and filesystem restriction per execution.
Verify A hostile step cannot read the host filesystem or reach an undeclared address
accountfree with existing accounts
Why Retry safety differs entirely between a service that accepts an idempotency key and one that does not, and both cases need building.
Get it Accounts on two services you would actually automate against, checking each one's documentation for idempotency support.
Verify One accepts an idempotency key and one does not, and both are handled
decidefree
Why Whether an ambiguous outcome retries or halts is a decision about which failure you prefer, and it must be made before the runner exists rather than during an incident.
Get it Decide the retry counts, the backoff and what happens on an ambiguous outcome per step kind.
Verify A written policy covers retryable errors, ambiguous outcomes and the maximum attempts

Data model

Create these before the first phase that stores anything. Changing a table later is the expensive kind of change.

- flows(id, name, definition_json, version, enabled, created_at)
- runs(id, flow_id, flow_version, trigger_id, state, started_at, finished_at, failed_step_id)
- step_results(run_id, step_id, attempt, state, input_hash, output_json, idempotency_key, started_at, finished_at, error)
- triggers(id, flow_id, kind, config_json, cursor, dedupe_key_field)
- trigger_events(id, trigger_id, external_key, received_at, run_id) · deduplicated on the external key
- connections(id, kind, credentials_ref, scopes_json, expires_at) · never stored inline in a flow definition
- A run pins the flow version it started with, because editing a flow mid-run would otherwise change the steps underneath an execution already in progress.

Environment variables

These go in a .env file the app reads at startup. The pack's .env.example is this table as a file · copy it, never commit the filled-in version.

Variable	Needed	Example	Where the value comes from
`DATABASE_URL`secret	required	`postgres://localhost:5432/flows`	Run state, step outputs and connection references.
`STEP_TIMEOUT_SECONDS`	required	`60`	Per step. A hanging step should fail the run rather than the worker.
`MAX_ATTEMPTS`	required	`5`	Then stop and surface. Retrying forever is a queue that never drains.
`AMBIGUOUS_OUTCOME`	required	`halt`	A timeout after a possible success is not a retry candidate.
`STEP_OUTPUT_RETENTION_DAYS`	required	`14`	Debugging needs them and they contain other people's data.
`INJECT_UNDECLARED_CONNECTIONS`	required	`never`	Present as a constant. Flows get shared and ambient credentials go with them.

The build, in order

1
A run that survives a restart
Killing the worker mid-flow does not lose or repeat work.
1. In-memory state loses every run in progress on a deployment.
  Files src/runner.ts
2. Editing a flow must not change steps underneath a running execution.
  Files src/runs.ts
3. Between steps, repeatedly. Not by simulating it.
  Files tests/resume.test.ts
done when · tick each as it passes
watch out
- Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory.
next phase
2
Retrying without doing it twice
A step that timed out after succeeding does not repeat its effect.
1. Stable across attempts of the same step, different across steps.
  Files src/idempotency.ts
2. So an ambiguous outcome is visible rather than assumed failed.
  Files src/runner.ts
3. A validation error retried forever is a queue that never drains.
  Files src/errors.tssrc/errors.test.ts
done when · tick each as it passes
watch out
- A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry.
next phase
3
Triggers that fire once
A duplicate webhook or a re-polled item does not run the flow twice.
1. Before starting a run, not inside it.
  Files src/triggers.tstests/dedupe.test.ts
2. Losing an item is worse than seeing it twice, given deduplication.
  Files src/poll.ts
3. A stated rule rather than an assumption of uniqueness.
  Files src/triggers.ts
done when · tick each as it passes
watch out
- Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative.
next phase
4
Running steps you did not write
A step's code cannot reach anything it did not declare.
1. Flows get shared. Ambient credentials go with them.
  Files src/sandbox.tstests/isolation.test.ts
2. Undeclared access fails rather than warning.
  Files src/sandbox.ts
3. Step outputs are logged and shown. Credentials must not be in them.
  Files src/redact.ts
done when · tick each as it passes
watch out
- Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn.
5
Seeing what a run actually didproduct builder
A failed run can be diagnosed and replayed from the failing step.
1. Replaying from the start re-sends whatever step one sent.
  Files src/replay.ts
2. Debugging needs them and they contain other people's data.
  Files src/observability.ts
3. A flow failing every hour for a week is normally nobody's job.
  Files src/health.ts
done when · tick each as it passes
watch out
- Replaying a failed run from the beginning re-executes steps that already succeeded, which for a step that sent a message or charged a card is a second effect caused by your debugging. Replay from the failure using stored earlier results, and make replaying from the start a separate and clearly labelled action.

what this build does not replace

Breadth of integrations.
A builder for non-technical users.
Multi-tenant noisy neighbour isolation.
Exactly-once delivery to services that do not support it.
managed hosting, connector updates, execution capacity, support, and reliability
thousands of connectors
maintained OAuth applications
schema-drift handling
high-scale durable execution and support

after v1, if you want it

Scheduled flows with a missed-window policy, since a scheduler that catches up floods everything
Branching and looping, which multiply every resumption question here
A test mode that runs a flow with side effects disabled, which is what people actually want

Need the files? The project pack on the verdict page hands your agent the whole brief · more automation.