Vibecode Activepieces Cloud
track this build4 phases, 12 steps, beginner friendly0%KINDA · weekend project
price variesbuild time closest consolation build: multi-daycategory 🤖 automationreplaced by 0 people
The core loop is buildable, but a dependable replacement becomes a real weekend or multi-day project. For Activepieces Cloud, use a managed open-source workflow runner instead of operating queues and upgrades. The hard boundary is managed hosting, connector updates, execution capacity, support, and reliability, plus connector breadth and execution reliability.
the project pack5 files · written for this build, step by step
You are building a lean indie version of Activepieces Cloud. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Activepieces Cloud · indie build A workflow automation engine whose entire difficulty is failure: a flow of six steps that fails on the fourth must be resumable without repeating the first three, and every step that touches the outside world must be safe to retry. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Execution | Durable step-by-step runs with each result persisted | resuming needs the previous results, which means storing them | | Idempotency | A key per external effect, supplied by the engine | retrying a step that already sent an email is the failure that matters | | Triggers | Webhooks and polling with deduplication at the boundary | a duplicate trigger runs the whole flow again | | Isolation | Step code runs sandboxed with declared network access | steps are user-written and run on your infrastructure | | Runtime | TypeScript with PostgreSQL and a worker pool | runs are stateful, long and must survive a restart | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the engine and the worker pool. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **PostgreSQL 16 or newer** · free locally - Why: Durable run state, deduplication and cursors all want real transactions. - Get it: Docker or your package manager. - Verify: psql connects and a unique constraint enforces trigger deduplication - [ ] **A sandbox for executing step code** · free - Why: Steps are user-written code running on your infrastructure, and running them in the worker process is not something to do once. - Get it: A container or isolate runtime with network and filesystem restriction per execution. - Verify: A hostile step cannot read the host filesystem or reach an undeclared address - [ ] **Two real services, one with idempotency support** · free with existing accounts - Why: Retry safety differs entirely between a service that accepts an idempotency key and one that does not, and both cases need building. - Get it: Accounts on two services you would actually automate against, checking each one's documentation for idempotency support. - Verify: One accepts an idempotency key and one does not, and both are handled - [ ] **Your retry and ambiguity policy** · free - Why: Whether an ambiguous outcome retries or halts is a decision about which failure you prefer, and it must be made before the runner exists rather than during an incident. - Get it: Decide the retry counts, the backoff and what happens on an ambiguous outcome per step kind. - Verify: A written policy covers retryable errors, ambiguous outcomes and the maximum attempts ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Breadth of integrations. - A builder for non-technical users. - Multi-tenant noisy neighbour isolation. - Exactly-once delivery to services that do not support it. - managed hosting, connector updates, execution capacity, support, and reliability - thousands of connectors - maintained OAuth applications - schema-drift handling - high-scale durable execution and support If one of those is essential to you, that is the reason to keep paying for Activepieces Cloud, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Activepieces Cloud The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a workflow runner where a step that half succeeded does not run twice to replace Activepieces Cloud. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Execution: Durable step-by-step runs with each result persisted · resuming needs the previous results, which means storing them - Idempotency: A key per external effect, supplied by the engine · retrying a step that already sent an email is the failure that matters - Triggers: Webhooks and polling with deduplication at the boundary · a duplicate trigger runs the whole flow again - Isolation: Step code runs sandboxed with declared network access · steps are user-written and run on your infrastructure - Runtime: TypeScript with PostgreSQL and a worker pool · runs are stateful, long and must survive a restart ### Data model (create this before Phase 1) - flows(id, name, definition_json, version, enabled, created_at) - runs(id, flow_id, flow_version, trigger_id, state, started_at, finished_at, failed_step_id) - step_results(run_id, step_id, attempt, state, input_hash, output_json, idempotency_key, started_at, finished_at, error) - triggers(id, flow_id, kind, config_json, cursor, dedupe_key_field) - trigger_events(id, trigger_id, external_key, received_at, run_id) · deduplicated on the external key - connections(id, kind, credentials_ref, scopes_json, expires_at) · never stored inline in a flow definition - A run pins the flow version it started with, because editing a flow mid-run would otherwise change the steps underneath an execution already in progress. ### Phase 1 · A run that survives a restart Build: Persist each step's result before moving to the next, so a run resumes from the last completed step. Pin the flow version at run start. Verify by killing the worker between steps and restarting. Trap: Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. Done when: Killing the worker between steps and restarting resumes the run from the last completed step, tested repeatedly. Completed steps are not re-executed on resume. Editing a flow does not change a run already in progress, because the version is pinned. A run whose flow was deleted completes or fails cleanly rather than hanging. Do not build yet: No idempotency, triggers or sandboxing. ### Phase 2 · Retrying without doing it twice Build: Supply an idempotency key per step attempt and pass it to services that accept one. For those that do not, record the attempt before making the call so an ambiguous outcome is detectable. Classify errors into retryable and not. Trap: A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. Done when: A step that timed out after its effect succeeded does not repeat it where the service supports idempotency keys. Where it does not, the ambiguous outcome is surfaced rather than retried blindly, tested by injecting a timeout after success. Errors are classified and only retryable classes retry. Retries use backoff with a cap and then stop. Do not build yet: No triggers or sandboxing yet. ### Phase 3 · Triggers that fire once Build: Deduplicate incoming triggers on an external key before starting a run. Keep a polling cursor per trigger that advances only on success. Handle a webhook delivered three times and a poll that returns overlapping pages. Trap: Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. Done when: A webhook delivered three times starts one run, tested directly. A polling cursor advances only after the triggered run is durably recorded. Overlapping poll pages do not produce duplicate runs. A trigger whose external key is missing is handled by a stated rule rather than assumed unique. Do not build yet: No sandboxing or connections yet. ### Phase 4 · Running steps you did not write Build: Execute step code sandboxed with no ambient credentials and network access limited to declared destinations. Inject only the connections the flow declares. Enforce a timeout and a memory ceiling per step. Trap: Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn. Done when: A step cannot access a connection the flow did not declare, tested with a deliberately hostile step. Network access outside declared destinations fails. Steps terminate at a timeout and a memory ceiling with the run marked failed rather than the worker dying. Credentials do not appear in run logs or step outputs. Do not build yet: No observability or versioning yet. ### Phase 5 · Seeing what a run actually did Build: Store each step's input and output for a retention period, redacted. Support replaying a run from its failed step with the earlier results reused. Report flow health per version and alert on a flow failing repeatedly. Trap: Replaying a failed run from the beginning re-executes steps that already succeeded, which for a step that sent a message or charged a card is a second effect caused by your debugging. Replay from the failure using stored earlier results, and make replaying from the start a separate and clearly labelled action. Done when: A failed run replays from its failed step reusing earlier results, verified against a flow whose first step has a visible effect. Step inputs and outputs are retained for a stated period with credentials redacted. A flow failing repeatedly alerts rather than accumulating silently. Health is reported per flow version so a bad edit is identifiable. Do not build yet: Nothing after this. ### Out of scope (and why) - Breadth of integrations, which is what these platforms actually sell and is an endless maintenance commitment. - A visual builder good enough for non-technical users, which is most of the product's engineering. - Multi-tenant scale with noisy neighbour isolation, which is where running this as a service becomes hard. - Exactly-once delivery to services that do not support it, which is not achievable and should not be claimed. ### README must contain - That every step result is persisted so a run resumes rather than restarting - That a run pins its flow version, so editing a flow cannot change a running execution - That an ambiguous step outcome is surfaced rather than retried, because a retry may send a second message - That only connections the flow declares are injected into its steps - That replay resumes from the failed step rather than from the beginning ===== AGENTS.md ===== # Agent instructions · Activepieces Cloud indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Durable step-by-step runs with each result persisted, A key per external effect, supplied by the engine, Webhooks and polling with deduplication at the boundary, Step code runs sandboxed with declared network access, TypeScript with PostgreSQL and a worker pool. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. - A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. - Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. - Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn. ===== BUILD_PLAN.md ===== # Build plan · Activepieces Cloud A workflow automation engine whose entire difficulty is failure: a flow of six steps that fails on the fourth must be resumable without repeating the first three, and every step that touches the outside world must be safe to retry. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · A run that survives a restart Killing the worker mid-flow does not lose or repeat work. ### Steps 1. Persist after every step In-memory state loses every run in progress on a deployment. Files: `src/runner.ts` 2. Pin the flow version at run start Editing a flow must not change steps underneath a running execution. Files: `src/runs.ts` 3. Test by killing the worker Between steps, repeatedly. Not by simulating it. Files: `tests/resume.test.ts` ### Done when - [ ] A killed worker resumes from the last completed step - [ ] Completed steps are not re-executed - [ ] Flow edits do not affect running executions - [ ] A deleted flow's run resolves cleanly ### Watch out - Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. ## Phase 2 · Retrying without doing it twice A step that timed out after succeeding does not repeat its effect. ### Steps 1. Generate an idempotency key per step Stable across attempts of the same step, different across steps. Files: `src/idempotency.ts` 2. Record before calling So an ambiguous outcome is visible rather than assumed failed. Files: `src/runner.ts` 3. Classify errors A validation error retried forever is a queue that never drains. Files: `src/errors.ts`, `src/errors.test.ts` ### Done when - [ ] An idempotent service does not repeat the effect - [ ] An ambiguous outcome is surfaced rather than retried - [ ] Errors are classified and only retryables retry - [ ] Retries back off and eventually stop ### Watch out - A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. ## Phase 3 · Triggers that fire once A duplicate webhook or a re-polled item does not run the flow twice. ### Steps 1. Deduplicate on the source's key Before starting a run, not inside it. Files: `src/triggers.ts`, `tests/dedupe.test.ts` 2. Advance cursors conservatively Losing an item is worse than seeing it twice, given deduplication. Files: `src/poll.ts` 3. Handle a missing external key A stated rule rather than an assumption of uniqueness. Files: `src/triggers.ts` ### Done when - [ ] A thrice-delivered webhook starts one run - [ ] Cursors advance only after durable recording - [ ] Overlapping pages produce no duplicates - [ ] A missing external key follows a stated rule ### Watch out - Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. ## Phase 4 · Running steps you did not write A step's code cannot reach anything it did not declare. ### Steps 1. Inject only declared connections Flows get shared. Ambient credentials go with them. Files: `src/sandbox.ts`, `tests/isolation.test.ts` 2. Restrict network to declared destinations Undeclared access fails rather than warning. Files: `src/sandbox.ts` 3. Redact credentials from outputs Step outputs are logged and shown. Credentials must not be in them. Files: `src/redact.ts` ### Done when - [ ] An undeclared connection is inaccessible - [ ] Undeclared network access fails - [ ] Steps terminate at timeout and memory limits - [ ] Credentials never appear in logs or outputs ### Watch out - Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn. ## Not in this build - Breadth of integrations. - A builder for non-technical users. - Multi-tenant noisy neighbour isolation. - Exactly-once delivery to services that do not support it. ## After v1, if you want it - Scheduled flows with a missed-window policy, since a scheduler that catches up floods everything - Branching and looping, which multiply every resumption question here - A test mode that runs a flow with side effects disabled, which is what people actually want ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. Run state, step outputs and connection references. DATABASE_URL=postgres://localhost:5432/flows # Required. Per step. A hanging step should fail the run rather than the worker. STEP_TIMEOUT_SECONDS=60 # Required. Then stop and surface. Retrying forever is a queue that never drains. MAX_ATTEMPTS=5 # Required. A timeout after a possible success is not a retry candidate. AMBIGUOUS_OUTCOME=halt # Required. Debugging needs them and they contain other people's data. STEP_OUTPUT_RETENTION_DAYS=14 # Required. Present as a constant. Flows get shared and ambient credentials go with them. INJECT_UNDECLARED_CONNECTIONS=never
You are building a lean indie version of Activepieces Cloud. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Activepieces Cloud · indie build A workflow automation engine whose entire difficulty is failure: a flow of six steps that fails on the fourth must be resumable without repeating the first three, and every step that touches the outside world must be safe to retry. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Execution | Durable step-by-step runs with each result persisted | resuming needs the previous results, which means storing them | | Idempotency | A key per external effect, supplied by the engine | retrying a step that already sent an email is the failure that matters | | Triggers | Webhooks and polling with deduplication at the boundary | a duplicate trigger runs the whole flow again | | Isolation | Step code runs sandboxed with declared network access | steps are user-written and run on your infrastructure | | Runtime | TypeScript with PostgreSQL and a worker pool | runs are stateful, long and must survive a restart | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the engine and the worker pool. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **PostgreSQL 16 or newer** · free locally - Why: Durable run state, deduplication and cursors all want real transactions. - Get it: Docker or your package manager. - Verify: psql connects and a unique constraint enforces trigger deduplication - [ ] **A sandbox for executing step code** · free - Why: Steps are user-written code running on your infrastructure, and running them in the worker process is not something to do once. - Get it: A container or isolate runtime with network and filesystem restriction per execution. - Verify: A hostile step cannot read the host filesystem or reach an undeclared address - [ ] **Two real services, one with idempotency support** · free with existing accounts - Why: Retry safety differs entirely between a service that accepts an idempotency key and one that does not, and both cases need building. - Get it: Accounts on two services you would actually automate against, checking each one's documentation for idempotency support. - Verify: One accepts an idempotency key and one does not, and both are handled - [ ] **Your retry and ambiguity policy** · free - Why: Whether an ambiguous outcome retries or halts is a decision about which failure you prefer, and it must be made before the runner exists rather than during an incident. - Get it: Decide the retry counts, the backoff and what happens on an ambiguous outcome per step kind. - Verify: A written policy covers retryable errors, ambiguous outcomes and the maximum attempts ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Breadth of integrations. - A builder for non-technical users. - Multi-tenant noisy neighbour isolation. - Exactly-once delivery to services that do not support it. - managed hosting, connector updates, execution capacity, support, and reliability - thousands of connectors - maintained OAuth applications - schema-drift handling - high-scale durable execution and support If one of those is essential to you, that is the reason to keep paying for Activepieces Cloud, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Activepieces Cloud The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a workflow runner where a step that half succeeded does not run twice to replace Activepieces Cloud. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Execution: Durable step-by-step runs with each result persisted · resuming needs the previous results, which means storing them - Idempotency: A key per external effect, supplied by the engine · retrying a step that already sent an email is the failure that matters - Triggers: Webhooks and polling with deduplication at the boundary · a duplicate trigger runs the whole flow again - Isolation: Step code runs sandboxed with declared network access · steps are user-written and run on your infrastructure - Runtime: TypeScript with PostgreSQL and a worker pool · runs are stateful, long and must survive a restart ### Data model (create this before Phase 1) - flows(id, name, definition_json, version, enabled, created_at) - runs(id, flow_id, flow_version, trigger_id, state, started_at, finished_at, failed_step_id) - step_results(run_id, step_id, attempt, state, input_hash, output_json, idempotency_key, started_at, finished_at, error) - triggers(id, flow_id, kind, config_json, cursor, dedupe_key_field) - trigger_events(id, trigger_id, external_key, received_at, run_id) · deduplicated on the external key - connections(id, kind, credentials_ref, scopes_json, expires_at) · never stored inline in a flow definition - A run pins the flow version it started with, because editing a flow mid-run would otherwise change the steps underneath an execution already in progress. ### Phase 1 · A run that survives a restart Build: Persist each step's result before moving to the next, so a run resumes from the last completed step. Pin the flow version at run start. Verify by killing the worker between steps and restarting. Trap: Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. Done when: Killing the worker between steps and restarting resumes the run from the last completed step, tested repeatedly. Completed steps are not re-executed on resume. Editing a flow does not change a run already in progress, because the version is pinned. A run whose flow was deleted completes or fails cleanly rather than hanging. Do not build yet: No idempotency, triggers or sandboxing. ### Phase 2 · Retrying without doing it twice Build: Supply an idempotency key per step attempt and pass it to services that accept one. For those that do not, record the attempt before making the call so an ambiguous outcome is detectable. Classify errors into retryable and not. Trap: A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. Done when: A step that timed out after its effect succeeded does not repeat it where the service supports idempotency keys. Where it does not, the ambiguous outcome is surfaced rather than retried blindly, tested by injecting a timeout after success. Errors are classified and only retryable classes retry. Retries use backoff with a cap and then stop. Do not build yet: No triggers or sandboxing yet. ### Phase 3 · Triggers that fire once Build: Deduplicate incoming triggers on an external key before starting a run. Keep a polling cursor per trigger that advances only on success. Handle a webhook delivered three times and a poll that returns overlapping pages. Trap: Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. Done when: A webhook delivered three times starts one run, tested directly. A polling cursor advances only after the triggered run is durably recorded. Overlapping poll pages do not produce duplicate runs. A trigger whose external key is missing is handled by a stated rule rather than assumed unique. Do not build yet: No sandboxing or connections yet. ### Phase 4 · Running steps you did not write Build: Execute step code sandboxed with no ambient credentials and network access limited to declared destinations. Inject only the connections the flow declares. Enforce a timeout and a memory ceiling per step. Trap: Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn. Done when: A step cannot access a connection the flow did not declare, tested with a deliberately hostile step. Network access outside declared destinations fails. Steps terminate at a timeout and a memory ceiling with the run marked failed rather than the worker dying. Credentials do not appear in run logs or step outputs. Do not build yet: No observability or versioning yet. ### Phase 5 · Seeing what a run actually did Build: Store each step's input and output for a retention period, redacted. Support replaying a run from its failed step with the earlier results reused. Report flow health per version and alert on a flow failing repeatedly. Trap: Replaying a failed run from the beginning re-executes steps that already succeeded, which for a step that sent a message or charged a card is a second effect caused by your debugging. Replay from the failure using stored earlier results, and make replaying from the start a separate and clearly labelled action. Done when: A failed run replays from its failed step reusing earlier results, verified against a flow whose first step has a visible effect. Step inputs and outputs are retained for a stated period with credentials redacted. A flow failing repeatedly alerts rather than accumulating silently. Health is reported per flow version so a bad edit is identifiable. Do not build yet: Nothing after this. ### Out of scope (and why) - Breadth of integrations, which is what these platforms actually sell and is an endless maintenance commitment. - A visual builder good enough for non-technical users, which is most of the product's engineering. - Multi-tenant scale with noisy neighbour isolation, which is where running this as a service becomes hard. - Exactly-once delivery to services that do not support it, which is not achievable and should not be claimed. ### README must contain - That every step result is persisted so a run resumes rather than restarting - That a run pins its flow version, so editing a flow cannot change a running execution - That an ambiguous step outcome is surfaced rather than retried, because a retry may send a second message - That only connections the flow declares are injected into its steps - That replay resumes from the failed step rather than from the beginning ===== AGENTS.md ===== # Agent instructions · Activepieces Cloud indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Durable step-by-step runs with each result persisted, A key per external effect, supplied by the engine, Webhooks and polling with deduplication at the boundary, Step code runs sandboxed with declared network access, TypeScript with PostgreSQL and a worker pool. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. - A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. - Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. - Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn. ===== BUILD_PLAN.md ===== # Build plan · Activepieces Cloud A workflow automation engine whose entire difficulty is failure: a flow of six steps that fails on the fourth must be resumable without repeating the first three, and every step that touches the outside world must be safe to retry. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · A run that survives a restart Killing the worker mid-flow does not lose or repeat work. ### Steps 1. Persist after every step In-memory state loses every run in progress on a deployment. Files: `src/runner.ts` 2. Pin the flow version at run start Editing a flow must not change steps underneath a running execution. Files: `src/runs.ts` 3. Test by killing the worker Between steps, repeatedly. Not by simulating it. Files: `tests/resume.test.ts` ### Done when - [ ] A killed worker resumes from the last completed step - [ ] Completed steps are not re-executed - [ ] Flow edits do not affect running executions - [ ] A deleted flow's run resolves cleanly ### Watch out - Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. ## Phase 2 · Retrying without doing it twice A step that timed out after succeeding does not repeat its effect. ### Steps 1. Generate an idempotency key per step Stable across attempts of the same step, different across steps. Files: `src/idempotency.ts` 2. Record before calling So an ambiguous outcome is visible rather than assumed failed. Files: `src/runner.ts` 3. Classify errors A validation error retried forever is a queue that never drains. Files: `src/errors.ts`, `src/errors.test.ts` ### Done when - [ ] An idempotent service does not repeat the effect - [ ] An ambiguous outcome is surfaced rather than retried - [ ] Errors are classified and only retryables retry - [ ] Retries back off and eventually stop ### Watch out - A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. ## Phase 3 · Triggers that fire once A duplicate webhook or a re-polled item does not run the flow twice. ### Steps 1. Deduplicate on the source's key Before starting a run, not inside it. Files: `src/triggers.ts`, `tests/dedupe.test.ts` 2. Advance cursors conservatively Losing an item is worse than seeing it twice, given deduplication. Files: `src/poll.ts` 3. Handle a missing external key A stated rule rather than an assumption of uniqueness. Files: `src/triggers.ts` ### Done when - [ ] A thrice-delivered webhook starts one run - [ ] Cursors advance only after durable recording - [ ] Overlapping pages produce no duplicates - [ ] A missing external key follows a stated rule ### Watch out - Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. ## Phase 4 · Running steps you did not write A step's code cannot reach anything it did not declare. ### Steps 1. Inject only declared connections Flows get shared. Ambient credentials go with them. Files: `src/sandbox.ts`, `tests/isolation.test.ts` 2. Restrict network to declared destinations Undeclared access fails rather than warning. Files: `src/sandbox.ts` 3. Redact credentials from outputs Step outputs are logged and shown. Credentials must not be in them. Files: `src/redact.ts` ### Done when - [ ] An undeclared connection is inaccessible - [ ] Undeclared network access fails - [ ] Steps terminate at timeout and memory limits - [ ] Credentials never appear in logs or outputs ### Watch out - Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn. ## Not in this build - Breadth of integrations. - A builder for non-technical users. - Multi-tenant noisy neighbour isolation. - Exactly-once delivery to services that do not support it. ## After v1, if you want it - Scheduled flows with a missed-window policy, since a scheduler that catches up floods everything - Branching and looping, which multiply every resumption question here - A test mode that runs a flow with side effects disabled, which is what people actually want ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. Run state, step outputs and connection references. DATABASE_URL=postgres://localhost:5432/flows # Required. Per step. A hanging step should fail the run rather than the worker. STEP_TIMEOUT_SECONDS=60 # Required. Then stop and surface. Retrying forever is a queue that never drains. MAX_ATTEMPTS=5 # Required. A timeout after a possible success is not a retry candidate. AMBIGUOUS_OUTCOME=halt # Required. Debugging needs them and they contain other people's data. STEP_OUTPUT_RETENTION_DAYS=14 # Required. Present as a constant. Flows get shared and ambient credentials go with them. INJECT_UNDECLARED_CONNECTIONS=never
You are building a production product version of Activepieces Cloud. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # Activepieces Cloud · product brief ## Problem The core loop is buildable, but a dependable replacement becomes a real weekend or multi-day project. For Activepieces Cloud, use a managed open-source workflow runner instead of operating queues and upgrades. The hard boundary is managed hosting, connector updates, execution capacity, support, and reliability, plus connector breadth and execution reliability. ## Product outcome A workflow engine where a deployment mid-run costs nothing, a timeout does not send a second message, and a shared flow cannot reach credentials it never declared. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - OAuth or API credentials for two chosen services - PostgreSQL - Redis - public HTTPS endpoint - background worker ## Explicit non-goals for v1 - Breadth of integrations. - A builder for non-technical users. - Multi-tenant noisy neighbour isolation. - Exactly-once delivery to services that do not support it. - managed hosting, connector updates, execution capacity, support, and reliability - thousands of connectors - maintained OAuth applications - schema-drift handling - high-scale durable execution and support ## Success criteria - A killed worker is confirmed to resume from the last completed step - An ambiguous outcome is confirmed surfaced rather than retried - A thrice-delivered webhook is confirmed to start one run - An undeclared connection is confirmed inaccessible - Replay is confirmed to resume from the failed step ===== BRIEF.md ===== # Build brief · Activepieces Cloud The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a workflow runner where a step that half succeeded does not run twice to replace Activepieces Cloud. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Execution: Durable step-by-step runs with each result persisted · resuming needs the previous results, which means storing them - Idempotency: A key per external effect, supplied by the engine · retrying a step that already sent an email is the failure that matters - Triggers: Webhooks and polling with deduplication at the boundary · a duplicate trigger runs the whole flow again - Isolation: Step code runs sandboxed with declared network access · steps are user-written and run on your infrastructure - Runtime: TypeScript with PostgreSQL and a worker pool · runs are stateful, long and must survive a restart ### Data model (create this before Phase 1) - flows(id, name, definition_json, version, enabled, created_at) - runs(id, flow_id, flow_version, trigger_id, state, started_at, finished_at, failed_step_id) - step_results(run_id, step_id, attempt, state, input_hash, output_json, idempotency_key, started_at, finished_at, error) - triggers(id, flow_id, kind, config_json, cursor, dedupe_key_field) - trigger_events(id, trigger_id, external_key, received_at, run_id) · deduplicated on the external key - connections(id, kind, credentials_ref, scopes_json, expires_at) · never stored inline in a flow definition - A run pins the flow version it started with, because editing a flow mid-run would otherwise change the steps underneath an execution already in progress. ### Phase 1 · A run that survives a restart Build: Persist each step's result before moving to the next, so a run resumes from the last completed step. Pin the flow version at run start. Verify by killing the worker between steps and restarting. Trap: Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. Done when: Killing the worker between steps and restarting resumes the run from the last completed step, tested repeatedly. Completed steps are not re-executed on resume. Editing a flow does not change a run already in progress, because the version is pinned. A run whose flow was deleted completes or fails cleanly rather than hanging. Do not build yet: No idempotency, triggers or sandboxing. ### Phase 2 · Retrying without doing it twice Build: Supply an idempotency key per step attempt and pass it to services that accept one. For those that do not, record the attempt before making the call so an ambiguous outcome is detectable. Classify errors into retryable and not. Trap: A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. Done when: A step that timed out after its effect succeeded does not repeat it where the service supports idempotency keys. Where it does not, the ambiguous outcome is surfaced rather than retried blindly, tested by injecting a timeout after success. Errors are classified and only retryable classes retry. Retries use backoff with a cap and then stop. Do not build yet: No triggers or sandboxing yet. ### Phase 3 · Triggers that fire once Build: Deduplicate incoming triggers on an external key before starting a run. Keep a polling cursor per trigger that advances only on success. Handle a webhook delivered three times and a poll that returns overlapping pages. Trap: Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. Done when: A webhook delivered three times starts one run, tested directly. A polling cursor advances only after the triggered run is durably recorded. Overlapping poll pages do not produce duplicate runs. A trigger whose external key is missing is handled by a stated rule rather than assumed unique. Do not build yet: No sandboxing or connections yet. ### Phase 4 · Running steps you did not write Build: Execute step code sandboxed with no ambient credentials and network access limited to declared destinations. Inject only the connections the flow declares. Enforce a timeout and a memory ceiling per step. Trap: Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn. Done when: A step cannot access a connection the flow did not declare, tested with a deliberately hostile step. Network access outside declared destinations fails. Steps terminate at a timeout and a memory ceiling with the run marked failed rather than the worker dying. Credentials do not appear in run logs or step outputs. Do not build yet: No observability or versioning yet. ### Phase 5 · Seeing what a run actually did Build: Store each step's input and output for a retention period, redacted. Support replaying a run from its failed step with the earlier results reused. Report flow health per version and alert on a flow failing repeatedly. Trap: Replaying a failed run from the beginning re-executes steps that already succeeded, which for a step that sent a message or charged a card is a second effect caused by your debugging. Replay from the failure using stored earlier results, and make replaying from the start a separate and clearly labelled action. Done when: A failed run replays from its failed step reusing earlier results, verified against a flow whose first step has a visible effect. Step inputs and outputs are retained for a stated period with credentials redacted. A flow failing repeatedly alerts rather than accumulating silently. Health is reported per flow version so a bad edit is identifiable. Do not build yet: Nothing after this. ### Out of scope (and why) - Breadth of integrations, which is what these platforms actually sell and is an endless maintenance commitment. - A visual builder good enough for non-technical users, which is most of the product's engineering. - Multi-tenant scale with noisy neighbour isolation, which is where running this as a service becomes hard. - Exactly-once delivery to services that do not support it, which is not achievable and should not be claimed. ### README must contain - That every step result is persisted so a run resumes rather than restarting - That a run pins its flow version, so editing a flow cannot change a running execution - That an ambiguous step outcome is surfaced rather than retried, because a retry may send a second message - That only connections the flow declares are injected into its steps - That replay resumes from the failed step rather than from the beginning ===== ARCHITECTURE.md ===== # Architecture · Activepieces Cloud ## Stack | Part | Choice | Why | | --- | --- | --- | | Execution | Durable step-by-step runs with each result persisted | resuming needs the previous results, which means storing them | | Idempotency | A key per external effect, supplied by the engine | retrying a step that already sent an email is the failure that matters | | Triggers | Webhooks and polling with deduplication at the boundary | a duplicate trigger runs the whole flow again | | Isolation | Step code runs sandboxed with declared network access | steps are user-written and run on your infrastructure | | Runtime | TypeScript with PostgreSQL and a worker pool | runs are stateful, long and must survive a restart | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Runner | Durable step execution and resumption. | Not swappable for in-memory state, which loses every run on a deployment. | | Idempotency | Keys and ambiguous outcome handling. | Any scheme, provided ambiguity halts rather than retries. | | Triggers | Boundary deduplication and cursors. | Any implementation, provided deduplication happens before the run starts. | | Sandbox | Isolated step execution with declared access. | Any runtime, provided undeclared access fails rather than warns. | | Observability | Redacted step records and targeted replay. | Any store, provided replay resumes from the failure. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DATABASE_URL` · required, secret · Run state, step outputs and connection references. - `STEP_TIMEOUT_SECONDS` · required · Per step. A hanging step should fail the run rather than the worker. - `MAX_ATTEMPTS` · required · Then stop and surface. Retrying forever is a queue that never drains. - `AMBIGUOUS_OUTCOME` · required · A timeout after a possible success is not a retry candidate. - `STEP_OUTPUT_RETENTION_DAYS` · required · Debugging needs them and they contain other people's data. - `INJECT_UNDECLARED_CONNECTIONS` · required · Present as a constant. Flows get shared and ambient credentials go with them. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · Activepieces Cloud product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Durable step-by-step runs with each result persisted, A key per external effect, supplied by the engine, Webhooks and polling with deduplication at the boundary, Step code runs sandboxed with declared network access, TypeScript with PostgreSQL and a worker pool. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. - A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. - Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. - Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn. - Replaying a failed run from the beginning re-executes steps that already succeeded, which for a step that sent a message or charged a card is a second effect caused by your debugging. Replay from the failure using stored earlier results, and make replaying from the start a separate and clearly labelled action. ===== MILESTONES.md ===== # Delivery milestones · Activepieces Cloud Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · A run that survives a restart Killing the worker mid-flow does not lose or repeat work. ### Steps 1. Persist after every step In-memory state loses every run in progress on a deployment. Files: `src/runner.ts` 2. Pin the flow version at run start Editing a flow must not change steps underneath a running execution. Files: `src/runs.ts` 3. Test by killing the worker Between steps, repeatedly. Not by simulating it. Files: `tests/resume.test.ts` ### Done when - [ ] A killed worker resumes from the last completed step - [ ] Completed steps are not re-executed - [ ] Flow edits do not affect running executions - [ ] A deleted flow's run resolves cleanly ### Watch out - Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. ## M2 · Retrying without doing it twice A step that timed out after succeeding does not repeat its effect. ### Steps 1. Generate an idempotency key per step Stable across attempts of the same step, different across steps. Files: `src/idempotency.ts` 2. Record before calling So an ambiguous outcome is visible rather than assumed failed. Files: `src/runner.ts` 3. Classify errors A validation error retried forever is a queue that never drains. Files: `src/errors.ts`, `src/errors.test.ts` ### Done when - [ ] An idempotent service does not repeat the effect - [ ] An ambiguous outcome is surfaced rather than retried - [ ] Errors are classified and only retryables retry - [ ] Retries back off and eventually stop ### Watch out - A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. ## M3 · Triggers that fire once A duplicate webhook or a re-polled item does not run the flow twice. ### Steps 1. Deduplicate on the source's key Before starting a run, not inside it. Files: `src/triggers.ts`, `tests/dedupe.test.ts` 2. Advance cursors conservatively Losing an item is worse than seeing it twice, given deduplication. Files: `src/poll.ts` 3. Handle a missing external key A stated rule rather than an assumption of uniqueness. Files: `src/triggers.ts` ### Done when - [ ] A thrice-delivered webhook starts one run - [ ] Cursors advance only after durable recording - [ ] Overlapping pages produce no duplicates - [ ] A missing external key follows a stated rule ### Watch out - Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. ## M4 · Running steps you did not write A step's code cannot reach anything it did not declare. ### Steps 1. Inject only declared connections Flows get shared. Ambient credentials go with them. Files: `src/sandbox.ts`, `tests/isolation.test.ts` 2. Restrict network to declared destinations Undeclared access fails rather than warning. Files: `src/sandbox.ts` 3. Redact credentials from outputs Step outputs are logged and shown. Credentials must not be in them. Files: `src/redact.ts` ### Done when - [ ] An undeclared connection is inaccessible - [ ] Undeclared network access fails - [ ] Steps terminate at timeout and memory limits - [ ] Credentials never appear in logs or outputs ### Watch out - Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn. ## M5 · Seeing what a run actually did (production only) A failed run can be diagnosed and replayed from the failing step. ### Steps 1. Replay from the failure Replaying from the start re-sends whatever step one sent. Files: `src/replay.ts` 2. Retain inputs and outputs, redacted Debugging needs them and they contain other people's data. Files: `src/observability.ts` 3. Alert on repeated flow failure A flow failing every hour for a week is normally nobody's job. Files: `src/health.ts` ### Done when - [ ] Replay resumes from the failed step - [ ] Inputs and outputs are retained redacted for a stated period - [ ] Repeated failures alert - [ ] Health is reported per flow version ### Watch out - Replaying a failed run from the beginning re-executes steps that already succeeded, which for a step that sent a message or charged a card is a second effect caused by your debugging. Replay from the failure using stored earlier results, and make replaying from the start a separate and clearly labelled action. ===== OPERATIONS.md ===== # Operations · Activepieces Cloud ## Backup Run state and flow definitions are the operational data. Step outputs contain other people's data and should be backed up under the same retention as the live records rather than indefinitely. ## Restore Restore and confirm in-flight runs resume correctly rather than restarting, since a restore that loses step results will re-execute steps whose effects already happened. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Ambiguous outcome count and per-flow failure rate. The first is where duplicate effects come from, and the second identifies a flow that has been broken for a week without anyone noticing. ## Incident checklist A duplicated external effect, such as two identical messages, is the failure that reaches other people. It means either a retry on an ambiguous outcome or a trigger that was not deduplicated, and the step records distinguish the two. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] A killed worker is confirmed to resume from the last completed step - [ ] An ambiguous outcome is confirmed surfaced rather than retried - [ ] A thrice-delivered webhook is confirmed to start one run - [ ] An undeclared connection is confirmed inaccessible - [ ] Replay is confirmed to resume from the failed step ## Launch constraint Do not market omitted Activepieces Cloud capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. Run state, step outputs and connection references. DATABASE_URL=postgres://localhost:5432/flows # Required. Per step. A hanging step should fail the run rather than the worker. STEP_TIMEOUT_SECONDS=60 # Required. Then stop and surface. Retrying forever is a queue that never drains. MAX_ATTEMPTS=5 # Required. A timeout after a possible success is not a retry candidate. AMBIGUOUS_OUTCOME=halt # Required. Debugging needs them and they contain other people's data. STEP_OUTPUT_RETENTION_DAYS=14 # Required. Present as a constant. Flows get shared and ambient credentials go with them. INJECT_UNDECLARED_CONNECTIONS=never
README.md
# Activepieces Cloud · indie build A workflow automation engine whose entire difficulty is failure: a flow of six steps that fails on the fourth must be resumable without repeating the first three, and every step that touches the outside world must be safe to retry. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Execution | Durable step-by-step runs with each result persisted | resuming needs the previous results, which means storing them | | Idempotency | A key per external effect, supplied by the engine | retrying a step that already sent an email is the failure that matters | | Triggers | Webhooks and polling with deduplication at the boundary | a duplicate trigger runs the whole flow again | | Isolation | Step code runs sandboxed with declared network access | steps are user-written and run on your infrastructure | | Runtime | TypeScript with PostgreSQL and a worker pool | runs are stateful, long and must survive a restart | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the engine and the worker pool. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **PostgreSQL 16 or newer** · free locally - Why: Durable run state, deduplication and cursors all want real transactions. - Get it: Docker or your package manager. - Verify: psql connects and a unique constraint enforces trigger deduplication - [ ] **A sandbox for executing step code** · free - Why: Steps are user-written code running on your infrastructure, and running them in the worker process is not something to do once. - Get it: A container or isolate runtime with network and filesystem restriction per execution. - Verify: A hostile step cannot read the host filesystem or reach an undeclared address - [ ] **Two real services, one with idempotency support** · free with existing accounts - Why: Retry safety differs entirely between a service that accepts an idempotency key and one that does not, and both cases need building. - Get it: Accounts on two services you would actually automate against, checking each one's documentation for idempotency support. - Verify: One accepts an idempotency key and one does not, and both are handled - [ ] **Your retry and ambiguity policy** · free - Why: Whether an ambiguous outcome retries or halts is a decision about which failure you prefer, and it must be made before the runner exists rather than during an incident. - Get it: Decide the retry counts, the backoff and what happens on an ambiguous outcome per step kind. - Verify: A written policy covers retryable errors, ambiguous outcomes and the maximum attempts ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Breadth of integrations. - A builder for non-technical users. - Multi-tenant noisy neighbour isolation. - Exactly-once delivery to services that do not support it. - managed hosting, connector updates, execution capacity, support, and reliability - thousands of connectors - maintained OAuth applications - schema-drift handling - high-scale durable execution and support If one of those is essential to you, that is the reason to keep paying for Activepieces Cloud, and the README should say so rather than pretend.
BRIEF.md
# Build brief · Activepieces Cloud The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a workflow runner where a step that half succeeded does not run twice to replace Activepieces Cloud. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Execution: Durable step-by-step runs with each result persisted · resuming needs the previous results, which means storing them - Idempotency: A key per external effect, supplied by the engine · retrying a step that already sent an email is the failure that matters - Triggers: Webhooks and polling with deduplication at the boundary · a duplicate trigger runs the whole flow again - Isolation: Step code runs sandboxed with declared network access · steps are user-written and run on your infrastructure - Runtime: TypeScript with PostgreSQL and a worker pool · runs are stateful, long and must survive a restart ### Data model (create this before Phase 1) - flows(id, name, definition_json, version, enabled, created_at) - runs(id, flow_id, flow_version, trigger_id, state, started_at, finished_at, failed_step_id) - step_results(run_id, step_id, attempt, state, input_hash, output_json, idempotency_key, started_at, finished_at, error) - triggers(id, flow_id, kind, config_json, cursor, dedupe_key_field) - trigger_events(id, trigger_id, external_key, received_at, run_id) · deduplicated on the external key - connections(id, kind, credentials_ref, scopes_json, expires_at) · never stored inline in a flow definition - A run pins the flow version it started with, because editing a flow mid-run would otherwise change the steps underneath an execution already in progress. ### Phase 1 · A run that survives a restart Build: Persist each step's result before moving to the next, so a run resumes from the last completed step. Pin the flow version at run start. Verify by killing the worker between steps and restarting. Trap: Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. Done when: Killing the worker between steps and restarting resumes the run from the last completed step, tested repeatedly. Completed steps are not re-executed on resume. Editing a flow does not change a run already in progress, because the version is pinned. A run whose flow was deleted completes or fails cleanly rather than hanging. Do not build yet: No idempotency, triggers or sandboxing. ### Phase 2 · Retrying without doing it twice Build: Supply an idempotency key per step attempt and pass it to services that accept one. For those that do not, record the attempt before making the call so an ambiguous outcome is detectable. Classify errors into retryable and not. Trap: A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. Done when: A step that timed out after its effect succeeded does not repeat it where the service supports idempotency keys. Where it does not, the ambiguous outcome is surfaced rather than retried blindly, tested by injecting a timeout after success. Errors are classified and only retryable classes retry. Retries use backoff with a cap and then stop. Do not build yet: No triggers or sandboxing yet. ### Phase 3 · Triggers that fire once Build: Deduplicate incoming triggers on an external key before starting a run. Keep a polling cursor per trigger that advances only on success. Handle a webhook delivered three times and a poll that returns overlapping pages. Trap: Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. Done when: A webhook delivered three times starts one run, tested directly. A polling cursor advances only after the triggered run is durably recorded. Overlapping poll pages do not produce duplicate runs. A trigger whose external key is missing is handled by a stated rule rather than assumed unique. Do not build yet: No sandboxing or connections yet. ### Phase 4 · Running steps you did not write Build: Execute step code sandboxed with no ambient credentials and network access limited to declared destinations. Inject only the connections the flow declares. Enforce a timeout and a memory ceiling per step. Trap: Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn. Done when: A step cannot access a connection the flow did not declare, tested with a deliberately hostile step. Network access outside declared destinations fails. Steps terminate at a timeout and a memory ceiling with the run marked failed rather than the worker dying. Credentials do not appear in run logs or step outputs. Do not build yet: No observability or versioning yet. ### Phase 5 · Seeing what a run actually did Build: Store each step's input and output for a retention period, redacted. Support replaying a run from its failed step with the earlier results reused. Report flow health per version and alert on a flow failing repeatedly. Trap: Replaying a failed run from the beginning re-executes steps that already succeeded, which for a step that sent a message or charged a card is a second effect caused by your debugging. Replay from the failure using stored earlier results, and make replaying from the start a separate and clearly labelled action. Done when: A failed run replays from its failed step reusing earlier results, verified against a flow whose first step has a visible effect. Step inputs and outputs are retained for a stated period with credentials redacted. A flow failing repeatedly alerts rather than accumulating silently. Health is reported per flow version so a bad edit is identifiable. Do not build yet: Nothing after this. ### Out of scope (and why) - Breadth of integrations, which is what these platforms actually sell and is an endless maintenance commitment. - A visual builder good enough for non-technical users, which is most of the product's engineering. - Multi-tenant scale with noisy neighbour isolation, which is where running this as a service becomes hard. - Exactly-once delivery to services that do not support it, which is not achievable and should not be claimed. ### README must contain - That every step result is persisted so a run resumes rather than restarting - That a run pins its flow version, so editing a flow cannot change a running execution - That an ambiguous step outcome is surfaced rather than retried, because a retry may send a second message - That only connections the flow declares are injected into its steps - That replay resumes from the failed step rather than from the beginning
AGENTS.md
# Agent instructions · Activepieces Cloud indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Durable step-by-step runs with each result persisted, A key per external effect, supplied by the engine, Webhooks and polling with deduplication at the boundary, Step code runs sandboxed with declared network access, TypeScript with PostgreSQL and a worker pool. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. - A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. - Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. - Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn.
BUILD_PLAN.md
# Build plan · Activepieces Cloud A workflow automation engine whose entire difficulty is failure: a flow of six steps that fails on the fourth must be resumable without repeating the first three, and every step that touches the outside world must be safe to retry. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · A run that survives a restart Killing the worker mid-flow does not lose or repeat work. ### Steps 1. Persist after every step In-memory state loses every run in progress on a deployment. Files: `src/runner.ts` 2. Pin the flow version at run start Editing a flow must not change steps underneath a running execution. Files: `src/runs.ts` 3. Test by killing the worker Between steps, repeatedly. Not by simulating it. Files: `tests/resume.test.ts` ### Done when - [ ] A killed worker resumes from the last completed step - [ ] Completed steps are not re-executed - [ ] Flow edits do not affect running executions - [ ] A deleted flow's run resolves cleanly ### Watch out - Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. ## Phase 2 · Retrying without doing it twice A step that timed out after succeeding does not repeat its effect. ### Steps 1. Generate an idempotency key per step Stable across attempts of the same step, different across steps. Files: `src/idempotency.ts` 2. Record before calling So an ambiguous outcome is visible rather than assumed failed. Files: `src/runner.ts` 3. Classify errors A validation error retried forever is a queue that never drains. Files: `src/errors.ts`, `src/errors.test.ts` ### Done when - [ ] An idempotent service does not repeat the effect - [ ] An ambiguous outcome is surfaced rather than retried - [ ] Errors are classified and only retryables retry - [ ] Retries back off and eventually stop ### Watch out - A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. ## Phase 3 · Triggers that fire once A duplicate webhook or a re-polled item does not run the flow twice. ### Steps 1. Deduplicate on the source's key Before starting a run, not inside it. Files: `src/triggers.ts`, `tests/dedupe.test.ts` 2. Advance cursors conservatively Losing an item is worse than seeing it twice, given deduplication. Files: `src/poll.ts` 3. Handle a missing external key A stated rule rather than an assumption of uniqueness. Files: `src/triggers.ts` ### Done when - [ ] A thrice-delivered webhook starts one run - [ ] Cursors advance only after durable recording - [ ] Overlapping pages produce no duplicates - [ ] A missing external key follows a stated rule ### Watch out - Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. ## Phase 4 · Running steps you did not write A step's code cannot reach anything it did not declare. ### Steps 1. Inject only declared connections Flows get shared. Ambient credentials go with them. Files: `src/sandbox.ts`, `tests/isolation.test.ts` 2. Restrict network to declared destinations Undeclared access fails rather than warning. Files: `src/sandbox.ts` 3. Redact credentials from outputs Step outputs are logged and shown. Credentials must not be in them. Files: `src/redact.ts` ### Done when - [ ] An undeclared connection is inaccessible - [ ] Undeclared network access fails - [ ] Steps terminate at timeout and memory limits - [ ] Credentials never appear in logs or outputs ### Watch out - Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn. ## Not in this build - Breadth of integrations. - A builder for non-technical users. - Multi-tenant noisy neighbour isolation. - Exactly-once delivery to services that do not support it. ## After v1, if you want it - Scheduled flows with a missed-window policy, since a scheduler that catches up floods everything - Branching and looping, which multiply every resumption question here - A test mode that runs a flow with side effects disabled, which is what people actually want
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. Run state, step outputs and connection references. DATABASE_URL=postgres://localhost:5432/flows # Required. Per step. A hanging step should fail the run rather than the worker. STEP_TIMEOUT_SECONDS=60 # Required. Then stop and surface. Retrying forever is a queue that never drains. MAX_ATTEMPTS=5 # Required. A timeout after a possible success is not a retry candidate. AMBIGUOUS_OUTCOME=halt # Required. Debugging needs them and they contain other people's data. STEP_OUTPUT_RETENTION_DAYS=14 # Required. Present as a constant. Flows get shared and ambient credentials go with them. INJECT_UNDECLARED_CONNECTIONS=never
PRODUCT.md
# Activepieces Cloud · product brief ## Problem The core loop is buildable, but a dependable replacement becomes a real weekend or multi-day project. For Activepieces Cloud, use a managed open-source workflow runner instead of operating queues and upgrades. The hard boundary is managed hosting, connector updates, execution capacity, support, and reliability, plus connector breadth and execution reliability. ## Product outcome A workflow engine where a deployment mid-run costs nothing, a timeout does not send a second message, and a shared flow cannot reach credentials it never declared. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - OAuth or API credentials for two chosen services - PostgreSQL - Redis - public HTTPS endpoint - background worker ## Explicit non-goals for v1 - Breadth of integrations. - A builder for non-technical users. - Multi-tenant noisy neighbour isolation. - Exactly-once delivery to services that do not support it. - managed hosting, connector updates, execution capacity, support, and reliability - thousands of connectors - maintained OAuth applications - schema-drift handling - high-scale durable execution and support ## Success criteria - A killed worker is confirmed to resume from the last completed step - An ambiguous outcome is confirmed surfaced rather than retried - A thrice-delivered webhook is confirmed to start one run - An undeclared connection is confirmed inaccessible - Replay is confirmed to resume from the failed step
BRIEF.md
# Build brief · Activepieces Cloud The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a workflow runner where a step that half succeeded does not run twice to replace Activepieces Cloud. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Execution: Durable step-by-step runs with each result persisted · resuming needs the previous results, which means storing them - Idempotency: A key per external effect, supplied by the engine · retrying a step that already sent an email is the failure that matters - Triggers: Webhooks and polling with deduplication at the boundary · a duplicate trigger runs the whole flow again - Isolation: Step code runs sandboxed with declared network access · steps are user-written and run on your infrastructure - Runtime: TypeScript with PostgreSQL and a worker pool · runs are stateful, long and must survive a restart ### Data model (create this before Phase 1) - flows(id, name, definition_json, version, enabled, created_at) - runs(id, flow_id, flow_version, trigger_id, state, started_at, finished_at, failed_step_id) - step_results(run_id, step_id, attempt, state, input_hash, output_json, idempotency_key, started_at, finished_at, error) - triggers(id, flow_id, kind, config_json, cursor, dedupe_key_field) - trigger_events(id, trigger_id, external_key, received_at, run_id) · deduplicated on the external key - connections(id, kind, credentials_ref, scopes_json, expires_at) · never stored inline in a flow definition - A run pins the flow version it started with, because editing a flow mid-run would otherwise change the steps underneath an execution already in progress. ### Phase 1 · A run that survives a restart Build: Persist each step's result before moving to the next, so a run resumes from the last completed step. Pin the flow version at run start. Verify by killing the worker between steps and restarting. Trap: Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. Done when: Killing the worker between steps and restarting resumes the run from the last completed step, tested repeatedly. Completed steps are not re-executed on resume. Editing a flow does not change a run already in progress, because the version is pinned. A run whose flow was deleted completes or fails cleanly rather than hanging. Do not build yet: No idempotency, triggers or sandboxing. ### Phase 2 · Retrying without doing it twice Build: Supply an idempotency key per step attempt and pass it to services that accept one. For those that do not, record the attempt before making the call so an ambiguous outcome is detectable. Classify errors into retryable and not. Trap: A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. Done when: A step that timed out after its effect succeeded does not repeat it where the service supports idempotency keys. Where it does not, the ambiguous outcome is surfaced rather than retried blindly, tested by injecting a timeout after success. Errors are classified and only retryable classes retry. Retries use backoff with a cap and then stop. Do not build yet: No triggers or sandboxing yet. ### Phase 3 · Triggers that fire once Build: Deduplicate incoming triggers on an external key before starting a run. Keep a polling cursor per trigger that advances only on success. Handle a webhook delivered three times and a poll that returns overlapping pages. Trap: Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. Done when: A webhook delivered three times starts one run, tested directly. A polling cursor advances only after the triggered run is durably recorded. Overlapping poll pages do not produce duplicate runs. A trigger whose external key is missing is handled by a stated rule rather than assumed unique. Do not build yet: No sandboxing or connections yet. ### Phase 4 · Running steps you did not write Build: Execute step code sandboxed with no ambient credentials and network access limited to declared destinations. Inject only the connections the flow declares. Enforce a timeout and a memory ceiling per step. Trap: Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn. Done when: A step cannot access a connection the flow did not declare, tested with a deliberately hostile step. Network access outside declared destinations fails. Steps terminate at a timeout and a memory ceiling with the run marked failed rather than the worker dying. Credentials do not appear in run logs or step outputs. Do not build yet: No observability or versioning yet. ### Phase 5 · Seeing what a run actually did Build: Store each step's input and output for a retention period, redacted. Support replaying a run from its failed step with the earlier results reused. Report flow health per version and alert on a flow failing repeatedly. Trap: Replaying a failed run from the beginning re-executes steps that already succeeded, which for a step that sent a message or charged a card is a second effect caused by your debugging. Replay from the failure using stored earlier results, and make replaying from the start a separate and clearly labelled action. Done when: A failed run replays from its failed step reusing earlier results, verified against a flow whose first step has a visible effect. Step inputs and outputs are retained for a stated period with credentials redacted. A flow failing repeatedly alerts rather than accumulating silently. Health is reported per flow version so a bad edit is identifiable. Do not build yet: Nothing after this. ### Out of scope (and why) - Breadth of integrations, which is what these platforms actually sell and is an endless maintenance commitment. - A visual builder good enough for non-technical users, which is most of the product's engineering. - Multi-tenant scale with noisy neighbour isolation, which is where running this as a service becomes hard. - Exactly-once delivery to services that do not support it, which is not achievable and should not be claimed. ### README must contain - That every step result is persisted so a run resumes rather than restarting - That a run pins its flow version, so editing a flow cannot change a running execution - That an ambiguous step outcome is surfaced rather than retried, because a retry may send a second message - That only connections the flow declares are injected into its steps - That replay resumes from the failed step rather than from the beginning
ARCHITECTURE.md
# Architecture · Activepieces Cloud ## Stack | Part | Choice | Why | | --- | --- | --- | | Execution | Durable step-by-step runs with each result persisted | resuming needs the previous results, which means storing them | | Idempotency | A key per external effect, supplied by the engine | retrying a step that already sent an email is the failure that matters | | Triggers | Webhooks and polling with deduplication at the boundary | a duplicate trigger runs the whole flow again | | Isolation | Step code runs sandboxed with declared network access | steps are user-written and run on your infrastructure | | Runtime | TypeScript with PostgreSQL and a worker pool | runs are stateful, long and must survive a restart | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Runner | Durable step execution and resumption. | Not swappable for in-memory state, which loses every run on a deployment. | | Idempotency | Keys and ambiguous outcome handling. | Any scheme, provided ambiguity halts rather than retries. | | Triggers | Boundary deduplication and cursors. | Any implementation, provided deduplication happens before the run starts. | | Sandbox | Isolated step execution with declared access. | Any runtime, provided undeclared access fails rather than warns. | | Observability | Redacted step records and targeted replay. | Any store, provided replay resumes from the failure. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DATABASE_URL` · required, secret · Run state, step outputs and connection references. - `STEP_TIMEOUT_SECONDS` · required · Per step. A hanging step should fail the run rather than the worker. - `MAX_ATTEMPTS` · required · Then stop and surface. Retrying forever is a queue that never drains. - `AMBIGUOUS_OUTCOME` · required · A timeout after a possible success is not a retry candidate. - `STEP_OUTPUT_RETENTION_DAYS` · required · Debugging needs them and they contain other people's data. - `INJECT_UNDECLARED_CONNECTIONS` · required · Present as a constant. Flows get shared and ambient credentials go with them. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · Activepieces Cloud product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Durable step-by-step runs with each result persisted, A key per external effect, supplied by the engine, Webhooks and polling with deduplication at the boundary, Step code runs sandboxed with declared network access, TypeScript with PostgreSQL and a worker pool. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. - A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. - Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. - Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn. - Replaying a failed run from the beginning re-executes steps that already succeeded, which for a step that sent a message or charged a card is a second effect caused by your debugging. Replay from the failure using stored earlier results, and make replaying from the start a separate and clearly labelled action.
MILESTONES.md
# Delivery milestones · Activepieces Cloud Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · A run that survives a restart Killing the worker mid-flow does not lose or repeat work. ### Steps 1. Persist after every step In-memory state loses every run in progress on a deployment. Files: `src/runner.ts` 2. Pin the flow version at run start Editing a flow must not change steps underneath a running execution. Files: `src/runs.ts` 3. Test by killing the worker Between steps, repeatedly. Not by simulating it. Files: `tests/resume.test.ts` ### Done when - [ ] A killed worker resumes from the last completed step - [ ] Completed steps are not re-executed - [ ] Flow edits do not affect running executions - [ ] A deleted flow's run resolves cleanly ### Watch out - Holding run state in memory means a deployment mid-flow loses every run in progress, and since deployments happen constantly the failure is routine rather than exceptional. Persist after every step and resume from the store rather than from memory. ## M2 · Retrying without doing it twice A step that timed out after succeeding does not repeat its effect. ### Steps 1. Generate an idempotency key per step Stable across attempts of the same step, different across steps. Files: `src/idempotency.ts` 2. Record before calling So an ambiguous outcome is visible rather than assumed failed. Files: `src/runner.ts` 3. Classify errors A validation error retried forever is a queue that never drains. Files: `src/errors.ts`, `src/errors.test.ts` ### Done when - [ ] An idempotent service does not repeat the effect - [ ] An ambiguous outcome is surfaced rather than retried - [ ] Errors are classified and only retryables retry - [ ] Retries back off and eventually stop ### Watch out - A step that sends an email, times out waiting for the response and is retried sends a second email, and the recipient sees two while your run log shows one failure. Record before calling, use idempotency keys where the service supports them, and treat an ambiguous outcome as needing a check rather than a retry. ## M3 · Triggers that fire once A duplicate webhook or a re-polled item does not run the flow twice. ### Steps 1. Deduplicate on the source's key Before starting a run, not inside it. Files: `src/triggers.ts`, `tests/dedupe.test.ts` 2. Advance cursors conservatively Losing an item is worse than seeing it twice, given deduplication. Files: `src/poll.ts` 3. Handle a missing external key A stated rule rather than an assumption of uniqueness. Files: `src/triggers.ts` ### Done when - [ ] A thrice-delivered webhook starts one run - [ ] Cursors advance only after durable recording - [ ] Overlapping pages produce no duplicates - [ ] A missing external key follows a stated rule ### Watch out - Advancing a polling cursor before the run succeeds means a failure loses the item entirely, while advancing only on success means a crash re-processes it, so deduplication at the boundary is what makes either safe. Deduplicate on the source's own identifier and keep the cursor conservative. ## M4 · Running steps you did not write A step's code cannot reach anything it did not declare. ### Steps 1. Inject only declared connections Flows get shared. Ambient credentials go with them. Files: `src/sandbox.ts`, `tests/isolation.test.ts` 2. Restrict network to declared destinations Undeclared access fails rather than warning. Files: `src/sandbox.ts` 3. Redact credentials from outputs Step outputs are logged and shown. Credentials must not be in them. Files: `src/redact.ts` ### Done when - [ ] An undeclared connection is inaccessible - [ ] Undeclared network access fails - [ ] Steps terminate at timeout and memory limits - [ ] Credentials never appear in logs or outputs ### Watch out - Injecting every connection the account has into every step means one flow's custom code can read the credentials for services it has nothing to do with, and flows get shared between people constantly. Inject only what the flow declares, and make an undeclared access fail rather than warn. ## M5 · Seeing what a run actually did (production only) A failed run can be diagnosed and replayed from the failing step. ### Steps 1. Replay from the failure Replaying from the start re-sends whatever step one sent. Files: `src/replay.ts` 2. Retain inputs and outputs, redacted Debugging needs them and they contain other people's data. Files: `src/observability.ts` 3. Alert on repeated flow failure A flow failing every hour for a week is normally nobody's job. Files: `src/health.ts` ### Done when - [ ] Replay resumes from the failed step - [ ] Inputs and outputs are retained redacted for a stated period - [ ] Repeated failures alert - [ ] Health is reported per flow version ### Watch out - Replaying a failed run from the beginning re-executes steps that already succeeded, which for a step that sent a message or charged a card is a second effect caused by your debugging. Replay from the failure using stored earlier results, and make replaying from the start a separate and clearly labelled action.
OPERATIONS.md
# Operations · Activepieces Cloud ## Backup Run state and flow definitions are the operational data. Step outputs contain other people's data and should be backed up under the same retention as the live records rather than indefinitely. ## Restore Restore and confirm in-flight runs resume correctly rather than restarting, since a restore that loses step results will re-execute steps whose effects already happened. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Ambiguous outcome count and per-flow failure rate. The first is where duplicate effects come from, and the second identifies a flow that has been broken for a week without anyone noticing. ## Incident checklist A duplicated external effect, such as two identical messages, is the failure that reaches other people. It means either a retry on an ambiguous outcome or a trigger that was not deduplicated, and the step records distinguish the two. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] A killed worker is confirmed to resume from the last completed step - [ ] An ambiguous outcome is confirmed surfaced rather than retried - [ ] A thrice-delivered webhook is confirmed to start one run - [ ] An undeclared connection is confirmed inaccessible - [ ] Replay is confirmed to resume from the failed step ## Launch constraint Do not market omitted Activepieces Cloud capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. Run state, step outputs and connection references. DATABASE_URL=postgres://localhost:5432/flows # Required. Per step. A hanging step should fail the run rather than the worker. STEP_TIMEOUT_SECONDS=60 # Required. Then stop and surface. Retrying forever is a queue that never drains. MAX_ATTEMPTS=5 # Required. A timeout after a possible success is not a retry candidate. AMBIGUOUS_OUTCOME=halt # Required. Debugging needs them and they contain other people's data. STEP_OUTPUT_RETENTION_DAYS=14 # Required. Present as a constant. Flows get shared and ambient credentials go with them. INJECT_UNDECLARED_CONNECTIONS=never
$ choose a build depth, inspect the files, then open the complete pack in your agent
People still pay for Activepieces Cloud because one automation is easy; maintaining hundreds of connectors and making every run recoverable is the business. The recurring cost buys OAuth review, token refresh, webhooks, polling, schemas, retries, idempotency, queues, secrets, rate limits, observability, and upgrades, not just the visible interface.
xmanaged hosting, connector updates, execution capacity, support, and reliability
xthousands of connectors
xmaintained OAuth applications
xschema-drift handling
xhigh-scale durable execution and support
share on X ↗"I just replaced Activepieces Cloud with one prompt"
Don't feel like building it? These folks already made it free.
Activepieces Free CloudManaged Activepieces with ten free active flows and unlimited runs; team controls still cost money.free↗no votes, no pay-to-list · just what's real
Activepieces Cloud pricing
| plan | monthly | annual (per mo) | what you get |
|---|---|---|---|
| free | $0/workspace | $0/workspace | 1 user; unlimited flows; Flows, Agents, Tables, MCPs and API access; credits refresh daily with a hard cap |
| plus | $20/workspace | $16/workspace | Up to 5 users; unlimited flows; monthly credit pool equal to 3× the Free baseline; BYO AI keys; pay-as-you-go overage |
| team | $200/workspace | $166/workspace | 25 users included; unlimited flows and projects; 5× Plus credits; SSO, standard roles, global connections and email support |
| ultimate | custom | — | Custom users and yearly credit pool; Git Sync/releases, dedicated workers, SCIM, custom RBAC, audit logs and dedicated support |
| embed | — | $3000/workspace | Embeddable white-label builder, JavaScript SDK, user provisioning, branding, piece and template management; credit-based |
free tier1 user, unlimited flows, Agents/Tables/MCPs/API and a hard-capped daily credit refresh; the numeric daily credit allowance is not published
billingfree + monthly and annual Plus/Team billing; annual is advertised as 2 months free; Ultimate is custom and Embed starts at $36,000/year
hidden costs$0.007 per extra credit on Plus/Team; Team users beyond 25 cost $25/user/month; AI actions consume 2, 10 or 20 credits by model tier, or 1 with a BYO key; Embed has a $36,000/year starting commitment
verified 2026-08-14 · source ↗
Vibecode Activepieces Cloud
Kinda. The core of Activepieces Cloud is buildable in a weekend with the prompt on this page, but there are real gaps: managed hosting, connector updates, execution capacity, support, and reliability, thousands of connectors. Read the honest list above before committing.
How much does Activepieces Cloud cost?
Activepieces Cloud's pricing is usage-based or varies by plan · Typical Plus monthly tier; usage limits apply..
What do I lose by replacing Activepieces Cloud?
Honestly: managed hosting, connector updates, execution capacity, support, and reliability; thousands of connectors; maintained OAuth applications; schema-drift handling; high-scale durable execution and support. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to Activepieces Cloud?
Yes: Activepieces Free Cloud (Managed Activepieces with ten free active flows and unlimited runs; team controls still cost money.) The prompt is for when you want it exactly your way.