Build Checkly

KINDAreplaces $24/mosaves $288/yrback to the verdict

0%0 of 20 items done

Saved on this device only. Tick prerequisites first, then work the phases in order · do not start one until the checks above it pass.

A checker running outside your infrastructure that performs API and browser checks on a schedule, distinguishes a real failure from a blip before alerting, sends one alert per incident rather than one per check, and publishes a status page whose uptime figure is computed honestly including its own gaps.

estimated effort multi-daythe files for this build are in the project pack

RuntimeNode 22 with PostgreSQLBrowser checksPlaywright, headlessLocationA server with a different provider and region from what it monitorsAlertingA path independent of your own infrastructureStatus pageStatic, generated and served from somewhere separate

Before step 1

Everything below is assumed from the first step. Tick each one when you actually have it, not when you plan to.

installfree
Why Runs the checker and Playwright.
Get it nodejs.org, or nvm install 22. open ↗
Verify node --version prints v22 or higher
have ready5 to 10 US dollars a month
Why This is the entire design constraint. A monitor in the same failure domain cannot report the outage you built it for.
Get it A small instance with a provider you do not otherwise use. It needs very little capacity; independence is the requirement rather than power.
Verify A written list of shared dependencies exists and is short
installfree self-hosted
Why Results, incidents and alerts, with the incident logic needing transactions.
Get it On the checker server, or a managed instance from a third provider. open ↗
Verify psql connects and reports 16 or higher
API keyfree tiers cover this volume
Why An alert routed through something you host is an alert you will not receive during the outage that matters.
Get it A transactional email provider or an external chat service. Not a webhook into your own application.
Verify An alert arrives with your monitored infrastructure fully down
have readyfree on most static hosts
Why A status page hosted on the failing infrastructure is the oldest mistake in this category and it is made repeatedly.
Get it Static hosting with a different provider, or the checker server itself if that is genuinely separate.
Verify The page serves while the monitored system is down

Data model

Create these before the first phase that stores anything. Changing a table later is the expensive kind of change.

- checks(id, name, kind, target, interval_seconds, timeout_ms, expected_json, enabled, degraded_after_ms) · kind is http or browser
- results(check_id, at, ok, duration_ms, status_code, error, screenshot_key)
- incidents(id, check_id, started_at, confirmed_at, resolved_at, state, cause_summary) · confirmed_at is when it passed the confirmation threshold
- alerts(incident_id, at, channel, delivered, error)
- maintenance(id, check_id, starts_at, ends_at, reason) · declared downtime excluded from uptime, and labelled on the status page
- checker_gaps(started_at, ended_at, reason) · when the checker itself was not running, so uptime can be honest about it
- An outage is a confirmed incident, not a single failed result. The confirmation threshold is what separates a monitor from a noise generator.

Environment variables

These go in a .env file the app reads at startup. The pack's .env.example is this table as a file · copy it, never commit the filled-in version.

Variable	Needed	Example	Where the value comes from
`DATABASE_URL`secret	required	`postgres://user:pass@localhost:5432/checks`	On the checker, not on the monitored infrastructure.
`CONFIRM_FAILURES`	required	`3`	Consecutive failures before an incident is confirmed. With the interval, this defines your detection time.
`RESOLVE_SUCCESSES`	required	`3`	Consecutive successes before resolving. Higher values suppress flapping.
`DEFAULT_INTERVAL_SECONDS`	required	`60`	How often checks run. Shorter detects faster and costs more requests.
`ALERT_CHANNEL_URL`secret	required	`https://api.provider.example/send`	Independent delivery path. Never a webhook into the monitored system.
`STATUS_PAGE_DEST`	required	`s3://status-bucket`	Where the generated status page is published. Independent hosting.
`CHECKER_HEARTBEAT_SECONDS`	required	`120`	How long without a checker run before the period is recorded as a gap rather than uptime.

The build, in order

1
A checker somewhere else
Checks run from outside the failure domain of what they watch, and you have verified that.
1. Different provider and region. Not a different instance in the same account.
  Files docs/separation.md
2. With a timeout, recording duration, status and error for every run.
  Files src/http-check.js
3. Provider, region, DNS, certificate authority. Each one is a way the monitor can fail with the target.
  Files docs/separation.md
done when · tick each as it passes
watch out
- A monitor in the same cloud region as the application will miss a regional outage entirely, and that is precisely the event you built it for. Write down what the monitor and the target share, including provider, region, DNS and certificate authority, and be deliberate about each.
next phase
2
Checks that assert something real
A check fails when the application is broken, not only when it is down.
1. A value only a working application produces. Status codes lie routinely.
  Files src/assertions.js
2. A slow response is information; treating it as an outage produces noise.
  Files src/assertions.js
3. A real flow, with a screenshot and the failing selector captured on failure.
  Files src/browser-check.js
  terminal
  npx playwright install chromium
done when · tick each as it passes
watch out
- A check asserting only a two hundred status code passes while the application returns an error page, an empty result set or a login screen. Assert something that proves the application actually worked, which usually means a value that could only be produced by a working system.
next phase
3
Incidents, not results
One outage produces one incident, and a blip produces nothing.
1. Consecutive failures past a threshold. The threshold and interval together define your detection time; state it.
  Files src/incidents.js
2. Consecutive successes, so flapping does not close and reopen.
  Files src/incidents.js
3. Alternate failure and success around the threshold. This is where naive logic produces a storm.
  Files test/incidents.test.js
done when · tick each as it passes
watch out
- Alerting on every failed result turns a thirty second network blip into a page at three in the morning, and after two of those the alerts get muted. Confirm with consecutive failures, and require consecutive successes to resolve so a flapping service does not open and close incidents repeatedly.
next phase
4
Alerting that survives the outage
You are told once, through a path that does not depend on what is broken.
1. Confirmed and resolved. Not per failed check.
  Files src/alerts.js
2. External provider, not your own infrastructure. Test by taking everything down.
  Files src/channels.js
3. Suppress the alert, keep the downtime visible. Hiding it makes uptime figures dishonest.
  Files src/maintenance.js
done when · tick each as it passes
watch out
- Alerting through a webhook into a service you also host means the outage takes the alert with it. Use an external provider for delivery, and test the whole path by taking your infrastructure down deliberately rather than by taking one service down.
next phase
5
A status page that tells the truth
The uptime figure is honest, including about the times the monitor was not looking.
1. From incidents, published somewhere separate. Static because it must serve when everything else is gone.
  Files src/status.js
2. Whenever the checker was not running. These become unknown periods rather than uptime.
  Files src/gaps.js
3. One month, computed by hand from the incident list. Uptime arithmetic is easy to get subtly wrong.
done when · tick each as it passes
watch out
- Computing uptime as the proportion of successful checks counts a period when the checker was down as no data and therefore, silently, as up. Record checker gaps explicitly and show them, because an uptime figure that quietly excludes its own blind spots is the number every status page gets wrong.
6
More locations, and knowing what you cannot seeproduct builder
A regional problem is distinguishable from a global one, and the blind spots are documented.
1. Different provider again. Two locations that share a provider tell you less than they appear to.
  Files docs/separation.md
2. One location failing is regional or a checker problem. Both failing is an outage.
  Files src/incidents.js
3. Regions with no checker, times with no coverage, and failure modes no check asserts.
  Files docs/coverage.md
done when · tick each as it passes
watch out
- One checker cannot tell the difference between your service being down and its own network path being broken, and it will report the second as the first. A second location that disagrees turns a false alarm into useful information about where the problem is.

what this build does not replace

A global probe network.
Phone and text message delivery.
Managed browser check operations at scale.
Anyone to call when the monitoring breaks.
global browser fleet, Playwright operations, alerting, CI integration, and support
global probe network
phone and SMS delivery
massive retention
advanced incident response and support

after v1, if you want it

A third location, once two have disagreed usefully at least once
Response time percentiles per check, which reveal degradation before failure
Public incident write-ups attached to the status page history

Need the files? The project pack on the verdict page hands your agent the whole brief · more monitoring.