Build SharpAPI

KINDAreplaces $50/mosaves $600/yrback to the verdict

0%0 of 16 items done

Saved on this device only. Tick prerequisites first, then work the phases in order · do not start one until the checks above it pass.

A local task API whose hard problems are keeping a response contract stable while the model underneath it is retired on a published schedule, never letting a failed extraction become a stored result, and charging a job to your card exactly once.

estimated effort weekendthe files for this build are in the project pack

RuntimeNode 22 and Fastify bound to the loopback address, no browser interfaceStorageSQLite for jobs and results, with prompts and schemas as files on diskTask definitionOne directory per task holding prompt.md, schema.json, a dated model id and a golden setExecutionAsynchronous by default: a post returns a job id and one worker drains the queueValidationEvery response checked against that task's schema before anything is stored

Before step 1

Everything below is assumed from the first step. Tick each one when you actually have it, not when you plan to.

installfree
Why Runs the API and the worker.
Get it nodejs.org, or your version manager. open ↗
Verify node --version prints v22 or higher
API keypay per call; a few hundred jobs a month is small, and the ceiling is what keeps it that way
Why Every task is one call, and this is the only part of the build that costs money.
Get it Create a key and set a spend limit in the provider's console as well as the ceiling in this app.
Verify One task returns schema-valid JSON
have readyfree
Why The golden set is the only thing standing between a model change and a silent change in your output, and clean inputs never catch anything.
Get it Pull them from whatever system you were going to point this at.
Verify Each task directory has a golden set with at least one input you would call difficult
decidefree
Why Provider retirement is scheduled and announced, and a task pinned to an alias changes shape on somebody else's release day.
Get it Read the provider's deprecation page, write the exact dated identifier and the published retirement date into the task file.
Verify Every task file has a dated identifier and a date, and none uses an alias

Data model

Create these before the first phase that stores anything. Changing a table later is the expensive kind of change.

- tasks(name, dir, model_id, prompt_sha, schema_sha, created_at, retire_after) · retire_after is the date the pinned model stops working, copied by hand from the provider's own deprecation page
- jobs(id, task, input_ref, idempotency_key, state, attempts, claimed_by, created_at, started_at, finished_at) · state is queued, running, succeeded, failed or invalid
- results(job_id, raw_response, parsed_json, schema_ok, input_tokens, output_tokens, cost_minor, model_id, prompt_sha)
- failures(job_id, attempt, kind, detail, at) · kind is transport, rate_limit, refusal, schema or timeout, because those are four different problems and one of them is not retryable
- golden(task, case_id, input_ref, expected_json, notes) · the fixed inputs a task is re-checked against whenever anything moves
- runs(id, task, model_id, at, n_cases, n_pass, n_changed, diff_ref) · one golden run, so swapping a model is a measurement rather than a hope
- Every stored result carries the model id and the prompt hash that produced it. A task whose output changed cannot be debugged without knowing which of those two moved.

Environment variables

These go in a .env file the app reads at startup. The pack's .env.example is this table as a file · copy it, never commit the filled-in version.

Variable	Needed	Example	Where the value comes from
`MODEL_API_KEY`secret	required	`sk-...redacted`	The model provider. Give it a provider-side spend limit too.
`TASKS_DIR`	required	`./tasks`	One directory per task. This is the whole catalogue.
`DATABASE_PATH`	required	`./data/tasks.db`	Jobs, results, failures and golden runs. Back this up.
`API_TOKEN`secret	required	`a long random string`	Required on every route including job status. Rotate it by changing this.
`BIND_ADDRESS`	required	`127.0.0.1`	Loopback by default. Changing it is a deliberate act with consequences.
`MONTHLY_COST_CEILING`	required	`20.00`	New jobs are refused above this. In-flight jobs still finish.
`RETIREMENT_WARN_DAYS`	required	`45`	Startup warns this far before a pinned model's published retirement date.

The build, in order

1
One task, one folder, one contract
A task directory is loaded at startup and its endpoint either returns schema-valid JSON or an error, never prose.
1. The hash is what lets you tell a prompt change from a model change later.
  Files src/tasks/load.tstasks/summarise/prompt.mdtasks/summarise/schema.json
2. The schema is the contract callers depend on, so it is the gate.
  Files src/run/validate.tstests/validate.test.ts
3. You cannot diagnose a bad extraction you threw away.
  Files src/store/results.ts
done when · tick each as it passes
watch out
- Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result.
next phase
2
Jobs that bill exactly once
A post returns a job id immediately and a crashed worker never pays for the same call twice.
1. The window between call and write is where double billing lives.
  Files src/worker/claim.tstests/claim.test.ts
2. A database constraint is the only deduplication that survives a crash.
  Files src/db/migrations/002_idempotency.sql
3. Retrying a refusal burns money to get the same answer.
  Files src/run/failures.tstests/failures.test.ts
done when · tick each as it passes
watch out
- Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo.
next phase
3
A golden set per task
Every task has fixed inputs with expected outputs, and one command says what changed.
1. Clean inputs never catch a regression.
  Files tasks/summarise/golden/src/golden/run.ts
2. You need to know which field moved, not that something did.
  Files src/golden/diff.tstests/diff.test.ts
3. The bug you just found is the test you did not have.
  Files src/cli/promote.ts
done when · tick each as it passes
watch out
- A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day.
next phase
4
Cost you can see and a door you can shut
Every job carries what it cost, the month has a ceiling, and only your own services can call in.
1. Your estimate and their meter will disagree, and theirs is the one that bills.
  Files src/run/cost.ts
2. A warning in a log is not a limit.
  Files src/worker/ceiling.tstests/ceiling.test.ts
3. The status route leaks results, so it is not the safe one.
  Files src/http/auth.ts
done when · tick each as it passes
watch out
- A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning.
5
Living with a retirement dateproduct builder
The pinned model's end date is known in advance, and migrating to its replacement is a measured diff rather than a leap.
1. An alias is a change you did not schedule.
  Files tasks/*/task.jsontests/no-alias.test.ts
2. Providers publish these. Nobody reads them on the day.
  Files src/tasks/retirement.ts
3. The diff is the evidence that the contract survived.
  Files src/cli/migrate.ts
done when · tick each as it passes
watch out
- Pointing every task at a convenience alias that always means the newest model turns a provider release into an unannounced change in your output shape, at an hour you did not choose, with no diff to read. Pin dated identifiers, keep the retirement dates in the database, and let the golden set decide when the pin moves.

what this build does not replace

Packaged SDKs in other languages.
Billing tiers and per-caller quotas.
The other tasks in somebody else's catalogue.
Fine-tuning or self-hosted models.
tuned prompts per use case, tested against messy real-world inputs
response contracts that stay stable when the underlying models change
the managed async queue with polling, throttling, and rate limits
ready SDKs for PHP/Laravel, Node, and Python
80+ language coverage tested per endpoint

after v1, if you want it

A second provider behind the same task folder, chosen per task, once the golden set can tell you whether it is worse
Batch submission for tasks where latency does not matter, which is usually much cheaper per call
A small dashboard over the failures table, because the kinds of failure are more interesting than the count

Need the files? The project pack on the verdict page hands your agent the whole brief · more dev tools.