Vibecode SharpAPI
track this build4 phases, 12 steps, beginner friendly0%KINDA · weekend project
Any single endpoint here, summarize a text, categorize a product, parse a resume, is one LLM call with a decent prompt and a JSON schema, so the piece you actually need is often an evening of work. What you will not rebuild in a weekend is the catalog: 30+ endpoints with tuned prompts, response contracts that survive model upgrades, an async job queue with polling, and maintained SDKs. Rebuild the two or three endpoints you use, pay when you need the shelf.
the project pack5 files · written for this build, step by step
You are building a lean indie version of SharpAPI. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # SharpAPI · indie build A local task API whose hard problems are keeping a response contract stable while the model underneath it is retired on a published schedule, never letting a failed extraction become a stored result, and charging a job to your card exactly once. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 and Fastify bound to the loopback address, no browser interface | this is something your other services call, not something a person opens | | Storage | SQLite for jobs and results, with prompts and schemas as files on disk | a task definition you can diff and revert beats one buried in a table | | Task definition | One directory per task holding prompt.md, schema.json, a dated model id and a golden set | adding a task is adding a folder, and that is the only way this grows without turning into a framework | | Execution | Asynchronous by default: a post returns a job id and one worker drains the queue | a model call that takes forty seconds must not hold an HTTP connection open | | Validation | Every response checked against that task's schema before anything is stored | an unvalidated response is a guess wearing a content type | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the API and the worker. - Get it: nodejs.org, or your version manager. - Verify: node --version prints v22 or higher - [ ] **An API key for a model provider with structured output** · pay per call; a few hundred jobs a month is small, and the ceiling is what keeps it that way - Why: Every task is one call, and this is the only part of the build that costs money. - Get it: Create a key and set a spend limit in the provider's console as well as the ceiling in this app. - Verify: One task returns schema-valid JSON - [ ] **Ten real inputs per task, including the ugly ones** · free - Why: The golden set is the only thing standing between a model change and a silent change in your output, and clean inputs never catch anything. - Get it: Pull them from whatever system you were going to point this at. - Verify: Each task directory has a golden set with at least one input you would call difficult - [ ] **Which dated model each task pins, and its retirement date** · free - Why: Provider retirement is scheduled and announced, and a task pinned to an alias changes shape on somebody else's release day. - Get it: Read the provider's deprecation page, write the exact dated identifier and the published retirement date into the task file. - Verify: Every task file has a dated identifier and a date, and none uses an alias ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Packaged SDKs in other languages. - Billing tiers and per-caller quotas. - The other tasks in somebody else's catalogue. - Fine-tuning or self-hosted models. - tuned prompts per use case, tested against messy real-world inputs - response contracts that stay stable when the underlying models change - the managed async queue with polling, throttling, and rate limits - ready SDKs for PHP/Laravel, Node, and Python - 80+ language coverage tested per endpoint If one of those is essential to you, that is the reason to keep paying for SharpAPI, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · SharpAPI The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a small self-hosted task API that turns one prompt file and one JSON schema into an endpoint your other projects can call to replace SharpAPI. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Runtime: Node 22 and Fastify bound to the loopback address, no browser interface · this is something your other services call, not something a person opens - Storage: SQLite for jobs and results, with prompts and schemas as files on disk · a task definition you can diff and revert beats one buried in a table - Task definition: One directory per task holding prompt.md, schema.json, a dated model id and a golden set · adding a task is adding a folder, and that is the only way this grows without turning into a framework - Execution: Asynchronous by default: a post returns a job id and one worker drains the queue · a model call that takes forty seconds must not hold an HTTP connection open - Validation: Every response checked against that task's schema before anything is stored · an unvalidated response is a guess wearing a content type ### Data model (create this before Phase 1) - tasks(name, dir, model_id, prompt_sha, schema_sha, created_at, retire_after) · retire_after is the date the pinned model stops working, copied by hand from the provider's own deprecation page - jobs(id, task, input_ref, idempotency_key, state, attempts, claimed_by, created_at, started_at, finished_at) · state is queued, running, succeeded, failed or invalid - results(job_id, raw_response, parsed_json, schema_ok, input_tokens, output_tokens, cost_minor, model_id, prompt_sha) - failures(job_id, attempt, kind, detail, at) · kind is transport, rate_limit, refusal, schema or timeout, because those are four different problems and one of them is not retryable - golden(task, case_id, input_ref, expected_json, notes) · the fixed inputs a task is re-checked against whenever anything moves - runs(id, task, model_id, at, n_cases, n_pass, n_changed, diff_ref) · one golden run, so swapping a model is a measurement rather than a hope - Every stored result carries the model id and the prompt hash that produced it. A task whose output changed cannot be debugged without knowing which of those two moved. ### Phase 1 · One task, one folder, one contract Build: Load every directory under tasks/ into the tasks table, hashing the prompt and the schema. Generate one route per task. Call the model with structured output against that task's schema, validate the parsed response against the same schema file, and store the raw response either way. Trap: Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. Done when: Adding a directory with prompt.md and schema.json produces a working endpoint with no code change, tested by adding one. A response that does not validate is stored as state invalid with the raw text kept, and the endpoint returns an error rather than a partial object, tested with a stubbed provider returning prose. A model refusal is recorded as kind refusal and is never retried. The prompt hash and schema hash are recorded on the task row at load time. Do not build yet: No queue, no retries, no cost accounting and no golden set yet. ### Phase 2 · Jobs that bill exactly once Build: A jobs table drained by one worker. Claim a row with a conditional update that sets state and claimed_by before the model call, not after. Accept an idempotency key from the caller and make it unique, so a retried post returns the existing job instead of creating a second one. Trap: Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. Done when: Two posts with the same idempotency key produce one job and one model call, tested. Killing the worker mid-call leaves a row in running with its request id, and restarting does not issue a second call for it, tested by killing the process. Backoff on a rate limit honours the provider's retry header when one is present rather than a fixed sleep. A transport failure retries and a refusal does not, asserted in the suite. Do not build yet: No golden set, no cost ceiling and no callers other than curl yet. ### Phase 3 · A golden set per task Build: For each task keep a handful of real inputs under golden/ with the output you accept. A run command executes them against the current pinned model, compares field by field, and reports pass, changed and failed with a readable diff. Store the run so two runs can be compared. Trap: A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. Done when: Each task has at least one golden case and the run command exits non-zero when any case fails, tested by breaking a schema. The diff shows which fields changed rather than printing two blobs. A run is stored with its model id, case count and pass count, so two runs are comparable. Promoting a live job's input into the golden set takes one command. Do not build yet: No cost ceiling, no access control and no migration tooling yet. ### Phase 4 · Cost you can see and a door you can shut Build: Record input and output tokens and a computed cost in minor units on every result, from the provider's usage figures rather than an estimate. Sum per task per month. A shared token in a header gates every route, and the process binds to the loopback address by default. Trap: A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning. Done when: Every result row has a non-null cost in minor units taken from the provider's reported usage, checked against one real call. Reaching the monthly ceiling refuses new jobs with a clear error and leaves in-flight ones to finish, tested at the boundary. A request without the token is rejected on every route including the job status route. The process refuses to bind to a non-loopback address unless a variable is explicitly set. Do not build yet: No per-caller quotas, no SDKs and no admin interface. ### Phase 5 · Living with a retirement date Build: Pin an exact dated model identifier per task, never a moving alias. Store the retirement date from the provider's deprecation page on the task row and warn on startup when it is close. A migrate command runs the golden set against a candidate model and shows the per-field diff before you change the pin. Export every job, result and run as JSONL. Trap: Pointing every task at a convenience alias that always means the newest model turns a provider release into an unannounced change in your output shape, at an hour you did not choose, with no diff to read. Pin dated identifiers, keep the retirement dates in the database, and let the golden set decide when the pin moves. Done when: No task references a moving alias, asserted by a test that scans the task files. Each task row carries a retirement date and startup prints a warning inside a configured window before it. The migrate command runs the golden set against a candidate and prints a per-field diff without changing any pin, tested. Export produces JSONL for jobs, results and runs, and reimports into an empty database with identical counts. Do not build yet: Nothing after this. ### Out of scope (and why) - Packaged client SDKs for other languages, since your callers can send a POST and the maintenance of three SDKs is the actual product you are not buying. - Billing tiers, credit accounting and per-caller rate limits, which only matter when the callers are strangers. - The rest of the catalogue. Build the two or three tasks you use and let the folder convention make the fourth cheap. - Fine-tuning or hosting your own model, which changes this from a weekend of glue into an infrastructure project. ### README must contain - That a task is a directory, and exactly which files it must contain - That the pinned model identifier is dated, where its retirement date came from, and when it was last checked - That every result records the model id and prompt hash, and why you cannot debug an output without both - What one call of each task costs, measured rather than estimated, and where the monthly ceiling is set - That the service binds to loopback and requires a token, and what to change if you move it ===== AGENTS.md ===== # Agent instructions · SharpAPI indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 and Fastify bound to the loopback address, no browser interface, SQLite for jobs and results, with prompts and schemas as files on disk, One directory per task holding prompt.md, schema.json, a dated model id and a golden set, Asynchronous by default: a post returns a job id and one worker drains the queue, Every response checked against that task's schema before anything is stored. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. - Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. - A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. - A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning. ===== BUILD_PLAN.md ===== # Build plan · SharpAPI A local task API whose hard problems are keeping a response contract stable while the model underneath it is retired on a published schedule, never letting a failed extraction become a stored result, and charging a job to your card exactly once. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · One task, one folder, one contract A task directory is loaded at startup and its endpoint either returns schema-valid JSON or an error, never prose. ### Steps 1. Load tasks from the filesystem, hash both files The hash is what lets you tell a prompt change from a model change later. Files: `src/tasks/load.ts`, `tasks/summarise/prompt.md`, `tasks/summarise/schema.json` 2. Validate every response against the task's own schema The schema is the contract callers depend on, so it is the gate. Files: `src/run/validate.ts`, `tests/validate.test.ts` 3. Keep the raw response even when it is rubbish You cannot diagnose a bad extraction you threw away. Files: `src/store/results.ts` ### Done when - [ ] A new task directory becomes an endpoint with no code change - [ ] A non-validating response is stored as invalid, not as a result - [ ] A refusal is classified as a refusal and not retried - [ ] Prompt and schema hashes are recorded per task ### Watch out - Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. ## Phase 2 · Jobs that bill exactly once A post returns a job id immediately and a crashed worker never pays for the same call twice. ### Steps 1. Claim the row before the call, not after The window between call and write is where double billing lives. Files: `src/worker/claim.ts`, `tests/claim.test.ts` 2. Make the caller's idempotency key unique in the schema A database constraint is the only deduplication that survives a crash. Files: `src/db/migrations/002_idempotency.sql` 3. Classify failures into four kinds and retry only two Retrying a refusal burns money to get the same answer. Files: `src/run/failures.ts`, `tests/failures.test.ts` ### Done when - [ ] A repeated idempotency key yields one job and one call - [ ] A killed worker does not reissue a call it may have paid for - [ ] Rate limit backoff uses the provider's own retry hint - [ ] Refusals are not retried ### Watch out - Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. ## Phase 3 · A golden set per task Every task has fixed inputs with expected outputs, and one command says what changed. ### Steps 1. Put your worst real inputs in the golden set Clean inputs never catch a regression. Files: `tasks/summarise/golden/`, `src/golden/run.ts` 2. Diff field by field, not blob against blob You need to know which field moved, not that something did. Files: `src/golden/diff.ts`, `tests/diff.test.ts` 3. Promote a bad live result into golden in one command The bug you just found is the test you did not have. Files: `src/cli/promote.ts` ### Done when - [ ] The run command fails loudly on a broken case - [ ] Diffs are per field and readable - [ ] Runs are stored and comparable - [ ] Promoting a live input takes one command ### Watch out - A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. ## Phase 4 · Cost you can see and a door you can shut Every job carries what it cost, the month has a ceiling, and only your own services can call in. ### Steps 1. Take usage from the provider, not from a token estimate Your estimate and their meter will disagree, and theirs is the one that bills. Files: `src/run/cost.ts` 2. Stop queueing at the ceiling A warning in a log is not a limit. Files: `src/worker/ceiling.ts`, `tests/ceiling.test.ts` 3. Gate every route including status The status route leaks results, so it is not the safe one. Files: `src/http/auth.ts` ### Done when - [ ] Costs come from reported usage and are never null - [ ] The monthly ceiling refuses new jobs - [ ] Every route requires the token - [ ] Non-loopback binding needs an explicit opt-in ### Watch out - A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning. ## Not in this build - Packaged SDKs in other languages. - Billing tiers and per-caller quotas. - The other tasks in somebody else's catalogue. - Fine-tuning or self-hosted models. ## After v1, if you want it - A second provider behind the same task folder, chosen per task, once the golden set can tell you whether it is worse - Batch submission for tasks where latency does not matter, which is usually much cheaper per call - A small dashboard over the failures table, because the kinds of failure are more interesting than the count ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. The model provider. Give it a provider-side spend limit too. MODEL_API_KEY=sk-...redacted # Required. One directory per task. This is the whole catalogue. TASKS_DIR=./tasks # Required. Jobs, results, failures and golden runs. Back this up. DATABASE_PATH=./data/tasks.db # Required · secret. Required on every route including job status. Rotate it by changing this. API_TOKEN=a long random string # Required. Loopback by default. Changing it is a deliberate act with consequences. BIND_ADDRESS=127.0.0.1 # Required. New jobs are refused above this. In-flight jobs still finish. MONTHLY_COST_CEILING=20.00 # Required. Startup warns this far before a pinned model's published retirement date. RETIREMENT_WARN_DAYS=45
You are building a lean indie version of SharpAPI. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # SharpAPI · indie build A local task API whose hard problems are keeping a response contract stable while the model underneath it is retired on a published schedule, never letting a failed extraction become a stored result, and charging a job to your card exactly once. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 and Fastify bound to the loopback address, no browser interface | this is something your other services call, not something a person opens | | Storage | SQLite for jobs and results, with prompts and schemas as files on disk | a task definition you can diff and revert beats one buried in a table | | Task definition | One directory per task holding prompt.md, schema.json, a dated model id and a golden set | adding a task is adding a folder, and that is the only way this grows without turning into a framework | | Execution | Asynchronous by default: a post returns a job id and one worker drains the queue | a model call that takes forty seconds must not hold an HTTP connection open | | Validation | Every response checked against that task's schema before anything is stored | an unvalidated response is a guess wearing a content type | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the API and the worker. - Get it: nodejs.org, or your version manager. - Verify: node --version prints v22 or higher - [ ] **An API key for a model provider with structured output** · pay per call; a few hundred jobs a month is small, and the ceiling is what keeps it that way - Why: Every task is one call, and this is the only part of the build that costs money. - Get it: Create a key and set a spend limit in the provider's console as well as the ceiling in this app. - Verify: One task returns schema-valid JSON - [ ] **Ten real inputs per task, including the ugly ones** · free - Why: The golden set is the only thing standing between a model change and a silent change in your output, and clean inputs never catch anything. - Get it: Pull them from whatever system you were going to point this at. - Verify: Each task directory has a golden set with at least one input you would call difficult - [ ] **Which dated model each task pins, and its retirement date** · free - Why: Provider retirement is scheduled and announced, and a task pinned to an alias changes shape on somebody else's release day. - Get it: Read the provider's deprecation page, write the exact dated identifier and the published retirement date into the task file. - Verify: Every task file has a dated identifier and a date, and none uses an alias ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Packaged SDKs in other languages. - Billing tiers and per-caller quotas. - The other tasks in somebody else's catalogue. - Fine-tuning or self-hosted models. - tuned prompts per use case, tested against messy real-world inputs - response contracts that stay stable when the underlying models change - the managed async queue with polling, throttling, and rate limits - ready SDKs for PHP/Laravel, Node, and Python - 80+ language coverage tested per endpoint If one of those is essential to you, that is the reason to keep paying for SharpAPI, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · SharpAPI The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a small self-hosted task API that turns one prompt file and one JSON schema into an endpoint your other projects can call to replace SharpAPI. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Runtime: Node 22 and Fastify bound to the loopback address, no browser interface · this is something your other services call, not something a person opens - Storage: SQLite for jobs and results, with prompts and schemas as files on disk · a task definition you can diff and revert beats one buried in a table - Task definition: One directory per task holding prompt.md, schema.json, a dated model id and a golden set · adding a task is adding a folder, and that is the only way this grows without turning into a framework - Execution: Asynchronous by default: a post returns a job id and one worker drains the queue · a model call that takes forty seconds must not hold an HTTP connection open - Validation: Every response checked against that task's schema before anything is stored · an unvalidated response is a guess wearing a content type ### Data model (create this before Phase 1) - tasks(name, dir, model_id, prompt_sha, schema_sha, created_at, retire_after) · retire_after is the date the pinned model stops working, copied by hand from the provider's own deprecation page - jobs(id, task, input_ref, idempotency_key, state, attempts, claimed_by, created_at, started_at, finished_at) · state is queued, running, succeeded, failed or invalid - results(job_id, raw_response, parsed_json, schema_ok, input_tokens, output_tokens, cost_minor, model_id, prompt_sha) - failures(job_id, attempt, kind, detail, at) · kind is transport, rate_limit, refusal, schema or timeout, because those are four different problems and one of them is not retryable - golden(task, case_id, input_ref, expected_json, notes) · the fixed inputs a task is re-checked against whenever anything moves - runs(id, task, model_id, at, n_cases, n_pass, n_changed, diff_ref) · one golden run, so swapping a model is a measurement rather than a hope - Every stored result carries the model id and the prompt hash that produced it. A task whose output changed cannot be debugged without knowing which of those two moved. ### Phase 1 · One task, one folder, one contract Build: Load every directory under tasks/ into the tasks table, hashing the prompt and the schema. Generate one route per task. Call the model with structured output against that task's schema, validate the parsed response against the same schema file, and store the raw response either way. Trap: Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. Done when: Adding a directory with prompt.md and schema.json produces a working endpoint with no code change, tested by adding one. A response that does not validate is stored as state invalid with the raw text kept, and the endpoint returns an error rather than a partial object, tested with a stubbed provider returning prose. A model refusal is recorded as kind refusal and is never retried. The prompt hash and schema hash are recorded on the task row at load time. Do not build yet: No queue, no retries, no cost accounting and no golden set yet. ### Phase 2 · Jobs that bill exactly once Build: A jobs table drained by one worker. Claim a row with a conditional update that sets state and claimed_by before the model call, not after. Accept an idempotency key from the caller and make it unique, so a retried post returns the existing job instead of creating a second one. Trap: Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. Done when: Two posts with the same idempotency key produce one job and one model call, tested. Killing the worker mid-call leaves a row in running with its request id, and restarting does not issue a second call for it, tested by killing the process. Backoff on a rate limit honours the provider's retry header when one is present rather than a fixed sleep. A transport failure retries and a refusal does not, asserted in the suite. Do not build yet: No golden set, no cost ceiling and no callers other than curl yet. ### Phase 3 · A golden set per task Build: For each task keep a handful of real inputs under golden/ with the output you accept. A run command executes them against the current pinned model, compares field by field, and reports pass, changed and failed with a readable diff. Store the run so two runs can be compared. Trap: A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. Done when: Each task has at least one golden case and the run command exits non-zero when any case fails, tested by breaking a schema. The diff shows which fields changed rather than printing two blobs. A run is stored with its model id, case count and pass count, so two runs are comparable. Promoting a live job's input into the golden set takes one command. Do not build yet: No cost ceiling, no access control and no migration tooling yet. ### Phase 4 · Cost you can see and a door you can shut Build: Record input and output tokens and a computed cost in minor units on every result, from the provider's usage figures rather than an estimate. Sum per task per month. A shared token in a header gates every route, and the process binds to the loopback address by default. Trap: A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning. Done when: Every result row has a non-null cost in minor units taken from the provider's reported usage, checked against one real call. Reaching the monthly ceiling refuses new jobs with a clear error and leaves in-flight ones to finish, tested at the boundary. A request without the token is rejected on every route including the job status route. The process refuses to bind to a non-loopback address unless a variable is explicitly set. Do not build yet: No per-caller quotas, no SDKs and no admin interface. ### Phase 5 · Living with a retirement date Build: Pin an exact dated model identifier per task, never a moving alias. Store the retirement date from the provider's deprecation page on the task row and warn on startup when it is close. A migrate command runs the golden set against a candidate model and shows the per-field diff before you change the pin. Export every job, result and run as JSONL. Trap: Pointing every task at a convenience alias that always means the newest model turns a provider release into an unannounced change in your output shape, at an hour you did not choose, with no diff to read. Pin dated identifiers, keep the retirement dates in the database, and let the golden set decide when the pin moves. Done when: No task references a moving alias, asserted by a test that scans the task files. Each task row carries a retirement date and startup prints a warning inside a configured window before it. The migrate command runs the golden set against a candidate and prints a per-field diff without changing any pin, tested. Export produces JSONL for jobs, results and runs, and reimports into an empty database with identical counts. Do not build yet: Nothing after this. ### Out of scope (and why) - Packaged client SDKs for other languages, since your callers can send a POST and the maintenance of three SDKs is the actual product you are not buying. - Billing tiers, credit accounting and per-caller rate limits, which only matter when the callers are strangers. - The rest of the catalogue. Build the two or three tasks you use and let the folder convention make the fourth cheap. - Fine-tuning or hosting your own model, which changes this from a weekend of glue into an infrastructure project. ### README must contain - That a task is a directory, and exactly which files it must contain - That the pinned model identifier is dated, where its retirement date came from, and when it was last checked - That every result records the model id and prompt hash, and why you cannot debug an output without both - What one call of each task costs, measured rather than estimated, and where the monthly ceiling is set - That the service binds to loopback and requires a token, and what to change if you move it ===== AGENTS.md ===== # Agent instructions · SharpAPI indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 and Fastify bound to the loopback address, no browser interface, SQLite for jobs and results, with prompts and schemas as files on disk, One directory per task holding prompt.md, schema.json, a dated model id and a golden set, Asynchronous by default: a post returns a job id and one worker drains the queue, Every response checked against that task's schema before anything is stored. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. - Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. - A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. - A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning. ===== BUILD_PLAN.md ===== # Build plan · SharpAPI A local task API whose hard problems are keeping a response contract stable while the model underneath it is retired on a published schedule, never letting a failed extraction become a stored result, and charging a job to your card exactly once. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · One task, one folder, one contract A task directory is loaded at startup and its endpoint either returns schema-valid JSON or an error, never prose. ### Steps 1. Load tasks from the filesystem, hash both files The hash is what lets you tell a prompt change from a model change later. Files: `src/tasks/load.ts`, `tasks/summarise/prompt.md`, `tasks/summarise/schema.json` 2. Validate every response against the task's own schema The schema is the contract callers depend on, so it is the gate. Files: `src/run/validate.ts`, `tests/validate.test.ts` 3. Keep the raw response even when it is rubbish You cannot diagnose a bad extraction you threw away. Files: `src/store/results.ts` ### Done when - [ ] A new task directory becomes an endpoint with no code change - [ ] A non-validating response is stored as invalid, not as a result - [ ] A refusal is classified as a refusal and not retried - [ ] Prompt and schema hashes are recorded per task ### Watch out - Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. ## Phase 2 · Jobs that bill exactly once A post returns a job id immediately and a crashed worker never pays for the same call twice. ### Steps 1. Claim the row before the call, not after The window between call and write is where double billing lives. Files: `src/worker/claim.ts`, `tests/claim.test.ts` 2. Make the caller's idempotency key unique in the schema A database constraint is the only deduplication that survives a crash. Files: `src/db/migrations/002_idempotency.sql` 3. Classify failures into four kinds and retry only two Retrying a refusal burns money to get the same answer. Files: `src/run/failures.ts`, `tests/failures.test.ts` ### Done when - [ ] A repeated idempotency key yields one job and one call - [ ] A killed worker does not reissue a call it may have paid for - [ ] Rate limit backoff uses the provider's own retry hint - [ ] Refusals are not retried ### Watch out - Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. ## Phase 3 · A golden set per task Every task has fixed inputs with expected outputs, and one command says what changed. ### Steps 1. Put your worst real inputs in the golden set Clean inputs never catch a regression. Files: `tasks/summarise/golden/`, `src/golden/run.ts` 2. Diff field by field, not blob against blob You need to know which field moved, not that something did. Files: `src/golden/diff.ts`, `tests/diff.test.ts` 3. Promote a bad live result into golden in one command The bug you just found is the test you did not have. Files: `src/cli/promote.ts` ### Done when - [ ] The run command fails loudly on a broken case - [ ] Diffs are per field and readable - [ ] Runs are stored and comparable - [ ] Promoting a live input takes one command ### Watch out - A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. ## Phase 4 · Cost you can see and a door you can shut Every job carries what it cost, the month has a ceiling, and only your own services can call in. ### Steps 1. Take usage from the provider, not from a token estimate Your estimate and their meter will disagree, and theirs is the one that bills. Files: `src/run/cost.ts` 2. Stop queueing at the ceiling A warning in a log is not a limit. Files: `src/worker/ceiling.ts`, `tests/ceiling.test.ts` 3. Gate every route including status The status route leaks results, so it is not the safe one. Files: `src/http/auth.ts` ### Done when - [ ] Costs come from reported usage and are never null - [ ] The monthly ceiling refuses new jobs - [ ] Every route requires the token - [ ] Non-loopback binding needs an explicit opt-in ### Watch out - A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning. ## Not in this build - Packaged SDKs in other languages. - Billing tiers and per-caller quotas. - The other tasks in somebody else's catalogue. - Fine-tuning or self-hosted models. ## After v1, if you want it - A second provider behind the same task folder, chosen per task, once the golden set can tell you whether it is worse - Batch submission for tasks where latency does not matter, which is usually much cheaper per call - A small dashboard over the failures table, because the kinds of failure are more interesting than the count ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. The model provider. Give it a provider-side spend limit too. MODEL_API_KEY=sk-...redacted # Required. One directory per task. This is the whole catalogue. TASKS_DIR=./tasks # Required. Jobs, results, failures and golden runs. Back this up. DATABASE_PATH=./data/tasks.db # Required · secret. Required on every route including job status. Rotate it by changing this. API_TOKEN=a long random string # Required. Loopback by default. Changing it is a deliberate act with consequences. BIND_ADDRESS=127.0.0.1 # Required. New jobs are refused above this. In-flight jobs still finish. MONTHLY_COST_CEILING=20.00 # Required. Startup warns this far before a pinned model's published retirement date. RETIREMENT_WARN_DAYS=45
You are building a production product version of SharpAPI. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # SharpAPI · product brief ## Problem Any single endpoint here, summarize a text, categorize a product, parse a resume, is one LLM call with a decent prompt and a JSON schema, so the piece you actually need is often an evening of work. What you will not rebuild in a weekend is the catalog: 30+ endpoints with tuned prompts, response contracts that survive model upgrades, an async job queue with polling, and maintained SDKs. Rebuild the two or three endpoints you use, pay when you need the shelf. ## Product outcome A task API where adding a capability is adding a folder, where a bad model response is an error rather than a row, where a crash never bills twice, and where a model retirement is something you migrate through with a diff in your hand. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - OpenAI or Anthropic API key - a JSON schema per task for structured outputs - a job queue (BullMQ + Redis, or a SQLite job table) - real sample inputs to calibrate each prompt against ## Explicit non-goals for v1 - Packaged SDKs in other languages. - Billing tiers and per-caller quotas. - The other tasks in somebody else's catalogue. - Fine-tuning or self-hosted models. - tuned prompts per use case, tested against messy real-world inputs - response contracts that stay stable when the underlying models change - the managed async queue with polling, throttling, and rate limits - ready SDKs for PHP/Laravel, Node, and Python - 80+ language coverage tested per endpoint ## Success criteria - A non-validating response is confirmed to store no result - A repeated idempotency key is confirmed to produce one call - The monthly ceiling is confirmed to refuse new jobs - No task is confirmed to reference a moving model alias - Export and reimport are confirmed to round-trip every table ===== BRIEF.md ===== # Build brief · SharpAPI The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a small self-hosted task API that turns one prompt file and one JSON schema into an endpoint your other projects can call to replace SharpAPI. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Runtime: Node 22 and Fastify bound to the loopback address, no browser interface · this is something your other services call, not something a person opens - Storage: SQLite for jobs and results, with prompts and schemas as files on disk · a task definition you can diff and revert beats one buried in a table - Task definition: One directory per task holding prompt.md, schema.json, a dated model id and a golden set · adding a task is adding a folder, and that is the only way this grows without turning into a framework - Execution: Asynchronous by default: a post returns a job id and one worker drains the queue · a model call that takes forty seconds must not hold an HTTP connection open - Validation: Every response checked against that task's schema before anything is stored · an unvalidated response is a guess wearing a content type ### Data model (create this before Phase 1) - tasks(name, dir, model_id, prompt_sha, schema_sha, created_at, retire_after) · retire_after is the date the pinned model stops working, copied by hand from the provider's own deprecation page - jobs(id, task, input_ref, idempotency_key, state, attempts, claimed_by, created_at, started_at, finished_at) · state is queued, running, succeeded, failed or invalid - results(job_id, raw_response, parsed_json, schema_ok, input_tokens, output_tokens, cost_minor, model_id, prompt_sha) - failures(job_id, attempt, kind, detail, at) · kind is transport, rate_limit, refusal, schema or timeout, because those are four different problems and one of them is not retryable - golden(task, case_id, input_ref, expected_json, notes) · the fixed inputs a task is re-checked against whenever anything moves - runs(id, task, model_id, at, n_cases, n_pass, n_changed, diff_ref) · one golden run, so swapping a model is a measurement rather than a hope - Every stored result carries the model id and the prompt hash that produced it. A task whose output changed cannot be debugged without knowing which of those two moved. ### Phase 1 · One task, one folder, one contract Build: Load every directory under tasks/ into the tasks table, hashing the prompt and the schema. Generate one route per task. Call the model with structured output against that task's schema, validate the parsed response against the same schema file, and store the raw response either way. Trap: Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. Done when: Adding a directory with prompt.md and schema.json produces a working endpoint with no code change, tested by adding one. A response that does not validate is stored as state invalid with the raw text kept, and the endpoint returns an error rather than a partial object, tested with a stubbed provider returning prose. A model refusal is recorded as kind refusal and is never retried. The prompt hash and schema hash are recorded on the task row at load time. Do not build yet: No queue, no retries, no cost accounting and no golden set yet. ### Phase 2 · Jobs that bill exactly once Build: A jobs table drained by one worker. Claim a row with a conditional update that sets state and claimed_by before the model call, not after. Accept an idempotency key from the caller and make it unique, so a retried post returns the existing job instead of creating a second one. Trap: Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. Done when: Two posts with the same idempotency key produce one job and one model call, tested. Killing the worker mid-call leaves a row in running with its request id, and restarting does not issue a second call for it, tested by killing the process. Backoff on a rate limit honours the provider's retry header when one is present rather than a fixed sleep. A transport failure retries and a refusal does not, asserted in the suite. Do not build yet: No golden set, no cost ceiling and no callers other than curl yet. ### Phase 3 · A golden set per task Build: For each task keep a handful of real inputs under golden/ with the output you accept. A run command executes them against the current pinned model, compares field by field, and reports pass, changed and failed with a readable diff. Store the run so two runs can be compared. Trap: A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. Done when: Each task has at least one golden case and the run command exits non-zero when any case fails, tested by breaking a schema. The diff shows which fields changed rather than printing two blobs. A run is stored with its model id, case count and pass count, so two runs are comparable. Promoting a live job's input into the golden set takes one command. Do not build yet: No cost ceiling, no access control and no migration tooling yet. ### Phase 4 · Cost you can see and a door you can shut Build: Record input and output tokens and a computed cost in minor units on every result, from the provider's usage figures rather than an estimate. Sum per task per month. A shared token in a header gates every route, and the process binds to the loopback address by default. Trap: A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning. Done when: Every result row has a non-null cost in minor units taken from the provider's reported usage, checked against one real call. Reaching the monthly ceiling refuses new jobs with a clear error and leaves in-flight ones to finish, tested at the boundary. A request without the token is rejected on every route including the job status route. The process refuses to bind to a non-loopback address unless a variable is explicitly set. Do not build yet: No per-caller quotas, no SDKs and no admin interface. ### Phase 5 · Living with a retirement date Build: Pin an exact dated model identifier per task, never a moving alias. Store the retirement date from the provider's deprecation page on the task row and warn on startup when it is close. A migrate command runs the golden set against a candidate model and shows the per-field diff before you change the pin. Export every job, result and run as JSONL. Trap: Pointing every task at a convenience alias that always means the newest model turns a provider release into an unannounced change in your output shape, at an hour you did not choose, with no diff to read. Pin dated identifiers, keep the retirement dates in the database, and let the golden set decide when the pin moves. Done when: No task references a moving alias, asserted by a test that scans the task files. Each task row carries a retirement date and startup prints a warning inside a configured window before it. The migrate command runs the golden set against a candidate and prints a per-field diff without changing any pin, tested. Export produces JSONL for jobs, results and runs, and reimports into an empty database with identical counts. Do not build yet: Nothing after this. ### Out of scope (and why) - Packaged client SDKs for other languages, since your callers can send a POST and the maintenance of three SDKs is the actual product you are not buying. - Billing tiers, credit accounting and per-caller rate limits, which only matter when the callers are strangers. - The rest of the catalogue. Build the two or three tasks you use and let the folder convention make the fourth cheap. - Fine-tuning or hosting your own model, which changes this from a weekend of glue into an infrastructure project. ### README must contain - That a task is a directory, and exactly which files it must contain - That the pinned model identifier is dated, where its retirement date came from, and when it was last checked - That every result records the model id and prompt hash, and why you cannot debug an output without both - What one call of each task costs, measured rather than estimated, and where the monthly ceiling is set - That the service binds to loopback and requires a token, and what to change if you move it ===== ARCHITECTURE.md ===== # Architecture · SharpAPI ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 and Fastify bound to the loopback address, no browser interface | this is something your other services call, not something a person opens | | Storage | SQLite for jobs and results, with prompts and schemas as files on disk | a task definition you can diff and revert beats one buried in a table | | Task definition | One directory per task holding prompt.md, schema.json, a dated model id and a golden set | adding a task is adding a folder, and that is the only way this grows without turning into a framework | | Execution | Asynchronous by default: a post returns a job id and one worker drains the queue | a model call that takes forty seconds must not hold an HTTP connection open | | Validation | Every response checked against that task's schema before anything is stored | an unvalidated response is a guess wearing a content type | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Task loader | Directories, prompts, schemas and pins. | Any layout, provided the prompt and schema stay as files you can diff. | | Runner | The model call, validation and failure classification. | Any provider with structured output, provided validation stays the gate. | | Queue | Claiming, idempotency and retries. | Any queue, provided the claim happens before the call. | | Golden | Fixed cases, runs and diffs. | Not swappable for a smoke test, which is what it becomes if you fill it with easy inputs. | | Cost | Reported usage, per-task totals and the ceiling. | Any accounting, provided the figures come from the provider. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `MODEL_API_KEY` · required, secret · The model provider. Give it a provider-side spend limit too. - `TASKS_DIR` · required · One directory per task. This is the whole catalogue. - `DATABASE_PATH` · required · Jobs, results, failures and golden runs. Back this up. - `API_TOKEN` · required, secret · Required on every route including job status. Rotate it by changing this. - `BIND_ADDRESS` · required · Loopback by default. Changing it is a deliberate act with consequences. - `MONTHLY_COST_CEILING` · required · New jobs are refused above this. In-flight jobs still finish. - `RETIREMENT_WARN_DAYS` · required · Startup warns this far before a pinned model's published retirement date. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · SharpAPI product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Node 22 and Fastify bound to the loopback address, no browser interface, SQLite for jobs and results, with prompts and schemas as files on disk, One directory per task holding prompt.md, schema.json, a dated model id and a golden set, Asynchronous by default: a post returns a job id and one worker drains the queue, Every response checked against that task's schema before anything is stored. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. - Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. - A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. - A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning. - Pointing every task at a convenience alias that always means the newest model turns a provider release into an unannounced change in your output shape, at an hour you did not choose, with no diff to read. Pin dated identifiers, keep the retirement dates in the database, and let the golden set decide when the pin moves. ===== MILESTONES.md ===== # Delivery milestones · SharpAPI Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · One task, one folder, one contract A task directory is loaded at startup and its endpoint either returns schema-valid JSON or an error, never prose. ### Steps 1. Load tasks from the filesystem, hash both files The hash is what lets you tell a prompt change from a model change later. Files: `src/tasks/load.ts`, `tasks/summarise/prompt.md`, `tasks/summarise/schema.json` 2. Validate every response against the task's own schema The schema is the contract callers depend on, so it is the gate. Files: `src/run/validate.ts`, `tests/validate.test.ts` 3. Keep the raw response even when it is rubbish You cannot diagnose a bad extraction you threw away. Files: `src/store/results.ts` ### Done when - [ ] A new task directory becomes an endpoint with no code change - [ ] A non-validating response is stored as invalid, not as a result - [ ] A refusal is classified as a refusal and not retried - [ ] Prompt and schema hashes are recorded per task ### Watch out - Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. ## M2 · Jobs that bill exactly once A post returns a job id immediately and a crashed worker never pays for the same call twice. ### Steps 1. Claim the row before the call, not after The window between call and write is where double billing lives. Files: `src/worker/claim.ts`, `tests/claim.test.ts` 2. Make the caller's idempotency key unique in the schema A database constraint is the only deduplication that survives a crash. Files: `src/db/migrations/002_idempotency.sql` 3. Classify failures into four kinds and retry only two Retrying a refusal burns money to get the same answer. Files: `src/run/failures.ts`, `tests/failures.test.ts` ### Done when - [ ] A repeated idempotency key yields one job and one call - [ ] A killed worker does not reissue a call it may have paid for - [ ] Rate limit backoff uses the provider's own retry hint - [ ] Refusals are not retried ### Watch out - Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. ## M3 · A golden set per task Every task has fixed inputs with expected outputs, and one command says what changed. ### Steps 1. Put your worst real inputs in the golden set Clean inputs never catch a regression. Files: `tasks/summarise/golden/`, `src/golden/run.ts` 2. Diff field by field, not blob against blob You need to know which field moved, not that something did. Files: `src/golden/diff.ts`, `tests/diff.test.ts` 3. Promote a bad live result into golden in one command The bug you just found is the test you did not have. Files: `src/cli/promote.ts` ### Done when - [ ] The run command fails loudly on a broken case - [ ] Diffs are per field and readable - [ ] Runs are stored and comparable - [ ] Promoting a live input takes one command ### Watch out - A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. ## M4 · Cost you can see and a door you can shut Every job carries what it cost, the month has a ceiling, and only your own services can call in. ### Steps 1. Take usage from the provider, not from a token estimate Your estimate and their meter will disagree, and theirs is the one that bills. Files: `src/run/cost.ts` 2. Stop queueing at the ceiling A warning in a log is not a limit. Files: `src/worker/ceiling.ts`, `tests/ceiling.test.ts` 3. Gate every route including status The status route leaks results, so it is not the safe one. Files: `src/http/auth.ts` ### Done when - [ ] Costs come from reported usage and are never null - [ ] The monthly ceiling refuses new jobs - [ ] Every route requires the token - [ ] Non-loopback binding needs an explicit opt-in ### Watch out - A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning. ## M5 · Living with a retirement date (production only) The pinned model's end date is known in advance, and migrating to its replacement is a measured diff rather than a leap. ### Steps 1. Pin dated model identifiers and test that you did An alias is a change you did not schedule. Files: `tasks/*/task.json`, `tests/no-alias.test.ts` 2. Keep the retirement date in the database and warn early Providers publish these. Nobody reads them on the day. Files: `src/tasks/retirement.ts` 3. Migrate by golden diff, then move the pin The diff is the evidence that the contract survived. Files: `src/cli/migrate.ts` ### Done when - [ ] No task uses a moving model alias - [ ] A near retirement date warns at startup - [ ] Migration shows a diff before anything changes - [ ] Export and reimport round-trips every table ### Watch out - Pointing every task at a convenience alias that always means the newest model turns a provider release into an unannounced change in your output shape, at an hour you did not choose, with no diff to read. Pin dated identifiers, keep the retirement dates in the database, and let the golden set decide when the pin moves. ===== OPERATIONS.md ===== # Operations · SharpAPI ## Backup The SQLite file and the tasks directory, and they are different kinds of thing: the directory is source you should be keeping in version control anyway, the database is the history of what was asked and what came back. ## Restore Restore the database, restore or check out the tasks directory, then run the golden set. If the golden run passes on the restored pins, the restore is good. If it does not, the pins and the prompts have drifted apart and that is exactly what the hashes are for. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Three numbers: the share of jobs ending in state invalid, the monthly spend against the ceiling, and days remaining on the nearest model retirement. A rising invalid rate usually means the model changed under a pin you thought was fixed, or an input source started sending something new. ## Incident checklist The bad day here is not an outage, it is a task that quietly started returning a slightly different shape and nobody noticed for a fortnight. The stored raw responses, prompt hashes and model ids are what let you find the day it changed. Run the golden set first, then diff a stored result from before and after. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] A non-validating response is confirmed to store no result - [ ] A repeated idempotency key is confirmed to produce one call - [ ] The monthly ceiling is confirmed to refuse new jobs - [ ] No task is confirmed to reference a moving model alias - [ ] Export and reimport are confirmed to round-trip every table ## Launch constraint Do not market omitted SharpAPI capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. The model provider. Give it a provider-side spend limit too. MODEL_API_KEY=sk-...redacted # Required. One directory per task. This is the whole catalogue. TASKS_DIR=./tasks # Required. Jobs, results, failures and golden runs. Back this up. DATABASE_PATH=./data/tasks.db # Required · secret. Required on every route including job status. Rotate it by changing this. API_TOKEN=a long random string # Required. Loopback by default. Changing it is a deliberate act with consequences. BIND_ADDRESS=127.0.0.1 # Required. New jobs are refused above this. In-flight jobs still finish. MONTHLY_COST_CEILING=20.00 # Required. Startup warns this far before a pinned model's published retirement date. RETIREMENT_WARN_DAYS=45
README.md
# SharpAPI · indie build A local task API whose hard problems are keeping a response contract stable while the model underneath it is retired on a published schedule, never letting a failed extraction become a stored result, and charging a job to your card exactly once. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 and Fastify bound to the loopback address, no browser interface | this is something your other services call, not something a person opens | | Storage | SQLite for jobs and results, with prompts and schemas as files on disk | a task definition you can diff and revert beats one buried in a table | | Task definition | One directory per task holding prompt.md, schema.json, a dated model id and a golden set | adding a task is adding a folder, and that is the only way this grows without turning into a framework | | Execution | Asynchronous by default: a post returns a job id and one worker drains the queue | a model call that takes forty seconds must not hold an HTTP connection open | | Validation | Every response checked against that task's schema before anything is stored | an unvalidated response is a guess wearing a content type | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the API and the worker. - Get it: nodejs.org, or your version manager. - Verify: node --version prints v22 or higher - [ ] **An API key for a model provider with structured output** · pay per call; a few hundred jobs a month is small, and the ceiling is what keeps it that way - Why: Every task is one call, and this is the only part of the build that costs money. - Get it: Create a key and set a spend limit in the provider's console as well as the ceiling in this app. - Verify: One task returns schema-valid JSON - [ ] **Ten real inputs per task, including the ugly ones** · free - Why: The golden set is the only thing standing between a model change and a silent change in your output, and clean inputs never catch anything. - Get it: Pull them from whatever system you were going to point this at. - Verify: Each task directory has a golden set with at least one input you would call difficult - [ ] **Which dated model each task pins, and its retirement date** · free - Why: Provider retirement is scheduled and announced, and a task pinned to an alias changes shape on somebody else's release day. - Get it: Read the provider's deprecation page, write the exact dated identifier and the published retirement date into the task file. - Verify: Every task file has a dated identifier and a date, and none uses an alias ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Packaged SDKs in other languages. - Billing tiers and per-caller quotas. - The other tasks in somebody else's catalogue. - Fine-tuning or self-hosted models. - tuned prompts per use case, tested against messy real-world inputs - response contracts that stay stable when the underlying models change - the managed async queue with polling, throttling, and rate limits - ready SDKs for PHP/Laravel, Node, and Python - 80+ language coverage tested per endpoint If one of those is essential to you, that is the reason to keep paying for SharpAPI, and the README should say so rather than pretend.
BRIEF.md
# Build brief · SharpAPI The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a small self-hosted task API that turns one prompt file and one JSON schema into an endpoint your other projects can call to replace SharpAPI. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Runtime: Node 22 and Fastify bound to the loopback address, no browser interface · this is something your other services call, not something a person opens - Storage: SQLite for jobs and results, with prompts and schemas as files on disk · a task definition you can diff and revert beats one buried in a table - Task definition: One directory per task holding prompt.md, schema.json, a dated model id and a golden set · adding a task is adding a folder, and that is the only way this grows without turning into a framework - Execution: Asynchronous by default: a post returns a job id and one worker drains the queue · a model call that takes forty seconds must not hold an HTTP connection open - Validation: Every response checked against that task's schema before anything is stored · an unvalidated response is a guess wearing a content type ### Data model (create this before Phase 1) - tasks(name, dir, model_id, prompt_sha, schema_sha, created_at, retire_after) · retire_after is the date the pinned model stops working, copied by hand from the provider's own deprecation page - jobs(id, task, input_ref, idempotency_key, state, attempts, claimed_by, created_at, started_at, finished_at) · state is queued, running, succeeded, failed or invalid - results(job_id, raw_response, parsed_json, schema_ok, input_tokens, output_tokens, cost_minor, model_id, prompt_sha) - failures(job_id, attempt, kind, detail, at) · kind is transport, rate_limit, refusal, schema or timeout, because those are four different problems and one of them is not retryable - golden(task, case_id, input_ref, expected_json, notes) · the fixed inputs a task is re-checked against whenever anything moves - runs(id, task, model_id, at, n_cases, n_pass, n_changed, diff_ref) · one golden run, so swapping a model is a measurement rather than a hope - Every stored result carries the model id and the prompt hash that produced it. A task whose output changed cannot be debugged without knowing which of those two moved. ### Phase 1 · One task, one folder, one contract Build: Load every directory under tasks/ into the tasks table, hashing the prompt and the schema. Generate one route per task. Call the model with structured output against that task's schema, validate the parsed response against the same schema file, and store the raw response either way. Trap: Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. Done when: Adding a directory with prompt.md and schema.json produces a working endpoint with no code change, tested by adding one. A response that does not validate is stored as state invalid with the raw text kept, and the endpoint returns an error rather than a partial object, tested with a stubbed provider returning prose. A model refusal is recorded as kind refusal and is never retried. The prompt hash and schema hash are recorded on the task row at load time. Do not build yet: No queue, no retries, no cost accounting and no golden set yet. ### Phase 2 · Jobs that bill exactly once Build: A jobs table drained by one worker. Claim a row with a conditional update that sets state and claimed_by before the model call, not after. Accept an idempotency key from the caller and make it unique, so a retried post returns the existing job instead of creating a second one. Trap: Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. Done when: Two posts with the same idempotency key produce one job and one model call, tested. Killing the worker mid-call leaves a row in running with its request id, and restarting does not issue a second call for it, tested by killing the process. Backoff on a rate limit honours the provider's retry header when one is present rather than a fixed sleep. A transport failure retries and a refusal does not, asserted in the suite. Do not build yet: No golden set, no cost ceiling and no callers other than curl yet. ### Phase 3 · A golden set per task Build: For each task keep a handful of real inputs under golden/ with the output you accept. A run command executes them against the current pinned model, compares field by field, and reports pass, changed and failed with a readable diff. Store the run so two runs can be compared. Trap: A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. Done when: Each task has at least one golden case and the run command exits non-zero when any case fails, tested by breaking a schema. The diff shows which fields changed rather than printing two blobs. A run is stored with its model id, case count and pass count, so two runs are comparable. Promoting a live job's input into the golden set takes one command. Do not build yet: No cost ceiling, no access control and no migration tooling yet. ### Phase 4 · Cost you can see and a door you can shut Build: Record input and output tokens and a computed cost in minor units on every result, from the provider's usage figures rather than an estimate. Sum per task per month. A shared token in a header gates every route, and the process binds to the loopback address by default. Trap: A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning. Done when: Every result row has a non-null cost in minor units taken from the provider's reported usage, checked against one real call. Reaching the monthly ceiling refuses new jobs with a clear error and leaves in-flight ones to finish, tested at the boundary. A request without the token is rejected on every route including the job status route. The process refuses to bind to a non-loopback address unless a variable is explicitly set. Do not build yet: No per-caller quotas, no SDKs and no admin interface. ### Phase 5 · Living with a retirement date Build: Pin an exact dated model identifier per task, never a moving alias. Store the retirement date from the provider's deprecation page on the task row and warn on startup when it is close. A migrate command runs the golden set against a candidate model and shows the per-field diff before you change the pin. Export every job, result and run as JSONL. Trap: Pointing every task at a convenience alias that always means the newest model turns a provider release into an unannounced change in your output shape, at an hour you did not choose, with no diff to read. Pin dated identifiers, keep the retirement dates in the database, and let the golden set decide when the pin moves. Done when: No task references a moving alias, asserted by a test that scans the task files. Each task row carries a retirement date and startup prints a warning inside a configured window before it. The migrate command runs the golden set against a candidate and prints a per-field diff without changing any pin, tested. Export produces JSONL for jobs, results and runs, and reimports into an empty database with identical counts. Do not build yet: Nothing after this. ### Out of scope (and why) - Packaged client SDKs for other languages, since your callers can send a POST and the maintenance of three SDKs is the actual product you are not buying. - Billing tiers, credit accounting and per-caller rate limits, which only matter when the callers are strangers. - The rest of the catalogue. Build the two or three tasks you use and let the folder convention make the fourth cheap. - Fine-tuning or hosting your own model, which changes this from a weekend of glue into an infrastructure project. ### README must contain - That a task is a directory, and exactly which files it must contain - That the pinned model identifier is dated, where its retirement date came from, and when it was last checked - That every result records the model id and prompt hash, and why you cannot debug an output without both - What one call of each task costs, measured rather than estimated, and where the monthly ceiling is set - That the service binds to loopback and requires a token, and what to change if you move it
AGENTS.md
# Agent instructions · SharpAPI indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 and Fastify bound to the loopback address, no browser interface, SQLite for jobs and results, with prompts and schemas as files on disk, One directory per task holding prompt.md, schema.json, a dated model id and a golden set, Asynchronous by default: a post returns a job id and one worker drains the queue, Every response checked against that task's schema before anything is stored. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. - Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. - A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. - A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning.
BUILD_PLAN.md
# Build plan · SharpAPI A local task API whose hard problems are keeping a response contract stable while the model underneath it is retired on a published schedule, never letting a failed extraction become a stored result, and charging a job to your card exactly once. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · One task, one folder, one contract A task directory is loaded at startup and its endpoint either returns schema-valid JSON or an error, never prose. ### Steps 1. Load tasks from the filesystem, hash both files The hash is what lets you tell a prompt change from a model change later. Files: `src/tasks/load.ts`, `tasks/summarise/prompt.md`, `tasks/summarise/schema.json` 2. Validate every response against the task's own schema The schema is the contract callers depend on, so it is the gate. Files: `src/run/validate.ts`, `tests/validate.test.ts` 3. Keep the raw response even when it is rubbish You cannot diagnose a bad extraction you threw away. Files: `src/store/results.ts` ### Done when - [ ] A new task directory becomes an endpoint with no code change - [ ] A non-validating response is stored as invalid, not as a result - [ ] A refusal is classified as a refusal and not retried - [ ] Prompt and schema hashes are recorded per task ### Watch out - Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. ## Phase 2 · Jobs that bill exactly once A post returns a job id immediately and a crashed worker never pays for the same call twice. ### Steps 1. Claim the row before the call, not after The window between call and write is where double billing lives. Files: `src/worker/claim.ts`, `tests/claim.test.ts` 2. Make the caller's idempotency key unique in the schema A database constraint is the only deduplication that survives a crash. Files: `src/db/migrations/002_idempotency.sql` 3. Classify failures into four kinds and retry only two Retrying a refusal burns money to get the same answer. Files: `src/run/failures.ts`, `tests/failures.test.ts` ### Done when - [ ] A repeated idempotency key yields one job and one call - [ ] A killed worker does not reissue a call it may have paid for - [ ] Rate limit backoff uses the provider's own retry hint - [ ] Refusals are not retried ### Watch out - Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. ## Phase 3 · A golden set per task Every task has fixed inputs with expected outputs, and one command says what changed. ### Steps 1. Put your worst real inputs in the golden set Clean inputs never catch a regression. Files: `tasks/summarise/golden/`, `src/golden/run.ts` 2. Diff field by field, not blob against blob You need to know which field moved, not that something did. Files: `src/golden/diff.ts`, `tests/diff.test.ts` 3. Promote a bad live result into golden in one command The bug you just found is the test you did not have. Files: `src/cli/promote.ts` ### Done when - [ ] The run command fails loudly on a broken case - [ ] Diffs are per field and readable - [ ] Runs are stored and comparable - [ ] Promoting a live input takes one command ### Watch out - A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. ## Phase 4 · Cost you can see and a door you can shut Every job carries what it cost, the month has a ceiling, and only your own services can call in. ### Steps 1. Take usage from the provider, not from a token estimate Your estimate and their meter will disagree, and theirs is the one that bills. Files: `src/run/cost.ts` 2. Stop queueing at the ceiling A warning in a log is not a limit. Files: `src/worker/ceiling.ts`, `tests/ceiling.test.ts` 3. Gate every route including status The status route leaks results, so it is not the safe one. Files: `src/http/auth.ts` ### Done when - [ ] Costs come from reported usage and are never null - [ ] The monthly ceiling refuses new jobs - [ ] Every route requires the token - [ ] Non-loopback binding needs an explicit opt-in ### Watch out - A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning. ## Not in this build - Packaged SDKs in other languages. - Billing tiers and per-caller quotas. - The other tasks in somebody else's catalogue. - Fine-tuning or self-hosted models. ## After v1, if you want it - A second provider behind the same task folder, chosen per task, once the golden set can tell you whether it is worse - Batch submission for tasks where latency does not matter, which is usually much cheaper per call - A small dashboard over the failures table, because the kinds of failure are more interesting than the count
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. The model provider. Give it a provider-side spend limit too. MODEL_API_KEY=sk-...redacted # Required. One directory per task. This is the whole catalogue. TASKS_DIR=./tasks # Required. Jobs, results, failures and golden runs. Back this up. DATABASE_PATH=./data/tasks.db # Required · secret. Required on every route including job status. Rotate it by changing this. API_TOKEN=a long random string # Required. Loopback by default. Changing it is a deliberate act with consequences. BIND_ADDRESS=127.0.0.1 # Required. New jobs are refused above this. In-flight jobs still finish. MONTHLY_COST_CEILING=20.00 # Required. Startup warns this far before a pinned model's published retirement date. RETIREMENT_WARN_DAYS=45
PRODUCT.md
# SharpAPI · product brief ## Problem Any single endpoint here, summarize a text, categorize a product, parse a resume, is one LLM call with a decent prompt and a JSON schema, so the piece you actually need is often an evening of work. What you will not rebuild in a weekend is the catalog: 30+ endpoints with tuned prompts, response contracts that survive model upgrades, an async job queue with polling, and maintained SDKs. Rebuild the two or three endpoints you use, pay when you need the shelf. ## Product outcome A task API where adding a capability is adding a folder, where a bad model response is an error rather than a row, where a crash never bills twice, and where a model retirement is something you migrate through with a diff in your hand. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - OpenAI or Anthropic API key - a JSON schema per task for structured outputs - a job queue (BullMQ + Redis, or a SQLite job table) - real sample inputs to calibrate each prompt against ## Explicit non-goals for v1 - Packaged SDKs in other languages. - Billing tiers and per-caller quotas. - The other tasks in somebody else's catalogue. - Fine-tuning or self-hosted models. - tuned prompts per use case, tested against messy real-world inputs - response contracts that stay stable when the underlying models change - the managed async queue with polling, throttling, and rate limits - ready SDKs for PHP/Laravel, Node, and Python - 80+ language coverage tested per endpoint ## Success criteria - A non-validating response is confirmed to store no result - A repeated idempotency key is confirmed to produce one call - The monthly ceiling is confirmed to refuse new jobs - No task is confirmed to reference a moving model alias - Export and reimport are confirmed to round-trip every table
BRIEF.md
# Build brief · SharpAPI The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a small self-hosted task API that turns one prompt file and one JSON schema into an endpoint your other projects can call to replace SharpAPI. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Runtime: Node 22 and Fastify bound to the loopback address, no browser interface · this is something your other services call, not something a person opens - Storage: SQLite for jobs and results, with prompts and schemas as files on disk · a task definition you can diff and revert beats one buried in a table - Task definition: One directory per task holding prompt.md, schema.json, a dated model id and a golden set · adding a task is adding a folder, and that is the only way this grows without turning into a framework - Execution: Asynchronous by default: a post returns a job id and one worker drains the queue · a model call that takes forty seconds must not hold an HTTP connection open - Validation: Every response checked against that task's schema before anything is stored · an unvalidated response is a guess wearing a content type ### Data model (create this before Phase 1) - tasks(name, dir, model_id, prompt_sha, schema_sha, created_at, retire_after) · retire_after is the date the pinned model stops working, copied by hand from the provider's own deprecation page - jobs(id, task, input_ref, idempotency_key, state, attempts, claimed_by, created_at, started_at, finished_at) · state is queued, running, succeeded, failed or invalid - results(job_id, raw_response, parsed_json, schema_ok, input_tokens, output_tokens, cost_minor, model_id, prompt_sha) - failures(job_id, attempt, kind, detail, at) · kind is transport, rate_limit, refusal, schema or timeout, because those are four different problems and one of them is not retryable - golden(task, case_id, input_ref, expected_json, notes) · the fixed inputs a task is re-checked against whenever anything moves - runs(id, task, model_id, at, n_cases, n_pass, n_changed, diff_ref) · one golden run, so swapping a model is a measurement rather than a hope - Every stored result carries the model id and the prompt hash that produced it. A task whose output changed cannot be debugged without knowing which of those two moved. ### Phase 1 · One task, one folder, one contract Build: Load every directory under tasks/ into the tasks table, hashing the prompt and the schema. Generate one route per task. Call the model with structured output against that task's schema, validate the parsed response against the same schema file, and store the raw response either way. Trap: Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. Done when: Adding a directory with prompt.md and schema.json produces a working endpoint with no code change, tested by adding one. A response that does not validate is stored as state invalid with the raw text kept, and the endpoint returns an error rather than a partial object, tested with a stubbed provider returning prose. A model refusal is recorded as kind refusal and is never retried. The prompt hash and schema hash are recorded on the task row at load time. Do not build yet: No queue, no retries, no cost accounting and no golden set yet. ### Phase 2 · Jobs that bill exactly once Build: A jobs table drained by one worker. Claim a row with a conditional update that sets state and claimed_by before the model call, not after. Accept an idempotency key from the caller and make it unique, so a retried post returns the existing job instead of creating a second one. Trap: Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. Done when: Two posts with the same idempotency key produce one job and one model call, tested. Killing the worker mid-call leaves a row in running with its request id, and restarting does not issue a second call for it, tested by killing the process. Backoff on a rate limit honours the provider's retry header when one is present rather than a fixed sleep. A transport failure retries and a refusal does not, asserted in the suite. Do not build yet: No golden set, no cost ceiling and no callers other than curl yet. ### Phase 3 · A golden set per task Build: For each task keep a handful of real inputs under golden/ with the output you accept. A run command executes them against the current pinned model, compares field by field, and reports pass, changed and failed with a readable diff. Store the run so two runs can be compared. Trap: A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. Done when: Each task has at least one golden case and the run command exits non-zero when any case fails, tested by breaking a schema. The diff shows which fields changed rather than printing two blobs. A run is stored with its model id, case count and pass count, so two runs are comparable. Promoting a live job's input into the golden set takes one command. Do not build yet: No cost ceiling, no access control and no migration tooling yet. ### Phase 4 · Cost you can see and a door you can shut Build: Record input and output tokens and a computed cost in minor units on every result, from the provider's usage figures rather than an estimate. Sum per task per month. A shared token in a header gates every route, and the process binds to the loopback address by default. Trap: A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning. Done when: Every result row has a non-null cost in minor units taken from the provider's reported usage, checked against one real call. Reaching the monthly ceiling refuses new jobs with a clear error and leaves in-flight ones to finish, tested at the boundary. A request without the token is rejected on every route including the job status route. The process refuses to bind to a non-loopback address unless a variable is explicitly set. Do not build yet: No per-caller quotas, no SDKs and no admin interface. ### Phase 5 · Living with a retirement date Build: Pin an exact dated model identifier per task, never a moving alias. Store the retirement date from the provider's deprecation page on the task row and warn on startup when it is close. A migrate command runs the golden set against a candidate model and shows the per-field diff before you change the pin. Export every job, result and run as JSONL. Trap: Pointing every task at a convenience alias that always means the newest model turns a provider release into an unannounced change in your output shape, at an hour you did not choose, with no diff to read. Pin dated identifiers, keep the retirement dates in the database, and let the golden set decide when the pin moves. Done when: No task references a moving alias, asserted by a test that scans the task files. Each task row carries a retirement date and startup prints a warning inside a configured window before it. The migrate command runs the golden set against a candidate and prints a per-field diff without changing any pin, tested. Export produces JSONL for jobs, results and runs, and reimports into an empty database with identical counts. Do not build yet: Nothing after this. ### Out of scope (and why) - Packaged client SDKs for other languages, since your callers can send a POST and the maintenance of three SDKs is the actual product you are not buying. - Billing tiers, credit accounting and per-caller rate limits, which only matter when the callers are strangers. - The rest of the catalogue. Build the two or three tasks you use and let the folder convention make the fourth cheap. - Fine-tuning or hosting your own model, which changes this from a weekend of glue into an infrastructure project. ### README must contain - That a task is a directory, and exactly which files it must contain - That the pinned model identifier is dated, where its retirement date came from, and when it was last checked - That every result records the model id and prompt hash, and why you cannot debug an output without both - What one call of each task costs, measured rather than estimated, and where the monthly ceiling is set - That the service binds to loopback and requires a token, and what to change if you move it
ARCHITECTURE.md
# Architecture · SharpAPI ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 and Fastify bound to the loopback address, no browser interface | this is something your other services call, not something a person opens | | Storage | SQLite for jobs and results, with prompts and schemas as files on disk | a task definition you can diff and revert beats one buried in a table | | Task definition | One directory per task holding prompt.md, schema.json, a dated model id and a golden set | adding a task is adding a folder, and that is the only way this grows without turning into a framework | | Execution | Asynchronous by default: a post returns a job id and one worker drains the queue | a model call that takes forty seconds must not hold an HTTP connection open | | Validation | Every response checked against that task's schema before anything is stored | an unvalidated response is a guess wearing a content type | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Task loader | Directories, prompts, schemas and pins. | Any layout, provided the prompt and schema stay as files you can diff. | | Runner | The model call, validation and failure classification. | Any provider with structured output, provided validation stays the gate. | | Queue | Claiming, idempotency and retries. | Any queue, provided the claim happens before the call. | | Golden | Fixed cases, runs and diffs. | Not swappable for a smoke test, which is what it becomes if you fill it with easy inputs. | | Cost | Reported usage, per-task totals and the ceiling. | Any accounting, provided the figures come from the provider. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `MODEL_API_KEY` · required, secret · The model provider. Give it a provider-side spend limit too. - `TASKS_DIR` · required · One directory per task. This is the whole catalogue. - `DATABASE_PATH` · required · Jobs, results, failures and golden runs. Back this up. - `API_TOKEN` · required, secret · Required on every route including job status. Rotate it by changing this. - `BIND_ADDRESS` · required · Loopback by default. Changing it is a deliberate act with consequences. - `MONTHLY_COST_CEILING` · required · New jobs are refused above this. In-flight jobs still finish. - `RETIREMENT_WARN_DAYS` · required · Startup warns this far before a pinned model's published retirement date. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · SharpAPI product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Node 22 and Fastify bound to the loopback address, no browser interface, SQLite for jobs and results, with prompts and schemas as files on disk, One directory per task holding prompt.md, schema.json, a dated model id and a golden set, Asynchronous by default: a post returns a job id and one worker drains the queue, Every response checked against that task's schema before anything is stored. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. - Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. - A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. - A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning. - Pointing every task at a convenience alias that always means the newest model turns a provider release into an unannounced change in your output shape, at an hour you did not choose, with no diff to read. Pin dated identifiers, keep the retirement dates in the database, and let the golden set decide when the pin moves.
MILESTONES.md
# Delivery milestones · SharpAPI Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · One task, one folder, one contract A task directory is loaded at startup and its endpoint either returns schema-valid JSON or an error, never prose. ### Steps 1. Load tasks from the filesystem, hash both files The hash is what lets you tell a prompt change from a model change later. Files: `src/tasks/load.ts`, `tasks/summarise/prompt.md`, `tasks/summarise/schema.json` 2. Validate every response against the task's own schema The schema is the contract callers depend on, so it is the gate. Files: `src/run/validate.ts`, `tests/validate.test.ts` 3. Keep the raw response even when it is rubbish You cannot diagnose a bad extraction you threw away. Files: `src/store/results.ts` ### Done when - [ ] A new task directory becomes an endpoint with no code change - [ ] A non-validating response is stored as invalid, not as a result - [ ] A refusal is classified as a refusal and not retried - [ ] Prompt and schema hashes are recorded per task ### Watch out - Asking a model for JSON in the prompt and parsing whatever comes back means that on a bad day you store the string 'Sure, here is the JSON:' plus a code fence, and the caller three services downstream sees a null field instead of an error. Validate against the schema file, treat a validation miss as its own job state called invalid, and never let a parse failure produce a row that looks like a result. ## M2 · Jobs that bill exactly once A post returns a job id immediately and a crashed worker never pays for the same call twice. ### Steps 1. Claim the row before the call, not after The window between call and write is where double billing lives. Files: `src/worker/claim.ts`, `tests/claim.test.ts` 2. Make the caller's idempotency key unique in the schema A database constraint is the only deduplication that survives a crash. Files: `src/db/migrations/002_idempotency.sql` 3. Classify failures into four kinds and retry only two Retrying a refusal burns money to get the same answer. Files: `src/run/failures.ts`, `tests/failures.test.ts` ### Done when - [ ] A repeated idempotency key yields one job and one call - [ ] A killed worker does not reissue a call it may have paid for - [ ] Rate limit backoff uses the provider's own retry hint - [ ] Refusals are not retried ### Watch out - Retrying a failed job by running it from the top means a worker killed after the model responded but before the write pays for the same extraction twice, and you only discover it on the invoice. Claim the row first, record the provider's request identifier as soon as you have it, and on restart treat a row stuck in running with a request id as finished-unknown rather than as work to redo. ## M3 · A golden set per task Every task has fixed inputs with expected outputs, and one command says what changed. ### Steps 1. Put your worst real inputs in the golden set Clean inputs never catch a regression. Files: `tasks/summarise/golden/`, `src/golden/run.ts` 2. Diff field by field, not blob against blob You need to know which field moved, not that something did. Files: `src/golden/diff.ts`, `tests/diff.test.ts` 3. Promote a bad live result into golden in one command The bug you just found is the test you did not have. Files: `src/cli/promote.ts` ### Done when - [ ] The run command fails loudly on a broken case - [ ] Diffs are per field and readable - [ ] Runs are stored and comparable - [ ] Promoting a live input takes one command ### Watch out - A golden set of three clean inputs passes forever and tells you nothing, because the cases that break when a model changes are the messy ones: the truncated CV, the product description in two languages, the review that is one emoji. Put your worst real inputs in the golden set, and when a live call produces something wrong, promote that input into golden/ the same day. ## M4 · Cost you can see and a door you can shut Every job carries what it cost, the month has a ceiling, and only your own services can call in. ### Steps 1. Take usage from the provider, not from a token estimate Your estimate and their meter will disagree, and theirs is the one that bills. Files: `src/run/cost.ts` 2. Stop queueing at the ceiling A warning in a log is not a limit. Files: `src/worker/ceiling.ts`, `tests/ceiling.test.ts` 3. Gate every route including status The status route leaks results, so it is not the safe one. Files: `src/http/auth.ts` ### Done when - [ ] Costs come from reported usage and are never null - [ ] The monthly ceiling refuses new jobs - [ ] Every route requires the token - [ ] Non-loopback binding needs an explicit opt-in ### Watch out - A task API with no ceiling and no authentication on a machine you later put on a network is a way for someone else to spend your model budget, and the first sign is the bill. Bind to loopback, require a token, and stop queueing new jobs when the month's ceiling is reached rather than merely logging a warning. ## M5 · Living with a retirement date (production only) The pinned model's end date is known in advance, and migrating to its replacement is a measured diff rather than a leap. ### Steps 1. Pin dated model identifiers and test that you did An alias is a change you did not schedule. Files: `tasks/*/task.json`, `tests/no-alias.test.ts` 2. Keep the retirement date in the database and warn early Providers publish these. Nobody reads them on the day. Files: `src/tasks/retirement.ts` 3. Migrate by golden diff, then move the pin The diff is the evidence that the contract survived. Files: `src/cli/migrate.ts` ### Done when - [ ] No task uses a moving model alias - [ ] A near retirement date warns at startup - [ ] Migration shows a diff before anything changes - [ ] Export and reimport round-trips every table ### Watch out - Pointing every task at a convenience alias that always means the newest model turns a provider release into an unannounced change in your output shape, at an hour you did not choose, with no diff to read. Pin dated identifiers, keep the retirement dates in the database, and let the golden set decide when the pin moves.
OPERATIONS.md
# Operations · SharpAPI ## Backup The SQLite file and the tasks directory, and they are different kinds of thing: the directory is source you should be keeping in version control anyway, the database is the history of what was asked and what came back. ## Restore Restore the database, restore or check out the tasks directory, then run the golden set. If the golden run passes on the restored pins, the restore is good. If it does not, the pins and the prompts have drifted apart and that is exactly what the hashes are for. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Three numbers: the share of jobs ending in state invalid, the monthly spend against the ceiling, and days remaining on the nearest model retirement. A rising invalid rate usually means the model changed under a pin you thought was fixed, or an input source started sending something new. ## Incident checklist The bad day here is not an outage, it is a task that quietly started returning a slightly different shape and nobody noticed for a fortnight. The stored raw responses, prompt hashes and model ids are what let you find the day it changed. Run the golden set first, then diff a stored result from before and after. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] A non-validating response is confirmed to store no result - [ ] A repeated idempotency key is confirmed to produce one call - [ ] The monthly ceiling is confirmed to refuse new jobs - [ ] No task is confirmed to reference a moving model alias - [ ] Export and reimport are confirmed to round-trip every table ## Launch constraint Do not market omitted SharpAPI capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. The model provider. Give it a provider-side spend limit too. MODEL_API_KEY=sk-...redacted # Required. One directory per task. This is the whole catalogue. TASKS_DIR=./tasks # Required. Jobs, results, failures and golden runs. Back this up. DATABASE_PATH=./data/tasks.db # Required · secret. Required on every route including job status. Rotate it by changing this. API_TOKEN=a long random string # Required. Loopback by default. Changing it is a deliberate act with consequences. BIND_ADDRESS=127.0.0.1 # Required. New jobs are refused above this. In-flight jobs still finish. MONTHLY_COST_CEILING=20.00 # Required. Startup warns this far before a pinned model's published retirement date. RETIREMENT_WARN_DAYS=45
$ choose a build depth, inspect the files, then open the complete pack in your agent
Because they need five of these workflows, not one, and someone else keeps the prompts, schemas, and model choices working while they ship their actual product. A team that only needs one endpoint is exactly who should DIY it.
xtuned prompts per use case, tested against messy real-world inputs
xresponse contracts that stay stable when the underlying models change
xthe managed async queue with polling, throttling, and rate limits
xready SDKs for PHP/Laravel, Node, and Python
x80+ language coverage tested per endpoint
share on X ↗"I just replaced SharpAPI ($50/mo) with one prompt"
SharpAPI pricing
build$50/mo · monthly, credit-based · $600/yr
free tier14-day trial with 100,000 processed words, no credit card.
verified 2026-08-10 · source ↗
Is SharpAPI free?
14-day trial with 100,000 processed words, no credit card. Paid is Build at $50/mo (checked 2026-08-10).
Vibecode SharpAPI
Kinda. The core of SharpAPI is buildable in a weekend with the prompt on this page, but there are real gaps: tuned prompts per use case, tested against messy real-world inputs, response contracts that stay stable when the underlying models change. Read the honest list above before committing.
How much does SharpAPI cost?
SharpAPI costs about $50/month (Build, checked 2026-08-10), which is $600 per year.
What do I lose by replacing SharpAPI?
Honestly: tuned prompts per use case, tested against messy real-world inputs; response contracts that stay stable when the underlying models change; the managed async queue with polling, throttling, and rate limits; ready SDKs for PHP/Laravel, Node, and Python; 80+ language coverage tested per endpoint. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to SharpAPI?
Yes: OpenAI Node SDK (official client with structured outputs, the engine behind most DIY task endpoints), Instructor (schema-validated structured extraction from LLMs, with retries on invalid output), BullMQ (Redis-backed job queue if the SQLite job table stops being enough). Using prior art is also vibecoding; the prompt is for when you want it exactly your way.