Build Littlebird

KINDAreplaces $20/mosaves $240/yrback to the verdict

0%0 of 17 items done

Saved on this device only. Tick prerequisites first, then work the phases in order · do not start one until the checks above it pass.

A personal assistant whose hard problems are answering only from retrieved context and saying so when it cannot, keeping a scheduled watcher from spending money on pages that did not change, and noticing when the job that makes it useful has quietly stopped running.

estimated effort weekendthe files for this build are in the project pack

RuntimeNode 20 with TypeScript and Fastify, one processStorageSQLite via better-sqlite3, including the vectorsInterfaceServer-rendered HTML with htmx, plain CSSModel accessOne provider behind one module, key and base URL from the environmentFetchingConditional requests with stored validators, content extraction, hashed snapshots

Before step 1

Everything below is assumed from the first step. Tick each one when you actually have it, not when you plan to.

installfree
Why Runs the server and the scheduler in one process.
Get it nodejs.org, or nvm install 20. open ↗
Verify node --version prints v20 or higher
API keypay per use, typically small for a personal library
Why Chat, digests and embeddings all go through it, and the cost ledger exists to keep it from surprising you.
Get it Create an account with a provider, generate a key, and note the chat and embedding model names and their current prices.
Verify A ten chunk embedding run and a one sentence chat completion both succeed
have readya few pounds a month
Why A scheduled watcher on a laptop that sleeps is a digest page that silently stops updating, which is the failure this build is organised around.
Get it A small VPS, a home server, or any always-on machine.
Verify A scheduled pass runs overnight and writes a run row
have readyfree
Why One that supports conditional requests, one that does not, and one that changes cosmetically every load. Each exists to make a different gate in phase three fail before it passes.
Get it Pick from sites you already read, and check their response headers by hand first.
Verify You know which validators each returns and whether its extracted text is stable
decidefree
Why The two numbers that decide whether this stays cheap, and choosing them after a bill means choosing whatever already happened.
Get it Pick a monthly figure you would not mind, and a character cap per digest call, and write down why.
Verify Both are in configuration and named in tests

Data model

Create these before the first phase that stores anything. Changing a table later is the expensive kind of change.

- sources(id, kind, title, url, added_at, bytes, sha256) · kind is paste, upload or url
- chunks(id, source_id, ordinal, text, token_estimate, embedding) · the unit that gets cited
- conversations(id, started_at, title) and turns(conversation_id, ordinal, role, content, cited_chunk_ids_json, at)
- watch_items(id, name, url, cadence, last_fetch_at, etag, last_modified, last_sha256, last_status, consecutive_failures, paused)
- snapshots(watch_item_id, at, extracted_sha256, extracted_path, bytes) · kept for a stated number of versions
- digests(watch_item_id, snapshot_at, summary, model_id, tokens_in, tokens_out, created_at) · only written when something changed
- calls(id, at, purpose, model_id, tokens_in, tokens_out, latency_ms, estimated_cost) · every model call, no exceptions
- runs(id, kind, started_at, finished_at, items_checked, items_changed, calls_made, error_text) · the heartbeat that proves the job is alive
- The watcher's cheapest path is the common one: a 304 response ends the check before extraction, hashing or any model call.

Environment variables

These go in a .env file the app reads at startup. The pack's .env.example is this table as a file · copy it, never commit the filled-in version.

Variable	Needed	Example	Where the value comes from
`DATABASE_PATH`	required	`./data/assistant.db`	Sources, chunks, conversations, watch items, digests and the call ledger.
`OUT_DIR`	required	`./out`	The digest Markdown written after each run, so it survives the process.
`MODEL_API_KEY`secret	required	`sk-...redacted`	Chat, digest and embedding calls. The only credential here.
`MODEL_BASE_URL`	optional	`https://api.example.com/v1`	Set it to point at a different provider or a local server.
`CHAT_MODEL`	required	`a chat model id from your provider`	Recorded on every call in the ledger, so a model change is visible in the costs.
`EMBED_MODEL`	required	`an embedding model id from your provider`	Changing it invalidates existing embeddings, which the app must say rather than silently mix.
`DIGEST_CHAR_CAP`	required	`20000`	Maximum characters sent per digest call. Exceeding it truncates with a visible note.
`MONTHLY_COST_CEILING_USD`	required	`10`	Reaching it pauses scheduled work and says so on every page.

The build, in order

1
Context with provenance
Anything the assistant can use came from a source you can point at.
1. The difference between a useful retrieval and a confident fragment.
  Files src/ingest/chunk.tstests/chunk-structure.test.ts
2. A stored login page poisons every later answer.
  Files src/ingest/extract.ts
3. A deleted source that still answers questions is not deleted.
  Files src/sources/delete.ts
done when · tick each as it passes
watch out
- Chunking by a fixed character count cuts sentences and tables in half, so retrieval returns fragments that read as nonsense and the answers built on them are subtly wrong with no visible cause. Chunk on structure first, paragraphs and headings, falling back to size only when a block is too large, and store enough overlap that a split sentence appears whole in one of the two chunks.
next phase
2
Answers that cite, or say they cannot
Every reply names the chunks it used, and a question with no relevant context gets an honest refusal.
1. Ids in, citations out, or the answer is unverifiable.
  Files src/chat/prompt.tssrc/chat/parse.ts
2. An honest 'I do not have that' beats a fluent guess.
  Files src/chat/retrieve.tstests/no-context-refusal.test.ts
3. The audit trail for the answer that was wrong.
  Files src/chat/turns.ts
done when · tick each as it passes
watch out
- Dropping retrieved text into the prompt without ids and rendering whatever comes back produces answers that mix your documents with the model's general knowledge, indistinguishably, so you cannot tell which sentences your own notes support. Pass chunk ids, require citations, and show an answer citing nothing as exactly that.
next phase
3
A watcher that pays only for change
A daily check on a page that did not change costs one conditional request and nothing else.
1. The cheapest possible check, and the polite one.
  Files src/watch/fetch.tstests/conditional-request.test.ts
2. Advertisements and timestamps change; the article does not.
  Files src/watch/extract.ts
3. A broken watcher should be visible, not persistent.
  Files src/watch/health.ts
done when · tick each as it passes
watch out
- Refetching every watched page daily and asking a model what changed spends real money on pages that did not change at all, and since page bodies are large, a handful of items becomes a monthly bill out of all proportion to the value. Send conditional requests, then compare the extracted text hash, and treat a model call as the last resort rather than the first step.
next phase
4
A bill you can see before it arrives
Every model call is recorded and a ceiling stops the job rather than the credit card.
1. One wrapper, and no provider call outside it.
  Files src/model/call.tstests/all-calls-logged.test.ts
2. Diff first, truncate second, and say that you did.
  Files src/watch/digest.ts
3. The alternative is finding out from a statement.
  Files src/scheduler.ts
done when · tick each as it passes
watch out
- Sending an entire long page into a summarisation call each time it changes turns one watched documentation site into the majority of your bill, and because each individual call looks reasonable the pattern is invisible until the month ends. Cap the characters sent per call, send the diff rather than the whole page where you can, and put spend by purpose on a page you actually look at.
5
Noticing when it has stoppedproduct builder
A dead scheduler is visible within one cadence, and everything on disk is readable without this app.
1. Including the ones that failed, especially those.
  Files src/scheduler.ts
2. On the digest page, on the chat page, on the source list.
  Files src/views/layout.tsxtests/heartbeat-stale.test.ts
3. So a dead process still leaves something greppable.
  Files src/digest/write.ts
done when · tick each as it passes
watch out
- A personal assistant does not fail with an error, it fails by quietly not running: the process dies after a reboot, the digest page keeps showing last week's summaries, and you carry on believing it is watching until you notice the dates. Record every run, show the last successful one on every page, and make a stale heartbeat impossible to miss.

what this build does not replace

OAuth connectors to third-party accounts.
A mobile app and push notifications.
Multi-user access and sharing.
Autonomous actions on your behalf.
Mobile app and notifications; your version is a browser tab
Prebuilt connectors to email, calendar, Slack and whatever else the paid product authenticates against
Someone else tuning prompts and swapping models when a better one ships
Reliability: your cron job dies silently and nobody pages you
Any team or sharing features, if the product has them

after v1, if you want it

A second watch cadence such as hourly for one item, which needs nothing new beyond a column already present
Re-embedding on an embedding model change, once the app detects and reports the mismatch
A weekly rollup of digests, built from rows you already store and one extra call per week

Need the files? The project pack on the verdict page hands your agent the whole brief · more ai assistants.