Build BazQux Reader

YESreplaces $2.5/mosaves $30/yrback to the verdict

0%0 of 17 items done

Saved on this device only. Tick prerequisites first, then work the phases in order · do not start one until the checks above it pass.

A feed reader whose hard problems are polling without being a nuisance, deciding what counts as the same item when publishers change identifiers and republish posts, and rendering a stranger's HTML without handing them your browser.

estimated effort weekendthe files for this build are in the project pack

RuntimeTypeScript on Node 22, a poller process and a web processStorageSQLite with FTS5 over item titles and contentFetchingConditional requests, per-host serialisation and honoured retry intervalsParsingOne parser producing a common item shape from RSS, Atom and JSON FeedRenderingServer-side sanitisation against an allowlist, no remote subresources

Before step 1

Everything below is assumed from the first step. Tick each one when you actually have it, not when you plan to.

installfree
Why Runs the poller and the web process.
Get it nodejs.org, or nvm install 22. open ↗
Verify node --version prints v22 or higher
installfree
Why The archive is the point of the reader, and searching it needs FTS5, which is not in every build.
Get it Most distributions include it; check rather than assume. open ↗
Verify A create virtual table using fts5 statement succeeds
have readya few pounds a month, or free if you own the box
Why A poller that only runs when a laptop is open turns polite intervals into a burst every time you sit down.
Get it A small server, a home machine or a single-board computer.
Verify A feed polled overnight shows fetches spread across the night
have readyfree
Why Missing guids, republished items and relative image URLs are only visible in feeds nobody tidied up, and clean fixtures hide every bug in phase two.
Get it Export the OPML from whatever you read now, and include at least one feed with no guid and one that republishes.
Verify Each imports and its identity source distribution is visible
decidefree
Why Whether remote images load at all is a tracking decision, and how long items are kept is a disk decision; both are asserted by tests.
Get it Decide whether images load directly, through a proxy, or on request, and how long items are kept after they leave a feed.
Verify Both are in the configuration and named in the README

Data model

Create these before the first phase that stores anything. Changing a table later is the expensive kind of change.

- feeds(id, url, site_url, title, etag, last_modified, last_fetched_at, next_fetch_at, consecutive_failures, interval_seconds, state)
- fetches(id, feed_id, at, status, bytes, duration_ms, note) · the 304s are recorded too, because they are the evidence that the conditional requests work
- items(id, feed_id, stable_id, stable_id_source, url, title, author, published_at, first_seen_at, content_sanitised, content_sha256)
- item_state(item_id, read_at, starred_at, hidden_by_filter_id) · separate from the item, so a republish never resets it
- filters(id, scope_feed_id, field, pattern, action, created_at) · applied at display, recorded on the item, never destructive
- folders(id, name) and subscriptions(feed_id, folder_id, sort_order)
- An item is kept after it falls out of the feed, because the feed is a window of the last few posts and the database is the archive.

Environment variables

These go in a .env file the app reads at startup. The pack's .env.example is this table as a file · copy it, never commit the filled-in version.

Variable	Needed	Example	Where the value comes from
`DATABASE_PATH`	required	`./data/reader.db`	Feeds, items, read state and the search index.
`USER_AGENT`	required	`MyReader/1.0 (+https://example.com/reader)`	Names the software and a contact address, so a publisher can reach you rather than block you.
`MIN_POLL_INTERVAL_SECONDS`	required	`900`	The floor, regardless of what any feed suggests.
`MAX_POLL_INTERVAL_SECONDS`	required	`86400`	The ceiling a quiet feed widens to.
`MAX_CONSECUTIVE_FAILURES`	required	`10`	After this a feed is marked unwell and polled at the ceiling.
`IMAGE_MODE`	required	`proxy`	Direct, proxy or off. A remote image is a per-open tracking pixel.
`ITEM_RETENTION_DAYS`	required	`0`	Zero keeps everything. The archive is why the reader exists.

The build, in order

1
Polling like a good citizen
Fetching is conditional, paced, and backs off from a feed that is unwell.
1. A reader that cannot produce a 304 is a reader that is costing someone money.
  Files src/poll/fetch.tstests/conditional.test.ts
2. Guessing a backoff when the server told you the number gets you blocked.
  Files src/poll/backoff.tstests/retry-after.test.ts
3. One request at a time to a host, and a user agent with a contact address.
  Files src/poll/scheduler.ts
done when · tick each as it passes
watch out
- Polling every feed on a fixed timer with no conditional headers downloads the same megabyte from someone's small server all day, and when they respond with a 429 a naive client treats it as a failure, retries immediately and gets blocked outright. Send the validators, treat 304 as the expected answer, and wait exactly as long as the response asked.
next phase
2
Deciding what counts as the same item
An edited or republished post appears once, and its read state survives the edit.
1. When deduplication goes wrong, the source column is the first thing you read.
  Files src/parse/identity.tstests/identity.test.ts
2. State on the item row is state you lose to an edit.
  Files migrations/002_item_state.sql
3. Publishers fix typos, and fifty duplicates is how a reader loses a user.
  Files src/ingest/upsert.ts
done when · tick each as it passes
watch out
- Using the entry link as the identity works until a publisher fixes a typo and republishes, or reuses one URL for a running live blog, and the reader either shows fifty duplicates or silently hides new posts behind an old one; storing read state on the item row then loses it every time the content changes. Record which identity source was used per item, keep state in a separate table, and treat a changed content hash as an update rather than a new item.
next phase
3
Rendering a stranger's HTML
Feed content displays correctly without executing or fetching anything the publisher chose.
1. Deny lists are a list of the attacks you happened to think of.
  Files src/render/sanitise.tstests/xss-fixtures.test.ts
2. Half the images in real feeds are relative to something.
  Files src/render/urls.ts
3. Belt and braces, because sanitisers have had bugs.
  Files src/web/security.ts
done when · tick each as it passes
watch out
- Rendering feed HTML directly because it comes from sites you trust is how one compromised blog reaches every reader's session, and even without a script, a single remote image is a per-open pixel that tells the publisher when and where you read. Sanitise against an allowlist rather than a deny list, and decide deliberately whether remote images load at all.
next phase
4
Unread, filters and finding things again
Reading ten thousand items stays fast and filters hide without deleting.
1. Counting the items table per request stops being viable in the first year.
  Files src/state/counts.ts
2. An over-broad filter should be discoverable and reversible.
  Files src/filters/apply.tstests/filters-reversible.test.ts
3. The archive is the point; an archive you cannot search is storage.
  Files src/search/index.ts
done when · tick each as it passes
watch out
- Deleting items a filter matched means a filter written slightly too broadly silently removes a month of posts and there is no way to discover it, let alone undo it. Mark the item hidden with the filter's identifier, show a count of what each filter is hiding, and make unhiding one action.
5
Coming in, going out and staying welcomeproduct builder
Subscriptions move in and out freely and the poller stays a good neighbour at a hundred feeds.
1. OPML carries subscriptions and folders and nothing else.
  Files src/opml/import.tssrc/opml/export.tstests/opml-roundtrip.test.ts
2. A hundred simultaneous requests every hour is indistinguishable from an attack.
  Files src/poll/scheduler.tstests/jitter.test.ts
3. It is the number that tells you whether the manners actually work.
  Files src/web/routes/stats.tsx
done when · tick each as it passes
watch out
- Scheduling every feed on the same interval from the moment of import sends a hundred requests in one second every hour, which looks exactly like an attack from the receiving end and gets the address blocked; meanwhile a feed that updates twice a year is polled as often as one that updates hourly. Jitter the schedule at import and widen a quiet feed's interval from its own observed history.

what this build does not replace

Full-text scraping of sites that publish summaries.
Comment threads from platforms whose terms forbid it.
A multi-user hosted service.
Mobile clients and a sync protocol.
team libraries and institutional access
resilient polling of malformed and rate-limited feeds
licensed scholarly metadata
publisher-specific import reliability
citation graph scale

after v1, if you want it

WebSub or a similar push mechanism for the feeds that offer it, which removes polling for those entirely
Per-feed reading statistics computed locally, to decide what to unsubscribe from
An offline archive export per item, once the retention policy has run for a few months

Need the files? The project pack on the verdict page hands your agent the whole brief · more rss & research.