Build Pinboard

YESreplaces $1.83/mosaves $21.96/yrback to the verdict

0%0 of 19 items done

Saved on this device only. Tick prerequisites first, then work the phases in order · do not start one until the checks above it pass.

Save a page from a bookmarklet or the command line, extract and store its readable text so it survives the site going away, tag and annotate it, search the full text instantly, and export the whole archive as plain files you could read without this application.

estimated effort weekendthe files for this build are in the project pack

RuntimeNode 22 with a small HTTP serverExtractionReadability plus a sanitiserStorageSQLite with full text search, plus the extracted text on diskCaptureA bookmarklet and a command line entry pointSnapshotsText and a single-file HTML snapshot, images optional

Before step 1

Everything below is assumed from the first step. Tick each one when you actually have it, not when you plan to.

installfree
Why Runs the server, the fetcher and the indexer.
Get it nodejs.org, or nvm install 22. open ↗
Verify node --version prints v22 or higher
have readyfree
Why Text snapshots are small, but ten thousand articles still add up, and the index roughly doubles it.
Get it Estimate from your current bookmark count. Text-only archiving is modest; enabling images changes the picture entirely.
Verify Free space comfortably exceeds your estimate
have readyfree
Why Phase 2 is only meaningful against real pages: a clean article, a JavaScript-only page, one with an aggressive content security policy, one behind a soft wall and a very long page.
Get it Collect them from your own reading. Keep them as fixtures so extraction changes can be compared.
Verify All five are recorded with their expected extraction outcome
have ready5 to 10 USD a month, or free on hardware you own
Why Only needed if you want the archive reachable from more than one machine.
Get it Any small virtual server, or a machine at home with a tunnel. This archive is private, so authentication comes before exposure.
Verify The archive is reachable over HTTPS and refuses unauthenticated requests

Data model

Create these before the first phase that stores anything. Changing a table later is the expensive kind of change.

- bookmarks(id, url, canonical_url, title, description, added_at, read_at, starred, source)
- tags(id, name), bookmark_tags(bookmark_id, tag_id)
- snapshots(id, bookmark_id, kind, path, bytes, fetched_at, status) · kind is text, html or pdf
- notes(id, bookmark_id, body, created_at)
- bookmarks_fts · full text over title, description, notes and extracted text
- The URL is normalised on save so the same page saved from two places is one bookmark, not two.

Environment variables

These go in a .env file the app reads at startup. The pack's .env.example is this table as a file · copy it, never commit the filled-in version.

Variable	Needed	Example	Where the value comes from
`DB_PATH`	required	`./data/archive.db`	SQLite database with bookmarks, tags, notes and the search index.
`SNAPSHOT_DIR`	required	`./data/snapshots`	Where extracted text and HTML snapshots are written. Back this up alongside the database.
`FETCH_TIMEOUT_MS`	required	`10000`	Per-page fetch timeout. A slow page should fail rather than hold up a save.
`FETCH_MAX_BYTES`	required	`5000000`	Streaming byte cap per page, enforced during download.
`SAVE_TOKEN`secret	required	`replace-with-a-random-token`	Used by the bookmarklet. Scoped to saving only, because a bookmarklet cannot keep a secret well.
`LINKCHECK_INTERVAL_DAYS`	required	`90`	How often each URL is re-checked. Link rot works on months, so checking often is rude and pointless.
`LINKCHECK_CONCURRENCY`	required	`2`	Simultaneous checks. Low, and per-host limited, so you do not look like a crawler.

The build, in order

1
Save a link and normalise it
A URL becomes a bookmark, and the same page saved twice is one entry.
1. URL, optional title, optional tags. Return the bookmark so a caller knows whether it created or updated.
  Files src/save.js
2. Strip known tracking parameters, lowercase the host, remove a default port and a trailing slash. Do not strip unknown parameters, because some of them are the page.
  Files src/normalise.js
3. A table of URL pairs with expected same or different verdicts. This is where over-eager normalisation gets caught.
  Files test/normalise.test.js
done when · tick each as it passes
watch out
- Tracking parameters make every share of the same article a different URL, and an archive that treats them as different fills with duplicates within a month. Strip the known tracking parameters, but keep parameters that identify content, because for many sites the query string is the page.
next phase
2
Archiving the readable page
The article survives the site disappearing.
1. Timeout, byte cap enforced during streaming, and a check of robots.txt. You are a well-behaved client saving pages for yourself.
  Files src/fetch.js
2. Readability for the article, then strip scripts, event handlers and remote references so the snapshot is inert and self-contained.
  Files src/extract.js
  terminal
  npm install @mozilla/readability jsdom dompurify
3. A page that needs JavaScript, returns an error or is behind a wall is a failed snapshot with a reason. An empty snapshot pretending to be a success is worse than nothing.
  Files src/extract.js
done when · tick each as it passes
watch out
- Storing raw fetched HTML and rendering it later means executing someone else's scripts and loading their trackers every time you open your own archive. Sanitise at capture: strip scripts, event handlers and remote references, and store the cleaned version.
next phase
3
Tags, notes and fast search
Finding something you saved two years ago takes a few seconds.
1. Ten thousand bookmarks with article-length text. Without this fixture every performance claim is a guess.
  Files scripts/generate-archive.js
2. Title, description, notes and extracted text in one full text index, updated on save.
  Files src/search.js
3. The index is derived from the stored text, so rebuilding must be possible and must produce identical results.
  Files src/search.js
done when · tick each as it passes
watch out
- Ten thousand bookmarks with full article text is a large index, and a naive query with wildcards on both sides cannot use it. Use the full text index properly and measure with a realistic corpus, because the design that works at two hundred entries will not be obvious until it fails.
next phase
4
Capture that takes one click
Saving a page is fast enough that you actually do it.
1. Title, URL and any selected text. Keep it small enough to paste, and use a save-only token.
  Files public/bookmarklet.js
2. A strict content security policy will stop a bookmarklet. Detect and tell the user rather than failing silently, and offer the command line as the fallback.
  Files public/bookmarklet.js
3. The standard bookmarks HTML export, plus at least one service's JSON. Deduplicate through the same normalisation as saving.
  Files src/import.js
done when · tick each as it passes
watch out
- A bookmarklet runs on every site you visit, and a naive one exposes your archive token to any page that looks for it. Keep the token in the request rather than in a global, use a token scoped only to saving, and accept that a bookmarklet cannot be as secure as an extension.
next phase
5
Export that makes leaving easy
The archive is portable, and its value does not depend on this application existing.
1. Everything including snapshot paths, with a schema version.
  Files src/export.js
2. One file per bookmark with front matter and the extracted text. This is the format that outlives the application.
  Files src/export.js
3. Import into an empty database and compare a normalised checksum.
  Files test/roundtrip.test.js
done when · tick each as it passes
watch out
- An archive whose contents are only readable through its own application is the situation people are escaping when they leave a hosted service. The Markdown directory is not a convenience feature; it is the difference between an archive and a lock-in.
6
Hosting it, refreshing it and keeping itproduct builder
It runs somewhere reachable, notices dead links, and you have restored it once.
1. A private archive of what you read is sensitive. Put a login in front and get a certificate.
  Files src/auth.jsCaddyfile
2. Low concurrency, per-host limits, and a long interval. Mark dead rather than deleting, because the snapshot is now the only copy.
  Files src/linkcheck.js
3. The database and the snapshot directory. Restore onto a clean machine and record the date.
  Files scripts/backup.shREADME.md
done when · tick each as it passes
watch out
- Re-checking every URL on a schedule looks like a crawler to the sites you are checking and will get your address blocked. Spread checks over time, limit concurrency per host, and check each URL rarely: link rot works on a scale of months.

what this build does not replace

Archiving pages behind paywalls or logins.
Full asset archiving with images and stylesheets.
The proven longevity of a service that has run for years.
Convenient mobile capture without a browser extension.
hosted longevity, simple API, archive service, and zero-maintenance reliability
publisher bypasses
cross-device mobile capture
high-quality recommendation graph
licensed full-text feeds

after v1, if you want it

A weekly digest of pages saved but never opened
Optional image archiving for a subset of tags where it actually matters
Public sharing of a single tag as a static page

Need the files? The project pack on the verdict page hands your agent the whole brief · more reading.