Vibecode OpenPTE

You are building a lean indie version of OpenPTE.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== README.md =====
# OpenPTE · indie build

A practice platform for exam-style speaking, writing, reading and listening tasks, where the honest engineering problem is producing genuinely useful feedback on a spoken or written response without pretending to be the official scoring engine.

Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | TypeScript with Fastify and PostgreSQL | the work is task delivery, timing and storing attempts |
| Speech | Local speech recognition with word timings and confidence | fluency and pronunciation feedback both derive from timings and confidence |
| Writing | Rule-based checks plus a model for discourse, each reported separately | a grammar error and a weak argument are different feedback and should not be merged into a number |
| Timing | Server-authoritative task timers | an exam practice tool where the timer can be paused teaches the wrong habit |
| Content | Original tasks written for the platform, never real exam items | reproducing exam questions is unlawful and teaches memorisation |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Node.js 22 or newer** · free
  - Why: Runs the server and task delivery.
  - Get it: nodejs.org, or nvm install 22.
  - Verify: node --version prints v22 or higher
- [ ] **PostgreSQL 16 or newer** · free locally
  - Why: Attempts, recognitions and observations with per-dimension queries.
  - Get it: Docker or your package manager.
  - Verify: psql connects and creates a table
- [ ] **A local speech recogniser with word timings and confidence** · free, needs machine time
  - Why: Fluency feedback comes from timings and uncertainty from confidence, and sending learners' voice recordings to a service is a poor default.
  - Get it: whisper.cpp or an equivalent, confirmed to emit per-word timings and confidence rather than segments only.
  - Verify: A spoken response produces per-word timings and confidences offline
- [ ] **Original practice tasks you wrote** · free, and it is real work
  - Why: Real exam items are copyrighted, and a platform built on them cannot be shared and does not teach the skill.
  - Get it: Write tasks matching the published format descriptions rather than reproducing items. Aim for a handful per format before building the delivery.
  - Verify: At least five original tasks exist per format, with none copied from an exam
- [ ] **Recordings from several first languages** · free, and requires asking people
  - Why: The fairness requirement of this build is that an accent is never reported as an error, and that can only be tested against varied speakers.
  - Get it: Ask people who would actually use this, with their permission, and keep the recordings only as long as needed for testing.
  - Verify: Recordings from at least four first languages exist and none is reported as mispronunciation

## Quick start

Follow Phase 1 in `BUILD_PLAN.md`; it creates the project.

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- Any predicted or estimated official score.
- Real exam questions.
- The official integrated skill scoring model.
- Accreditation or any claim of affiliation with the exam board.
- Scores calibrated to the real automated marker, so your numbers are vibes, not predictions
- A question bank that is maintained and rotated as the exam changes
- Full mock tests with official section timing, weighting and score report layout
- Model answers, templates and community discussion around each item
- Mobile apps, cross-device sync, and anyone to blame when the grader is wrong

If one of those is essential to you, that is the reason to keep paying for OpenPTE, and the README should say so rather than pretend.

===== BRIEF.md =====
# Build brief · OpenPTE

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a practice platform for an English proficiency exam, with feedback rather than fake scores to replace OpenPTE. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Rule zero
People take this exam for visas and university admission. A practice score they believe is
a prediction can lead someone to book a test they are not ready for, at real cost, or to
delay one they would have passed.
You cannot reproduce the official scoring. The real exam is machine scored on a scale from
10 to 90 per skill, using integrated tasks where a single spoken or written response feeds
into several communicative skill scores at once, plus enabling skills such as oral fluency,
pronunciation, grammar, spelling, vocabulary and written discourse. That engine is
proprietary, trained on data you do not have, and periodically revised.
So do not output a predicted overall band. Give component feedback that helps someone
improve: where the fluency broke down, which words were mispronounced, whether the essay
addressed the prompt. State plainly and in the interface, not only in a footer, that these
are practice observations and not a predicted result.
Also: do not reproduce real exam questions. They are copyrighted and using them is both
unlawful and a disservice, since memorised items do not build the skill.

### Stack (fixed, do not substitute)
- Runtime: TypeScript with Fastify and PostgreSQL · the work is task delivery, timing and storing attempts
- Speech: Local speech recognition with word timings and confidence · fluency and pronunciation feedback both derive from timings and confidence
- Writing: Rule-based checks plus a model for discourse, each reported separately · a grammar error and a weak argument are different feedback and should not be merged into a number
- Timing: Server-authoritative task timers · an exam practice tool where the timer can be paused teaches the wrong habit
- Content: Original tasks written for the platform, never real exam items · reproducing exam questions is unlawful and teaches memorisation

### Data model (create this before Phase 1)
- tasks(id, skill, format, prompt_text, audio_path, time_limit_seconds, source) · source is always original
- attempts(id, user_id, task_id, started_at, submitted_at, response_text, response_audio_path)
- recognitions(attempt_id, word_ordinal, text, start_ms, end_ms, confidence) · the basis for speech feedback
- observations(id, attempt_id, dimension, detail, evidence_json) · dimension is fluency, pronunciation, grammar, content or discourse
- progress(user_id, dimension, week, observation_count, trend) · improvement over time, never a predicted score
- There is deliberately no column anywhere holding a predicted overall band, because a field like that gets displayed eventually.

### Phase 1 · Tasks delivered under exam conditions
Build: Deliver reading, listening, speaking and writing tasks with server-authoritative timers,
  a preparation period where the format has one, and automatic submission at the limit.
  Handle the browser closing mid-task.
Trap: A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.
Done when: A task submits automatically at the time limit even if the browser was closed, tested by closing it mid-task. The timer cannot be extended from the client, verified by attempting it. Preparation and response periods are separate where the format requires. An interrupted attempt is recorded as interrupted rather than discarded.
Do not build yet: No recognition, feedback or progress.

### Phase 2 · Hearing what was actually said
Build: Recognise speech locally with word timings and per-word confidence. Handle accented
  speech without treating an accent as an error. Record audio at a quality that makes
  recognition meaningful and store it with the attempt.
Trap: Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.
Done when: A spoken response produces words with timings and confidences. A strongly accented but correct response is not reported as mispronounced, checked against recordings from several first languages. Poor audio is detected and reported as an audio problem rather than a speaking problem. Recognition runs with networking disabled.
Do not build yet: No feedback generation yet.

### Phase 3 · Feedback that names something specific
Build: Generate observations per dimension, each citing evidence: the timestamp where fluency
  broke, the word that was unclear, the sentence with the grammar issue. Reject any
  observation without evidence. Report dimensions separately and never combine them.
Trap: Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.
Done when: Every observation cites evidence that resolves to the response, with an uncited one rejected by a validator tested directly. Dimensions are reported separately with no aggregate anywhere in the model or interface, verified by inspection. Clicking an observation plays or highlights the evidence. Feedback on a good response says what worked rather than only what did not.
Do not build yet: No progress tracking yet.

### Phase 4 · Written responses, checked honestly
Build: Rule-based checks for spelling, grammar and word count reported deterministically, plus
  a model assessing whether the response addresses the prompt and how it is structured.
  Report the two sources separately so the user knows which is which.
Trap: Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.
Done when: Spelling, grammar and word count checks are deterministic and identical across runs, verified by running an essay twice. The model's observations quote the text they describe. The two sources are visually distinct. An essay that ignores the prompt is identified as such rather than praised for fluency.
Do not build yet: No progress or export yet.

### Phase 5 · Progress, without inventing a number
Build: Track observation counts per dimension over time and show trends. State plainly in the
  interface that this is practice feedback and not a predicted exam result. Export attempts
  and observations, and delete them completely on request.
Trap: A trend line drawn on a zero to ninety axis will be read as a predicted score no matter what the caption says, because the axis is the claim. Use counts and rates on their own scales, and keep the exam's scale out of the interface entirely.
Done when: Progress shows per-dimension trends with the exam's scale appearing nowhere in the interface, verified by reading every screen. The disclaimer appears with the feedback rather than only in a footer. Export contains attempts, audio and observations. Deletion removes attempts, recordings and derived observations together.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Any predicted or estimated official score, which cannot be produced honestly and is the one thing users will ask for most.
- Real exam questions, which are copyrighted and whose use would be both unlawful and pedagogically worthless.
- The official scoring engine's integrated skill model, where one response contributes to several skill scores through a proprietary method.
- Accredited preparation or any claim of affiliation with the exam board, which would be a misrepresentation.

### README must contain
- That no predicted score is produced and why that cannot be done honestly
- That all tasks are original and no real exam questions are used
- That low recogniser confidence is reported as uncertainty and never as mispronunciation
- That every observation must cite evidence in the response or it is rejected
- That the exam's scale appears nowhere in the interface, including on charts

===== AGENTS.md =====
# Agent instructions · OpenPTE indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: TypeScript with Fastify and PostgreSQL, Local speech recognition with word timings and confidence, Rule-based checks plus a model for discourse, each reported separately, Server-authoritative task timers, Original tasks written for the platform, never real exam items. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.
- Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.
- Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.
- Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.

===== BUILD_PLAN.md =====
# Build plan · OpenPTE

A practice platform for exam-style speaking, writing, reading and listening tasks, where the honest engineering problem is producing genuinely useful feedback on a spoken or written response without pretending to be the official scoring engine.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · Tasks delivered under exam conditions

A practice task runs with the same timing pressure as the real thing.

### Steps

1. Keep the clock on the server
   The client shows a countdown. It does not own one.
   Files: `src/timer.ts`
2. Submit automatically at the limit
   Even if the browser is gone. The attempt still happened.
   Files: `src/attempts.ts`
3. Record interruptions honestly
   An interrupted attempt is data, not something to hide.
   Files: `src/attempts.ts`

### Done when

- [ ] A task submits at the limit with the browser closed
- [ ] The timer cannot be extended from the client
- [ ] Preparation and response periods are separate
- [ ] An interrupted attempt is recorded as such

### Watch out

- A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.

## Phase 2 · Hearing what was actually said

A spoken response becomes words with times and confidences.

### Steps

1. Recognise with word timings and confidence
   Both are needed. Fluency comes from timings, uncertainty from confidence.
   Files: `src/recognise.ts`
2. Never equate low confidence with error
   Test with several accents. This is the fairness requirement of the whole build.
   Files: `tests/accents.md`
3. Detect audio problems separately
   A bad microphone is not a speaking weakness and must not be reported as one.
   Files: `src/audio-quality.ts`

### Done when

- [ ] Responses produce words with timings and confidence
- [ ] Accented correct speech is not reported as mispronounced
- [ ] Poor audio is reported as an audio problem
- [ ] Recognition works offline

### Watch out

- Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.

## Phase 3 · Feedback that names something specific

Every observation points at a moment or a phrase in the response.

### Steps

1. Require evidence per observation
   A timestamp or a span. Validate it and reject the output otherwise.
   Files: `src/observations.ts`, `src/validate.test.ts`
2. Keep dimensions separate everywhere
   No aggregate field. A number that exists will eventually be shown.
   Files: `src/schema.sql`
3. Say what worked too
   Feedback that only lists failures is demoralising and less useful.
   Files: `src/observations.ts`

### Done when

- [ ] Every observation cites resolvable evidence
- [ ] An uncited observation is rejected
- [ ] No aggregate score exists in model or interface
- [ ] Good responses receive positive observations

### Watch out

- Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.

## Phase 4 · Written responses, checked honestly

An essay gets specific feedback on grammar, content and structure separately.

### Steps

1. Run deterministic checks separately
   Same essay, same findings, every time. Label them as mechanical.
   Files: `src/checks.ts`
2. Require the model to quote
   An observation about a paragraph must include the paragraph.
   Files: `src/essay.ts`
3. Distinguish the two visually
   So the user knows which finding is reliable and which is an opinion.
   Files: `src/feedback-view.ts`

### Done when

- [ ] Mechanical checks are deterministic across runs
- [ ] Model observations quote the text
- [ ] The two sources are visually distinct
- [ ] An off-prompt essay is identified as off-prompt

### Watch out

- Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.

## Not in this build

- Any predicted or estimated official score.
- Real exam questions.
- The official integrated skill scoring model.
- Accreditation or any claim of affiliation with the exam board.

## After v1, if you want it

- Spaced repetition over the dimensions a learner is weakest in
- Model responses for each task, which is what learners ask for after feedback
- A study plan built from observation trends rather than from a score

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required · secret. Attempts and observations, including learners' voice recordings by reference.
DATABASE_URL=postgres://localhost:5432/practice

# Required. Voice recordings. Personal data, so retention and deletion apply to this directory too.
AUDIO_DIR=./attempts/audio

# Required. The local recogniser. Larger models give better timings and fairer confidence across accents.
ASR_MODEL=./models/ggml-medium.en.bin

# Required. How long voice recordings are kept. Shorter than the observations derived from them.
AUDIO_RETENTION_DAYS=90

# Required. Present as a constant so the answer is visible in configuration. There is no other value.
SHOW_PREDICTED_SCORE=never

You are building a lean indie version of OpenPTE.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== README.md =====
# OpenPTE · indie build

A practice platform for exam-style speaking, writing, reading and listening tasks, where the honest engineering problem is producing genuinely useful feedback on a spoken or written response without pretending to be the official scoring engine.

Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | TypeScript with Fastify and PostgreSQL | the work is task delivery, timing and storing attempts |
| Speech | Local speech recognition with word timings and confidence | fluency and pronunciation feedback both derive from timings and confidence |
| Writing | Rule-based checks plus a model for discourse, each reported separately | a grammar error and a weak argument are different feedback and should not be merged into a number |
| Timing | Server-authoritative task timers | an exam practice tool where the timer can be paused teaches the wrong habit |
| Content | Original tasks written for the platform, never real exam items | reproducing exam questions is unlawful and teaches memorisation |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Node.js 22 or newer** · free
  - Why: Runs the server and task delivery.
  - Get it: nodejs.org, or nvm install 22.
  - Verify: node --version prints v22 or higher
- [ ] **PostgreSQL 16 or newer** · free locally
  - Why: Attempts, recognitions and observations with per-dimension queries.
  - Get it: Docker or your package manager.
  - Verify: psql connects and creates a table
- [ ] **A local speech recogniser with word timings and confidence** · free, needs machine time
  - Why: Fluency feedback comes from timings and uncertainty from confidence, and sending learners' voice recordings to a service is a poor default.
  - Get it: whisper.cpp or an equivalent, confirmed to emit per-word timings and confidence rather than segments only.
  - Verify: A spoken response produces per-word timings and confidences offline
- [ ] **Original practice tasks you wrote** · free, and it is real work
  - Why: Real exam items are copyrighted, and a platform built on them cannot be shared and does not teach the skill.
  - Get it: Write tasks matching the published format descriptions rather than reproducing items. Aim for a handful per format before building the delivery.
  - Verify: At least five original tasks exist per format, with none copied from an exam
- [ ] **Recordings from several first languages** · free, and requires asking people
  - Why: The fairness requirement of this build is that an accent is never reported as an error, and that can only be tested against varied speakers.
  - Get it: Ask people who would actually use this, with their permission, and keep the recordings only as long as needed for testing.
  - Verify: Recordings from at least four first languages exist and none is reported as mispronunciation

## Quick start

Follow Phase 1 in `BUILD_PLAN.md`; it creates the project.

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- Any predicted or estimated official score.
- Real exam questions.
- The official integrated skill scoring model.
- Accreditation or any claim of affiliation with the exam board.
- Scores calibrated to the real automated marker, so your numbers are vibes, not predictions
- A question bank that is maintained and rotated as the exam changes
- Full mock tests with official section timing, weighting and score report layout
- Model answers, templates and community discussion around each item
- Mobile apps, cross-device sync, and anyone to blame when the grader is wrong

If one of those is essential to you, that is the reason to keep paying for OpenPTE, and the README should say so rather than pretend.

===== BRIEF.md =====
# Build brief · OpenPTE

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a practice platform for an English proficiency exam, with feedback rather than fake scores to replace OpenPTE. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Rule zero
People take this exam for visas and university admission. A practice score they believe is
a prediction can lead someone to book a test they are not ready for, at real cost, or to
delay one they would have passed.
You cannot reproduce the official scoring. The real exam is machine scored on a scale from
10 to 90 per skill, using integrated tasks where a single spoken or written response feeds
into several communicative skill scores at once, plus enabling skills such as oral fluency,
pronunciation, grammar, spelling, vocabulary and written discourse. That engine is
proprietary, trained on data you do not have, and periodically revised.
So do not output a predicted overall band. Give component feedback that helps someone
improve: where the fluency broke down, which words were mispronounced, whether the essay
addressed the prompt. State plainly and in the interface, not only in a footer, that these
are practice observations and not a predicted result.
Also: do not reproduce real exam questions. They are copyrighted and using them is both
unlawful and a disservice, since memorised items do not build the skill.

### Stack (fixed, do not substitute)
- Runtime: TypeScript with Fastify and PostgreSQL · the work is task delivery, timing and storing attempts
- Speech: Local speech recognition with word timings and confidence · fluency and pronunciation feedback both derive from timings and confidence
- Writing: Rule-based checks plus a model for discourse, each reported separately · a grammar error and a weak argument are different feedback and should not be merged into a number
- Timing: Server-authoritative task timers · an exam practice tool where the timer can be paused teaches the wrong habit
- Content: Original tasks written for the platform, never real exam items · reproducing exam questions is unlawful and teaches memorisation

### Data model (create this before Phase 1)
- tasks(id, skill, format, prompt_text, audio_path, time_limit_seconds, source) · source is always original
- attempts(id, user_id, task_id, started_at, submitted_at, response_text, response_audio_path)
- recognitions(attempt_id, word_ordinal, text, start_ms, end_ms, confidence) · the basis for speech feedback
- observations(id, attempt_id, dimension, detail, evidence_json) · dimension is fluency, pronunciation, grammar, content or discourse
- progress(user_id, dimension, week, observation_count, trend) · improvement over time, never a predicted score
- There is deliberately no column anywhere holding a predicted overall band, because a field like that gets displayed eventually.

### Phase 1 · Tasks delivered under exam conditions
Build: Deliver reading, listening, speaking and writing tasks with server-authoritative timers,
  a preparation period where the format has one, and automatic submission at the limit.
  Handle the browser closing mid-task.
Trap: A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.
Done when: A task submits automatically at the time limit even if the browser was closed, tested by closing it mid-task. The timer cannot be extended from the client, verified by attempting it. Preparation and response periods are separate where the format requires. An interrupted attempt is recorded as interrupted rather than discarded.
Do not build yet: No recognition, feedback or progress.

### Phase 2 · Hearing what was actually said
Build: Recognise speech locally with word timings and per-word confidence. Handle accented
  speech without treating an accent as an error. Record audio at a quality that makes
  recognition meaningful and store it with the attempt.
Trap: Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.
Done when: A spoken response produces words with timings and confidences. A strongly accented but correct response is not reported as mispronounced, checked against recordings from several first languages. Poor audio is detected and reported as an audio problem rather than a speaking problem. Recognition runs with networking disabled.
Do not build yet: No feedback generation yet.

### Phase 3 · Feedback that names something specific
Build: Generate observations per dimension, each citing evidence: the timestamp where fluency
  broke, the word that was unclear, the sentence with the grammar issue. Reject any
  observation without evidence. Report dimensions separately and never combine them.
Trap: Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.
Done when: Every observation cites evidence that resolves to the response, with an uncited one rejected by a validator tested directly. Dimensions are reported separately with no aggregate anywhere in the model or interface, verified by inspection. Clicking an observation plays or highlights the evidence. Feedback on a good response says what worked rather than only what did not.
Do not build yet: No progress tracking yet.

### Phase 4 · Written responses, checked honestly
Build: Rule-based checks for spelling, grammar and word count reported deterministically, plus
  a model assessing whether the response addresses the prompt and how it is structured.
  Report the two sources separately so the user knows which is which.
Trap: Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.
Done when: Spelling, grammar and word count checks are deterministic and identical across runs, verified by running an essay twice. The model's observations quote the text they describe. The two sources are visually distinct. An essay that ignores the prompt is identified as such rather than praised for fluency.
Do not build yet: No progress or export yet.

### Phase 5 · Progress, without inventing a number
Build: Track observation counts per dimension over time and show trends. State plainly in the
  interface that this is practice feedback and not a predicted exam result. Export attempts
  and observations, and delete them completely on request.
Trap: A trend line drawn on a zero to ninety axis will be read as a predicted score no matter what the caption says, because the axis is the claim. Use counts and rates on their own scales, and keep the exam's scale out of the interface entirely.
Done when: Progress shows per-dimension trends with the exam's scale appearing nowhere in the interface, verified by reading every screen. The disclaimer appears with the feedback rather than only in a footer. Export contains attempts, audio and observations. Deletion removes attempts, recordings and derived observations together.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Any predicted or estimated official score, which cannot be produced honestly and is the one thing users will ask for most.
- Real exam questions, which are copyrighted and whose use would be both unlawful and pedagogically worthless.
- The official scoring engine's integrated skill model, where one response contributes to several skill scores through a proprietary method.
- Accredited preparation or any claim of affiliation with the exam board, which would be a misrepresentation.

### README must contain
- That no predicted score is produced and why that cannot be done honestly
- That all tasks are original and no real exam questions are used
- That low recogniser confidence is reported as uncertainty and never as mispronunciation
- That every observation must cite evidence in the response or it is rejected
- That the exam's scale appears nowhere in the interface, including on charts

===== AGENTS.md =====
# Agent instructions · OpenPTE indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: TypeScript with Fastify and PostgreSQL, Local speech recognition with word timings and confidence, Rule-based checks plus a model for discourse, each reported separately, Server-authoritative task timers, Original tasks written for the platform, never real exam items. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.
- Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.
- Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.
- Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.

===== BUILD_PLAN.md =====
# Build plan · OpenPTE

A practice platform for exam-style speaking, writing, reading and listening tasks, where the honest engineering problem is producing genuinely useful feedback on a spoken or written response without pretending to be the official scoring engine.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · Tasks delivered under exam conditions

A practice task runs with the same timing pressure as the real thing.

### Steps

1. Keep the clock on the server
   The client shows a countdown. It does not own one.
   Files: `src/timer.ts`
2. Submit automatically at the limit
   Even if the browser is gone. The attempt still happened.
   Files: `src/attempts.ts`
3. Record interruptions honestly
   An interrupted attempt is data, not something to hide.
   Files: `src/attempts.ts`

### Done when

- [ ] A task submits at the limit with the browser closed
- [ ] The timer cannot be extended from the client
- [ ] Preparation and response periods are separate
- [ ] An interrupted attempt is recorded as such

### Watch out

- A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.

## Phase 2 · Hearing what was actually said

A spoken response becomes words with times and confidences.

### Steps

1. Recognise with word timings and confidence
   Both are needed. Fluency comes from timings, uncertainty from confidence.
   Files: `src/recognise.ts`
2. Never equate low confidence with error
   Test with several accents. This is the fairness requirement of the whole build.
   Files: `tests/accents.md`
3. Detect audio problems separately
   A bad microphone is not a speaking weakness and must not be reported as one.
   Files: `src/audio-quality.ts`

### Done when

- [ ] Responses produce words with timings and confidence
- [ ] Accented correct speech is not reported as mispronounced
- [ ] Poor audio is reported as an audio problem
- [ ] Recognition works offline

### Watch out

- Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.

## Phase 3 · Feedback that names something specific

Every observation points at a moment or a phrase in the response.

### Steps

1. Require evidence per observation
   A timestamp or a span. Validate it and reject the output otherwise.
   Files: `src/observations.ts`, `src/validate.test.ts`
2. Keep dimensions separate everywhere
   No aggregate field. A number that exists will eventually be shown.
   Files: `src/schema.sql`
3. Say what worked too
   Feedback that only lists failures is demoralising and less useful.
   Files: `src/observations.ts`

### Done when

- [ ] Every observation cites resolvable evidence
- [ ] An uncited observation is rejected
- [ ] No aggregate score exists in model or interface
- [ ] Good responses receive positive observations

### Watch out

- Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.

## Phase 4 · Written responses, checked honestly

An essay gets specific feedback on grammar, content and structure separately.

### Steps

1. Run deterministic checks separately
   Same essay, same findings, every time. Label them as mechanical.
   Files: `src/checks.ts`
2. Require the model to quote
   An observation about a paragraph must include the paragraph.
   Files: `src/essay.ts`
3. Distinguish the two visually
   So the user knows which finding is reliable and which is an opinion.
   Files: `src/feedback-view.ts`

### Done when

- [ ] Mechanical checks are deterministic across runs
- [ ] Model observations quote the text
- [ ] The two sources are visually distinct
- [ ] An off-prompt essay is identified as off-prompt

### Watch out

- Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.

## Not in this build

- Any predicted or estimated official score.
- Real exam questions.
- The official integrated skill scoring model.
- Accreditation or any claim of affiliation with the exam board.

## After v1, if you want it

- Spaced repetition over the dimensions a learner is weakest in
- Model responses for each task, which is what learners ask for after feedback
- A study plan built from observation trends rather than from a score

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required · secret. Attempts and observations, including learners' voice recordings by reference.
DATABASE_URL=postgres://localhost:5432/practice

# Required. Voice recordings. Personal data, so retention and deletion apply to this directory too.
AUDIO_DIR=./attempts/audio

# Required. The local recogniser. Larger models give better timings and fairer confidence across accents.
ASR_MODEL=./models/ggml-medium.en.bin

# Required. How long voice recordings are kept. Shorter than the observations derived from them.
AUDIO_RETENTION_DAYS=90

# Required. Present as a constant so the answer is visible in configuration. There is no other value.
SHOW_PREDICTED_SCORE=never

You are building a production product version of OpenPTE.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== PRODUCT.md =====
# OpenPTE · product brief

## Problem

The mechanics of a PTE trainer are not hard: record audio, transcribe it, time the task, score against a rubric, keep a history. An agent can build that in a weekend using Whisper for transcription and an LLM for rubric feedback, and you will genuinely practice more because the loop is yours. What you cannot build is the part people actually pay for: a question bank that tracks what is currently showing up in the real exam, and a scoring model calibrated against Pearson's automated marker so the number you see means something. Your DIY grader will be directionally useful and numerically fictional. Good enough for drilling fluency and essay structure, not good enough to decide whether you are ready to book the test.

## Product outcome

A practice platform that runs tasks under real timing pressure and gives specific, evidenced feedback, while refusing to produce the predicted score that would be the easiest thing to sell and the least honest.

## Target user

A builder who needs a maintainable product foundation, not a one-off demo.

## Required capabilities

- Node 20 and a modern browser with mic permission
- An OpenAI-compatible API key in .env for transcription and rubric scoring, or a local whisper.cpp build
- Your own question set: prompts, images, and audio you supply as JSON
- Time to write the rubric prompts per task type, which is most of the actual work

## Explicit non-goals for v1

- Any predicted or estimated official score.
- Real exam questions.
- The official integrated skill scoring model.
- Accreditation or any claim of affiliation with the exam board.
- Scores calibrated to the real automated marker, so your numbers are vibes, not predictions
- A question bank that is maintained and rotated as the exam changes
- Full mock tests with official section timing, weighting and score report layout
- Model answers, templates and community discussion around each item
- Mobile apps, cross-device sync, and anyone to blame when the grader is wrong

## Success criteria

- No aggregate or predicted score is confirmed absent from model and interface
- The exam's scale is confirmed absent from every chart
- Accented correct speech is confirmed not reported as mispronunciation
- An uncited observation is confirmed rejected
- A task is confirmed to submit at the limit with the browser closed

===== BRIEF.md =====
# Build brief · OpenPTE

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a practice platform for an English proficiency exam, with feedback rather than fake scores to replace OpenPTE. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Rule zero
People take this exam for visas and university admission. A practice score they believe is
a prediction can lead someone to book a test they are not ready for, at real cost, or to
delay one they would have passed.
You cannot reproduce the official scoring. The real exam is machine scored on a scale from
10 to 90 per skill, using integrated tasks where a single spoken or written response feeds
into several communicative skill scores at once, plus enabling skills such as oral fluency,
pronunciation, grammar, spelling, vocabulary and written discourse. That engine is
proprietary, trained on data you do not have, and periodically revised.
So do not output a predicted overall band. Give component feedback that helps someone
improve: where the fluency broke down, which words were mispronounced, whether the essay
addressed the prompt. State plainly and in the interface, not only in a footer, that these
are practice observations and not a predicted result.
Also: do not reproduce real exam questions. They are copyrighted and using them is both
unlawful and a disservice, since memorised items do not build the skill.

### Stack (fixed, do not substitute)
- Runtime: TypeScript with Fastify and PostgreSQL · the work is task delivery, timing and storing attempts
- Speech: Local speech recognition with word timings and confidence · fluency and pronunciation feedback both derive from timings and confidence
- Writing: Rule-based checks plus a model for discourse, each reported separately · a grammar error and a weak argument are different feedback and should not be merged into a number
- Timing: Server-authoritative task timers · an exam practice tool where the timer can be paused teaches the wrong habit
- Content: Original tasks written for the platform, never real exam items · reproducing exam questions is unlawful and teaches memorisation

### Data model (create this before Phase 1)
- tasks(id, skill, format, prompt_text, audio_path, time_limit_seconds, source) · source is always original
- attempts(id, user_id, task_id, started_at, submitted_at, response_text, response_audio_path)
- recognitions(attempt_id, word_ordinal, text, start_ms, end_ms, confidence) · the basis for speech feedback
- observations(id, attempt_id, dimension, detail, evidence_json) · dimension is fluency, pronunciation, grammar, content or discourse
- progress(user_id, dimension, week, observation_count, trend) · improvement over time, never a predicted score
- There is deliberately no column anywhere holding a predicted overall band, because a field like that gets displayed eventually.

### Phase 1 · Tasks delivered under exam conditions
Build: Deliver reading, listening, speaking and writing tasks with server-authoritative timers,
  a preparation period where the format has one, and automatic submission at the limit.
  Handle the browser closing mid-task.
Trap: A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.
Done when: A task submits automatically at the time limit even if the browser was closed, tested by closing it mid-task. The timer cannot be extended from the client, verified by attempting it. Preparation and response periods are separate where the format requires. An interrupted attempt is recorded as interrupted rather than discarded.
Do not build yet: No recognition, feedback or progress.

### Phase 2 · Hearing what was actually said
Build: Recognise speech locally with word timings and per-word confidence. Handle accented
  speech without treating an accent as an error. Record audio at a quality that makes
  recognition meaningful and store it with the attempt.
Trap: Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.
Done when: A spoken response produces words with timings and confidences. A strongly accented but correct response is not reported as mispronounced, checked against recordings from several first languages. Poor audio is detected and reported as an audio problem rather than a speaking problem. Recognition runs with networking disabled.
Do not build yet: No feedback generation yet.

### Phase 3 · Feedback that names something specific
Build: Generate observations per dimension, each citing evidence: the timestamp where fluency
  broke, the word that was unclear, the sentence with the grammar issue. Reject any
  observation without evidence. Report dimensions separately and never combine them.
Trap: Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.
Done when: Every observation cites evidence that resolves to the response, with an uncited one rejected by a validator tested directly. Dimensions are reported separately with no aggregate anywhere in the model or interface, verified by inspection. Clicking an observation plays or highlights the evidence. Feedback on a good response says what worked rather than only what did not.
Do not build yet: No progress tracking yet.

### Phase 4 · Written responses, checked honestly
Build: Rule-based checks for spelling, grammar and word count reported deterministically, plus
  a model assessing whether the response addresses the prompt and how it is structured.
  Report the two sources separately so the user knows which is which.
Trap: Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.
Done when: Spelling, grammar and word count checks are deterministic and identical across runs, verified by running an essay twice. The model's observations quote the text they describe. The two sources are visually distinct. An essay that ignores the prompt is identified as such rather than praised for fluency.
Do not build yet: No progress or export yet.

### Phase 5 · Progress, without inventing a number
Build: Track observation counts per dimension over time and show trends. State plainly in the
  interface that this is practice feedback and not a predicted exam result. Export attempts
  and observations, and delete them completely on request.
Trap: A trend line drawn on a zero to ninety axis will be read as a predicted score no matter what the caption says, because the axis is the claim. Use counts and rates on their own scales, and keep the exam's scale out of the interface entirely.
Done when: Progress shows per-dimension trends with the exam's scale appearing nowhere in the interface, verified by reading every screen. The disclaimer appears with the feedback rather than only in a footer. Export contains attempts, audio and observations. Deletion removes attempts, recordings and derived observations together.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Any predicted or estimated official score, which cannot be produced honestly and is the one thing users will ask for most.
- Real exam questions, which are copyrighted and whose use would be both unlawful and pedagogically worthless.
- The official scoring engine's integrated skill model, where one response contributes to several skill scores through a proprietary method.
- Accredited preparation or any claim of affiliation with the exam board, which would be a misrepresentation.

### README must contain
- That no predicted score is produced and why that cannot be done honestly
- That all tasks are original and no real exam questions are used
- That low recogniser confidence is reported as uncertainty and never as mispronunciation
- That every observation must cite evidence in the response or it is rejected
- That the exam's scale appears nowhere in the interface, including on charts

===== ARCHITECTURE.md =====
# Architecture · OpenPTE

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | TypeScript with Fastify and PostgreSQL | the work is task delivery, timing and storing attempts |
| Speech | Local speech recognition with word timings and confidence | fluency and pronunciation feedback both derive from timings and confidence |
| Writing | Rule-based checks plus a model for discourse, each reported separately | a grammar error and a weak argument are different feedback and should not be merged into a number |
| Timing | Server-authoritative task timers | an exam practice tool where the timer can be paused teaches the wrong habit |
| Content | Original tasks written for the platform, never real exam items | reproducing exam questions is unlawful and teaches memorisation |

## Modules

Each module has one owner concern and a documented way to replace it.

| Module | Owns | How to replace it |
| --- | --- | --- |
| Delivery | Server-authoritative timing and submission. | Any implementation, provided the clock is never the client's. |
| Recognition | Word timings and confidence. | Any recogniser, provided confidence is never presented as correctness. |
| Observations | Evidenced, per-dimension feedback. | Any generator, provided uncited observations are rejected and no aggregate exists. |
| Checks | Deterministic mechanical findings. | Any checker, provided its findings stay distinguishable from a model's opinions. |
| Progress | Trends that are not scores. | Any presentation that keeps the exam's scale out of the interface. |

## Configuration

Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists.

- `DATABASE_URL` · required, secret · Attempts and observations, including learners' voice recordings by reference.
- `AUDIO_DIR` · required · Voice recordings. Personal data, so retention and deletion apply to this directory too.
- `ASR_MODEL` · required · The local recogniser. Larger models give better timings and fairer confidence across accents.
- `AUDIO_RETENTION_DAYS` · required · How long voice recordings are kept. Shorter than the observations derived from them.
- `SHOW_PREDICTED_SCORE` · required · Present as a constant so the answer is visible in configuration. There is no other value.

## Production baseline

- Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives.
- Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised.
- Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts.
- Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists.
- Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path.

## Decision records

For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.

===== AGENTS.md =====
# Agent instructions · OpenPTE product build

- Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: TypeScript with Fastify and PostgreSQL, Local speech recognition with word timings and confidence, Rule-based checks plus a model for discourse, each reported separately, Server-authoritative task timers, Original tasks written for the platform, never real exam items.
- Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit.
- Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present.
- Never invent cryptography or silently weaken a requirement to make a check pass.
- Put every external service behind an interface with a deterministic fake for tests.
- Add migrations and rollback or recovery notes for every persistent data change.
- Log useful operational context without credentials, tokens, passwords or personal data.
- Update documentation and run every check before completing a milestone.

## Known traps

- A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.
- Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.
- Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.
- Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.
- A trend line drawn on a zero to ninety axis will be read as a predicted score no matter what the caption says, because the axis is the claim. Use counts and rates on their own scales, and keep the exam's scale out of the interface entirely.

===== MILESTONES.md =====
# Delivery milestones · OpenPTE

Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer.

## M1 · Tasks delivered under exam conditions

A practice task runs with the same timing pressure as the real thing.

### Steps

1. Keep the clock on the server
   The client shows a countdown. It does not own one.
   Files: `src/timer.ts`
2. Submit automatically at the limit
   Even if the browser is gone. The attempt still happened.
   Files: `src/attempts.ts`
3. Record interruptions honestly
   An interrupted attempt is data, not something to hide.
   Files: `src/attempts.ts`

### Done when

- [ ] A task submits at the limit with the browser closed
- [ ] The timer cannot be extended from the client
- [ ] Preparation and response periods are separate
- [ ] An interrupted attempt is recorded as such

### Watch out

- A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.

## M2 · Hearing what was actually said

A spoken response becomes words with times and confidences.

### Steps

1. Recognise with word timings and confidence
   Both are needed. Fluency comes from timings, uncertainty from confidence.
   Files: `src/recognise.ts`
2. Never equate low confidence with error
   Test with several accents. This is the fairness requirement of the whole build.
   Files: `tests/accents.md`
3. Detect audio problems separately
   A bad microphone is not a speaking weakness and must not be reported as one.
   Files: `src/audio-quality.ts`

### Done when

- [ ] Responses produce words with timings and confidence
- [ ] Accented correct speech is not reported as mispronounced
- [ ] Poor audio is reported as an audio problem
- [ ] Recognition works offline

### Watch out

- Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.

## M3 · Feedback that names something specific

Every observation points at a moment or a phrase in the response.

### Steps

1. Require evidence per observation
   A timestamp or a span. Validate it and reject the output otherwise.
   Files: `src/observations.ts`, `src/validate.test.ts`
2. Keep dimensions separate everywhere
   No aggregate field. A number that exists will eventually be shown.
   Files: `src/schema.sql`
3. Say what worked too
   Feedback that only lists failures is demoralising and less useful.
   Files: `src/observations.ts`

### Done when

- [ ] Every observation cites resolvable evidence
- [ ] An uncited observation is rejected
- [ ] No aggregate score exists in model or interface
- [ ] Good responses receive positive observations

### Watch out

- Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.

## M4 · Written responses, checked honestly

An essay gets specific feedback on grammar, content and structure separately.

### Steps

1. Run deterministic checks separately
   Same essay, same findings, every time. Label them as mechanical.
   Files: `src/checks.ts`
2. Require the model to quote
   An observation about a paragraph must include the paragraph.
   Files: `src/essay.ts`
3. Distinguish the two visually
   So the user knows which finding is reliable and which is an opinion.
   Files: `src/feedback-view.ts`

### Done when

- [ ] Mechanical checks are deterministic across runs
- [ ] Model observations quote the text
- [ ] The two sources are visually distinct
- [ ] An off-prompt essay is identified as off-prompt

### Watch out

- Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.

## M5 · Progress, without inventing a number (production only)

A learner sees improvement over weeks without ever being shown a predicted result.

### Steps

1. Trend counts and rates, not scores
   And never on the exam's scale. The axis is the claim.
   Files: `src/progress.ts`
2. Put the disclaimer with the feedback
   Where it is read, not in a footer where it is not.
   Files: `src/feedback-view.ts`
3. Export and delete completely
   Including voice recordings, which are biometric-adjacent and personal.
   Files: `src/export.ts`, `src/deletion.ts`

### Done when

- [ ] The exam's scale appears nowhere in the interface
- [ ] The disclaimer appears with the feedback
- [ ] Export contains attempts, audio and observations
- [ ] Deletion removes recordings and observations together

### Watch out

- A trend line drawn on a zero to ninety axis will be read as a predicted score no matter what the caption says, because the axis is the claim. Use counts and rates on their own scales, and keep the exam's scale out of the interface entirely.

===== OPERATIONS.md =====
# Operations · OpenPTE

## Backup

Voice recordings are personal data belonging to learners. Back up with encryption, apply the retention period to backups, and consider whether you need to keep audio at all once observations exist.

## Restore

Restore and confirm retention resumes, since restoring expired voice recordings recreates personal data that should have been deleted.

Do a restore drill before the first real user, and write the date here when it passes.

## Monitoring

Watch the rate at which low-confidence recognition is being produced per speaker, since a speaker whose every response comes back uncertain is being failed by the recogniser rather than by their speaking.

## Incident checklist

Any aggregate score appearing in the interface is the incident, whether added deliberately or derived by a chart axis. The schema has no such field, and that absence is the control worth protecting.

1. Contain the issue without destroying evidence or user data.
2. Record the timeline and affected scope.
3. Rotate exposed secrets and revoke compromised sessions or credentials.
4. Restore from a verified backup when needed.
5. Document the root cause, the remediation and the regression test.

## Release gate

- [ ] No aggregate or predicted score is confirmed absent from model and interface
- [ ] The exam's scale is confirmed absent from every chart
- [ ] Accented correct speech is confirmed not reported as mispronunciation
- [ ] An uncited observation is confirmed rejected
- [ ] A task is confirmed to submit at the limit with the browser closed

## Launch constraint

Do not market omitted OpenPTE capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required · secret. Attempts and observations, including learners' voice recordings by reference.
DATABASE_URL=postgres://localhost:5432/practice

# Required. Voice recordings. Personal data, so retention and deletion apply to this directory too.
AUDIO_DIR=./attempts/audio

# Required. The local recogniser. Larger models give better timings and fairer confidence across accents.
ASR_MODEL=./models/ggml-medium.en.bin

# Required. How long voice recordings are kept. Shorter than the observations derived from them.
AUDIO_RETENTION_DAYS=90

# Required. Present as a constant so the answer is visible in configuration. There is no other value.
SHOW_PREDICTED_SCORE=never

# OpenPTE · indie build

A practice platform for exam-style speaking, writing, reading and listening tasks, where the honest engineering problem is producing genuinely useful feedback on a spoken or written response without pretending to be the official scoring engine.

Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | TypeScript with Fastify and PostgreSQL | the work is task delivery, timing and storing attempts |
| Speech | Local speech recognition with word timings and confidence | fluency and pronunciation feedback both derive from timings and confidence |
| Writing | Rule-based checks plus a model for discourse, each reported separately | a grammar error and a weak argument are different feedback and should not be merged into a number |
| Timing | Server-authoritative task timers | an exam practice tool where the timer can be paused teaches the wrong habit |
| Content | Original tasks written for the platform, never real exam items | reproducing exam questions is unlawful and teaches memorisation |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Node.js 22 or newer** · free
  - Why: Runs the server and task delivery.
  - Get it: nodejs.org, or nvm install 22.
  - Verify: node --version prints v22 or higher
- [ ] **PostgreSQL 16 or newer** · free locally
  - Why: Attempts, recognitions and observations with per-dimension queries.
  - Get it: Docker or your package manager.
  - Verify: psql connects and creates a table
- [ ] **A local speech recogniser with word timings and confidence** · free, needs machine time
  - Why: Fluency feedback comes from timings and uncertainty from confidence, and sending learners' voice recordings to a service is a poor default.
  - Get it: whisper.cpp or an equivalent, confirmed to emit per-word timings and confidence rather than segments only.
  - Verify: A spoken response produces per-word timings and confidences offline
- [ ] **Original practice tasks you wrote** · free, and it is real work
  - Why: Real exam items are copyrighted, and a platform built on them cannot be shared and does not teach the skill.
  - Get it: Write tasks matching the published format descriptions rather than reproducing items. Aim for a handful per format before building the delivery.
  - Verify: At least five original tasks exist per format, with none copied from an exam
- [ ] **Recordings from several first languages** · free, and requires asking people
  - Why: The fairness requirement of this build is that an accent is never reported as an error, and that can only be tested against varied speakers.
  - Get it: Ask people who would actually use this, with their permission, and keep the recordings only as long as needed for testing.
  - Verify: Recordings from at least four first languages exist and none is reported as mispronunciation

## Quick start

Follow Phase 1 in `BUILD_PLAN.md`; it creates the project.

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- Any predicted or estimated official score.
- Real exam questions.
- The official integrated skill scoring model.
- Accreditation or any claim of affiliation with the exam board.
- Scores calibrated to the real automated marker, so your numbers are vibes, not predictions
- A question bank that is maintained and rotated as the exam changes
- Full mock tests with official section timing, weighting and score report layout
- Model answers, templates and community discussion around each item
- Mobile apps, cross-device sync, and anyone to blame when the grader is wrong

If one of those is essential to you, that is the reason to keep paying for OpenPTE, and the README should say so rather than pretend.

# Build brief · OpenPTE

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a practice platform for an English proficiency exam, with feedback rather than fake scores to replace OpenPTE. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Rule zero
People take this exam for visas and university admission. A practice score they believe is
a prediction can lead someone to book a test they are not ready for, at real cost, or to
delay one they would have passed.
You cannot reproduce the official scoring. The real exam is machine scored on a scale from
10 to 90 per skill, using integrated tasks where a single spoken or written response feeds
into several communicative skill scores at once, plus enabling skills such as oral fluency,
pronunciation, grammar, spelling, vocabulary and written discourse. That engine is
proprietary, trained on data you do not have, and periodically revised.
So do not output a predicted overall band. Give component feedback that helps someone
improve: where the fluency broke down, which words were mispronounced, whether the essay
addressed the prompt. State plainly and in the interface, not only in a footer, that these
are practice observations and not a predicted result.
Also: do not reproduce real exam questions. They are copyrighted and using them is both
unlawful and a disservice, since memorised items do not build the skill.

### Stack (fixed, do not substitute)
- Runtime: TypeScript with Fastify and PostgreSQL · the work is task delivery, timing and storing attempts
- Speech: Local speech recognition with word timings and confidence · fluency and pronunciation feedback both derive from timings and confidence
- Writing: Rule-based checks plus a model for discourse, each reported separately · a grammar error and a weak argument are different feedback and should not be merged into a number
- Timing: Server-authoritative task timers · an exam practice tool where the timer can be paused teaches the wrong habit
- Content: Original tasks written for the platform, never real exam items · reproducing exam questions is unlawful and teaches memorisation

### Data model (create this before Phase 1)
- tasks(id, skill, format, prompt_text, audio_path, time_limit_seconds, source) · source is always original
- attempts(id, user_id, task_id, started_at, submitted_at, response_text, response_audio_path)
- recognitions(attempt_id, word_ordinal, text, start_ms, end_ms, confidence) · the basis for speech feedback
- observations(id, attempt_id, dimension, detail, evidence_json) · dimension is fluency, pronunciation, grammar, content or discourse
- progress(user_id, dimension, week, observation_count, trend) · improvement over time, never a predicted score
- There is deliberately no column anywhere holding a predicted overall band, because a field like that gets displayed eventually.

### Phase 1 · Tasks delivered under exam conditions
Build: Deliver reading, listening, speaking and writing tasks with server-authoritative timers,
  a preparation period where the format has one, and automatic submission at the limit.
  Handle the browser closing mid-task.
Trap: A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.
Done when: A task submits automatically at the time limit even if the browser was closed, tested by closing it mid-task. The timer cannot be extended from the client, verified by attempting it. Preparation and response periods are separate where the format requires. An interrupted attempt is recorded as interrupted rather than discarded.
Do not build yet: No recognition, feedback or progress.

### Phase 2 · Hearing what was actually said
Build: Recognise speech locally with word timings and per-word confidence. Handle accented
  speech without treating an accent as an error. Record audio at a quality that makes
  recognition meaningful and store it with the attempt.
Trap: Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.
Done when: A spoken response produces words with timings and confidences. A strongly accented but correct response is not reported as mispronounced, checked against recordings from several first languages. Poor audio is detected and reported as an audio problem rather than a speaking problem. Recognition runs with networking disabled.
Do not build yet: No feedback generation yet.

### Phase 3 · Feedback that names something specific
Build: Generate observations per dimension, each citing evidence: the timestamp where fluency
  broke, the word that was unclear, the sentence with the grammar issue. Reject any
  observation without evidence. Report dimensions separately and never combine them.
Trap: Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.
Done when: Every observation cites evidence that resolves to the response, with an uncited one rejected by a validator tested directly. Dimensions are reported separately with no aggregate anywhere in the model or interface, verified by inspection. Clicking an observation plays or highlights the evidence. Feedback on a good response says what worked rather than only what did not.
Do not build yet: No progress tracking yet.

### Phase 4 · Written responses, checked honestly
Build: Rule-based checks for spelling, grammar and word count reported deterministically, plus
  a model assessing whether the response addresses the prompt and how it is structured.
  Report the two sources separately so the user knows which is which.
Trap: Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.
Done when: Spelling, grammar and word count checks are deterministic and identical across runs, verified by running an essay twice. The model's observations quote the text they describe. The two sources are visually distinct. An essay that ignores the prompt is identified as such rather than praised for fluency.
Do not build yet: No progress or export yet.

### Phase 5 · Progress, without inventing a number
Build: Track observation counts per dimension over time and show trends. State plainly in the
  interface that this is practice feedback and not a predicted exam result. Export attempts
  and observations, and delete them completely on request.
Trap: A trend line drawn on a zero to ninety axis will be read as a predicted score no matter what the caption says, because the axis is the claim. Use counts and rates on their own scales, and keep the exam's scale out of the interface entirely.
Done when: Progress shows per-dimension trends with the exam's scale appearing nowhere in the interface, verified by reading every screen. The disclaimer appears with the feedback rather than only in a footer. Export contains attempts, audio and observations. Deletion removes attempts, recordings and derived observations together.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Any predicted or estimated official score, which cannot be produced honestly and is the one thing users will ask for most.
- Real exam questions, which are copyrighted and whose use would be both unlawful and pedagogically worthless.
- The official scoring engine's integrated skill model, where one response contributes to several skill scores through a proprietary method.
- Accredited preparation or any claim of affiliation with the exam board, which would be a misrepresentation.

### README must contain
- That no predicted score is produced and why that cannot be done honestly
- That all tasks are original and no real exam questions are used
- That low recogniser confidence is reported as uncertainty and never as mispronunciation
- That every observation must cite evidence in the response or it is rejected
- That the exam's scale appears nowhere in the interface, including on charts

# Agent instructions · OpenPTE indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: TypeScript with Fastify and PostgreSQL, Local speech recognition with word timings and confidence, Rule-based checks plus a model for discourse, each reported separately, Server-authoritative task timers, Original tasks written for the platform, never real exam items. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.
- Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.
- Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.
- Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.

# Build plan · OpenPTE

A practice platform for exam-style speaking, writing, reading and listening tasks, where the honest engineering problem is producing genuinely useful feedback on a spoken or written response without pretending to be the official scoring engine.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · Tasks delivered under exam conditions

A practice task runs with the same timing pressure as the real thing.

### Steps

1. Keep the clock on the server
   The client shows a countdown. It does not own one.
   Files: `src/timer.ts`
2. Submit automatically at the limit
   Even if the browser is gone. The attempt still happened.
   Files: `src/attempts.ts`
3. Record interruptions honestly
   An interrupted attempt is data, not something to hide.
   Files: `src/attempts.ts`

### Done when

- [ ] A task submits at the limit with the browser closed
- [ ] The timer cannot be extended from the client
- [ ] Preparation and response periods are separate
- [ ] An interrupted attempt is recorded as such

### Watch out

- A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.

## Phase 2 · Hearing what was actually said

A spoken response becomes words with times and confidences.

### Steps

1. Recognise with word timings and confidence
   Both are needed. Fluency comes from timings, uncertainty from confidence.
   Files: `src/recognise.ts`
2. Never equate low confidence with error
   Test with several accents. This is the fairness requirement of the whole build.
   Files: `tests/accents.md`
3. Detect audio problems separately
   A bad microphone is not a speaking weakness and must not be reported as one.
   Files: `src/audio-quality.ts`

### Done when

- [ ] Responses produce words with timings and confidence
- [ ] Accented correct speech is not reported as mispronounced
- [ ] Poor audio is reported as an audio problem
- [ ] Recognition works offline

### Watch out

- Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.

## Phase 3 · Feedback that names something specific

Every observation points at a moment or a phrase in the response.

### Steps

1. Require evidence per observation
   A timestamp or a span. Validate it and reject the output otherwise.
   Files: `src/observations.ts`, `src/validate.test.ts`
2. Keep dimensions separate everywhere
   No aggregate field. A number that exists will eventually be shown.
   Files: `src/schema.sql`
3. Say what worked too
   Feedback that only lists failures is demoralising and less useful.
   Files: `src/observations.ts`

### Done when

- [ ] Every observation cites resolvable evidence
- [ ] An uncited observation is rejected
- [ ] No aggregate score exists in model or interface
- [ ] Good responses receive positive observations

### Watch out

- Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.

## Phase 4 · Written responses, checked honestly

An essay gets specific feedback on grammar, content and structure separately.

### Steps

1. Run deterministic checks separately
   Same essay, same findings, every time. Label them as mechanical.
   Files: `src/checks.ts`
2. Require the model to quote
   An observation about a paragraph must include the paragraph.
   Files: `src/essay.ts`
3. Distinguish the two visually
   So the user knows which finding is reliable and which is an opinion.
   Files: `src/feedback-view.ts`

### Done when

- [ ] Mechanical checks are deterministic across runs
- [ ] Model observations quote the text
- [ ] The two sources are visually distinct
- [ ] An off-prompt essay is identified as off-prompt

### Watch out

- Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.

## Not in this build

- Any predicted or estimated official score.
- Real exam questions.
- The official integrated skill scoring model.
- Accreditation or any claim of affiliation with the exam board.

## After v1, if you want it

- Spaced repetition over the dimensions a learner is weakest in
- Model responses for each task, which is what learners ask for after feedback
- A study plan built from observation trends rather than from a score

# Copy to .env and fill in. Never commit .env; this file documents it.

# Required · secret. Attempts and observations, including learners' voice recordings by reference.
DATABASE_URL=postgres://localhost:5432/practice

# Required. Voice recordings. Personal data, so retention and deletion apply to this directory too.
AUDIO_DIR=./attempts/audio

# Required. The local recogniser. Larger models give better timings and fairer confidence across accents.
ASR_MODEL=./models/ggml-medium.en.bin

# Required. How long voice recordings are kept. Shorter than the observations derived from them.
AUDIO_RETENTION_DAYS=90

# Required. Present as a constant so the answer is visible in configuration. There is no other value.
SHOW_PREDICTED_SCORE=never

# OpenPTE · product brief

## Problem

The mechanics of a PTE trainer are not hard: record audio, transcribe it, time the task, score against a rubric, keep a history. An agent can build that in a weekend using Whisper for transcription and an LLM for rubric feedback, and you will genuinely practice more because the loop is yours. What you cannot build is the part people actually pay for: a question bank that tracks what is currently showing up in the real exam, and a scoring model calibrated against Pearson's automated marker so the number you see means something. Your DIY grader will be directionally useful and numerically fictional. Good enough for drilling fluency and essay structure, not good enough to decide whether you are ready to book the test.

## Product outcome

A practice platform that runs tasks under real timing pressure and gives specific, evidenced feedback, while refusing to produce the predicted score that would be the easiest thing to sell and the least honest.

## Target user

A builder who needs a maintainable product foundation, not a one-off demo.

## Required capabilities

- Node 20 and a modern browser with mic permission
- An OpenAI-compatible API key in .env for transcription and rubric scoring, or a local whisper.cpp build
- Your own question set: prompts, images, and audio you supply as JSON
- Time to write the rubric prompts per task type, which is most of the actual work

## Explicit non-goals for v1

- Any predicted or estimated official score.
- Real exam questions.
- The official integrated skill scoring model.
- Accreditation or any claim of affiliation with the exam board.
- Scores calibrated to the real automated marker, so your numbers are vibes, not predictions
- A question bank that is maintained and rotated as the exam changes
- Full mock tests with official section timing, weighting and score report layout
- Model answers, templates and community discussion around each item
- Mobile apps, cross-device sync, and anyone to blame when the grader is wrong

## Success criteria

- No aggregate or predicted score is confirmed absent from model and interface
- The exam's scale is confirmed absent from every chart
- Accented correct speech is confirmed not reported as mispronunciation
- An uncited observation is confirmed rejected
- A task is confirmed to submit at the limit with the browser closed

# Build brief · OpenPTE

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a practice platform for an English proficiency exam, with feedback rather than fake scores to replace OpenPTE. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Rule zero
People take this exam for visas and university admission. A practice score they believe is
a prediction can lead someone to book a test they are not ready for, at real cost, or to
delay one they would have passed.
You cannot reproduce the official scoring. The real exam is machine scored on a scale from
10 to 90 per skill, using integrated tasks where a single spoken or written response feeds
into several communicative skill scores at once, plus enabling skills such as oral fluency,
pronunciation, grammar, spelling, vocabulary and written discourse. That engine is
proprietary, trained on data you do not have, and periodically revised.
So do not output a predicted overall band. Give component feedback that helps someone
improve: where the fluency broke down, which words were mispronounced, whether the essay
addressed the prompt. State plainly and in the interface, not only in a footer, that these
are practice observations and not a predicted result.
Also: do not reproduce real exam questions. They are copyrighted and using them is both
unlawful and a disservice, since memorised items do not build the skill.

### Stack (fixed, do not substitute)
- Runtime: TypeScript with Fastify and PostgreSQL · the work is task delivery, timing and storing attempts
- Speech: Local speech recognition with word timings and confidence · fluency and pronunciation feedback both derive from timings and confidence
- Writing: Rule-based checks plus a model for discourse, each reported separately · a grammar error and a weak argument are different feedback and should not be merged into a number
- Timing: Server-authoritative task timers · an exam practice tool where the timer can be paused teaches the wrong habit
- Content: Original tasks written for the platform, never real exam items · reproducing exam questions is unlawful and teaches memorisation

### Data model (create this before Phase 1)
- tasks(id, skill, format, prompt_text, audio_path, time_limit_seconds, source) · source is always original
- attempts(id, user_id, task_id, started_at, submitted_at, response_text, response_audio_path)
- recognitions(attempt_id, word_ordinal, text, start_ms, end_ms, confidence) · the basis for speech feedback
- observations(id, attempt_id, dimension, detail, evidence_json) · dimension is fluency, pronunciation, grammar, content or discourse
- progress(user_id, dimension, week, observation_count, trend) · improvement over time, never a predicted score
- There is deliberately no column anywhere holding a predicted overall band, because a field like that gets displayed eventually.

### Phase 1 · Tasks delivered under exam conditions
Build: Deliver reading, listening, speaking and writing tasks with server-authoritative timers,
  a preparation period where the format has one, and automatic submission at the limit.
  Handle the browser closing mid-task.
Trap: A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.
Done when: A task submits automatically at the time limit even if the browser was closed, tested by closing it mid-task. The timer cannot be extended from the client, verified by attempting it. Preparation and response periods are separate where the format requires. An interrupted attempt is recorded as interrupted rather than discarded.
Do not build yet: No recognition, feedback or progress.

### Phase 2 · Hearing what was actually said
Build: Recognise speech locally with word timings and per-word confidence. Handle accented
  speech without treating an accent as an error. Record audio at a quality that makes
  recognition meaningful and store it with the attempt.
Trap: Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.
Done when: A spoken response produces words with timings and confidences. A strongly accented but correct response is not reported as mispronounced, checked against recordings from several first languages. Poor audio is detected and reported as an audio problem rather than a speaking problem. Recognition runs with networking disabled.
Do not build yet: No feedback generation yet.

### Phase 3 · Feedback that names something specific
Build: Generate observations per dimension, each citing evidence: the timestamp where fluency
  broke, the word that was unclear, the sentence with the grammar issue. Reject any
  observation without evidence. Report dimensions separately and never combine them.
Trap: Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.
Done when: Every observation cites evidence that resolves to the response, with an uncited one rejected by a validator tested directly. Dimensions are reported separately with no aggregate anywhere in the model or interface, verified by inspection. Clicking an observation plays or highlights the evidence. Feedback on a good response says what worked rather than only what did not.
Do not build yet: No progress tracking yet.

### Phase 4 · Written responses, checked honestly
Build: Rule-based checks for spelling, grammar and word count reported deterministically, plus
  a model assessing whether the response addresses the prompt and how it is structured.
  Report the two sources separately so the user knows which is which.
Trap: Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.
Done when: Spelling, grammar and word count checks are deterministic and identical across runs, verified by running an essay twice. The model's observations quote the text they describe. The two sources are visually distinct. An essay that ignores the prompt is identified as such rather than praised for fluency.
Do not build yet: No progress or export yet.

### Phase 5 · Progress, without inventing a number
Build: Track observation counts per dimension over time and show trends. State plainly in the
  interface that this is practice feedback and not a predicted exam result. Export attempts
  and observations, and delete them completely on request.
Trap: A trend line drawn on a zero to ninety axis will be read as a predicted score no matter what the caption says, because the axis is the claim. Use counts and rates on their own scales, and keep the exam's scale out of the interface entirely.
Done when: Progress shows per-dimension trends with the exam's scale appearing nowhere in the interface, verified by reading every screen. The disclaimer appears with the feedback rather than only in a footer. Export contains attempts, audio and observations. Deletion removes attempts, recordings and derived observations together.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Any predicted or estimated official score, which cannot be produced honestly and is the one thing users will ask for most.
- Real exam questions, which are copyrighted and whose use would be both unlawful and pedagogically worthless.
- The official scoring engine's integrated skill model, where one response contributes to several skill scores through a proprietary method.
- Accredited preparation or any claim of affiliation with the exam board, which would be a misrepresentation.

### README must contain
- That no predicted score is produced and why that cannot be done honestly
- That all tasks are original and no real exam questions are used
- That low recogniser confidence is reported as uncertainty and never as mispronunciation
- That every observation must cite evidence in the response or it is rejected
- That the exam's scale appears nowhere in the interface, including on charts

# Architecture · OpenPTE

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | TypeScript with Fastify and PostgreSQL | the work is task delivery, timing and storing attempts |
| Speech | Local speech recognition with word timings and confidence | fluency and pronunciation feedback both derive from timings and confidence |
| Writing | Rule-based checks plus a model for discourse, each reported separately | a grammar error and a weak argument are different feedback and should not be merged into a number |
| Timing | Server-authoritative task timers | an exam practice tool where the timer can be paused teaches the wrong habit |
| Content | Original tasks written for the platform, never real exam items | reproducing exam questions is unlawful and teaches memorisation |

## Modules

Each module has one owner concern and a documented way to replace it.

| Module | Owns | How to replace it |
| --- | --- | --- |
| Delivery | Server-authoritative timing and submission. | Any implementation, provided the clock is never the client's. |
| Recognition | Word timings and confidence. | Any recogniser, provided confidence is never presented as correctness. |
| Observations | Evidenced, per-dimension feedback. | Any generator, provided uncited observations are rejected and no aggregate exists. |
| Checks | Deterministic mechanical findings. | Any checker, provided its findings stay distinguishable from a model's opinions. |
| Progress | Trends that are not scores. | Any presentation that keeps the exam's scale out of the interface. |

## Configuration

Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists.

- `DATABASE_URL` · required, secret · Attempts and observations, including learners' voice recordings by reference.
- `AUDIO_DIR` · required · Voice recordings. Personal data, so retention and deletion apply to this directory too.
- `ASR_MODEL` · required · The local recogniser. Larger models give better timings and fairer confidence across accents.
- `AUDIO_RETENTION_DAYS` · required · How long voice recordings are kept. Shorter than the observations derived from them.
- `SHOW_PREDICTED_SCORE` · required · Present as a constant so the answer is visible in configuration. There is no other value.

## Production baseline

- Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives.
- Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised.
- Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts.
- Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists.
- Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path.

## Decision records

For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.

# Agent instructions · OpenPTE product build

- Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: TypeScript with Fastify and PostgreSQL, Local speech recognition with word timings and confidence, Rule-based checks plus a model for discourse, each reported separately, Server-authoritative task timers, Original tasks written for the platform, never real exam items.
- Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit.
- Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present.
- Never invent cryptography or silently weaken a requirement to make a check pass.
- Put every external service behind an interface with a deterministic fake for tests.
- Add migrations and rollback or recovery notes for every persistent data change.
- Log useful operational context without credentials, tokens, passwords or personal data.
- Update documentation and run every check before completing a milestone.

## Known traps

- A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.
- Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.
- Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.
- Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.
- A trend line drawn on a zero to ninety axis will be read as a predicted score no matter what the caption says, because the axis is the claim. Use counts and rates on their own scales, and keep the exam's scale out of the interface entirely.

# Delivery milestones · OpenPTE

Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer.

## M1 · Tasks delivered under exam conditions

A practice task runs with the same timing pressure as the real thing.

### Steps

1. Keep the clock on the server
   The client shows a countdown. It does not own one.
   Files: `src/timer.ts`
2. Submit automatically at the limit
   Even if the browser is gone. The attempt still happened.
   Files: `src/attempts.ts`
3. Record interruptions honestly
   An interrupted attempt is data, not something to hide.
   Files: `src/attempts.ts`

### Done when

- [ ] A task submits at the limit with the browser closed
- [ ] The timer cannot be extended from the client
- [ ] Preparation and response periods are separate
- [ ] An interrupted attempt is recorded as such

### Watch out

- A timer running in the browser can be paused by opening the developer tools or simply losing focus, which turns timed practice into untimed practice and removes the only thing that makes it resemble the exam. Keep the clock on the server and treat the client display as a hint.

## M2 · Hearing what was actually said

A spoken response becomes words with times and confidences.

### Steps

1. Recognise with word timings and confidence
   Both are needed. Fluency comes from timings, uncertainty from confidence.
   Files: `src/recognise.ts`
2. Never equate low confidence with error
   Test with several accents. This is the fairness requirement of the whole build.
   Files: `tests/accents.md`
3. Detect audio problems separately
   A bad microphone is not a speaking weakness and must not be reported as one.
   Files: `src/audio-quality.ts`

### Done when

- [ ] Responses produce words with timings and confidence
- [ ] Accented correct speech is not reported as mispronounced
- [ ] Poor audio is reported as an audio problem
- [ ] Recognition works offline

### Watch out

- Treating low recogniser confidence as mispronunciation punishes accents the recogniser was simply not trained on, which is both wrong and demoralising to exactly the people using the tool. Report low confidence as uncertain rather than incorrect, and never present the recogniser's difficulty as the speaker's failure.

## M3 · Feedback that names something specific

Every observation points at a moment or a phrase in the response.

### Steps

1. Require evidence per observation
   A timestamp or a span. Validate it and reject the output otherwise.
   Files: `src/observations.ts`, `src/validate.test.ts`
2. Keep dimensions separate everywhere
   No aggregate field. A number that exists will eventually be shown.
   Files: `src/schema.sql`
3. Say what worked too
   Feedback that only lists failures is demoralising and less useful.
   Files: `src/observations.ts`

### Done when

- [ ] Every observation cites resolvable evidence
- [ ] An uncited observation is rejected
- [ ] No aggregate score exists in model or interface
- [ ] Good responses receive positive observations

### Watch out

- Combining dimensions into a single figure recreates exactly the predicted score this build refuses to produce, and users will read it as one. Keep the dimensions apart in the model and in the interface, and make sure no aggregate exists anywhere that could be displayed as an overall result.

## M4 · Written responses, checked honestly

An essay gets specific feedback on grammar, content and structure separately.

### Steps

1. Run deterministic checks separately
   Same essay, same findings, every time. Label them as mechanical.
   Files: `src/checks.ts`
2. Require the model to quote
   An observation about a paragraph must include the paragraph.
   Files: `src/essay.ts`
3. Distinguish the two visually
   So the user knows which finding is reliable and which is an opinion.
   Files: `src/feedback-view.ts`

### Done when

- [ ] Mechanical checks are deterministic across runs
- [ ] Model observations quote the text
- [ ] The two sources are visually distinct
- [ ] An off-prompt essay is identified as off-prompt

### Watch out

- Presenting a model's opinion about an essay's quality with the same authority as a spelling checker's finding misleads the user about how reliable each is, and a model asked to grade will confidently invent criteria. Label the deterministic checks and the model's observations differently, and require the model to quote the text it is describing.

## M5 · Progress, without inventing a number (production only)

A learner sees improvement over weeks without ever being shown a predicted result.

### Steps

1. Trend counts and rates, not scores
   And never on the exam's scale. The axis is the claim.
   Files: `src/progress.ts`
2. Put the disclaimer with the feedback
   Where it is read, not in a footer where it is not.
   Files: `src/feedback-view.ts`
3. Export and delete completely
   Including voice recordings, which are biometric-adjacent and personal.
   Files: `src/export.ts`, `src/deletion.ts`

### Done when

- [ ] The exam's scale appears nowhere in the interface
- [ ] The disclaimer appears with the feedback
- [ ] Export contains attempts, audio and observations
- [ ] Deletion removes recordings and observations together

### Watch out

- A trend line drawn on a zero to ninety axis will be read as a predicted score no matter what the caption says, because the axis is the claim. Use counts and rates on their own scales, and keep the exam's scale out of the interface entirely.

# Operations · OpenPTE

## Backup

Voice recordings are personal data belonging to learners. Back up with encryption, apply the retention period to backups, and consider whether you need to keep audio at all once observations exist.

## Restore

Restore and confirm retention resumes, since restoring expired voice recordings recreates personal data that should have been deleted.

Do a restore drill before the first real user, and write the date here when it passes.

## Monitoring

Watch the rate at which low-confidence recognition is being produced per speaker, since a speaker whose every response comes back uncertain is being failed by the recogniser rather than by their speaking.

## Incident checklist

Any aggregate score appearing in the interface is the incident, whether added deliberately or derived by a chart axis. The schema has no such field, and that absence is the control worth protecting.

1. Contain the issue without destroying evidence or user data.
2. Record the timeline and affected scope.
3. Rotate exposed secrets and revoke compromised sessions or credentials.
4. Restore from a verified backup when needed.
5. Document the root cause, the remediation and the regression test.

## Release gate

- [ ] No aggregate or predicted score is confirmed absent from model and interface
- [ ] The exam's scale is confirmed absent from every chart
- [ ] Accented correct speech is confirmed not reported as mispronunciation
- [ ] An uncited observation is confirmed rejected
- [ ] A task is confirmed to submit at the limit with the browser closed

## Launch constraint

Do not market omitted OpenPTE capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.

# Copy to .env and fill in. Never commit .env; this file documents it.

# Required · secret. Attempts and observations, including learners' voice recordings by reference.
DATABASE_URL=postgres://localhost:5432/practice

# Required. Voice recordings. Personal data, so retention and deletion apply to this directory too.
AUDIO_DIR=./attempts/audio

# Required. The local recogniser. Larger models give better timings and fairer confidence across accents.
ASR_MODEL=./models/ggml-medium.en.bin

# Required. How long voice recordings are kept. Shorter than the observations derived from them.
AUDIO_RETENTION_DAYS=90

# Required. Present as a constant so the answer is visible in configuration. There is no other value.
SHOW_PREDICTED_SCORE=never

$ choose a build depth, inspect the files, then open the complete pack in your agent