Build ekto

KINDAreplaces $29.99/mosaves $359.88/yrback to the verdict

0%0 of 17 items done

Saved on this device only. Tick prerequisites first, then work the phases in order · do not start one until the checks above it pass.

A speech translation pipeline where every design decision is a latency trade: recognition wants complete sentences, translation wants complete clauses, speech synthesis wants complete phrases, and a conversation wants none of them to take longer than a natural pause.

estimated effort multi-daythe files for this build are in the project pack

RecognitionStreaming-shaped recognition with voice activity segmentationTranslationA model run per stable segment, never per partial wordSynthesisSpeech generated per translated segment with a small queueLatencyA stated budget per stage, measured continuouslyRuntimeLocal models where they fit, with the trade stated

Before step 1

Everything below is assumed from the first step. Tick each one when you actually have it, not when you plan to.

installfree
Why Orchestrates recognition, translation and synthesis.
Get it python.org or your package manager, then a virtual environment.
Verify python3 --version prints 3.12 or higher
installfree locally, needs machine time
Why Segmentation and latency measurement both depend on timings, and quality differs sharply between languages.
Get it A local engine with models for both languages, checked for whether it emits word timings.
Verify Both languages recognise with word timings
installfree locally; per-character if hosted
Why Quality varies enormously by pair, and a pair that translates poorly makes every other decision in the build irrelevant.
Get it A local or hosted model for your specific pair, evaluated on real conversational speech rather than written text.
Verify Twenty real conversational sentences translate acceptably in both directions
installfree for open models
Why Output speech is half the experience and synthesis latency is a large part of the total budget.
Get it A local engine with voices for both languages, measured for time to first audio.
Verify A sentence synthesises in both languages within your latency budget
have readyfree, and requires asking people
Why Every parameter here is tuned against real speech with interruptions, overlaps and people who do not pause, and none of that appears in read-aloud test sentences.
Get it Record a genuine conversation with both speakers' consent, including a stretch where they talk over each other.
Verify A recording exists with natural pauses, overlap and at least one long unbroken utterance

Data model

Create these before the first phase that stores anything. Changing a table later is the expensive kind of change.

- conversations(id, language_a, language_b, started_at, ended_at, disclosed_at)
- segments(id, conversation_id, speaker_side, start_ms, end_ms, source_text, stability, superseded_by)
- translations(segment_id, target_language, text, model, confidence, at)
- utterances(translation_id, audio_path, played_at, duration_ms)
- latencies(segment_id, stage, ms) · stage is recognition, translation, synthesis or total
- corrections(segment_id, before, after, at) · when recognition revised after translation had already spoken
- A segment records whether it was superseded, because a translation of a revised segment is a correction the listener already heard the wrong version of.

Environment variables

These go in a .env file the app reads at startup. The pack's .env.example is this table as a file · copy it, never commit the filled-in version.

Variable	Needed	Example	Where the value comes from
`LANGUAGE_A`	required	`en`	One side of the conversation. Segmentation is tuned per language.
`LANGUAGE_B`	required	`pt`	The other side. Pause patterns differ, so tuning is per side.
`MAX_SEGMENT_MS`	required	`6000`	A speaker who does not pause must still be translated.
`PAUSE_THRESHOLD_MS`	required	`500`	The central trade. Shorter is faster and produces fragments.
`SYNTHESIS_QUEUE_MAX`	required	`2`	Bounded. Growing lag is worse than dropping a segment.
`LATENCY_BUDGET_MS`	required	`2500`	Total speech to translated audio. Beyond this a conversation stops working.

The build, in order

1
Deciding where a sentence ends
Speech is cut into segments that are complete enough to translate.
1. Both. Activity alone cuts mid-clause.
  Files src/segment.py
2. A speaker who does not pause must still be translated.
  Files src/segment.py
3. The trade is the project. Measure it rather than guessing.
  Files src/latency.py
done when · tick each as it passes
watch out
- Segmenting on short pauses gives low latency and fragments that translate into nonsense, because most languages need a complete clause before the grammar is determined. Segmenting on long pauses translates well and lags so far behind that the conversation becomes unusable. This trade is the whole project, so measure both sides of it rather than picking a number.
next phase
2
Translating complete thoughts
Each segment translates correctly and the pipeline knows when it is unsure.
1. A partial clause translates into confident nonsense.
  Files src/translate.py
2. Equal fluency for a guess and a certainty is the dangerous default.
  Files src/confidence.py
3. A mistranslated number in a conversation is a real problem.
  Files src/protect.pytests/test_protect.py
done when · tick each as it passes
watch out
- Speaking a low-confidence translation with the same fluency as a high-confidence one means the listener has no way to tell a good translation from a guess, and in a conversation that matters the guess is the one that causes harm. Signal uncertainty, either by saying so or by asking the speaker to repeat.
next phase
3
Speaking without falling behind
Translated speech plays promptly and the queue never grows unbounded.
1. Unbounded means a minute behind during a fast exchange.
  Files src/synthesise.py
2. Not by accumulating. Being far behind is worse than skipping.
  Files src/queue.py
3. It is the number the whole build is judged on.
  Files src/latency.py
done when · tick each as it passes
watch out
- A synthesis queue with no bound grows during a fast exchange until the translation is a minute behind, at which point it is worse than useless because the listener is hearing an answer to a question two turns ago. Cap the queue, and when it is exceeded, drop or summarise rather than lengthening the lag.
next phase
4
Two people, two directions
Both sides are translated without the tool talking over itself.
1. Independently segmented. Languages pause differently.
  Files src/duplex.py
2. Gate input during playback or identify your own output. Test it deliberately.
  Files src/feedback.pytests/test_loop.py
3. People talk over each other. It is not an error case.
  Files src/duplex.py
done when · tick each as it passes
watch out
- Synthesised translation picked up by the microphone gets recognised, translated back and spoken again, producing a loop that fills the channel within seconds. Gate the input during playback, or identify your own output, and test it deliberately rather than discovering it in a real conversation.
5
Corrections, transcripts and consentproduct builder
Mistakes are correctable, the record exists if wanted, and the other party knows.
1. The listener was told something wrong. Fixing the log does not fix that.
  Files src/corrections.py
2. A transcript is a recording of another person's speech.
  Files src/transcript.py
3. A conversation is not an archive unless someone chose that.
  Files src/retention.py
done when · tick each as it passes
watch out
- Silently correcting a segment whose wrong translation has already been spoken leaves the listener holding a false statement they were never told was wrong, and in a real conversation that is the failure with consequences. Speak the correction, or at minimum show it, rather than only fixing the transcript.

what this build does not replace

Commercial translation quality across many language pairs.
Preserving the speaker's voice in translation.
Simultaneous interpretation quality.
Use where a mistranslation carries real consequence.
Long session reliability: memory growth, drifting segmentation and dropped turns after the first 20 minutes
Clean sentence by sentence pacing and turn detection, which is most of the perceived quality
A mobile app, so no translating anything while standing up
Offline or low-bandwidth behavior tuned for actual travel
Latency budgets someone else already fought for: streaming partial results instead of waiting for a full segment

after v1, if you want it

A shared screen view showing both transcripts, which helps far more than it sounds
Per-domain term lists, since the vocabulary that matters is usually specific
Latency tuning presets, so a slow conversation and a fast one use different settings

Need the files? The project pack on the verdict page hands your agent the whole brief · more voice ai.