Decoding video in parallel, in a tab
Video2Any has one constraint that decides everything else: your video never leaves your machine. There is no upload, so there is no server doing the work, so the decoding, the comparing and the rendering all happen in a browser tab that also has to stay responsive.
Here is what that actually takes, and what the rest of the site is built on.
You have to write the demuxer
WebCodecs gives you a VideoDecoder that turns encoded chunks into frames. It does not give you the chunks. The browser has an MP4 parser — it is how <video> works — and no API hands it to you, so you parse the container yourself: walk the boxes, find moov, read the sample table, and feed VideoDecoder one EncodedVideoChunk at a time.
Doing that has a nice side effect. Once you are reading the container yourself you can tell people why a file will not open before spending five minutes failing: a ProRes file is an editing format no browser decodes, and you find that out by reading its moov, which for ProRes is often at the end of the file rather than the start. Our probe scans both ends for that reason.
Two phases, because memory
The obvious approach is to decode every frame and compare. A 92-minute recording at 25fps is 138,000 frames; at 1080p that is not a thing a tab can hold.
So it runs in two passes. The sweep decodes to a grid of sample points and keeps each sample at 160×90 — small enough that a whole video is a few tens of megabytes of pixels, big enough to tell one slide from another. Only after the detector has chosen does the capture pass go back for full-resolution frames, and only for the ones it chose. A ceiling of 900 samples caps the first pass, which has an odd consequence worth knowing: a 29-minute video and a 92-minute video cost about the same, because the cost is per sample, not per minute.
One worker per chunk, each with its own decoder
The video is split by time and each chunk gets a worker with its own VideoDecoder. The pool is min(4, cores / 2): half the cores, because the tab still has a UI to run, and capped at four for reasons the next section is about.
Each chunk sweeps from slightly before its own range, so the detector has a previous frame to judge its first one against, and then drops anything outside the range it owns. Slides stream out as each capture lands rather than at the end.
What splitting costs you
Two things, and they are the interesting part. Each chunk calibrates its detection threshold on its own slice, so the same video split four ways and eight ways produces different decks — the output depends on the number of cores in the machine. And each chunk removed duplicate shots only within itself, so a view a meeting keeps cutting back to survived once per chunk it appeared in: three duplicate slides out of nine on a 29-minute recording.
The duplicates are fixed — the check moved to the main thread and runs as each slide arrives. The calibration is not. We built the global version, and it made the deck worse: on that same meeting, where ffmpeg counts seven real cuts, per-chunk calibration gives six and global gives four. So the pool stays at four, because more chunks is about 25% faster and still changes the answer.
Nothing can be shown until the video has been read
The threshold comes from the distribution of frame-to-frame differences across the whole video, so the first slide cannot be chosen until the last frame has been sampled. On a 92-minute recording the sweep takes 52 seconds and the first slide appears at 50 of them.
That is not a performance problem and no amount of parallelism fixes it. It is a feedback problem, and it was solved by writing "Reading the video · 45:46 of 92:08" where a bare percentage used to be.
The rest of the stack
Cloudflare Workers with Hono for the API, D1 for accounts and metadata, R2 for the few things that are stored. React 19 with TanStack Router, Tailwind v4, better-auth. Eight locales, each with its own URL prefix and its own router basepath.
The site is a prerendered SPA rather than server-rendered: every public page is a static HTML snapshot built at deploy time, which is why the marketing pages are cacheable and fast and why the converter itself works with no server at all. There is exactly one place we break that. The /app pages are static shells, so the editor used to boot knowing nothing about your account and had to ask — and starting work before the answer came back is how a paying subscriber once got the free tier’s thirty-minute limit. The worker already holds the session when it serves those pages, so it writes the plan into the document and the first render knows. Those responses are private and never cached; every other page keeps its public caching.
Why bother
A server would make most of this go away. It would also mean every lecture, every internal meeting and every medical recording someone converts passes through our machines, and the whole point of the product is that they do not.
The constraint is the feature. Everything above is what it costs.