A 130 MB WASM model on Cloudflare Pages

Machine-learning models run in browsers now. Not toy ones — a source-separation model that splits a song into vocals and instrumental, and a text-to-speech engine with voice cloning, both compiled to WebAssembly and executed on the visitor's own processor. No upload, no inference server, no GPU bill.

The inference part turned out to be the easy half. Getting 130 MB of model onto a static host, and getting the browser to agree to run it, produced four failures. Three give you no usable error message. One gives you the opposite of the truth.

All the numbers below are measured on the deployed artefacts, not estimated.

The setup

A pthreads WebAssembly build needs SharedArrayBuffer. Browsers only expose that to a cross-origin isolated page, which means three response headers:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp
Cross-Origin-Resource-Policy: cross-origin

The third one is the interesting one, and it is where most write-ups get it wrong. Under COEP: require-corp, emscripten's pthread workers and the nested worker the engine spawns fetch their own subresources from what the browser treats as an opaque origin. Cross-Origin-Resource-Policy: same-origin blocks those. It has to be cross-origin, which is a looser value than instinct suggests, scoped to the one path prefix that needs it rather than applied site-wide, because cross-origin also lets any site embed those resources.

On Cloudflare Pages you set that in a _headers file. Which is where the first failure lives.

Failure 1: Pages merges same-name headers, and COOP is a single token

To be safe, I declared both the directory and everything under it:

/ai-text-to-speech/
  Cross-Origin-Opener-Policy: same-origin
  Cross-Origin-Embedder-Policy: require-corp

/ai-text-to-speech/*
  Cross-Origin-Opener-Policy: same-origin
  Cross-Origin-Embedder-Policy: require-corp

Both rules match the document. Pages does not let the more specific rule win — it merges same-named headers into one comma-joined value:

cross-origin-opener-policy: same-origin, same-origin
cross-origin-embedder-policy: require-corp, require-corp

COOP and COEP are single-token structured headers. That value fails to parse, and the browser does the worst possible thing with a header it cannot parse: it falls back to unsafe-none silently. No console warning about the header. What you get instead, from somewhere else entirely, is:

DataCloneError: SharedArrayBuffer transfer requires self.crossOriginIsolated

Nothing in that message points at a response header, let alone at a duplicated one.

Two things to take away:

• * matches the empty string. /prefix/* already matches /prefix/ itself. The extra exact-path rule was not belt-and-braces; it was the bug.
• Merging is harmless for Cache-Control, which is a directive list. It is fatal for any header whose grammar is one token.

The duplication is also how I proved the first point: seeing every header appear exactly twice is only possible if both patterns matched.

My deploy script now fails the build if any file is matched by two rules that set the same header. That check costs twenty lines and would have saved a day.

Failure 2: the HTTP cache stores response headers, not just bodies

With the _headers file fixed and deployed, it still failed. Same page, same error class, but now from a different place: the worker would not start at all. The ErrorEvent had an empty message, empty filename, and line number 0. It fired about 4 ms after the worker was constructed.

The cause: the browser's HTTP cache held a copy of tts-worker.js fetched before the isolation headers existed. Cached responses carry the headers they were stored with, and that entry had a max-age of four hours. A cross-origin isolated page refuses to run a worker whose own script lacks COEP: require-corp — and it refuses with no detail whatsoever.

So the server was correct, the network tab looked correct, and the page was loading a four-hour-old response that was not.

The diagnostic that settled it, which is worth keeping around:

const headers = async (url, cache) =>
  [...(await fetch(url, { cache })).headers]
    .filter(([k]) => k.startsWith('cross-origin'));

console.log('fresh ', await headers(url, 'no-store'));
console.log('cached', await headers(url, 'force-cache'));

If those two disagree, you are looking at a poisoned cache entry, not a server problem. In my case fresh had all three headers and cached had none.

Two fixes, because either alone is insufficient:

• Cache-Control: no-cache on the front-end sources. They are 13–31 KB; revalidating them costs nothing next to a four-hour outage that no server-side change can end.
• A version token on the worker URL (tts-worker.js?v=…) so an already-poisoned entry can be stepped around. Bump it whenever the isolation headers change.

There is a corollary that cost me more time than the bug itself: my local dev server treated any URL carrying ?v= as immutable. So the moment I added a version token, local stopped picking up worker edits — and, worse, local and production now behaved differently in exactly the area I was debugging. Divergence between local and production caching is what made this hard, not the caching itself.

Failure 3: a 25 MiB per-file wall

Cloudflare Pages rejects any single file over 25 MiB. The largest model file is 82.0 MiB. Another is 37.9 MiB.

Splitting is obvious; the part worth writing down is where the source of truth goes. The slicing tool records each part's path, byte count and SHA-256 in a manifest, and the manifest — not the original file — becomes what the client trusts:

models/flow_lm_main_delta_attn_flow_int8.onnx   82.0 MiB → 4 parts
models/mimi_encoder.onnx                        37.9 MiB → 2 parts

The reassembly in the worker is unremarkable except for two assertions that have both fired during development:

async function fetchAsset(entry, onDelta) {
  if (!entry.parts?.length) return await fetchBytes(`${BASE}${entry.path}`, onDelta);
  const out = new Uint8Array(entry.bytes);
  let off = 0;
  for (const p of entry.parts) {
    const buf = await fetchBytes(`${BASE}${p.path}`, onDelta);
    if (off + buf.length > out.length) throw new Error(`${entry.path}: parts exceed the declared size`);
    out.set(buf, off);
    off += buf.length;
  }
  if (off !== entry.bytes) throw new Error(`${entry.path}: assembled ${off} of ${entry.bytes} bytes`);
  return out;
}

A truncated model does not throw when you load it. It produces plausible-looking garbage audio. Assert the byte count.

Failure 4: the host will not compress the bytes that matter, and HEAD lies about it

Cloudflare compresses responses by content type. .wasm, .js, .css and .html are handled automatically. .onnx, model slices and voice embeddings are application/octet-stream, and those go out uncompressed.

That is most of the payload. So the bytes get pre-compressed at pack time and stored under their original names, with Content-Encoding: br declared per file in the _headers file.

Per file, not by wildcard. The compressor skips anything it cannot meaningfully shrink, and a raw file served with Content-Encoding: br is simply corrupt. A /models/* rule would claim files that were never compressed. The pack tool generates one rule per file it actually compressed, from the result rather than the intention.

Now the part that wasted an afternoon. To check whether it worked, the obvious command is:

curl -sI https://example.net/models/big.onnx | grep -i content-encoding

It reports content-encoding: br every time. It is wrong. Cloudflare answers a HEAD request with that header while the real GET for a large application/octet-stream is not compressed at all. Reading response headers gives you the exact opposite of the truth.

The only reliable check counts bytes off a GET:

U=https://example.net/models/big.onnx
echo "wire $(curl -s -H 'Accept-Encoding: br' "$U" | wc -c)  decoded $(curl -s --compressed "$U" | wc -c)"

Pipe to wc -c, do not use %{size_download} — that counts wire bytes, not decoded bytes, so it silently agrees with whichever answer you were hoping for. --compressed needs a curl built with brotli (curl -V | grep -i brotli).

What the compression is actually worth

Measured on the deployed slices:

The 40.7% is more interesting than it looks. 40% of the bytes in that slice are 0x00, and its order-0 entropy is 4.19 bit/byte — meaning no coder that looks only at symbol frequencies can get below 52.4%. Brotli reached 40.7%, comfortably under that floor, which is a concrete demonstration that its context modelling is finding structure across bytes and not merely counting them. Quantised weight tensors are full of such structure: runs of zeros, repeated exponent patterns, near-duplicated blocks.

If you are shipping quantised models over the wire and have not measured this, it is likely the cheapest win available to you.

One more, without a section of its own

For a while the worker was reporting its failure and I could not see it. The messages went through console.debug, which Chrome hides at its default console level. Emscripten's stderr and every routing failure were being logged into a level nobody looks at. Anything a diagnostic depends on belongs at warn or above.

The checklist

If you are putting a pthreads WASM model on a static host:

1. Never let two _headers rules set the same header on one path. Assert it in your build. * matches the empty string.
2. Set CORP: cross-origin on the isolated prefix, and only there.
3. Serve front-end sources no-cache, and version your worker URLs. A cached response carries its old headers with it.
4. Keep local and production caching identical. Divergence is what turns a one-hour bug into a one-day bug.
5. Assert reassembled byte counts. A truncated model produces output, not an exception.
6. Declare Content-Encoding per file, generated from what was actually compressed.
7. Verify compression by counting bytes off a GET. HEAD will lie to you.
8. Log diagnostics at warn or above.

The tool this came out of is Audiofilter — browser-side vocal separation, noise reduction and text-to-speech, none of which uploads your file. If you want the DSP side rather than the delivery side, why phase cancellation takes the bass with the vocal covers that.