Voice Studio: a voice for every character, and nobody had to record it

Every character on our platform speaks, and until this week every voice started the same way: either somebody recorded a clip of themselves, or the character borrowed one of a few dozen built-in voices. The first asks a lot of a user who only wants to invent a grumpy old sea captain. The second means a thousand characters end up sharing the same handful of voices. Voice Studio removes both. You describe the voice in a sentence, listen to a few takes, keep one, and it becomes your own. Nobody records anything.

Design once, clone forever

The central decision was to keep designing a voice separate from speaking with it. Designing produces exactly one artifact: a short reference clip (5–10 seconds) of the new voice saying one line, plus that line. That pair is precisely what our cloning path already consumes. So a designed voice is stored as an ordinary cloned voice, and from then on it goes through the same speech engine, streaming, realtime calls and billing as a voice somebody recorded.

This keeps the hot path untouched. The voice-design model is large and slow compared to our speech engine, and it is only needed once per character, not once per line. Putting it in front of every sentence would have made every chat reply slower to buy nothing.

description ──▶ avionics ──▶ engine-server ──NATS──▶ GPU worker
                 (auth,         (pick a worker            voice-design runtime
                  credits)       that has the              loads on demand,
                                 runtime selected)         designs N clips in 1 batch
                                                           │
   clips ◀── avionics ◀── engine-server ◀── object store ◀─┘ (1 WAV per description)

kept clip ──▶ POST /v1/audio/voices  ──▶  an ordinary cloned voice
                                          (speech, streaming, calls: unchanged)

Three takes, one pass

Our first prototype designed the same description twice to offer a choice. Both takes sounded like the same person, which makes the choice pointless. Sampling noise varies delivery, not identity. What does vary identity is the description. So the app first rewrites what the user typed into three clearly different readings of it: same character, different pitch, texture, age within the stated range, pace and attitude. It also writes one in-character line for them to speak.

All three go to the GPU as a single batch. On an RTX 5090 a batch of three ~6-second clips took 8.2 seconds on average, against 6.1 seconds for one clip, so the extra takes cost about two seconds of waiting instead of twelve:

A runtime that does not squat on the GPU

Workers on our network are people's own machines, often a single 16 GB card already holding a chat model, an embedding model and the speech engine. The voice-design model adds about 4.2 GB of VRAM (4.4 GB peak) and loads in under 8 seconds. Paying that permanently for something a user does once per character would be a bad trade, so the runtime loads on the first request and unloads itself after ten idle minutes:

// The design runtime is needed once per character, not once per line.
// It loads on the first job and gives its ~4 GB back after 10 idle minutes.
frame, err := sc.Call(ctx, requestID, "design", params)
v.idle = time.AfterFunc(10*time.Minute, func() {
    v.log.Info("voice design: idle, unloading")
    v.Stop()
})

On the worker it is an optional runtime with its own switch, not a speech engine. That distinction mattered in practice: the desktop app keeps exactly one speech engine selected at a time, and an early build that filed voice design under the speech slots would have switched off the worker's actual speech engine the moment you enabled it.

Moving audio over a message bus

Jobs reach workers over NATS, and a NATS reply tops out at 1 MB. Three WAV clips don't fit. The worker writes each clip into a short-lived object store bucket (15-minute TTL) under a key bound to the request id and the clip's position, and replies with references. The server only reads a key that matches its own request, so a worker can never make it fetch somebody else's object. It deletes each clip once it has read it. This is the same pattern our audio-watermark runtime uses for watermarked songs.

Keeping “no recording” honest

Cloning a recorded voice asks for consent first: whose voice it is, and what it will be used for. A designed voice is nobody's, so saving one skips that step. That opens an obvious hole: a save route that skips consent must not accept any audio a client sends, or it becomes a way to clone a real person without asking. So every designed clip carries a server-issued token, an HMAC over the user, the language, the line and a hash of the clip itself, valid for an hour. Save checks it and accepts nothing else:

// A designed voice belongs to nobody, so saving it skips the consent a
// recording needs. The save route therefore only accepts clips it issued:
mac := hmac.New(sha256.New, secret)
for _, part := range []string{issuedAt, userID, lang, line, sha256Hex(clip)} {
    mac.Write([]byte(part)); mac.Write([]byte{0})
}
// Different user, different clip, different line, or older than an hour → 403.

The rewriting step is also told never to describe or imitate a real, identifiable person's voice, and to describe an original voice with similar general qualities instead.

Use it

In the character app, “Describe a voice” has replaced the built-in voice list: a new character gets a voice its author designed or recorded. In the TTS playground, the Design a voice tab lets you write up to three descriptions, hear them side by side and save one to “My voices”. Through the API it is two calls, billed like speech, per second of audio returned:

# 1. Design: one reference clip per description, all in one batch
curl https://api.openalchemy.io/v1/audio/voices/design \
  -H "Authorization: Bearer $OA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "items": [
      {"instruct": "A warm young woman, clear gentle voice, speaks slowly with a smile",
       "text": "Hey, it is really good to finally meet you.", "language": "en"},
      {"instruct": "A calm middle-aged man, low and slightly husky, unhurried",
       "text": "Hey, it is really good to finally meet you.", "language": "en"}
    ]
  }'
# → {"voices": [{"audio_b64": "UklGR…", "format": "wav", "sample_rate": 24000, "seconds": 3.4}, …]}

# 2. Keep one: upload the clip with its line as the reference text
curl https://api.openalchemy.io/v1/audio/voices \
  -H "Authorization: Bearer $OA_API_KEY" \
  -F file=@voice1.wav -F name=hana -F language=en \
  -F reference_text="Hey, it is really good to finally meet you."

# 3. Speak anything in it
curl https://api.openalchemy.io/v1/audio/speech \
  -H "Authorization: Bearer $OA_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "cosyvoice3", "voice": "hana", "input": "See you tomorrow!"}' -o hana.wav

Ten languages are supported for designing: Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish and Italian. Voices saved to the library are currently Chinese, English or Japanese. Workers can opt in from Settings → Runtime → Voice design in Engine Desktop 0.6.29 or later.

Measured in October 2026. Download Engine Desktop to serve voice design from your own GPU.