Mindfire Technology

Inside the Qwen3-TTS Engine Code (Qwen3-TTS, Part 2)

Thu, 07 May 2026 00:00:00 -0600

This is the follow-up to my previous post on adding Qwen3-TTS to Book2Audio. That post covered how to use Qwen3-TTS from the command line and how I refactored the code to support multiple TTS engines. This post walks through the actual engine code — how it's structured, what each piece does, and how Qwen3-TTS works under the hood.

All the code discussed here is available in my GitHub repo.

The Engine Abstraction

The starting point is a simple abstract base class that defines what any TTS engine needs to do:

from abc import ABC, abstractmethod
import numpy as np

class TTSEngine(ABC):
    @abstractmethod
    def generate(self, text: str) -> np.ndarray:
        ...

    @property
    @abstractmethod
    def sample_rate(self) -> int:
        ...

Two methods, that's it. generate takes a string of text and returns a numpy array of audio samples. sample_rate returns the sample rate in Hz so the caller knows how to save the audio correctly. Any TTS backend — Kokoro, Qwen, or something else entirely — just needs to implement these two things.

AudioGenerator then wraps any engine and handles the model-agnostic parts:

class AudioGenerator:
    def __init__(self, engine: TTSEngine) -> None:
        self._engine = engine

    def generate(self, text: str) -> np.ndarray:
        return self._engine.generate(text)

    def save(self, audio: np.ndarray, output_file: str) -> None:
        sf.write(output_file, audio, self._engine.sample_rate)

    def generate_and_save(self, text: str, output_file: str) -> None:
        self.save(self.generate(text), output_file)

The key thing here is save — it pulls sample_rate from the engine rather than hardcoding it. Different engines could theoretically produce audio at different sample rates, and this handles that transparently. generate_and_save is just a convenience method that chains the two together.

Loading the Model

The QwenCustomVoiceEngine constructor handles model loading. If you don't pass in a pre-loaded model, it figures everything out from the model_size parameter:

QWEN_MODEL_SIZES = {
    '0.6b': 'Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice',
    '1.7b': 'Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice',
}

def __init__(self,
             speaker: str = 'vivian',
             language: str = 'Auto',
             instruct: str | None = None,
             model_size: str = '0.6b',
             model: Qwen3TTSModel | None = None) -> None:
    if model is None:
        model_id = QWEN_MODEL_SIZES.get(
            model_size.lower(), QWEN_MODEL_SIZES['0.6b']
        )
        attn_impl = 'sdpa'
        try:
            import flash_attn
            attn_impl = 'flash_attention_2'
        except ImportError:
            pass
        device = 'cuda:0' if torch.cuda.is_available() else 'cpu'
        model = Qwen3TTSModel.from_pretrained(
            model_id,
            device_map=device,
            dtype=torch.bfloat16,
            attn_implementation=attn_impl,
        )

The QWEN_MODEL_SIZES dictionary maps friendly size names to full Hugging Face model identifiers. This way the rest of the code just passes around "0.6b" or "1.7b" instead of long model strings. It also means that if Qwen releases new checkpoints, there's only one place to update. Note to self: I should really allow you to pass a full model name here and only use this dictionary for shorthands. I need to implement that still.

There are a few things worth noting about the from_pretrained call.

device_map controls where the model runs. The code checks torch.cuda.is_available() and uses the GPU if present, falling back to CPU otherwise. This is the same pattern that the Kokoro engine uses, so both engines behave consistently.

dtype=torch.bfloat16 halves the memory footprint compared to full float32 precision with negligible quality loss. For the 1.7B model, this is the difference between fitting on a consumer GPU and not fitting at all.

The attention implementation check is about GPU memory efficiency. FlashAttention 2 is an optimized attention algorithm that reduces VRAM usage during inference. But it requires the flash-attn package, which compiles from source and needs the CUDA Toolkit and a C++ compiler installed — a nontrivial setup on Windows. If it's not available, the engine falls back to PyTorch's built-in scaled dot product attention (sdpa), which works fine but uses a bit more VRAM. The code handles this gracefully: try the import, use it if it's there, move on if it's not. To be frank, I never got flash attention working — getting the CUDA Toolkit and C++ compiler set up on Windows was more than I wanted to take on right now. So the code is there for when I get around to it, but for now I use sdpa.

The constructor also accepts a pre-loaded model via the model parameter. This is useful for testing — you can inject a mock — and it also means you could share a single model instance across multiple engine objects if you needed to.

Generating Speech

The generate method is where text actually becomes audio:

def generate(self, text: str) -> np.ndarray:
    kwargs = {
        'text': text,
        'language': self._language,
        'speaker': self._speaker,
    }
    if self._instruct is not None:
        kwargs['instruct'] = self._instruct

    wavs, sr = self._model.generate_custom_voice(**kwargs)
    self._sample_rate = sr
    return wavs[0]

It assembles the keyword arguments for generate_custom_voice, conditionally including the instruct parameter, makes the call, and returns the first waveform.

The instruct parameter is only included when it's not None. This matters because the 0.6B model doesn't support instruction control — only the 1.7B CustomVoice model does. Passing instruct to the 0.6B model won't cause an error, but it will be silently ignored.

generate_custom_voice returns a list of waveforms because the Qwen3-TTS API supports batched generation — you can pass a list of strings and get back multiple audio arrays in one call. For our paragraph-at-a-time use case, we always pass a single string and take wavs[0]. However, I should really change things to allow this code to handle everything at once as an option. As I mentioned in the previous post, Qwen3-TTS produces a somewhat different voice each time you call it, which makes for a questionable audiobook experience when you're generating paragraph by paragraph. Batching everything into a single call might help with that consistency.

The sample rate is captured from the return value rather than hardcoded, though in practice Qwen3-TTS always returns 24000 Hz. By reading it from the response, the code stays correct even if a future model version changes the rate.

Wiring It Together

The CLI entry point in book_to_audio.py ties everything together. When the user passes --engine qwen, a small factory function creates the right engine:

def _create_engine(args):
    if args.engine == 'qwen':
        return QwenCustomVoiceEngine(
            speaker=args.speaker,
            language=args.language,
            instruct=args.instruct,
            model_size=args.model_size,
        )
    else:
        return KokoroEngine(voice=args.voice)

That engine gets wrapped in an AudioGenerator, which gets handed to BookToAudio, which does the actual document processing. BookToAudio doesn't know or care whether it's using Kokoro or Qwen — it just calls generate and gets audio back.

This is the payoff of the strategy pattern. Adding a third engine later — say, for voice cloning with the Qwen3-TTS Base model — means writing a new engine class, adding an option to _create_engine, and nothing else changes.

What's Next

Voice cloning is the natural next step. The Qwen3-TTS Base model can clone a voice from just a few seconds of reference audio, which opens up the possibility of generating an entire audiobook in a specific narrator's voice. That will be a separate engine class since it uses a different model and a different API (generate_voice_clone instead of generate_custom_voice), but the abstraction is already in place to support it.

If you need help with your Artificial Intelligence solutions, we're here to help.

Implementing Qwen3-TTS in My PDF-to-Audiobook Pipeline (Qwen3-TTS, Part 1)

Fri, 17 Apr 2026 10:32:51 -0600

In my last post, I walked through building a PDF-to-audiobook pipeline using Kokoro for text-to-speech. The pipeline worked well enough that I've been actively using it to listen to books that only exist as PDFs. But I mentioned wanting to try Alibaba's recently open-sourced Qwen3-TTS as an alternative voice engine, and I've now done exactly that. (My code is found in my github repo.)

This post covers how to use Qwen3-TTS to generate speech from the command line and a discussion of how I refactored the code to support multiple TTS engines. A follow-up post will walk through the Qwen engine code itself.

Trying Qwen3-TTS

Before touching any of my existing code, I wanted to hear what Qwen3-TTS actually sounded like. The setup is straightforward. Install the package:

pip install -U qwen-tts

The first time you use a model, the weights download automatically from Hugging Face. The 0.6B model is roughly 1.2GB and the 1.7B model is around 3.4GB.

Once installed, generating speech from Python is only a few lines:

import torch
import soundfile as sf
from qwen_tts import Qwen3TTSModel

model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice",
    device_map="cuda:0",
    dtype=torch.bfloat16,
)

wavs, sr = model.generate_custom_voice(
    text="The philosopher argued that all knowledge is provisional.",
    language="English",
    speaker="ryan",
)

sf.write("output.wav", wavs[0], sr)

Code found here.

You load a model with from_pretrained, call generate_custom_voice with your text, a language, and a speaker name, and you get back a list of waveform arrays and a sample rate. Write the first waveform to a file and you have a WAV you can play.

Qwen3-TTS comes with nine built-in speakers: aiden, dylan, eric, onoanna, ryan, serena, sohee, unclefu, and vivian. They vary quite a bit in tone and accent. I'd recommend generating a short sample with each one to find what works for your use case. However, my experience is that you get a somewhat different voice each time you run the voice. This makes it less than desirable for reading an audio book.

The 1.7B CustomVoice model also supports an instruct parameter that lets you control the delivery style with natural language:

wavs, sr = model.generate_custom_voice(
    text="The philosopher argued that all knowledge is provisional.",
    language="English",
    speaker="ryan",
    instruct="Read in a calm, steady audiobook narration style",
)

This is a genuinely interesting feature, but be aware that instruction control only works on the 1.7B models. The 0.6B models silently ignore the instruct parameter.

Find a list of all the available models on Hugging Face here.

Refactoring for Multiple Engines

My original code for Book2Audio had the Kokoro TTS model wired directly into the AudioGenerator class. To support Qwen3-TTS as an alternative, I needed to pull the model-specific logic out and make it swappable. (Apologies for naming the repo Book2Audio and the python file booktoaudio. I need to rename the repo at some point to match.)

The approach was a straightforward application of the strategy pattern. I created a TTSEngine abstract base class with two methods: generate, which takes text and returns a numpy audio array, and a sample_rate property. Then I wrote two concrete implementations: KokoroEngine wrapping the existing Kokoro pipeline, and QwenCustomVoiceEngine wrapping Qwen3-TTS.

AudioGenerator, which previously owned the Kokoro pipeline directly, now takes any TTSEngine. It delegates audio generation to whatever engine it's given and handles only the model-agnostic work of saving WAV files. BookToAudio, the class that orchestrates document parsing and paragraph-by-paragraph generation, didn't need to change at all. It still talks to AudioGenerator the same way it always did.

I also split the single book_to_audio.py file into several files. The engines live in their own directory, AudioGenerator and BookToAudio each got their own module, and book_to_audio.py became a thin CLI entry point. This makes it easy to add more engines later without everything piling up in one file.

From the command line, switching engines is just a flag:

python book_to_audio.py "documents/MyBook.pdf" --engine kokoro --voice af_heart

python book_to_audio.py "documents/MyBook.pdf" --engine qwen --speaker ryan --language English

You can also convert plain text directly without a PDF:

python book_to_audio.py --text "Hello world" --engine qwen --speaker vivian

By default, the Qwen engine uses the 0.6B model. To use the larger 1.7B model, which supports instruction control:

python book_to_audio.py "documents/MyBook.pdf" --engine qwen --speaker ryan --language English --model-size 1.7b --instruct "Read in a calm, steady audiobook narration style"

For long documents, you can limit the page range to test on a small section before committing to a full run:

python book_to_audio.py "documents/MyBook.pdf" --engine qwen --speaker ryan --start-page 10 --end-page 15

To process the document without generating audio — useful for inspecting the extracted text before spending time on generation:

python book_to_audio.py "documents/MyBook.pdf" --dry-run --generate-text-file

And to specify an output file name instead of the default:

python book_to_audio.py "documents/MyBook.pdf" --engine qwen --speaker ryan --output-file "my_audiobook.wav"

The Kokoro path works exactly as before. The Qwen path adds a few extra options for speaker, language, model size, and style instructions.

Kokoro vs. Qwen3-TTS for Audiobooks

After testing both engines on the same material, I have to be honest: I still prefer Kokoro for most audiobook listening. The Qwen3-TTS voices, while technically impressive, tend to have cadence patterns and occasional accent shifts that can be fatiguing over long listening sessions. The ryan speaker comes closest to a natural audiobook narrator in English, and I may switch to it in the future as I experiment more with the instruct parameter on the 1.7B model. But for now, Kokoro's more neutral delivery wins for extended listening.

Hardware Considerations

One thing that surprised me during this process was discovering that my laptop had been running Kokoro on CPU the whole time. PyTorch had been installed without CUDA support, which meant torch.cuda.is_available() returned False and everything silently fell through to CPU inference. It worked, just slower than it needed to be.

If you're running this on a machine with an NVIDIA GPU, make sure you install the CUDA-enabled version of PyTorch:

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126

You can verify it worked with:

python -c "import torch; print(torch.cuda.is_available())"

In any case, my code will use your GPU if it's available and CUDA is properly installed.

For Qwen3-TTS specifically, the 0.6B model needs roughly 1.5GB of VRAM and runs comfortably on a 6GB laptop GPU. The 1.7B model needs 4-6GB and may be tight on consumer hardware, especially if other applications are using the GPU. I'd recommend starting with the 0.6B model and only moving to the 1.7B if you want instruction control or find the quality insufficient. Theoretically the 1.7B should work, but I haven't really tested it yet on my laptop. I'll do that in a future post.

What's Next

In the next post, I'll walk through the actual Qwen engine code, explaining how it works and how to use the Qwen3-TTS API. I also plan to explore Qwen3-TTS voice cloning, which lets you train the model on a specific narrator's voice from just a short audio clip and then generate an entire audiobook in that style.

If you need help with your Artificial Intelligence solutions, we're here to help.