Deploying real-time personalized speech with Qwen3-TTS on Amazon SageMaker AI

來源原標題

Deploying real-time personalized speech with Qwen3-TTS on Amazon SageMaker AI

作者

Suneesh T

原始標籤

Advanced (300) · Amazon SageMaker AI · Technical How-to

原始連結

https://aws.amazon.com/blogs/machine-learning/deploying-real-time-personalized-speech-with-qwen3-tts-on-amazon-sagemaker-ai/

摘要原文

With voice cloning, you can generate new speech in a target speaker’s voice from a short reference recording, without retraining a model. You can now deploy the publicly available Qwen3-TTS-12Hz-1.7B-Base text-to-speech model from Amazon SageMaker JumpStart to a fully managed, real-time inference endpoint. Voice cloning reproduces the vocal identity of a specific speaker. Start with a short recording of the speaker and its transcript. Then supply the new text to synthesize. The model speaks that text in the reference speaker’s voice, without retraining. Media teams, educators, and application developers can use this capability to create personalized voice experiences and localize multilingual content. They can also support accessible communication and preserve a speaker’s identity across languages. With a self-hosted, publicly available voice cloning model, you control cost and keep audio data within your AWS environment. You can also adapt the model to your domain. With Amazon SageMaker AI , you can run the model on a fully managed real-time endpoint and handle infrastructure provisioning, health monitoring, and automatic scaling. You don’t manage the underlying GPU servers. This post shows how to deploy Qwen3-TTS-12Hz-1.7B-Base from Amazon SageMaker JumpStart using the Amazon SageMaker Python SDK , and how to invoke the resulting endpoint to clone a voice from a reference clip. It also covers the configuration settings that make this deployment work in practice, along with the Amazon CloudWatch metrics you can use to monitor and right-size the endpoint. Qwen3-TTS is a publicly available text-to-speech model family developed by the Qwen team at Alibaba Cloud. It covers 10 languages: Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, and Italian. The models use the Qwen3-TTS-Tokenizer-12Hz speech tokenizer and support streaming generation for low-latency, interactive scenarios. This post uses the Base variant, Qwen3-TTS-12Hz-1.7B-Base . It performs voice cloning from only a few seconds of user audio. It can also serve as a base for fine-tuning. For voice cloning, the model takes a reference audio clip and its transcript. It captures the speaker’s vocal characteristics, such as timbre, pitch, and cadence, and applies them to new text. This differs from the CustomVoice variant, which generates speech from a fixed set of predefined speakers rather than a user-supplied reference. The model also supports cross-lingual cloning: You can capture a voice from a reference in one language and generate speech in another while preserving the speaker’s vocal identity. Qwen3-TTS-12Hz-1.7B-Base is available in Amazon SageMaker JumpStart alongside Qwen3-TTS-12Hz-1.7B-CustomVoice and Qwen3-ASR-1.7B. With these JumpStart options, you can use the streamlined deployment shown in this post. With voice cloning, applications can reproduce the vocal identity of a chosen speaker from a reference recording, rather than being limited to a fixed set of predefined voices. This supports a range of use cases across media, customer engagement, education, and conversational AI. Key benefits of this deployment approach include: Common applications for voice cloning include: This solution deploys Qwen3-TTS-12Hz-1.7B-Base from Amazon SageMaker JumpStart to a real-time endpoint. JumpStart provides the model artifacts and a pre-built serving container, so you don’t write a custom inference handler. You construct a JumpStartModel object, call its deploy method, and invoke the resulting endpoint with the Amazon SageMaker runtime client. The JumpStart container serves the model to generate 24 kHz audio output from text input. This stage design affects how you size GPU memory, as the following section explains. The following diagram shows the real-time inference architecture for the deployment. Figure 1: Real-time voice cloning architecture on Amazon SageMaker AI The deployment follows the standard Amazon SageMaker AI real-time inference pattern: Before you begin, confirm that you have the following resources and permissions: Construct a JumpStartModel with the model ID and deploy it. The GPU memory override is the most important setting for this deployment. The following section explains how to configure it. Before running the following code, replace <sagemaker-execution-role-arn> with the ARN of the IAM execution role that Amazon SageMaker AI uses to deploy the model. The endpoint takes several minutes to reach the InService status while the container loads the model onto the GPU. You can monitor the endpoint status in the Amazon SageMaker AI console or by calling the describe_endpoint API. As described earlier, the model runs as two stages (talker and code2wav) on the same GPU. vLLM reserves GPU memory up front based on gpu_memory_utilization , the fraction of GPU memory a stage can claim. Both stages share one GPU. To avoid out-of-memory errors during startup, keep their combined reservation within the GPU’s capacity. Set the value so the two stages together fit comfortably. With one shared setting applied equally to both stages, 0.45 works well (0.45 + 0.45 = 0.90, leaving a approximately 10 percent buffer): At container startup, each stage logs its memory use to the endpoint’s log group in Amazon CloudWatch Logs. The following values confirm the model fits well within a 24 GB GPU: The model weights are small (3.66 GiB for the talker and 0.45 GiB for code2wav). Most of the remaining memory each stage is allowed becomes KV cache, the working memory that holds tokens for in-flight requests. At a utilization of 0.45 on a 24 GB GPU (about 22 GiB usable), each stage can use up to roughly 10 GiB. Together, the two stages fit within the GPU with headroom to spare. This is why a 24 GB GPU is well suited to the 1.7B model, and a larger GPU is not required. With the endpoint InService , test voice cloning. Two details are specific to this JumpStart container and are essential to get a response: Audio formats. Provide the reference clip as a base64-encoded data:audio/wav;base64,... URI. Converting the reference to 24 kHz mono WAV before encoding is the recommended input form. The endpoint returns audio in the format you request through response_format . This post uses "wav" , and the model generates 24 kHz mono audio. The following code converts the reference clip and base64-encodes it: Invoke the endpoint with the target text in input and the reference clip in ref_audio . Note the CustomAttributes routing argument, which is required to reach the text-to-speech handler: The preceding request is the most basic case: one sentence of text returned as speech in the cloned voice. The following patterns build on it, using the same ref_audio , ref_text , and route custom attribute. Sample audio for this example (open the files provided with this post): Reference input: reference_original_input.wav Generated output: cloned_output.wav A single request works well for one sentence or a short passage. For longer content, such as an article or a multi-line script, split it and send one request per sentence or paragraph so the voice stays consistent throughout. To produce speech in a different language while keeping the same voice, use the same reference clip and set "language" to another supported language. The following example uses the English reference to generate Chinese speech, preserving the speaker’s vocal identity across languages. Generated output: cross_lingual_cloning.wav Choose an instance in three steps. First, confirm that JumpStart supports it. Then, check your service quota and confirm that its GPU memory fits the model. For a 1.7B model, a 24 GB GPU (the g6 family, NVIDIA L4) is well matched and cost effective. To estimate concurrent request capacity, check the KV cache token budget in the endpoint’s log group in Amazon CloudWatch Logs. The log group path is /aws/sagemaker/Endpoints/<endpoint-name> .