Fish Audio Releases Drama 3, a 35B Open-Weight MoE Text-to-Speech Model
https://www.marktechpost.com/2026/10/14/fish-audio-drama-3-open-weight-tts-model/📌 Fish Audio Drama 3：35B MoE 架構的開源語音模型，讀得懂「演技指令」
TL;DR：35B MoE 模型支援括號式表演指導與 83 種語言，可自架部署，單串流首音延遲僅 88.9 毫秒。
當劇本裡寫著「[咬著牙忍住]」，一般的 TTS 只會把文字唸出來，不會懂語氣——這正是 Fish Audio 這週發布的 Drama 3 想處理的落差。
🤔 從唸字到演戲：方括號裡的指示要怎麼「演」出來
Drama 3 是 Fish Audio 推出的 text-to-speech 模型，接受純文字腳本搭配方括號式的表演指示。例如在臺詞前加上「[barely holding it together]」，或用成對標籤如
🧩 從 Qwen3.5-35B-A3B 到 Codebook-MTP：拆解 Drama 3 的架構
Drama 3 是一個 35B 參數的 mixture-of-experts 模型，每個 token 只啟用約 3B 參數。根據發布文章，其骨幹模型從 Qwen3.5-35B-A3B 語言模型出發，共 40 層，每 4 層中有 3 層是 Gated DeltaNet，1 層是 gated full-attention——也就是每 4 層才有 1 層保留 KV cache，其餘的 DeltaNet 層改用固定大小的狀態來儲存歷史資訊。
解碼端引入新的 Codebook-MTP decoder，取代前代 S2 使用的逐幀 Fast AR decoder：4 個共享層之後接上 9 層，每層對應一個 acoustic codebook，但只有 codebook 0 與 1 的結果會回饋進骨幹模型（S2 則是全部 10 個 codebook 都回饋）。音訊 tokenizer 沿用 S2 版本，基於 Descript Audio Codec，輸出 44.1kHz 音訊，運作速率約每秒 21.5 frames。
訓練流程分為 3 個預訓練階段，接著是指令微調，最後用強化學習收尾。RL 階段採用不含 KL 懲罰項的 GRPO，搭配單一學習到的人類偏好 reward model。
📊 發音準確度拿下第一，但整體品質仍落後對手
在 Artificial Analysis Pronunciation Robustness 測試中（針對 4 類困難英文單字由真人評審判斷發音），Drama 3 整體得分 93.2%，排名第一，高於前代 S2.1 Pro 的 88.1%。不過這項測試量的是發音，不是情感演繹，Fish Audio 對情感表現只給出質化描述。在 Artificial Analysis 的品質排行榜上，「Drama 3 Preview」的 Elo 為 1242，勝過 S2.1 Pro 的 1190，但仍落後 Eleven v4 Turbo 的 1328。
延遲方面，模型卡報告在單張 H200 上的串流表現：單一串流時，首個音訊片段的中位延遲為 88.9 毫秒，real-time factor 為 0.131；16 個串流同時進行時，吞吐量可達每秒 927.6 frames。推論透過 SGLang-Omni 服務，其融合 kernel 僅在 Hopper GPU 上驗證過。
⚠️ 自架門檻與授權限制
Fish Speech 的安裝指南要求至少 48GB NVIDIA GPU 記憶體（FP8），不支援 CPU、Apple MPS 或 AMD ROCm；MoE 專家層以 FP8 存放，其餘權重為 bf16。若使用 Hosted API，呼叫 POST /v1/tts 時要把 model header 設成 drama-3，文件特別警告：header 缺漏或打錯會「靜默」退回成 s2.1-pro。官方牌價為每 100 萬字元 50 美元，到 2026 年 11 月 12 日前有每 100 萬字元 10 美元的上市優惠價。
權重採用 Fish Audio Community License v1.0：研究與非商業用途免費；商業用途在年營收低於 50 萬美元時也免費，但要以第三方 API 方式提供服務需另外取得授權，產品上須標示「Built with Fish Audio」，且禁止未經同意複製真人聲音。
🎯 實務啟示
如果你要做的是有角色、有情緒張力的語音內容（有聲書、劇本朗讀、多角色配音），Drama 3 把「表演指導」直接做進 prompt 介面，省去另外設計情緒控制系統的工夫；但自架需要一張 48GB 以上的 Hopper 等級 GPU，商業授權門檻也要先盤清楚，中小規模試用前建議先用 hosted API 評估效果再決定是否自建。
🔗 來源
- 標題：Fish Audio Releases Drama 3, a 35B Open-Weight MoE Text-to-Speech Model
- 作者／機構：Asif Razzaq
- 連結：https://www.marktechpost.com/2026/10/14/fish-audio-drama-3-open-weight-tts-model/
#TextToSpeech #FishAudio #OpenWeight #MixtureOfExperts #TTS #SpeechSynthesis #AIAudio #Qwen #GRPO #VoiceAI
原始資料 MarkTechPost · 收集於 2026-10-15
摘要原文
This week, Fish Audio released Fish Audio Drama 3 . Drama 3 is a text-to-speech model from Fish Audio that takes plain-language acting direction. It is a 35B mixture-of-experts model with about 3B parameters active per token. The weights are on Hugging Face , and the model is also live in the Fish Audio API . For developers, it is a self-hostable option for scripted dialogue, audiobooks and dubbing. Drama 3 turns a script along with bracketed directions into acted speech. Writers place cues like [barely holding it together] before a line. Paired tags such as <whisper> change only the enclosed words, per the Drama 3 docs . Directions can be written in any language, independent of the spoken line. The model supports 83 languages . Speaker tags <|speaker:0|> to <|speaker:15|> allow up to 16 characters in 1 request. The API takes 1 voice ID per speaker, so a full scene renders in a single call. According to the launch post , the backbone starts from the Qwen3.5-35B-A3B language model. It has 40 layers, grouped as 3 Gated DeltaNet layers per 1 gated full-attention layer. Only 1 layer in 4 keeps a KV cache. DeltaNet layers store history in a constant-size state instead. A new Codebook-MTP decoder replaces the per-frame Fast AR decoder used in S2 . It has 4 shared layers feeding 9 layers, 1 per acoustic codebook. Only codebooks 0 and 1 feed back into the backbone; S2 fed back all 10. The audio tokenizer is unchanged from S2. It is built on the Descript Audio Codec and outputs 44.1 kHz audio. It runs at about 21.5 frames per second. Training ran 3 pre-training stages, then instruction tuning, then reinforcement learning. RL used GRPO without a KL penalty and a single learned human-preference reward model. Fish Audio reports Drama 3 at 93.2% overall on Artificial Analysis Pronunciation Robustness , ranked #1. Its predecessor S2.1 Pro scored 88.1%. Human listeners judge hard English words across 4 categories: This benchmark tests pronunciation, not emotional delivery. Fish Audio describes emotive quality only qualitatively. On the Artificial Analysis quality leaderboard , ‘Drama 3 Preview’ holds an Elo of 1242. That beats S2.1 Pro at 1190 but trails Eleven v4 Turbo at 1328. The Fish Speech install guide lists at least 48 GB of NVIDIA GPU memory in FP8. CPU, Apple MPS and AMD ROCm are not supported. MoE experts ship in FP8, with the remaining weights in bf16. The model card reports streaming results on 1 H200. At 1 stream, median time to first audio is 88.9 ms with a real-time factor of 0.131. At 16 streams, throughput reaches 927.6 frames per second. Serving runs through SGLang-Omni , whose fused kernels are validated only on Hopper GPUs. In the hosted API, set the model header to drama-3 on POST /v1/tts . The docs warn that a missing or misspelled header silently falls back to s2.1-pro . List price is $50 per 1M characters. A $10 launch price runs until November 12, 2026, per the launch post . The weights use the Fish Audio Community License v1.0 . Research and non-commercial use are free. Commercial use is free under USD 500,000 in annual revenue. Hosting Drama 3 as a third-party API needs a separate license. Products must display ‘Built with Fish Audio.’ Cloning a real voice without consent is prohibited. Check out the weights on Hugging Face , the technical report , the GitHub repo and the launch post . All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter . Wait! are you on telegram? now you can join us on telegram as well. The post Fish Audio Releases Drama 3, a 35B Open-Weight MoE Text-to-Speech Model appeared first on MarkTechPost .
由 claude-code/sonnet 自動生成