multimodal-art-projection/YuE

Looking for the original YuE? Its code, documentation, and license are preserved on the YuE-v1 branch . YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality Compose in symbols. Create in sound. 📄 arXiv · PDF · 🎧 Demos · 🚀 Try online (free) · 🗳️ Music Arena · 📰 News · 🤗 YuE2 · 🚀 Quick start · 🤖 Agent skill · 📊 Benchmarks · 🤗 MERT2 · 🤗 SheetSage2 · 🤗 WSB · 📦 Release · All languages · September 14, 2026 Global: September 17, 2026 · Text-to-Audio: September 20, 2026 YuE2 brings frontier song quality to music generation with an editable composition. Give it lyrics and a style prompt: it writes a melody-and-chord plan, then realizes that plan as a complete song with vocals and accompaniment. 🎧 YuE2 needs your ears We're running a public blind listening study comparing YuE2 with leading proprietary music generation systems. Hear anonymous clips and choose A, B, or a tie. Listen & vote → · No account needed; headphones recommended. Frontier quality. YuE2 is competitive with Suno v5/v6 on WildSongBench. YuE2 (best-of-8) achieves 6.9632 SongBench Avg , the highest observed mean among all evaluated settings. White-box music generation through symbolic planning. Read, play, and change the composition before rendering it. Melody and chords become explicit controls that a person or an agent can inspect and edit. Zero-shot covers and agentic editing. Reimagine a transcribed song in a new style, or refine a song through a conversation about its score, arrangement, and lyrics—all with the same generation checkpoint. 192 WildSongBench prompts. Both YuE2 settings use symbolic planning. Bo8 = best-of-8. The axes are normalized comparison indices; bubble area represents AudioBox production quality. Scores and evaluation protocol . Vector PDF · SVG . News 📄 October 6, 2026 — SheetSage2 on arXiv. Read the paper and use the BibTeX citation . 📄 October 2, 2026 — SheetSage2 technical report. The SheetSage2 report describes the transcription model used for covers and score editing: MIDI-derived synthetic supervision, structured decoding, and autoregressive distillation, with evaluations on eight benchmark collections. 📄 September 29, 2026 — YuE2 on arXiv. Read the paper and use the BibTeX citation . 📄 September 26, 2026 — YuE2 technical report. Our technical report is now available, with the full model and training methods, automatic and expert listening results, and evaluations of score editing and zero-shot covers. 🎹 September 25, 2026 — Instrumental generation and covers. The yue2-music agent skill now turns a simple description, ABC score, or reference recording into instrumental music. YuE2 writes the score by default; the skill moves the vocal melody into the instrumental part before rendering. Recommended agent: GPT-6 Astra. Download the updated skill → 🚀 Try YuE2 online for free. Create a song in your browser with the NOIZ-hosted demo → . No installation required. ⚡ YuE2-Turbo. NOIZ's inference and serving toolkit → accelerates YuE2 and supports concurrent requests. 🎛️ ComfyUI. YuE2 has native nodes and an official text-to-music workflow → . 🎬 Maestro. Maestro → includes YuE2 for local song generation, composition planning, and covers. Creator's post → Hear what you can make Create Cover Edit with an agent Lyrics + style → score → full song Source recording → melody score → a new interpretation Musical feedback → score, style, or lyric revisions → a new recording Listen and inspect the score Hear zero-shot covers Follow an editing conversation The agentic demo follows The Last Train through 9 steps and 14 versions , from Mandarin pop to English jazz with new harmony and a saxophone solo. Listen to each version and inspect its conversation, score, prompt, and lyrics. How it works One AR–NAR Mixture-of-Transformers backbone predicts the score and semantic tokens autoregressively, then generates acoustic latents with flow matching. A VAE decodes those latents into stereo audio. Creation, covering, and editing differ in where the score comes from: YuE2, a transcribed recording, or an edited composition. The staged Python API exposes plan() → generate_semantic() → synthesize() → decode() . See the generation guide for exact-plan reuse and decoder selection. Quick start Linux · Python 3.12 · NVIDIA GPU with BF16 support and 24 GB VRAM. YuE2 produces 48 kHz stereo audio without quantization. Model files download from Hugging Face on first use. git clone https://github.com/multimodal-art-projection/YuE.git cd YuE python3.12 -m venv .venv source .venv/bin/activate python -m pip install --upgrade pip python -m pip install . python examples/generate.py --output outputs/first-song Open outputs/first-song/audio.flac . The output directory also retains the score, semantic tokens, acoustic latents, generation settings, and model identities. The Python interface is equally short: import json from pathlib import Path from yue2 import YuE2Pipeline request = json . loads ( Path ( "examples/song.json" ). read_text ( encoding = "utf-8" )) with YuE2Pipeline . from_pretrained ( "m-a-p/YuE2-3B" , device = "cuda" ) as pipe : song = pipe ( ** request ) song . save_artifacts ( "outputs/my-song" ) print ( song . truncated ) Setting Behavior cot="full" Generate an editable melody-and-chord plan; the default for new songs cot="melody" Use a melody plan with free accompaniment; recommended for covers cot="off" Generate directly from lyrics and style abc=... Supply your own score in full or melody mode Generation guide · Original example inputs · v0.1.6 wheel archive Cover a song Transcribe a source recording with 🤗 SheetSage2 , review its melody ABC, and provide new lyrics or a target style. For covers, use cot="melody" and a score without chord symbols so the accompaniment can adapt to the new style. from pathlib import Path from yue2 import YuE2Pipeline with YuE2Pipeline . from_pretrained ( "m-a-p/YuE2-3B" , device = "cuda" ) as pipe : cover = pipe ( style = "English, jazz-funk, warm lead vocal, Rhodes, bass and drums" , lyrics = Path ( "cover-lyrics.txt" ). read_text ( encoding = "utf-8" ), abc = Path ( "cover-score/score.abc" ). read_text ( encoding = "utf-8" ), cot = "melody" , seed = 42 , ) cover . save_artifacts ( "outputs/cover" ) SheetSage2 runs in a separate environment and loads its MERT2 encoder automatically. The cover guide gives the complete transcription and generation commands. An included original melody example also lets you try score-conditioned generation immediately. Edit a composition Export a plan, revise the musical details, and render the edited score: import json from pathlib import Path from yue2 import YuE2Pipeline request = json . loads ( Path ( "examples/song.json" ). read_text ( encoding = "utf-8" )) with YuE2Pipeline . from_pretrained ( "m-a-p/YuE2-3B" , device = "cuda" ) as pipe : plan = pipe . plan ( ** request ) plan . save ( "outputs/plan" ) Copy outputs/plan/score.abc to edited.abc , then ask an agent to change its harmony, melody, tempo, or form. Supply the edited file as a new score: python examples/generate.py --request examples/song.json \ --abc-file edited.abc --cot full --output outputs/edited The editable score is the white-box interface: you can inspect the intended composition and intervene on it. Editing generates a new complete recording; it does not preserve the original waveform outside an edit. Editing guide and a reproducible harmony example . Agent skill The yue2-music skill teaches an agent how to generate songs and instrumental music, transcribe and cover recordings, edit ABC scores, check musical invariants, and organize listening comparisons. We recommend GPT-6 Astra as the agent. YuE2 remains the model that composes the default score and generates the audio. Use skills/yue2-music/ from this repository or download the updated skill ZIP . Install it using your agent's skill-directory or import mechanism. The song workflow uses the Python runtime installed with pip install .