MakazhanAlpamys/Soup

🌍 English | Türkçe | العربية | 日本語 Soup Fine-tune and post-train LLMs in one command. No SSH, no config hell. Website · Quick Start · Web UI · Config · Docs · Commands · Models · Discord · Telegram · Product Hunt Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done. pip install " soup-cli[train] " # add [train] to fine-tune; bare `soup-cli` is the light CLI soup init --template chat soup train Fine-tune an 8B model on a 4 GB laptop GPU. Layer streaming keeps the frozen base out of VRAM and feeds it to the GPU one decoder layer at a time. Measured on an RTX 3050 Laptop 4 GB: Llama-3.1-8B-Instruct + NF4 at 119.6 tok/s, 3.32 GB peak — bit-exact against a normal resident run, and reproduced independently on an H100 at 113.00 tok/s in the same 3.32 GB. (Both figures were measured on v0.72.2, before the v0.73.0 correctness repair that cost −4.8% at 32B; neither has been re-run on a 4 GB card since — re-measurement pending in issue #361 .) Opt-in ( stream_layers: true ) and still BETA — how it works · all measurements · paper · check it yourself on a free Colab T4 (caps the process to 4 GB, then asserts a streamed model is bit-identical to a normal one) Llama-3.1-8B-Instruct + NF4, LoRA, batch 1, seq 512 on an RTX 3050 Laptop 4 GB — 3.32 GB peak, 119.6 tok/s (measured on v0.72.2, before the #331 repair; re-measurement pending in issue #361). Full video (90s) Why Soup? Training LLMs is still painful. Even experienced teams spend 30-50% of their time fighting infrastructure instead of improving models. Soup fixes that. Zero SSH. Never SSH into a broken GPU box again. One config. A simple YAML file is all you need. Auto everything. Batch size, GPU detection, quantization — handled. Works locally. Train on your own GPU with QLoRA. No cloud required. What's New v0.75.0 — the same soup.yaml trained a different recipe on MLX than on transformers, silently. Six training options were validated, documented, accepted — and read by nothing on that backend. All 60 pull requests in this release came from outside the maintainer , by 22 people. Breaking: an unknown config key now refuses the load. v0.74 warned and named this release as the deadline. A typo like quantizaton , or a key that only exists on a newer Soup, used to be dropped while the run proceeded with the setting not applied; it now fails on the CLI (exit 1) and in the API ( ValueError ), naming the field you probably meant. The detector applies the root-level lora: remap the schema has honoured since v0.40.1, so that spelling is accepted, not refused; the two soup fetch examples files using it moved to the canonical training.lora . Every recipe and template loads clean, key names are escaped before they reach the terminal, and the scan is bounded. MLX honours the config it accepted. train_on_responses_only , warmup_ratio / scheduler / weight_decay / optimizer , max_grad_norm , gradient_accumulation_steps and gradient_checkpointing were each validated and then dropped on backend: mlx . Only 8 of the 32 optimizer names have an MLX equivalent; the other 24 are refused by name instead of silently becoming AdamW. MLX also drives the live dashboard, the tracker and soup ui , and soup doctor --config lists the settings a backend does not read. Validation loss existed nowhere. It was computed on every backend and thrown away: no metrics column, no event field, nothing on the panel. It is now recorded, streamed and displayed. Breaking: grpo_variant: gspo is the published sequence-level objective (arXiv:2507.18071), replacing a column-centering heuristic in which a padding token also shifted the gradient of every row sharing its column. Existing gspo configs will not reproduce prior runs. Web UI read endpoints and SSE require auth , with short-lived single-use tickets instead of a token in a query string; --public no longer serves /docs and /openapi.json to the LAN; and a training subprocess no longer hangs when nothing reads its output. torch>=2.6.0 closes v0.74.0's known limitation: at 2.5.1 trl>=0.29 could not import and every preference trainer was dead. Also fixed: training.loraplus_lr_ratio crashed every run that set it, and packing: true raised on TRL 0.29. Python 3.10–3.12 only. On 3.13+, pip used to resolve untested PyTorch wheels that crash in the native extension before Soup runs at all. Older highlights live on the GitHub Releases page. Quick Start 1. Install Soup is a command-line application, so the cleanest install gives it its own environment and puts soup on your PATH : # Light core: CLI + config + data tools, no PyTorch pipx install soup-cli uv tool install soup-cli # same idea, if you already use uv # Add the training stack (torch, transformers, peft, trl, datasets, …) pipx install " soup-cli[train] " # Everything (train + serve + ui + data) in one shot pipx install " soup-cli[all] " # Or from GitHub (latest dev) pipx install " git+https://github.com/MakazhanAlpamys/Soup.git " Already inside a virtualenv, a Colab notebook, or a Docker image? Use pip directly, with the same names and extras: pip install soup-cli pip install " soup-cli[train] " pip install " soup-cli[all] " pip install git+https://github.com/MakazhanAlpamys/Soup.git Use pip rather than pipx if you also want to import soup_cli from your own code, since pipx deliberately isolates the application from everything else. The full extras table ( fast , mlx , serve , eval , ui , vision , audio , …) lives in docs/models.md . error: externally-managed-environment ? That is PEP 668 , not a Soup problem. Debian 12, Ubuntu 23.04 and later stop pip from writing into the system Python, because apt manages those files too. pipx and uv tool sidestep it by giving Soup its own environment, which is why they are listed first above. python3 -m venv .venv && source .venv/bin/activate then plain pip works just as well. Double quotes, not single. "soup-cli[train]" is the only spelling that works in every shell — cmd.exe , PowerShell, bash and zsh. If you copied 'soup-cli[train]' from an older tutorial and pip rejected it, that is the reason: why, and the exact error . soup init , soup data … , and the other data/inspection commands work on the light install. Fine-tuning ( soup train ) needs the [train] extra. 2. Create a config soup init # interactive wizard soup init --template chat # or start from a template Templates: chat , code , tool-calling , medical , reasoning , vision , kto , orpo , simpo , ipo , bco , rlhf , pretrain , moe , longcontext , embedding , audio . 3. Train, test, ship soup train --config soup.yaml # LoRA, quantization, batching — all handled soup chat --model ./output # talk to your model soup push --model ./output --repo you/my-model soup merge --adapter ./output # merge LoRA into the base soup export --model ./output --format gguf --quant q4_k_m # GGUF for Ollama / llama.cpp More export targets (ONNX, TensorRT, AWQ, GPTQ, BitNet) and deployment options live in docs/serving-and-export.md . Web UI Prefer a browser? soup ui serves a local dashboard for experiments, training setup, live metrics, dataset exploration and model chat. pip install " soup-cli[ui] " soup ui # Opens http://127.0.0.1:7860 Web UI documentation Configuration A complete soup.yaml : base : meta-llama/Llama-3.1-8B-Instruct task : sft # backend: unsloth # 2-5x faster, pip install "soup-cli[fast]" data : train : ./data/train.jsonl format : alpaca val_split : 0.1 training : epochs : 3 lr : 2e-5 batch_size : auto lora : r : 64 alpha : 16 quantization : 4bit output : ./output config/schema.py is the single source of truth for every field. Advanced data, training, and PEFT options are documented under Documentation . Unknown config keys are rejected since v0.75. A key no model declares — a typo like quantizaton , or a field that only exists on a newer Soup — used to validate clean and be discarded, so the run proceeded with the setting simply not applied. v0.74 reported it at load with the field you probably meant; from v0.