nobodywho-ooo/nobodywho

On-device AI for any device. NobodyWho is an inference engine that lets you run LLMs locally and efficiently. Features Run locally, offline — no API keys needed or hidden fees Run any chat LLM — Gemma, Qwen, Mistral and more Fast, type-safe tool calling — automatically generates structured grammars from your function signatures, no schema writing needed Multimodal input — provide image and audio information to your LLM Text-to-speech — synthesize local WAV audio with Kokoro, Pocket TTS and Supertonic backends Speech-to-text — transcribe audio into text with Whisper Voice Activity Detection — know when to stop listening and start transcribing with Silero Model downloading — load models directly from Hugging Face or any URL Under the Hood flowchart TD FL["Flutter"]:::lang PY["Python"]:::lang GO["Godot"]:::lang K["Kotlin"]:::lang S["Swift"]:::lang RN["React Native"]:::lang FRB["flutter_rust_bridge"]:::glue P3["PyO3"]:::glue GX["gdext"]:::glue U["UniFFI"]:::glue FL --> FRB PY --> P3 GO --> GX K --> U S --> U RN --> U CORE["NobodyWho · Rust<br/>chat · templates · grammars · sampling · context shifting"]:::core FRB --> CORE P3 --> CORE GX --> CORE U --> CORE CORE --> LCPP["llama.cpp<br/>text · vision · embeddings · reranking"]:::engine CORE --> ORT["ONNX Runtime<br/>speech-to-text · text-to-speech · VAD"]:::engine LCPP --> HW["Hardware<br/>Vulkan · Metal · CUDA · GPU · CPU"]:::hw ORT --> HW classDef lang fill:#e8eefc,stroke:#5b7bd5,color:#11204a classDef glue fill:#f3f0fb,stroke:#8b7bd5,color:#2a1f4a classDef core fill:#eaf6ee,stroke:#4fa46a,color:#0f3b1f classDef engine fill:#fdf0e3,stroke:#d58f3b,color:#4a2d0b classDef hw fill:#f0eeea,stroke:#a09a90,color:#3a352e Loading GPU-accelerated inference via Vulkan or Metal — runs fast on any OS Compatible with thousands of pre-trained LLMs — use any LLM in the GGUF format Powered by the wonderful llama.cpp You can test our inference engine on iOS , Android , Vision Pro and Apple Watch . Platforms Binding Install Runs on Documentation Kotlin Maven Central Desktop, Android docs.nobodywho.ooo/kotlin Swift SPM macOS, iOS, visionOS, watchOS docs.nobodywho.ooo/swift React Native / Expo npm Desktop, Android, iOS docs.nobodywho.ooo/react-native Flutter pub.dev Desktop, Android, iOS docs.nobodywho.ooo/flutter Python PyPI Desktop docs.nobodywho.ooo/python Godot AssetLib Desktop, Android docs.nobodywho.ooo/godot Desktop means Linux, macOS and Windows throughout. Three gaps worth knowing before you start: Godot has no iOS export. Use the Flutter, React Native or Swift binding on iOS. Windows ARM64 is not supported yet. There is no web export. It is tracked in issue #111 . ⭐ Useful to you? Star the repo, it's the easiest way to say thanks. Requirements Inference uses the GPU where available and CPU where not. The real constraint is memory. Desktop Hardware — any 64-bit Linux, macOS or Windows machine. Windows is x86_64 only, there are no ARM64 Windows builds. macOS accelerates through Metal out of the box, Linux and Windows need a GPU driver with Vulkan support and fall back to CPU without one. Memory — roughly 1.5× the model file in free RAM, or 2× on a machine that's already busy. 8 GB is a comfortable floor for models up to ~2 GB; 16 GB or more above that. Discrete GPUs — NobodyWho offloads as many layers as fit in free VRAM and runs the rest on the CPU, so a model larger than your VRAM still works, just slower. If barely any of it fits, it skips the GPU and stays on CPU. It uses one GPU, the card with the most free memory. Mobile iOS — iPhone 11 or newer, 4 GB RAM or more. Android — Snapdragon 855 / Adreno 640 / 6 GB RAM or better. Rule of thumb — the device needs roughly twice the model file size in available RAM. iOS reserves around 2 GB, Android 2 to 4 GB depending on vendor. Models under 1 GB run smoothly on any phone. Models Any model in the GGUF format works. Pass a hf:owner/repo:QUANT reference, an HTTPS URL, or a local path anywhere a model path is expected; remote models are downloaded and cached on first use. Pass "auto" to fit one to available memory. Start with Qwen3 0.6B : at ~330 MB it is small enough for any phone and good enough to tell whether the integration works. Quick Start Python pip install nobodywho from nobodywho import Chat chat = Chat ( 'hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M' ) response = chat . ask ( 'What is the capital of Denmark?' ) print ( response . completed ()) # The capital of Denmark is Copenhagen. Python documentation · PyPI Kotlin // Android implementation( " ai.nobodywho:nobodywho-android:<version> " ) // Desktop JVM (Linux, macOS, Windows) implementation( " ai.nobodywho:nobodywho:<version> " ) Use the version from the Maven Central badge above. import ai.nobodywho.Chat val chat = Chat .fromPath( modelPath = " hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M " ) val response = chat.ask( " What is the capital of Denmark? " ).completed() println (response) // The capital of Denmark is Copenhagen. Kotlin documentation Swift Add via Swift Package Manager: https://github.com/nobodywho-ooo/nobodywho-swift.git import NobodyWho let chat = try await Chat . fromPath ( modelPath : " hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M " ) let response = try await chat . ask ( " What is the capital of Denmark? " ) . completed ( ) print ( response ) // The capital of Denmark is Copenhagen. Swift documentation · GitHub · starter app React Native / Expo # React Native npm install react-native-nobodywho # Expo npx expo install react-native-nobodywho import { Chat } from "react-native-nobodywho" ; const chat = await Chat . fromPath ( { modelPath : "hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M" , } ) ; const msg = await chat . ask ( "What is the capital of Denmark?" ) . completed ( ) ; console . log ( msg ) ; // The capital of Denmark is Copenhagen. RN / Expo documentation · NPM · RN starter app · Expo starter app Flutter flutter pub add nobodywho import 'package:nobodywho/nobodywho.dart' as nobodywho; void main () async { await nobodywho. NobodyWho . init (); final chat = await nobodywho. Chat . fromPath ( modelPath : 'hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M' , ); final msg = await chat. ask ( 'What is the capital of Denmark?' ). completed (); print (msg); // The capital of Denmark is Copenhagen. } Flutter documentation · pub.dev · starter app Godot Install NobodyWho from inside the editor: In Godot 4.5+, open the AssetLib tab and search for NobodyWho . Download and import it, making sure Ignore asset root is ticked in the import dialogue. Reload the project. You can also grab a specific version from the releases page and import the zip the same way. Godot documentation Local Server NobodyWho provides an experimental local server that implements the OpenAI Chat Completions API. Start the server : uvx --from ' git+https://github.com/nobodywho-ooo/nobodywho.git#subdirectory=nobodywho/server ' nobodywho-server --model hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M --name qwen It listens on http://127.0.0.1:8888 and serves /v1/models and /v1/chat/completions . See the docs for more info. Documentation The documentation has everything you might want to know: https://docs.nobodywho.ooo/ Working with a coding agent? Point it at llms.txt or llms-full.txt , or install the NobodyWho skill so your agent can look up the current APIs and documentation: npx skills add https://github.com/nobodywho-ooo/nobodywho --skill nobodywho Community Discord & Matrix — ask us anything Issues and Discussions — bugs and feature requests CONTRIBUTING.md — set up the repo and send a PR CHANGELOG.md · SECURITY.md · CODE_OF_CONDUCT.md ⭐ Star the repo if NobodyWho is useful to you. It is how people find us. License NobodyWho is licensed under the EUPL-1.2 . You may use it in proprietary and commercial projects, free of charge.