Everything here runs on your own hardware. The theme is that "local" no longer means "toy": a 26M-parameter net can call functions on a watch, a 7M-parameter net can out-reason frontier LLMs on puzzles, and a 744B mixture-of-experts model can be streamed off an SSD on a single desktop. I keep this page updated as new releases land, with the last update date noted per entry.

Small language models

Needle (Cactus Compute)

A 26M-parameter model distilled from Google's Gemini 3.1 into a "Simple Attention Network" built for resource-constrained devices such as phones and watches. It is specifically trained for function calling and, on the Cactus runtime, hits roughly 6000 tokens/second prefill and 1200 tokens/second decode. A neat proof that agentic tool-use does not require billions of parameters. Last update: active (235+ commits).

VibeThinker (Weibo AI)

A series of compact reasoning models. VibeThinker-1.5B punched far above its weight on math competitions like AIME and HMMT, and the newer VibeThinker-3B claims frontier-level scores on several verifiable reasoning benchmarks at only 3B parameters. The interesting bit is how much reasoning you can extract from a model small enough to run comfortably on a laptop. Last update: Jun 16, 2026 (VibeThinker-3B release).

TinyRecursiveModels (Samsung SAIL Montréal)

The official 7M-parameter TRM from the "Less is More" paper. It solves ARC-AGI, Sudoku and Maze tasks by recursively refining a latent scratchpad instead of emitting chain-of-thought, reaching 45% on ARC-AGI-1 and 8% on ARC-AGI-2 while using under 0.01% of a frontier LLM's parameters. I wrote a full deep dive on HRM & TRM covering how it works. Last update: Apr 1, 2026 (repo archived, read-only).

Large language models

Colibrì

A lightweight C inference engine that runs GLM-5.2, a 744B-parameter mixture-of-experts model, on a consumer machine with about 25GB of RAM by streaming expert weights from disk instead of holding them all in memory. It treats VRAM, RAM and storage as one unified memory hierarchy and adds speculative decoding, grammar-forced drafts and a web dashboard, so you get high-quality inference from a huge model without a GPU cluster. Last update: Jul 12, 2026.

Uncensored

Qwythos-9B (Claude-Mythos, heretic GGUF)

A GGUF-quantized, decensored derivative of Empero AI's Qwythos-9B (itself fine-tuned from Qwen3.5-9B). "Heretic" refers to the decensoring method used, Magnitude-Preserving Orthogonal Ablation (MPOA), which strips safety refusals while trying to keep reasoning intact: the card reports about 85% fewer refusals with near-identical MMLU (77.35% vs 77.03%). It ships a 1M-token context via YaRN rope-scaling and native function calling. Listed here for completeness on the uncensored end of the local-model landscape; use responsibly. Last update: actively downloaded (~45k downloads/month).

This is a living list. If a model here gets a major update or a new local-first release is worth adding, I refresh the entry and bump its date.