The usual rule for running a model at home is simple: if the weights do not fit in your memory, you cannot run it. A small open-source project called Edge0 is built on the idea that, for one kind of model, the rule is wrong.
Edge0 describes itself as "an open-source streaming MoE inference framework for running large sparse models on consumer hardware". On 13 November its team released two models, Edge0 145B and Edge0 120B, built to run on a single NVIDIA GPU. On 20 November, release 0.3.1 added streaming on Apple silicon, so both models now run on a Mac with 64GB of memory. The repository has about 12,100 stars, 1,900 forks and 28 contributors, and it is updated almost daily.
How it works
A mixture-of-experts (MoE) model is made of many small sub-networks, the experts, and for each token it only uses a few of them. Edge0 145B has 512 experts per layer and uses 10 at a time, according to its model table. Most of the model sits idle on any given step.
Edge0 takes advantage of that. Instead of loading every expert into GPU memory, it keeps the experts on your SSD and streams in the ones the next step will need, into a fixed-size area of memory sized to your machine. A router trained to predict which experts will be needed a step ahead hides most of the waiting. The team's technical report from September, "The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction", describes the method on its earlier 35B model.
The new 145B and 120B models are derived from Qwen3.8-Flash-Next and, according to the project, were trained natively at 4 bits. They come in two formats, Q3_K and NVFP4, and run on edge0-strata, a fork of the MIT-licensed Strata engine with added support for this kind of model.
What the project claims
The headline figure in the README is Edge0 145B in Q3_K on an RTX 5090 D with 24GB of memory: 121.2 tokens per second when decoding agent tasks, and 3,554.9 tokens per second to read a 16,000-token prompt. For a model of that size on one consumer card, that would be fast enough for real agent work.
On quality, the project reports Edge0 145B at 86.5 on Terminal-Bench 2.1 and 64.6 on DeepSWE 1.1, ahead of the Qwen3.8-Flash-Next model it was derived from (83.3 and 58.7) on those two tests, and level with it or a little behind on GPQA-Diamond, IFBench and HLE. Edge0 120B scores lower on the coding tests (75.3 and 51.3) but holds up on long-context reading.
All of these are the project's own measurements. I could not find independent reproductions yet, and a model tuned on agent workloads can look better on agent benchmarks than in general use. Treat them as a reason to test it, not as a verdict.
The 20 November release also changed something small but useful for anyone wiring it into tools. The local server now returns errors in the standard OpenAI and Anthropic formats, so an agent tool can recognise a context overflow and compact its history, and a prompt that would leave fewer than 2,048 tokens for the answer is refused instead of being cut short.
How to try it
There are two ways in, both described in the README.
The desktop app. Download Edge0 Chat for Windows or macOS (Apple silicon) from the project's GitHub Releases, open it, and download a model under Models. This is the quickest way to see whether your machine copes.
The command line and local server. The
edge0-infertool providesserveandchatcommands and picks the right engine from the model folder.servestarts an OpenAI-compatible server on your machine, so any tool that talks to the OpenAI API can point at it instead. The 145B and 120B setup guide covers CUDA 13 on Linux and Windows, and Metal on macOS.
Check the hardware first. For the new models you need an NVIDIA GPU with CUDA 13 on Linux or Windows, or an Apple silicon Mac with 64GB of memory. A fast NVMe SSD matters, since the experts are read from it constantly. The earlier 35B model runs on far more devices, including iPhone and Android, through the GGML-based native engine.
A sensible first test: start the server, point a coding tool you already use at it, and give it three tasks from a real project that you have already solved with a hosted model. Compare the answers, the speed, and how often it needs correcting.
How it compares
Tools like llama.cpp and Ollama run models that fit in your memory, and they are mature, widely supported and easy to install. Edge0 does something different: it runs models that do not fit, by streaming the parts it needs. If a 27B or 35B model is good enough for your work, the established tools are the simpler choice. If you want a model in the 120B to 145B class on hardware you already own, Edge0 is one of very few open projects attempting it.
The trade-offs are real. It is young (the first release was on 8 September), the new models depend on a forked engine, and speed will depend heavily on your SSD. Expect rough edges.
The licence, which matters here
The framework code is under Apache-2.0, and bundled components keep their own licences. The models are different. Edge0's own model contributions are under the Edge0 Community Model License 1.0, which allows personal use, research, education, modification, internal use and commercial use of the outputs. It requires separate written permission for selling or distributing the models for money, for commercial services that offer inference or fine-tuning, and for independent commercial coding or office assistants. The Qwen materials the models are built on remain under Qwen's own licence.
So you can use it inside your business and sell what it helps you make. You cannot build a paid product around the model itself without talking to them first.
Who should care
I'd try it if you run agents on private code or client data and want a larger local model than the usual 30B class, or if you already own a 24GB NVIDIA card or a 64GB Mac and pay for hosted tokens every month. I'd wait if you need a stable, supported setup for a team, or if your product would count as a commercial assistant under the model licence.
The bigger point is about where useful work in open AI is happening. A team most people have not heard of is running a 145B-parameter model on one consumer GPU by treating the SSD as part of the memory, and that idea, if it holds up under independent tests, changes what "fits on your machine" means.

