Edge0 runs a 145B-parameter model on one consumer GPU by streaming it from your SSD
An open-source project with 12,000 GitHub stars now runs 145B and 120B mixture-of-experts models on a single NVIDIA card or a 64GB Mac. What it does, how to try it, and what the licence allows.