A 638 MB file claims to turn a mid-sized open model into a coding agent that fixes 15 more of the 51 hardest SWE-bench Pro tasks than it managed before. If that holds up, a small team can run a capable coding agent on a machine it owns, with no code leaving the building and no per-token bill.
My take is simple. I'd try it for a private coding agent on my own hardware, and I'd wait for someone else to reproduce the numbers before I trusted them. Both halves matter, and the rest of this post explains why.
What the adapter does
The release is Qwen3.8-27B-pi-adapted, a LoRA adapter for Alibaba's Qwen3.8-27B. A LoRA (low-rank adaptation) is a small set of extra weights trained on top of a frozen model, so you download the base model once and bolt the adapter on. This one was trained with reinforcement learning, meaning the model attempted thousands of real software tasks and was rewarded when a verifier said the task was done, plus a small bonus for using tools efficiently.
It is unusual in what it was trained for. It was not trained to be better at coding in general. It was trained to be better at driving one specific agent harness, pi, an open-source coding agent CLI that its GitHub page describes as an "AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI". The harness is the program around the model: it decides how tool calls are formatted, how files are read and edited, and how many steps the agent gets. pi is MIT-licensed and showed about 118.2k stars and 15.1k forks on GitHub as of 22 Nov 2026.
The model card reports, with thinking turned off and one attempt per task, a jump on the 51 hardest SWE-bench Pro tasks from 39.2% to 64.7% (15 tasks newly solved, 2 lost), and on 86 graded Terminal-Bench 2.1 tasks from 39.5% to 55.8% (16 newly solved, 2 lost). It also says the adapter needs under half the turns of the base model on tasks both solve, and generates about 4.5 times fewer tokens than the base model running with maximum-effort thinking. In plain terms, the claim is that the adapter gets most of the benefit of "thinking hard" without paying for the thinking.
Who is behind it
The adapter sits under the harness-hub organisation on Hugging Face, and the card names no individual authors. The companion write-up, "Introducing harness adapters", was published on the Hugging Face blog on 20 Nov 2026 with ten named contributors, among them Lewis Tunstall and Sergio Paniego, and the card says the training script lives in Hugging Face's own TRL library. So this is Hugging Face research, released as open weights with the recipe attached.
Interest is real but early. As of 22 Nov 2026 the adapter showed 939 likes and about 2.4K downloads on its Hugging Face page, and the quantised GGUF version about 5.5K downloads. Downloads measure attention and say nothing yet about quality.
How to try it
Start with hardware, because it decides everything else. Merged into the base model, the checkpoint is about 54 GB in BF16 (16-bit) weights, according to the card. My estimate, not the card's, is that serving that with vLLM and the 131,072-token context the evaluations used means an 80 GB-class data-centre GPU or a pair of smaller ones. If you have a large-memory Mac or a workstation instead, the card's Llama.app example pulls a 4-bit base model of about 17 GB and loads the adapter on top.
The card's GPU route, as written, has three steps:
Serve the base model with the adapter in vLLM: vllm serve Qwen/Qwen3.8-27B --enable-lora --max-lora-rank 32 --lora-modules qwen3.8-pi=harness-hub/Qwen3.8-27B-pi-adapted --language-model-only --enable-prefix-caching --mamba-cache-mode align --enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3 --default-chat-template-kwargs '{"enable_thinking": false}' --max-model-len 131072. The card notes that without --mamba-cache-mode align, every agent turn re-reads its whole history.
Install pi (it needs Node.js 22.19 or newer): npm install -g --ignore-scripts @earendil-works/pi-coding-agent.
Add a provider called local to ~/.pi/agent/models.json with "baseUrl": "http://localhost:8000/v1", "api": "openai-completions" and a model id of qwen3.8-pi, then run pi --provider local --model qwen3.8-pi --print "Find out why the tests in tests/test_parser.py fail and fix the bug."
Two settings are easy to get wrong. Keep thinking off, because that is what the adapter was trained for, and keep "api": "openai-completions", because the card says that without it pi uses the Responses API, which the adapter was neither trained nor evaluated with. For the llama.cpp route, the card suggests a Q8_0 base model as a good default and warns that lower-bit quants drift further from the trained behaviour.
How it compares
The fairest comparison is with the same base model under the same harness, and that is what the card gives. Qwen3.8-27B with maximum-effort thinking scores 74.5% on the hard SWE-bench Pro subset under pi, ahead of the adapter's 64.7%, though the card says the gap is not statistically significant on 51 tasks. On Terminal-Bench 2.1 the adapter without thinking scores 56.6% against the thinking base model's 54.2%, measured on 83 tasks that all four runs graded. The cost difference is large: median generated tokens per SWE-bench Pro task were 48.8k for the thinking base model against 9.4k for the adapter. If you pay for compute by the hour on your own box, that difference shows up as speed rather than money, but it still matters.
Hugging Face trained sibling adapters for two other open harnesses. The blog reports the OpenCode adapter moving SWE-bench Pro (hard 51) from 35.3% to 51.0%, and the Hermes Agent adapter from 29.4% to 43.1%. The authors are explicit that "you cannot compare the results across different harnesses", so treat those as evidence that the recipe transfers rather than as a ranking. Do not compare any of this with the numbers on the Qwen3.8-27B card either; the adapter card says Qwen used other harnesses, maximum-effort thinking, larger budgets and the full SWE-bench Pro.
I'm not going to put this next to hosted frontier coding agents, because nobody has run them on the same 51 tasks under the same budget. What you can say is that the adapter competes on a different axis: it runs where you choose, under a licence you can read.
Limits and licence
Every number above is self-reported. As of 22 Nov 2026 no independent reproduction had been published in the sources checked for this post; an explainx.ai explainer summarises the claims but shows no runs of its own.
The card is unusually candid about its weak spots, which I count in its favour. Step 160 was picked by looking at checkpoints' scores on SWE-bench Pro and on an 18-task Terminal-Bench subset during training, so those results are not a clean hold-out. The 68 Terminal-Bench tasks that were not used to pick the checkpoint show a similar gain, 39.7% to 55.9%, but the SWE-bench Pro gain has no such check, and its confidence interval is wide at plus or minus 14.2 points. The headline has already moved twice: an earlier version of the card reported 29.4% to 62.7% before the team found nine base-model runs had failed because a network tunnel went down, and the thinking-on figures were rerun after a 16k output cap cut long reasoning short. The card also says part of the gain is simply finishing inside the 100-step budget, since 20 of the base model's 31 failed SWE-bench Pro attempts ran out of steps.
Beyond the numbers, the GGUF version has not been re-evaluated, quantisation below Q8_0 is untested, the vision side of the model is not served, and the sandboxes had internet access during evaluation (the card says it found no attempt to look up a task's upstream fix). On licensing, the adapter and the base model are both tagged Apache-2.0 on Hugging Face and pi is MIT, so commercial use inside a small business is allowed, with the usual notice requirements.
What small teams can take from the method
The blog's argument fits in one line: "Models are not fully optimized for all harnesses and the tasks you're doing." Each harness formats tools and context its own way, and a general model leaves some performance on the table in every one of them. The fix here was cheap by model-training standards. According to the card, each adapter trained on two NVIDIA B200 GPUs (one serving, one training) for 160 steps over 750 scale800 tasks interleaved with 300 TermGrade tasks, and the run was stopped at step 191 of 1,000 planned. What comes out is a 638 MB file that sits on top of the unchanged base model.
For a small team, the recipe is worth copying. If you run your own internal agent with its own tools, the same recipe (a verifier that says pass or fail, a set of real tasks, a LoRA on an open base) could teach an open model your harness specifically. We covered a related idea in Shopify's loop for training cheaper task models. You still need tasks with automatic checks, which is why evals for your agent come first.
What I'd do
My decision rule is short. If you already have a GPU server or a large-memory Mac and code you would rather not send to a hosted model, try the adapter this week on chores you can check: failing tests, small refactors, dependency bumps. If you are buying hardware just for this, wait.
Run the card's exact vLLM settings first, including thinking off and the completions API, so your results are comparable.
Pick ten real tasks from your own repo that have a test proving success, and run the base model and the adapter on each under pi.
Count solved tasks, turns and wall-clock time per task. If you want a feel for where agents lose money, our post on cutting coding agent costs lists the usual leaks.
Only switch to a lower-bit GGUF after you have a baseline, and rerun the same ten tasks to see how much it drifts.
Keep the agent away from production credentials until someone outside Hugging Face has rerun the benchmarks or your own ten tasks say it is reliable.
If an independent group reproduces even most of that 25-point gain, it will be a practical option for anyone running agents on their own machines, and until then your own ten tasks are the evidence worth trusting.

