You don’t need to worry about the split in strix halo, it runs a unified memory architecture so Linux will dynamically allocate VRAM on demand up to maximum possible (which would be around 120GB I think)

I’m on an nvidia system so can’t answer the rest of your questions other than to say that in my experience ollama gave me a much worse tokens per second speed compared to running the same model using llama.cpp, so if you get something slower than you expect it might be ollama caused.

Good luck with your £250 system 😉

I’ve been testing this model since yesterday through the Qwen API in opencode and deepseek harness. I actually prefer it over GLM-5.3-Flash! it’s super good at agentic coding and comfortably fast, now my goal is to be able to one day run it locally 😳 I think a Strix Halo with 128GB and a fast nvme should be able to run it well at near loseless quant

llama.cpp support for it got merged a few hours ago 🏆

This is better than GLM-5.2 which just 2 months ago was my go to high tier model!

Also in OpenRouter this model is listed to have a maximum context of 1.31M tokens! this is the first high-end model to my knowledge that supports more than 1M tokens context 🤯

You might want to try running Ling-3.0-Flash (124B-A5.1B) it might be perfect for your M4 and should perform close to the level of DeepSeek V4 Flash preview or MiniMax M2.7

Yeah I tried Kat Coder when it came out, I don’t remember the exact issue I ran into but I identified a definite regression in my specific use cases compared to using vanilla. Ornith seems to be equivalent or better so far 👍

My previous experience with finetunes is bad…

Today I tried Ornith-1.5-35B-A3B, and I’m quite positively surprised by it… I compared it to both Qwen3.7-Flash and MiMo-V2.5 (via OpenRouter) in a few coding tasks and Ornith seems to be better than both! Thinks much more but the result definitely speaks for itself.

Hoping for Qwen3.8-35B soon 🤞

I’ll add my vote to keeping the 32+8 system as that has the most ram.

I also have an 8GB RTX 3070 Ti and 32GB RAM that I run Qwen3.6-35B-A3B perfectly on (30+ t/s) and looking forward to a possible Qwen3.8 35B update.

Recently bought a used 8GB RTX 3070 for a bargain, planning to combine it with the 3070 Ti and hopefully run a lower quant Qwen3.8-27B which is crazy good for its size.

Cheers for the new local GOAT!

Also on ModelScope in case you-know-who bribes huggingface takedown because model is too dangerous.

free to use if you have the hardware. For this model because of its size, the main problem is the hardware availability/cost. But in general there are 3 ways to run an open weights model:

• pay a provider like DeepSeek/OpenRouter per usage
• buy hardware that can run it locally: actually not a bad investment for a business
• rent hardware that can run it from a cloud provider, hardware can be rented dedicated or time-shared (so called serverless).

Update: GLM-5.3 dropped today, open weights release in ~2 weeks!

Everything runs well if one is patient enough 😂

But yeah, with a smaller context and carefully offloading some of the layers to the CPU, it’ll run just not very fast. And definitely using Q4 quants.

No you don’t need a data center, this can run on any current gaming PC with a GPU that has 16GB+ VRAM.

Although to get good token generation speed and full context you need at least 24GB VRAM.

Appears they removed the branch and changesets

I only have 8gb vram and can run 35b-a3b perfectly (30+ tps), have you tried 35b?

Wow, got about 9 repos of mine there and a few from my startup’s, only public ones are scraped so I say fair enough. Happy to have switched to running my own git infra a year ago

I hope their decision to opensource Qwen3.8-Max means this version is going to be another open-weights frontier contender 👍

Minecraft with factory modkit for me