• 0 Posts
  • 12 Comments
Joined 3 years ago
cake
Cake day: June 21st, 2023

help-circle
  • Cheaper than expected given the economic hellscape, still more than the Index. Oh well. Glad they are attempting to prevent scalping, sad that it means that there’s a random draw to be able to buy them. Still trying to buy one. I figure the 256GB model will be entirely fine, especially with the SD card expansion slot. Pity that you can’t upgrade the storage like the Deck, but oh well.




  • Ryzen 5950X with a slight underclock
    2x32+2x16 DDR4 2666
    Radeon 7900XTX 24GB with a 250W power limit
    15tps is entirely usable as an agent, but I can’t go above 131k ctx and must set --parallel 1 to fit in available RAM+VRAM. It actually still OOMs periodically, as even with nothing else running it only has a couple GB to work with. If that happens with context mostly filled then you’re looking at ten minutes of prompt processing before next token.

    I’m not about to buy another 64GB pair to replace the 16s, but… I wish I had done that years ago.


  • This is exactly why I am so sad that I didn’t buy more DDR4 back when it was reasonable. I run Unsloth’s Qwen 3.5 122B A10B UD_Q4_k_XL and while it works great, I really wish I had enough ram for Q6 or even Q8. The speed difference won’t be wildly worse, but the quality of output is noticeable. I’m just glad that it works as well as it does in Q4. It’s mostly limited by my main ram bandwidth, the GPU helps but I only barely hit 15t/s decode with MTP hitting >80%.