Xylight@lemdro.id to LocalLLaMA@sh.itjust.worksEnglish · edit-21 个月前My 8gb vram system as i try to load GLM-4.6-Q0.00001_XXXS.gguf:media1.tenor.comimagemessage-square13linkfedilinkarrow-up188arrow-down15
arrow-up183arrow-down1imageMy 8gb vram system as i try to load GLM-4.6-Q0.00001_XXXS.gguf:media1.tenor.comXylight@lemdro.id to LocalLLaMA@sh.itjust.worksEnglish · edit-21 个月前message-square13linkfedilink
minus-squareffhein@lemmy.worldlinkfedilinkEnglisharrow-up1·30 天前Ah, multiple GPUs? For some reason I thought you meant that with exllamav3 you had managed to load a model which was larger than your VRAM.
Ah, multiple GPUs? For some reason I thought you meant that with exllamav3 you had managed to load a model which was larger than your VRAM.