Open-source Large Language Models (LLMs) have become much easier to run. But one thing is often misunderstood: Running a model locally is not the same as serving a model efficiently.
Ollama gained traction by being the first easy llama.cpp wrapper, then spent years dodging attribution, misleading users, and pivoting to cloud, all while riding VC money earned on someone else’s engine.
He shares that the people behind llama.cpp don’t act poorly (at least in those regards). My experience confirms llama.cpp can run just about any gguf while ollama can only run those that have been customized for ollama, and llama.cpp seems faster (no evidence, just anecdotal).
Here the article that spurred my change: Friends don’t let friends use Ollama
He shares that the people behind llama.cpp don’t act poorly (at least in those regards). My experience confirms llama.cpp can run just about any gguf while ollama can only run those that have been customized for ollama, and llama.cpp seems faster (no evidence, just anecdotal).