ggml.ai joins Hugging Face to ensure the long-term progress of Local AI
Background
ggml org is the parent of llama.cpp, which is the backbone of a a large number of local AI projects and was itself responsible for the spark that started the whole local LLM movement by allowing people to run Llama models on their consumer hardware when such a thing seemed completely impossible. ggml and gguf are basically a container format and spec for for quantized model weights. Georgi Gerganov is the founder and leader of that organization.
Huggingface is a start-up which was the first mover in distribution and development tools for local AI models, with an as-yet-unknown monetization strategy. They basically give away obscene amounts of bandwidth and storage space for anyone to host models, and provide top notch documentation and tooling to develop and run AI, specifically the transformers library and the safetensor weights standard.
As a metaphor, if we were talking about video playback, safetensors and transformers would be a the raw 4K data on a Blu-ray disc along with the hardware Blu-ray player, with ggml being the MPEG/VCEG org creating AVC/H264/HEVC and gguf being mp4 or mkv containers which house them.
The news
The ggml org has decided to come under the wing of huggingface. I view this with a deserved amount of cynicism because of the proven track record for start-ups giving shit away and latching on to the open source ecosystem to turn around and bite the hand that feeds when the founders decide they need to look for an exit, or when they get acquired by a FAANG.
ggml is really the lone pillar in this comic with regards to local AI. If it goes down, I don’t know anything that can replace it.
Let’s all who care about the ability to run AI free from the control of the big 4 light a stick of incense to our deity or non-deity of choice in the hope that ggml might make it out of this unscathed.