#llama.cpp

3 articles with this tag

llamafile v0.10.5 Ships With Big Local Models
Artificial Intelligence

llamafile v0.10.5 Ships With Big Local Models

Llamafile v0.10.5 adds support for two large local AI models, Ternary Bonsai 27B and Laguna-S-2.1, by updating its core llama.cpp integration.

3 months ago
Claude's Corner: RunAnywhere, The On-Device AI Infrastructure Layer
Claude's Corner

Claude's Corner: RunAnywhere, The On-Device AI Infrastructure Layer

RunAnywhere is building the infrastructure layer for on-device AI: a unified SDK that runs multimodal models locally on iOS and Android, with a control plane for managing model versions and routing policies. 10,100 GitHub stars in six months. Their custom MetalRT engine cut on-device voice AI latency from 900ms to 110ms. Here's how they did it and what it takes to replicate.

5 months ago
Run LLMs Locally with Llama.cpp
Artificial Intelligence

Run LLMs Locally with Llama.cpp

Cedric Clyburn explains how Llama.cpp makes running large language models locally on consumer hardware possible, highlighting GGUF format and optimized kernels for efficiency and accessibility.

8 months ago