#LLM Efficiency
5 articles with this tag
SSM RAG Prefill Speedup Shatters Limits
SSM RAG prefill speedup slashes latency by 4500x on edge hardware, enabling interactive AI by pre-computing context.
LOCKS Unlocks LLM Long Context Efficiency
LOCKS revolutionizes long-context LLMs by approximating KV cache with spectral summaries, achieving near full-quality inference while drastically cutting latency and computation.
From LLM APIs to Local Neural Artifacts
Fuzzy-function programming enables compiling LLM-powered functions locally, matching large model performance with minimal resources.
AdaCodec: Efficient Video MLLM Encoding
AdaCodec revolutionizes video MLLMs by using predictive visual coding to drastically cut tokenization costs and latency, achieving superior performance at a fraction of the budget.
DMax: Parallel Decoding for Diffusion LLMs
DMax revolutionizes diffusion language models with Soft Parallel Decoding, boosting TPF significantly while preserving accuracy and achieving 1,338 TPS.