Neural Magic Acquired; Red Hat, Nvidia, and AMD Race to Acquire AI Model Optimization Startups

Red Hat, a leader in open-source software, announced the acquisition of Neural Magic, a machine learning startup focused on AI model optimization. Neural Magic’s technology is designed to make AI models, and large language models (LLMs) run efficiently on standard CPUs and GPUs, minimizing the need for specialized AI hardware. Neural Magic's acquisition marks the fifth acquisition of a startup accelerating AI inference.

Founded in 2018 by MIT Research Scientist Alex Matveev and Professor Nir Shavit, Neural Magic enables LLMs to operate on off-the-shelf processors at comparable speeds to custom AI chips. This capability allows companies to deploy AI models on readily available hardware, enhancing efficiency and scalability.

“We’re thrilled to complement our hybrid cloud-focused AI portfolio with Neural Magic’s groundbreaking AI innovation, furthering our drive to not only be the ‘Red Hat’ of open source, but the ‘Red Hat’ of AI as well," commented Matt Hicks, president and CEO, Red Hat, in a press release today.

Neural Magic’s open-source project, vLLM, was instrumental in their decision. The vLLM library accelerates AI model deployment, supporting multiple hardware types and enabling rapid inference speeds, essential for business scalability.

This move follows Nvidia’s recent acquisition of Deci AI, another Israeli deep learning startup specializing in model compression to enhance the efficiency of generative AI models. Deci AI’s technology compresses models to operate more effectively on GPUs, providing a solution that reduces the hardware and energy costs associated with large-scale AI deployments.

AMD also bucked the trend with its recent acquisition of Mipsology, a French startup to strengthen its AI inference software capabilities

Microchip Technology followed suit with its acquisition of Neuronix AI Labs, a company specializing in neural network sparsity optimization.

Nano Dimension originally set the stage for these acquisitions in 2021 with its purchase of DeepCube, one of the first startups to focus on deep learning acceleration. DeepCube’s advancements in machine learning optimization paved the way for efficient AI model deployment across a variety of hardware environments.