#Reliability

2 articles with this tag

Databricks Tackles LLM Inference Costs
Technology

Databricks Tackles LLM Inference Costs

Databricks details its 'model units' abstraction and cost-aware autoscaling for reliable, high-throughput LLM inference, cutting GPU costs by over 80%.

24 days ago
GitHub Battles Outages Amid AI Boom
Technology

GitHub Battles Outages Amid AI Boom

GitHub faces massive infrastructure challenges due to the rapid adoption of AI-powered agentic development workflows, leading to recent outages and a push for 30X scalability.

about 2 months ago