Uber's GitFarm Solves Monorepo Woes

Uber's sprawling codebase, managed within massive monorepos, presented a significant engineering challenge. Traditional Git workflows, involving multi-gigabyte repository clones and constant syncing, became a drag on hundreds of automation systems.

To combat this, Uber Engineering built GitFarm, a novel Git-as-a-Service platform designed to streamline operations for large-scale monorepos. It aims to eliminate the costly overhead associated with local Git checkouts.

The Monorepo Bottleneck

Cloning Uber's Go monorepo alone could take 15 minutes and consume substantial compute resources. Services needing access to multiple monorepos required even more, leading to significant storage and processing demands.

Even optimizations like shallow clones failed to address the root issue: upstream Git servers were overwhelmed by millions of individual clone and fetch operations. These requests forced servers to enumerate objects and stream data, creating a shared scalability bottleneck.

Beyond CI Limitations

While CI platforms like Jenkins and Buildkite offer caching, they proved heavyweight for Uber's needs. They required provisioning full build environments for simple Git commands and lacked a standalone API for Git operations.

This approach also led to redundant work, as agents fetched independently, and stale states accumulated on long-lived agents. Furthermore, Git operations were coupled to the build lifecycle, limiting programmatic access and pre-warming capabilities.

GitFarm's Service-Based Approach

GitFarm operates not as a Git SCM, but as a cloud-based Git client. It executes standard Git commands on behalf of other services within secure, ephemeral sandboxes.

These sandboxes leverage pre-warmed repository checkouts and container pools, dramatically reducing latency. Clients gain access to a full Git checkout in under 500 milliseconds.

Architecture for Speed

The GitFarm Gateway handles authentication, authorization, and routing requests to the Backend. The Backend maintains warm, up-to-date repository states by periodically fetching changes from upstream.

It manages a pool of isolated, pre-initialized sandboxes. When a request arrives, an available sandbox is provisioned with a repository checkout, minimizing per-request initialization costs.

This pooling model reduces the overhead of providing a ready sandbox and checkout to less than a second.

Enabling Complex Workflows

GitFarm supports multi-command workflows through a gRPC streaming API. This allows clients to execute sequences of Git commands within a single persistent session, preserving checkout consistency.

This persistent session model minimizes connection setup and environment initialization overhead for chained operations.

Specialized Clusters for Performance

The system employs clustering, grouping Backend nodes into specialized clusters for specific use cases. This prevents noisy neighbor effects and ensures predictable performance.

Uber operates both specialized high-throughput clusters and a generic shared cluster for lighter-weight integrations. Routing decisions are managed centrally by the Gateway.

Tangible Results

In production since early 2025, GitFarm has demonstrated significant improvements. One read-heavy service, previously running on six hosts, now uses GitFarm, drastically reducing compute needs.

Overall, GitFarm has reduced client-side resource utilization by over 80% while maintaining the flexibility of native Git semantics.