Artificial Intelligence
OpenAI describes scaling Habitat on Azure Cosmos DB
OpenAI published a second instalment on how it reworked Habitat, its storage platform, to serve more than 1 billion users by changing how Azure Cosmos DB handled multi-tenancy and partitioning.
- OpenAI said Habitat had to support more than 1 billion users across 40 regions and manage more than 500 petabytes of data.
- The post said OpenAI scaled the underlying storage layer, Azure Cosmos DB, with help from the Azure team.
- OpenAI said it moved from a shared container approach to separate containers for some critical uses, then later returned to shared tenancy by default.
- The company said added controls for hot keys, request priority and serverless scaling reduced manual capacity work and had prevented provisioning and noisy-neighbour incidents for months.
OpenAI said Habitat was the storage platform behind its products and that it needed to answer in milliseconds at very large scale. The post said the team had to change how the system was built and run as usage, reliability demands and new product uses grew.
The article described an early approach based on a single shared Azure Cosmos DB container, with clients choosing partition keys directly. OpenAI said that led to collisions on common keys, interference between workloads and reduced throughput as data grew.
It then said the company tried separate containers for some important new uses. OpenAI said that reduced the blast radius of problems, but created more capacity-planning work and led to launch-day scaling issues when teams did not pre-provision enough throughput.
In the final phase, OpenAI said it worked with the Azure team to make shared tenancy work better. The changes it described included throttling the hottest keys first, prioritising interactive requests over batch work, and a serverless mode that could scale each physical partition to 10K RU/s.
Named in this story
Companies
- OpenAI
- said Habitat served its products and described the scaling changes
Organisations
- Azure team
- worked with OpenAI on the shared-tenancy changes
Products and systems
- Habitat
- the storage platform OpenAI built for product data access
- Azure Cosmos DB
- the storage service OpenAI scaled and reconfigured
How the source tells it
A technical vendor blog with mild boosterism and a problem-solution structure.
- vendor boosterism first-person praise of the platform’s scale, reliability and partnership with Azure
- urgency references to mission-critical traffic, explosive growth and launch-day capacity problems