Rapidly Scaling Online Storage to Serve Over 1 Billion ChatGPT Users

OpenAI's engineering team details how it built and scaled Habitat, its internal online storage platform, from a Python client library to a distributed service now handling 70 million requests per second, serving over 1 billion weekly ChatGPT users across nearly 40 geographic regions and managing more than 500 petabytes of data. The platform scaled more than 10x year-over-year for three consecutive years, a rate the team describes as far exceeding conventional infrastructure planning cycles. A key architectural decision was converting Habitat from a client-side library into a standalone service to eliminate cross-team deployment coordination failures and establish centralized control over routing, access policy enforcement, audit logging, and data security. The team made a deliberate, time-bounded technical debt decision to retain Python despite its CPU and latency overhead, betting that OpenAI's own coding models (Codex, GPT) would eventually make a rewrite tractable — a bet they report has been validated. Managing asyncio scheduling delay emerged as the dominant tail-latency challenge at scale, requiring real-time event loop monitoring beyond standard CPU, memory, and network metrics. A second installment will cover multi-tenancy reliability, read-performance optimization, and the Azure Cosmos DB scaling partnership.