OpenAI published a second account of its Habitat storage platform on October 23, describing the tradeoffs behind storing application data for ChatGPT and other products. The engineering report focuses on Azure Cosmos DB and the difficulties of keeping shared infrastructure reliable as workloads change.
One of the central choices is how many customers or workloads should share a database container. Sharing can make capacity easier to use efficiently. A particularly demanding workload can also affect the resources available to its neighbors.
The report traces attempts to rely on database primitives, move toward isolation, and then use shared infrastructure by default with the ability to move data when needed. That last capability makes the sharing decision less permanent.
A design that works well for the average account still needs a path for moving an unusually demanding customer.
The operational burden, traffic pattern and cost of such a design need to match the product. Moving data safely introduces its own work, including checking that reads and writes continue to behave as expected during the transition.
OpenAI describes its own engineering choices and failure modes in the report.
Microsoft's partitioning documentation explains the constraint behind part of the story: an ordinary logical partition can store up to 20GB, and uneven demand on partition keys can produce hot partitions and rate limiting. Splitting a physical partition does not split the data for a single logical key. Hierarchical keys offer another design route, with their own query-planning tradeoffs.
That means a growing database and one unusually demanding tenant are different scaling problems. More total capacity can coexist with a bottleneck concentrated on one key. The distribution of storage and request demand across keys can reveal contention that the container's combined size does not show.
The design changed in three stages
| Stage in the report | Approach | Problem to examine |
|---|---|---|
| I | Rely on Cosmos DB primitives | Partition limits and demanding neighbors |
| II | Try workload isolation and single tenancy | Capacity planning and uneven growth |
| III | Share by default, with data movement | Move exceptional workloads without isolating everyone |
For an existing application, the choice of isolation mechanism depends on which tenant or operation causes the contention. Data movement can relieve an exceptional workload, but its implementation and operating cost still have to be justified by the bottleneck it resolves.
