OpenAI·Software Engineer·Onsite - System Design / Architecture
Jul 2026
OpenAI SWE interview that went deep on memory allocator design, starting from a basic first-fit implementation and pushing all the way through coalescing, best-fit optimization, and complexity trade-offs. Four parts total, each building on the last, and the follow-ups got pretty hairy toward the end.
- Design and implement a simplified memory allocator with malloc and free operating over a single fixed-size heap array. Walk from a correct first-fit implementation to an optimized one, covering block layout, splitting, and per-operation complexity.
- How should free handle adjacent free blocks, and what is the difference between external and internal fragmentation?
- Propose a best-fit optimization for the allocator. What data structure makes the lookup efficient, and why is a plain min-heap not a clean fit?
- Compare first-fit and best-fit on allocation time, metadata overhead, and long-run fragmentation behavior. When would you prefer each?
- How would you make the allocator thread-safe, starting from a global lock and then reducing contention?
- How do boundary tags enable O(1) coalescing without a prev pointer in the block header?
- How would you detect heap corruption, such as through canaries or header checksums, without unacceptable overhead?
“This is the kind of question where you think you know it and then the details eat you alive.” The rest of the author's notes on Software Engineer interview at OpenAI, Onsite - System Design / Architecture round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post