OpenAI's Vijaye Raji on the architecture behind a $1B ads business

At ICONIQ Frontier, Vijaye Raji, CTO of Applications at OpenAI, gave a look inside how OpenAI is building new product layers without changing the model users actually talk to.Take ads. The main model generates its response without knowing ads exist. Only afterward does a separate LLM decide whether the conversation is an appropriate place for one, keeping the two experiences deliberately isolated.

For Vijaye, that architecture points to a broader shift. Vijaye believes routing is becoming its own layer of intelligence, with smaller models deciding where requests should go, what context matters, and how to protect sensitive information along the way.Codex shows how quickly that future is arriving inside OpenAI itself. Among the company’s top 10 engineers, token use increased roughly 400x in six months, pushing compute spend to levels Raji said can now exceed headcount by about 10x.