Tech Times

OpenAI Halves Inference Costs With Software Alone: GPUs Drop to Hundreds

OpenAI inference cost reduction cut ChatGPT guest traffic from tens of thousands of Nvidia GPUs to just a couple hundred, ...

OpenAI engineers cut ChatGPT guest traffic to a few hundred Nvidia GPUs, with no new hardware deployed.

OpenAI inference cost reduction cut ChatGPT guest traffic from tens of thousands of Nvidia GPUs to just a couple hundred, using software optimization alone. Engineers achieved more than 50% savings ...
Tech Times

LLM Data Mixture Breaks When Training Pools Shift: Causal Inference Offers Fix

LLM training data mixture optimization breaks when training pools shift — every prior proxy experiment becomes stale.