LLM InfraTalk (25 min)Intermediate
The Inference Bill: A Postmortem
Monday, October 12 · 11:00 AM to 11:25 AM PDT · Mission Hall
We cut inference spend by 71 percent over two quarters. Most of it was not model selection.
- Track
- LLM Infra
- Format
- Talk (25 min)
- Level
- Intermediate
- Room
- Mission Hall
Speaker
- ML Platform Lead, Northbeam
Rafael runs inference for a few hundred million requests a month and would like to talk about the bill.