The bug that halved a gateway's throughput without a single error
· 6 min read
Pfizer runs an AI gateway — the single service every internal tool talks to when it wants a model. During a routine upgrade check, its throughput fell by half. No errors. No failed requests. Nothing wrong on any dashboard.
They published what they found together with the LiteLLM team, and the cause is small enough to fit in a paragraph: someone wrote a setting that said don't encrypt this connection, and the system encrypted it anyway.