Cloud Bill Shock: How Batching Hurts LLM Apps in the Wallet

I was working on cost optimization for an LLM-based document translation At that point, the LLM translation flow was still very direct: one extracted It worked, but it was not ideal for cost. For a document with many text segments, the number of API calls grew linearly. In simpler terms: Instead of