GPT-3.5-Turbo's accuracy plummets to 50% with lengthy prompts.

GPT-3.5-Turbo drops from 90% accuracy to 50% when the answer sits in the middle of a 20k-token prompt instead of the start or end. Liu et al. (2023) documented this in "Lost in the Middle: How Language Models Use Long Contexts" at ACL. The edges of your context window are prime real estate. The midd