Multi-turn Evaluation — References

Material on measuring exchanges rather than responses, on simulated participants, and on what happens to earlier context as a conversation lengthens.

Measuring an exchange

• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (Zheng et al., 2023), arXiv:2306.05685 — cited here for its two-turn construction and the finding that the second turn is where models separate, which is the smallest useful demonstration that single-response measurement is incomplete.

Simulated participants

• The user-simulation literature from task-oriented dialogue systems, which long predates language-model products, is the established source for goal-driven simulated users, agenda representations and the validity question of whether a simulator's conversations resemble real ones. Described as a body of work rather than attributed to a single title.
• The benchmark designs that pair a simulated customer with a written policy and check the resulting system state are the current reference for combining conversational behaviour with a checkable outcome. Described rather than named precisely, to avoid attributing a title inaccurately.

Why late turns degrade

• Lost in the Middle: How Language Models Use Long Contexts (Liu et al., 2023), arXiv:2307.03172 — the measured positional effect on how material in a long context is used, which is the mechanism behind a constraint stated early being dropped once the exchange grows.

Multi-agent conversation harnesses

• AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation (Wu et al., 2023), arXiv:2308.08155 — conversational coordination between model-driven participants, relevant here as the machinery a user simulator is usually built on.

No verified reference is recorded for first-failure turn identification, for retention assertions at controlled conversational distance, or for within-scenario variance reporting in multi-turn suites.