PARLEY: Evaluating LLM Collaboration Through Cooperative Asymmetric Puzzles
4.60T1 sourcearXiv cs.MA
Source record
Published by arXiv cs.MA (T1 source). The original is at https://arxiv.org/abs/2611.14242.
Pipeline notes
The summary and note below are generated by the signal pipeline — they are Beyond Desk’s reading, not quotations from the source.
SummaryPARLEY evaluates pairs of GPT-4o agents on four asymmetric puzzles covering referential communication, joint coordination, perspective-taking, and spatial reasoning. Across 400 conversations, task success varied from 34% to 80%. A six-type failure taxonomy identifies perception, adaptation, and coordination problems, while a taxonomy-guided intervention raises average success to 70.5%.
Why it mattersProvides a concrete test design for separating perception, adaptation, and coordination failures before choosing prompts, role assignments, or verification steps.
Cited by
No citations on record.