Source record

Published by arXiv cs.MA (T1 source). The original is at https://arxiv.org/abs/2610.11559.

Pipeline notes

The summary and note below are generated by the signal pipeline — they are Beyond Desk’s reading, not quotations from the source.

SWE-Journey is a benchmark for evaluating LLM coding assistants like Claude Code and Codex through long-horizon, multi-turn interactions. It uses a weak-to-strong synthesis pipeline for task generation and four user personas simulated by an agent. Models pass over 75% of tests with software architects but fewer than 25% with non-coders.

Quantifies a gap users sense but cannot measure: assistants work for developers, fail for everyone else. The persona-based failure breakdown and 'ask/find/fix right' framing offer a concrete diagnostic for tool selection.

Cited by

No citations on record.