Source record

Published by arXiv cs.MA (T1 source). The original is at https://arxiv.org/abs/2608.26295.

Pipeline notes

The summary and note below are generated by the signal pipeline — they are Beyond Desk’s reading, not quotations from the source.

MemToC is a controlled benchmark for evaluating how LLM agents arbitrate between parametric memory and tool returns when the two conflict. Across five 7-9B open-weight models, instruction-tuned variants retain a verified-correct answer against an incorrect tool in only 6.5-17.1% of cases. SFT and DPO improve arbitration on some backbones but reduce abstention after tool errors.

Quantifies a specific failure mode in tool-using agents and offers a benchmark plus fine-tuning baseline for anyone designing agent arbitration policies.

Cited by

No citations on record.