I have created a tool to observe what happens inside an LLM. Also allows for the comparison of two quantizations of the same model to see at what point they diverge, token by token
2.55T2 sourcer/LocalLLaMA
r/LocalLLaMAoriginal source ↗
Source record
Published by r/LocalLLaMA (T2 source). The original is at https://www.reddit.com/r/LocalLLaMA/comments/1xx7qbf/i_have_created_a_tool_to_observe_what_happens/.
Pipeline notes
The summary and note below are generated by the signal pipeline — they are Beyond Desk’s reading, not quotations from the source.
SummaryThe author presents a tool for inspecting an LLM’s internal processing and comparing two quantized versions of the same model to identify the token where their outputs diverge.
Why it mattersToken-level quantization comparisons could help readers diagnose subtle model changes and select appropriate quantization levels.
Cited by
No citations on record.