Source record

Published by Hacker News · Show HN (50+ points) (T2 source). The original is at https://github.com/cactus-compute/cactus-hybrid.

Pipeline notes

The summary and note below are generated by the signal pipeline — they are Beyond Desk’s reading, not quotations from the source.

Cactus post-trained Gemma 4 E2B with a 68k-parameter probe layer that reads intermediate hidden states to predict response correctness, returning structured confidence scores for routing between on-device and cloud inference. The probe averages 0.814 AUROC across 12 benchmarks, including zero-shot audio tasks. Weights, code, and runtime patches are published under MIT.

Concrete mechanism for hybrid AI routing that avoids self-assessment parsing and token entropy heuristics. Weights and runnable code across multiple inference backends are openly available.

Cited by

No citations on record.