Large language models can hold an answer and still not give it. A model may sandbag on a capability evaluation or answer against what it internally knows, and outputs alone cannot tell whether it is hiding knowledge or simply lacks it. In "A Lie Detector Test for Language Models: Reading Knowledge a Model Won't Reveal," researchers borrow the Concealed Information Test from forensic science to close that gap. Their method, Probe of Internal Knowledge, presents the model with the true detail hidden among plausible decoys and measures a stronger internal response to the item it recognizes, revealing guilty knowledge even when the generated text conceals it.
The approach reframes evaluation from what a model says to what it recognizes. Instead of relying on prompted answers that can be steered or suppressed, the test probes internal signals for familiarity with the correct detail. For teams training, auditing, and shipping models, that matters because sandbagging, alignment faking, and strategic underperformance all look identical to ignorance at the output layer. A detector that reads knowledge the model will not reveal offers a more reliable check on capabilities, safety claims, and deployment readiness than accuracy on a benchmark alone.
When correct outputs still hide flawed reasoning
Two complementary lines of work show why output-only checks fall short. "LogicTrack: Auditing Reasoning Trajectories of Large Language Models with Formal Logic Solvers" tackles chain of thought that reaches the right final answer through logically invalid intermediate steps. LogicTrack auto-formalizes each reasoning step and checks it with a solver, auditing the trajectory rather than just the outcome. "An Interpretable Memory Decision Controller for LLM Agents Based on Three-Signal Complementarity: Decoupling Confidence and Consistency" tackles a different failure mode in retrieval augmented generation. Standard retrieval blindly injects memories and can amplify hallucinations when the memory store contains conflicting positions, with hallucination rates markedly higher than a memory-free baseline. Its controller learns when retrieved memories should be trusted by decoupling confidence and consistency signals.
Together, the three papers point to a single shift in how to evaluate modern systems. Whether knowledge is hidden, reasoning is unsound, or memory is conflicted, reliable assessment requires looking inside the process, not just scoring the final text. For builders, that means adding internal probes, trajectory audits, and trust decisions to the evaluation stack before a model is trusted in production.