Unclear prompts cut the accuracy of LLM-generated code by 20 to 40%, Rajat Kotyal said during a talk hosted by AI Engineer on the Keto Specification Analyzer. Kotyal, who works in the automated reasoning group at AWS, demoed a check that runs before a coding agent writes code. The target is the spec file itself, not the output, so no remote attacker is needed for it to fail.
One example used two order rules that cannot both be true.
Kotyal put valid-but-wrong code at 60 to 90% after generation, with twice as many regressions once that code ships, figures he tied to public research. His pipeline first asks the model to clean wording, then converts English rules to SMT-LIB formulas for a satisfiability checker. If the solver finds two readings or a direct conflict, it sends the sentence back for another pass.
Kotyal said the team samples each translation repeatedly to test whether the model keeps making the same assumption. Those samples are grouped by logical equivalence to show distinct interpretations, then a human picks a fix or tells the system to skip the rule. He stressed there is no oracle for spec correctness beyond the user because the model doesn't hold that domain knowledge.
The Keto name already means other things in software and hardware. Keto Software says it connects strategy, capital and execution in one strategic operating system, while Keto-Mojo sells clinical-grade blood glucose and ketone tracking for home use. Neither product checks spec logic, which leaves the AWS project without a direct named rival in those results.
Kotyal did not give a dataset, a false-positive rate or a test on a production repo, so the figures describe the problem more than the fix. A controlled run that counts caught conflicts against human review would change that read. How many teams will stop an agent to resolve a spec gap before it codes?
