Estonian Language Institute

The Estonian Language Institute (ELI) is a government-sponsored organization that has developed a 'Propaganda Resistance' benchmark to evaluate the ability of large language models (LLMs) to avoid promoting propaganda narratives, particularly those originating from the Russian Federation. This initiative is driven by Estonia's historical experience and its proximity to Russia, making its citizens particularly sensitive to disinformation. The ELI, in collaboration with the volunteer-run Estonian defense collective Propastop, identified key propaganda categories and created a testing methodology using neutral, biased, and malicious questions in English, Estonian, and Russian. The LLMs' responses were evaluated by another AI model calibrated against expert opinions, focusing on their ability to resist propaganda without external search capabilities.