AI Agents Need Skills: Martin Keen on LLM Tooling

In a recent IBM "think series" video, Martin Keen, a Master Inventor at IBM, breaks down the crucial concept of "AI agent skills." These skills are the building blocks that enable large language models (LLMs) to perform complex tasks by interacting with tools and external services. Keen illustrates how these skills are defined and why they are essential for creating reliable and functional AI agents.

Understanding AI Agent Skills

Keen explains that LLMs, while powerful, often lack the procedural knowledge to execute tasks in the real world. They "know a lot of facts," but not necessarily "how work actually gets done." This is where AI agent skills come in. These skills act as a bridge, providing the LLM with specific instructions on how to use tools or services to achieve a desired outcome.

The Structure of a Skill File

A fundamental skill file, according to Keen, is a simple markdown file with a name and a description. The name clearly identifies the skill, and the description explains when the agent should use it. For instance, a skill named "PDF Builder" might have a description like, "Use when the user asks to extract a PDF."

The full discussion can be found on IBM's YouTube channel.

Beyond these mandatory fields, skill files can also include optional components:

• scripts/: This directory can house executable code, such as Python or Bash scripts, that the agent can run to perform the skill.
• references/: This can contain static resources like templates or data files that the scripts might need.

These optional components allow for more sophisticated and dynamic skill implementations.

Progressive Disclosure in AI Agents

Keen introduces the concept of "progressive disclosure" as a way to manage the complexity of AI agent skills. This approach involves organizing information in layers, starting with the most essential details and revealing more as needed. He outlines three tiers:

• Tier 1: Metadata This includes the name and description of the skill, providing a high-level understanding of its purpose.
• Tier 2: Body This level contains the core instructions, such as the detailed steps or logic required to execute the skill.
• Tier 3: Optional Folders These can include scripts and references, providing the necessary executable code and data for the skill.

This tiered approach ensures that the LLM receives the right amount of information at the right time, preventing cognitive overload and improving efficiency.

Types of Knowledge for AI Agents

Keen highlights four key types of knowledge that AI agents can leverage:

• Model Context: This refers to the knowledge inherently present within the LLM itself.
• Tool Access (MCP): This is the Model Context Protocol, which allows agents to call external APIs and services.
• Factual Knowledge (RAS): This refers to Retrieval Augmented Generation, where agents retrieve relevant information from external knowledge bases to augment their responses.
• Procedural Knowledge (Skills): This is the knowledge of how to perform tasks, which is explicitly encoded in the skills files.

By combining these forms of knowledge, AI agents can achieve a much higher level of functionality and autonomy.

Building Trust in AI Agents

The ability to define and manage skills is crucial for building trust in AI agents. Keen emphasizes that understanding how an agent operates, what it can do, and when it can do it, is fundamental to user confidence. He notes that skills, particularly those that allow agents to interact with external systems or execute code, carry inherent risks such as prompt injection and tool poisoning. Therefore, rigorous testing and careful implementation are essential.

Keen concludes by drawing a parallel to human cognitive science, noting that humans also possess different types of memory – semantic, episodic, and procedural. By mirroring these structures in AI agent design, developers can create more capable and trustworthy AI systems.