Nikolay Petrov
AI Engineer speaker at Intel
About Nikolay Petrov
Nikolay Petrov works at Intel and spoke at AI Engineer in November 2026 on evaluating agent skills. His talk, "Your SKILL.md Is 70% Stuff the Model Already Knows", presents a method for finding which parts of a skill give a model new knowledge and which repeat what it already has.
The work covers popular public skill repositories, including Anthropic's skills, measured across several Claude models.
Notable Positions
On Scoring Skills Atom by Atom
Petrov splits a skill into around 25 to 30 items, each a question with an expected answer, and asks each with and without the skill loaded. A judge, ideally from a different model family, scores the answers. Each atom lands in one of four outcomes: already known, newly taught, made worse by the skill, or not learned either way.
On General Knowledge Versus Exact Details
Frontier models know the concepts in most skills but far fewer of the specifics. In his measurements Opus fell from 70% on a lenient check of general understanding to 22% on a strict check of exact keys, parameters, values and steps. His conclusion is that skills should carry the exact details.
On Evals as the Final Check
He describes the novelty score as a starting direction for optimizing a skill. After removing what the model knows, the task evals have to be rerun to confirm the result holds. See Skill Engineering.
Key Quotes
- "Teach model something new." (on what a knowledge skill is for)
- "It's basically initial vector for optimization." (on novelty scores)
Related Reading
- Intel's Nikolay Petrov: Measure What Your SKILL.md Adds - Full breakdown of the talk
- Agent Skills - What skills are and how agents load them
- Skill Engineering - Designing and maintaining skills