LinkedIn says its user-understanding agent now improves itself, as Ziwei Guo and Saloni Gandhi described on AI Engineer. The obvious reading is an agent that fixes its own mistakes. I do not buy that reading yet.
Guo said the team built a seven-step loop around a career agent that turns a member ID plus search history, profile and preference data into fields like preferred job titles with confidence scores, citations and reasoning. Guo said traces go to an L1 judge scored 1 to 3 across 15 dimensions, then to reflection that clusters failures, assigns root causes, merges duplicates and ranks issues, then to proposed prompt edits that are validated on a holdout set before becoming the new baseline. Gandhi said the first loop-generated prompt change passed human review and quantitative checks before reaching production.
The loop runs. The judge does not earn trust.
Gandhi said analysis of 180 to 200 human reviews found full agreement at about 55% for score 1 and 57% for score 2, rising to about 80% for score 3, which means the judge recognized good outputs far better than it sorted bad from mediocre. Gandhi said agreement fell to about 47% on domain expertise and 50% on work type, with the judge often accepting a plausible title while missing omitted intent signals and confusing domain with function and skill, as when software engineering, machine learning engineer and machine learning were scored without consistent partial credit rules. That is fatal for automation because every later edit inherits the same rubric error.
For the obvious reading to hold, evaluation would need to be objective and stable enough to drive edits without people in the middle. Guo said evaluation must be objective and aligned with user feedback, yet the current focus is only prompt optimization with weekly or biweekly runs and human-in-the-loop production updates, while workflow redesign, harness code and optimizer code remain future rungs on what Guo called the ladder. This is narrow next to the stronger industry definition of recursive self-improvement as an AI system capable of fully autonomously designing its own successor, per Anthropic, not a prompt tuner supervised by annotators.
To be fair, the LinkedIn team reported the failure instead of hiding it and used the disagreements to tighten rubrics and study judge variance. If you want this pattern, budget for 200 human labels before you trust the loop.
