# Recursive Self-Improvement Meets a Strong Baseline Agent

Researchers already use bounded forms of recursive self-improvement (RSI) to train language models. A model can generate training examples or judge its own answers. The model uses that signal to update its weights, and the updated model provides the signal for the next round. [Self-Rewarding Language Models](https://proceedings.mlr.press/v235/yuan24d.html), [A Model Can Help Itself](https://arxiv.org/abs/2510.18814)

This article tests the thesis that a similar loop can improve an agent’s work while the underlying model stays fixed. In this form of recursive agentic self-improvement, a separate component reviews the agent’s actions and outcomes, then revises its instructions. The next round starts with those instructions and carries forward the evidence from earlier runs, including where a change made performance worse.

> The thesis that a similar \[RSI\] loop can improve an agent’s work while the underlying model stays fixed.

The counter-thesis is that a strong baseline agent may leave too little useful headroom for that process to justify itself. The baseline is the agent running with its original system instructions. The first runs show which requirements the baseline meets and which it misses. If it already meets the tested requirements, there may still be room to improve speed or cost. A revision can also add overhead or improve one measure without benefiting the whole workflow. This post examines that tradeoff through a series of experiments built around OpenAI models and APIs, using Codex to help engineer the surrounding system.

> A strong baseline agent may leave too little useful headroom for that process to justify itself.

We chose retail to see how the same agent handled different kinds of work within one business. A simulated online store gave us tasks with concrete outcomes, from finding a product within a shopper’s budget to reconciling a payout against the store’s records.

We tested twelve different retail tasks and, in a separate study, twelve variations of a return-and-exchange workflow. We measured correctness, task time, and model cost in both.

## Source Code

All open-source code used to produce this post is available on [GitHub](https://github.com/garystafford/recursive-agent-instruction-improvement).

%[https://github.com/garystafford/recursive-agent-instruction-improvement]

## Inspecting task outcomes with Promptfoo

We used [Promptfoo](https://www.promptfoo.dev/) to evaluate saved runs in both studies. Its built-in assertions and custom JavaScript checks examined the agent’s final reply, recorded actions and approvals, and the resulting store records. For each task, we applied the same checks across instruction versions.

Promptfoo’s trace view shows the sequence of model steps, tool calls, and approvals so we can inspect what happened during a run. The time and cost comparisons use the runner’s task timer and the original model usage records.

![Promptfoo displays recorded model generations, tool calls, and their timing within a retail task.](https://cdn.hashnode.com/uploads/covers/6abe4a89fbf68ad459d66120/8a12c82f-4f14-41d9-8883-25052c8065fc.png align="center")

*Figure 1. Promptfoo shows the model steps and tool calls behind a task result.*

We used the comparison view to see where an instruction revision changed the outcome. Each row aligns the three versions for the same task and pass, and the column headers show how many runs each version passed.

In one payout reconciliation task, the agent correctly explained a discrepancy in its reply. It recorded $160 as the amount settled that day. Promptfoo flagged the mismatch because the correct total was $100.

![Promptfoo compares both passes of the twelve retail tasks. Baseline passes 23 of 24 cases, Revision 1 passes 24, and Revision 2 passes 22.](https://cdn.hashnode.com/uploads/covers/6abe4a89fbf68ad459d66120/5af37674-d36f-41cf-8b00-3b3f33ff2a75.jpg align="center")

*Figure 2. Both passes of the twelve retail tasks in Promptfoo, with Baseline, Revision 1, and Revision 2 side by side.*

## Twelve different retail tasks

For the baseline, we gave the retail agent the same **system instructions** for every task, along with a workflow-specific request. Those instructions told the agent to check the store’s rules and records, get approval before actions such as placing orders or issuing refunds, and verify the outcome before reporting it.

### Baseline results by task

Each workflow had one scenario with its own simulated records and success checks. The table shows the baseline averages for each workflow. Each model step is a recorded model generation. Tool calls include business tool calls, scripted approval calls, and local commands. Total tokens include input and output across all model steps, including cached input, rounded to the nearest thousand.

| # | Workflow and expected outcome | Avg. model steps | Avg. tool calls | Avg. task time | Avg. total tokens |
| --: | --- | --: | --: | --: | --: |
| 1 | **Product selection.** Recommend a waterproof jacket within budget and the delivery deadline. | 2 | 3 | 20.6 s | 45k |
| 2 | **Purchase and checkout.** Apply the promotion and gift credit, charge the balance, and place the order. | 6 | 22.5 | 41.4 s | 166k |
| 3 | **Order modification.** Change the jacket size and address while preserving the belt. | 3.5 | 13 | 29.8 s | 95k |
| 4 | **Order cancellation.** Cancel fulfillment, refund the payment, and release stock. | 3 | 13 | 27.2 s | 95k |
| 5 | **Delivery exception.** Replace only the missing package using the existing investigation. | 3 | 12 | 26.0 s | 95k |
| 6 | **Return and exchange.** Exchange the jacket, refund the shoes ($35 to the original card and $15 to gift credit), and keep the belt. | 6.5 | 24 | 48.0 s | 159k |
| 7 | **Warranty service.** Book a covered repair after the return window. | 3 | 13 | 31.5 s | 95k |
| 8 | **Loyalty account.** Add missing points, reverse duplicates, and keep pending points. | 4.5 | 14 | 29.9 s | 104k |
| 9 | **Subscription.** Skip one delivery, change quantities later, and keep the discount. | 3.5 | 6 | 27.4 s | 83k |
| 10 | **Stock replenishment.** Account for demand and inbound stock and order the right pack quantity. | 4 | 11.5 | 26.9 s | 109k |
| 11 | **Product recall.** Quarantine the recalled stock and initiate customer follow-up without duplicates. | 4.5 | 14.5 | 37.6 s | 102k |
| 12 | **Payout reconciliation.** Explain the difference and record the unresolved adjustment. | 4 | 14 | 34.0 s | 99k |

### How we revised and tested the instructions

For each task, a local runner sent the request and system instructions to the OpenAI Agents API. The agent used the simulated store’s tools and records in a fresh hosted sandbox. The runner handled scripted approvals and saved the results for grading.

![The local experiment runner sends tasks to the hosted Agents API. Each run uses a sandbox with one retail tool service and fresh SQLite records. Promptfoo checks the saved results locally.](https://cdn.hashnode.com/uploads/covers/6abe4a89fbf68ad459d66120/01f461f0-d61a-4b1f-ad28-3e2350c61710.png align="center")

*Figure 3. The hosted setup for the twelve distinct retail workflows.*

We ran the twelve tasks with the baseline instructions and saved the evaluation results, task times, and model costs. A separate model call reviewed the instructions, the recorded runs, and those measurements, then wrote a complete set of revised instructions. The prompt put accuracy first and asked the model to preserve or improve the correctness of the agent’s actions and reports. It also asked for ways to improve speed and reduce cost while meeting the task and approval requirements.

We gave the revised instructions to the retail agent and reran the same tasks with fresh store records. The tools, business rules, approval requirements, and checks stayed fixed. For Revision 2, the model started from Revision 1’s instructions and reviewed the results available from both earlier versions.

![Four steps for revising instructions. Run the task, check accuracy and record time and cost, revise the instructions in a separate model call, then retest with fresh store records.](https://cdn.hashnode.com/uploads/covers/6abe4a89fbf68ad459d66120/6ed94803-fb21-4973-a4e5-f79d27861b89.png align="center")

*Figure 4. We applied the same checks before and after each instruction revision.*

GPT-6.1 Sol ran the tasks at low reasoning effort. The calls that wrote the revisions used the same model at high reasoning effort. The table compares results across instruction versions.

| Measure | Baseline | Revision 1 | Revision 2 |
| --- | --: | --: | --: |
| Tasks passed | 23/24 | 24/24 | 22/24 |
| Model steps | 95 | 90 | 86 |
| Tool calls | 321 | 298 | 298 |
| Local command calls, included above | 22 | 0 | 0 |
| Total task time | 760.6 s | 705.1 s | 835.3 s |
| Estimated task model cost | $1.142–$1.324 | $1.053–$1.233 | $1.070–$1.248 |

Task time runs from submission to the final answer and includes approvals. It excludes setup, grading, evidence collection, and cleanup. The cost estimates use recorded token usage and exclude hosting and the separate requests that wrote the revisions.

**Revision 1 cut local work.** The baseline used local commands to create output folders and save reports. The revision told the agent to return its structured answer directly while still writing any required business records. Local command calls fell from 22 to zero, total task time fell by 7.3%, and the upper estimate of model cost fell by 6.8%. Revision 1 passed every check. The baseline correctly changed a subscription but attached the future quantity update to a single delivery cycle in its structured reply.

**Revision 2 reduced progress text but took longer overall.** It asked for shorter updates and more concise reasons for actions requiring approval. Compared with Revision 1, progress text fell 24%, yet the agent generated 3% more output tokens and total task time rose 18.5%. In one case, the agent made the right order changes but assigned the belt’s “unchanged” result to the whole order. It also failed the payout check described in the evaluation section. Revision 2 passed both cases when repeated.

Writing the two instruction revisions for the twelve-task study cost an estimated $1.45 and $2.85 in model usage, before the cost of running and checking the tasks. Whether the loop pays for itself depends on how often its useful changes hold up in practice.

### When the reviser kept the instructions unchanged

In a follow-up experiment, we ran the same twelve retail tasks with GPT-6.1 Terra, a lower-cost model in the same family as Sol. Both models used `low` reasoning effort. The baseline passed every check on store records and approvals, but three structured reports failed. Revision 1 clarified how to report a verified price, use the exact record ID, and distinguish a new action from one already on record. When we repeated the cases, all twelve passed.

For Revision 2, the reviser considered shortening the reporting guidance and removing some record checks. The guidance addressed the three errors we had observed. The checks served different purposes. Reading a record before an action helped the agent decide what to do. Reading it afterward helped verify what happened. The revision protocol lets the reviser keep the current instructions when the results don’t support another change.

> The revision protocol lets the reviser keep the current instructions when the results don’t support another change.

## Twelve variations of one task

The retail return-and-exchange study kept the workflow fixed and varied the customer request. A request covering several items shows how many different decisions can fit into that one workflow.

### Five items, one customer request

In one example, a customer wanted to exchange a jacket, get refunds for shoes and earphones, and keep a second jacket. They also asked for a refund on an opened game and agreed to take another copy if the store allowed only an exchange. The agent checked the rules for each item, confirmed stock availability, and calculated the refunds before requesting approval. When the payment record changed, it recalculated the earphones refund before issuing it.

![One recorded retail request progresses through policy checks, refund calculations, approval, and execution.](https://cdn.hashnode.com/uploads/covers/6abe4a89fbf68ad459d66120/3214b244-0a58-456e-a233-7540a309b24c.png align="center")

*Figure 5. The agent prepares the plan on the left and carries it out on the right. The run used simulated store records and scripted approval.*

### Comparing the variations

The twelve variations tested different parts of the workflow. Some asked the agent to identify the right item or handle an unavailable size. Others asked the agent to apply different return policies, check prior refunds, or wait for an inspection.

Revision 1 and Revision 2 both started from Baseline. For each revision, we drafted instructions from one set of examples and compared the versions on a separate set of cases. All tasks passed, including those used to draft the revisions.

The table shows the Baseline averages for each variation. Tool calls include business tool calls and scripted approval calls.

| Scenario and expected behavior | Avg. model steps | Avg. tool calls | Avg. task time (s) | Avg. total tokens |
| --- | --: | --: | --: | --: |
| **1\. Mixed returns and exchanges.** Exchange a jacket, refund shoes, and leave another jacket unchanged. | 3 | 9 | 33 | 81k |
| **2\. Ambiguous item.** Clarify which of the two matching jackets on different orders the customer means. | 6 | 13 | 54 | 106k |
| **3\. Customer changes size.** Update an exchange from medium to large while preserving the customer’s other instructions. | 4 | 12 | 41 | 106k |
| **4\. Unavailable size.** Follow the customer’s fallback: keep the jacket if large is unavailable, without issuing a refund. | 4 | 11 | 43 | 103k |
| **5\. Split payment.** Refund the original card and gift credit correctly, without moving gift credit onto the card. | 3 | 9 | 35 | 81k |
| **6\. Previous partial refund.** Refund only the remaining pair of shoes, without duplicating an earlier refund. | 3 | 11 | 36 | 77k |
| **7\. Different return windows.** Complete eligible clothing requests while denying an electronics return outside its allowed window. | 3 | 12 | 37 | 86k |
| **8\. Exchange-only item.** Exchange an opened game for the authorized same-title replacement when a refund is prohibited. | 3 | 18 | 37 | 112k |
| **9\. Pending jewelry inspection.** Wait for inspection approval before refunding a ring, then complete the eligible return. | 4 | 19 | 45 | 143k |
| **10\. Altered jewelry.** Deny a refund for an altered ring, but complete other eligible requests. | 3 | 12 | 33 | 87k |
| **11\. Different policy versions.** Apply the policy governing each item, even when similar products have different return rules. | 3 | 17 | 35 | 105k |
| **12\. Combined constraints.** Handle a size change, split payment, an expired return, and pending inspection within one five-item request. | 4 | 18 | 46 | 128k |

Both revisions cut business tool calls by about a fifth compared with Baseline. Revision 1 raised median task time from 36.8 to 40.0 seconds and the upper estimate of model cost by 1.3%. Revision 2 lowered the median from 38.6 to 36.7 seconds, while the upper cost estimate rose 0.4%. Revision 2 was faster in half of its paired comparisons.

Promptfoo’s comparison view pairs each Baseline run with the revision tested on the same case. That lets us inspect the replies and checks behind the aggregate results.

![Promptfoo compares experiment 11 baseline and revised task outputs side by side across both evaluation rounds.](https://cdn.hashnode.com/uploads/covers/6abe4a89fbf68ad459d66120/e7cd0070-8ec0-44cb-b0f0-b772276fb6f0.png align="center")

*Figure 6. Matched Baseline and revised results from the return-and-exchange study.*

The four examples below show how Baseline and Revision 2 differed by variation. Times are medians, and business tool calls are totals for each version.

| Return or exchange variation | Median task time, Baseline → Revision 2 | Business tool calls, Baseline → Revision 2 |
| --- | --: | --: |
| Multiple orders with a previous partial refund | 38.9 → 30.0 s | 18 → 16 |
| Two matching items requiring clarification | 54.4 → 47.1 s | 22 → 14 |
| Opened item eligible for exchange only | 36.1 → 39.5 s | 35 → 30 |
| Jewelry awaiting inspection | 42.7 → 48.7 s | 34 → 26 |

With Revision 2, both the clarification and inspection cases used eight fewer business tool calls. Median time fell for clarification and rose for inspection. That variation matters when deciding whether to adopt a shared instruction set. The study’s rule required every case to pass. A revision also had to lower estimated model cost or median task time without worsening the other. Neither revision met that rule. Revision 2 still reduced median task time, a benefit a team might value differently.

## Framework for applied AI engineering

The [applied AI engineering experiment framework](https://blog.latentthoughts.com/from-hypothesis-to-understanding-a-practical-framework-for-applied-ai-engineering-experiments) helped us turn a broad claim about self-improvement into tests with measurable outcomes.

![The engineering experiment framework moves from a hypothesis through testing and interpretation to a decision.](https://cdn.hashnode.com/uploads/covers/6abe4a89fbf68ad459d66120/2bda05df-81c3-4bbb-acea-a4f6a8f4fcf9.png align="center")

*Figure 7. The eight-step framework used to structure the investigation.*

The model proposed changes within the experiment. We chose the questions, set the checks, and decided whether the evidence supported continuing.

![Experiment results feed into interpretation and a decision to revise the investigation or stop.](https://cdn.hashnode.com/uploads/covers/6abe4a89fbf68ad459d66120/1c826f09-2586-48a0-8b4e-ebaec57e3907.png align="center")

*Figure 8. Findings guide the next experiment or a decision to stop.*

## A revised thesis: continuous adaptation

The results support the counter-thesis within this workload: a strong baseline left little measured correctness headroom, while some revisions still improved efficiency. That makes the value of an improvement loop a practical question of its benefits versus the effort it requires. How well revisions transfer to unfamiliar requests, and whether gains accumulate over time, remain open questions.

A revised thesis is that the larger opportunity may be **continuous adaptation as requirements change**. A new return policy, a changed service interface, or different customer behavior could create new work for an improvement loop even when the original agent performed well. The tests here varied customer choices within individual tasks. A next experiment could introduce a lasting change to the operating environment and measure whether autonomous revision restores the required behavior, along with the time and cost of doing so.

---

*This blog represents my viewpoints, not those of my past or current employers. All product names, images, logos, and brands are the property of their respective owners.*