Mitigating LLM Self-Evaluation Bias via Multi-Agent Iterative Auditing: An Analysis of the Claudex Loop Workflow

One of the most persistent challenges in Large Language Model (LLM) orchestration is the phenomenon of self-favorable bias. When an agent is tasked with planning, executing, and grading its own work, it inherently lacks the critical distance required to identify architectural flaws or edge-case vulnerabilities. In short: you cannot trust Claude to grade its own work. This inherent lack of objectivity can lead to "hallucinated success," where a model validates a plan that is fundamentally broken in execution.

The Claudex Loop introduces a multi-agent, adversarial auditing architecture designed specifically to break this feedback loop. By introducing a secondary, independent model—Codex—to act as an auditor within a structured, iterative framework, the system moves from a single-agent "stream of consciousness" approach to a rigorous, multi-phase verification pipeline.

The Architecture of the Claudex Loop

The workflow is partitioned into four distinct phases: Reconnaissance, Interrogation, Review, and Build. This separation of concerns ensures that technical requirements are validated before a single line of code is committed to the repository.

Phase 0: Reconnaissance and Information Retrieval

Before any planning occurs, the system enters a reconnaissance phase. The objective here is to determine if the necessary technical documentation or API specifications exist within the model's training data or can be retrieved via live web search.

The workflow offers two modes of retrieval:

1. Standard Web Search: Utilizing Claude’s native sub-agents to scrape relevant documentation.
2. Deep Research Mode: A dynamic, high-intensity workflow that triggers specialized sub-agents for exhaustive technical deep dives (e.g., investigating Google Calendar API constraints or OAuth 2.0 implementation nuances).

A critical output of this phase is the Assumptions Ledger. This document serves as a single source of truth, listing every architectural assumption made during the research phase. By formalizing these assumptions early, the system prevents "requirement drift" later in the lifecycle.

Phase 1: The Interrogation Phase

Once the research is complete, the system enters an enhanced interrogation mode. Rather than proceeding directly to a plan.md file, the agent presents a series of load-bearing questions. These are high-impact technical decisions that will dictate the fundamental architecture of the project (e.g., "Which Google account identity provider should be used for OAuth?").

Following the load-ability questions, the system moves to cosmetic decisioning. While these do not alter the core logic, they refine the implementation details (such as UI/UX preferences or secondary library choices), ensuring that the subsequent planning phase is highly specific.

Phase 2: The Iterative Review Loop

This is the core of the Claudex Loop's technical efficacy. After Claude generates a standard plan.md based on the interrogation results, the plan is passed to Codex for review within a read-only sandbox.

The interaction follows an iterative loop pattern:

1. Audit: Codex analyzes the plan.md for logical inconsistencies, missing edge cases, or architectural flaws.
2. Feedback: Codex returns specific revisions (e.g., "Revise X, Y, and Z").
3. Re-evaluation: Claude processes these revisions and submits a new version of the plan.

To prevent infinite loops and excessive token consumption (token burning), the system implements a hard stop threshold—defaulting to five rounds of back-and-forth. However, as demonstrated in complex scenarios, this limit can be extended if the models reach a deadlock. In one observed instance, an initial 27 issues were identified in Round 1; through seven rounds of iterative refinement, the models reached an approved verdict.

Phase 3: Execution and Post-Build Verification

The final phase is the Build phase. The user can opt for Claude to build while Codex audits, or vice versa, or even a tandem build where both models contribute to different modules (e.g., one handling asset generation via DALL-E/GPT-Image tools, and another handling logic).

Crucially, the loop does not end when the code is written. A secondary audit occurs where Codex reviews the actual implementation against the original plan.md specification. This "post-build" check uses a fresh context window to identify discrepancies between the plan and the execution.

Case Study: Recreating a Scheduling Engine (Calendly Clone)

To demonstrate the efficacy of this loop, we applied it to the development of a scheduling web application utilizing Google Meet and Google Calendar integrations.

During the Review Phase, Codex identified critical logic gaps that Claude’s initial plan overlooked, such as:

• Concurrency Constraints: The potential for two simultaneous booking requests to both succeed (race conditions).
• OAuth Flow Vulnerabilities: Incomplete implementation of the connection handshake.
• Double-Booking Logic: Lack of a robust constraint check within the scheduling algorithm.

During the Build Phase, the post-execution audit identified even more granular technical regressions:

• Time Grid Drift: The UI calendar grid shifting incorrectly after meeting updates.
• Security Regressions: Management tokens being stored in plain text rather than encrypted environment variables.
• Logic Errors: Event blocking overlapping incorrect time slots.

In this specific run, Codex identified 23 findings; 19 were accepted and fixed by the developer/agent, while four were rejected as non-critical or out of scope.

Conclusion: The Economic and Technical Value Proposition

The Claudex Loop architecture shifts the cost of error from production to planning. While the multi-agent loop increases initial token usage during the planning phase, it drastically reduces the "token burn" associated with debugging broken production code and fixing architectural flaws after deployment. By treating LLM development as a verifiable engineering pipeline rather than a single-prompt generation task, developers can achieve much higher levels of reliability in autonomous agentic workflows.