Architecting an Autonomous Agentic Harness: Implementing Tool Calling, MCP Integration, and Model Routing in Pure Python

In the current landscape of AI development, high-level frameworks like LangChain or CrewAI have become the standard for building agents. While these abstractions are powerful, they often obscure the fundamental mechanics of how an LLM interacts with external environments. To truly master agentic workflows, one must understand how to build an agentic harness from the ground scratch—implementing the loop, the tool schemas, and the communication protocols manually.

This post details the architectural process of building a custom Python-based harness that supports multi-step reasoning (the agentic loop), Model Context Protocol (MCP) integration, persistent memory via context injection, and dynamic model routing between local and cloud-based LLMs.

The Core Architecture: LLM as the Reasoning Engine

At its fundamental level, a Large Language Model (LLM) is a text prediction engine. It receives an input sequence and predicts the most probable subsequent tokens. To transform this "brain" into an "agent," we must build a wrapper—the harness—that provides the model with "hands" (tools) and "memory."

The foundation of our harness begins with a basic chat completion loop. Using the openai Python library, we can interface with both cloud providers (Openary/Anthropic) and local inference servers like Ollama by pointing the base_url to a local endpoint (e.g., http://localhost:11434/v1).

The Basic Chat Completion Loop

The primary structure involves managing a list of message objects, categorized into three types:

1. System Messages: Defining the persona and operational constraints.
2. User Messages: The current task or query.
3. Assistant Messages: The model's previous responses and tool call outputs.

Implementing Tool Calling via JSON Schemas

The most critical leap from a simple chatbot to an agent is tool calling. It is a common misconception that LLMs "call" functions. In reality, the LLM merely generates a structured text string—typically in JSON format—that requests a function call. The harness is responsible for parsing this request, executing the Python function, and feeding the result back to the model.

To enable this, we must define a Tool Schema. This schema acts as the interface between the Python environment and the LLM's reasoning engine. Each tool requires:

• Name: A unique identifier (e.g., list_files).
• Description: A semantic explanation of what the tool does, which the LLM uses to decide when to invoke it.
• Parameters: An object defining the required inputs, including types (e.g., string, object) and properties.

For example, a read_file tool requires a file_name parameter of type string. By passing these schemas into the tools argument of the chat completion request, we provide the model with the necessary metadata to perform structured reasoning.

The Agentic Loop: Enabling Multi-Step Reasoning

A single-shot tool call is insufficient for complex tasks. If an agent needs to list files, read a specific file, and then summarize its contents, it must be able to iterate. This requires the implementation of an Agentic Loop (or while True loop).

The logic follows this recursive pattern:

1. Request: The harness sends the current message history to the LLM.
2. Inspect: The harness checks if the response contains tool_calls.
3. Execute: If tool calls exist, the harness iterates through each call, parses the arguments via json.loads(), executes the corresponding Python function, and appends a tool role message containing the result to the history.
4. Repeat: The updated history is sent back to the LLM.
5. Terminate: The loop breaks only when the LLM provides a standard text response without requesting further tool execution.

This loop allows for "chain-of-thought" style execution where the agent can perform an arbitrary number of operations until the task is complete.

Expanding Capabilities with Model Context Protocol (MCP)

To avoid hardcoding every possible tool into our Python script, we can implement the Model Context Protocol (MCP). MCP is an open standard that allows agents to dynamically discover and connect to external servers.

By utilizing stdio type MCP servers, our harness can launch separate processes that serve specific capabilities (e.g., a web-search server or a filesystem server). This introduces complexity via asynchronous programming. Because MCP interactions are I/O bound and involve network or process communication, we must use asyncio to prevent the agent loop from blocking while waiting for tool results.

The implementation involves:

• Dynamic Discovery: Connecting to an MCP server and querying its available tools.
• moving these discovered tools into our primary tool_schema.
• Async Execution: Using async/await patterns to handle the lifecycle of the MCP session.

Persistent Memory via Context Injection

Standard LLM sessions are stateless; once the Python process terminates, all context is lost. To implement persistent memory, we can use a simple but effective strategy: Context Injection.

We maintain a local memory.md file. We then equip our agent with two specific tools: load_memory and save_memory.

• At Startup: The harness reads the contents of memory.md and injects it into the initial System Message. This ensures the model is "pre-loaded" with user preferences, names, or past interactions.
• During Runtime: When the agent learns something new (e.g., "My name is Tim"), it calls the save_memory tool to append this information to the markdown file.

Model Routing and Unified Configuration

A sophisticated harness should be model-agnostic. By implementing a configuration-driven architecture, we can switch between local models—such as Quen 3.5 (4B or 2B) running via Ollama—and high-performance cloud models like GPT-4 or Claude.

By using a models.json configuration file and a command-driven interface (e.g., /model <name>), we can swap the client instance at runtime. This allows developers to use small, local models for simple tasks (like file manipulation) and route complex reasoning tasks to larger, more capable cloud models, optimizing both cost and latency.

Conclusion

Building an agentic harness from scratch reveals the true complexity of AI orchestration. By managing the tool schemas, the recursive execution loop, MCP integration, and persistent state manually, we gain a granular level of control that high-level frameworks cannot provide. This architecture provides a robust foundation for building specialized, autonomous agents capable of interacting with any programmable environment.