We built our own custom harness, was it worth it?
A few years ago, the product engineer was the hot new profile on the engineering job market: someone with the product intuition required to own features end to end and the technical ability to actually ship them.
Then came the AI engineer, someone who understood the model landscape, could navigate providers, and knew how to apply AI in product.
Now, there’s a new title making the rounds. If you’ve been tuned into the AI discourse over the past few months, you’ve almost certainly heard it: the harness engineer.
Here’s the thing, though. Harness engineering isn’t a new discipline. It’s just a fancy new name for a natural shift in engineering leverage that becomes obvious once you understand how agent systems work.
The agent loop
For all the excitement surrounding AI agents in recent months, the primitives they’re built on are surprisingly uncomplicated.
The first generation of AI products was extremely simple. A prompt goes in, a response comes out.
Then, as products became more ambitious, we saw the arrival of workflows: chains of LLM calls designed to accomplish more complex tasks. They were more capable, but still rigid. Every execution path was predetermined, so they couldn’t adapt, share context, or build on what they’d learned along the way.
The agent loop changed that.
In an agent loop, rather than prescribing every step in advance, the model is given a goal and a set of tools, then trusted to figure out how to accomplish tasks independently. For any given request, it decides which tool to call, evaluates the result, determines whether the task is complete, and either returns an answer if it is or repeats the process if not.
That’s it.
Whether an agent runs for thirty seconds or thirty hours, it’s executing that same stupidly simple loop.
Once you realize every team building AI agents is working from those same fundamental primitives, an interesting question naturally follows:
Where does product differentiation come from?
Where the leverage is
There are really only two places you can intervene within an agent loop.
The first is at the level of the model itself. You can experiment with prompting techniques, evaluate providers, fine-tune models, or invest in reinforcement learning.
But the model ultimately remains a black box. You can influence its behaviour, but you can’t fundamentally change how it reasons. And as frontier models continue to improve, the return on investment from model customization is diminishing for most teams.
So, that leaves only what surrounds the model: decisions about what context gets loaded, which tools are available, what belongs in memory, how to manage compaction, whether tool execution should be synchronous or asynchronous, etc.
Collectively, these decisions form the orchestration layer around the agent loop. That layer is the harness, and it’s where the engineering leverage is today.
Our harness journey
Last month we launched Runneth, a marketing agent that runs inside a dedicated cloud VM for every customer.
Today, Runneth runs on our own custom harness. But that’s not where we started.
Our first implementation used the Claude Agent SDK, the same engine behind Claude Code. Since we were already building on Anthropic models, it was the obvious place to begin. It let us validate the product quickly without needing to build an orchestration layer from scratch.
As Runneth matured, we eventually transitioned to Pi, an open-source harness with a minimal core and an extensive plugin architecture, because it was a better fit for our needs. It made different decisions around things like extensibility, context management, and orchestration that aligned more closely with the product we were building.
Still, though, we ran into recurring problems related to:
Opinionated defaults: Every open-source harness makes unique architectural decisions about how its agent loop operates. Those defaults make sense for the use cases they were designed for, but they aren’t necessarily the same decisions we’d make ourselves. Even Pi’s intentionally minimal architecture still had core assumptions baked in that we wished we could customize.
Latency: Every trip around the agent loop costs time, and not every request needs multiple iterations. Many of our agent’s requests follow predictable patterns where the required context, tools, and execution path are already known. Without deeper control over the loop, there’s no clean way to bypass unnecessary reasoning steps, which adds latency to requests that could otherwise be handled more efficiently.
Cost: Most popular harnesses are designed to run locally, where compute costs aren’t a constraint. Our environment looks very different, however. Because our agents run in VMs, memory and compute translate directly into infrastructure cost. Pi was consuming ~100 MB of RAM per conversation, and when you’re running hundreds of those concurrently, it quickly adds up.
We discussed these challenges with the Anthropic team, and their advice was clear:
Try building your own harness.
So we did.
We analyzed the source code of all the open-source harnesses we could find and compared the architectural decisions each one made. From there, we designed our own harness, drawing on what we found worked well for others, and built it in Rust for its memory efficiency, lack of runtime overhead, performance, and safety benefits.
What owning the harness unlocked for us
The harness is still new, but the early results are promising. We’ve already reduced memory usage from ~100 MB to ~10 MB per conversation, and gained significantly more control over things like:
Prompt augmentation: For well-understood patterns, we can now intercept requests before the model ever sees them and augment them, replace them, inject them with domain-specific context, preselect execution paths, or otherwise optimize how they’re handled.
Learning extraction: When users correct the model’s output, our harness captures those corrections and stores them. On semantically similar requests, those learnings are automatically applied.
Detached tool execution: Many harnesses block on tool execution. The agent calls a tool, waits for it to finish, then continues reasoning. That works well for short-lived operations, but in our VM environment, some commands can take several minutes. Rather than leaving the agent idle, our harness detaches long-running tool calls and polls them asynchronously, allowing execution to continue in parallel.
Multiplayer: Off-the-shelf harnesses are designed for single users working in isolation. As a result, collaboration requires constant shuttling of context and feedback between separate agents, each time effectively starting from scratch. We built our harness to support multiple users on the same VM, meaning an entire team can work in the same environment, on the same files, with shared memory and conversation history.
Permissions: To support multiple users on a shared VM, the harness must reliably distinguish between them. Rather than specifying user identity in the system prompt, our harness hard-codes it into every tool call, making enforcement deterministic. And because our VMs run Linux, we also map each user directly to the Unix permissions system, so read, write, and execute access is enforced at the operating-system level. This gives us a secure, reliable foundation for multiplayer access.
The tradeoffs
Reading this, it might be tempting to conclude that every agent team should explore building their own harness.
But that’s not true.
Building a custom harness is a significant investment. You inherit every challenge that open-source frameworks have already solved: tool execution, context management, memory compaction, observability, debugging, testing, and the countless edge cases that only show up once your agent is running in production.
For many teams, that tradeoff isn’t worth it.
If your product fits comfortably within the assumptions of an existing harness, it’s almost certainly more sensible to build on top of that rather than start from scratch. That means that for many teams, the Claude Agent SDK with well-designed tools and skills will be enough. For others, Pi with rich plugins will be a great fit.
For us, though, it’s been a bet worth taking. Over time, the benefits of owning the harness gradually came to outweigh the costs of building it.
Harness engineering may be the latest buzzworthy title in the industry, but strip away the noise and it really just comes down to understanding a simple agent loop, knowing where you can and should intervene within it, and making decisions at the harness level to optimize your agent’s behaviour.
Join the Motion team 💜
 |
|
Thanks for sharing! I am considering owning agent or open-source agent too.
I took the same layers apart in a runnable Python tutorial, 400+ stars on GitHub.
https://github.com/hardness1020/awesome-agent-architecture