On Big Tech's Harness Engineering Futility Thesis and Uncritical Next-Generation Loop Advocacy

After an AI Times article discussing big tech's harness engineering futility thesis was published, various tech leaders exchanged opinions on the matter. Reports indicated that following Google, OpenAI also made statements suggesting that next-generation models could absorb significant portions of current harness functionality.

This post is an addition to that discussion. To state my conclusion upfront, I agree with the outlook that most harness engineering will become obsolete. In my own post Harness Engineering Explained Comprehensively — Birth, Concepts, Gaps, and Future in One Go, I wrote that complex harness designs created to compensate for model weaknesses are rapidly replaced as models improve.

However, using this as grounds to claim that harness engineering as a whole will soon become unnecessary misses the essence of harness engineering. The problem of models absorbing functionality is different from the problem of humans and organizations managing responsibility as they delegate work to AI.

First, we must examine how keywords circulate

However, before anything else, tech keywords are often consumed in ways and meanings disconnected from their overseas origins in Korea, so I first verified this. Again, we need to distinguish between the original statements, actual technical discussions, and the frameworks created within Korea.

A comparison from my post Harness Engineering Explained Comprehensively — Birth, Concepts, Gaps, and Future in One Go of how the harness engineering keyword is consumed globally versus in Korea.

One caveat about over-interpretation: there is no clear evidence that "harness is dead" or "harness is useless" is a widely used slogan in English-language discourse. The discussion is closer to models improving their tool use abilities, some scaffolding being absorbed into models or platforms, and manually constructed complex orchestration layers having shorter lifespans. The statement from Vice President Noam Brown that I saw was not something I directly verified to the original source, but rather learned through domestic and international reporting. That's why I want to examine the 'harness futility thesis' frame that has quickly solidified within Korea.

Harness that will disappear and harness that must remain are different

The harness that will disappear is clear. The practice of splitting prompts into multiple stages, temporary rules to work around specific model defects, and shallow workflows where tool invocation order is artificially fixed by humans can be absorbed as models improve. This harness should disappear.

However, as A2Sys CEO Dong-Soo Lee organized it, I believe it is risky to extrapolate this to all harness. To summarize, it looks like this:

• Prompt and workflow harness can be absorbed by models. I agree.
• Tool execution harness addresses permissions, approvals, and failure recovery. Is this really harness that will disappear?
• Runtime, memory, and infrastructure harness address context, cache, model routing, cost, latency, and audit logs. I find it difficult to say this will also disappear.

The essence of harness that I discussed from chapters 7 through 11 in 『Harness Engineering: Re:Starting from Zero in AI Software Engineering』 lies here. Contracts define allowable scope, structure and isolation narrow the scope of change, and traceability and verification connect intended goals to actual results. Measurement and operations manage cost, permissions, incidents, and recovery. Harness engineering in this scope is not something solved by better models.

On the other hand, the claim that harness, loops, and prompt engineering are human heuristics is also correct. All methodologies start with heuristics. Types, tests, CI, code review, and permission separation are all methods humans created to handle complex systems. But what's more important here is: can we delegate transparency and accountability entirely to the flagship model? AI cannot do this on its own—verify that work is proceeding as intended and ensure accidents don't occur. Of course, there could be an AI that monitors and corrects this, but then that is the harness.

I wrote this position in a comment on Professor Han Sang-ki's post like this:

It seems to depend on how broadly you define harness engineering. As long as organizations and developers have intent, I don't think it will disappear.

The better the model becomes, the greater the possibility that AI will accomplish increasingly nonsensical things with greater sophistication until completion. That's why harness is not a device to correct model weaknesses, but an operational device to fix direction and responsibility.

In actual operations, responsibility boundaries come before model capabilities

Nextain conducts work with development clients through Discord. And today, I put into that an agent from Claude Code that was handling a specific issue. A Claude Code instance (actually a server-deployed Claude Code) that participates in a Discord channel reads that channel to provide answers.

A non-developer operator present in the channel conveyed test results, shortcomings, and requirements. In handling what could be addressed, when important deployments or resources that shouldn't be touched needed modification, the agent asks me for approval.

Also, since they were initially frustrated that the agent started work right after hearing requirements (without delay), I instructed work process. Upon hearing requirements, acknowledge receipt, conduct analysis, create a plan, report the plan back to the Discord channel, and then proceed with work. When work is finally complete, file a final report. If any issues arise mid-process, call me (development lead) and ask on Discord when a decision is needed.

The core of this harness is Nextain's development process and the permissions of each member. If this were left to a flagship model, would it automatically handle work in the best way? Is there even one best method? Consistency in organization and work process is ultimately harness. This responsibility boundary is separate from model capability. The better a model becomes at handling more work, the more rigorously we must divide what may and may not be done and where to stop.

The real problem facing practitioners is drift and cost

Developers using Codex and Claude in practice still face the problem of drift. Even when narrowing objectives, work leaks in other directions. Prohibited changes get made. Tests get changed to pass. Operational permission boundaries are lost. As models become more sophisticated, this problem can be concealed more smoothly.

In such cases, "just run more" is possible for big tech. They can extend reasoning time, generate more candidates, discard failed paths, and attach external verification. However, working developers and cost-conscious startups cannot do this. The reality of domestic companies is the same.

In the comments with Professor Han Sang-ki linked above, former AI Future Planning Senior Secretary Hah Jung-woo mentioned the "total token cost efficiency" that is precisely this criterion. Harness is not a device that rejects flagship models. It's a cost control device—using expensive models only for truly necessary judgments, and separating repetitive work into smaller models, local tools, scripts, and deterministic validators.

Recent OpenAI achievements in mathematics also need to be distinguished at two points. I rate the first achievement, related to the counterexample related to the Erdős unit distance problem in the plane, highly as showing the possibility of finding mathematical connections different from existing approaches. This is a case demonstrating the possibility of new discoveries only AI could make.

In contrast, the subsequently highlighted test-time compute is a direction of investing more computation, candidate exploration, verification, and iteration at inference time. This is also important technology. However, talking about it solely as pure intelligence improvement of a single model erases the cost structure. It's a quantity strategy, but a technique particularly favorable to big tech.

Harness is the defense line for cost and data

The saying "models will consume harness" is provocative. But who uses that model and at what cost is a separate issue. As models absorb more functionality, user code can become simpler. Simultaneously, users may become more dependent on heavier, more expensive model calls. Complexity doesn't disappear—it shifts from the user's workspace to the model provider's billing layer.

Data is the same. For agents to become truly useful, they must see documents, code, schedules, emails, operational logs, incident history, approval flows, and customer data. The moment an agent becomes a workspace, the user's entire work becomes input.

Good harness sends only necessary context to external models and keeps the rest in the user's workspace. It routes models, reuses cache, divides permissions, and attaches reproducible validation. That's why harness is a cost defense line and a data defense line.

From this perspective, it's possible that global model companies would find strong user-side harness inconvenient. If users first process with small models and local tools, calling on flagships only when necessary, then call volumes and context sent decreases. This is not speculation but a natural consideration of interests in an industry structure where frontier model development, inference, GPU, and power costs grow.

OpenAI and Anthropic are at the forefront of harness competition through Codex and Claude Code. Google, meanwhile, is both a model company and a cloud provider. It can't be said that all companies have exactly the same interests, but the reason this debate is difficult to hear as pure technical prediction is clear. Good harness reduces big tech's desire to 'move workspaces to the model, collect data, and increase dependency on flagships.'

Loops are not the next-generation default

Loops are the same. I use loops, but I think for most missions, not using loops is the right approach. If it's work where regular developers could deliver appropriate results even if delegated, using a good model once well might be cheaper and faster. Surprisingly, effective use cases for loops are narrow.

First is research and development. Work where there is no single answer, hypotheses change based on experimental results, and the next experiment must actively respond by reading previous results. Internal voice and song research is like this. In such cases, multiple AIs review experimental results and full data, judge continuity with existing experiments, drift, and necessity for the next cycle. The completion criterion is not plausible results but the judgment that metrics no longer improve. Since convergence of exploration is the goal, loops are appropriate.

Second is using small models in restricted environments. Small LLMs easily truncate objectives and stop at "this is good enough." But what's needed in this case is not lengthy loops but a mechanism that closes objectives and termination conditions. It's closer to Claude Code's goal feature than a loop—a harness device. If you produce loops indiscriminately for general user daily tasks, it becomes wasteful. Use loops for convergence of exploration, and harness for fixing objectives and responsibility. They are not interchangeable.

Naia aims to keep the workspace on the user side

Naia does not reject flagship models. Development will thoroughly utilize cloud flagship models. Good models must be used.

However, we have no intention of delegating all workspaces and data to big tech platforms. Naia's direction is to combine local, P2P, open-weight, small models, deterministic validators, and explicit permission harness. Use expensive flagships for necessary judgments, but the basic workspace should remain in the user's hands.

From this perspective, the development of open-weight models like DeepSeek, Qwen, Kimi, and GLM is important. It becomes possible to combine models by purpose, protect data locally, and call closed flagships only when necessary. In Korea, proprietary model strategies from Upstage and Naver have meaning in this flow as well. Rather than simply following global big tech flagships, binding open-weight models, proprietary models, specialized harness, and domestic business context well may be more realistic.

Models can be borrowed. But there's no need to borrow the workspace and data too.

Conclusion: More necessary than futility thesis is technical-level discussion

Harness that will disappear will disappear. Functionality that models absorb will increase. There's no reason to deny this. However, I don't think the functionality models absorb, the execution systems that fix organizational intent and responsibility, and the operational systems that protect cost and data boundaries will disappear. If the subject directing AI is human and the subject taking responsibility for results is human, then contracts, structure, traceability, validation, and operations will not disappear.

I hope we can stop the keyword debate about whether harness is dead or alive. I'd prefer discussions centered on real technology: what to put inside the model, what to keep under user and organizational control, how to validate and reverse drift, who bears the burden of token cost and data exposure.

References and Sources

My writings and books

• 『Harness Engineering: Re:Starting from Zero in AI Software Engineering』
• "Harness Engineering Explained Comprehensively — Birth, Concepts, Gaps, and Future in One Go"
• "Harness Engineering Publication News"

Direct catalysts for this discussion

• AI Times, "Following Google, OpenAI also embraces 'harness futility thesis'... next-generation models will absorb function"
• AI Times, "[June 25] 'Models will consume harness'..."
• Professor Han Sang-ki's Facebook post and comment discussion
• A2Sys CEO Dong-Soo Lee, "Harness futility thesis? Half right and half wrong"

Original materials and technical references

• OpenAI, "Learning to Reason with LLMs"
• OpenAI, "An OpenAI model has disproved a central conjecture in discrete geometry"
• OpenAI, "Introducing Codex"
• OpenAI, "Introducing OpenAI o3 and o4-mini"
• Google, "Gemini 2.0: our new AI model for the agentic era"
• "The Interplay of Harness Design and Post-Training in LLM Agents"
• "More Is Not Always Better: Cross-Component Interference in LLM Agent Scaffolding"
• "AutoHarness"