[{"data":1,"prerenderedAt":1272},["ShallowReactive",2],{"knowledge:en:\u002Fknowledge\u002Fai\u002Fagents\u002Frepository-centered-agent-harness":3,"knowledge-locales:\u002Fknowledge\u002Fai\u002Fagents\u002Frepository-centered-agent-harness":1271},{"id":4,"title":5,"appliesTo":6,"body":11,"description":1251,"extension":1252,"meta":1253,"navigation":1254,"path":1255,"publishedAt":1256,"seo":1257,"status":1258,"stem":1259,"tags":1260,"technologies":1266,"updatedAt":1269,"__hash__":1270},"knowledge_en\u002Fknowledge\u002Fai\u002Fagents\u002Frepository-centered-agent-harness.md","Designing an Agent Harness That Survives Model Changes",[7,8,9,10],"RAG decision placement and alternative-model evaluation checked on 2026-10-01","Local decision models and mlx-serve checked on September 29, 2026","Public information available as of 2026-09-03","Jev decision delegation and document selection based on public information as of 2026-09-19",{"type":12,"value":13,"toc":1228},"minimark",[14,19,23,26,29,33,68,72,75,78,104,121,124,128,131,198,201,211,214,218,221,227,244,247,331,334,338,341,359,388,411,496,499,503,506,571,583,588,598,602,605,608,631,639,642,660,664,677,727,736,745,784,793,796,814,823,828,831,908,911,914,917,938,951,960,964,967,973,976,979,983,992,995,1001,1004,1007,1015,1019,1022,1031,1034,1038,1044,1047,1064,1067,1071,1074,1100,1104,1130,1134],[15,16,18],"h2",{"id":17},"overview","Overview",[20,21,22],"p",{},"When choosing a coding agent, it is easy to focus on the model: GPT, Claude, Qwen, or another option. Development results, however, do not depend on the model alone. They are also shaped by the execution environment that controls repository access, tools, permissions, context, skills, subagents, and how results are presented for review. This article calls that layer the agent harness.",[20,24,25],{},"If every harness owns a separate copy of the project's knowledge, changing products means migrating rules and procedures. Keeping durable knowledge in the repository and making harness-specific configuration thin allows the model and the harness to be evaluated separately.",[20,27,28],{},"This is not a ranking of products. It describes how to grow an agentic coding environment incrementally while leaving room to evaluate local LLMs later.",[15,30,32],{"id":31},"conclusions","Conclusions",[34,35,36,40,43,46,53,56,59,62,65],"ul",{},[37,38,39],"li",{},"Kev, SemIf, Laya, and Tev1 offer local decision-evaluation options, but their readouts, calibration, licenses, and API coverage need separate checks.",[37,41,42],{},"The model performs inference; the harness manages context, tools, permissions, the execution loop, and human review.",[37,44,45],{},"Shared knowledge and procedures should have an authoritative version in the repository rather than living only in product-specific memory or settings.",[37,47,48,52],{},[49,50,51],"code",{},"AGENTS.md"," should be a lightweight entry point to relevant documents and checks, not a comprehensive manual.",[37,54,55],{},"Documentation, skills, scripts, CI, and memory have different responsibilities and should not duplicate the same content everywhere.",[37,57,58],{},"What can be shared across tools is primarily semantic content. Skill discovery paths, agent definitions, authentication, and permissions may need thin adapters.",[37,60,61],{},"Do not hard-code model routing in advance. Expand it after measuring quality, elapsed time, retries, and the amount of human correction on comparable tasks.",[37,63,64],{},"Automatic knowledge capture should create candidates. Human review and deterministic validation should happen before a candidate becomes authoritative repository content.",[37,66,67],{},"Skill and document selection can be delegated to a specialized model, while the harness and code retain control of mandatory rules, permissions, and authoritative-source updates.",[15,69,71],{"id":70},"what-is-an-agent-harness","What Is an Agent Harness?",[20,73,74],{},"An LLM proposes the next text or tool call from the supplied context. Turning those proposals into development work requires a loop that executes actions and returns the results to the model.",[20,76,77],{},"The exact boundary differs by product, but a harness commonly handles:",[34,79,80,83,86,89,92,95,98,101],{},[37,81,82],{},"reading and writing repositories and working directories;",[37,84,85],{},"terminal, browser, and external-service tools;",[37,87,88],{},"instructions and context supplied to the model;",[37,90,91],{},"permission for tool execution, file changes, and network access;",[37,93,94],{},"task state and context compaction;",[37,96,97],{},"skills and reusable workflows;",[37,99,100],{},"delegation to subagents and synthesis of their results; and",[37,102,103],{},"a review surface for diffs, logs, and test results.",[20,105,106,107,114,115,120],{},"OpenAI describes the ",[108,109,113],"a",{"href":110,"rel":111},"https:\u002F\u002Fopenai.com\u002Findex\u002Funrolling-the-codex-agent-loop\u002F",[112],"nofollow","Codex agent loop"," as a harness that connects a model to tools and iterates on their results. Its internal ",[108,116,119],{"href":117,"rel":118},"https:\u002F\u002Fopenai.com\u002Findex\u002Fharness-engineering\u002F",[112],"harness-engineering report"," also describes making applications, logs, and metrics legible to agents and keeping structured repository knowledge as the system of record.",[20,122,123],{},"It is therefore not enough to know that two agents used the same model. Tool availability, context construction, permissions, retry behavior, and completion criteria must also be controlled before model differences can be compared meaningfully.",[15,125,127],{"id":126},"separate-the-environment-into-four-layers","Separate the Environment into Four Layers",[20,129,130],{},"An agentic coding environment is easier to change when divided into four layers.",[132,133,134,150],"table",{},[135,136,137],"thead",{},[138,139,140,144,147],"tr",{},[141,142,143],"th",{},"Layer",[141,145,146],{},"Primary responsibility",[141,148,149],{},"What changes when it is replaced",[151,152,153,165,176,187],"tbody",{},[138,154,155,159,162],{},[156,157,158],"td",{},"Model \u002F provider",[156,160,161],{},"Inference, text generation, and proposed tool calls",[156,163,164],{},"Decision quality, speed, price, privacy, and context length",[138,166,167,170,173],{},[156,168,169],{},"Agent harness",[156,171,172],{},"Agent loop, tools, context, permissions, and delegation",[156,174,175],{},"Workflow, available capabilities, safety boundaries, and interaction model",[138,177,178,181,184],{},[156,179,180],{},"Repository contract",[156,182,183],{},"Rules, design knowledge, procedures, and task evidence",[156,185,186],{},"The development method shared by the team",[138,188,189,192,195],{},[156,190,191],{},"Deterministic enforcement",[156,193,194],{},"Tests, linting, builds, CI, and approvals",[156,196,197],{},"Acceptance criteria and pass\u002Ffail decisions",[20,199,200],{},"This separation preserves the repository contract and verification methods when the harness changes. When only the model changes, keeping the harness and task fixed makes comparison easier.",[202,203,209],"pre",{"className":204,"code":206,"language":207,"meta":208},[205],"language-text","Model \u002F Provider\n       ↓\nAgent Harness\n       ↓\nRepository Contract\n       ↓\nTests \u002F CI \u002F Human Review\n","text","",[49,210,206],{"__ignoreMap":208},[20,212,213],{},"No lower layer should trust the layer above it completely. A model may produce a syntactically valid tool call that is not authorized for the current task. Harness permissions and acceptance checks in scripts or CI should remain separate.",[15,215,217],{"id":216},"make-the-repository-authoritative","Make the Repository Authoritative",[20,219,220],{},"The repository should contain a map to the information an agent needs, not a giant instruction file that sends everything to the model on every task.",[202,222,225],{"className":223,"code":224,"language":207,"meta":208},[205],"Repository\n├─ AGENTS.md          # Short rules and routes to deeper sources\n├─ docs\u002F              # Authoritative design, decisions, and operations\n├─ .agents\u002Fskills\u002F    # Reusable procedures loaded when relevant\n├─ scripts\u002F           # Deterministic repeated operations\n└─ tests \u002F CI         # Machine-checked completion criteria\n",[49,226,224],{"__ignoreMap":208},[20,228,229,233,234,236,237,239,240,243],{},[108,230,232],{"href":117,"rel":231},[112],"OpenAI's harness-engineering report"," says that a large ",[49,235,51],{}," consumed task context, obscured priority, and became stale. Its implementation instead uses ",[49,238,51],{}," as a table of contents and stores detailed knowledge under ",[49,241,242],{},"docs\u002F",".",[20,245,246],{},"The goal is not to store everything in the repository. The destination should match the information's role.",[132,248,249,262],{},[135,250,251],{},[138,252,253,256,259],{},[141,254,255],{},"Location",[141,257,258],{},"Suitable content",[141,260,261],{},"Content to avoid",[151,263,264,276,287,298,309,320],{},[138,265,266,270,273],{},[156,267,268],{},[49,269,51],{},[156,271,272],{},"Always-relevant constraints and routes to deeper documents",[156,274,275],{},"Long background explanations, run history, and every product setting",[138,277,278,281,284],{},[156,279,280],{},"Documentation",[156,282,283],{},"Design decisions, operating rules, public knowledge, and rationale",[156,285,286],{},"Frequently changing temporary state",[138,288,289,292,295],{},[156,290,291],{},"Skill",[156,293,294],{},"Reusable procedures loaded for a defined situation",[156,296,297],{},"One-off prompts and unverified claims",[138,299,300,303,306],{},[156,301,302],{},"Script \u002F CI",[156,304,305],{},"Processes that should return the same decision for the same input",[156,307,308],{},"Ambiguous design judgment and final editorial decisions",[138,310,311,314,317],{},[156,312,313],{},"Memory",[156,315,316],{},"Personal preferences and compact continuity",[156,318,319],{},"The team's only source of truth or an auditable decision",[138,321,322,325,328],{},[156,323,324],{},"Issue \u002F PR",[156,326,327],{},"Task purpose, status, review, and links to changes",[156,329,330],{},"The only location for long-lived design knowledge",[20,332,333],{},"Memory is useful, but it is tied to an agent or profile and can retain incorrect or outdated assumptions. Decisions shared across people and harnesses should be moved into documentation or issues, while memory keeps small preferences or references to the authoritative source.",[15,335,337],{"id":336},"what-is-portable-and-what-needs-an-adapter","What Is Portable and What Needs an Adapter",[20,339,340],{},"A repository-centered design does not mean that every harness discovers the same file in the same way.",[20,342,343,348,349,352,353,358],{},[108,344,347],{"href":345,"rel":346},"https:\u002F\u002Fopencode.ai\u002Fdocs\u002Fskills",[112],"OpenCode"," discovers ",[49,350,351],{},".agents\u002Fskills\u002F\u003Cname>\u002FSKILL.md"," in a project and loads the full skill only when needed. It can configure ",[108,354,357],{"href":355,"rel":356},"https:\u002F\u002Fopencode.ai\u002Fv2\u002Fdocs\u002Fagents",[112],"a model and tool permissions per agent"," and supports multiple providers, including local models.",[20,360,361,362,367,368,373,374,379,380,383,384,387],{},"Qwen Code also supports ",[108,363,366],{"href":364,"rel":365},"https:\u002F\u002Fgithub.com\u002FQwenLM\u002Fqwen-code\u002Fblob\u002Fmain\u002Fdocs\u002Fusers\u002Ffeatures\u002Fskills.md",[112],"project skills"," and ",[108,369,372],{"href":370,"rel":371},"https:\u002F\u002Fgithub.com\u002FQwenLM\u002Fqwen-code\u002Fblob\u002Fmain\u002Fdocs\u002Fusers\u002Ffeatures\u002Fsub-agents.md",[112],"subagents"," and can connect to Ollama, vLLM, LM Studio, and other local servers through an ",[108,375,378],{"href":376,"rel":377},"https:\u002F\u002Fqwenlm.github.io\u002Fqwen-code-docs\u002Fen\u002Fusers\u002Fconfiguration\u002Fmodel-providers\u002F",[112],"OpenAI-compatible API",". Its current documented paths, however, are ",[49,381,382],{},".qwen\u002Fskills\u002F"," for skills and ",[49,385,386],{},".qwen\u002Fagents\u002F"," for subagents.",[20,389,390,395,396,398,399,404,405,410],{},[108,391,394],{"href":392,"rel":393},"https:\u002F\u002Fhermes-agent.nousresearch.com\u002Fdocs\u002Fuser-guide\u002Ffeatures\u002Fcontext-files",[112],"Hermes Agent"," reads ",[49,397,51],{}," as project context and loads ",[108,400,403],{"href":401,"rel":402},"https:\u002F\u002Fhermes-agent.nousresearch.com\u002Fdocs\u002Fuser-guide\u002Ffeatures\u002Fskills\u002F",[112],"skills"," on demand. Its default skill directory belongs to a profile, but it can scan external directories and redirect creation to a Git-tracked directory. It can gate ",[108,406,409],{"href":407,"rel":408},"https:\u002F\u002Fhermes-agent.nousresearch.com\u002Fdocs\u002Fuser-guide\u002Ffeatures\u002Fmemory\u002F",[112],"memory and skill writes"," on human approval, but both gates allow free writes by default.",[132,412,413,426],{},[135,414,415],{},[138,416,417,420,423],{},[141,418,419],{},"Asset",[141,421,422],{},"Portability",[141,424,425],{},"Recommended treatment",[151,427,428,439,451,465,476,486],{},[138,429,430,433,436],{},[156,431,432],{},"Markdown design and operational documents",[156,434,435],{},"High",[156,437,438],{},"Keep one authoritative repository version",[138,440,441,445,448],{},[156,442,443],{},[49,444,51],{},[156,446,447],{},"Relatively high",[156,449,450],{},"Verify each harness's scope and precedence rules",[138,452,453,459,462],{},[156,454,455,456],{},"The content of a ",[49,457,458],{},"SKILL.md",[156,460,461],{},"Medium",[156,463,464],{},"Choose a source of truth and add path synchronization or a thin wrapper if needed",[138,466,467,470,473],{},[156,468,469],{},"Agent and subagent definitions",[156,471,472],{},"Low",[156,474,475],{},"Keep them as minimal harness-specific configuration",[138,477,478,481,483],{},[156,479,480],{},"Provider authentication and permissions",[156,482,472],{},[156,484,485],{},"Configure them explicitly per harness and keep them out of Git",[138,487,488,491,493],{},[156,489,490],{},"Sessions, memory, and task queues",[156,492,472],{},[156,494,495],{},"Return durable decisions to documentation, issues, and PRs",[20,497,498],{},"The objective is not zero adapters. It is one semantic source with small product-specific connections.",[15,500,502],{"id":501},"compare-products-by-their-emphasis","Compare Products by Their Emphasis",[20,504,505],{},"As of September 3, 2026, these harnesses can be positioned by their documented capabilities and the reason for evaluating them. This is not an overall ranking.",[132,507,508,521],{},[135,509,510],{},[138,511,512,515,518],{},[141,513,514],{},"Harness",[141,516,517],{},"Documented emphasis",[141,519,520],{},"Reason to evaluate it",[151,522,523,536,550,561],{},[138,524,525,528,533],{},[156,526,527],{},"Codex",[156,529,530,532],{},[49,531,51],{},", skills, sandboxing, and tools for long-running tasks",[156,534,535],{},"Establish a baseline that includes the current workflow and human review",[138,537,538,540,547],{},[156,539,347],{},[156,541,542,543,546],{},"Many providers and local models, ",[49,544,545],{},".agents\u002Fskills\u002F",", and per-agent models and permissions",[156,548,549],{},"Change models while reusing repository assets",[138,551,552,555,558],{},[156,553,554],{},"Qwen Code",[156,556,557],{},"Multiple providers, OpenAI-compatible local servers, project skills, and subagents",[156,559,560],{},"Compare configurations that include Qwen or another local model",[138,562,563,565,568],{},[156,564,394],{},[156,566,567],{},"Persistent memory, skill creation, project context, and long-running agents",[156,569,570],{},"Evaluate learning across tasks and the creation of knowledge candidates",[20,572,573,574,577,578,367,580,582],{},"OpenCode documents support for more than 75 providers and local models, with a ",[49,575,576],{},"provider\u002Fmodel"," choice per agent. For a repository that already manages ",[49,579,51],{},[49,581,545],{},", it is a practical candidate for a second coding harness.",[20,584,585,586,243],{},"Despite its name, Qwen Code documents protocols for OpenAI, Anthropic, Gemini, and OpenAI-compatible local servers. Migrating existing skills still requires an explicit synchronization policy for ",[49,587,382],{},[20,589,590,591,367,594,597],{},"Hermes Agent extends beyond short coding tasks with memory, skill updates, profiles, and long-running operation. Before allowing it to modify knowledge automatically, enable ",[49,592,593],{},"memory.write_approval",[49,595,596],{},"skills.write_approval"," and review every candidate. Its security documentation also describes deny rules as guardrails for accidental behavior rather than a sandbox against an adversarial process. Use an isolated runtime where the threat model requires it.",[15,599,601],{"id":600},"derive-model-routing-from-measurements","Derive Model Routing from Measurements",[20,603,604],{},"The ability to change models does not justify immediately fixing rules such as \"research is local\" and \"implementation is cloud.\" Task difficulty and language quality can invalidate those assumptions.",[20,606,607],{},"Record comparable results with the same repository, task contract, and completion criteria.",[34,609,610,613,616,619,622,625,628],{},[37,611,612],{},"percentage of completion criteria satisfied;",[37,614,615],{},"volume and severity of human edits;",[37,617,618],{},"compliance with rules, allowed paths, and task scope;",[37,620,621],{},"retries and reasons for interruption;",[37,623,624],{},"elapsed time, token use, and inference cost;",[37,626,627],{},"memory, GPU occupancy, and interference with other work for local inference; and",[37,629,630],{},"defects discovered during review.",[20,632,633,638],{},[108,634,637],{"href":635,"rel":636},"https:\u002F\u002Fwww.perplexity.ai\u002Fuk\u002Fhub\u002Fblog\u002Fa-local-first-agent-for-private-and-cost-effective-knowledge-work",[112],"Perplexity"," reported different results for Portable Computer, Pi, and Hermes while holding Qwen 3.8 27B constant on its own Local Knowledge Work Bench. This first-party, unpublished benchmark does not establish a universal ranking, but it is an example of a harness affecting results while the model remains fixed.",[20,640,641],{},"Expand model routing incrementally.",[643,644,645,648,651,654,657],"ol",{},[37,646,647],{},"Record a few bounded tasks with the current harness and model as a baseline.",[37,649,650],{},"Run the same task category in a second harness and compare permissions and review effort as well as output.",[37,652,653],{},"Start local-model evaluation with low-risk formatting, search, or deterministic verification tasks.",[37,655,656],{},"Promote only roles with a stable success rate into the default routing policy.",[37,658,659],{},"Repeat the evaluation when the model or harness version changes materially.",[15,661,663],{"id":662},"delegate-bounded-judgments-to-a-specialized-model","Delegate Bounded Judgments to a Specialized Model",[20,665,666,667,670,671,676],{},"When the candidate skills or reference documents are already known, their selection can be evaluated as a separate model responsibility. TypeSafe's Jev returns selections and evaluations against a supplied ",[49,668,669],{},"state",". Its ",[108,672,675],{"href":673,"rel":674},"https:\u002F\u002Fdocs.typesafe.ai\u002Fconcepts\u002Fhow-to-build-with-system-one",[112],"documented design approach"," assigns narrow judgments to the model while keeping control flow, deterministic rules, and actual operations in code.",[132,678,679,692],{},[135,680,681],{},[138,682,683,686,689],{},[141,684,685],{},"Primitive",[141,687,688],{},"Returns",[141,690,691],{},"Example harness use",[151,693,694,705,716],{},[138,695,696,699,702],{},[156,697,698],{},"Choice",[156,700,701],{},"A selected option and probability distribution",[156,703,704],{},"Suggest a skill or classify a request",[138,706,707,710,713],{},[156,708,709],{},"Score",[156,711,712],{},"A rating against defined levels and probability distribution",[156,714,715],{},"Evaluate relevance or information completeness",[138,717,718,721,724],{},[156,719,720],{},"Noul",[156,722,723],{},"The probability of yes",[156,725,726],{},"Check candidate fit or the presence of evidence",[20,728,729,730,735],{},"These ",[108,731,734],{"href":732,"rel":733},"https:\u002F\u002Fdocs.typesafe.ai\u002Fintroduction",[112],"typed questions"," are evaluated independently against the same state. They do not pass one answer into the next question as sequential reasoning. Jev does not handle text or code generation; surrounding code and generative LLMs collect context, retrieve documents, explain findings, and implement changes.",[20,737,738,739,744],{},"TypeSafe's ",[108,740,743],{"href":741,"rel":742},"https:\u002F\u002Fdocs.typesafe.ai\u002Fcookbooks\u002Fskill_suggestion",[112],"skill suggestion experiment"," illustrates the approach. Two API calls rank all candidates and inspect the top three more closely before suggesting at most one skill. The agent retains its original skill index and its own judgment.",[132,746,747,760],{},[135,748,749],{},[138,750,751,754,757],{},[141,752,753],{},"Provider-reported error rate",[141,755,756],{},"Agent alone",[141,758,759],{},"With a Jev suggestion",[151,761,762,773],{},[138,763,764,767,770],{},[156,765,766],{},"Wrong skill or no skill loaded when one fits",[156,768,769],{},"16.8%",[156,771,772],{},"7.3%",[138,774,775,778,781],{},[156,776,777],{},"Unnecessary skill loaded when none fits",[156,779,780],{},"9.8%",[156,782,783],{},"4.0%",[20,785,786,787,792],{},"This is a provider experiment using Jev 1.12, Claude Haiku 4.5, 182 skills from Hermes, and 488 requests. Some requests were generated, and the measurement covers skill loading in the agent's first response. It does not establish the same improvement for Jev 1.13, Japanese, or Copilot, nor does it prove better end-to-end work quality or token savings. The ",[108,788,791],{"href":789,"rel":790},"https:\u002F\u002Fgithub.com\u002Ftypesafe-ai\u002Fskills",[112],"official TypeSafe skill"," likewise supplies guidance for designing and implementing API workflows; installing it does not itself replace an existing agent's skill-selection mechanism.",[20,794,795],{},"Measure candidate selection, actual loading by the agent, instruction compliance, and the final artifact separately. Include requests with no suitable skill, missing context, or several required skills, and compare total time, cost, and missed skills against the existing selection method. Mandatory rules and authorization checks must remain applicable even when no candidate is selected.",[20,797,798,803,804,809,810,813],{},[108,799,802],{"href":800,"rel":801},"https:\u002F\u002Fdocs.typesafe.ai\u002Fconfidence",[112],"Confidence"," is derived from the probability distribution of a Choice or Score answer and should not be treated as accuracy. Tune thresholds on your own data and record the model and question wording. The delivery path confirmed in the ",[108,805,808],{"href":806,"rel":807},"https:\u002F\u002Fdocs.typesafe.ai\u002Fmodels",[112],"model documentation"," is a cloud API. As of September 19, 2026, the model ID is ",[49,811,812],{},"jev-1.13.0",", and the documentation notes lower accuracy outside English. Adding it to a local LLM setup still sends the supplied state externally. Not training on customer data is distinct from not storing or retaining it.",[20,815,816,817,822],{},"The ",[108,818,821],{"href":819,"rel":820},"https:\u002F\u002Fdocs.typesafe.ai\u002Fprimitives",[112],"official multi-question contract"," allows independent judgments over the same state to be batched. If an earlier answer determines which documents to fetch or which options to evaluate next, a separate request is necessary. The full-candidate ranking followed by a top-three reassessment above has a dependency between stages and cannot simply be treated as one call.",[824,825,827],"h3",{"id":826},"compare-local-decision-models-by-readout-and-runtime","Compare Local Decision Models by Readout and Runtime",[20,829,830],{},"Checked on September 29, 2026. There are more local candidates for the judgment delegation illustrated with Jev. These are independent implementations, not official local editions of TypeSafe's Jev.",[132,832,833,846],{},[135,834,835],{},[138,836,837,840,843],{},[141,838,839],{},"Candidate",[141,841,842],{},"Decision readout",[141,844,845],{},"Runtime and evaluation considerations",[151,847,848,863,878,893],{},[138,849,850,857,860],{},[156,851,852],{},[108,853,856],{"href":854,"rel":855},"https:\u002F\u002Fgithub.com\u002Fjaredpalmer\u002Fkev",[112],"Kev",[156,858,859],{},"A trained pointer head on Qwen scores candidates",[156,861,862],{},"Dedicated server; CUDA \u002F ROCm, with automatic MLX use on Apple Silicon",[138,864,865,872,875],{},[156,866,867],{},[108,868,871],{"href":869,"rel":870},"https:\u002F\u002Fgithub.com\u002FTheoLeeCJ\u002FSemIf-OpenJev",[112],"SemIf",[156,873,874],{},"Reads option logits from an existing model without generating an answer",[156,876,877],{},"Pin backend, quantization, prompt, and probability calibration",[138,879,880,887,890],{},[156,881,882],{},[108,883,886],{"href":884,"rel":885},"https:\u002F\u002Fhuggingface.co\u002Fconvaiinnovations\u002Flaya-multilingual",[112],"Laya multilingual",[156,888,889],{},"A 322M encoder and decision head evaluate choice, score, and yes\u002Fno questions",[156,891,892],{},"Small size still requires workload-specific accuracy and calibration checks",[138,894,895,902,905],{},[156,896,897],{},[108,898,901],{"href":899,"rel":900},"https:\u002F\u002Fhuggingface.co\u002Ftogethercomputer\u002FTev1-4B-experimental",[112],"Tev1-4B-experimental",[156,903,904],{},"Generates an option letter through Qwen's standard LM head",[156,906,907],{},"An experimental model choosing among 2–24 options, not a non-generative classifier",[20,909,910],{},"Kev's MLX path is implemented at the check date. This differs from loading an ordinary chat GGUF. Similar API formats do not establish identical question counts, option limits, context budgets, accuracy, or confidence semantics. Validate your own inputs and error handling when changing providers.",[20,912,913],{},"In SemIf's author measurements, the same Qwen3.5-4B model, state, and 21 binary questions on an RTX 3090 took 1.023 seconds with direct logit readout versus 5.332 seconds to generate a JSON array, each the median of three runs. Answers agreed on only 18 of 21 questions, so this does not demonstrate a speedup of semantically identical processing. Measure decision quality separately from speed.",[20,915,916],{},"Laya multilingual's model card discloses uncalibrated, overconfident outputs and zero-shot typed-decisions accuracy of 0.342, against 0.318 random and 0.461 majority-class baselines. Multilingual support alone does not establish adequate Japanese skill selection or document evaluation; consider workload-specific evaluation, fine-tuning, and calibration. Tev1 also lacks comprehensive multilingual, prompt-injection, calibration, and out-of-distribution evaluation, and its weight license was still being finalized at the check date. Do not substitute a base-model license for the derived weights' distribution terms.",[20,918,919,924,925,928,929,932,933,937],{},[108,920,923],{"href":921,"rel":922},"https:\u002F\u002Fgithub.com\u002Fddalcu\u002Fmlx-serve\u002Freleases\u002Ftag\u002Fv26.9.6",[112],"mlx-serve v26.9.6"," adds ",[49,926,927],{},"POST \u002Fv1\u002Fdecisions"," for Laya checkpoints and documents batching concurrent requests. This API should not be treated as identical to Jev's ",[49,930,931],{},"\u002Fv1\u002Fsystemone",". A working connection also does not establish sufficient quality for skill selection or game decisions. Alongside the ",[108,934,936],{"href":935},"\u002Fen\u002Fknowledge\u002Fgame\u002Fllm\u002Fgame-npc-llm-adoption-boundaries","small game experiment",", compare response time including queues, wrong decisions, and fallback to existing behavior.",[20,939,940,941,946,947,950],{},"An ",[108,942,945],{"href":943,"rel":944},"https:\u002F\u002Fqiita.com\u002Fmoritalous\u002Fitems\u002F41c9402a5dd9d80fc7a9",[112],"individual implementation using Qwen3-VL-4B"," returns judgments about images from a one-token output and option log probabilities. This is a separate implementation that calls the model multiple times for multiple questions, not image support in Jev itself. Measure initial processing separately from cached processing, recording the model, quantization, runtime, and input reuse conditions. Verify that probabilities are available for every candidate; an option missing from ",[49,948,949],{},"top_logprobs"," must not be treated as having zero probability. Image-based document classification or skill suggestion remains an untested application idea.",[20,952,953,954,959],{},"Evaluate alternatives separately for output validity (parsing, allowed options, probability sums), decision quality against human-defined labels, probability calibration, and time and cost including retries. An ",[108,955,958],{"href":956,"rel":957},"https:\u002F\u002Fqiita.com\u002Fnasuvitz\u002Fitems\u002F4d27833640ff36b5d8c5",[112],"individual Nova Micro versus Jev comparison"," likewise separates output validation from agreement between models; agreement is not accuracy. Do not interchange generated probability values, token probabilities, and a dedicated decision model's probabilities as if they had identical semantics. Nova Micro is a cloud candidate, separate from local alternatives.",[15,961,963],{"id":962},"return-implementation-experience-to-the-knowledge-base","Return Implementation Experience to the Knowledge Base",[20,965,966],{},"When an agent identifies reusable knowledge during a task, it does not need to update the authoritative source immediately. Use a separate review loop.",[202,968,971],{"className":969,"code":970,"language":207,"meta":208},[205],"Implementation or research\n   ↓\nRecord a knowledge candidate\n   ↓\nClassify fact, decision, procedure, or temporary state\n   ↓\nDraft documentation, a skill, or a script\n   ↓\nTests, validation, and human review\n   ↓\nUpdate the authoritative repository source\n",[49,972,970],{"__ignoreMap":208},[20,974,975],{},"Repeated explanations are candidates for documentation. Repeated multi-step procedures are candidates for skills. Repeated pass\u002Ffail decisions are candidates for scripts or CI. One-off events and unverified inferences do not need to become durable knowledge.",[20,977,978],{},"This boundary retains the speed of agent-assisted discovery while reducing incorrect generalization, duplication, and stale knowledge.",[15,980,982],{"id":981},"select-and-reuse-accumulated-knowledge","Select and Reuse Accumulated Knowledge",[20,984,985,986,991],{},"When reusing saved notes, separate retrieval, semantic evaluation, text extraction, and explanation. TypeSafe's ",[108,987,990],{"href":988,"rel":989},"https:\u002F\u002Fdocs.typesafe.ai\u002Fcookbooks\u002Fclassifying_rag_passages",[112],"document-classification example"," evaluates retrieved passages for relevance, usable evidence, and contradictions to the question's assumptions, then lets code route the material. Conflicting information is passed to the generative LLM in a separate block.",[20,993,994],{},"The following is a possible adaptation for repository notes. It is a proposed architecture, not an official ready-made integration between conversation-history search such as funes and Jev, or a system measured for this article.",[202,996,999],{"className":997,"code":998,"language":207,"meta":208},[205],"Current problem and task state\n   ↓\nRetrieve candidate notes\n   ↓\nEvaluate relevance, evidence, and conflicts separately with Jev\n   ↓\nCode selects documents and retrieves text, sources, and status\n   ↓\nA generative LLM explains and investigates applicability to current code\n",[49,1000,998],{"__ignoreMap":208},[20,1002,1003],{},"Give notes an ID, affected feature and revision, problem, cause, adopted solution, failed approaches, validation results, status such as adopted, retired, or superseded, and source links to issues or PRs. Code can extract existing fields; a generative LLM can explain findings across several documents. Selection by Jev should not itself persist history into memory or modify the authoritative source.",[20,1005,1006],{},"A previously successful fix is not necessarily correct today. Include the current specification and relevant revision, and preserve failed approaches and contradictions instead of discarding them merely as low-relevance material. Handle explicit retirement status and date comparisons in code. When retrieval finds no suitable note, provide a path back to conversation-history search or the original code.",[20,1008,1009,1014],{},[108,1010,1013],{"href":1011,"rel":1012},"https:\u002F\u002Fdocs.typesafe.ai\u002Fmodel-jaggedness\u002Fjev-1.13",[112],"Known limitations of Jev 1.13"," include numbers and dates, multi-step judgments, states crowded with irrelevant information, and adversarial text in the input. Document selection does not verify factual correctness or guarantee protection against prompt injection. Evaluate missed important documents and preservation of conflicts on real Japanese inputs, then measure incremental value and total cost against existing retrieval and reranking before expanding adoption.",[824,1016,1018],{"id":1017},"evaluate-decision-placement-and-false-blocking","Evaluate Decision Placement and False Blocking",[20,1020,1021],{},"A pre-retrieval judgment asks whether a request belongs within the search source's scope; a post-retrieval, pre-generation judgment asks whether the retrieved material supports an answer. Give the former a scope description and necessary conversation history, and continue retrieval or seek clarification when uncertain. If the latter finds insufficient evidence, broaden retrieval or return to original sources and code. Missing evidence in the retrieved material does not establish that the repository contains no answer. Route contradictory material to investigation and explanation rather than stopping solely because no supporting passage was selected.",[20,1023,1024,1025,1030],{},"In an ",[108,1026,1029],{"href":1027,"rel":1028},"https:\u002F\u002Fqiita.com\u002Fkikuziro\u002Fitems\u002F2be9091b328d8b844640",[112],"individual RAG experiment with Jev 1.13",", answerability classification over 400 fictional chunks and 120 questions reached 99.2% accuracy but falsely blocked 1.7% of answerable questions. This does not guarantee the same quality for Company's skill selection or real Japanese documents. Do not count skipped answerable requests as successful processing savings.",[20,1032,1033],{},"Assuming a constant downstream cost per request, a simple savings condition is: decision cost \u003C fraction of requests legitimately skipped × downstream cost. Include additional retrieval, retry, and validation costs in an actual comparison. Since requests that continue also incur the decision latency, record waiting times separately for answered requests and legitimately skipped requests.",[15,1035,1037],{"id":1036},"what-to-evaluate-first","What to Evaluate First",[20,1039,1040,1041,1043],{},"Add a second harness as a comparison target instead of replacing the existing environment. If the repository already contains ",[49,1042,51],{}," and skills, OpenCode is a reasonable first candidate because it can test both reuse of those assets and provider switching.",[20,1045,1046],{},"Keep the first evaluation narrow.",[34,1048,1049,1052,1055,1058,1061],{},[37,1050,1051],{},"Use a dedicated branch or worktree, and do not let two harnesses edit the same files concurrently.",[37,1053,1054],{},"Begin with reversible tasks such as read-only research or small documentation changes.",[37,1056,1057],{},"Explicitly restrict Git writes, network access, and credential handling.",[37,1059,1060],{},"Use the current harness as a baseline and compare completion rate and human correction.",[37,1062,1063],{},"Do not compare a new harness and a new local model at the same time; hold one variable constant first.",[20,1065,1066],{},"Evaluate Hermes Agent separately when persistent memory or automatic skill-candidate generation becomes a real requirement. When local Qwen is the priority, either OpenCode or Qwen Code can connect to an OpenAI-compatible local server. Make the adoption decision from comparable task results rather than product names.",[15,1068,1070],{"id":1069},"checklist","Checklist",[20,1072,1073],{},"Before adding an agent harness, confirm that:",[34,1075,1076,1079,1082,1085,1088,1091,1094,1097],{},[37,1077,1078],{},"the authoritative repository source is distinct from harness-specific configuration;",[37,1080,1081],{},"secrets, tokens, and personal memory are excluded from version control;",[37,1083,1084],{},"readable paths and writable paths are defined separately;",[37,1086,1087],{},"generated skills and memory do not become active before human review;",[37,1089,1090],{},"deterministic checks such as tests, linting, and builds are not left to prompts alone;",[37,1092,1093],{},"parallel agents cannot silently claim the same files;",[37,1095,1096],{},"harness, model, version, reasoning effort, and results can be compared; and",[37,1098,1099],{},"failures can be rolled back without leaving credentials or external state behind.",[15,1101,1103],{"id":1102},"limitations","Limitations",[34,1105,1106,1109,1115,1118,1121,1124,1127],{},[37,1107,1108],{},"Harnesses, skill formats, and settings change. Recheck official documentation for the version being adopted.",[37,1110,1111,1112,1114],{},"Similar ",[49,1113,458],{}," formats do not guarantee identical discovery paths, frontmatter, tools, or permission semantics.",[37,1116,1117],{},"Local execution is not automatically private, inexpensive, or fast. Evaluate external tools, telemetry, update checks, and hardware cost.",[37,1119,1120],{},"Memory and automatic learning reuse errors too. Define approval, provenance, refresh, and deletion procedures first.",[37,1122,1123],{},"A permission dialog or deny rule is not equivalent to operating-system isolation.",[37,1125,1126],{},"Multi-agent execution helps only when work can be separated. Measure coordination, review, and conflict-resolution overhead.",[37,1128,1129],{},"A vendor benchmark is evidence for its model, harness, hardware, and task set, not a promise for another repository.",[15,1131,1133],{"id":1132},"references","References",[34,1135,1136,1142,1148,1154,1160,1167,1173,1179,1185,1191,1197,1203,1210,1216,1222],{},[37,1137,1138],{},[108,1139,1141],{"href":117,"rel":1140},[112],"OpenAI: Harness engineering",[37,1143,1144],{},[108,1145,1147],{"href":110,"rel":1146},[112],"OpenAI: Unrolling the Codex agent loop",[37,1149,1150],{},[108,1151,1153],{"href":355,"rel":1152},[112],"OpenCode: Agents",[37,1155,1156],{},[108,1157,1159],{"href":345,"rel":1158},[112],"OpenCode: Agent Skills",[37,1161,1162],{},[108,1163,1166],{"href":1164,"rel":1165},"https:\u002F\u002Fdev.opencode.ai\u002Fdocs\u002Fproviders\u002F",[112],"OpenCode: Providers",[37,1168,1169],{},[108,1170,1172],{"href":376,"rel":1171},[112],"Qwen Code: Model Providers",[37,1174,1175],{},[108,1176,1178],{"href":364,"rel":1177},[112],"Qwen Code: Skills",[37,1180,1181],{},[108,1182,1184],{"href":370,"rel":1183},[112],"Qwen Code: Subagents",[37,1186,1187],{},[108,1188,1190],{"href":392,"rel":1189},[112],"Hermes Agent: Context Files",[37,1192,1193],{},[108,1194,1196],{"href":401,"rel":1195},[112],"Hermes Agent: Skills System",[37,1198,1199],{},[108,1200,1202],{"href":407,"rel":1201},[112],"Hermes Agent: Persistent Memory",[37,1204,1205],{},[108,1206,1209],{"href":1207,"rel":1208},"https:\u002F\u002Fhermes-agent.nousresearch.com\u002Fdocs\u002Fuser-guide\u002Fsecurity",[112],"Hermes Agent: Security",[37,1211,1212],{},[108,1213,1215],{"href":635,"rel":1214},[112],"Perplexity: A Local-First Agent for Private Knowledge Work",[37,1217,1218],{},[108,1219,1221],{"href":1220},"\u002Fen\u002Fknowledge\u002Fai\u002Fllm\u002Flocal-llm-landscape-2026","Major Local LLMs in 2026",[37,1223,1224],{},[108,1225,1227],{"href":1226},"\u002Fen\u002Fknowledge\u002Fweb\u002Fgithub\u002Fgithub-projects-issues-prs-team-coordination","Coordinating Multiple Agents with GitHub Projects, Issues, and Pull Requests",{"title":208,"searchDepth":1229,"depth":1229,"links":1230},2,[1231,1232,1233,1234,1235,1236,1237,1238,1239,1243,1244,1247,1248,1249,1250],{"id":17,"depth":1229,"text":18},{"id":31,"depth":1229,"text":32},{"id":70,"depth":1229,"text":71},{"id":126,"depth":1229,"text":127},{"id":216,"depth":1229,"text":217},{"id":336,"depth":1229,"text":337},{"id":501,"depth":1229,"text":502},{"id":600,"depth":1229,"text":601},{"id":662,"depth":1229,"text":663,"children":1240},[1241],{"id":826,"depth":1242,"text":827},3,{"id":962,"depth":1229,"text":963},{"id":981,"depth":1229,"text":982,"children":1245},[1246],{"id":1017,"depth":1242,"text":1018},{"id":1036,"depth":1229,"text":1037},{"id":1069,"depth":1229,"text":1070},{"id":1102,"depth":1229,"text":1103},{"id":1132,"depth":1229,"text":1133},"Learn how to separate models from agent harnesses, keep knowledge and procedures authoritative in the repository, and evaluate coding agents and local LLMs incrementally.","md",{},true,"\u002Fknowledge\u002Fai\u002Fagents\u002Frepository-centered-agent-harness","2026-09-03",{"title":5,"description":1251},"published","knowledge\u002Fai\u002Fagents\u002Frepository-centered-agent-harness",[1261,1262,1263,1264,1265],"agent-harness","agentic-coding","model-routing","agent-skills","knowledge-management",[1267,1268],"ai-agent","git","2026-10-01","ZtmKOv4EbP9oukMSwv-U4whs8hf-zXrh_Q5T7l-CNx4",{"ja":1254,"en":1254},1791891818830]