Managing AI Agents Like an Offshore Team

Six months building a production site with AI agents, six failures that reached production, and the checks I added after each one.

Managing AI Agents Like an Offshore Team

Managing AI agents resembles managing an offshore team in one important respect: giving instructions is not enough. After six months building this site with agents, I turn failures I can describe as testable conditions into automated checks. I still review changes and verify the live site myself.

TL;DR (Executive Summary)

  • Problem: I estimate that AI agents wrote about 90% of the code for my three-language site. They still broke documented rules. One bug remained in production for 25 days while the local environment behaved correctly.
  • Approach: I use four layers: written rules, build gates, a Claude Code Stop hook, and checks in the live environment.
  • Outcome: I added corresponding checks after six production failures. I do not have a measured defect rate before and after the changes.

Context: one person, one site, most code written by AI

Nguyenchau.dev runs in Vietnamese, English, and Japanese on Cloudflare Pages. Between 18 March and 25 September 2026, the repo recorded 1,279 commits. Git shows 694 commits with a Claude co-author line. Including Codex, I estimate that I wrote only about 10% of the code myself; the co-author count does not measure what percentage of the code AI wrote.

The reason is practical: I no longer have as much time to code as I used to. But I have not handed everything over. I keep three responsibilities:

  1. Set the goal and write it down so the agent can read it.
  2. Supply enough context for the agent to reason: data, background, past decisions, and why those decisions were made.
  3. Accept the result by reviewing changes and checking the live environment.

I often delegate code, information organization, and copy edits to agents. That division of work is close to my role as a Delivery Manager on an offshore team. Many of the lessons below first came from working with people.

An agent can follow an outdated document perfectly

When I still treated the site as a way to find clients, the agent instructions said content pages should include an author block and an invitation to get in touch. Agents added that block to /learn three times: on 29 July, 4 August, and 10 August 2026. Each time, the reasoning sounded plausible: strengthen E-E-A-T signals and add another contact point.

In those three cases, the agents followed an old goal in the documentation. I later removed the author block from /learn for privacy reasons and abandoned the idea of turning the site into a lead funnel. The agents could not know that unless I updated the material they read. My first task was therefore to rewrite the instructions, including the history of those three reintroductions and why the block should stay gone.

Offshore teams face the same problem: a team can execute an obsolete specification diligently and still deliver the wrong thing. I wrote about making tacit knowledge explicit in why “Japanese quality” is hard to reproduce in a Vietnam offshore team. With agents, a new decision has to reach a source they can actually read.

Six production failures and the checks I added

Correct documentation was still insufficient. These failures reached production. Here is why I missed them initially and what I added afterward:

Production failure Why I missed it Check added afterward
Thirty-four English and Japanese URLs across 17 pages redirected to Vietnamese for 25 days (8 August–2 September 2026) A fix for a different bug caused it; the local environment could not reproduce it A validator locks down the language-routing decision table, plus a direct production check after deployment
The chatbot emailed a visitor's answer to me although the chat box said the answer was “stored temporarily on this device” There were no chatbot tests; I found it during review A validator checks five behavioral rules, with tests for the validator itself
English and Japanese structured data described the author differently from Vietnamese: missing credentials, wrong title, wrong URL (three occurrences) The static version was correct, so the old check passed; the defect was in JavaScript that rewrote schema on language change A validator reads that JavaScript as well
FAQ entries appeared in structured data but not on the page Structured data is invisible on the page, so visual inspection missed it A validator compares visible FAQs with the schema
The homepage showed one sentence to crawlers and another to visitors HTML and JavaScript versions existed side by side; the README already warned about this failure A validator compares the two versions sentence by sentence
JSON-LD used two phone-number formats for the same business The information was written by hand in 20 files A validator compares each declared value with one canonical source

Ten validators currently run in the build command before the interface is bundled. If one detects a violation, the build stops and that version is not deployed. They only check conditions that have been defined.

The chatbot failure is the one I am least comfortable sharing. The message sent contained no contact details, but the interface had told visitors one thing while the system did another. I found it in a review on 5 September 2026, fixed it the same day, and changed the chatbot so it sends nothing unless the visitor chooses to leave contact details.

The 34-URL failure: why “it works locally” proved too little

This case is worth examining because it exposes a limit of both agents and tests.

On 8 August, an agent audited the site and found a real defect: several Vietnamese-only pages still returned content at English-language URLs, creating dozens of duplicate URLs. Its fix read the page body to see whether a translation existed and redirected to Vietnamese if it did not.

The logic looked sound. On Cloudflare, however, middleware cannot read a static asset response as page text in this situation. Every static page was consequently treated as untranslated, including 17 pages that really had translations. In the local simulation, the body remained readable, so the checks passed.

I noticed something suspicious in the language-routing changes while reviewing recent commits. Locally, they worked. In production, they failed. I gave the agent that evidence, and it changed the routing decision to use data available at build time rather than trying to read the page body at request time.

I took three lessons from it:

  • A fix for bug A can create a larger bug B. The agent solved the immediate problem without seeing a consequence in an environment it had not been shown.
  • A passing local test only establishes local behavior. I now check production directly after a deployment that changes routing. For an offshore team, this is why UAT has to run in the client's environment.
  • Human review remains the last line of discovery. No existing gate caught this bug before it happened because we had not yet specified that failure mode.

When should an instruction become an enforced check?

The repo's README has a “Common AI Mistakes — DON'T DO THIS” section. Item 4b warns against editing JSX without updating the HTML fallback and notes that it had already happened on three landing pages. On 6 August 2026, it happened again in the homepage form. The same day, I turned that rule into a validator.

A commit that created the /learn Stop hook summed up the idea: turn a rule in CLAUDE.md from a request into a guarantee. In Claude Code, the Stop hook runs when the agent is about to end its turn. If /learn content has changed but its validator still fails, the hook blocks the stop and returns the error for the agent to fix. In a human team, this is like refusing to mark work “done” before it meets the Definition of Done. I documented the hook setup in my note on Claude Code permissions and hooks.

I applied the same principle in a client project. Before I understood the model's behavior in depth, seven out of ten AI features in a SaaS system I managed for a Japanese client relied entirely on prompts: they depended on the model both reasoning correctly and following instructions. The simplest example is an honorific. Content addressed to a Japanese customer had to include 様, but the model sometimes omitted it despite the prompt rule.

I then proposed a refactor. Code would use customer data to select the applicable honorific, including 様 under the rule for Japanese customers. The prompt would handle only what genuinely required judgment. There were other cases, but the NDA limits me to this simple example. My rule of thumb is: if a business rule can be determined from known inputs, do not leave it to the model's memory.

By a “gate” I mean an automated check that stops work from progressing when it fails: a build fails, a merge is blocked, or an agent's turn cannot end. Unlike a checklist, it does not depend on someone remembering to run it.

Four layers of control for AI-agent work

Layer On this site In an offshore team
1. Written rules CLAUDE.md, README, SEO documentation Specifications and coding conventions
2. Build gates Ten validators; a violation stops deployment A CI gate blocks a merge
3. In-session check A Claude Code Stop hook blocks the end of a turn when the /learn validator fails Work cannot be reported “done” before it meets the Definition of Done
4. Live-environment check Verify production after deployment UAT in the client's environment

Each layer has its own use and limit:

  • Written rules preserve goals, exceptions, and the reasons behind old decisions, but an agent may overlook or rationalize them away.
  • Build gates stop violations of defined conditions, but cannot cover every way a defect might arise.
  • The in-session check returns errors for an agent to fix before human review. On this site, the Claude Code Stop hook covers only /learn.
  • Live-environment checks find failures that appear only in production, but they take human time and are not automated here.

After a production failure, I ask which part can become a build condition. I keep the written rule too: it holds the goal and reasoning that an automated check cannot express.

A useful gate checks behavior, not wording

A badly designed gate becomes a maintenance burden. I follow three principles:

  • Check behavior, not copy. The chatbot validator runs the real engine against real prompts and checks invariants such as “send nothing before a visitor leaves contact details.” It does not match the text on a button. A copy-matching test would fail whenever wording changed and encourage someone to patch the test rather than examine the behavior.
  • Check consistency, not completeness. The business-information validator does not require all 20 pages to declare every field. It checks that any field a page does declare matches the canonical source.
  • Test the gate itself. The chatbot validator has its own test suite, which deliberately reintroduces each old failure to confirm the validator turns red. A gate that never fails might be broken.

Limits and cost

Presenting this as a tidy process without its cost would overstate the result. Here are the limits I have seen:

  • A gate only catches conditions it was designed to check. I found the 34-URL failure by reviewing changes and checking production. I still need human review.
  • Every gate has a maintenance cost. One FAQ answer on the portfolio page currently lives in four places. The validator keeps them aligned, but editing one sentence still means editing it four times.
  • Some gates perform static source checks, not a real render. The schema validator cannot catch every form of indirect assignment. It covers failures I have seen, not every possible one.
  • I have no before-and-after defect rate. I did not count monthly defects before adding the gates, so I cannot claim they “cut bugs by X%.” I added corresponding checks for the six failures above; that does not prove they catch every variation.
  • This is one person running one site. After six months, I have not encountered a false positive severe enough to remove a gate, but a 30-person team would face different friction. I described a proposed design for offshore teams in AI Quality Gates for offshore development. That article is an unmeasured design; this one describes what I have actually run at a small scale.

AI agents have not removed the Delivery Manager's role from my work; they have shifted where I spend my attention. Rather than inspect every line, I set the goal, decide which failures warrant a gate, and still review what changed. I wrote more about reading and accepting AI output in From BrSE to AI Bridge SE. For the related question of what to automate with rules and what to leave to model judgment, see my notes on AI automation for SMEs.

Frequently Asked Questions

Should AI agents write most of the code for a production system?

They can, provided someone owns the goal, supplies the context agents need, and accepts the result. I estimate that AI agents wrote roughly 90% of the code on nguyenchau.dev. I review changes, run build checks, and verify behavior in production rather than assuming the agents are right.

Why aren't rules in CLAUDE.md or a prompt enough?

A written rule is a request, not an enforced condition. An agent can overlook it or rationalize an exception. My repo explicitly warned against editing JSX without updating its HTML counterpart, yet that happened again. A validator now fails the build when it detects that drift.

What belongs in code rather than a prompt?

Put deterministic business rules in code, such as selecting a customer honorific from known customer data. Leave the parts that require judgment to the model. Asking it to remember a fixed rule makes compliance harder to verify.