Agentic Testing Workshop: Let AI Write Tests, Let Humans Own Quality

Testing is one of the most natural uses for a coding agent. The work is repetitive, the patterns are well known and the existing code provides a specification of sorts. It is also a place where agents can create a false sense of security: a suite of a hundred tests that all pass can still miss everything that matters. The workshop teaches how to get the benefits while keeping the safety net real.

What agents do well in testing

• Drafting unit tests for pure functions and clear modules
• Listing edge cases a developer may have forgotten: empty input, very large values, unusual characters, time zones
• Generating test data and fixtures
• Translating a bug report into a failing reproduction test
• Updating many tests after a mechanical refactor
• Writing the first version of integration or UI tests that a human then refines

Where tests go wrong

Participants learn to recognize the typical failure modes of agent-written tests:

1. Tests that mirror the implementation. They compute the expected value with the same logic as the code, so they can never disagree with it.
2. Over-mocking. Everything is replaced by a mock, so the test verifies the mocks, not the behavior.
3. Weakened assertions. When a test fails, the agent changes the assertion instead of the code.
4. Snapshot sprawl. Large snapshots nobody reads, accepted blindly on every change.
5. Coverage theater. Lines are executed but nothing meaningful is asserted.

The test-the-tests exercise

A core exercise in the workshop is simple and effective. Participants take a function with an agent-written test suite and deliberately break the function in small ways, for example flipping a comparison or removing a validation. If the tests still pass, they are not protecting anything. Participants then ask the agent to strengthen the tests and repeat the experiment. This builds the habit of checking that a test can fail, a check as important as seeing it pass. Where a mutation testing tool exists for your language, we show how it automates this idea; check your ecosystem's current tools.

After the exercise, we also discuss a related habit: asking the agent to explain what each test is protecting against. If the explanation is vague, the test probably is too. Participants keep a short list of such questions beside their editor and use it during review. Over time, the team builds a shared sense of what a good test looks like in their codebase, and this sense is more durable than any single tool or prompt.

Humans own the strategy

Deciding what to test is a product and risk decision, not a typing task. We work with teams to define a short test strategy: which flows are critical (sign-up, payment, data export), which risks matter most and what level of testing fits each. Agents then fill in the volume. For mobile apps, the strategy always includes manual checks on real devices, since emulators miss many real-world problems such as permission prompts, poor network conditions and device-specific layout issues. Our guide on mobile app agentic coding expands on this.

Workshop agenda

1. Calibration: what testing looks like today and what hurts.
2. Generating tests with an agent for an unfamiliar module, including a written plan of cases.
3. The test-the-tests exercise.
4. Reproducing real bugs as failing tests before fixing them.
5. Drafting the team's test strategy and review checklist for agent-written tests.

The apprat.io view

At jocapps, quality is managed under ISO 9001 certified processes, and the principle is AI speed, human responsibility. Agents write much of the code and many of the tests, while senior AI Code Engineers plan, review every change, test on real devices and take accountability. A green test run is evidence, not proof, and a person decides whether the evidence is enough to ship. If you inherited an AI-generated app with weak tests, our rescue playbook shows how we approach it, and our code audit overview describes what a review delivers.

If you would like a testing workshop for your team, you can book a call for free.

Practical rules to take back

• Write or approve the list of test cases before the agent writes the test code.
• Watch each new test fail once before trusting it.
• Never let the agent change an existing assertion without a human reading why.
• Keep integration tests few but meaningful, and run them in CI on every change.