Every industry has a moment when the old way of doing things becomes outdated. For QA, that moment is now — and AI-powered QA testing is what’s replacing it.

For years, quality assurance operated on a simple model: more testing equals more quality. Measure it in hours. Bill it in hours. Report it in hours. The thickness of the test plan served as proof of diligence, and the invoice served as proof of work.

That model had a flaw that no one openly discussed: it prioritized volume over judgment. The longer it took to write tests, the more billable hours were generated. The more manual steps involved, the more justification there was for the next sprint. There was no structural incentive to build something smarter.

Trembit recognised this. And instead of working around it quietly, our team made a deliberate choice: either rebuild the process entirely or continue defending a model that no longer served clients as well as it could.

We chose to rebuild.

The shift — a business decision, not a technology experiment

This was not a team that stumbled onto AI tooling and decided to try it out. The decision to rebuild Trembit’s QA process was a business decision first. Clients — startups, scaling products, lean engineering teams — needed coverage fast and needed it to stay current as their products changed. The traditional QA model was not built for that reality. It was built for slower cycles, bigger teams, and longer runways. The question Trembit asked was specific: what would QA look like if it were designed around outcomes rather than effort? If coverage were the deliverable, not hours? If the test plan stayed current rather than becoming stale two sprints after it was written? That question pointed directly toward AI — not as a shortcut, but as the only way to restructure the process at the level it needed to be restructured.

“We were not looking for a faster way to do the same thing. We were looking for a fundamentally better thing to do.”

The team is committed to understanding AI properly before deploying it in client work. That meant formal investment: structured courses in AI engineering, hands-on sessions with AI specialists, months of internal iteration — building workflows, stress-testing outputs, discarding what was unreliable, refining what worked. More than six tools were evaluated before the current stack was settled on. The process has been revised continuously since. What Trembit uses today is not what was tried first — it is what survived serious scrutiny.

• Formal AI engineering training completed by the core QA team.
• Specialist advisory sessions on agent design and prompt engineering for test generation.
• Internal evaluation of 6+ AI tools and frameworks.
• Ongoing iteration — the process is reviewed and updated as tooling matures.
• Every AI output is human-reviewed before it reaches a client or a test suite.

The result is a team that has earned the right to trust its AI-augmented output — because it understands exactly where that output is reliable and exactly where human judgment must take over.

What actually changed — and by how much

The process is rebuilt around three components: a structured input protocol, AI agent execution, and mandatory engineer review. The combination produces a measurable difference.

~1,000 hrs

Previous cost to reach meaningful automated coverage on a new product

Manual test plan creation + case design + automation scripting + documentation

100–160 hrs

The same coverage today using Trembit’s AI-augmented process

AI agent execution + structured algorithm + engineer review and calibration

6–8×

Reduction in engineering hours for equivalent output

Measured across real client projects over the past twelve months

These are not projections. They are the before and after of how Trembit actually works — measured across real projects as the process matured. The hours did not shrink because the team cut scope. They shrank because the parts of QA that should never have required a senior engineer — repetitive test generation, boilerplate scripting, pattern-based case design — are now handled by AI. Engineers spend their time on the work that cannot be automated: adversarial thinking, risk calibration, and the scenarios nobody anticipated.

How the process runs

1. Structured input protocol

Before any agent runs, available inputs are catalogued and assessed: requirements, user stories, API specifications, codebase, design files, bug history. Gaps in inputs are documented as confidence risks before a single test is written. This prevents the most common failure mode in AI-assisted testing — generating tests that are syntactically correct but assert the wrong behaviour.

2. AI agent execution

With inputs structured, AI agents produce the initial test artefacts: test cases mapped to requirements, coverage gap analysis, initial test plan, and environment documentation. Work that previously required two to three weeks now takes one to three days.

Trembit uses Claude Code as the primary agent — it handles context understanding, test generation, execution, and failure interpretation in a single environment, maintaining context across the full codebase rather than generating isolated snippets. This produces tests that reflect actual system behaviour. Supporting tools include GitHub Copilot for inline suggestions during review, Cursor for navigating large, unfamiliar codebases, Playwright for end-to-end and API automation, and pytest for the backend and integration layer.

3. Engineer review and calibration

Every artefact the agents produce is reviewed by a Trembit QA engineer before it is used or shared. AI-generated tests fail in specific, predictable ways: asserting the wrong behaviour confidently, missing scenarios that require business context, and failing to discover what was never specified. The engineer review layer catches all of this — and is what makes the process reliable, not just fast.

“The AI handles the volume. The engineer handles the meaning. Both are necessary. Neither is sufficient alone.”

The difference, in one place

Manual QA versus Trembit’s AI-augmented process — compared across the dimensions that matter to engineering teams and the clients they work with.

	Manual QA	Trembit AI-Augmented QA
Hours to cover a new project	800 – 1,000 hrs	100 – 160 hrs
Test plan creation	2–5 days, fully manual	4–8 hours, auto-generated + review
Test suite bootstrap	1–3 weeks	1–3 days
Documentation	Separate effort, often skipped	Generated as a byproduct
Coverage visibility	Estimated, hard to quantify	Explicit targets + named gap register
Engineer role	Test writer & executor	Test architect & AI reviewer
Scalability	Linear — headcount grows with volume	Logarithmic — process scales, headcount does not
Client reporting	Hours logged + bug counts	Coverage %, AI confidence score, gap log

The scalability row deserves attention. In the manual model, more projects meant more engineers proportionally. In the AI-augmented model, the algorithm and tooling scale independently of headcount. Trembit can take on a higher volume of work at a higher quality bar without a corresponding growth in team size — and clients receive a consistent process regardless of who is assigned.

Where are you?

Before any tool runs. Before any plan is written.

What do you actually have to work with?

The AI needs inputs. The quality of those inputs directly determines the quality of what comes out. And the gap between “what we think we have” and “what we actually have” is where most QA projects quietly fail. Almost every project Trembit takes on falls into one of three situations. Not project types — situations, because the same product can move between them as documentation improves, degrades, or never existed at all.

We call them The Optimist, The Inheritor, and The Firefighter.

	The Optimist	The Inheritor	The Firefighter
Situation	Building from scratch. Requirements and designs exist.	Existing product. Has requirements, code, designs, API.	Existing product. No requirements. Code and API only. Bug-fixing mode.
Available inputs	Requirements, user stories, designs, wireframes	Figma, API spec, codebase, partial tests, issue tracker	Running code, API endpoints, bug reports
Real danger	Overconfidence. Requirements describe intent, not behaviour.	Stale docs. Contradictions between spec and reality.	No definition of “working.” False stability.
What AI does	Generates test cases from specs, builds traceability matrix, flags gaps	Cross-references docs vs code vs API, surfaces contradictions	Reverse-engineers system behaviour, clusters bug patterns
Test plan outputs	• Traceability matrix• Assumptions log• Shift-left plan• Risk prioritisation• Versioning triggers	• Discrepancy register• Coverage gap analysis• API contract tests• Regression scope• Living doc protocol	• System discovery map• Bug pattern clusters• Smoke test baseline• Behavioural snapshots• Risk acceptance log
Engineer focus	Adversarial thinking — what did no one anticipate?	Trust calibration — which contradictions actually matter?	Defining “stable” before any fix begins
Timeline to first suite	1–2 days generation + 0.5 days review	2–3 days discovery + 1 day calibration	1 day mapping + rolling coverage build

🌱  The Optimist — Building from scratch

You have requirements. You have designs. The product is being built right now, and there is a real chance to do things properly from the start. This is the best situation to be in — and the one most prone to overconfidence.

Requirements describe intent, not behaviour. Designs show happy paths. Neither tells you what happens when a user does the right steps in the wrong order, or inputs valid data into an unexpected flow. The danger is not lack of information — it is unverified assumptions that nobody has thought to challenge.

The AI builds a complete traceability matrix — every requirement mapped to test cases, gaps flagged immediately — and an assumptions log that makes explicit what was inferred versus what was documented. The engineer provides the adversarial thinking that no specification anticipates. A shift-left plan is produced so that testing begins before development finishes, compressing the cost of finding defects early.

🏗  The Inheritor — Existing product with documentation

You have been handed something. It has history. There are requirements somewhere, designs in Figma, and an API spec that may or may not match what is actually deployed. People made decisions you were not part of. Some of those decisions were good.

The challenge here is not volume — it is trust. Do the docs reflect reality? Do the tests, if any exist, actually catch real bugs — or do they just pass?

The AI cross-references requirements, codebase, and API spec simultaneously and surfaces contradictions between them. Trembit consistently finds three to five meaningful discrepancies per project that human engineers had learned to work around without noticing. Those contradictions are where defects live. The output includes a discrepancy register, full coverage gap analysis, API contract test suite, and a living documentation protocol that keeps the plan current as the product evolves.

🚒  The Firefighter — Existing product, no requirements

Something is broken. Nobody fully agrees on what “working” looks like. There are no requirements, or they are so stale that they are worse than useless. There is code. There is an API. There are bug reports. That is it.

This is the hardest situation. And the one where honesty matters most. AI can map what exists. It cannot tell you what was intended. The first output is a system discovery map — inferred flows, entity relationships, API surface, dependency structure — that becomes a de facto requirements substitute. Bug pattern analysis identifies where defects historically cluster, and coverage concentrates there first. The critical concept here is the behavioural snapshot: automated tests that lock in what the system currently does, irrespective of whether that behaviour is correct. Not correctness tests — stability anchors. Every area with zero coverage is named explicitly in a risk acceptance log, reviewed with the client, and signed off. No silent gaps.

“In firefighting mode, the most valuable thing a test plan can do is define a floor. You may not know what ‘correct’ looks like yet — but you can guarantee ‘stable.'”

What every Trembit test plan includes

Regardless of which situation a project is in, five elements appear in every test plan Trembit produces. They are not additions — they are the infrastructure that makes the rest trustworthy and actionable.

AI confidence score per section.  Each section carries a transparent indicator of what was AI-generated versus engineer-verified. Clients and internal engineers know exactly where human judgment was applied and where additional scrutiny is warranted.

Automation coverage target with rationale.  Not just a percentage — an explanation of why that target was chosen, what the uncovered scenarios are, and who owns them. A number without reasoning is not useful.

Test environment and data specification.  A complete description of required environments, data that must be present before tests run, and what must be seeded or mocked. Environment mismatches are one of the most common causes of test failures in practice.

Entry and exit criteria.  Defined before any testing begins. All stakeholders agree in advance on what “ready to test” and “testing complete” mean. This eliminates the most common category of scope disagreement late in an engagement.

Time and effort breakdown.  A transparent account of how hours were spent: AI agent execution versus engineer review and calibration. Instead of “40 hours on test development,” Trembit reports what the AI produced, how long the engineer spent reviewing it, and what coverage was achieved. That is a different kind of conversation about value.

What this means for clients

Three things change concretely when working with Trembit on QA.

Speed to real coverage

A product that previously required three to four weeks to reach meaningful automated test coverage now reaches that point in three to five days. Coverage exists before stakeholder presentations and investor demos — not after.

Consistency that does not depend on who is assigned

Manual QA quality varies based on who is available, how much context they have accumulated, and how much bandwidth they have for that sprint. The AI-augmented process produces consistent outputs because the algorithm is consistent. The engineer layer calibrates and improves — but the baseline does not fluctuate.

A test plan that stays current

Traditional test plans are point-in-time documents. They become inaccurate with every sprint. The Trembit test plan is a living artefact with explicit versioning triggers, ownership, and update protocols. When a feature ships, the algorithm generates incremental coverage. The engineer integrates it. The plan reflects the product as it is — not as it was when the engagement started.

Transparency, not reassurance

Clients see exactly what is covered, exactly what is not, and exactly why. The AI confidence score and the risk acceptance log replace comfortable ambiguity with an accurate picture. Some clients find this adjustment uncomfortable at first. Most find it is what they actually needed.

For QA engineers: what this work actually requires

This section is written directly for engineers considering whether Trembit is the right place to do this work. The role is not a test writer. It is a test architect. That distinction changes the day-to-day completely. A test writer executes a defined process. A test architect defines what should be tested and why, evaluates whether AI-generated output is correct and complete, identifies the scenarios that require human ingenuity to discover, and owns the quality outcome of the engagement.

The skills this demands:

• System architecture literacy.  The ability to read an unfamiliar codebase and understand component relationships and failure modes — not just surface behaviour.
• Prompt engineering for testing.  Knowing how to give an AI agent the context and constraints that produce meaningful test scenarios, not syntactically valid but semantically shallow outputs.
• Critical evaluation of AI output.  Recognising when a generated test is asserting the wrong thing, when coverage gaps exist that the agent did not surface, and when the confidence score should be lower than it appears.
• Risk-based prioritisation.  Ordering coverage not by what is easiest to test, but by what failure would cost the product and the client the most.
• Clear communication.  Translating coverage gaps, confidence levels, and risk acceptance decisions into language that non-technical stakeholders can act on.

The tools the team works with daily reflect this evolution. Claude Code serves as the primary agent environment — context-aware, codebase-wide, handling generation through to failure interpretation. ChatGPT and Perplexity are used for research, edge case exploration, and rapid knowledge lookup during test design. Windsurf supports agentic coding workflows where codebase navigation and test scaffolding happen in parallel. Chatbox is used for conversational AI-assisted review and documentation drafting. Engineers on the team are expected to be fluent across all of them — knowing which tool fits which problem is itself a skill.

Trembit works with engineers who are at this intersection — quality engineering and AI tooling, intellectually demanding, with a process that is still being built rather than inherited. The team is remote-first. The work is real.

“The bar for QA at Trembit went up when we introduced AI — not down. We needed engineers who could evaluate machine output critically. That is a harder skill than writing tests manually.”

Where the approach has real limits

Trembit is direct about what AI-augmented QA does not solve.

• Input dependency.  Vague requirements generate vague tests. Missing context produces missing coverage. The algorithm cannot manufacture understanding; it was not given. If inputs are weak, the process says so before a test is written — not after.
• Exploratory testing.  AI cannot discover unknown unknowns — defects that no specification anticipated and no bug report described. Every engagement includes explicit time-boxed, human-led exploratory sessions targeting the highest-risk areas. This is not automatable, and Trembit does not pretend otherwise.
• Usability and accessibility.  Automated tests cannot evaluate whether a product is usable by a real person or accessible to users with disabilities. These require human evaluation.
• Performance under realistic load.  AI-generated tests cover defined scenarios. They do not replace dedicated load testing under concurrent real-world conditions.
• Highly regulated environments.  In domains requiring full manual traceability and human sign-off on every test case, AI can accelerate generation but cannot replace the certification layer.

These limitations are documented in every Trembit test plan. The objective is not the appearance of complete coverage — it is an accurate picture of what is covered, what is not, and what that means for the product.

From testing hours to quality infrastructure

The shift from 1,000 hours to 100–160 hours is a data point. What it represents is something larger: a different definition of what QA actually delivers.

Traditional QA delivered artefacts — test plans, test cases, bug reports — documents that described the product at a moment in time and became less accurate with every sprint that followed. Trembit delivers infrastructure. Automated coverage that runs on every merge. A test plan that updates as the product evolves. Coverage gaps that are named, visible, and managed — not silently assumed not to exist. A confidence score on every section so clients know exactly what they have and what they do not. That requires more transparency from Trembit — about what the AI produced, where engineers intervened, and where the limits are. It requires more clarity from clients about what acceptable risk looks like. But it produces something more durable than a spreadsheet of test cases ever could.

The question is not how many hours your QA will take.

The question is: which situation are you in — and what does real coverage look like for your product at this stage? Trembit can answer that concretely, usually in a single technical conversation. Reach out to the team.

Trembit — Quality Engineering

Trembit is not the team that waits to see how AI shakes out. It is the team that studied it, built it, stress-tested it, and rebuilt it when the first version was not good enough. That restlessness — the refusal to defend yesterday’s process when a better one is available — is what clients get when they work with Trembit. Not just faster QA. A team that will keep improving it.