Using AI to Generate Test Cases from User Stories: Agentic Testing in Practice

At an IT consultancy, no two projects look the same. One sprint you're testing a claims workflow for an insurance client; the next you're on a manufacturing client's inventory system; and the sprint after that, it's a public-sector portal with its own compliance rules. The tech stack changes. The domain changes. What doesn't change is the starting point of almost every testing cycle: a user story with acceptance criteria, and a QA engineer trying to turn that into a solid set of test cases before the sprint runs out.

That's exactly the kind of repetitive, context-heavy, but fundamentally pattern-based task that's worth testing AI against. So I did, using the same real workflow I'd normally follow by hand, feeding user stories into an LLM and comparing what came back to what I'd have written myself.

The naive approach is to paste a user story into an AI chat and ask for "test cases." It works; technically, you'll get a list. But the output tends to be generic: happy-path cases, maybe one negative case, and almost nothing tied to the actual business rules buried in the acceptance criteria. The gap isn't the model's reasoning; it's context. A user story alone rarely contains everything a human tester actually draws on: the underlying data model, edge cases from past bugs on similar features, or company-specific validation rules that live in a wiki somewhere, not in the ticket.

This is where the "agentic" part earns its name. Rather than a single prompt-response exchange, an agentic setup gives the model:

• The user story and acceptance criteria
• Access to related artifacts: a linked PBI, a Figma spec, an API contract, or a schema file.
• The ability to ask clarifying questions or flag ambiguity instead of guessing silently
• A defined output format matching your test management tool (so cases land directly into your Test Plan structure rather than needing reformatting)

That last point matters more than it sounds. If a coding agent can output test cases in a format your Test management tool (Test Plan) already understands- title, steps, expected result, priority- you've removed the tedious part of the workflow without removing your own judgment from the ones that matter.

Where It Genuinely Helped

Across a handful of real user stories, the pattern that held up consistently:

• Boundary and negative cases came out stronger than expected. Given a field with a stated max length or a numeric range, the model reliably generated boundary tests (min, max, one over, one under, empty, null) without being asked explicitly- the kind of case that's easy to forget by hand at 4 PM on a Friday.

• It surfaced ambiguity I'd have otherwise resolved by guessing. More than once, the AI's first response wasn't a test case at all; it was a question, like "should this field accept accented letters and non-Latin characters (José, Müller, 田中), or is it expecting plain English text only?" That's arguably more valuable than a generated test case: it's catching a gap in the requirement itself before a test gets written against an assumption.

• Volume and first-draft speed. For a story with 5–6 acceptance criteria, having a reasonable first draft of 15–20 test cases in a couple of minutes, rather than 30–40 minutes of manual writing, is a real time gain as long as you budget time to review, not just accept.

Where It Fell Short

• Domain-specific business rules stayed invisible unless you explicitly provided them. If a rule lives only in internal documentation or a senior tester's head, the AI has no way to know it exists.

• It's confidently wrong sometimes, not just incomplete. A fluent, well-formatted test case that tests the wrong behavior is worse than an obviously incomplete one; it's more likely to be trusted without a second look.

• Cross-story dependencies were consistently missed. A checkout flow depends on cart, auth, and inventory state from other stories entirely, and an agent working from one ticket at a time won't naturally reach across the backlog to catch that.

A Practical Way to Use This Without Overtrusting It

The workflow that held up best wasn't "generate and accept"; it was "generate, then triage":

1. Feed the agent the user story plus any linked artifacts (API spec, mockups, related tickets).

2. Ask it to flag ambiguous acceptance criteria before generating cases.

3. Generate a first draft in your team's format.

4. Review as a reviewer, not an author: ask "is this true about our system," not "does this look reasonable."

5. Keep a running list of domain rules the AI consistently misses, and feed those in on future prompts; this is the part that compounds.

The Comparison: a Password Reset Story

To make this concrete rather than abstract, here's a simplified (anonymized) version of a real user story I ran this against:

As a registered user, I want to reset my password via email, so that I can regain access to my account if I forget my credentials.

Acceptance criteria:

• User enters their email on the "Forgot Password" screen.

• If the email exists, the system sends a reset link valid for 30 minutes.

• The reset link allows setting a new password meeting the site's complexity rules (8+ characters, 1 number, 1 special character)

• After a successful reset, all existing sessions for that account are logged out.

What the AI generated on the first pass (7 cases):

1. Verify reset link is sent when a valid, registered email is entered

2. Verify an appropriate message is shown when an unregistered email is entered

3. Verify the reset link works within the 30-minute validity window

4. Verify the reset link is rejected after 30 minutes have passed

5. Verify a new password is accepted when it meets all complexity rules

6. Verify a new password is rejected when it's missing a number or special character

7. Verify the password field enforces the minimum 8-character length

What I added by hand, after reviewing (6 more cases):

8. Verify the reset link is invalidated after first use, even if still within the 30-minute window (a link should be single-use, not just time-limited; this wasn't stated explicitly in the AC, but it's standard security practice the AI had no way to infer from the ticket alone)

9. Verify all other active sessions are actually terminated after a successful reset (this was in the AC, but the AI's version of this case just checked "reset succeeds" without verifying the session-logout side effect- an easy detail to generate a case for in words but miss in substance)

10. Verify requesting a second reset link invalidates the first one (a known edge case from a similar bug on a past project; no way for the AI to know this without being told)

11. Verify behavior when the reset request is submitted with an email containing leading/trailing whitespace or mixed case (domain-specific: this system does case-insensitive email matching, which isn't obvious from the story)

12. Verify rate-limiting behavior if the same email requests multiple reset links in quick succession (a non-functional requirement that lived in a security policy doc, not the ticket)

13. Verify the messaging shown for an unregistered email doesn't reveal whether the account exists (a security nuance; the AI's version of case #2 assumed showing "email not found" was fine, which is actually an information-disclosure issue in most systems handling this correctly)

What this shows in practice: the AI's 7 cases weren't wrong; they were a genuinely solid, well-structured first pass covering the stated acceptance criteria cleanly, including boundary cases like the whitespace-adjacent password length check. But every one of the 6 cases I added by hand came from something outside the ticket: a security convention, a past incident, a policy document, or plain domain judgment about what "reset" implies beyond the literal words. That's the actual shape of the gap: not sloppiness, but a blind spot around anything that isn't written down where the AI can see it.

From Test Cases to Automated Scripts

Generating the test cases is only half the workflow; the other half is turning some of them into automated scripts instead of manual steps someone re-runs by hand every sprint. Once a test case is reviewed and approved- say, case #5 above, "verify a new password is accepted when it meets all complexity rules" that same case can be handed to a coding agent as the spec for a script:

	
	test('accepts a new password meeting complexity rules', async ({ page }) => { 
	  await page.goto('/reset-password?token=valid-test-token'); 
	  await page.getByLabel('New password').fill('Str0ng!Pass'); 
	  await page.getByRole('button', { name: 'Reset Password' }).click(); 
	  await expect(page.getByText('Password updated successfully')).toBeVisible(); 
	});  
	

The order matters: the test case comes first, reviewed as a case, and the automation code is generated from it afterward. That keeps the same review discipline intact; you're not asking AI to invent both what to test and how to test it in one ungoverned step. Only a subset of cases are worth automating this way (regression-prone paths, boundary checks); the generation step earns a faster what to test, the automation step a faster, repeatable how to keep testing it.

Coding Agents as a Bridge Into Automation

There's a second, quieter shift happening alongside all this: coding agents are lowering the barrier between "manual tester" someone who knows the domain deeply but doesn't write code- and "automation tester." Historically, that gap was a programming-skills gap: a manual tester could describe exactly what needed testing but had no way to turn it into a runnable script. A coding agent moves where that skill requirement sits: instead of writing Playwright code by hand, a tester describes the test case in plain language, the same way they already write manual steps, and iterates on the generated script conversationally.

This doesn't remove the need for technical understanding entirely; someone still needs to read the generated script critically. But the starting point moves dramatically closer to something a business tester already knows how to produce: a clear, well-structured test case and further from "learn TypeScript first." That's a real shift in who can plausibly become an automation tester, not just how fast an existing one can work. It's a pattern I've seen play out first-hand: testers who know a domain cold, hand off the coding to an agent, and focus their effort on what they're already good at.

The Honest Takeaway

AI-generated test cases are a genuinely useful first draft, not a replacement for the tester's judgment about what actually matters to a given business. The password reset example above is a fair snapshot of the split: roughly half the coverage a thorough review needs can come straight from a well-structured story, and the AI gets there fast. The other half- the security conventions, the past incidents, the "obviously you'd also check that" cases- still needs a human who's seen enough real systems break to know what a ticket never says out loud. Treated as a fast, slightly literal-minded junior tester whose work you always review, it's a solid addition to the toolkit. Treated as a replacement for actually understanding the feature, it'll eventually generate a beautifully formatted test case for a behavior your system doesn't have.