Using Generative AI in Software Testing: Practical Guide

For most of the history of test automation, the workflow barely changed. Specialist engineers wrote code, built frameworks and spent much of their time repairing scripts every time the application changed.

Large language models have changed what’s possible. Teams can now describe a test in plain English and get a draft back, convert old Selenium scripts into readable automation, generate realistic test data from a sentence, and get a plain explanation of why a test failed. Used well, generative AI removes a lot of the slow, repetitive work in testing.

Used carelessly, it can also produce tests that look convincing and check the wrong things.

This guide explains what generative AI testing is, how it works, where it adds real value, what its limits are, and how to adopt it while keeping people in control of what counts as a test.

What is Generative AI?

Generative AI is a branch of artificial intelligence that creates new content, such as text, code, images or structured data, based on patterns it learned from very large amounts of training data. ChatGPT, released by OpenAI in late 2022, brought it to mainstream attention, and many similar models have followed.

Earlier AI systems in testing mostly classified or predicted things, such as recognising an element or flagging an unstable test. Generative AI produces something new: a test case, a block of test data, a plain English explanation or a converted script.

Two Meanings of “Generative AI Testing”

The phrase is used in two different ways:

This guide focuses mainly on the first meaning.

What Generative AI Does in Test Automation

Drafting Test Cases

Generative AI can turn requirements, user stories and acceptance criteria into draft test cases, including positive paths, negative cases and edge cases a person might overlook.

Creating Test Data

It can generate realistic synthetic test data, such as names, addresses, orders and transactions, including unusual values and edge cases, without copying real customer data.

Converting Legacy Tests

It can read existing coded tests, work out what they’re checking and convert them into more readable, maintainable forms.

Helping with Maintenance

It can suggest how to repair a test when the application changes, for example by finding an element that has moved or been renamed.

Explaining Failures

It can read logs, network traffic and screenshots and explain in plain language why a test probably failed.

Benefits of Generative AI in Testing

1. Faster Test Creation

Drafting test cases, data and steps takes a fraction of the time it would by hand, which helps new features get coverage within the same sprint.

2. Broader Scenarios

AI is good at suggesting variations, such as boundary values, unusual inputs and error conditions, which helps teams cover more than the obvious path.

3. More of the Team Can Contribute

When tests are drafted and written in plain English, business analysts and manual testers can create and review automation, not just engineers.

4. Better Use of People’s Time

With drafting and repetitive work handled faster, testers can spend more time on test strategy, exploratory testing and reviewing whether tests check what matters.

How Generative AI Testing Works

1. Understanding Intent, Not Just Syntax

Good generative AI testing tools work out what a test is trying to prove, not just what it clicks. When converting a script that logs in, fills a form and checks a message, the tool recognises the business journey behind it: sign in, enter data, submit, confirm.

That understanding helps it:

• Keep the important test logic while dropping fragile implementation details

• Spot steps repeated across tests that could become reusable components

• Suggest assertions that fit the context

• Produce tests people can read and review

2. Turning Fragile Scripts into Readable Steps

Legacy tests often depend on fixed XPath or CSS locators that break when the page changes. Generative AI can rewrite them as plain steps tied to what the user sees.

A legacy Selenium test might look like this:

driver.findElement(By.id("username")).sendKeys("jane@example.com");
driver.findElement(By.id("password")).sendKeys("Secure123!");
driver.findElement(By.xpath("//form[@id='login']//button[@class='submit-btn']")).click();
WebElement welcome = new WebDriverWait(driver, Duration.ofSeconds(10))
    .until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector(".dashboard h1")));
Assert.assertEquals(welcome.getText(), "Welcome back, Jane");

driver.findElement(By.id("username")).sendKeys("jane@example.com");
driver.findElement(By.id("password")).sendKeys("Secure123!");
driver.findElement(By.xpath("//form[@id='login']//button[@class='submit-btn']")).click();
WebElement welcome = new WebDriverWait(driver, Duration.ofSeconds(10))
    .until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector(".dashboard h1")));
Assert.assertEquals(welcome.getText(), "Welcome back, Jane");

driver.findElement(By.id("username")).sendKeys("jane@example.com");
driver.findElement(By.id("password")).sendKeys("Secure123!");
driver.findElement(By.xpath("//form[@id='login']//button[@class='submit-btn']")).click();
WebElement welcome = new WebDriverWait(driver, Duration.ofSeconds(10))
    .until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector(".dashboard h1")));
Assert.assertEquals(welcome.getText(), "Welcome back, Jane");

driver.findElement(By.id("username")).sendKeys("jane@example.com");
driver.findElement(By.id("password")).sendKeys("Secure123!");
driver.findElement(By.xpath("//form[@id='login']//button[@class='submit-btn']")).click();
WebElement welcome = new WebDriverWait(driver, Duration.ofSeconds(10))
    .until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector(".dashboard h1")));
Assert.assertEquals(welcome.getText(), "Welcome back, Jane");

Converted into plain English, it becomes:

Write "jane@example.com" in field "Email"
Write "Secure123!" in field "Password"
Click "Sign in"
Look for "Welcome back, Jane"

Write "jane@example.com" in field "Email"
Write "Secure123!" in field "Password"
Click "Sign in"
Look for "Welcome back, Jane"

Write "jane@example.com" in field "Email"
Write "Secure123!" in field "Password"
Click "Sign in"
Look for "Welcome back, Jane"

Write "jane@example.com" in field "Email"
Write "Secure123!" in field "Password"
Click "Sign in"
Look for "Welcome back, Jane"

The second version is easier for anyone to read and review, and it describes what the user sees rather than how the page is built.

3. Suggesting Additional Coverage

Generative AI can look at a test and suggest what’s missing. For a login test, it might propose:

• Edge cases, such as empty passwords and special characters

• Accessibility checks, such as keyboard navigation

• Security checks, such as rejecting injection attempts in input fields

• Error handling, such as how the page behaves when a service is unavailable

These suggestions are a starting point. A person should decide which ones matter for the business before they become part of the suite.

4. Generating Data and Assertions

Test Data

Generative AI can look at field types and validation rules and create data for normal cases, boundaries and invalid input.

Assertions

It can suggest what to check after an action, such as a confirmation message, saved data or a status change. Assertions are where errors matter most, so they need the most careful human review.

5. Improving Through Feedback

When people review, accept and correct what AI generates, and record why tests failed, that feedback helps the team improve its prompts, components and review process over time. The key is keeping a record of what the AI proposed and what people decided.

Three Sources Generative AI Can Draft Tests From

Source 1: Legacy Test Suites

The Challenge

Many organisations have years of coded automation spread across several tools, each needing specialist skills. Rewriting it all by hand looks too expensive, so old suites linger and absorb maintenance effort.

How Generative AI Helps

Generative AI can read legacy scripts, work out what each test checks and convert it into readable automation, preserving the knowledge built into the tests while dropping fragile implementation details.

A Typical Migration Approach

Validation is the step that can’t be skipped. A converted test that passes but no longer checks the original condition is worse than no test at all.

Source 2: Requirements, Stories and Manual Test Cases

The Challenge

A lot of testing knowledge sits in requirements documents, user stories, Gherkin scenarios, manual test cases and process documents. Turning that knowledge into automation has traditionally needed engineering effort.

How Generative AI Helps

Generative AI can read these documents and draft executable tests from them, keeping a link from each test back to the requirement it came from.

For example, from a Gherkin scenario like this:

Scenario: Customer updates their delivery address
  Given a signed-in customer with an existing delivery address
  When they change their address to "10 King Street, Leeds, LS1 2HL"
  And they save the change
  Then the new address appears on their account page
  And a confirmation email is sent to the customer

Scenario: Customer updates their delivery address
  Given a signed-in customer with an existing delivery address
  When they change their address to "10 King Street, Leeds, LS1 2HL"
  And they save the change
  Then the new address appears on their account page
  And a confirmation email is sent to the customer

Scenario: Customer updates their delivery address
  Given a signed-in customer with an existing delivery address
  When they change their address to "10 King Street, Leeds, LS1 2HL"
  And they save the change
  Then the new address appears on their account page
  And a confirmation email is sent to the customer

Scenario: Customer updates their delivery address
  Given a signed-in customer with an existing delivery address
  When they change their address to "10 King Street, Leeds, LS1 2HL"
  And they save the change
  Then the new address appears on their account page
  And a confirmation email is sent to the customer

AI can draft the full journey, including signing in, finding the address form, entering valid data and checking both the account page and the confirmation email.

From a user story like this:

As a returning customer, I want my basket saved when I sign out,
so that I can finish my purchase later on any device

As a returning customer, I want my basket saved when I sign out,
so that I can finish my purchase later on any device

As a returning customer, I want my basket saved when I sign out,
so that I can finish my purchase later on any device

As a returning customer, I want my basket saved when I sign out,
so that I can finish my purchase later on any device

AI can draft tests for the basket persisting after signing out, appearing after signing in on another browser, and expiring after the agreed period.

Why Traceability Matters

When every generated test cites the requirement it came from, reviewers can check the AI understood the requirement correctly, and auditors can see why each test exists.

Source 3: The Application Itself

The Challenge

New features need coverage quickly, and waiting for manual test design delays releases or leaves gaps.

How Generative AI Helps

Some tools can look at the application, either its screens or its API specifications, and suggest tests based on what they find, such as forms to fill, buttons to click and common patterns like login, search and checkout.

The Limitation to Keep in Mind

A test generated only from the application checks what the application currently does, not what the business intended it to do. If the feature is wrong, a test generated from it will often be wrong in the same way.

Tests drafted from requirements avoid this trap, which is why many teams use application-based suggestions to fill gaps rather than as their main source.

The Risks of Generative AI Testing and How to Manage Them

The common thread is governance. Generative AI is good at proposing tests and changes. People should decide which ones are kept, and every decision should leave a record.

Putting Generative AI Testing into Practice

Phase 1: Assess Your Current Testing

Inventory Your Automation

List the tools and frameworks in use, how many tests each holds, how much maintenance they need and which skills they depend on.

Map Coverage Gaps

Compare what’s tested with what should be, including critical scenarios still tested by hand and compliance requirements.

Decide Where to Start

Start where generative AI helps most: high-maintenance legacy tests, critical compliance tests, and new features waiting for coverage.

Phase 2: Run a Pilot

Choose a few hundred tests or a few important features that cover different parts of the application, simple and complex scenarios, and different data needs.

• Generate or convert tests for the pilot scope

• Review every generated test, with particular attention to assertions

• Run them against the application and compare results with existing tests

• Record what needed correcting, and use it to refine your approach

Set goals for the pilot, such as how much faster tests are created, how many need significant correction and how confident reviewers are in the results.

Phase 3: Scale Up

Extend to more applications and suites in stages. Train each group on what it needs:

Phase 4: Make Generation Part of Everyday Work

New Features

Draft tests from each story as it’s refined, so coverage is ready when development finishes.

Application Changes

Let AI propose updates to affected tests, and have people review them before they’re kept.

Ongoing Clean-Up

Keep converting remaining manual and legacy tests, and keep improving shared components.

Generative AI Testing with Virtuoso QA

Virtuoso QA uses generative AI to speed up writing, converting, maintaining and diagnosing tests for browser-based applications, while a deterministic engine runs every test and people approve the decisions that matter.

Every run records which requirement each test covers, how each step went and who approved the test.

• Touchstone agents draft requirements and tests from specs, user stories and Jira tickets, citing the document behind each one, and a named person approves every requirement before anything is built

• GENerator converts existing test suites, including Selenium tests, into plain English journeys, so years of testing knowledge carry over

• Testers write in plain English, each step runs live as it’s typed, and StepIQ suggests the next step from the page in front of them

• AI test data generation creates realistic data from a short description, including edge cases

• When the UI changes, Virtuoso QA proposes repairs for a reviewer to accept or reject, so generated tests stay current without hidden edits

• A “Why did this fail?” option uses generative AI to explain failures in plain English, often with a suggested fix, and the data stays inside Virtuoso QA
  
  

To learn how AI agents draft tests from your requirements while people keep the final say, read about our agentic test generation.

Related Reads

‍The Shift from Automation to Autonomy: What are AI Agents?‍

Agentic AI vs Generative AI: The Key Differences‍

What are Agentic Workflows - Capabilities, Working, and Implementation

What is Autonomous Testing? Five Levels, Examples and Behaviours

Autonomous Testing with Agentic AI: The Next Evolution in QA

Autonomous Test Generation: What it is and How it Works