Insurance
Insurance Testing Challenges: 14 Reasons Releases Fail
Rishabh Kumar
Published On
Table of contents
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Insurance releases fail for a small number of recurring reasons: change the QA team doesn’t control, insurance logic that’s harder to test than it looks, an estate that fails at its joins, test assets that decay, and proof that’s assembled after the fact. Each reason has a known fix, and most of the fixes reinforce each other.
The fourteen insurance testing challenges below come up in almost every conversation with insurance QA leaders, and in the forums and communities where test engineers compare notes. They’re grouped into five families. For each one, we describe what it looks like on the ground, why it happens, what fixes it, and how Touchstone, Virtuoso QA’s agentic QA platform, helps.
Change You Don’t Control
The first family of problems has nothing to do with the quality of the insurer's own work. Vendors ship on their calendars, business teams change configuration directly, and code arrives faster than anyone can review it. A testing approach built for occasional, engineering-led change cannot keep up.
The Vendor Release Treadmill
What it looks like
A core platform release lands and a third of the automated suite goes red. Almost none of the failures are real defects. The team spends two weeks repairing scripts, and by the time the suite is green again the next release is in pre-production. A common refrain: we spend more time fixing the regression pack than running it.
Why it happens
Guidewire ships three cloud releases a year, and Duck Creek runs three feature releases plus monthly updates. Each can move screens, element structures and API behaviour. Scripts written against page structure break when the structure moves, even when the insurance behaviour is unchanged.
What fixes it
Tie tests to business intent, not page mechanics. Run the regression pack against the new release in pre-production before any go-live decision. Measure repair effort per release, because that number is the business case for better tooling.
How Virtuoso QA Touchstone helps
Journeys are defined from requirements, and self-healing execution absorbs routine interface movement. When a release genuinely changes behaviour, Touchstone traces which requirements and journeys it affects and proposes updates as reviewable differences, so the team reviews a short list instead of rebuilding a suite.
Configuration That Skips the Pipeline
What it looks like
A pricing analyst uploads a new rating table on a Thursday afternoon. An underwriting manager adds a referral rule. A product owner switches on a new coverage. None of it goes through the engineering pipeline, so none of it triggers regression. The first test is a live customer.
Why it happens
Modern core platforms put configuration in the hands of the business on purpose, because it speeds up product change. Governance has not caught up with the speed.
What fixes it
Treat configuration as a release. Every configuration deployment to production should trigger the regression journeys that depend on it, with sign-off from someone other than the person who made the change.
How Touchstone helps
Rating guides, rules documentation and product specifications are sources Touchstone reads.
When one is revised, every dependent requirement and test is identified and an update is proposed for approval, so configuration change and test change move together and the approval is visible in the evidence.
Code Arriving Faster Than Anyone Can Check It
What it looks like
Delivery teams use AI coding assistants, and core platform vendors have begun shipping assistants into their own developer toolchains. Code volume rises. Unit tests are often written by the same assistant that wrote the code, so they confirm what the code does rather than what the business asked for.
Why it happens
Writing code got cheaper. Deciding whether it is correct did not.
What fixes it
Separate the definition of correct from the thing being tested. Requirements should come from business sources and be approved by people accountable for them, and tests should run on an engine that does not change its mind between runs.
How Touchstone helps
Touchstone derives requirements from your own documents, not from the code. Every requirement and test has a named approver, and a deterministic engine runs the tests, so the AI that reasons is never the system that executes. That independence is the point.
Insurance Logic That is Harder Than it Looks
The second family is about the logic itself. Rating, dates, jurisdictions and documents carry most of an insurer's commercial and regulatory exposure, and each one has properties that generic test design misses.
Rating and Renewal Pricing Drift
What it looks like
Prices are slightly wrong for a slice of customers, and nobody notices until a complaint, a broker query or a pricing review. In UK home and motor, the stakes are higher: renewal prices must not exceed the equivalent new business price, and in 2023 one large insurer agreed to pay redress estimated at around £30 million after an error in implementing that calculation.
Why it happens
Rating spans several systems: the pricing team's model, the rating engine, the factor mapping in the policy system and channel-specific discounts. A change to any one can shift prices without failing a single functional test, because the journey still completes.
What fixes it
Test prices, not just journeys. Quote the same risks through every channel and compare. Test renewal prices against new business prices for equivalent risks. Re-run price tests on every model or table deployment, not only on code releases.
How Touchstone helps
The Analyst reads the rating guide and proposes requirements, including channel parity and ENBP checks. Journeys quote across channels; API steps call the rating service with known inputs. A new table version is traced to every dependent test. See How to Test an Insurance Rating Engine.
Logic That Depends on the Calendar
What it looks like
A backdated endorsement calculates the wrong additional premium. A renewal invited 30 days out picks up the wrong rate table. A cancellation within the cooling-off period refunds on the wrong basis. Tests pass because they only ever run "today".
Why it happens
Insurance is full of time: effective dates, renewal windows, grace periods, anniversary dates and pro-rata calculations. Test environments rarely make it easy to move time, and test data usually holds only freshly created policies.
What fixes it
Design tests around the policy calendar. Create policies with past and future effective dates, run journeys at key points in the lifecycle, and use the platform's test clock in non-production environments where one exists.
How Touchstone helps
API steps create policies at any point in their lifecycle with chosen effective dates, so journeys can start mid-term, at renewal invitation or inside a cooling-off window. Requirements drawn from the product specification name the date rules explicitly, so they are tested deliberately rather than by accident.
Variation by Jurisdiction, Product and Brand
What it looks like
A change works for one state, territory or brand and breaks another. A form appears for the wrong jurisdiction. A multi-brand insurer discovers that one price book was never updated.
Why it happens
The same platform carries many products, brands and regulatory regimes, each with its own rules, rates and forms. Test packs tend to cover the main variant thoroughly and the rest by assumption.
What fixes it
Make variation explicit. Build a matrix of products, brands and jurisdictions that matter, and drive the same journeys through each with data rather than copying scripts.
How Touchstone helps
Reusable building blocks and data-driven journeys run the same business process across every variant. Requirements carry the jurisdiction or brand they apply to, so coverage gaps are visible rather than assumed.
Documents and Wordings
What it looks like
The policy is right but the schedule is wrong. A renewal notice is missing a mandatory value. An old wording is attached after a product change.
Why it happens
Document composition sits in a separate platform with its own templates and release cycle, and document checks are often manual and late.
What fixes it
Check, at every policy event, that the right document was requested from the right template version with the right values, and that it reached the customer.
How Touchstone helps
Journeys confirm the document request, the values passed to the composition service and availability in the portal. Template versions trace back to the wording or rule that changed them.
An Estate That Fails at Its Joins
The third family sits between systems. Most serious production incidents in insurance start at an integration, and integrations are where test ownership is weakest.
Integrations That Fail Quietly
What it looks like
A credit or claims history call times out, the system falls back to a default, and the quote completes at the wrong price. No error is raised. The test passes.
Why it happens
Integrations are built to degrade gracefully, which is right for customers and dangerous for testing. A journey that completes is not the same as a journey that used the right data.
What fixes it
Assert on outcomes that prove the integration was used: the enriched value, the decision it drove, the flag that shows no fallback occurred. Test timeouts and errors deliberately with stubbed services.
How Touchstone helps
Journeys combine browser steps with API and data checks, so a test can confirm that the value returned by a service actually reached the rating or decision step. A fallback becomes a failure, not a silent pass.
Work That Happens Overnight
What it looks like
Renewal invitations, billing runs, lapse processing and claims payments run in batches. UI tests never see them. The defect appears the next morning in a customer's inbox or bank account.
Why it happens
Batch and asynchronous processes sit outside the screens that most automated tests drive, and they often only run on a schedule.
What fixes it
Include batch outcomes in regression. Trigger jobs on demand in test environments where the platform allows it, then check the results.
How Touchstone helps
API steps trigger exposed jobs and services, and data and API checks confirm what the batch produced: the invitation generated, the instalment collected, the policy lapsed or not. The outcome is proven inside the same journey as the change that affected it.
Environments and Third-Party Sandboxes
What it looks like
Two teams share one UAT environment. A refresh wipes the test data overnight. A data provider's sandbox is down, rate-limited or returns the same answer for every request. Tests fail for reasons nobody can reproduce.
Why it happens
Full insurance environments are expensive, and third-party services are outside the insurer's control.
What fixes it
Make tests independent of environment state. Create the data each test needs at the start. Virtualise third-party services in lower environments, with known responses, and keep a small set of tests that run against the real sandboxes.
How Touchstone helps
API steps create the policies, customers and claims each journey needs, so tests do not depend on what the last refresh left behind. Deterministic execution means that when a test fails, it fails the same way again, which separates environment noise from real defects.
Test Assets That Decay
The fourth family is about the tests themselves. Many insurers have tried automation more than once. The suites decayed, trust collapsed, and regression drifted back to manual.
Brittle Automation and the Abandoned Framework
What it looks like
A Selenium framework built by a contractor who has since left. Generated element identifiers, nested frames and multi-step wizards that break scripts on every release. A suite that passes on Monday and fails on Tuesday with no change. Eventually the team stops looking at the results.
Why it happens
Insurance user interfaces are complex, and code-based frameworks concentrate knowledge in a few people. Maintenance cost grows faster than the value the suite returns.
What fixes it
Authoring that the wider team can read and maintain, execution that tolerates interface change, and a clear line between a flaky test and a real failure.
How Touchstone helps
Tests are authored and approved in plain language, so underwriters, analysts and testers can all read them. Self-healing absorbs interface movement. Deterministic execution and full step evidence make failures reproducible and diagnosable.
Existing scripts and manual test cases can be used as sources for new journeys, so a migration does not start from nothing.
One honest caveat
Heavily nested iframes and shadow DOM components remain workable but higher friction for any automation, Touchstone included, and some application edge cases still need a person to guide the journey.
Test Data That Runs Out or Cannot Be Used
What it looks like
A policy can only be cancelled once, so the cancellation test consumes its data. Production copies cannot be used because of personal data. Synthetic data does not reflect the real mix of risks, covers and history.
Why it happens
Insurance data is stateful, long-lived and personal. Good test data is hard to create and easy to use up.
What fixes it
Create fresh data per test through APIs, cover the risk profiles that matter with deliberate synthetic sets, and keep personal data out of test environments.
How Touchstone helps
API steps generate the data each journey needs in a known state, and data-driven journeys run the same process across many risk profiles.
People and Proof
The fifth family is the one boards and regulators notice. Knowledge lives in a few heads, specialists are always busy, and evidence of testing is pieced together when someone asks for it.
The Specialist Bottleneck and Tribal Knowledge
What it looks like
The regression pack is a spreadsheet. Only two people understand the commercial property rating, and both are needed for UAT in the same week as quarter-end. Releases wait on their calendars.
Why it happens
Insurance rules are expert knowledge. When tests are written in code or kept in spreadsheets, experts cannot review them efficiently, so they are pulled into manual testing instead.
What fixes it
Put experts where their judgement matters: deciding what correct looks like. Let them approve requirements and journeys in plain language, and let automation do the repetition.
How Touchstone helps
The Analyst drafts requirements from the documents experts already wrote; experts approve or correct them. Their approval becomes part of the evidence, and their time goes on judgement rather than clicking through regression.
Evidence Assembled After the Fact
What it looks like
Internal audit asks which tests covered last quarter's pricing release. A regulator asks how a defect reached customers. The answer takes weeks of screenshots, emails and interviews, and still has gaps.
Why it happens
Most testing produces results, not evidence. Results show tests ran. Evidence shows the right things were tested, against which version, approved by whom, with results that reproduce.
What fixes it
Produce evidence as part of the work. Link every requirement to its source, every test to its requirement, every approval to a person and every run to a build.
How Touchstone helps
This is what Touchstone is built for. Every artefact cites its origin, every step has a named approver, every change is a visible difference and every run reproduces, so the answer to an auditor's question is a query.
How Do These Problems Show up in Your Team?
Use the symptoms below to find where to start. Most teams recognise four or five; the problem families with the most matches are where improvement pays back fastest.
Symptom you recognise | Likely problem | Start with |
|---|---|---|
Suite goes red after every vendor release | 1, 11 | Intent-based journeys and change tracing |
Pricing issues found by brokers or complaints | 2, 4 | Price tests on every table deployment |
Defects in backdated or renewal scenarios | 5 | Policy calendar test design |
One brand or state breaks when another is changed | 6 | Variant matrix and data-driven journeys |
Customers receive wrong or missing documents | 7 | Document checks per policy event |
Prices change with no error raised | 8 | Assertions that integrations were used |
Problems appear the morning after a release | 9 | Batch outcomes in regression |
Failures nobody can reproduce | 10, 11 | Self-contained data and deterministic runs |
Tests consume their own data | 12 | API-created test data |
Releases wait on two experts | 13 | Plain-language approval by specialists |
Audit questions take weeks to answer | 14 | Release evidence produced as you test |
Unit tests written by the same assistant as the code | 3 | Independent, approved requirements |
Across all fourteen, Virtuoso QA’s insurance testing brings governed regression to the full policy and claims lifecycle, from quote to bind through to first notice of loss and settlement.
Interesting Reads
Duck Creek Testing: How to Automate P&C Insurance Application Testing
Test Automation for SAP Commerce Cloud
Guidewire Testing: What to Test, When, and How to Keep Up With the Release Calendar
Microsoft Dynamics 365 Test Automation: Enterprise ERP Testing with Self-Healing AI
Frequently Asked Questions
What is the most common reason insurance releases fail?
What are the biggest insurance testing challenges?
Why does insurance regression testing take so long?
How do you stop automated tests breaking after every Guidewire or Duck Creek release?
How should insurers test configuration changes made by business teams?
Can AI-generated tests be trusted for insurance software?
