Navigate to
Acceptance Testing: The What And Why
- 8 mins read
-
By Marjan Venema
- Updated on January 31, 2024
Acceptance Testing: An Overview
A couple of decades ago, that happened quite often due to the often lengthy time since requirements were specified.
Agile practices have done much to close the gap. But even when your Agile adoption is top-notch, it’s still crucial to have acceptance testing in your arsenal because it involves more than verifying requirements.
This article will fill you in on what you need to know to put it to work effectively and ensure that you deliver software your customers and end-users will be happy to use.
What Is Acceptance Testing, and Why Is It Important?
Historically, as a user, you’d only get your first look at a new software product long after most developers and testers had moved on.
Because of that long gap, acceptance testing was crucial in determining whether the software met expectations and was workable for its users, in other words: whether it was acceptable and fit to go live. Only if it was would it be added to a production environment and used in the normal course of doing business.
In agile development, acceptance testing is part of the process and not an afterthought. However, the intent is still the same: verifying that the software meets expectations from the customer’s and end-users’ point of view.
When you work with a mature agile team, you won’t even see any difference between specifications and verification. You collaborate with the team to create user stories and their acceptance criteria. And you or they write these acceptance criteria in a notation that tools can turn into automated acceptance tests.
The tremendous benefit is that the product under development is always in an acceptable state — no more nasty surprises.
How Does It Differ From Functional Testing and System Testing?
Functional testing is the old name of acceptance testing. The name changed, so it better reflected the testing’s intent.
You now use the term functional testing to distinguish it from non-functional testing — verifying non-functional requirements (more on this in the next section).
The difference between acceptance and system testing is in whose point of view you take. With system testing, you verify the software behaves as expected according to how the developers interpreted and understood the requirements.
Types of Acceptance Criteria
To do an acceptance test and decide whether you’ll accept the software, you need clear criteria. These are known as acceptance criteria.
Functional acceptance criteria say how software needs to behave to help its users do their job. They’re about the functions, or features, your software offers.
Non-functional acceptance criteria specify the requirements for everything else. They’re about how your software does what it does: aspects like its accessibility, user-friendliness, security and privacy safeguards, speed, reliability, and many, many more.
Types of Acceptance Testing
User Acceptance Testing is what you generally think of when you hear the term acceptance testing. But there’s a lot more to acceptance testing than verifying it meets users’ expectations.
User Acceptance Testing (UAT)
During UAT, you’ll run a software product through its paces to ensure that it meets your expectations as a user and that it’s workable for you. In other words, that it helps you do your work and delivers the benefits you set out to get.
In mature agile teams and teams using CI/CD, (automated) acceptance testing combines both user acceptance testing and system testing.
You keep UAT’s user-centricity by close collaboration between end-users and the development team in specifying the acceptance criteria for each user story developed. The development team then uses these to create automated test cases that run whenever an integration builds runs.
Operational Acceptance Testing (OAT)
You use operational acceptance testing to assess whether you can properly run and administer a software system. OAT is also called Operational Readiness Testing (ORT) or Operations Readiness and Assurance (OR&A) Testing.
During OAT, you’ll look at everything you need to keep a software system running smoothly: its security, its scalability, its speed and performance, graceful degradation under peak loads, etc. You’ll also look at its ability to recover from failures, your backup and restore procedures, load balancing, and more.
In short, OAT is the primary type of acceptance testing for all non-functional requirements. The only functional tests you’d perform deal with the functions you need to run and verify these non-functional requirements.
Governance & Compliance, or Regulatory Acceptance Testing
You perform Regulatory Acceptance Testing when the field for which your software gets used is regulated in any form.
Software for medical purposes perhaps comes first to mind. You’ll find other examples in aviation, automotive, oil & gas, shipping, agriculture, fishing. There’s hardly an industry left where you don’t have to deal with rules and regulations.
When you do compliance or regulatory testing, you verify that the software complies with the applicable rules and regulations. These can be functional requirements but quite often are non-functional. Think, for example, of audit trails required to know who updated what and when.
Alpha and Beta Acceptance Testing
Alpha and Beta Acceptance Testing are practices to get early feedback from a limited number of end-users and customers, or even customers’ customers.
The aim is to gather feedback from real-life use to iron out any kinks, such as incorrect or cumbersome behavior, before releasing your software to the general public.
Alpha acceptance testing is also known as Internal Acceptance Testing. And Beta acceptance testing is also known as Field or External Acceptance Testing.
Both are informal forms of user acceptance testing. Alpha or beta testers get early access to new features in return for their feedback. They don’t get a vote in the decision to go live and don’t execute any of your test cases.
Alpha testers are usually internal users and maybe a few select (potential) clients. Sometimes dedicated independent test teams do the alpha testing.
Beta testing follows the alpha phase. The difference is that you let in (a lot) more external end-users.
Contract or Contractual Acceptance Testing
A software contract usually has clauses around how long a customer has after delivery to notify a software company of problems it must address as part of the agreement. A customer will have to pay separately to resolve issues it did not report during the “acceptance period.”
As such, it’s more a reason to do acceptance testing than it is a type of testing. However, a contract can include acceptance criteria you defined when negotiating your agreement with a software vendor. If these didn’t make it into user story or non-functional acceptance criteria, you’d do specific contract testing to verify them.
When Do You Perform Acceptance Testing?
Src: Usersnap
You execute formal acceptance tests as the final test before new software goes live.
However, when you practice CI/CD or trunk-based development with feature toggles, “go live” is when you throw the switch (the feature toggle) to make a feature available to everyone.
So, even then, the idea is still the same. Acceptance tests must all succeed before you go live.
How To Perform User Acceptance Tests in 5 Easy Steps
Acceptance testing informs the decision to go live with a product or feature. It requires a thorough test plan and diligent execution.
The steps to perform an acceptance test as a separate activity are:
Collect the Acceptance Criteria from the user stories that are ready to go live.
- If your user stories don’t yet have acceptance criteria, create them. Do this in collaboration with business professionals, testers, and developers. Check out the section “Three Amigos as the Core Idea of Behavior Driven Development” in Behaviour Driven Development to find out why.
Taking the Acceptance Criteria as input, create at least one test case for each.
- Describe the factual data and situation you need as the starting point.
- Describe the precise actions a user should take from that starting point, including the data s/he is to enter.
- Describe the software’s expected behavior, including the factual data it should display in response to the user’s actions.
Execute the test cases.
- Diagnose failures: determine whether it’s a problem with the test case or the software. Fix issues in test cases and re-execute. Report issues with the software and re-execute when fixed.
Even if it’s no longer a separate activity, you’ll need to go through these steps. They’ll just be part of other development activities. For example, you’d create acceptance criteria as part of requirements gathering or defining user stories.
Close the Gap Between What You Mean and What You Get
Kudos to you. You’ve learned what acceptance testing is. Actually, you’ve learned a lot more.
So, I’m sure you’re with me when I say that acceptance testing is an essential part of any software development process.
You also know now that you can avoid the misinterpretations that lead to getting what you didn’t mean by adopting a test-first approach like BDD. Or at least by creating the acceptance criteria for user stories first and letting them inform both development and QA testing.
So take the first step — create acceptance criteria for every user story you intend to develop — and grow from there.
Speed up your Agile planning and execution!
Signup for a FREE Trial of Nimble Agile
