Why Test Planning is essential and how to write an actionable test plan.
First, what is a Test Plan? A Test Plan is a blueprint for how testing will be conducted in a project. A Test Plan is a collaborative development between Quality Assurance and the Project teams that helps ensure that the right testing resources are available when they are needed. A Test Plan documents the scope, purpose, and procedures of testing.
Creating test plans is an important part of software testing. Unfortunately, many organizations do not invest in creating test plans. Such plans can greatly benefit projects throughout their lifecycles. Some testers and test managers may say that they do not need to write test plans because they use the Agile approach, or that they do not have time for test plans. A test plan is important to any of the software lifecycle approaches, ensuring that the right resources are in place to meet the test objectives.
Why do you Want to Create a Test Plan?
A test plan gives you a clear understanding of how testing will be conducted and makes it easier to monitor the testing process. Creating a test plan requires you to answer a number of questions concerning your testing needs, priorities, and expected results. This allows you, your team, and the client to have clarity around what we are working with on the quality side.
Investing time into test planning will help you:
- To ensure the project goals, expectations, and requirements are clearly defined.
- To determine the time, effort, and expenses you'll need to deliver the quality of your product.
- To obtain better control over the testing process in all its stages.
How to Create a Helpful and Actionable Test Plan
To help you better understand the test planning process that we follow here at Diamond, let me break the process into multiple steps and elaborate on each of them.
1 - Perform a Product Analysis from the QA’s standpoint
A thorough analysis of the product, its features, and its functionalities is the first step in a successful test plan creation process. After that, the Quality Assurance Lead needs to find out the product objectives to ensure that the test plan and the test cases meet them.
To better understand the user experience, the testing team must be familiar with the target audience of the product. Keep in mind both aspects of the product acceptance - business and end user.
Do a thorough analysis of competitive and similar applications. Identify an application/s that stakeholders consider as competitors. A better analysis comes out if all these activities are done in collaboration of the QA with the Product and Engineering teams.
2 - Define the Scope
When determining the range of tests to be performed, you must capture a full picture of how the product may be used. Create a list of all components that need to be tested, as well as a list of all the hardware and software platforms, tools, and third-party services.
3 - Test Strategy
You may employ different testing techniques on one project depending on use cases and business requirements. So your test strategy should describe all the testing methods your team will use to verify the product's features meet requirements.
4 - Types of testing
Once you have a clear idea for upcoming tests, you can choose different types of tests based on the product, goals, and particular functional and non-functional components under the test.
Here are some examples of the test approaches that we use at Diamond to meet the highest quality level of our products:
- Boundary testing is a process of testing between extreme ends or boundaries between partitions of the input values.
- API testing is a practice that tests the APIs directly from the functionality, performance, reliability, and security angles.
- Integration testing occurs when all of the individual software modules are combined and tested as a whole. The purpose is to identify any faults that might appear when the modules interact with each other.
- System testing verifies the proper functioning of an entire product (End-to-End), ensuring that it performs as intended.
- Accessibility testing is assuring that content on a website or in a mobile app is available to as many people as possible. It makes your web and mobile apps accessible to those with disabilities, such as hearing disabilities, vision impairment, and other physical or cognitive conditions. Here at Diamond, we take Accessible development very seriously and always develop different tools and frameworks to suffice this important type of testing.
- Automated testing reduces the time needed for manual testing by performing many of the tasks that were previously carried out by manual testers. It is especially useful in continuous integration and continuous delivery, where software is developed, tested, and deployed multiple times a day. Please note that test automation cannot be established as the only approach on the project due to limitations and longer establishment time.
At this point, you should have a list of all features you plan to test. Take the list of features and their components and prioritize them according to the delivery plan (feature roadmap). To simplify the priority definition, you can use the same prioritization scale used for bugs and user stories (P0, P1, P2, etc).
Determine Test Criteria
During this stage, you and your QA team should establish success criteria.
Here are some KPIs that can be used as QA success criteria on a project:
- Bug Acceptance rate.
- Accessibility violations rate on screen/page.
- Task accuracy and productivity.
- Regression bug rate.
- Application performance.
- Data indicators.
When planning a QA environment, it is important to consider the right hardware and software. The testing environment is a combination of software, hardware, and other resources used by the testing team when running a Test Plan. Designing an environment that meets the needs of your QA team is important so that you can develop a product that delivers on its intended purpose. Choosing hardware or software that is markedly different from the user-facing environment could cause tests to pass in the test environment, but fail when reaching the end user.
You need to list all the resources necessary for performing tests. Those may be time, people, software, and equipment.
Continue to Identify Test Deliverables
As a product is developed, the scope of testing should also be developed to match the product. It is extremely important that the Product, Engineering and QA teams reach an agreement on deliverables after each product iteration. It is especially important that all QA team members agree on deliverables, because they will be delivering the test results.
Test planning can be challenging because plans often change; however, you should not feel discouraged by these changes. Spend time to prepare for the situation where the scope or requirements get changed. Outline several scenarios of how you would act in that case. For example, what if the number of test platforms or environments increases? What if some user stories are being prioritized over the other tickets or bugs? What if testers need to validate more data than was scoped at the beginning?
A test plan that details every aspect of the project’s test activities and resources will help you work effectively with your QA and the rest of the project teams. It’s important to consider all aspects of the testing process together with your team, so you can ensure successful collaboration and achieve the best results.
A test plan communicates how testing will be conducted to the development, product teams, and to the rest of the organization. Without a test plan, communication about testing is less organized and efficient.