How to use Azure DevOps Test Plans

Azure Test Plans provides tools for testers to improve software quality and collaboration throughout the development process. The browser-based test management solution provides capabilities for planned manual testing, user acceptance testing, exploratory testing, and gathering feedback from stakeholders.

The typical use case of Test Plans is manual testing that should either happen along the development within sprints, or release testing that aims for the stable release of software.

Features

The test plans use test cases and test suites to structure the testing. Test cases are written as their own work items and they can be linked into user stories, requirements and features. Test suites are used to create whole of test cases that are tested. For example our suite can contain all the test cases that are created for current sprint user stories (sprint based testing). The Azure DevOps Test Plan has also browser based test runner, configuration and parameter management and reporting tools to monitor progress of testing.

Sprint based testing

If we want to organize our testing around development sprints and we want to test work that is done within the sprint, we need to link our test cases into user stories (work items). User story (or other kind of work item) can contain multiple test cases which all should be added with link “Tested by”. The tested by link type is used in requirement based test suite. The suite gets automatically all the test cases that needs to be run during the sprint.

The requirement based test suite should target the current sprint and correct area path.

After we have created requirement based suite for sprint work, we can then create charts to monitor the sprint testing progress. Charts shows us the current situation of test runs and their success/failure rate.

Release based testing

If our testing aims for stable release and our testing is not sync’d with development, we can arrange our testing around releases. In this case we don’t link test cases into user stories or features. To define the test cases we can use two different strategies: First we can tag our test cases with release label. Let’s say we add tag “spring_2022” into all test cases that should be run in this release. Then we can create a query based suite which picks all the spring_2022 tagged test cases and build our testing around that. Now when we create new tests cases we can simply tag them with release tag and they will be automatically added into suite.

Another way is to create static suite and hand pick all the test cases which are relevant. We can use queries to get base set of test cases and then add the special cases. The downside of this approach is that new test cases needs to be manually added into suite.

Regression testing

Release or sprint test plan can contain multiple suites and typically we want to run regression tests against our product when the release is getting close. Simple way to handle regression testing is to add “regression” tag into test cases and then add query based suite into test plan that adds all the regression tests. Now if we create any new regression test cases (with regression tag) they are automatically added into test suite.

Typical test plan structure

Typically our situation isn’t so simple, that we could do the testing around sprints or release. The reality has a nasty behavior to be more complex and messy. So usually we combine the release and sprint testing and plan our testing around new features, regression and special cases that might be required by the environment, customer customization or something else. The problem with complex test plan is to track the progress of testing. To do that we need to use the Test Plan progress report tool.

Summary

Azure Test Plans can be used to do the manual testing. Use different suites to add required test cases under test plan and monitor the progress with progress report tool, or with charts if the testing can be focused around sprint or release. In next blog post we will dig deeper into test parameters and configurations, which can be used to tackle the complexity of different environments, test setups and devices.