Code coverage is one of the metrics that determine the quality of the code. It measures, in percentage values, how well unit tests cover the code of an application. In general, every team should try to achieve high coverage as it usually means less bugs.
It’s often possible to add code coverage into existing pipelines in many CI/CD systems. Combining this with viewable reports tied to each build produces a powerful tool. It may help in directing team efforts and increases a product owner’s confidence. Finally, they often aid in highlighting areas of an application that need some more work.
Code coverage and Azure DevOps
Azure DevOps uses the Publish Code Coverage Results task, which supports Cobertura and JaCoCo formats. This means a need for conversion if a team uses dotCover or OpenCover as their code coverage tool.
Daniel Palme’s Report Generator tool supports a variety of input and output formats. It enables easy conversion from dotCover or OpenCover formats to Cobertura.
There are third party tasks that should perform coverage testing and report conversion. But using a custom PowerShell script will provide more customization options. It will also be usable under many other CI/CS solutions.
Script requirements and workflow
Time to determine the necessary tools along with the basic script workflow.
For certain, the script will require the following tools:
- dotCover Command Line Tools from Jetbrains.
- Report Generator available on GitHub.
- dotnet or VSTest executable depending on team preference or project requirements. Available with Visual Studio.
The script itself should perform the following tasks:
- Create a coverage-report subdirectory in System.DefaultWorkingDirectory. It will contain XML reports provided by dotCover.
- If the coverage-report subdirectory exists, empty it to avoid clutter from previous builds.
- Locate all test projects.
- For each test project, run dotCover CLI with the appropriate parameters. If possible, support both dotnet test and VSTest executables.
- If dotCover execution succeeds use the Report Generator to convert reports to Cobertura format.
There are also some extra requirements:
- The script should rely on Azure DevOps predefined variables. Bonus points for being clean and easy to understand by human beings.
- It should find test projects based on a user-defined pattern. It should also support test project exclusions.
- The script should support dotCover assembly exclusion patterns. There is no need to generate test coverage for third-party modules such as xUnit.
Using the script in a build pipeline
Configuring scripts falls under the area of basic configuration.
There are some checks that will stop the script from executing if something goes wrong.
The script will work with the PowerShell task built into Azure DevOps. It produces output suitable for the Publish Code Coverage Results task.
It’s possible to use the script with the inline setting in the PowerShell task. The script will also work when committed to a repository.
Dotnet restore or NuGet restore tasks should complete before the script is executed.
When using dotCover with a VSTest build, test assemblies before launching the PowerShell.
Recommended settings for the Publish Code Coverage Results task:
- Code coverage tool — Cobertura.
- Summary file — **/coverage-report/cobertura/*.xml.
- Report directory — **/coverage-report/html.
- Fail when code coverage results are missing — Checked.
The script’s code will work on a build server where it is possible to install or add own software. But with time, coffee and the will to tinker, it might be possible to run it on build agents hosted by Microsoft. This is because the software required by the script is available as NuGet packages.
The script supports dotCover with dotnet test and VSTest executables. You can switch between them by commenting out the following lines:
- Comment out or remove either line 6 or 7 to switch between dotnet and VSTest.
- If needed remove checking of the paths in lines 26 – 28 or 30 – 32.
- Finally, remove or comment out lines 92 – 110 or 112 – 143.