Setting up coverage reports on TFS with OpenCover

Code coverage is a metric which indicates the percentage of volume of your source code covered by your tests. It is
certainly a good idea to have code coverage reports generated as part of Continuous Integration – it allows you to keep track of quality of your tests or even set requirements for your builds to have a certain coverage.

Code coverage in Visual Studio is only available in the Enterprise edition. Fortunately, thanks to OpenCover you can still generate coverage reports even if you don’t have access to the Enterprise license.

In this article I will show you how to configure a Build Definition on Team Foundation Server 2015/2017 to use OpenCover to produce code coverage reports.

Preparations

We are going to put some files on TFS. We will need:

  • RunOpenCover.ps1 – PowerShell script that will run OpenCover – we are going to write it in a moment
  • vsts-task-lib – a PowerShell script library which provides some helpful util functions
  • OpenCover executable
  • OpenCoverToCoberturaConverter – a tool to convert the report to a format understandable by Visual Studio
  • (optional) ReportGenerator – a tool do generate HTML reports

The last three items are available as NuGet packages. I suggest organizing all these files into the following directory structure:

Once done, check it in to your TFS instance.

I’ve put the BuildTools directory on the top level of the repository. Next, I’ve added a mapping to my Build Definition in order to make that directory available during the build.

Create the PowerShell script

Let’s now write the PowerShell script. The script is going to perform a couple of steps:

  • We would like our script to use a file pattern to scan for test assemblies in the same way that the “native” Visual Studio Tests task does. For that, we can use Find-Files cmdlet available in vsts-task-lib.
  • Next, we run OpenCover and use the list of paths with test assemblies as parameters.
  • Next, we need to convert the results file produced by OpenCover to Cobertura – a file format which TFS can understand.
  • Finally, we can use the same results file to produce an HTML, human-readable report.

The script will take a couple of parameters as input:

Next, let’s run the Find-Files utility to search against the pattern defined in $testAssembly. This code is copied from the original Run Visual Studio Tests task source code.

We can finally run OpenCover. The command to do this is pretty complicated. OpenCover supports different test runners (VSTest being only one of them) so we need to specify the path to VSTest as one of the arguments. The path below ( %VS140COMNTOOLS%\..\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe ) is valid for Visual Studio 2015 installation.

Another important argument is -mergebyhash . It forces OpenCover to treat assemblies with the same hash as one. I’ve spent a few hours figuring out why my coverage score is so low. It turned out that OpenCover analyzed few copies of the same assembly.

Next, let’s convert the results generated by OpenCover to Cobertura format.

Finally, we will generate a HTML report based on the results from OpenCover.

And that’s it.

Configure the Build Definition

We will need to add three build steps to our Build Definition. If you have a Visual Studio Tests task in it, remove it – you will no longer need it.

  • PowerShell task – set the Script Path to point to RunOpenCover.ps1 and specify the Arguments:

  • Publish Test Results task – configure it as on the image below; as a by-product of generating coverage reports, we produce test results – we need to tell TFS where to find them

  • Publish Code Coverage Results task – configure it as on the image below; thanks to this task the results will be visible  on the build summary page

And that’s it! Run the build definition and enjoy your code coverage results. You can find the on the build summary page. The HTML report is available as one of the build artifacts.

10 thoughts on “Setting up coverage reports on TFS with OpenCover

  1. Hi,

    Do you see an issue creating this into a normal TFS / VSTS extension on the marketplace that can be used with any project? Is there something a blocking / missing feature stopping you(or me) from making this into a extension?

    Just asking before I set out to make it into one… 🙂

  2. hi, i am converting open cover xml to cobertura using below given code but cobertura xml file generated does not show any code coverage which is there originally

    Start-Process “Path To OpenCoverToCobertura\OpenCoverToCoberturaConverter.exe” -Wait -ArgumentList “-input:”” Path to open cover \Coverage.xml”” -output:””output path \Cobertura.xml”” ”

    the open cover xml is generated using sql cover tool for finding code coverage on database.

    Can you please help me out ?

  3. Hi Miłosz. Great post!

    I’m struggling in getting this to work, when my TFS executes the step that runs the RunOpenCover.ps1 script it fails saying:

    Find-Files : The ‘Find-Files’ command was found in the module ‘Microsoft.TeamFoundation.DistributedTask.Task.Common’, but the module could not be loaded. For more information, run ‘Import-Module Microsoft.TeamFoundation.DistributedTask.Task.Common’.

    I’ve followed the same directory structure that you suggested, so I’ve the script LegacyFindFunctions.ps1 (which is where I’ve found the function Find-Files) in: scripts\vsts-task-lib\LegacyFindFunctions.ps1.

    On the other hand, I’ve the script RunOpenCover.ps1 in: scripts\RunOpenCover.ps1.

    Any thoughts?

    Thanks!
    Walter

  4. Hi Walter,

    It seems that I forgot to mention that you should load the LegacyFindFunctions.ps1 file. This command should do the trick:

    . $PSScriptRoot\vsts-task-lib\LegacyFindFunctions.ps1

    Let me know if it helped.

Leave a Reply