Setting up coverage reports on TFS with OpenCover

Code coverage is a metric which indicates the percentage of volume of your source code covered by your tests. It is
certainly a good idea to have code coverage reports generated as part of Continuous Integration – it allows you to keep track of quality of your tests or even set requirements for your builds to have a certain coverage.

Code coverage in Visual Studio is only available in the Enterprise edition. Fortunately, thanks to OpenCover you can still generate coverage reports even if you don’t have access to the Enterprise license.

In this article I will show you how to configure a Build Definition on Team Foundation Server 2015/2017 to use OpenCover to produce code coverage reports.

Preparations

We are going to put some files on TFS. We will need:

  • RunOpenCover.ps1 – PowerShell script that will run OpenCover – we are going to write it in a moment
  • vsts-task-lib – a PowerShell script library which provides some helpful util functions
  • OpenCover executable
  • OpenCoverToCoberturaConverter – a tool to convert the report to a format understandable by Visual Studio
  • (optional) ReportGenerator – a tool do generate HTML reports

The last three items are available as NuGet packages. I suggest organizing all these files into the following directory structure:

Once done, check it in to your TFS instance.

I’ve put the BuildTools directory on the top level of the repository. Next, I’ve added a mapping to my Build Definition in order to make that directory available during the build.

Create the PowerShell script

Let’s now write the PowerShell script. The script is going to perform a couple of steps:

  • We would like our script to use a file pattern to scan for test assemblies in the same way that the “native” Visual Studio Tests task does. For that, we can use Find-Files cmdlet available in vsts-task-lib.
  • Next, we run OpenCover and use the list of paths with test assemblies as parameters.
  • Next, we need to convert the results file produced by OpenCover to Cobertura – a file format which TFS can understand.
  • Finally, we can use the same results file to produce an HTML, human-readable report.

The script will take a couple of parameters as input:

Next, let’s run the Find-Files utility to search against the pattern defined in $testAssembly. This code is copied from the original Run Visual Studio Tests task source code.

We can finally run OpenCover. The command to do this is pretty complicated. OpenCover supports different test runners (VSTest being only one of them) so we need to specify the path to VSTest as one of the arguments. The path below ( %VS140COMNTOOLS%\..\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe ) is valid for Visual Studio 2015 installation.

Another important argument is -mergebyhash . It forces OpenCover to treat assemblies with the same hash as one. I’ve spent a few hours figuring out why my coverage score is so low. It turned out that OpenCover analyzed few copies of the same assembly.

Next, let’s convert the results generated by OpenCover to Cobertura format.

Finally, we will generate a HTML report based on the results from OpenCover.

And that’s it.

Configure the Build Definition

We will need to add three build steps to our Build Definition. If you have a Visual Studio Tests task in it, remove it – you will no longer need it.

  • PowerShell task – set the Script Path to point to RunOpenCover.ps1 and specify the Arguments:

  • Publish Test Results task – configure it as on the image below; as a by-product of generating coverage reports, we produce test results – we need to tell TFS where to find them

  • Publish Code Coverage Results task – configure it as on the image below; thanks to this task the results will be visible  on the build summary page

And that’s it! Run the build definition and enjoy your code coverage results. You can find the on the build summary page. The HTML report is available as one of the build artifacts.

Understand monads with LINQ

This post is another attempt on explaining the M word in an approachable way. This explanation will best suite C# developers who are familiar with LINQ and query expressions. However, if you are not familiar with C# but would like to learn how powerful and expressive some of its features are, please read on!

Recap of LINQ and query expressions

LINQ is a technology introduced in C# 3.0 and .NET 3.5. One of its major applications is processing collections in an elegant, declarative way.

Here’s an example of LINQ’s select expression:

Query expressions are one of the language features which constitute LINQ. Thanks to it LINQ expressions can look in a way which resembles SQL expressions:

Before LINQ you would need to write a horrible, imperative loop which literates over the numbers array and appends the results to a new array.

Single element collection: Maybe class

It’s pretty easy to understand what select expression does in the above example: it apples a given expression to each element of a collection and produces a collection containing the results.

Let’s now imagine that instead of arbitrary collection, we are working with a special kind of collection – one that can have either one element or no elements at all. In other words, it’s either empty, or full.

How should select expression act on such a collection? Exactly the same way that it works with regular collections. If our collection has one element than apply the given expression to it and return a new collection with the result. If the collection is empty, just return an empty collection.

Note that such a special collection is actually quite interesting – it represents an object that either has a value or is empty. Let’s create such an object and call it Maybe.

Let’s create two factory methods to allow more convenient creation of instances of Maybe.

Thanks to type inference in generic method calls and the static using feature we can now simply write:

Making Maybe LINQ-friendly

Since we’ve already discussed how select would work on Maybe, let’s implement it!

Adding support for query expressions to your custom types is surprisingly easy. You just need to define a method which confirms to a specific signature (it’s an interesting design decision by C# creators which allows more flexibility than requiring the type to implement a specific interface).

What’s going on here? Firstly, let’s take a look at the signature. Our method takes a function which transforms the value contained by Maybe to another type.  It returns an instance of Maybe containing an instance of the result type.

If it’s confusing, just replace Maybe with List or IEnumerable. It makes perfect sense to write a select expression which transforms a list of ints to a list of strings. It works the same way with our Maybe type.

Now, the implementation. There are two cases:

  • If the object contains a value than apply the mapper function and return a new Maybe instance with the result
  • If the object is empty, there is nothing to convert – return a new empty Maybe instance

Let’s give it a try:

Nice! We can now use select expressions with Maybe type.

Taking it one step further

Let’s now imagine that given an employee’s id, our goal is to return the name of theirs supervisor’s supervisor. A person can but does not have to have a supervisor. We are given a repository class with the following method:

And a Person class:

In order to find the person’s supervisor’s supervisor’s name we would need to write a series of if statements:

Can we improve this code with our new Maybe type? Of course we can! First of all, since Maybe represents a value which may or may not exist, it seems reasonable for GetPersonById to return Maybe<Person> instead of Person.

Next, let’s modify the Person class. Since a person can either have or not have a supervisor, it’s again a good fit for the Maybe type:

Given these modifications we can now rewrite GetSupervisorSupervisorName in a neater and more elegant way:

Why is this better than the previous version? First of all, we explicitly represent the fact that given a person, the method might or might not return a valid result. Previously, the method always returned a string. There was no way to indicate that it can sometimes return null (apart from a comment). A user of such a method could forget to perform null check and in consequence be surprised by a runtime error.

What’s more, we avoid the nesting of if statements. In this example we only go two levels deep. What if there were 5 levels? Code without these nested if statements is much cleaner and more readable. It expresses the actual logic, not on the boilerplate of null-checking.

Making it work

If you’re copying these snippets to Visual Studio, you might have noticed that the last one won’t compile.

By implementing Select we told the compiler how to apply functions to values inside Maybe instances. However, here we have a slightly more complex situation. We take a value which sits inside a Maybe instance and apply a function to it. As a result we get another Maybe instance, so now we have a Maybe inside a Maybe. The compiler doesn’t know how to handle this situation and we need to tell it by implementing SelectMany.

The first parameter to SelectMany is a function which takes a value (which sits inside Maybe) and returns a new Maybe. In our example, that would be a function which takes a Person and returns its ReportsTo property.

The second parameter is a function which takes the original value, the value sitting inside Maybe returned by the first parameter and transforms both into a result. In our case that would be a function which takes a Person and returns its Name.

Inside the implementation we have the nested if statements that we had to write when we didn’t use the Maybe type. And this is the crucial idea about monads – they help you hide ugly boilerplate code and let the developer focus on the actual logic.

Again, let me draw a diagram for those of you who prefer visual aids:

Ok, so what’s exactly a Monad?

Monad is any generic type which implements SelectMany (strictly speaking, this is far from a formal definition, but I think it’s sufficient in this context and captures the core idea).

SelectMany is a slightly more general version of an operation which in the functional programming world is referred to as bind.

Monadic types are like wrappers around some values. Binding monads is all about composing them. By wrapping and unwrapping of the values inside monads, we can perform additional operations (such as handling empty results in our case) and hide them away from the user.

Maybe is a classic example of a monad. Another great candidate for monad is C#’s Task<T> type. You can think of it as a type that wraps some value (the one that will be returned when the task completes). By combining tasks you describe that one task should be executed after the other finishes.

Summary

I hope this article helped you understand what monads are about. If you find this interesting, check out the F# programming language where monads are much more common and feel more natural. Check out this excellent resource about F#: https://fsharpforfunandprofit.com/.

It’s also worth mentioning that there exists an interesting C# library which exploits the concepts I described in this article: https://github.com/louthy/csharp-monad. Check it out if you’re interested.