Using Pex and Microsoft Code Digger to Better Understand and Test Your Code
Code is a bit like an animal.
If a great deal of effort is placed into training the animal (through proper design practices, testing methodologies and reviews) then it can easily be domesticated by an experienced developer and can do just about any trick that its owner desires. In contrast, an untrained, untested and poorly written piece of code is the kind that you worry about when you leave the house and it proceeds to destroy everything you hold dear.
This is why testing is such a crucial part of software development and why it can often help to have any many tools as possible to help you train the animal within your code so it doesn't harbor bugs.
Microsoft Research's Pex team recently released the Microsoft Code Digger, which is a handy extension for Visual Studio 2012 that will allow you analyze all of the possible execution paths that a particular snippet of code could take. This can be extremely helpful when dealing with areas of complex code and to help better understand the code, discover why it behaves in a certain way and to uncover any bugs lurking within it.
About Pex
Pex is one of the many wonderful things going on at Microsoft Research and it is intended to a tool to assist with automating white-box and unit-testing. It can help with generating all kinds of different inputs that can be thrown at a specific set of code and will display each of these execution branches along with the corresponding output of the function. It can provide an easy way for those that aren’t crazy about writing unit tests to simply test their code by letting the Pex Engine run through it.
Pex can provide an excellent way for you to find those small edge-cases that can so often plague software and the fact that the process is completely automated makes it even easier!
Let's Get Digging
To get started using the Code Digger, you’ll just need to go and download the extension from the following site :
After installing it, you should be good to go and ready to get started!
A minor caveat; Code Digger was very recently released and as a result currently only works for Visual Studio 2012 and can only target code that is contained within Portable Class Libraries. But fear not, as there are still numerous other ways that you can use Pex even outside of Visual Studio 2012, which will be covered later within this post.
Getting Started and Swimming with Sharks
Let's take a look at the Code Digger in action, which can help provide a much better explanation of what is going on behind-the-scenes and what makes Pex so magical!
Firstly, we will create a simple Portable Class Library file that we can use to create a very simple function and get an idea of how the Code Digger works.
You'll be prompted to select which environments that you want the code to be compatible with after. Since this is for demonstration purposes, just click OK. Then we will need to create a very simple function to begin with, such as summing an array of integers :
// Simple method to sum an array of integers
public static int Sum(int[] values)
{
return values.Sum();
}
To actually put the Code Digger to work, all you will need to do is simply right-click on your function and select the Generate Inputs / Outputs Table option :
As you can see, the Code Digger provided the following valuable inputs that the function might receive as well as the different behaviors that it might exhibit. This information can be very useful in finding weaknesses and avoiding exceptions within our code as well as finding edge cases that may remain unseen :
Adding a bit more complexity
Now that we know it will work for a simple integer values, let’s try passing in a more complex model and see what kind of results it provides :
// More Complex Example
public static decimal CalculateCandyPricePerServing(IEnumerable<BagOfCandy> bagsOfCandy)
{
return bagsOfCandy.Average(p => p.Servings / p.Price);
}
// A Bag of Candy
public class BagOfCandy
{
public int Servings { get; set; }
public decimal Price { get; set; }
public BagOfCandy(){}
}
which yields the following :
As you can see from the output, adding some additional complexity can also introduce other execution paths which require testing to be performed.
We will need to add the necessary checks within this statement to fix up some of these problems, which should be able to be done through a few simple logic checks :
// A simple method to calculate the average price per
// piece of candy given several bags of candy
public static decimal CalculateCandyPricePerServing(IEnumerable<BagOfCandy> bagsOfCandy)
{
// Rough attempt to fix several more of the issues
// apparent in the first table
if(bagsOfCandy == null || !bagsOfCandy.Any(p => p != null && p.Price != 0 && p.Servings != 0))
{
// Return -1 for invalid results
return -1;
}
else
{
// Attempt to average the values otherwise
return bagsOfCandy.Average(p => p.Servings / p.Price);
}
}
And now after running it, let's see what our results look like :
That's a slight improvement, however there are still a few errors. A little more tinkering with it could help get rid of those few remaining red errors :
public static decimal CalculateCandyPricePerServing(IEnumerable<BagOfCandy> bagsOfCandy)
{
// If the bags of candy are not null and contain any bags
// that are not null
if(bagsOfCandy != null && bagsOfCandy.Any(b => b != null))
{
// Gather the unempty bags with valid prices
var unemptyBags = bagsOfCandy.Where(b => b != null && b.Price > 0);
// If there are any of these bags - output the
// average, otherwise -1
return (unemptyBags.Any()) ? unemptyBags.Average(p => p.Servings/p.Price) : -1;
}
else
{
// Otherwise return -1
return -1;
}
}
which yields :
As you can see, Pex is great and continually challenging your possible inputs to help you avoid errors. This also functions as a great method to teach yourself or others about unit-testing and how to properly develop methods (and hopefully develop solid habits so that you won’t have to do this for every one of your methods).
Don’t Have Visual Studio 2012? No problem!
Although the Microsoft Code Digger is the most recent tool released by the Pex Team, it isn't the only one out there. The team has previously released two of the components of the engine as Add-Ins for Visual Studio 2008 and Visual Studio 2010, which can be downloaded below :
- Pex and Moles - A Pex Engine Power Tools Add-In for Visual Studio 2008 and 2010
Pex consists of the following tools and functions :
- Pex command line (Visual Studio not required)
- Pex Visual Studio Add-in, for Visual Studio 2008 and Visual Studio 2010
- API reference, tutorials and documentation
- Samples and Behaviors
and Moles has the following features :
- Lightweight test stubs and detours for .NET
- API reference, tutorials and documentation
- Samples and Behaviors
These tools should be everything that you need to begin exploring, experimenting and hopefully more easily testing your code.
Note : In Visual Studio 2012, Moles has been replaced by the Fakes Framework. You can read more about handling and isolating your code using the Fakes Framework here.
Try it out yourself online!
If you aren't sure if the Code Digger, Pex or Moles are something that you would be interesting in, you should give it a try online using Pex for Fun!
Pex for Fun is a learning tool and puzzle game designed using the basic concepts behind Pex. It features a very simplified version of Pex, which analyzes a given function and provides you information about the output and challenges you to place the appropriate logic within the function to solve the puzzle.
For example, you initial puzzle might look like this :
using System;
public class Program {
public static int Puzzle(int x) {
return x;
}
}
along with the following output :
Looking at this, you might think "Well since I know the output, I'll just use a switch statement" :
using System;
public class Program {
public static int Puzzle(int x) {
switch(x)
{
case 0:
case 1:
default:
return x;
case 2:
return x + 1;
case 3:
return x * 2;
case 7:
return x * x + x;
}
}
}
Nope! If you have ever been in a programming competition before, you will know this feeling :
If you enjoy puzzles like these, you'll quickly figure out that the solution requires a bit of recursion :
using System;
public class Program {
public static int Puzzle(int x) {
if(x <= 1)
{
return x;
}
else
{
return Puzzle(x-1) + ((Puzzle(x-1) - Puzzle(x-2)) + (Puzzle(x-2) - Puzzle(x-3)));
}
}
}
And to the victor go the glorious, pixelated spoils :
More than Pixelated Golden Medals
The Pex for Fun example as mentioned earlier displays the most very basic features of the Pex Engine and is really meant more for recreational and educational purposes.
The features and functionality that exists there however is quite powerful and if leveraged correctly could dramatically change the way that unit-testing is done (or it could at least act as a great supplement). I encourage you to try downloading any of the Pex packages mentioned throughout this post and try it out yourself.
If you want to access the same examples used within this post, you can download them below :
Perhaps with the incredible continuing progress of the Pex Team and the rest of the folks over at Microsoft Research, unit-testing can and will become more like a game and less like a chore.