Jim's Blog Ramblings about novels, comics, programming, and other geek topics

29Nov/072

Simple Introduction to Unit Testing

Google AdSense

For some unknown reason, I fell in love with Unit Testing after first reading about it a few years ago. I think it's the concept of trying to break another developer's code, trying to out think another developer, and trying to find holes in their implementation that intrigues me. I haven't written as many Unit Tests as most developers, so maybe that's why I still like writing these tests.

I believe that it's not a good idea to have the programmer write their own unit tests, because most of the time if the programmer could think of various ways to break their code, they would have implemented solutions for it. It's much better for another developer to write Unit Test cases for the code, since they may not have intimate knowledge of the actual implementation details.

Although, sometimes you don't work on a team of programmers or have other programmers available to write Unit Tests for your code. In most of my recent projects, I was/am the only developer so I need to write my own Unit Tests. I've found that for the most part, the Unit Tests aren't very useful because I've accounted for everything that I can think to test in the implementation. But every now and then, I've found a few methods where testing with nulls or boundary conditions returns an error that I need to resolve.

What is Unit Testing?

Simply put, Unit Testing is when you write code that tests individual units (methods/classes) of a program. So if you write code to test a class method, then that's a Unit Test. Unit Testing does not eliminate other forms of testing. It compliments testing by testing staff and alpha/beta/end-users.

Why use Unit Testing?

Unit testing allows developers to test their own code before Testing staff gets a chance to test it. I also think that most of the time, Unit Tests will test parts of the code that normal testers won't or can't test. In my experience, most of my testers are usually testing items they can access via the user interface. If there's anything going on in the background, then they can't really test it unless something in the UI breaks and then you would need to start at the UI and debug backwards until you find the actual method causing the problem.

Example of Unit Testing:

Here's the signature of method that we will write some test cases for.

public static string ArrayToCommaDelimitedList(string[] valueArray)

This method takes an array of string values and returns a comma delimited list of items. For example, if we passed in two strings of "A" and "B", then we should get "A, B".

The first test we'll create will be to just validate the example we just mentioned. When naming Test classes, I use a standard such as:

  • Namespace naming: ProjectName.Tests.Helpers, where classes to be tested will be in ProjectName.Helpers. I just insert "Tests" as a namespace and try to use the same Namespace names to make organizing my tests easier to find. The above method that we are testing will be in ProjectName.Helpers namespace and the Tests will be in ProjectName.Tests.Helpers. I also create a second Visual Studio project to hold the test cases and never include the test cases in the same library as the code to be tested. 
  • Class naming: ProjectName.Tests.Helpers.ArrayHelperTests. I append "Tests" to the end of the class name. I like to have one Test class per implemented class. The only difference is that "Tests" is appended to the end of the testing class name and the test class includes a namespace of "Tests".

Here's the code for MyProject.Tests.Helpers.ArrayHelperTests class. In this example, I'll be using MbUnit's Unit Testing Framework.

using System;
using MyProject.Helpers;
using MbUnit.Framework;

namespace MyProject.Tests.Helpers
{
   [TestFixture]
   public class ArrayHelperTests
    {
       [Row(new string[] { "A", "B", "C" }, "A, B, C")]
       [Row(new string[] { }, "")]
       [Row(new string[] { null }, "")]
       [RowTest]
       public void ArrayToCommaDelimitedList(string[] listItems, string expectedOutput)
       {
           string actualOutput = ArrayHelper.ArrayToCommaDelimitedList(listItems);
           StringAssert.AreEqualIgnoreCase(expectedOutput, actualOutput);
       }
   }
}

The above test class has one test case (RowTest) with three sets of testing criteria (Row). This is one of the better functions of MbUnit. Some Unit Testing software doesn't allow you to use RowTests, so you would need to create a different Test Method for each set of criteria (if you're Unit Testing framework doesn't support a RowTest like feature, then you would need to create three test case methods for the above example).

Using TestDriven.NET Visual Studio add-on, I can just right click the project, namespace, class, or method to start the Unit Testing. After MbUnit is finished testing, it makes a nice Test Summary HTML report document that includes details of each Test with pass/fail and exception messages.  The output of the testing was:

2 passed, 1 failed, 0 skipped, took 2.23 seconds

Now, let's examine why we had a failed test. The testing results provides us details on why the test failed stating:

[failure] ArrayHelperTests.ArrayToCommaDelimitedList(System.String[],)
TestCase 'ArrayHelperTests.ArrayToCommaDelimitedList(System.String[],)'
     failed:  Equal assertion failed: [[]]!=[[, ]]

It appears that Test #3 failed because the method we are testing does not correctly render a list with a null value. Our expected output was "" and the actual output was ", ". Since the actual output is not what we expected or want, we can now go back and fix the method to properly handle a null value and then re-run the tests to validate the changes. After making the change by adding a IsNullOrEmpty check on the values passed to the method and rebuilding the projects, all three test cases now pass.

Unit Testing Frameworks and Tools

Frameworks:

Tools:

 

19Nov/072

Debugging assembly loading within a unit test project

Google AdSense

I reorganized one of my projects and discovered that my MbUnit tests no longer worked. The error stated that one of the assemblies could not be loaded, but it worked fine before I removed and added the project to another solution. The assembly in question was the SubSonic assembly.

This took me a little while to work through and I wasn't able to find any helpful blogs or documentation to help. So I figured that I'll contribute to the blogosphere and maybe someone else might find this useful.

I thought through the problem and finally figured it out. I had forgotten about how MbUnit executes. Even though, MbUnit's output is sent to the Visual Studio's output window, the tool runs in a separate process - remembering that I was able to debug and figure out how to solve my problem.

The root cause was that the assembly wasn't in the working directory for the test assembly so it couldn't be loaded by MbUnit. The SubSonic assembly was not in the GAC, the SubSonic directory wasn't in my %PATH%, and the assembly's "Copy Local"property was set to false.

By changing the "Copy Local" property, the test project automatically copied the SubSonic assembly to the bin directory and MbUnit was then able to load the assembly without any errors. Alternatively, you could add SubSonic to your GAC (Global Assembly Cache) or add the SubSonic directory (the directory with the .dll file) your environment path. Here's a little more information and the steps I used to resolve the problem.

Software

  • Visual Studio Professional 2005
  • SubSonic 2.0.3
  • MbUnit 1.0.2700 Add-in
  • TestDriven 2.8 Add-in

Symptoms

TestCase failed: An error occurred creating the configuration section handler for SubSonicService: Could not load file or assembly 'SubSonic' or one of its dependencies. The system cannot find the file specified.

System.Configuration.ConfigurationErrorsException

Message: An error occurred creating the configuration section handler for SubSonicService: Could not load file or assembly 'SubSonic' or one of its dependencies. The system cannot find the file specified.

Source: System.Configuration

StackTrace:
    at System.Configuration.BaseConfigurationRecord.FindAndEnsureFactoryRecord(String configKey, Boolean& isRootDeclaredHere)
    at System.Configuration.BaseConfigurationRecord.GetSectionRecursive(String configKey, Boolean getLkg, Boolean checkPermission, Boolean getRuntimeObject, Boolean requestIsHere, Object& result, Object& resultRuntimeObject)
    at System.Configuration.BaseConfigurationRecord.GetSection(String configKey, Boolean getLkg, Boolean checkPermission)
    at System.Configuration.BaseConfigurationRecord.GetSection(String configKey)
    at System.Configuration.ClientConfigurationSystem.System.Configuration.Internal.IInternalConfigSystem.GetSection(String sectionName)
    at System.Configuration.ConfigurationManager.GetSection(String sectionName)
    at SubSonic.DataService.LoadProviders()
    at SubSonic.DataService.GetInstance(String providerName)

Inner Exception:
    System.IO.FileNotFoundException
    Message: Could not load file or assembly 'SubSonic' or one of its dependencies. The system cannot find the file specified.

Source: System.Configuration

StackTrace:
    at System.Configuration.TypeUtil.GetTypeWithReflectionPermission(IInternalConfigHost host, String typeString, Boolean throwOnError)
    at System.Configuration.RuntimeConfigurationRecord.RuntimeConfigurationFactory.Init(RuntimeConfigurationRecord configRecord, FactoryRecord factoryRecord)
    at System.Configuration.RuntimeConfigurationRecord.RuntimeConfigurationFactory.InitWithRestrictedPermissions(RuntimeConfigurationRecord configRecord, FactoryRecord factoryRecord)
    at System.Configuration.RuntimeConfigurationRecord.RuntimeConfigurationFactory..ctor(RuntimeConfigurationRecord configRecord, FactoryRecord factoryRecord)
    at System.Configuration.RuntimeConfigurationRecord.CreateSectionFactory(FactoryRecord factoryRecord)
    at System.Configuration.BaseConfigurationRecord.FindAndEnsureFactoryRecord(String configKey, Boolean& isRootDeclaredHere)

Solution

  1. Expand "References" of the Test project
  2. Select "Subsonic" library
  3. Set "Copy Local" to true
  4. Rebuild the test project