Programmatically Running Unit Tests in C#

Unit testing is great for ensuring that you have good code coverage (where blocks of code have a corresponding test). The challenge is that integration testing better represents how the customer will use your products - and this is where the real bugs are found. In C#, Microsoft Test (MS Test) is directly supported in Visual Studio - and code coverage can be setup to enable live feedback for developers. This pushes the onus on build-gate, continuous integration, and nightly testing to perform integration tests. While this works, the original test still needs to be written by a developer and run locally.

A basic test class appears as follows:

[TestClass]
public class MyTests
{
	[ClassInitialize] // runs at the beginning of testing
	public void SetupClass()
	{
	}
	
	[TestInitialize] // runs at the beginning of each test
	public void SetupTest()
	{
	}
	
	[TestMethod]
	[Priority(0)]
	public void RunFirstTest(){
		Assert.IsTrue(1==1);
	}
	
	[TestMethod]
	[Priority(1)]
	public void RunSecondTest(){
		Assert.IsFalse(1==2);
	}

	[TestCleanup] // runs at the end of each test
	public void CleanupTest()
	{
	}
	
	[ClassCleanup] // runs after all tests
	public void CleanupClass()
	{
	}
}

This test class should complete successfully using MS Test.

Running a test outside of MS Test allows for an application to have a Built-In-Test (BIT). There are a few forms if BIT tests: Periodic BIT (PBIT), Startup BIT (SBIT), Initiated BIT (IBIT) and Maintenance BIT (MBIT). BIT tests are useful to ensure that various tests still pass during executino of your application. Sometimes it is hard to anticipate how a user will run your software, and so running BIT tests will let you know when actual use cases approach a failure mode - without actually monitoring their activity (some users don't like that).

Here is how to run the unit tests that are within a test assembly:

var asm = Assembly.Load("MyTestAssembly.dll");

// Get all the test classes in the assembly
var testClassTypes = asm.GetTypes()
	.Where(t => t.CustomAttributes.Any(a => a.AttributeType.Name == "TestClassAttribute"));

foreach (var t in testClassTypes){

	// Get all the test methods in each test class
	var methods = t.GetMethods().Where(m => m.CustomAttributes
		.Any(a => a.AttributeType.Name == "TestMethodAttribute") 
			&& m.CustomAttributes.Any(a => a.AttributeType.Name == "PriorityAttribute"))
		.ToList();
		
	// Order the methods by priority
	var orderedMethods = methods
		.OrderBy(m =>
			m.CustomAttributes.Last(a => a.AttributeType.Name == "PriorityAttribute")
			.ConstructorArguments.First().Value
		);
	
	// A test class should have a parameterless constructor
	var tc = Activator.CreateInstance(t, null);
	
	// Run the ordered test methods
	foreach (var m in orderedMethods) {
		// Invoke the test method
		// An error will be thrown if it fails
		m.Invoke(tc, null);
	}		
}

Any time an assertion does not evaluate correctly, an exception will be thrown.