There is a way to run parameterized tests in MSTest namely data-driven tests the downside is that you have to create an external file to host the parameters making the test less readable to my taste. Another alternative is to use MSTest that comes with VS2010 and extend it – I wrote about it in a previous blog post.
I’ve decided to create another solution for those of you that do not use VS2010 and .NET 4 or do not want to extend the testing framework – using data-driven test and the power of PostSharp.
The incidents
- One test marked as a test class no parameters
- One test with the same name with parameters
- One simple RowAttribute to keep the parameters
- One PostSharp OnInvocation attribute to do all of the magic
- Mix & Run
The Test(s)
Because of MSTest limitation will need two functions with the same name. The first one without parameters will be used as a host test method to run and the second will contain the actual test code:
[TestClass]
public class MyRowTests
{
public TestContext TestContext { get; set; }
[TestMethod, RowTest]
[DeploymentItem(@"..\..\SimpleRowTest.xml")]
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML",
"|DataDirectory|\\SimpleRowTest.xml",
"Row", DataAccessMethod.Sequential)]
public void SimpleRowTest()
{
}
[Row(1, 2, 3)]
[Row(3, 2, 5)]
[Row(3, 4, 9)]
[Row(3, 4, 7)]
public void SimpleRowTest(int x, int y, int result)
{
Assert.AreEqual(result, x + y);
}
}
The first method is the one MSTest know and respect and we’re going to use it to run the second method. I’ve added DeploymentItem and DataSource attribute to run the xml file that is going to be generated with the data from the Row attributes.
Note: It is important that the deployment item would point to the relative path of the test project file from the solution file – this is one nasty bit of code and I wish I could find a way to throw it away.
Note: It is crucial that you have TestContext defined in the test class as well – otherwise MSTest won’t be able to access the test’s parameters:
RowAttribute
This is a plain old C# attribute – nothing special, just a container to hold the row test data:
[AttributeUsage(AttributeTargets.Method, AllowMultiple = true)]
public class RowAttribute : Attribute
{
private readonly object[] _rowParameters;
public RowAttribute(params object[] rowParameters)
{
_rowParameters = rowParameters;
}
public object[] Parameters
{
get { return _rowParameters; }
}
}
It’s marked to be used on a method and that it can be applied multiple times and that’s it./p>
RowTestAttribute
This is where the magic happens! The attribute does two things:creates a xml data source file and run the method with the parameters. Inheriting from MethodInterceptionAspect enables hooking into two events – compilation of the class and when it runs:
[Serializable]
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
public class RowTestAttribute : MethodInterceptionAspect
{
private MethodInfo _methodToInvoke;
public override void CompileTimeInitialize(MethodBase method, AspectInfo aspectInfo)
{
base.CompileTimeInitialize(method, aspectInfo);
_methodToInvoke = method.DeclaringType.GetMethods(BindingFlags.Instance | BindingFlags.Public).Single(info => info.Name == method.Name && info != method);
var rowAttributes = _methodToInvoke.GetCustomAttributes<RowAttribute>();
CreateDataSource(_methodToInvoke, rowAttributes);
}
public override void OnInvoke(MethodInterceptionArgs args)
{
var testContext = args.Instance.GetPropertyValue<TestContext>("TestContext");
var parameters = _methodToInvoke.GetParameters();
var arguments = testContext.GetParameterValues(parameters);
try
{
_methodToInvoke.Invoke(args.Instance, arguments.ToArray());
}
catch (TargetInvocationException exc)
{
// fix stacktrace
FieldInfo remoteStackTraceString = typeof(Exception).GetField("_remoteStackTraceString", BindingFlags.Instance | BindingFlags.NonPublic); // MS.Net
remoteStackTraceString.SetValue(exc.InnerException, exc.InnerException.StackTrace + Environment.NewLine);
throw exc.InnerException;
}
}
private string CreateDataSource(MethodBase testMethod, IEnumerable<RowAttribute> customeAttribute)
{
var projectFileName = PostSharpEnvironment.Current.CurrentProject.EvaluateExpression("{$MSBuildProjectFullPath}");
var projectPath = Path.GetDirectoryName(projectFileName);
var filename = Path.Combine(projectPath, _methodToInvoke.Name + ".xml");
using (var sr = new XmlTextWriter(filename, Encoding.ASCII))
{
sr.WriteStartDocument();
sr.WriteStartElement("Rows");
foreach (var rowAttribute in customeAttribute)
{
sr.WriteStartElement("Row");
for (var i = 0; i < rowAttribute.Parameters.Length; i++)
{
sr.WriteStartElement("value" + i);
sr.WriteValue(rowAttribute.Parameters[i]);
sr.WriteEndElement();
}
sr.WriteEndElement();
}
sr.WriteEndElement();
sr.WriteEndDocument();
}
return filename;
}
}
There are three main functions:
- CompileTimeInitialize – that get called when the code compiles and is responsible to create the XML file – remember we need it before the test runs
- OnInvoke – called when the test runs. Reads the data from the data source and convert the parameters before running the second (parameterized) method
- CreateDataSource – you’ve guessed it – this is where we create the xml file – simple and dirty… I use PostSharp’s EvaluateExpression to find where the project’s file is – remember this bit runs on compile so no tricks using reflection could be used to find the target assembly.
Utilities
Simple methods I use in the RowTestAttribute – just in case you’re wondering:
public static class Extensions
{
public static IEnumerable<T> GetCustomAttributes<T>(this MethodBase method) where T : Attribute
{
return (T[])Attribute.GetCustomAttributes(method, typeof(T));
}
public static T GetPropertyValue<T>(this object instance, string propertyName)
{
var propertyInfo = instance.GetType().GetProperty(propertyName);
return (T)propertyInfo.GetValue(instance, null);
}
public static IEnumerable<object> GetParameterValues(this TestContext context, ParameterInfo[] parameters)
{
for (var i = 0; i < parameters.Length; i++)
{
var parameterType = parameters[i].ParameterType;
var inputData = context.DataRow["value" + i];
yield return Convert.ChangeType(inputData, parameterType);
}
}
}
Running the test
MSTest force me to run the empty parameter-less method. Running the test from the beginning of this post will result in the following:
How cool is that – The “test” is actually 4 tests run separately and one of them failed – with a clear error message. And there’s more clicking on the test we get to see the full error message:
In fact I was so impressed with how it works that I put it to us at work despite the fact that it’s not finished yet.
Is it done?
Not yet – I would really like to throw away the DeploymentItemAttribute and if I can get away with it the DataSourceAttribute. I do use it daily and it feels just wrong.
I’ve tried creating the attributes in runtime – using PostSharp but Visual Studio was not fooled – probably because DTE is used to collect these attribute and not reflection.
By publishing this post I was hoping to achieve two goals:
- Help other lost souls such as myself that are forced to use MSTest but wish they had the ability to run RowTests – BTW you’re welcome
- Get feedback, suggestions and ideas how this nifty idea can be improved
And please let me know if you put this code to use – that’s what the comments are for
Happy coding…
References: Enabling parameterized tests in MSTest using PostSharp from our NCG partner Dror Helper at the Helper Code blog.
