C# – Using the DynamicData attribute in unit tests

by

The purpose of parameterized tests is to eliminate duplicated tests. There are two ways to pass parameters into a parameterized test: the DataRow attribute and the DynamicData attribute.

With DataRow, the problem is you can only pass in constants and arrays. You can’t pass in reference types. When you try to pass in reference types, you get the following compile-time error:

An attribute argument must be a constant expression, typeof expression or array creation expression of an attribute parameter

This is where the DynamicData attribute comes in. You specify a test data generator method (or property). This generator method returns a list of test parameter arrays. Each bucket in the list is a different test run.

The following shows how to add the DynamicData attribute to a unit test, pointing to a static test method called GetTestData:

[DynamicData(nameof(GetTestData), DynamicDataSourceType.Method)] //arrange
[TestMethod()]
public void TestMathOps(decimal a, decimal b, Func<decimal, decimal, decimal> calculatorOperation, decimal expectedValue)
{
	//act
	var actual = calculatorOperation(a, b);

	//assert
	Assert.AreEqual(expectedValue, actual);
}
Code language: C# (cs)

And here’s the GetTestData test data generator method:

private static IEnumerable<object[]> GetTestData() 
{
	return new List<object[]>()
	{
		new object[]{ 1.2m, 2.3m, (Func<decimal, decimal, decimal>)Calculator.Add, 3.5m },
		new object[]{ 1.5m, 0.5m, (Func<decimal, decimal, decimal>)Calculator.Subtract, 1.0m },
		new object[]{ 1.5m, 2.0m, (Func<decimal, decimal, decimal>)Calculator.Multiply, 3.0m }
	};
}
Code language: PHP (php)

Each object[] is a different test run. In this example, the decimal parameters are an example of passing reference types. And the Func parameter is an example of passing a function to the parameterized test.

When I run the test, I get the following test results. As you can see, it ran the test with three sets of parameters.

Test has multiple result outcomes
   4 Passed

Results

    1)  TestMathOps
      Duration: 12 ms

    2)  TestMathOps (1.2,2.3,System.Func`3[System.Decimal,System.Decimal,System.Decimal],3.5)
      Duration: 4 ms

    3)  TestMathOps (1.5,0.5,System.Func`3[System.Decimal,System.Decimal,System.Decimal],1.0)
      Duration: < 1 ms

    4)  TestMathOps (1.5,2.0,System.Func`3[System.Decimal,System.Decimal,System.Decimal],3.0)
      Duration: < 1 msCode language: plaintext (plaintext)

DynamicData has many test smells – use at your own discretion

You may’ve looked at the DynamicData example above and your “code smell alarm” started going off, and for good reason. Using it leads to many test smells. Using DynamicData is a pragmatic choice. It’s a tradeoff between having duplicated tests and having test smells. It may make sense in your given situation.

I’ll list out a few of the test smells below.

  • Test Smell #1 – When one of the test cases fails, you get useless information about which test failed.

For example, let’s say the test against Calculator.Multiply() failed. This produces the following test results:

TestMathOps (1.5,2.0,System.Func`3[System.Decimal,System.Decimal,System.Decimal],3.0)
      Duration: 21 ms

      Message: 
        Assert.AreEqual failed. Expected:<3.0>. Actual:<-0.5>. Code language: plaintext (plaintext)

Can you easily tell which test case failed? Not really. You can only tell by going and looking at the test data generator method and matching up some of the parameters to the test case.

If you had separate unit tests instead, it would explicitly say that the Multiply test case failed.

  • Test Smell #2 – The arrange step is done outside of the test. Ideally the arrange-act-assert steps would all be contained in the test, making it easier to understand.

Note: This is the same reason why the ExpectedException attribute was a test smell and why they introduced Assert.ThrowsException in MSTestv2.

  • Test Smell #3 – DynamicData leads to overly complicated code.

DynamicData is hard to understand – it’s indirect and complex. You pass in the name of a test data generator method (indirect). This returns a list of object[]’s (indirect). Each object[] is a different test case that contains different parameters. Just looking at the DynamicData approach, it’s not intuitive. Furthermore, the object[] removes parameter safety and type safety at the same time. The compiler can’t possibly enforce passing in an object[] with the correct number of parameters, or parameters with the proper types.

This checks all the boxes for overly complicated code that’s really hard to understand and maintain. However, you’ll have to decide if this excessive complexity is worth it to get rid of duplicated tests.

Related Articles

Comments are closed.