Unarguably, test cases are really important in our software development life cycle. Good unit test cases save us the hassle of revisiting every part of our code often.
Let's dive into an example and try to understand what do we want to achieve here.
public class Name
{
public Prefix Prefix { get; set; }
public string First { get; set; }
public string Middle { get; set; }
public string Last { get; set; }
public string Suffix { get; set; }
}
When it comes to validating this Name class for every request, we expect 2 properties to be mandatory, FirstName, and LastName. We have created a NameValidator, Validating the scenarios. I'm using FluentValidations for the validations, but it's not relevant here. However, I would recommend using it.
Let's move to our main topic, how do we write test cases for these validators, How do I know the validators are working for all the scenarios and these are covered in code coverage
Xunit in .Net comes very handy. We write multiple test cases, something like
public class NameValidatorTests
{
private IValidator<Name> _validator;
[Fact]
public void Should_Pass()
{
_validator = new NameValidator();
var requet = new Name() { First = "First", Last = "Last", Middle = "Middle" };
var validationResult =_validator.Validate(requet);
Assert.True(validationResult.IsValid);
}
[Fact]
public void Should_Fail_FirstName()
{
_validator = new NameValidator();
var requet = new Name() { Last = "Last", Middle = "Middle" };
var validationResult = _validator.Validate(requet);
Assert.False(validationResult.IsValid);
Assert.Equal("FirstName is required in the request.", validationResult.Errors[0].ErrorMessage);
Assert.Equal("001", validationResult.Errors[0].ErrorCode);
}
[Fact]
public void Should_Fail_LastName()
{
_validator = new NameValidator();
var requet = new Name() { First = "First", Middle = "Middle" };
var validationResult = _validator.Validate(requet);
Assert.False(validationResult.IsValid);
Assert.Equal("Lastname is required in the request.", validationResult.Errors[0].ErrorMessage);
Assert.Equal("001", validationResult.Errors[0].ErrorCode);
}
}Do you mark something? We are writing the failure scenarios more than once, that too for the same Name Validator. You would say, okay Subham it's just 1 extra function, rather it's required to cover the scenarios. Cool, I agree. What if we have more than 5-6 mandatory properties in a class? Should we be writing that many test cases? We can do better.
So I thought of why not create a single function for the failing test cases and run it with multiple test data. The idea behind this is, each validator tests should have only 2 tests, Should_Pass and Should_Fail.
Well, the idea sounds fair but how do we do it? Considering we have different error messages codes to check. Over that, it shouldn't be even complex enough and flexible at the same time. Here I used [Theory] instead of [Fact], you can read about them, I wouldn't be able to cover them here.
I created a JSON file and listed all the scenarios, each scenario would be a property in the JSON File. We will talk about the structure that we want to use in our JSON file later in this blog. I also introduced a custom DataAttribute that can be used to read this JSON file and feed data to our test case with multiple scenarios.I created a JSON file and listed all the scenarios, each scenario would be a property in the JSON File. We will talk about the structure that we want to use in our JSON file later in this blog. I also introduced a custom DataAttribute that can be used to read this JSON file and feed data to our test case with multiple scenarios.
Let's see how does this Custom Data Attribute look like.
public class JsonFileDataAttribute : DataAttribute
{
private readonly string _filePath;
private readonly string _propertyName;
private string gb = Environment.CurrentDirectory;
/// <summary>
/// Load data from a JSON file as the data source for a theory
/// </summary>
/// <param name="filePath">The absolute or relative path to the JSON file to load</param>
public JsonFileDataAttribute(string filePath)
: this(filePath, null) { }
/// <summary>
/// Load data from a JSON file as the data source for a theory
/// </summary>
/// <param name="filePath">The absolute or relative path to the JSON file to load</param>
/// <param name="propertyName">The name of the property on the JSON file that contains the data for the test</param>
public JsonFileDataAttribute(string filePath, string propertyName)
{
_filePath = filePath;
_propertyName = propertyName;
}
/// <inheritDoc />
public override IEnumerable<object[]> GetData(MethodInfo testMethod)
{
if (testMethod == null) { throw new ArgumentNullException(nameof(testMethod)); }
// Get the absolute path to the JSON file
var path = Path.IsPathRooted(_filePath)
? _filePath
: Path.GetRelativePath(Directory.GetCurrentDirectory(), _filePath);
if (!File.Exists(path))
{
throw new ArgumentException($"Could not find file at path: {path}");
}
// Load the file
var fileData = File.ReadAllText(_filePath);
if (string.IsNullOrEmpty(_propertyName))
{
//whole file is the data
return JsonConvert.DeserializeObject<List<object[]>>(fileData);
}
// Only use the specified property as the data
var allData = JObject.Parse(fileData);
var data = allData[_propertyName];
return data.ToObject<List<object[]>>();
}
}We are good to use this property, by giving it just the file path(in case if we have only 1 property/scenario in a single file) or we can pass the file name and property name. PS: Path mentioned below can be relative or absolute. The relative path should be with respect to the DLL that was built.
However, you can also change the property of the file to enable it to get copied to the output directory. i.e with the dll. To do this:-
Right Click on the JSON file and click properties.
We have an option to Copy to Output Directory.
Click on the down arrow beside its value and click copy always.
I'll not be using the above method here, you are free to do it as it makes the code look much cleaner.
Now using the property in the test cases. I created a folder InputData to hold all the JSON files.
public class NameValidatorTests
{
private IValidator<Name> _validator;
[Theory]
[JsonFileData("..//..//..//Validator//InputData//NameValidator.json", "ShouldPass")]
public void Should_Pass(JObject name)
{
_validator = new NameValidator();
var validationResult = _validator.Validate(name.ToObject<Name>());
Assert.True(validationResult.IsValid);
}
[Theory]
[JsonFileData("..//..//..//Validator//InputData//NameValidator.json", "ShouldFail")]
public void Should_Fail(JObject name, string expectedMessage, string expectedErrorCode)
{
_validator = new NameValidator();
var validationResult = _validator.Validate(name.ToObject<Name>());
Assert.False(validationResult.IsValid);
Assert.Equal(expectedMessage, validationResult.Errors[0].ErrorMessage);
Assert.Equal(expectedErrorCode, validationResult.Errors[0].ErrorCode);
}
}Incase if you use Copy to Output Directory, you just need to mention the name of the file. We also get these values in our parameter of test cases, as you can see in the case of Should_Pass and Should_Fail.
The JsonFile looks Something like this.
{
"ShouldPass": [
[
{
"First": "Subham",
"Last": "Saraf"
}
]
],
"ShouldFail": [
[
{
"Middle": "Subham",
"Last": "Saraf"
},
"FirstName is required in the request.",
"934"
],
[
{
"First": "Subham",
"Middle": "Saraf"
},
"LastName is required in the request.",
"934"
]
]
}There are 2 properties in this Json File(ShouldPass, ShouldFail). ShouldFail has multiple scenarios in the form of an array. ShouldPass has only 1 scenario
That's it! Now, we are good to run our test cases.
For should fail scenario, it will run our test case for 2 times. Also, I have mentioned the errorCode and errorMessage that we are going to expect.
We can further refactor it by creating 2 more classes with our newly introduced JsonFileDataAttribute to define ShouldPass and ShouldFail DataAttributes. Now, these classes have ShouldPass and ShouldFail configured to pass.
public class ShouldPass : JsonFileDataAttribute
{
public ShouldPass(string filePath, string propertyName = "ShouldPass") : base("..//..//..//Validator//InputData//" + filePath, propertyName)
{
}
}
public class ShouldFail : JsonFileDataAttribute
{
public ShouldFail(string filePath, string propertyName = "ShouldFail") : base("..//..//..//Validator//InputData//" + filePath, propertyName)
{
}
}We can directly use these DataAttributes in our test cases and pass just the FileName.
public class NameValidatorTests
{
private IValidator<Name> _validator;
[Theory]
[ShouldPass("NameValidator.json")]
public void Should_Pass(JObject name)
{
_validator = new NameValidator();
var validationResult = _validator.Validate(name.ToObject<Name>());
Assert.True(validationResult.IsValid);
}
[Theory]
[ShouldFail("NameValidator.json")]
public void Should_Fail(JObject name, string expectedMessage, string expectedErrorCode)
{
_validator = new NameValidator();
var validationResult = _validator.Validate(name.ToObject<Name>());
Assert.False(validationResult.IsValid);
Assert.Equal(expectedMessage, validationResult.Errors[0].ErrorMessage);
Assert.Equal(expectedErrorCode, validationResult.Errors[0].ErrorCode);
}
}There are many other ways to do this, We can have only 1 data file for all the tests in case if we have few classes and we can name each class with 2 properties. We will just have 1 InputData.json file.
We can actually do a lot with this. Yes, I'll address all the questions/problems faced by you guys.
So, go ahead, play around with this. :)
For any suggestion or query, feel free to reach out to me.
Thanks,
I'm Subham Saraf, A software developer based in India,
A technological enhusiast who thinks data driven decison making(DDDM) is the best thing that has happend to the industry in this decade.
Propsing solutions for software development that makes developement process more intuative and updated.
Apart from Web-development, doing some personal research on implementing ELT by building Data Lake and it's macro components.
Also Pursuing my masters from Liverpool John Moores University