.NN#13: Unit Testing Rules From the Rules Engine

Blog

Download the Code Sample to follow along if you want.

I’ve recently given several talks on the Windows Workflow (WF) Rules Engine and during one of those talks someone asked the question, “So how do you unit test these rules?”  At the time I simply said, “wait for the demo I show on using the rules engine outside of workflow”.  Later in the presentation I showed a sample of executing rules out side of a workflow and again indicated you could do something like this for your unit testing.  I then also made a mental (recorded) note to post something on unit testing your rules.

Just to set up expectations for this post:

What this post isn’t about:

  • This post is NOT about unit testing your workflow.
  • This post is NOT about unit testing activities.
  • This post is NOT about unit testing code conditions.

What this post is about:

  • Unit testing declarative rules used by the rules engine (either inside or outside a workflow). (See conclusion for why you might not want to.)

Some of this article is going to assume a level of understanding regarding the Rule Engine and how it works.  Just so that I don’t recreate my talk on the subject in a blog post, please read “Introduction to the Windows Workflow Foundation Rules Engine” on MSDN.  This will give you a solid understanding on how the engine itself executes rules. 

Okay, with that out of the way let’s get started.  I’ve posted the sample code for this post to the (brand new) .NET Nugget Code Gallery on the MSDN Code Gallery site.  If you want to download it and follow along it might be helpful.  Note that this code was written in VS 2008 and makes use of the Microsoft Unit Testing Framework included in VS 2008 Pro and above.  You can easily adapt the unit tests to NUnit, MBUnit, or your favorite testing framework.

The Scenario:

Here’s our scenario: We have a workflow (in the code this is OrderWorkflow in the SomeWorkflowLibrary project) that contains some declarative rules that are executed by a policy activity.  We have rules within this RuleSet that determine the shipping costs by weight.  Then there is a rule that says if the OrderSubTotal is a hundred dollars or more we offer free shipping.    Simple enough.  Below is a snapshot of what the rules are defined as:

  1. WeightRuleOne: IF this.Order.Weight <= 10 THEN this.Order.ShippingAmount = 2.0m
  2. WeightRuleTwo: IF this.Order.Weight > 10 && this.Order.Weight <= 40 THEN this.Order.ShippingAmount = 14.0m
  3. WeightRuleThree: IF this.Order.Weight > 40 && this.Order.Weight <= 80 THEN this.Order.ShippingAmount = 40.0m
  4. WeightRuleFour: IF this.Order.Weight > 80 THEN this.Order.ShippingAmount = 50.0m FreeShippingForExpensiveOrders: IF this.Order.OrderSubTotal > 100.0m THEN this.Order.ShippingAmount = 0.0m

Note that in the RuleSet definition the FreeShippingForExpensiverOrders rule has a priority of zero, while the other four have a priority of 5.  This ensures that the rules engine processes the free shipping rule last. 

The code sample has a program host for the workflow and it will execute the workflow and spit out information to the console.  Feel free to mess around with the order amount and weight in the program.cs file of the SomeWorkflowApplication project.  If you’re not familiar with WF this file is starting an instance of the workflow runtime, creating an instance of the OrderWorkflow and then starting the workflow.  In the following code we are creating an instance of an Order object with a SubOrderTotal of $150 and a Weight of 50:

Order myOrder = new Order(int.MinValue, 150m, 15m);
Dictionary<string, object> workflowParams = new Dictionary<string, object>();
workflowParams.Add("Order", myOrder);

After the instance of Order is created we place it in a dictionary object and pass it to the OrderWorkflowInstance when we call CreateWorkflow.

WorkflowInstance instance = workflowRuntime.CreateWorkflow(typeof(SomeWorkflowLibrary.OrderWorkflow), workflowParams);

Under the hood this is assigning the myOrder object instance to a public property on the OrderWorkflow named Order.  This should be pretty intuitive that we want to pass in an Order instance since we are running a workflow against a specific Order, but I’ll indicate why this is important later.

The OrderWorkflow executes a policy activity (which runs the Order instance through the rules engine) and then branches to show some information.  The actual workflow isn’t that important to us since we are talking about unit testing the rules, not the workflow.  I provided it in the code sample for a little more realism and so that you can play around with the values to see how the rules engine behaved without tweaking the unit test code.  If you want to, change the rules in the rules editor by going to the properties of the orderRulesPolicy activity on the workflow designer and clicking on the ellipses next to the RuleSetReference property.  You might want to wait until after you’re finished with this article though. :)

On to the Unit Tests:

The goal of this post is to figure out what we need to do if we need to test the free shipping rule.  To that end let’s take a closer look at the TestOrderRules project.  First off note that it has some important references: System.Workflow.Activities and System.Workflow.ComponentModel.  These references are what contain the objects we need to read in the rules file, deserialize the rules into a RuleSet object and then execute the rules.  These are the same references you would use if you wanted to engage the rules engine outside of a workflow, and in fact, that’s what we are doing for our unit tests.

Remember when I said it was important that we were assigning our Order instance to a public property of the OrderWorkflow workflow?  The rules engine works against the instance of a specific Type.  For most rules that are executed within a workflow this type is the actual workflow itself.  Note in the rules listed above that the reference is to this.Order.ShippingAmount, the “this” is the instance of the OrderWorkflow.  When you define the rules in the rules editor tool from the workflow designer the workflow type is passed in and used for validation.  This means I couldn’t add a rule action or condition that used this.FOO unless a FOO property existed on the workflow; however, it is important to note that the type the rules are built against is NOT stored in the rules file.  This means that as long as the object being passed into the rules engine has all the properties and methods the rule set expects the rules engine doesn’t care.  We can substitute another object for the workflow, which is why we have the OrderTestContainer class.

public class OrderTestContainer
{
    private Order _order;
    public OrderTestContainer(Order order)
    {
      _order = order;
    }
    public Order Order
    {
        get { return _order; }
    }
}

This class simply has a public property of type Order named Order, just as the workflow has a dependency property of the same name.  When we want to test the rules we will create an instance of the OrderTestContainer, assign the Order to it and pass that in to the rules engine.  Note that this can be a little more complicated depending on how much your rules are doing.  We’ll touch on that a little more in the conclusion.

As I said earlier, our unit tests are actually going to have to execute the rules outside of the workflow.  To do this we need to perform the following steps:

  1. Load the rules file and deserialize it into a RuleSet object.
  2. Provide context for the rules in regards to the type they will be executed against to allow for validation.
  3. Execute the rules against a specific instance representing our test data.

I’ve created a static helper class named RulesHelper in the TestOrderRules project to accomplish these steps.  This static class has two methods that are called from our unit tests: RetrieveRulesSetFromFile and ExecutesRulesOnObject.

public static RuleSet RetrieveRuleSetFromFile(string fileNameAndPath, string ruleSetName)
{
    //guard and validation of parameters code removed....
    WorkflowMarkupSerializer serializer = new WorkflowMarkupSerializer();
    using (XmlReader rulesReader = XmlReader.Create(fileNameAndPath))
    {
        RuleDefinitions ruleDefs = serializer.Deserialize(rulesReader) as RuleDefinitions;
        if (ruleDefs == null)
        {
            throw new Exception("The rules could not be loaded from the rules file.");
        }
        if (!ruleDefs.RuleSets.Contains(ruleSetName))
        {
            throw new ArgumentOutOfRangeException(string.Format("The rule set {0} was not found with the rules file.", ruleSetName));
        }
        return ruleDefs.RuleSets[ruleSetName];
    }
}

The RetrieveRuleSetFromFile method accepts two string parameters, one is the file path location of the .rules file to load and the second is the name of the RuleSet to extract (a .rules file can contain multiple RuleSets).  The WorkflowMarkupSerializer class is in the System.Workflow.ComponentModel namespace and it is used to serialize and deserialize objects like RuleDefinitions and Workflows.  In this case we use it to deserialize the RuleDefinitions within the file loaded using the file name provided by the fileNameAndPath parameter.  After the RuleDefinitions are deserialized we look for a specific RuleSet, and if it exists, we return it as a RuleSet.  Note that the method would throw an exception if it can’t deserialize the RuleDefinitions or if it can’t find the specified RuleSet in the definitions.

public static void ExecuteRulesOnObject<T>(T objectToRunRulesAgainst, RuleSet rulesToExecute)
{
    RuleValidation validation = new RuleValidation(typeof(T), null);
    RuleExecution execution = new RuleExecution(validation, objectToRunRulesAgainst);
    try
    {
        rulesToExecute.Execute(execution);
    }
    catch (RuleSetValidationException ex)
    {
        Debug.WriteLine("Rules validation errors:");
        foreach (ValidationError error in ex.Errors)
        {
            Debug.WriteLine(error.ErrorText);
        }
    }
}

The ExecuteRulesOnObject method is a bit more interesting.  It takes a generic object to execute the ruleset against and the rules to execute.  The RuleValidation class is used to validate the RuleSet.  The API won’t let you execute a set of rules unless they are valid.  Earlier I stated that the rules engine executes the rules against a specific object and the RuleValidation object is being created to use the type of the object passed in as the objectToRunRulesAgainst parameter.  By using the generic type of T for this parameter we generalize the ExecuteRulesOnObject method to be used in many different unit tests that check different rule sets.

After the RuleValidation object is created the method creates a RuleExecution object which will be used to actually execute the rules.  It takes an instance of the RulesValidation object and the instance of the object we are trying to apply the rules to.  We then execute the rule set by calling Execute on the RuleSet that was passed in, passing the RuleExecution object.  This is what actually applies the rules.  In the code above I wrap that execution in a try - catch so that if there are validation errors they are written out to Debug and would be captured as part of the test results to the output window.

There is one final helper that I put into the actual OrderRulesTests.cs class and that is an overloaded method named getOrderRules.  This helper method is used to call the RetrieveRuleSetFromFile helper with either a default set of parameters, or allows calling with specific values.  This method isn’t that interesting so I’ll skip the code for it.  Just be aware that when called with no parameters the RetrieveRuleSetFromFile method is called using the file path to the .rules file in the workflow project and uses the name of the “OrderRules” ruleset.  The method will be called by each unit test that needs to retrieve a RuleSet to execute.

Now that the helpers are out of the way let’s take a look at the unit tests.  The first test I wrote was the one that validated an order of $100 or more received free shipping.

[TestMethod]
public void OrderRulesTests_Orders100DollarsAndOverGetFreeShipping()
{
    RuleSet orderRules = getOrderRules();
    Order testOrder = new Order(int.MinValue, 101m, 15m);
    OrderTestContainer orderContainer = new OrderTestContainer(testOrder);
    RulesHelper.ExecuteRulesOnObject<OrderTestContainer>(orderContainer, orderRules);
    Assert.AreEqual<decimal>(0.00m, testOrder.ShippingAmount, "Shipping amount was not zero.");
}

The test starts by getting the RuleSet via the getOrderRules helper.  Next it creates an instance of an Order class being sure to set the OrderSubTotal over $100.  Next we place the test order into an OrderTestContainer object.  If you recall the OrderTestContainer is used as a replacement for the workflow when the rules are executed.  The next line calls our ExecuteRulesOnObject helper indicating the type (OrderTestContainer) and passes our test container and rule set.  Finally we can do our asserts as we normally do to validate our results for the test. 

I’ve written a second test that validates that orders under $100 do not get free shipping.

[TestMethod]
public void OrderRulesTests_OrdersUnder100DollarsAreChargedForShipping()
{
    RuleSet orderRules = getOrderRules();
    Order testOrder = new Order(int.MinValue, 99m, 15m);
    OrderTestContainer orderContainer = new OrderTestContainer(testOrder);
    RulesHelper.ExecuteRulesOnObject<OrderTestContainer>(orderContainer, orderRules);
    Assert.AreNotEqual<decimal>(0.00m, testOrder.ShippingAmount, "Shipping amount was not applied.");
}

This is just as the first unit test, but specifically passes in an order that has a OrderSubTotal of less than $100.  Note that my assert ensures that the shipping cost is NOT zero.

Conclusion:

In this post I’ve discussed how to create some unit tests for your rules; however, I wonder if this is really a good thing or not.  The great thing about the Rules Engine and how it works is that the rules are abstracted from your code.  In the code samples the rules file is actually compiled into the assembly since I choose to do that, but I could have also created the instance of the workflow by passing in a reference to an external rules file (via an XMLReader), which means that the rules could change at any time.  In that scenario is it really a good idea to have unit tests based on rules that could change easily and be different at runtime?

Personally, I think the answer to if you should unit test your rules depends on a few things.  Cases where I think you’d want to unit test some of your rules:

  • The rules file is compiled into the assembly code, and thus won’t change unless someone updates the code anyway.  This is not a blanket statement, just a reason to think about doing it.
  • The rules are complex and using forward chaining you want to ensure the correct outcome.
  • You have a known set of rules that only change on rare occasions and are there because they are extremely important.  Examples of this would be that you have a set of rules that run make adjustments based on federal or state laws.
  • You have rules that call out to several methods in their condition or action items.  Here I would suggest having unit tests for the methods themselves first before thinking about unit tests for the rules.

As for when you would not want to unit test the rules, well that list is actually much larger, but I’ll give you an example of a simple one.  Take a look at the following unit test:

[TestMethod]
public void OrderRulesTests_IsCheckingShippingCostRulesReallyNecessary()
{
    RuleSet orderRules = getOrderRules();
    Order testOrder = new Order(int.MinValue, 99m, 15m);
    OrderTestContainer orderContainer = new OrderTestContainer(testOrder);
    RulesHelper.ExecuteRulesOnObject<OrderTestContainer>(orderContainer, orderRules);
    Assert.AreEqual<decimal>(14.00m, testOrder.ShippingAmount, "Shipping amount not calculated correctly for weight.");
}

This simply ensures that the correct shipping cost of $14.00 was applied when the weight was greater or equal to 10 and less than 40.  This seems like a reasonable test; however, I see some issues if you take this approach:

  • You are arbitrarily selecting a specific weight (15) to test a range.  You could select 10 or even 39.9, but again, this is just arbitrarily selecting a weight.  Since the rules can change it wouldn’t be much at all to add a rule that says IF this.Order.Weight = 15.00m THEN this.Order.ShippingAmount = 5000m.  This is worse if your rules are pulled in at runtime.
  • The rule is so simple that this would be similar to testing property setters.  Sure this is logic here, but based on the fact that it can change easily without changes to the code, what have we gained? (I’m probably going to hear it from the unit test zealots on that one… so fire away).

Another issue that comes up is the simplicity of the scenario above.  This scenario works with a single object on the workflow.  The rules only access properties of the Order object and they never call a function that is on the workflow or another object.  In these more advanced scenarios the method above becomes more cumbersome and tricky; however, there is a way you can help mitigate this.

To help simulate the workflow as we did with the OrderTestContainer you could create an interface that both your test container and the real workflow would implement.  For our example it would look like this:

public interface IOrderWorkflow
{
    Order Order { get; set; }
}

If we added more properties that the rules would be interested in, or methods for that matter, we could add this to this interface so we can be sure our test container could handle them.  If necessary we could use a mocking framework like RhinoMocks to simulate responses to calls to that testing container.

Note that this approach is also basically checking the entire RuleSet.  We could create another helper method that once it loaded the RuleSet went through and disabled all the rules except the rule you wanted to test.  You could then execute the ruleset for just that rule.  This may not be a good idea if you have a lot of implicit chaining going on though because within a real execution of the ruleset you may see that forward chaining causes the rule to be re-evaluated and the result of the rules may be completely different.

So there you have it.  After writing this I think there are several issues with testing your rules.  For one, they can change very easily.  Some might say that when code changes you have the same issue.  This is true; however, in my opinion because you are pulling these rules out into the rules engine so that they can be easier to change then by definition they will most likely change much more often than your code does.  The second issue is the complexity of testing a large set of rules.  New rules can cause your results to be incorrect in your unit tests and chaining can cause a lot of issues.  I think the testing of the rules should be retained not for unit testing, but functional testing instead.  Perhaps something like Fitnesse can help here (cue Jim Holmes)?

As always, please let me know what you think.  Also, let me know what you think about posting the code samples to the MSDN Code Gallery.