Testing Alteryx Macros - Part 2 - Our First Test and the Start of a Test Framework

At the end of my previous post I talked about the need for a test framework to start writing our tests.  I am going to build it out piece by piece as I need it.

Recall our first test was going to be

"The macro will add a new column called Total"

So what does a test look like for this?  Well it is an Alteryx module like this:

Let me talk you through it tool by tool.

The bottom left text input is our test input data.  It is configured like so

It is a single field called Field1 and has no data.  Having no data might seem a little strange, but if you look back to our test description, all we are testing is that a new column is added, at this point we don't care about the data just the columns.  Testing the data will be a future test.

The blue circle is the Add Totals tool we are building out, except that remember we haven't added any functionality yet, so right now all it does is pass the input data straight through and looks like this:

The top left text input is our expected data.  It is configured like this

So again we have no actual data, but this is setting the expectation of what we expect our macro to output in order for us to say it passes the test "The macro will add a new column called Total".

That is to say if we put a browse tool after the expected data text input and one after the macro we want the contents of those two browses to be identical.

Which brings us to our final tool and the first part of my test framework that I have built.  It is the "Expect Equal" macro.  The premise of it is fairly simple:  It takes two inputs: an Expected Data and an Actual data and if they differ will throw an error.  Hopefully the error will be detailed enough to quickly explain the difference.  

As I mentioned at the start of this post I am building up the functionality of the framework as I need it.  So right now all it compares is the field names on the two data streams, like so:

So we have a field info tool on each input. We join on name and then collect the mismatches.  If there are any we summarize these into a single record and use a message tool to issue an error message.

All of this together means that if we run our first test module (the one at the top of this page) we get the following output log:

Our first failing test!  With a useful error message telling us the field "Total" is missing from our actual data (of course because we haven't added anything to create it yet).  A good start to our TDD approach to macro building.  

Now before we go on to writing some code to make our test pass I want to look at a better way to run our tests.  My plan is that each test that we identify and build will be a separate Alteryx module and the name of that module will be the name and condensed description of the test.  That way if we break one of our tests we get fast feedback about what we broke.

My first module is called "Adds A Column Called Total.yxmd"  (I can use spaces in windows file names and it improves readability so why not?)

I will keep all of the tests together in a folder and what I want is a one button way to run them all at once.  And that brings us to the second part of our test framework the "Test Runner" macro

This macro will look in a particular folder (where we have saved all of our tests) and run each of the tests in turn and report a pass or fail for them.  So right now it produces this output log

From this log we can see straight away which test is failing and also the error message which is causing it to fail.

And how does the Test Runner macro work?  By using the CReW runner macros of course!

A quick run through of the tools from left to right:

• The Directory tool picks up all of our test modules from the folder we have saved them in.
• The List Runner macro runs each of those tests.
• The first Message tool issues a passed info message for tests which do not have errors.
• The second Message tool issues a failed error message for tests which have errors.
• The Log Parser macro breaks the logs out into a record per row.
• The final Message tool issues info messages for the errors that occurred in the failing test the second row that you can see in the log above.
• The Browse is just there for testing.

It was a great feeling when I realized that the runner macros were exactly what I needed to solve this problem.

So all there is left for us to do is write some functionality to make our test pass and that functionality is as simple as adding a formula tool to our macro

The macro creates a new field called Total and hard codes it to zero.  "Wait a second" you might be saying, "that doesn't seem very useful".  And right now it isn't.  But a guiding principal of TDD is small, fully tested steps.  Write just enough test to make it fail and then juts enough code to make it pass.  And right now we have no tests about what data is in the field, just that the field is created.  So that formula tool is "just enough" code to make our test pass; and if we save the macro and re-run our test runner macro we get the following log

Success! We have completed our first TDD cycle.  Given we have just written the code there really shouldn't be anything to re-factor at this point so time to write our next failing test.

We will continue to build out this macro and the Test Framework in the next post.