Auto-generating JavaScript Unit Tests

Unit tests are like apple pie, or so we’re told. There are several good unit test frameworks available for JavaScript though they require varying degrees of time and motivation on the part of the developer. In the meantime here’s a lightweight utility that will create a set of rudimentary tests with little more than a click of a button:

tester.testAll(adhocDesigner, {path:'adHocDesigner', debug: true});

Math.max = tester.test(Math.max, {
    customTestBefore: tester.argumentsCountTest);
Math.max(1)


The full source is gisted on github and printed at the end of this article.

How to use it

The module is wrapped by the tester object. It works in Firefox (with firebug), IE, Safari and Chrome (if you use IE7 or earlier you won’t get the benefit of the console).

The API

There are two ways to create tests: tester.testAll which writes tests for all descendant functions of the supplied object (the global object is not a permitted argument) and tester.test which generates a test for a given function. Unless you need to test a global function, or create custom tests on a given function, its recommended that you use tester.testAll. Not only will it save you time, it will assign a name to every function it finds (including anonymous functions) which will be very useful for debugging.

tester.testAll
@param root {Object} create a test for all functions that are descendants of this object
@param options {Object}:
    path {string} the path of the root object – useful for debugging
    debug {boolean} if true debugger will launch on every test failure

tester.test
@param fn {Object} create a test for this function
@param options {Object}:
    customTestBefore {Function} a custom function to be invoked before the original function is invoked
    customTestAfter {Function} a custom function to be invoked after the original function is invoked
    debug {boolean} if true, the script debugger (if installed) will launch on every test failure

tester.untestAll
Reverts all test functions to their original functions.

The Tests

There are two types of tests: ‘before’ tests and ‘after’ tests. These test functions must conform to a standard signature based on type:

‘before’ tests
@param fn {Function} the original function
@param args {Arguments} the value of the arguments in the current execution scope
@param thisBinding {Object} the ‘this’ value of the current execution scope

‘after’ tests
@param fn {Function} the original function
@param result {Object} the value returned when invoking the original function

The tester comes with a small library of built-in test functions. A subset of these functions (the base tests) will be invoked every time. The default base functions are:

tester.argumentsDefinedTest warn if any arguments passed to the function have the value undefined

argumentsDefinedTest: function(fn, args, thisBinding) {
    for (var i=0; args[i] !== undefined; i++);
    return (i != args.length) && "undefined arguments were passed";
}

tester.thisBindingTest warn if the value of this is not the owner of the function. Note the nifty check to eliminate constructor calls from the test ;-)

    thisBindingTest: function(fn, args, thisBinding) {
    //("thisBinding.constructor" checks for constructor functions)
    return !(thisBinding.constructor && (thisBinding.constructor == fn)) &&  
        fn.owners && (!~fn.owners.indexOf(thisBinding)) &&
        "this bound to " + thisBinding + ", but function owner is " +
            fn.owners.pluck('assignedName').join(" OR ");
}

tester.returnTest warn if the tester utility determines a return value was expected but was not returned. (Hopefully the RegEx is clever enough to ignore return statements in inner functions)

returnTest: function(fn, result) {
    return (fn.toString().match(/[\s]+return\s[^\s]+;\n[\s]?}$/) && (result === undefined)) &&
        "return value expected";
}

You can modify the base tests by updating the function tester.defineBaseTests.

Custom tests

When assigning a test function using the test function you can pass custom tests to be included before or after the original function execution. Set the customTestBefore and customTestAfter properties of the options argument and remember to conform to the ‘before’ and ‘after’ test signatures specified above.

Functions with multiple references

Sometimes we create multiple references to a function:

var a = {}, b = {};
a.b = function(s) {return s + 1};
b.b = a.b; 

The tester utility will create a separate test for each reference. This will avoid naming, debugging and thisBinding confusion when the tests are run.

Assigned names

The batch test generator, tester.testAll, will assign a name to every function in the specified object tree. The assigned name is based on the cumulative path of the property chain:

var a = {
    b: function(s){
        return s;
    },
    c: function(a,b) {
        f = "implicit global";
    }
};
a.aa = {
    bb: function(d) {
        return a.b(d) + 13;
    }
}

tester.testAll(a);
//writing tests for a.b
//writing tests for a.c
//writing tests for a.aa.bb

What it doesn’t do

1) Due to issues of parsing and traversing the DOM tree, external framework objects, web storage objects, developer tools, google gadget objects etc. not to mention the prohibitive size of the object tree (and this IE bug) you can not run testAll over the global object (e.g. window). Believe me – I tried :-(

To run tests over a global function (including native functions) call the test function:

alert = tester.test(alert, {customTestBefore: tester.argumentsCountTest});
alert();
//alert: 0 arguments supplied, 1 defined

2) Functions which are not referenced by properties are not visible to the tester utility (e.g. variables and anonymous functions defined inside another function)

How it works

The tester object comprises a handful of core methods:

tester.test

Given an existing function, creates a new function which appends a set of tests to either side of the original function call. This can be invoked standalone or from the batch testAll method. In addition to the fn argument the method takes an options object which can be used to set additional custom tests and to indicate whether a failing test should launch the debugger.

test: function(fn, options) {
    options = options || {};

    var fnName = fn.assignedName || fn.name;

    if (fn.original && (fn.assignedName != fn.original.assignedName)) {
        //there are multiple references to this function, concat owners
        [].push.apply(fn.original.owners,fn.owners);
    }

    //if this is already a test revert to original function and create new test
    fn = fn.original || fn;

    var getTestFunction = function(fn, testBefore, testAfter) {
        return function() {
            for (var i=0; i<testBefore.length; i++) {
                (msg = testBefore[i](fn, arguments, this)) && diagnostic(msg);
            }
            var result = fn.apply(this,arguments);
            for (var i=0; i<testAfter.length; i++) {
                (msg = testAfter[i](fn, result)) && diagnostic(msg);
            }
            return result;
        }
    }

    var diagnostic = function(msg) {
        this.console.warn(fnName + ": " + msg);
        if (options.debug) { debugger }
    };

    //build test suite
    !this.baseTestBefore && this.defineBaseTests();
    var testBefore = this.baseTestBefore.slice();
    var testAfter = this.baseTestAfter.slice();
    options.customTestBefore && testBefore.push(options.customTestBefore);
    options.customTestAfter && testAfter.push(options.customTestAfter);

    var testFunction = getTestFunction(fn, testBefore, testAfter);
    testFunction.original = fn;
    for (var prop in fn) {
        testFunction[prop] = fn[prop];
    }
    return testFunction;
}

The test function itself is defined in the anonymous getTestFunction. It simply runs the ‘before’ tests, uses apply to call the original function, then calls the ‘after’ tests. This is the core of the utility.

The function takes the original function fn and returns the newly created test function. The original function gets assigned as a property of the new test function, for easy reversion. When the test function is passed a function which already has an original property it means that function is itself already a test function. Therefore we revert it to the original function before using it as a seed for generating the new test function.

Morevoer if fn is already a test function and it’s assignedName property differs from that of it’s original function attribute it suggests there must be more than one property referencing the same function. In such cases, before reverting fn to the original function, we reset the original attribute’s assignedName and owner properties to those of fnso that each property will get a unique test function. (OK that’s probably a bit hard to follow at first – but check the code – it will sink in :-) )

The inner diagnostic function is used to shout out test failures and its values are baked in by closure. The options argument is checked for custom functions which are appended to the test suite. Finally any properties of the original function are copied to the test function.

tester.testAll

testAll: function(root, options) {
    if (!root || (root == (function(){return this})())) {
        alert("too many functions to iterate\nspecify a non-global root");  return;
    }

    options = options || {};

    var testChildren = function(root, path) {
        var nextPath, recurse;
        root && (root.assignedName = path);

        for (var key in root) {
            var obj = root[key];
            var objType = typeof obj;
            nextPath = path + (path && ".") + key;
            if ((objType == 'function')) {
                root[key].assignedName = obj.name || nextPath;
                root[key].owners = [root];
                root[key] = tester.test(obj, {debug: options.debug});
                this.console.log('writing tests for ' + nextPath);
                tester.testing.push({root:root, key:key});
            } else {
                recurse =
                    objType == "object" &&
                    obj != root &&
                    ('.' + path + '.').indexOf('.' + key + '.') == -1 &&
                    root.hasOwnProperty(key);
                recurse && testChildren(obj, nextPath);
            }
        }
    }
    testChildren(root || window, options.path || "");
}

This is basically a tree recursor and mostly speaks for itself. The opening statements might be of interest: the self invoking function provides the global object without guesswork (see this post)

var globalObject = (function(){return this})();

Before recursing we check obj != root and ('.' + path + '.').indexOf('.' + key + '.') == -1 to prevent stack overflow (the latter test is overkill and might even result in a few tests being omitted – you can comment it out if you suspect this is the case). Generated tests are added to the array tester.testing for easy batch reversion.

Wrap Up

So there it is. I hope this is useful to you – even if you don’t make use of the tester utility, maybe you’ll get some ideas from the code.

I only just finished the coding so you might find some errors. Also the library of test functions is not extensive. In particular I think a test for accidental implicit globals would be a great addition. Please let me know if you have any ideas or see any errors.

Happy testing!

Full code listing

(see also github)

var tester = {
    testing: [],
    console: window.console || {log: function(a) {window.status = a}, warn: alert},

    defineBaseTests: function() {
        this.baseTestBefore = [this.argumentsDefinedTest, this.thisBindingTest];
        this.baseTestAfter = [this.returnTest];
    },

    testAll: function(root, options) {
        if (!root || (root == (function(){return this})())) {
            alert("too many functions to iterate\nspecify a non-global root");  return;
        }

        options = options || {};

        var testChildren = function(root, path) {
            var nextPath, recurse;
            root && (root.assignedName = path);

            for (var key in root) {
                var obj = root[key];
                var objType = typeof obj;
                nextPath = path + (path && ".") + key;
                if ((objType == 'function')) {
                    root[key].assignedName = obj.name || nextPath;
                    root[key].owners = [root];
                    root[key] = tester.test(obj, {debug: options.debug});
                    this.console.log('writing tests for ' + nextPath);
                    tester.testing.push({root:root, key:key});
                } else {
                    recurse =
                        objType == "object" &&
                        obj != root &&
                        ('.' + path + '.').indexOf('.' + key + '.') == -1 &&
                        root.hasOwnProperty(key);
                    recurse && testChildren(obj, nextPath);
                }
            }
        }
        testChildren(root || window, options.path || "");
    },

    untestAll : function() {
        for (var i = 0, thisTest; thisTest = this.testing[i]; i++) {
            thisTest.root[thisTest.key] = thisTest.root[thisTest.key].original;
        }
        this.testing = [];
    },

    test: function(fn, options) {
        options = options || {};

        var fnName = fn.assignedName || fn.name;

        if (fn.original && (fn.assignedName != fn.original.assignedName)) {
            //there are multiple references to this function, concat owners
            [].push.apply(fn.original.owners,fn.owners);
        }

        //if this is already a test revert to original function and create new test
        fn = fn.original || fn;

        var getTestFunction = function(fn, testBefore, testAfter) {
            return function() {
                for (var i=0; i<testBefore.length; i++) {
                    (msg = testBefore[i](fn, arguments, this)) && diagnostic(msg);
                }
                var result = fn.apply(this,arguments);
                for (var i=0; i<testAfter.length; i++) {
                    (msg = testAfter[i](fn, result)) && diagnostic(msg);
                }
                return result;
            }
        }

        var diagnostic = function(msg) {
            this.console.warn(fnName + ": " + msg);
            if (options.debug) { debugger }
        };

        //build test suite
        !this.baseTestBefore && this.defineBaseTests();
        var testBefore = this.baseTestBefore.slice();
        var testAfter = this.baseTestAfter.slice();     
        options.customTestBefore && testBefore.push(options.customTestBefore);
        options.customTestAfter && testAfter.push(options.customTestAfter);

        var testFunction = getTestFunction(fn, testBefore, testAfter);
        testFunction.original = fn;
        for (var prop in fn) {
            testFunction[prop] = fn[prop];
        }
        return testFunction;
    },


    //LIBRARY OF TESTS
    // 'before' tests take original function, arguments and thisBinding
    argumentsCountTest: function(fn, args, thisBinding) {
            return (args.length < fn.length) &&
                args.length + " arguments supplied, " +  fn.length + " defined";
    },

    argumentsDefinedTest: function(fn, args, thisBinding) {
        for (var i=0; args[i] !== undefined; i++);
        return (i != args.length) && "undefined arguments were passed";
    },

    thisBindingTest: function(fn, args, thisBinding) {
        //("thisBinding.constructor" checks for constructor functions)
        return !(thisBinding.constructor && (thisBinding.constructor == fn)) &&  
            fn.owners && (!~fn.owners.indexOf(thisBinding)) &&
            "this bound to " + thisBinding + ", but function owner is " +
                fn.owners.pluck('assignedName').join(" OR ");
    },

    // 'after' tests take original function, and result of function call
    returnTest: function(fn, result) {
        return (fn.toString().match(/[\s]+return\s[^\s]+;\n[\s]?}$/) && (result === undefined)) &&
            "return value expected";
    },

    resultIsNumber: function(fn, result) {
        if (typeof result != 'number') {
            return "expecting number for result, got a " + typeof result;
        }
    }
}

//UTILITIES
Array.prototype.pluck = function(prop) {
    for (var i = 0, member, result = []; member = this[i]; i++) {
        result.push(member[prop] || member);
    }
    return result;  
}

//new function needed for non ECMA 5 compliant browsers only
Array.prototype.indexOf = [].indexOf || function(member) {
    for (var i = 0, len = this.length; i<len; i++) {
        if (this[i] === member) {
            return i;
        }
    }
    return -1;
}
About these ads

3 thoughts on “Auto-generating JavaScript Unit Tests

  1. Pingback: JavaScript Magazine Blog for JSMag » Blog Archive » News Roundup: Zepto, Zaphod & Narcissus, TLS in Javascript

  2. Pingback: » links for 2010-09-30 (Dhananjay Nene)

  3. Pingback: Patrones de Diseño y JavaScript: Decorator « Aijoona

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s