Lightweight Retrying for XMLHttpRequest

Today’s problem to solve is adding retry logic to XMLHttpRequest in JavaScript.

When the request fails I need to provide some basic retry a set number of times and with a delay in the request.

If you’re using jQuery – easy peasy. More than a few retry plugins to jQuery are available like this one by execjosh.

If you’re not using jQuery you could use one of the general Async libraries to make coding Async easier. Async.js includes retry options for you;

// try calling apiMethod 3 times
async.retry(3, apiMethod, function(err, result) {
    // do something with the result
});

Pretty nice. I would recommend doing some tests of your own but consuming something already out there which is actively maintained. This is pretty much the default advice for all development now. Try not to make it yourself.

However, I am working on a solution which needs to run extremely fast so my access to libraries/frameworks are extremely limited.

I need to hand craft the quickest simplest solution to the problem I can.

My secondary requirement came from me and that was I wanted to make it as transparent as possible to the XMLHttpRequest object, just replace XMLHttpRequest with my new object name and you’re done. Plus the retry strategy line.

I pretty much just want a wrapper around XMLHttpRequest which introduces a thin abstraction. That way I can proxy calls between my new object and the original XHR. Intercept the ones I need to capture state in order to retry on failure.

As TypeScript has won I went there first.

class NewXHR extends XMLHttpRequest {
    constructor() { super(); }
    open(method: string, uri: string) {
        console.log("open");
        super.open(method, uri);
    }
}

let xhr = new NewXHR();

xhr.open('GET', 'http://www.google.com');

This looks promising. But in the generated Javascript code there is no mention of a ‘new XMLHttpRequest()’.

TypeScript does some manipulation of prototypes to base the creation of the new inherited object. Which for 99% of the time is great. But in this edge case. I need to set the super class of ‘NewXHR’ to ‘new XMLHttpRequest()’. As Object.create and new Func have some differences.

One thing which was interesting while I was testing all this was the generated code for the super in the constructor.

//TypeScript
constructor() { super(); }

//Generated JavaScript
function NewXHR() {
    _super.call(this);
}

Drop in a ‘new’ keyword;

//TypeScript
constructor() { new super(); }

//Generated JavaScript
function NewXHR() {
    new _super.prototype();
}

For me unfortunately this still doesn’t work.

I played around for a while trying to get TypeScript to bend to my will but it’s inheritance pattern just didn’t seem to fit with what I needed so reluctantly I went to native JavaScript.

I went through a few different iterations using the revealing module pattern but I wanted to create a new object to keep parity with XMLHttpRequest coding style. This led me to using a constructor pattern.

var NewXhr = (function () {
	this.open = function(method, uri) {
  	console.log('open');
  }
});

var newXhr = new NewXhr();
newXhr.open('GET', 'https://www.google.com');

The basic form took shape into this;

var NewXhr = (function () {
  var xhr = new XMLHttpRequest(),
      _maxAttempts = 0,
      _retries = 0,
      _timeBetweenRetries = 0,
      _data,
      _open,
      _onerror;

  this.setRetryPolicy = function (options) {
    if (options.maxAttempts) {
      _maxAttempts = options.maxAttempts;
    }
    if (options.timeBetweenRetries) {
      _timeBetweenRetries = options.timeBetweenRetries;
    }
  };
	this.open = function(method, uri) {
  	_open = { method: method, uri: uri };	//Save for when failure occurs
    xhr.open(method, uri);
  };
  this.send = function (data) {
    _data = data;
    xhr.send(data);
  };
  xhr.onerror = function () {
    console.log('xhr.onerror');
    //retry
    if (_retries < _maxAttempts) {
      retryRequest();
    } else {
      if (_onerror && typeof _onerror === 'function') {
        _onerror.call();
      }
    }
  };
  function retryRequest() {
    if (_retries < _maxAttempts) {
      _retries++;
      setTimeout(function () {
        xhr.open(_open.method, _open.uri);
        xhr.send(_data);
      }, _timeBetweenRetries);
    }
  }
});

var xhr = new NewXhr();
xhr.setRetryPolicy({ 'maxAttempts': 5, 'timeBetweenRetries': 0 });
xhr.open('GET', 'https://www.google.com');
xhr.send();

This is the basic form but to get it working fully I need to add quite a lot of proxy calls and functions just to pass events correctly. Then there was upload progress events. All of this complicated the solution but it still worked well.

The final code is below but a couple of things to note;
I created a 'testFailure' function will destroy the uri slightly a few times until allowing it work proceed as expected. This is my poor mans validation test.

Also if you plug in your own onload function it will get called after mine but the scope is changed so where you would normally do something like 'if (state === this.DONE && xhr.status === 200)' You need to replace 'this' with 'xhr'.

var XMLHttpRequestWithRetry = (function () {
    var xmlHttpRequestWithRetry,
        xhr = new XMLHttpRequest(),
        _open,
        _data,
        _maxAttempts = 0,
        _retries = 0,
        _timeBetweenRetries = 0,
        _testFailure = 0,
        _onabort,
        _onerror,
        _onload,
        _onloadend,
        _onloadstart,
        _onprogress,
        _onreadystatechange,
        _ontimeout,
        _upload_onProgress,
        _readyState = xhr.readyState,
        _status = xhr.status;

    this.setRetryPolicy = function (options) {
        if (options.maxAttempts) {
            _maxAttempts = options.maxAttempts;
        }
        if (options.timeBetweenRetries) {
            _timeBetweenRetries = options.timeBetweenRetries;
        }
    };
    this.open = function (method, uri) {
        _open = { method: method, uri: uri };
        if (_testFailure > 0) {
            _testFailure--;
            uri = updateQueryStringParameter(uri, 'sp', '');
        }
        xhr.open(method, uri);
    };
    this.send = function (data) {
        _data = data;
        xhr.send(data);
    };
    this.upload = function () {
        return {
            id: null,
            filename: null,
            onProgress: null
        }
    };
    this.setRequestHeader = function (name, value) {
        xhr.setRequestHeader(name, value);
    }
    this.DONE = xhr.DONE;
    this.testFailure = function (numberTimesToFail) {
        _testFailure = numberTimesToFail;
    }
    
    //#region Defining Properties to wrap
    Object.defineProperties(this, {
        "readyState": {
            get: function () {
                return _readyState;
            }
        },
        "status": {
            get: function () {
                return _status;
            }
        },
        "onload": {
            get: function () {
                return _onload;
            },
            set: function (value) {
                _onload = value;
            },
        },
        "onerror": {
            get: function () {
                return _onerror;
            },
            set: function (value) {
                _onerror = value;
            },
        },
        "onabort": {
            get: function () {
                return _onabort;
            },
            set: function (value) {
                _onabort = value;
            },
        },
        "onloadend": {
            get: function () {
                return _onloadend;
            },
            set: function (value) {
                _onloadend = value;
            },
        },
        "onloadstart": {
            get: function () {
                return _onloadstart;
            },
            set: function (value) {
                _onloadstart = value;
            },
        },
        "onprogress": {
            get: function () {
                return _onprogress;
            },
            set: function (value) {
                _onprogress = value;
            },
        },
        "onreadystatechange": {
            get: function () {
                return _onreadystatechange;
            },
            set: function (value) {
                _onreadystatechange = value;
            },
        },
        "ontimeout": {
            get: function () {
                return _ontimeout;
            },
            set: function (value) {
                _ontimeout = value;
            },
        }
    });
    Object.defineProperty(this.upload, 'onprogress', {
        get: function () {
            return xhr.upload.onprogress;
        },
        set: function (value) {
            xhr.upload.onprogress = value;
        }
    });
    //#endregion

    //#region xhr functions
    xhr.onabort = _onabort;
    xhr.onloadend = _onloadend;
    xhr.onloadstart = _onloadstart;
    xhr.onprogress = _onprogress;
    xhr.onreadystatechange = _onreadystatechange;
    xhr.ontimeout = _ontimeout;
    //xhr.upload.onprogress = _upload_onProgress;
    xhr.onload = function () {
        var state = this.readyState,
            responseCode = xhr.status;

        _readyState = state;
        _status = xhr.status;

        if (!(state === this.DONE && (responseCode.toString()[0] === '2'))) {
            //retry
            if (_retries < _maxAttempts) {
                retryRequest();
            } else {
                if (_onload && typeof _onload === 'function') {
                    _onload.call();
                }
            }
        } else {
            if (_onload && typeof _onload === 'function') {
                _onload.call();
            }
        }
    };
    xhr.onerror = function () {
        console.log('xhr.onerror');
        //retry
        if (_retries < _maxAttempts) {
            retryRequest();
        } else {
            if (_onerror && typeof _onerror === 'function') {
                _onerror.call();
            }
        }
    };
    //#endregion

    function retryRequest() {
        if (_retries < _maxAttempts) {
            _retries++;
            setTimeout(function () {
                var uri = _open.uri;
                if (_testFailure > 0) {
                    _testFailure--;
                    uri = updateQueryStringParameter(uri, 'sp', '');
                }
                xhr.open(_open.method, _open.uri);
                xhr.send(_data);
            }, _timeBetweenRetries);
        }
    }

    function updateQueryStringParameter(uri, key, value) {
        var re = new RegExp("([?&])" + key + "=.*?(&|$)", "i");
        var separator = uri.indexOf('?') !== -1 ? "&" : "?";
        if (uri.match(re)) {
            return uri.replace(re, '$1' + key + "=" + value + '$2');
        }
        else {
            return uri + separator + key + "=" + value;
        }
    }
});

//Example Use
//var xhr = new XMLHttpRequestWithRetry();

//xhr.onload = function () {
//    console.log('xhr.onload');
//};
//xhr.onerror = function () {
//    console.log('xhr.onerror');
//};
//xhr.setRetryPolicy({ 'maxAttempts': 5, 'timeBetweenRetries': 0 });
//Force a failure in upload
//xhr.testFailure(Math.floor((Math.random() * 4) + 1));
//xhr.open('GET', 'https://www.google.com.au');
//xhr.send('cheese');

Javascript Sample: Return True x% of the time

Dear Future Mike,

Here is that code which you wanted to find.

I’m making a bit of a chaos monkey test and I needed to produce some native Javascript which would return true x% of the time.

function lucky(percent) {
  var r = Math.floor((Math.random() * 100) + 1);
  return r <= percent;
}

JSFiddle

You're welcome Future Mike

Web Workers – Fast messaging, extra threads for your JS, What a great idea – Lets productionize it. ARGH!!!!

TLDR;
A new starting template for a web worker complete with unit tests – as just beyond hello world is a bit of a chasm.

You’ve heard of web workers right? It’s how you get an actual non-UI thread for some big processing in JavaScript. The Hello World example looks like it’s the thing you’ve been looking for. Look how easy it is!

//Main JS
var worker = new Worker('doWork.js');

worker.addEventListener('message', function(e) {
  console.log('Worker said: ', e.data);
}, false);

worker.postMessage('Hello World'); // Send data to our worker.

//doWork.js
self.addEventListener('message', function(e) {
  self.postMessage(e.data);
}, false);

Boom – You know KungFu!!

Enjoy your new mastering of multithreaded JavaScript apps. Only it’s not quite that easy when you get into it. I’m probably going to write a short series of posts about common issues you’ll hit but I’ll start with ticking off a couple of heavy hitters in this article.

  1. Passing/Handling multiple actions
  2. Unit Testing
  3. Reporting Status of worker

Web workers are ace – but they are very ‘raw’. I kind of like that as they aren’t opinionated at all. They basically have no smarts. If JavaScript was written in 2 weeks then web workers were written in 5 minutes. Which means you get to tailor them to your needs. So lets nail my list of items which were the first issues I hit when using Web Workers in anger.

Passing/Handling Multiple Actions

worker.postMessage(message); is nice but what if you want more than just starting an action. You need some method of passing a more complex object. Nice easy support for that – web workers support simple object in the messaging.

This will work fine;
worker.postMessage({ action: 'start', options: { option1: true, option2: false } }); – This is how I’ve done it – I like the simple structure and feels very similar to every eventing model every used. Verb + Options. But message sending goes through a lovely cleansing process which might confuse you when you first see it.

worker.postMessage({ action: 'start', options: { option1: true, option2: false, myFunc: function () { //meh } } }); – When you receive this message in the worker the function ‘myFunc’ will be stripped from the object. BOOO!!! Web workers don’t support functions – but it’s JavaScript a functional language?! Yes but follow the thought exercise – if you could pass functions they the functional context would need to go with it – and web workers have no access to the UI/window/document/etc. So the context would explode in a closure fueled rage. It would be nice to know that it’s happening with maybe a console.info – but hey – I didn’t write the spec.

I’ve found a pattern that works well – lets grab it from the worker side;

    self.addEventListener('message', function (e) {
        var args = JSON.parse(e.data);
        switch (args.action) {
            case 'start':
                start();
                break;
            case 'stop':
                stop();
                break;
        }

    });

Hang on – do you know JavaScript best practices?! Where is the IIFE? – Just a sample guys, but in the instance of a web worker an IIFE isn’t really required – they are completely separate from the main JS code – so probably no fear of bundling issues or ‘use strict’ being ignored but that would depend on your build process. No global namespace bleeding or 3rd party library injection issues but TBH I include one anyway as I feel bad if I don’t.

Couple more useful tips

  1. Include an init function with options object.
  2. Include an Id for the worker – useful so you know where log messages are coming from

This will allow a setup prior to the worker starting – a web worker had a short startup cost ~40ms but you can prep the worker before work is required. Also passing options into the worker is a good idea to make it more flexible (it’s why you have config files for the app).

//worker.js
(function () {
    'use strict';

    var workerId;

    self.addEventListener('message', function (e) {
        var args = JSON.parse(e.data);
        switch (args.action) {
            case 'start':
                start();
                break;
            case 'stop':
                stop();
                break;
            case 'init':
                init(args.options);
                break;
        }
    });

    function init(options) {
        if (options.workerId) {
            workerId = options.workerId;
        }
    }

    //Other functions...

})();

Unit Testing

I’m only going to say this once – UNIT TEST YOUR JAVASCRIPT!! It’s easy and it’s the code closest to your users – Test it! You don’t have to go full TDD but at least put 1 test in your solution.

Setup – Jasmine – nice and easy running in Visual Studio for me using Chutzpah – don’t know this tech? I’ll help you set it up skype/tweet me @mikemengell. I honestly do want to help you test your JavaScript – ping me.

//worker.test.js
/// 
/// 

describe('Web Worker', function () {
    var worker;

    beforeEach(function () {
        worker = new Worker("webworker.js");
    });

    it('has loaded', function () {
        expect(worker).not.toBe(null);
        expect(worker.postMessage).toBeDefined();
    });

});

How easy was that?!

Now what? You want to test that the message passing is working correctly. Your options are being saved correctly, errors are being captured as expected, that sort of thing.

I add a ‘ping’ action which pings back – this tells me that my message passing is working as expected.


    self.addEventListener('message', function (e) {

        var args = JSON.parse(e.data);
        switch (args.action) {
            case 'ping':
                ping();
                break;
            case 'init':
                init(args.options);
                break;
        }

    });

    function ping() {
        self.postMessage('ping');
    }

    //More stuff...

And the test;


    it('ping action causes return message', function (done) {
        worker.onmessage = function (e) {
            expect(e.data).toBe('ping');
            done();
        }
        worker.postMessage(JSON.stringify({ action: 'ping' }));
    });

This test is an Ajax test – putting the expectation inline after the postMessage isn’t going to work for you as it’s an async task. So note the ‘done’ parameter in the test function then we execute it with ‘done()’ when the expectation has occurred.

When you do a certainly amount of unit testing and debugging you want two things;

  1. Was the right function called
  2. Is the internal state what I expect #Controversial

Just quickly on point 2; I don’t believe you should be testing internal state of object much – test the interface not the implementation – which is true. But in a web worker it’s a total black box – you can’t even check console logs/there is no trace/debug output. I found that this makes it really difficult to write and test. If you can write a complex web worker with out testing how things are looking internally then you’re better than me and I doff my cap. For me – I found testing internal state really useful. Buy me a coffee and we can argue about it 🙂

In Jasmine you have Spies which are how you monitor if functions are being called and with what parameters. For us it’s harder as we can’t inject a spy into the worker as it’s all private. So the best way I could think of to solve this one was to inject a console.log redirection and observe that.

I configure this during the init function. Also as an FYI console.log can slow down your code – depending on how many calls you’re doing so I included a console.log switch too.

(function () {
    'use strict';

    var workerId,
        logTo;

    self.addEventListener('message', function (e) {

        var args = JSON.parse(e.data);
        switch (args.action) {
            case 'init':
                init(args.options);
                break;
            case 'ping':
                ping();
                break;
        }

    });

    function init(options) {
        if (options.workerId) {
            workerId = options.workerId;
        }
        if (options.logTo) {
            logTo = options.logTo;
        }
    }

    function ping() {
        log('ping');
        self.postMessage('ping');
    }

    function log(message) {
        if (workerId) {
            message = workerId + '.' + message;
        }
        switch (logTo) {
            case 'console':
                console.log(message);
                break;
            case 'message':
                self.postMessage(JSON.stringify({ action: 'LogMessage', message: message }));
                break;
        }
    }

})();

And the unit test;

    it('redirecting logging to message results in expected messages', function (done) {
        worker.onmessage = function (e) {
            expect(JSON.parse(e.data)).toEqual({ action: 'LogMessage', message: 'ping' });
            done();
        }

        worker.postMessage(JSON.stringify({ action: 'init', options: { logTo: 'message' } }));
        worker.postMessage(JSON.stringify({ action: 'ping' }));
    });

Reporting Status of the Web Worker

Only having postMessage/onmessage to funnel all communication through is a bit of a pain but it’s not too bad. If you want the web worker to report on status/progress of a task then I played around with a few methods but working around the message passing is your only option. I wanted to provide a fairly generic approach to asking the worker for it’s status. There was an unexpected hoop to jump through.

I created a status action, calls the new log function with some additional options that include an array of object references which will be included in the return message.

The unit test

    var workerTestId = 'myWorker';

    it('status with options._this will report on internal state', function (done) {
        worker.onmessage = function (e) {
            expect(JSON.parse(e.data)).toEqual({ action: 'LogMessage', message: workerTestId + '.status', _this: { workerId: workerTestId } });
            done();
        }

        worker.postMessage(JSON.stringify({ action: 'init', options: { logTo: 'message', workerId: workerTestId } }));
        worker.postMessage(JSON.stringify({ action: 'status', options: { _this: ['workerId'] } }));
    });
//worker.js
(function () {
    'use strict';

    var workerId,
        logTo;

    self.addEventListener('message', function (e) {
        var args = JSON.parse(e.data);
        switch (args.action) {
            case 'ping':
                ping();
                break;
            case 'init':
                init(args.options);
                break;
            case 'status':
                status(args.options);
                break;
        }

    });

    function status(options) {
        log('status', options);
    }

    function log(message, options) {
        var returnObj = {},
            returnThis;

        if (workerId) {
            message = workerId + '.' + message;
        }
        if (options && options._this) {
            returnThis = {};
            for (var prop = 0; prop < options._this.length; prop++) {
                returnThis[options._this[prop]] = this[options._this[prop]];
            };
        }
        switch (logTo) {
            case 'console':
                console.log(message);
                break;
            case 'message':
                returnObj.action = 'LogMessage';
                returnObj.message = message;
                if (returnThis) {
                    returnObj._this = returnThis;
                }
                self.postMessage(JSON.stringify(returnObj));
                break;
        }
    }

})();

This doesn't work

Unfortunately this code doesn't work in a web worker. The issue is calling 'this' in the web worker. I could add state to it but couldn't dependably get access to it in another content. I tried things like 'var _this = this;' but it just raises an error '_this is not defined'.

My work around this issue is to put all my state in a single variable, which is a good practice for JavaScript apps anyway to reduce the global namespace footprint.

Now we have;

(function () {
    'use strict';

    var _this = {
        workerId: null,
        logTo: null
    };

    self.addEventListener('message', function (e) {
        var args = JSON.parse(e.data);
        switch (args.action) {
            case 'ping':
                ping();
                break;
            case 'init':
                init(args.options);
                break;
            case 'status':
                status(args.options);
                break;
        }

    });

    function ping() {
        log('ping');
        self.postMessage('ping');
    }

    function init(options) {
        if (options.workerId) {
            _this.workerId = options.workerId;
        }
        if (options.logTo) {
            _this.logTo = options.logTo;
        }
    }

    function status(options) {
        log('status', options);
    }

    function log(message, options) {
        var returnObj = {},
            returnThis;

        if (_this.workerId) {
            message = _this.workerId + '.' + message;
        }
        if (options && options._this) {
            returnThis = {};
            for (var prop = 0; prop < options._this.length; prop++) {
                returnThis[options._this[prop]] = _this[options._this[prop]];
            };
        }
        switch (_this.logTo) {
            case 'console':
                console.log(message);
                break;
            case 'message':
                returnObj.action = 'LogMessage';
                returnObj.message = message;
                if (returnThis) {
                    returnObj._this = returnThis;
                }
                self.postMessage(JSON.stringify(returnObj));
                break;
        }
    }
    
})();

Now the test works fine

    it('status with options.this will report on internal state', function (done) {
        worker.onmessage = function (e) {
            expect(JSON.parse(e.data)).toEqual({ action: 'LogMessage', message: workerTestId + '.status', _this: { workerId: workerTestId } });
            done();
        }

        worker.postMessage(JSON.stringify({ action: 'init', options: { logTo: 'message', workerId: workerTestId } }));
        worker.postMessage(JSON.stringify({ action: 'status', options: { _this: ['workerId'] } }));
    });

OK - Just about done

This covers most of what I think should go into a basic web worker template. It can get more complicated when you want to do more advanced things like Transferable Objects

I'll cover this in the next post.

Our final version + plus a couple of niceties is;

Worker.js Template

(function () {
    'use strict';

    var _this = {
        workerId: null,
        logTo: null
    };

    //Messages to be in the form;
    //{ action: 'VERB', options: {} };
    self.addEventListener('message', function (e) {
        var args = JSON.parse(e.data);
        switch (args.action) {
            case 'ping':
                ping();
                break;
            case 'throwError':
                throwError();
                break;
            case 'init':
                init(args.options);
                break;
            case 'status':
                status(args.options);
                break;
        }

    });

    function ping() {
        log('ping');
        self.postMessage('ping');
    }

    function throwError() {
        throw new Error('Test Error');
    }

    function init(options) {
        if (options.workerId) {
            _this.workerId = options.workerId;
        }
        if (options.logTo) {
            _this.logTo = options.logTo;
        }
    }

    function status(options) {
        log('status', options);
    }

    //#region PrivateFunctions

    function sendNotification(returnObj) {
        self.postMessage(JSON.stringify(returnObj));
    }

    function log(message, options) {
        var returnObj = {},
            returnThis;

        if (_this.workerId) {
            message = _this.workerId + '.' + message;
        }
        if (options && options._this) {
            returnThis = {};
            for (var prop = 0; prop < options._this.length; prop++) {
                returnThis[options._this[prop]] = _this[options._this[prop]];
            };
        }
        switch (_this.logTo) {
            case 'console':
                console.log(message);
                break;
            case 'message':
                returnObj.action = 'LogMessage';
                returnObj.message = message;
                if (returnThis) {
                    returnObj._this = returnThis;
                }
                self.postMessage(JSON.stringify(returnObj));
                break;
        }
    }

    //#endregion
    
})();

Worker.test.js Template

/// 
/// 

describe('Web Worker', function () {
    var worker,
        workerTestId = 'myWorker';

    beforeEach(function () {
        worker = new Worker("webworker.js");
    });

    it('has loaded', function () {
        expect(worker).not.toBe(null);
        expect(worker.postMessage).toBeDefined();
    });

    it('ping action causes return message', function (done) {
        worker.onmessage = function (e) {
            expect(e.data).toBe('ping');
            done();
        }
        worker.postMessage(JSON.stringify({ action: 'ping' }));
    });

    //Running in the browser with debugger attached with show error - but in another thread so Jasmine cant catch it.
    //onError event still gets called
    it('throwError action calls onerror', function (done) {
        worker.onerror = function (e) {
            expect(e.message).toBe('Error: Test Error');
            done();
        }
        worker.postMessage(JSON.stringify({ action: 'throwError' }));
    });

    it('redirecting logging to message results in expected messages', function (done) {
        worker.onmessage = function (e) {
            expect(JSON.parse(e.data)).toEqual({ action: 'LogMessage', message: 'status' });
            done();
        }

        worker.postMessage(JSON.stringify({ action: 'init', options: { logTo: 'message' } }));
        worker.postMessage(JSON.stringify({ action: 'status' }));
    });

    it('Init worker with Id includes Id in logs', function (done) {
        worker.onmessage = function (e) {
            expect(JSON.parse(e.data)).toEqual({ action: 'LogMessage', message: workerTestId + '.status' });
            done();
        }

        worker.postMessage(JSON.stringify({ action: 'init', options: { logTo: 'message', workerId: workerTestId } }));
        worker.postMessage(JSON.stringify({ action: 'status' }));
    });

    it('status with options.this will report on internal state', function (done) {
        worker.onmessage = function (e) {
            expect(JSON.parse(e.data)).toEqual({ action: 'LogMessage', message: workerTestId + '.status', _this: { workerId: workerTestId } });
            done();
        }

        worker.postMessage(JSON.stringify({ action: 'init', options: { logTo: 'message', workerId: workerTestId } }));
        worker.postMessage(JSON.stringify({ action: 'status', options: { _this: ['workerId'] } }));
    });

});

That's it - quite a long one but hopefully it will save you some time. Web workers are ace and much needed for more complex or CPU/IO intensive tasks.

If you have comments or have a better way of handling any of the problems I've mentioned let me know. These solutions have worked for me but it's not the only way to skin the cat.

Understanding the new ASP.net 5 Configuration in Startup.cs

ASP.net 5 has so many changes in it I’m starting to lose track. I know it’s all good stuff but there is a lot of cheese moving at the moment, although RC1 has reduced that a lot now. One element which I needed to understand a little more about was Configuration.

This little sucker in Startup.cs and how to use it;
ASPnet5configuration

Lets take a look at how the values get in there first

Internally Configuration is just a key value pair with some functions, awesome. I’m all about keeping it simple.
IConfiguration
IConfigurationRoot implements IConfiguration

When you define a ConfigurationBuilder you can give it any number of Sources in a nice chaining syntax. Each source (also called Provider) will be enumerated through and will produce a Key:string and a Value:string list.

In the case of AppSettings.json
ASPnet5configuration3
would produce something like;
"AppSettings:StorageConnectionString", "MySuperSecretString"
"AppSettings:SomeOtherSetting", "false"

When you call ConfigurationBuilder().Build() it will flatten out the configs into simple keys and values.

More hierarchy just means more colons (:) in the key, just like a file path.

In the Environment Variable example it’s a straight copy. For instance;
ASPnet5configuration4

Will become;
"WINDIR", "C:\WINDOWS"

However suppose you have conflicting settings, two JSON files who both produce the same key. The Configuration object would still remember all the original settings and where they came from. But if you tried to access the key the value would come from the newest source specified. This is exactly by design and a really elegant way or managing settings over multiple environments.

For instance, when (not if) you deploy to Azure you want to use the app settings from within the portal not the appSettings.json. So the ‘.AddEnvironmentVariables();‘ does exactly that with out you have to change a single line of code. In Azure App Settings you create settings with keys like ‘AppSettings:DbContext’. Azure is making the App Settings in the portal into environment variables and your solution is seamlessly pulling them into your app over your local settings.

A very neat solution but it does come with a caveat. Don’t get the ordering of the config sources wrong. It’s pretty easy in my example but a more complex solution would easily hide this subtle piece of the system.

Strong Typing Configuration Sections

So we know how to get config options into the application but how do we get at them? Easy Configuration["SomeKey"]. Easy for the odd value but crap if you want to get a load of settings.

We need to strong type the settings into a POCO. We can do this in two very simple steps.

  1. First build the POCO, something like this;
    public class AwesomeConfig
    {
        public string Thing1 { get; set; }
        public string Thing2 { get; set; }
        
        public AwesomeConfig()
        {
        }
    }
    
  2. Then to push the config values into the POCO use; services.Configure<AwesomeConfig>(Configuration.GetSection("Awesome"));

So after a suitably deep look into the configuration part of ASP.net 5 I’m still happy. It all makes sense and nothing is hidden from view and easily configurable.

Slick stuff Mr ASP.net *I doth my cap

Improved Cordova Project Template for Visual Studio 2015 adding Bower, Gulp and Angular

CordovaBowerGulp

I do quite a lot in the Cordova space at the moment and I was extremely happy when we announced our improved Tools for Cordova in Visual Studio. Being able to use Ripple was a breath of fresh air to simple app development and getting that cycle time of dev to local test down to seconds was lovely. Ripple has quite a lot of limitations for dealing with more complex apps, or at least apps which use more of the Cordova plugins. But it solves a need and I was happy to include it in my dev process.

For me though the biggest improvement was a short time after the initial announcement when we followed up with more support for Visual Studio 2015. I’m knee deep in Web tech most of the time and especially JavaScript land. Task runners in VS2015 were a god send but they were only supported in ASP.net projects. Until now! You can use Gulp/Grunt/Bower directly within the IDE for Cordova apps. A great move for us and incredibly useful for me personally as I really wanted the improved tooling support.

Why a new template?

The only downside was that the project template for Cordova didn’t get improved along the way. I’m sure our Cordova team are on the case and I will ping them to find out. But in the interim I thought I’d put a project template together. Let me position the thought behind my template. In my opinion you don’t need a build tool like Gulp or Grunt if you are doing the hello world of Cordova app development. It doesn’t make much sense, maybe you want a good linting tool but that’s about it. The power of improved build processes is necessary when you have a more complex app. What does a more complex app look like in Cordova? Well it’s a Single Page App. Cordova is wrapping HTML5 in a webview so in order to get multiple pages working right and calling into a RESTful backend a SPA is a natural evolution.

My project template takes a very simple Angular app and uses that to base the Cordova template on. Why Angular? Simple, I know it best. I’ve played around with other SPA frameworks but I’ve enjoyed Angular most and let’s be honest, it’s clearly the most popular one out there – for now (*cough Aurelia). There are lots of preferences for how to structure for Angular app and the coding practices within. I happen to really like John Papas style guide approach. I find it clean, clear and follows good JavaScript practices.

I prefer Gulp to Grunt at the moment so Gulp is in the template. I’ve included a few plugins which I think are useful for Cordova App Development. If you have others you use please suggest them or better yet do a pull request for the project template. There are so many plugins available and it doesn’t make sense to produce a very specific workflow for a generic project template but there may be some good ones which should be included.

Linting

A key gulp stream to include is ESLint. This is terrific linting tool for JavaScript, it’s highly configurable which means you won’t agree with my basic lint settings. To change them you just edit the eslint config file ‘.eslintrc’ in the project root. You probably want to head over to the list of rules for ESLint and pick you own. The template comes with no linting errors but if you start to edit the rules then obviously linting errors could occur. This is my opinionated view of what feels like good practices based on my experience and readings from the industry.

Get the Template

My Cordova Project Template is for Visual Studio 2015 as it uses the ‘project.json’ file. For now it solves a need for me. If you want to use it/tweak it/fork it then feel free.

Get the Cordova Project Template which uses Angular/Gulp/Bower for Visual Studio 2015

Visual Studio 2015 RC has an outstanding bug. It isn’t importing project templates for Cordova projects correctly. Once it’s fixed and I can confirm it works correctly I’ll publish the Project Template to the giuthub repo. For now just grab the solution, use it as your starting point and party on.

How to Add a Browser to Task Runner in Visual Studio Code

Once you start to use Visual Studio Code you’ll love it, it’s fast and really flexible. It’s not advertised as an IDE but I don’t think it’ll be long until it is considered one.

I mock up sites and samples a lot. I want to create basic site and hit F5 to run it. This is the guide to do just that.

If you want to run any Task Runners then you need to open Code to a folder as there are files Code needs to run. If you don’t see the options to create tasks or run them then remember to open a folder.

  1. Open Visual Studio Code
  2. Open a Folder
    If you want to start your journey to Code Rock Star do it from the command line. (Cheers Stuart!)
    CodeDot
  3. Hit Ctrl + Shift + P to bring up the command palette
  4. Type ctr – This will get you the ‘Configure Task Runner’ command
  5. You’ll get a default tasks.json created in a ‘.settings’ folder.
    Thanks Code, keeping my folder structure clean there
  6. Delete that and replace it with;
    {
        "version": "0.1.0",
        "command": "chrome",
        "windows": {
            "command": "C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe"
        },
        "args": ["index.html"]
    }

    Or an IE version;

    {
        "version": "0.1.0",
        "command": "IE",
        "windows": {
            "command": "explorer.exe"
        },
        "args": ["index.html"]
    }
  7. Now just create your index.html page
  8. Press Ctrl+Shift+B to build the solution and browse to your page

Go forth and Code.

Creating and Publishing a NuGet package for ASP.NET5 and DNX

ASP.NET5 is mega FACT! so many new cool bits. Visual Studio 2015 is really good and being a big fan of JSON all the things I’m very happy with the revised project config files. I wanted to build a little bit of middleware which others could grab from NuGet. The work flow to get this working had a few gotchas, especially working against the new prerelease version of ASP.NET5.

This guide should help you (and me) to get your code into the NuGet gallery and out into the world.

Create your Middleware

  1. Open Visual Studio 2015
  2. Select File > New Project
  3. Select the Class Library (Package) template
  4. Make sure that the Add Application Insights to Project check box is cleared
  5. Name your projectNew_aspnet5_library
  6. Click OK
  7. Add your awesome Middleware code
    If you want to know how to get started with writing some OWIN middleware then take a look at this video on Channel9
    Alternatively if you want to see some then take a look at one of my GitHub repos where there is a simple implementation of some Middleware

Test your Middleware

I found it very useful to test my middleware before getting NuGet in the process. Next we are going to create a test project which we can run our Middleware in.

  1. Select File > Add > New Project
  2. Select the ASP.NET Web Application template
  3. Again make sure that Add Application Insights to Project isn’t checked
  4. Name your project
  5. Click OK
  6. Select one of the ASP.NET5 Preview Templatesnew_aspnet5_site
  7. Open project.json
    project.json is the new config file for your project.
  8. Under dependencies add a new json string – start typing your Middleware project name
  9. Press Tab twice to accept the defaults
    After a few seconds the reference will be added to the DNX and DNX Core reference lists
    references
  10. Open Startup.cs and add your Middleware code
  11. Run your Test site and confirm your Middleware works correctly

Getting Ready to Publish to NuGet

To get ready for publishing to NuGet you need to have things setup the ‘right’ way, so lets go through getting this ready. NuGet packages are just a zip file of a folder. We need to build this correctly to make the process of consuming it nice and easy.

  1. Create a folder outside of your solution which is going to be the root of your NuGet package
  2. In the new folder create another folder called ‘lib’
    This is going to be where we put our dlls
  3. ASP.NET5 doesn’t generate output files by default so in your Middleware project properties, build tab check ‘Produce outputs on build’produce_outputs
  4. Set the build to a Release build
  5. Build your project
  6. Navigate to {Project Folder}\artifacts\bin\{ProjectName}\Release
  7. Copy the folders ‘dnx451’ and ‘dnxcore50’ into the lib folder we created earlier

Publish to NuGet

We’re going to publish to NuGet using the command line tools so as little is hidden from you as possible.

  1. Download the nuget.exe
    This link always has the latest published version
  2. Put nuget.exe somewhere on your system and add it to your environment pathGuide to add Environment Variables
  3. Open a command prompt
  4. Type nuget and press enterThis will test if the nuget is correctly in the pathnugetcmd
  5. Navigate to the project folder and type ‘nuget spec {projectname.xproj}’
    This will create a .nuspec file which is a little XML used to publish/index and install your package. You can create this by hand if you like but the ‘spec’ command will create a basic one for you.
  6. Copy the .nuspec file to the NuGet root folder
  7. Next you want to edit the .nuspec file
  8. Top Tip: If you’re working against a beta version of ASP.NET5 then you’ll need to set your version to {version}-beta. The NuGet Gallery will treat this as a prerelease version.
    Top Tip: Your Middleware has some dependencies if nothing else it has a dependency to ‘Microsoft.AspNet.Http’. You also need to add the DNX and DNXCore version dependencies.

  9. Edit your .nuspec to include the dependenciesnuspec_dependencies
  10. In the command prompt type ‘nuget pack {.nuspec file}’This will create the NuGet package.
  11. Lastly we need to get our package into the NuGet gallery

  12. Open www.nuget.org
  13. Either register or signin
  14. Go to your account and copy your APIKey
    nugetapikey
  15. In command prompt type ‘nuget setapikey ‘ and paste in your API key
  16. Last command ‘nuget push {.nupkg file}’

That’s it – bit fiddly but I’m sure the RTM of ASP.NET5 and Visual Studio 2015 will bring some tooling enhancements.

Before you run off – don’t forget to test your package! Try installing it yourself.
Top Tip:If your package doesn’t seem to work then it might be due to missing dependencies. In one of my tests the exceptions didn’t get visualized.

When there is no console

I was on a customer site trying to debug a site recently.  I had an issue in that I couldn’t open the F12 dev tools as it would cause the page to crash.  Bit odd but I’d put a little debug code in the web page in the form of some console logs.  But if I couldn’t press F12 and read my debug output then what options did I have?

I enjoy finding a hack around things like this so I redirected the console to alerts which I could see.

This solves an extremely thin set of requirements but it might be useful to you and me again in the future so I’m just going to put this here.

(function() { 
  console.log = function (s) { alert(s); }; 
  console.error = function (s) { alert(s); }; 
  console.warn = function (s) { alert(s); }; 
  alert('successfully redirected console logging'); })();

To use it just copy the code below into your address bar. It uses a simple bit of JavaScript injection to add some functionality to the page. When you copy and paste the code the starting ‘javascript:’ might be removed, just make sure that’s there otherwise it won’t work.

javascript:(function() { console.log = function (s) { alert(s); }; console.error = function (s) { alert(s); }; console.warn = function (s) { alert(s); }; alert('successfully redirected console logging'); })();

Also as a friend of mine reminded me that this could be made into a Bookmarklet, so here is that too.

Console Redirect

Drag this link into your favorites bar and then you can click it to inject the hack into any webpage.

OWIN.GNUTerryPratchett with ASP.NET5

discworld

As a child and later as an adult I enjoyed reading Terry Pratchett books, it was easy to get sucked into his crazy Discworld. As a tribute to Terry Pratchett his fans have taken an idea from Postal and kept his name alive within the internet by using a benign header.

X-Clacks-Overhead:GNU Terry Pratchett

You can read more about it all here.

I thought it would be a nice idea to make some OWIN Middleware for it using the newest ASP.NET5 standard. Note that it’s been created against ASP.NET5 Beta 4 – I’ll probably need to update for future versions and when we go RTM on ASP.NET5.

The project is on GitHub here, to use in your app it’s pretty easy just get it from NuGet then Use it.

Install-Package OWIN.GNU_TerryPratchett -Pre

Then to use it in your app;

public void Configure(IApplicationBuilder app)
{
    app.UseMvc();
    app.UseGNUTerryPratchett();
}

It’s only a few lines but if you have feedback let me know.