In a previous post I made a remark that when end to end testing routes on AngularJS with Protractor we might find our tests passing even when Angular throws an exception.
The post was written by Dan Mindru, the maker of Shipixen—the best Next.js boilerplate.
In which situation can checking for console errors help
In the example from that post, I was testing private routes (such as /profile) which had their own controller defined. They redirected to (/home) if logged in, otherwise stayed on the private page. Login was done from /home, which was completely independent from the ProfileCtrl.
Let’s imagine the profile controller threw an exception. The test would execute as follows:
- Navigate to /home - OK
- Login using Auth0 - OK
- Redirect to /profile and expect the route to NOT change - OK
To properly do the test we’ll also check for the opposite: if we go on /profile without being logged in we should to be redirected to /home. This would indeed make the test fail and solve our problem (at least initially). But what if an exception is being thrown in the profile controller only while being logged in?
Where to check the console
This is just one case where we would be fooled by the test, but there are many others. To be sure that we cover as many unforeseen situations, we can check the console in a ‘afterEach’ block:
describe("/profile", function () {
it("should do something", function () {
//test logic here
});
afterEach(function () {
//we'll check for console errors here
});
});
While this will ensure that we are doing a more thorough test, it will also fail our tests when we don’t want it to. Sometimes when our testing workflow includes third party modules, there will be console messages that we can’t control. Even worse, some browsers will pollute the console with all kinds of information, such as Chrome does:
“Consider using ‘dppx’ units, as in CSS ‘dpi’ means dots-per-CSS-inch, not dots-per-physical-inch, so does not correspond to the actual ‘dpi’ of a screen.”
It might be a good idea to make the console check a helper method, as seen here. In this post, I’ll just have it in the afterEach block to keep things simple.
Console reading
If you are planning to do this on a headless server using PhantomJS: it’s not gonna happen (as of Version 1.9). Phantom will crash and burn every time you try to read the console, so you need to do some tricks to make Chrome / Firefox work on a headless server.
After you at least have Chrome, here’s how to read the log:
...
afterEach(function() {
browser.manage().logs().get('browser').then(function(browserLog) {
console.log('\n log: ' + require('util').inspect(browserLog));
}
}
...
If you’ll run a test with this bit of code and output something in the console you’ll see the entire object.
I couldn’t find this documented anywhere, but the returned (browserLog) object will behave like so for the following console outputs:
console.info()
browserLog.level.value: 900
browserLog.level.name: 'WARNING'
browserLog.message.message.level: 'info'
(it also provides a very detailed message & stackTrace in browserLog.message.message.stackTrace)
console.warn()
browserLog.level.value: 900
browserLog.level.name: 'WARNING'
console.error()
browserLog.level.value: 1000
browserLog.level.name: 'SEVERE'
console.debug() & console.log()
Won’t be captured, you can use it as much as you want.
This is the info I got while performing tests in Chrome, please let me know if you encounter a different behavior.
At this point we can just ’expect’ the browserLog length to be 0 and we are good to go:
...
afterEach(function(done) {
browser.manage().logs().get('browser').then(function(browserLog) {
expect(browserLog.length).toEqual(0);
done();
}
}
...
Fine-tuning
It’s easy to spot how console.info()’s and console.warn()’s will make our life hard. As I mentioned, they might come from 3rd party modules or from the browser itself. Moreover, if you are running tests on your everyday browser, pay attention to what extensions/plugins you have on or just disable them altogether. Often browser extensions throw all kinds of funny exceptions that you don’t need to have in your console.
With that out of the way, we can be a bit more sure that at least the console.error()’s will come from the tested app. To only check for SEVERE messages, you can do the following:
...
afterEach(function(done) {
browser.manage().logs().get('browser').then(function(browserLog) {
var i = 0,
severWarnings = false;
for(i; i<=browserLog.length-1; i++){
if(browserLog[i].level.name === 'SEVERE'){
console.log('\n' + browserLog[i].level.name);
//uncomment to see the full error
//console.log('(Possibly exception) \n' + browserLog[i].message);
severWarnings = true;
}
}
expect(severWarnings).toBe(false);
done();
}
}
...
If you run into trouble while checking the console inside an afterEach block, I suggest using it as a helper method so you can control exactly when the console is checked. Plus, you won’t really need to check the console for every single assertion.
There you have it, now you can sleep better knowing that you’re one step closer to having a well-tested application.
If you liked the post, you might like Shipixen—a Next.js boilerplate that allows you to ship a beautifully designed Blog, Landing Page, SaaS, Waitlist or anything in between. Today!