Posts

unit testing
,

Why You Need Test Coverage, But Shouldn’t Trust It

I’ve never heard someone say that unit tests—or any kind of tests—are useless. Practically everyone agrees that testing our software is a good idea. There are objections about the amount of time it takes to set up the testing environment and create tests, but let’s assume you’ve gotten past all of that and are actually writing tests. How do we know that we’ve tested everything we need to? Tests can’t help if there are parts of your code that aren’t tested. That brings us to the topic of this article: test coverage.

What Is It?

There are tools out there—probably for every programming language, but we’ll focus on JavaScript—that run alongside your tests and track which code is executed during the tests. Don’t ask me how it’s done; it’s all magic as far as I’m concerned, but it works, even on code that is compiled/transformed as long as there are source maps available. Most of these tools are able to give you statistics on which lines, statements, and code branches have been executed, so you can know what parts of your project could use more testing.

So how do you get started with integrating test coverage tools into your testing? In some cases, such as with the Jest unit testing framework or with scaffolding tools such as Vue CLI, it’s built right in so you don’t have to do anything extra to enable code coverage, at least for your unit tests. Apart from that, you can use popular test coverage tools like Istanbul, Blanket.js, or JSCover. There are also tools that go beyond plain test coverage such as Sealights, which tracks coverage statistics over time and offers other enterprise-level features. However, it has a cost, so choose one of the free and open-source projects if you’re not looking for anything super advanced. To learn how to install the tools, you’ll have to check out each individual project’s site for their instructions.

The Problem With Test Coverage

While using test coverage tools can be considered by many to be essential, there’s something you’ll need to be aware of. Just because a test coverage tool tells you that something is covered, that’s not the end of the story. Coverage simply informs you what code was executed, not whether you are adequately testing to make sure the code produced the correct outcome.

Here are a few situations where coverage can be misleading:

if (a || b) {
  foo()
} else {
  bar()
}

In this example, if you test with a being true and b being false, and do another test where they are both false, then your results will come back saying that you’ve covered 100% of this code. But you haven’t tested the case where b is true and a is false. This may or may not be important depending on your code, but it’s important to test numerous cases to ensure there are no unexpected side effects.

Here’s another example:

eventEmitter.subscribe('event', () => foo('event'))
// ... more code ...
eventEmitter.trigger('event')

If this code is run by any of your tests, then your coverage tool will report that everything is covered, but if you don’t test to ensure that the execution of that event listener had the correct effects, then it isn’t fully tested.

Conclusion

Should you still use test coverage tools since they don’t give you perfectly accurate information? Yes! They may tell you something is tested when it technically isn’t, but if they say some code isn’t covered, then it certainly isn’t covered, and you’ll know to create tests that cover that code. This is quite useful even if sometimes they can give you a false confidence in your testing. In other words, if you aren’t checking your test coverage, it is likely in your best interest to do so.

,

Unit Testing Vue Components

Unit Testing Vue Components

If you’re wise, before you decide to use a framework, you check to make sure you’ll be able to adequately unit test your application if that framework is in place. So, before jumping onto the Vue bandwagon, this question must be answered: can you unit test a Vue application without going through more trouble than it’s worth? Well, let’s take a look at how to test Vue components and you can come to your own conclusion.

Setting Up

There are several unit testing library/framework options, but if you’re looking for a recommendation, then use the Vue CLI to scaffold out your Vue project, using either the “webpack”, “browserify”, or “pwa” template, and then answer “Y” to the “Setup unit tests with Karma + X?” (X is Jasmine when you use the browserify template and Mocha for the other two templates). Then it will pretty much all be set up for you. Many people would prefer to make their own decisions about what frameworks to use and how they set up the project to do testing. That’s not a bad thing, but sometimes it’s better to let someone/something else make that decision for you rather than spending precious developer time on researching your own choice.

In this case, Karma allows you to run all of your tests in multiple browsers at once or just PhantomJS as the default. It also integrates with webpack so you know your components are being compiled the same way they are when the application is built. This is especially useful when using Single File Components (aka .vue files), which can be tricky to compile otherwise. Mocha and Jasmine are both quite popular testing libraries, so you can’t go wrong with either one.

If, however, you really want to consider all of your options, then Jest may be a good solution. It’s touted as being very fast and the snapshots feature can make testing the final HTML output extremely simple. If you’re using .vue files, then you’ll need a pre-processor to compile them. This sadly doesn’t work well with CSS Modules, but there are workarounds that work to an extent.

As another consideration, you may want to have code coverage reporting. In that case Jest is a good option because it’s built in, but Vue CLI will also include code coverage if you decide to do unit testing. In any case, there are several code coverage libraries out there that can integrate with just about any unit testing framework.

Honestly, any framework should work as long as you either use plain JavaScript components or can find a way to compile the .vue files to JavaScript. There are too many options to be able to go into any kind of detail regarding setup, except to say that Vue CLI can do it for you if you’re OK with their choices.

How to Test

For these examples, it’s assumed that you’re using the unit testing setup that Vue CLI scaffolded out for you with the “webpack” template. For the most part, the concepts carry over to the other testing libraries. If you’re using something else, you should hopefully be able to follow along anyway.

To start with, let’s create a relatively simple component that’s pretty useless but will allow us to learn how to test multiple aspects of Vue components. We’ll call it Foo.vue:

%MINIFYHTMLfbae5dea5774a1b62fc529421c21620015%

Now let’s create our first unit test file for this file, which should be located in the test/unit/specs folder if you’re using the same scaffolding template. We’ll call it Foo.spec.js and just write out the very basic things to set up:

import Vue from 'vue'
import Foo from '@/components/Foo' // @ is configured to be our main src directory

decribe('Foo.vue', () => {
  // Our tests will go here 
}

Static Components

The first and simplest thing to test is the static functions on your components. In this case, we can test Foo.data and Foo.methods.giveZero, which should work without instantiating or mounting the component. Foo.computed.msg and Foo.watch.who both reference this, so they’ll need to be tested on an instance of Foo.

it('should have correct `data`', () => {
  expect(typeof Foo.data).to.equal('function')
  const data = Foo.data()
  expect(data.who).to.equal('world')
  expect(data.updates).to.equal(0)
})
it('should return 0 from `giveZero`', () => {
  expect(typeof Foo.methods.giveZero).to.equal('function')
  expect(Foo.methods.giveZero()).to.equal(0)
})

All of these tests should pass, so let’s take a look at testing a component instance since you can’t get very far simply by testing the static component definition.

Component Instances

To do this we need to use Vue to mount the component, but it won’t be mounted to any element in the DOM; it will just render it in memory. There are two ways to do this:

// Mount method 1
const Constructor = Vue.extend(Foo)
const vm1 = new Constructor().$mount()

// Mount method 2
const vm2 = new Vue(Foo).$mount()

While the first method is more verbose, it has the advantage of allowing you to pass in options for props. For example:

const Constructor = Vue.extend(Foo)
const vm1 = new Constructor({
  propsData: {
    someProp: 'custom value'
  }
}).$mount()

That’s equivalent to the following code inside a template:


You’ll likely end up mounting your components quite a bit during your testing, so it may be wise to create a helper function to do it and put it into a module that you can import into all of your tests.

function mount(component, options) {
  const Constructor = Vue.extend(component)
  return new Constructor(options).$mount()
}

Now let’s write a couple tests to see how to use mount:

it('someProp defaults to "default value"', () => {
  const foo = mount(Foo)
  expect(foo.someProp).to.equal('default value')
})

it('someProp can be set', () => {
  const foo = mount(Foo, { propsData: { someProp: 'custom value' } })
  expect(foo.someProp).to.equal('custom value')
})

Note, just like when working inside a component, once an instance is created, all props, data, and computed properties can be accessed directly off the root of the instance. There’s no need to find a computed property under foo.computed.msg or anything like that.

it('computed property is correct', () => {
  const foo = mount(Foo)
  expect(foo.msg).to.equal('Hello world')
})

Component DOM Rendering

Arguably, the most important thing that a component can do is render the correct DOM, so it’s probably important to test if the DOM was rendered correctly. We can access the DOM through the $el built-in property on our component instance. Through this, we can check anything we want, such as whether or not the correct text was rendered in a certain element. For example:

it('render proper DOM', () => {
  const foo = mount(Foo, { propsData: { someProp: 'custom value' } })
  expect(foo.$el.querySelector('h1').textContent).to.equal('Hello world')
  expect(foo.$el.querySelector('p').textContent).to.equal('custom value')
})

Reactivity

One of Vue’s greatest strengths is its ability to automatically propagate changes wherever they need to go when one change is made. For example, a computed property will be updated automatically when one of its dependent properties changes. Also, the component will re-render with new data when a change happens. So we need to be able to test the reactive outcomes of changes we make as well.

When testing computed properties, it’s a simple matter of checking to see if the computed property’s value reflects what it should be once a dependency changes:

it('computed property updates correctly', () => {
  const foo = mount(Foo)
  foo.who = 'universe'
  expect(foo.msg).to.equal('Hello universe')
})

Checking that those updates propagate to the rendered DOM is a bit more trouble, though. If we make a change to foo.who like we just did and then check the DOM, it’ll give us the same DOM output that it had when the foo was initialized, so the following tests will fail:

it('render proper DOM on changes', () => {
  const foo = mount(Foo, { propsData: { someProp: 'custom value' } })
  foo.who = 'universe'
  foo.someProp = 'really custom value'
  expect(foo.$el.querySelector('h1').textContent).to.equal('Hello universe') // FAIL!!
  expect(foo.$el.querySelector('p').textContent).to.equal('really custom value') // FAIL!!
})

This happens because Vue wisely renders asynchronously. This prevents it from blocking the JS thread, but it also allows the entire chain of reactive changes to finish taking place before it renders, so it doesn’t end up rendering multiple times. We can use Vue.nextTick and pass it a callback that will run once it finishes the next rendering cycle, which will allow us to test the DOM.

To allow us to test asynchronous functionality, we need to use a done parameter (you can use any name you want but “done” is pretty common) in the callback to it so we can tell our test runner when the test finishes.

it('render proper DOM on changes', (done) => { // <- Add `done` here
  const foo = mount(Foo, { propsData: { someProp: 'custom value' } })
  foo.who = 'universe'
  foo.someProp = 'really custom value'

  Vue.nextTick( () => {
    expect(foo.$el.querySelector('h1').textContent).to.equal('Hello universe')
    expect(foo.$el.querySelector('p').textContent).to.equal('really custom value')
    done() // Call `done` to say we're done
  })
})

Alternatively, you can return a Promise if your test runner supports it (I believe all latest releases of test runners do). And if you’re using Promises, you make it easier to read by using async functions with async and await. First, we’ll need to convert nextTick to use promises though by creating a helper function:

function nextVueTick() {
  return new Promise((res) => Vue.nextTick(res))
}

Now we can convert our previous test to look like this:

it('render proper DOM on changes', async () => { // <- remove `done` and add `async`
  const foo = mount(Foo, { propsData: { someProp: 'custom value' } })
  foo.who = 'universe'
  foo.someProp = 'really custom value'

  await nextVueTick() // <- use `await` instead of a callback

  expect(foo.$el.querySelector('h1').textContent).to.equal('Hello universe')
  expect(foo.$el.querySelector('p').textContent).to.equal('really custom value')
})

If you're using Jest or another testing framework that supports snapshots, the simplest way to test the DOM is simply to use foo.$el.outerHTML and compare it to a snapshot. That ensures you're able to test the entire rendered DOM rather than picking and choosing bits to check.

Spying on Functions

We can also use spies to make sure that certain functions are called:

it('call `giveZero` on click', () => {
  sinon.spy(Foo.methods, 'giveZero') // <- spy on `giveZero`
  const foo = mount(Foo, { propsData: { someProp: 'custom value' } })
  
  foo.$el.dispatchEvent(new Event('click')) // trigger event that will call giveZero
  
  expect(Foo.methods.giveZero.called).to.equal(true)
  Foo.methods.giveZero.restore() // Remove the spy
})

Note that we create the spy on the component definition rather than on the instance in this case. This is because when the instance is created, the click handler will have a reference to the method that was already defined. If we override it with a spy on the instance, it will never be called because the spy won't get registered as the click handler. Also, note that we're checking Foo.methods.giveZero.called rather than foo.giveZero.called. This is because when an instance is created, Vue will wrap giveZero, so foo.giveZero.called is undefined. Calling Foo.methods.giveZero.restore() will return giveZero to its original function and will prevent other tests from using the spy.

We can also use spies to check on watch functions, which are similar to DOM rendering in that they are asynchronous (there are ways to make them synchronous, but generally you won't be doing that). This means we'll need to bring back our nextVueTick helper:

it('watcher triggered when `who` changes', async () => {
  sinon.spy(Foo.watch, 'who') // <- spy on `who` watcher
  const foo = mount(Foo)

  foo.who = 'universe'
  await nextVueTick()

  expect(Foo.watch.who.called).to.equal(true)
  expect(foo.updates).to.equal(1)
  Foo.watch.who.restore() // Remove the spy
})

Conclusion

That's pretty much it. As you can see, just about everything about Vue components is testable without requiring too much finagling, so Vue's testability should not prevent anyone from making it their choice of web UI framework.

Additional Help

If that's not simple enough for you, then maybe you should check out one of the following libraries that are designed to simplify some of the aspects around testing Vue components.

Avoriaz: This library simplifies mounting, along with being able to pass more options in during mounting. It wraps components to help make DOM traversal and event triggering simpler. Avoriaz can also do shallow rendering so the rendered DOM isn't dependent on child components. With this in place, you won't need to update tests for parent components when the child component is changed. Finally, it offers a method to synchronously re-render, but asynchronous watchers still need to use Vue.nextTick.

vue-test-utils: This library is officially supported by the Vue development team, but currently it's only in beta. It took heavy inspiration from Avoriaz, but is looking to expand its capabilities as well as make improvements to the API.