titaniumbunker.com

Evil geniuses and world domination are 2 of our goals... we also like Dr Who

Unit testing Javascript

no comment

In previous posts I talked about unit testing an MVC controller for a home project I am working on.  A couple of days ago I started trying to get JavaScript unit testing implemented within the build system so that I could unit test any JavaScript that I included in my project.  For those who just want to look at the code, then feel free – the code for this is located on GitHub here.

So what’s unit testing then?

Well imagine I wrote a function that performs a series of calculations on some numbers.  Unit testing is testing that smallest part of the application. I can call this function with known parameters and check that I get expected outcomes.  Ideally a unit test should be quick to run, and should be capable of being run on any server.  In order to be totally unit testable, the code should be capable of being tested in isolation.  This allows a build server – Like Visual Studio Team Services, Jenkins or Bamboo – to run a collection of unit tests when code is checked into the source code repository. Unit tests that are automatic are repeatable and therefore quick and easy to run.

Can you unit test JavaScript?

Well – Yes.  Unit testing is capable for any system where you can write code which executes the relevant components of the system – so unit testing is supported in Java, and Python, Ruby and a bunch of other languages.

Visual Studio has some cool tools built in to support MS Build – it is possible to run MSTest tests from within Visual Studio.  This is great as it allows a developer to confirm that they haven’t broken anything.  Due to a historical experience, I tend to build my tests using Nunit but these are equally well supported in Visual Studio with some additional extensions.  By default the test explorer is unaware of the make-up of an Nunit test.  Adding an Nunit test adapter, enables Visual Studio to suddenly understand what is required of an Nunit test.  This adapter can be installed either as an extension to Visual Studio, or into the project itself.  I my test project I am going to install it into the project.  Infact I have installed 2 test adapters into this projet, the Nunit test adapter, and a QUnit test adapter.

Qunit?

In the same way that Nunit is a testing framework for .net, Qunit is a testing framework for Javascript.  Iinstalling this means we can now write javascript files that contain tests for javascript files that can be executed by the Qunit framework.To start out I created a simple javascript test script that will always pass

// Look at this page : https://github.com/alexbeletsky/rest.mvc.example/blob/master/src/Web/Scripts/Tests/api/tests.api.js - 
// might offer interesting approach to instantiating Ajax calls to REST API

QUnit.test("hello test", function (assert) {
    assert.ok(1 == "1", "Passed!");
});

QUnit.test("Get All Products", function (assert)
{
    function square(x) {
        return x * x;
    }

    var result = square(2);

    assert.equal(result, 4, "square(2) equals 4");

});

So now I could look at getting tests to run.  In the same way that I installed a test runner for Nunit to support running Nunit tests through visual studio, I needed to install a similar test runner for Qunit – and found one called Chutzpah.

Chutzpah can be installed as an extension through Visual Studio – Chutzpah has an extension for visual studio, and a test runner, so I’ll install the test adapter for Visual Studio, and the test runner into the project.

Chutzpah JavaScript Test Runner extension

Chutzpah JavaScript Test Runner extension

 Now all these extensions have been installed it should be possible to run all the unit tests from Visual Studio.
Nunit and JSunit tests together in Visual Studio

Nunit and JSunit tests together in Visual Studio

Setting Up CI

These instructions are for Visual Studio Team Services – however they should be equally applicable to other build systems.

  1. Create an account on Visual Systems Team Services – it might be possible to re-use an existing account -but for this example I’m going to create a new account.

    SettingupVisualStudioTeamSystems1

    Creating a Visual Studio Team Services Account

  2. The Visual Studio Team Services wizard prompts to create a project – I just close this screen as my code already exists on a GitHub repository.
    SettingupVisualStudioTeamSystems2

    Just ignore the wizard – I’ll bet setting up my project in the next steps.

    In terms of setting up source control, this isn’t a requirement because this project uses GitHub as source control. It is equally possible to set up a Visual Studio Team Services system for version control – however many of the Microsoft projects use GitHub internally, and it seems to be the Version control system with the most mind share at the moment. Click on close button to close the wizard. Now let’s set up a build process for our project.

  3. Click Build to get to the build menu, and now click on the green plus to add a new build.
    SettingupVisualStudioTeamSystems3

    Visual Studio Team Services – Build Screen

     The wizard here will automatically set up most of the options we need.
    • Click on Visual Studio

      Setting up a Visual Studio Build Project

      Setting up a Visual Studio Build Project

    • Click on GitHub
    • Click on Continuous Integration (build whenever this branch is updated)
    • Click Create
      Visual Studio Team Services - picking your Version Control System

      Visual Studio Team Services – locating your code

       

    • Visual studio Build System now shows the build plan.
      Visual Studio Build job - Note that Repository is highlighted in red

      Visual Studio Build set up

      Notice that the Repository option across the top is red.  Although we’ve told Visual Studio Team Servuces to use GitHub, we haven’t told it which repository to use.  Let’s solve that now.

Connecting Visual Studio Team Services to GitHub

GitHub offers 2 mechanisms for connecting. An easier “let Visual Studio Team Services pretend to be me” approach – which perhaps is not the most secure, and a more secure “Use a access token that I have created”.  Both of these approaches are discussed here.  For this step I’ll need a Personal Access Token.

  • Click on Repository.  There are no connections, available so click on Manage,
  • From the Services, click “New Service Endpoint”, and select GitHub.

    Setting Visual Studio Team Systems with GitHub Personal Access Token

    Connecting Visual Studio Team Services to GitHub using a Personal Access Token

  • Select Personal Access token
  • Enter the Personal Access token from GitHub – use the copy icon to copy the token to your clipboard
  • Give your connection a name
  • Click on OK.
  • Close the page and co back to your build definition.
  • Click the Refresh icon.

The rest of the repository information is filled in.  Click save to save your build definition, give it a suitable name and your build plan will appear in the left hand menu.

Let’s give it a go…  Click on the Queue build.

SettingupVisualStudioTeamSystems9

The build was successful, but the tests didn’t run… What gives?

The tests failed to run because visual studio was unaware that there were additional test runners.  We need to make a slight modification to the testing step.  If we had used MS test these would be supported automatically.  Fortunately the change needed is simple enough.

Modify the existing Visual Studio Test build step

The existing Visual Studio Test build step needs to be modified to instruct the build system as to where any custom test adapters can be found.   Path to Custom Test Adapters should be set to:

$(Build.SourcesDirectory)\packages

This change will allow the Nunit unit tests to operate.    This folder equates to the Nuget packages folder for the solution.  Queuing the build should now produce some a test result.

Adding a Visual Studio Test Build Step for JavaScript testing.

Now that the NUnit testing is processing, the build needs to be modified to process the JavaScript unit testing.  JS unit tests are stored in .JS files, so rather than looking for assemblies, the unit test should instead look for *.test.js files.  To accomplish this, add a new build step – a Test Assemblies step which should execute at approximately around the area where the existing unit testing is performed.  if should be set up as follows :

Javascipt Unit Test Settings

Javascipt Unit Test Settings

Running the build should now show 3 tests passing – 2 JS tests and 1 Nunit test.

Unit test results, covering both Nunit and Qunit tests

Unit test results, covering both Nunit and Qunit tests

Running the tests from the command line

So why would we want to run the tests from the command line?  Well this allows a build environment – like Bamboo or Jenkins  – environments where Visual Studio Test support isn’t necessarily supported – to execute the tests as part of the tests and receive the test results. Executing these tests can be done though the command line tool vstest.console.exe.  One of the flags for running the tests in this manner is TestAdapterPath – which informs the vstest.console.exe where to use additional test adapters.  Pointing vstest.console.exe at the nuget packages folder allows the application to ‘carry it’s own baggage’, regardless of the system it is being built on.  This is the same concept that the “path to custom Test Adapters” provides when executing tests Visual Studio Team Services.  With Nuget package restore, these dependencies will be automatically restored, but for an environment where nuget package restore is not a possibility (corporate firewalls for example) the nuget package folder can be checked into version control and will therefore be available when the source code is checked out.

Depending on your build environment, system environment variables can be used to ensure that the paths for the test object locations, the custom test adapters location can be set without needing to hard card any paths, allowing the potential re-use of these code libraries.

cd 'C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow'
& ".\vstest.console.exe" /TestAdapterpath:"PATH\TO\packages" "PATH\TO\JAVSCRIPT\TEST\Javascript.test.js" "PATH\TO\NUNIT\DLLS\UnitTestsLibrary.test.dll"

vstest.console.exe is based on the VS12 environment variable VS120COMNTOOLS, which typically points to C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\Tools\.

Running a Bamboo Job

A sample Bamboo script can be found here :

try{
cls
cd ..
echo "TESTING"
echo "======="
#Get-ChildItem Env:
    $a = @("bamboo_working_directory","bamboo_repository_revision_number","bamboo_buildNumber")

    for ($i=0; $i -lt $a.length; $i++) 
    {
        $env= ($a[$i]) 
        if (!(get-item env:$env)  )
        {
            throw [System.ArgumentNullException] "$env not found."
        }
    }
    # Calculate paths.
    $ScriptDir = Split-Path $script:MyInvocation.MyCommand.Path
    $scriptPath = (get-item $ScriptDir ).parent.FullName
    $testResultsPath = ${scriptPath} +  "\TestResults"
    $testAdapterPath = ${env:bamboo_working_directory} + "\packages"
    $javascriptTest = ${env:bamboo_working_directory} + "\UnitTestLibrary\Javascript.test.js"
    $nunitTest = ${env:bamboo_working_directory} + "\UnitTestLibrary\bin\Debug\UnitTestsLibrary.test.dll"
    $vsTest = (Split-Path -Path ${env:VS120COMNTOOLS} -Parent) + "\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe"


    cd $env:bamboo_working_directory

    echo "Configuration  : "
    echo ("Current Directory         " + $ScriptDir)
    echo ("Current Directory         " + $scriptPath)
    echo ("Test Results Directory    " + $testResultsPath)
    echo ("testAdapterPath           " + $testAdapterPath)
    echo ("javascriptTest            " + $javascriptTest)
    echo ("nunitTest                 " + $nunitTest)
    echo ("vsTest                    " + $vsTest)

    echo " "
    echo "-- Clear out old TRX files"
    Get-ChildItem -Path $testResultsPath -Include *.* -File -Recurse | foreach { $_.Delete()}


    & $vsTest /logger:trx /TestAdapterpath:"$testAdapterPath" /ListExecutors /UseVsixExtensions:true 
    & $vsTest /logger:trx /TestAdapterpath:"$testAdapterPath" "$javascriptTest" "$nunitTest"

    echo "-- Find Newest Results"
    $latest = ${testResultsPath} + "\"+(Get-ChildItem -Path $testResultsPath | Sort-Object LastAccessTime -Descending | Select-Object -First 1)
    $newName = ${testResultsPath} + "\TestResults-Rev_${env:bamboo_repository_revision_number}-Build_${env:bamboo_buildNumber}.trx"

    echo $latest
    echo $newName

    Rename-Item $latest $newName


    echo "-- END OF TESTS --"
  
} 
catch {
  echo "===="
  echo $_.Exception|format-list -force
  echo "===="
}

Major points for this script:

  • Script checks that expected environment variables are set – These are Bamboo specific variables.
  • Scripts constructs project paths based on environment variables
  • Script will eventually scan for *.test.js files, and *.test.dll files, rather than using hardcoded script names
  • Script clears previous test execution result records
  • Script executes all tests, creating a test result file
  • Script renames test result file using bamboo_repository_revision_number and bamboo_buildNumber
    • This allows a later step to find test results based on a standard name

 

Future Improvements

  • Investigate the possibility of altering the Bamboo script for Jenkins build systems
  • Investigate code coverage metrics for JavaScript

 

Tags: ,

Comments are closed.



Categories

Archives

Tags