jskit contains infrastructure and in particular a py.test plugin to enable running tests for JavaScript code inside browsers directly using py.test as the test driver. Running inside the browsers comes with some speed cost, on the other hand it means for example the code is tested against the real-world DOM implementations.
The approach also enables to write integration tests such that the JavaScript code is tested against server-side Python code mocked as necessary. Any server-side framework that can already be exposed through WSGI (or for which a subset of WSGI can be written to accommodate the jskit own needs) can play along.
jskit has also some support to run JavaScript tests from unittest.py based test suites.
jskit also contains code to help modularizing JavaScript code which can be used to describe and track dependencies dynamically during development and that can help resolving them statically when deploying/packaging.
Known supported browsers are Firefox, Internet Explorer >=7, and WebKit browsers.
jskit now supports both py.test 2.0 and late py.test 1.x.
jskit requires Python 2.6 or 2.7. It also uses MochiKit - of which it ships a version within itself for convenience - for its own working though in does not imposes its usage on tested code.
jskit was initially developed by Open End AB and is released under the MIT license.
For the simplest cases jskit allows to write tests simply as just JavaScript files. As an example one could have a jstest_example_any.js in a test directory in a project:
Tests = {
test_simple: function() {
ais(21*2, 42)
}
}
jstest_*.js is the globbing pattern used by jskit to recognize JavaScript test files. Each of these needs to contain a Tests object with test methods starting with test_. ais is one of the assertion functions predefined by jskit, it raises an exception causing the test to fail if the two arguments are not equal (in JavaScript == sense).
The suffix any separated by an underscore (_) in the test file name controls the kind of browsers the test should be run against, any is predefined by jskit to be
later we'll see how to define other groups of browsers.
To run the test we need to use py.test with the jskit plugin activated which happens if jskit in on the Python path (see also the py.test and plugin usage docs).
So running the test is just a matter of invoking py.test that has an installed jskit available on the Python path:
$ ../v/bin/py.test test/ inserting into sys.path: /u/pedronis/scratch/oejskit-play/v/lib/python2.5/site-packages ========================================= test session starts ========================================= python: platform linux2 -- Python 2.5.2 test object 1: /u/pedronis/scratch/oejskit-play/proj/test test/jstest_example_any.js . ====================================== 1 passed in 2.52 seconds =======================================
(concretely here I'm using a virtualenv with both the py.test and oejskit installed in it.)
Tests from different files with their Tests objects (or Python test classes are as we will explain later) are run isolated in IFRAMEs which will also display test results enumerating both assertions and tests, not just tests. Each JavaScript test_ method is run and its result reported by py.test as it would for Python tests. test_ methods from a Tests object are run in alphabetical order.
Normally one page is used for the whole test run with sections corresponding to the JavaScript test files or Python modules, links are progressively displayed on the upper right corner of the page to jump to the sections, this to help debugging the state of a failed test for example.
Tests of course need to be able to load and refer to tested code. Let's assume for example that JavaScript code lives under the static/js directory in the project, for example there could be a file ex.js there:
function ok() {
return "ok"
}
jskit way to refer to tested code is based on its modularity functionality, though for the simplest cases this is involve just providing enough information such that parts of the URL space can be mapped back to parts of the filesystem where JavaScript lives.
Adding the following to the conftest.py of the project:
class jstests_setup:
staticDirs = {
'/static': os.path.join(os.path.dirname(__file__), 'static')
}
jsRepos = ['/static']
tells jskit to serve files from the static directory (the value in the staticDirs mapping) for the static subtree of the URL space (the key).
The values in jsRepos list which prefixes of the URL space contain JavaScript, they play a bigger role when using the full modularity functionality, they are necessary for the simple case too though.
With this configuration we can write a test jstest_ok_any.js for the ok function in ex.js that looks like this:
OpenEnd.require("/static/js/ex.js")
Tests = {
test_ok: function() {
var res = ok()
ais(res, "ok")
}
}
With the information provided jskit can infer that ex.js is needed for the test and serve it.
Running all the tests now should give:
$ ../v/bin/py.test inserting into sys.path: /u/pedronis/scratch/oejskit-play/v/lib/python2.5/site-packages ========================================= test session starts ========================================= python: platform linux2 -- Python 2.5.2 test object 1: /u/pedronis/scratch/oejskit-play/proj test/jstest_example_any.js . test/jstest_ok_any.js . ====================================== 2 passed in 5.06 seconds =======================================
jskit predefines a set of functions to make assertions in tests, JavaScript not having an assertion statement at all
There's another predefined helper insertTestNode which will return a new DIV node at the end of test page styled with a border and labeled with the test name, the node can then be used to host DOM fragments to be manipulated by the test. If a test using this feature fails it should then be easy to identify the relevant DOM part and eyeball or inspect it with tools such as Firebug.
Given a failing test like (proj/fail/jstest_fail_any.js):
Tests = {
test_fail: function() {
aisDeeply([1].concat([2, 3]), [1,[2,3]])
}
}
this is part of the kind of output to be expected at least when the test is run against Firefox which provides proper traceback information:
...
> raise JsFailed(name, outcome['diag'])
E JsFailed: test_fail: FAILED: Structures begin differing at:
E got[1] = 2
E expected[1] = 2,3
E
E stack: Error()@:0
E ("isDeeply in test_fail in ?"," Structures begin differing at:
...
E ()@http://localhost:49900/test/jstest_fail_any.js:4
...
Notice that the line numbers and involved JavaScript files are presented:
E ()@http://localhost:49900/test/jstest_fail_any.js:4
On top of the predefined any, browser kind groupings against which to run tests can be defined that are meaningful to one's project. As we have seen these are used as suffixes for test files or as values for the later introduced jstests_browser_kind class attribute.
All is needed is to define a jstest_browser_specs mapping in the conftest.py file, with the kinds as keys and lists of browser names as values:
jstests_browser_specs = {
'supported': ['firefox', 'iexplore', 'safari'],
'extrafeatures': ['safari']
}
browser names themselves can also be used directly as kinds. Given the mapping jstests_basic_supported.js, jstests_extras_extrafeatures.js and in any case jstests_details_firefox.js or jstests_details_iexplore.js would be meaningful JavaScript test file names.
With the plugin activated a py.test command-line option --jstests-browser-spec=JSTESTS_BROWSER_SPEC can also be used to define browser kinds as in:
--jstests-browser-spec any=firefox,safari --jstests-browser-spec extra=safari
As we see here any itself can be redefined this way or via the mapping.
One central jskit feature is the ability to run JavaScript tests against server-side Python code setup through test code. For example one can add a test_integration.py:
from oejskit.testing import jstests_suite
import cgi
class TestIntegration(object):
jstests_browser_kind = 'any'
@jstests_suite('test_integration.js')
def integration_server_side(self):
def server(environ, start_response):
p = environ['PATH_INFO']
start_response('200 OK', [('content-type', 'text/plain')])
q = cgi.parse_qs(environ['QUERY_STRING'])
if p == '/add':
a, b = int(q['a'][0]), int(q['b'][0])
return [str(a+b)]
elif p == '/sub':
a, b = int(q['a'][0]), int(q['b'][0])
return [str(a-b)]
return []
return server
with a parallelly located test_integration.js:
OpenEnd.require("/static/js/get.js")
Tests = {
test_add: function() {
var res = http_get("/add?a=2&b=3")
ais(parseInt(res), 5)
},
test_sub: function() {
var res = http_get("/sub?a=6&b=2")
ais(parseInt(res), 4)
}
}
This test code will result in the tests in test_integration.js` to be run while the testing framework is serving the WSGI application server at the root of the URL space. The code in integration_server_side will be run once before the JavaScript tests in the suite will be run, in a sense it is akin to a setup method but funcargs can be requested in its arguments.
The tests will be run for each browser identified by the kind put in jstests_browser_kind. Both jstests_browser_kind and the decorator jstests_suite are required to achieve this behavior, jstests_browser_kind marks that the tests in the class need to be executed with a browser available to control.
get.js here defines http_get a simple function wrapping XMLHttpRequest t do synchronous GET requests.
Contained in the oejskit package there is a module browser.py which can be used as standalone script. The script can be run as a server that allows starting browsers on a remote machine, this means that the platforms for the python/server-side test code and the platforms for target browsers don't need to coincide. The script can be simply be copied as standalone file and it just needs a plain python installed without further dependencies to run. The script uses HMAC and a secret token to secure its access though it is still recommend to run it behind a firewall.
Setup servers to start browsers on the the relevant machine:
Z:\>c:\Python26\python.exe browser.py server 10010 JSTESTS_REMOTE_BROWSERS_TOKEN=eca424b610245337c80f32e3a08c50c6 serving browsers on 10010...
On the machine that will drive the tests set one environment variable with the secret token, and another that lists separated by spaces <hostname>:<port>:<browsers-separated-by-commas> for the browser servers:
$ export JSTESTS_REMOTE_BROWSERS_TOKEN=eca424b610245337c80f32e3a08c50c6 $ export JSTESTS_REMOTE_BROWSERS=bigboard:10010:iexplore $ ../v/bin/py.test --jstests-browser-spec any=iexplore,firefox inserting into sys.path: /u/pedronis/scratch/oejskit-play/v/lib/python2.5/site-packages ======================================================= test session starts ======================================================== python: platform linux2 -- Python 2.5.2 -- /u/pedronis/scratch/oejskit-play/v/bin/python test object 1: /u/pedronis/scratch/oejskit-play/proj test/jstest_example_any.js:1: proj.test.jstest_example_any.js[=iexplore][test_simple] PASS test/jstest_example_any.js:1: proj.test.jstest_example_any.js[=firefox][test_simple] PASS test/jstest_ok_any.js:1: proj.test.jstest_ok_any.js[=iexplore][test_ok] PASS test/jstest_ok_any.js:1: proj.test.jstest_ok_any.js[=firefox][test_ok] PASS test/test_integration.py:7: TestIntegration[=iexplore].integration_server_side[test_add] PASS test/test_integration.py:7: TestIntegration[=iexplore].integration_server_side[test_sub] PASS test/test_integration.py:7: TestIntegration[=firefox].integration_server_side[test_add] PASS test/test_integration.py:7: TestIntegration[=firefox].integration_server_side[test_sub] PASS ==================================================== 8 passed in 28.39 seconds =====================================================
Firefox is not listed in the environment variable, so a local one is started or reused.
There are also commands to shutdown browsers, respectively servers, they will also consider the environment variables to find their targets:
$ ../v/bin/python -m oejskit.browser cleanup iexplore $ ../v/bin/python -m oejskit.browser shutdown-servers
For convenience when setting up multiple machines there is the possibility to first generate a token and pass it in as an argument to the server command:
Z:\>c:\Python26\python.exe browser.py gentoken eca424b610245337c80f32e3a08c50c6 Z:\>c:\Python26\python.exe browser.py server 10010 eca424b610245337c80f32e3a08c50c6 serving browsers on 10010...
It is possible to give user-defined names for greater flexibility to browsers or browser command lines, for example to run tests against both Internet Explorer 7 and 8 one could run browser.py on a Windows machine with the latter this way:
Z:\>c:\Python26\python.exe browser.py gentoken eca424b610245337c80f32e3a08c50c6 Z:\>c:\Python26\python.exe browser.py server 10010 eca424b610245337c80f32e3a08c50c6 ie8=iexplore serving browsers on 10010...
and on other with the older IE7 with:
Y:\>c:\Python25\python.exe browser.py server 10010 eca424b610245337c80f32e3a08c50c6 ie7=iexplore serving browsers on 10010...
A series of definitions with user-defined-name=command-line can be specified after the main browser.py server parameters. user-defined-names can then be used as browser names and in browser specifications, like:
--jstests-browser-spec any=firefox,ie7,ie8
If it makes sense a full command line can be specified:
Y:\>c:\Python25\python.exe browser.py server 10010 eca424b610245337c80f32e3a08c50c6 "ff-testing=...\firefox.exe -P testing" ie7=iexplore
The oejskit package contains a unittest_support module whose content is a subclass of unittest.TestSuite which can be used to incorporate JavaScript tests with unittest.py tests. Typically one will make a per-project subclass of unittest_support.JSTestSuite, the parameters we previously described like staticDirs that go jstests_setup and the mapping jstests_browser_specs can be specified as attributes attached to such a subclass. An example usage would then be like:
class ProjectJSTestSuite(unittest_support.JSTestSuite):
jstests_browser_specs = {
'supported': ['firefox', 'iexplore', 'safari']
}
def ok_root():
def ok(environ, start_response):
start_response('200 OK', [('content-type', 'text/plain')])
return ['ok\n']
return ok
if __name__ == '__main__':
runner = unittest.TextTestRunner(verbosity=1)
all = unittest.TestSuite()
example_suite = ProjectJSTestSuite('jstest_example_supported.js')
integration_suite = ProjectJSTestSuite('test_integration_style.js',
root=ok_root,
browser_kind='supported')
all.addTest(example_suite)
all.addTest(integration_suite)
runner.run(all)
the suites are instantiated with a JavaScript test file which will be located relative to the Python test file, optionally a browser_kind and a factory root for e.g. a WSGI application (see also Integration testing) can be specified to serve for the test. If a browser_kind is not specified it will be extracted out of the JavaScript test file name.