wiki:RunningUnitTests

Version 33 (modified by jon, 6 years ago) (diff)

--

Unit Tests

== Running Unit Tests ==

There are currently five products with unit tests:

  • collective.contentlicensing
  • collective.plonebookmarklets
  • collective.searchandreplace
  • collective.imstransport
  • enpraxis.educommons

To run the tests, enter the name of the product name with the -s flag:

Example:
./bin/instance test -s collective.contentlicensing

Additionally, you can specify a specific test using the -t flag:

Example:
./bin/instance test -s collective.contentlicensing -t test_cite_resource

To run tests for all the products in developtment (including collective.zipfiletransport and enpraxis.leftskin that doesn't have any tests at this point but will eventually have):

./bin/test

Location of Tests

Tests are located in the tests directory of each product.

Scope of Tests

The unit tests for ContentLicensing, PloneBookMarklets, SearchAndReplace, and IMSTransport are specific to that product.

The unit test for eduCommons require these other products as dependencies.

Documentation Tests

Doctests for each product have '.txt' extensions and are run from the test_doctests module in each test folder.

Testing Technologies

A few technologies have aided and will continue to aid unit test development for eduCommons:

  • testrecorder - A plone product that can be added to a plone instance and records actions taken, outputting a functional test.
  • Mocker Objects - Provides a mocker object that can be used in unit tests to "train" input parameters.
  • Plone Dummy Objects - Provided by the PloneTestCase?, Dummy objects are similar to mocker objects but require explicitly setting functions and parameters.

Test Coverage

An eduCommons coverage report shows a need for additional unit tests for folder listing views (foldercontents.py and summarycontents.py), many of the event handlers (eventHandlers.py), some of the browser views, and the migration modules. In addition, current reports do not show which python files are not being tested at all. Only percentage of files being tested are displayed. Additional reporting is required in order to give a better assessment of test coverage.

Also, is should be noted that there are two products that don't have any unit tests:

Furthermore, the tests for collective.imstransport fail because, when placed in the buildout, the eventHandlers for eduCommons product override the imstransport handlers. Because a large part of this release will be focused on the refactoring of IMS, unit test development will be delayed for collective.imstransport until later stages of product development.

The buildout includes tools to generate test coverage reports. To generate the report run:

./bin/test-coverage

That will generate output in the 'coverage' directory of the buildout.

In addition, z3c.coverage is used to generate HTML reports on test coverage. To generate those reports run the following command after running the 'test-coverage' one:

./bin/coverage-report

This uses the 'coverage' directory as input and generates the report in 'coverage/report'. Then point your web browser to file:///path/to/buildout/coverage/report/all.html to access the report.

Updated Coverage Reports Jan. 7 2009:

Reviews of Unit Tests