Plugin Interface
While it is recommended that plugins subclass nose.plugins.Plugin, the only requirements for a plugin are that it implement the methods options(self, parser, env) and configure(self, options, conf), and have the attributes enabled, name and score.
Plugins may implement any or all of the methods documented below. Please note that they must not subclass IPluginInterface; IPluginInterface is a only description of the plugin API.
When plugins are called, the first plugin that implements a method and returns a non-None value wins, and plugin processing ends. The exceptions to this are methods marked as generative or chainable. generative methods combine the output of all plugins that respond with an iterable into a single flattened iterable response (a generator, really). chainable methods pass the results of calling plugin A as the input to plugin B, where the positions in the chain are determined by the plugin sort order, which is in order by score descending.
In general, plugin methods correspond directly to methods of nose.selector.Selector, nose.loader.TestLoader and nose.result.TextTestResult are called by those methods when they are called. In some cases, the plugin hook doesn't neatly match the method in which it is called; for those, the documentation for the hook will tell you where in the test process it is called.
Plugin hooks fall into four broad categories: selecting and loading tests, handling errors raised by tests, preparing objects used in the testing process, and watching and reporting on test results.
Selecting and loading tests
To alter test selection behavior, implement any necessary want* methods as outlined below. Keep in mind, though, that when your plugin returns True from a want* method, you will send the requested object through the normal test collection process. If the object represents something from which normal tests can't be collected, you must also implement a loader method to load the tests.
Examples:
- The builtin doctests plugin implements wantFile to enable loading of doctests from files that are not python modules. It also implements loadTestsFromModule to load doctests from python modules, and loadTestsFromFile to load tests from the non-module files selected by wantFile.
- The builtin attrib plugin implements wantFunction and wantMethod so that it can reject tests that don't match the specified attributes.
Handling errors
To alter error handling behavior -- for instance to catch a certain class of exception and handle it differently from the normal error or failure handling -- you should subclass ErrorClassPlugin. See the documentation for ErrorClassPlugin for more details.
Examples:
- The builtin skip and deprecated plugins are ErrorClass plugins.
Preparing test objects
To alter, get a handle on, or replace test framework objects such as the loader, result, runner, and test cases, use the appropriate prepare methods. The simplest reason to use prepare is if you need to use an object yourself. For example, the isolate plugin implements prepareTestLoader so that it can use the test loader later on to load tests. If you return a value from a prepare method, that value will be used in place of the loader, result, runner or test case, respectively. When replacing test cases, be aware that you are replacing the entire test case -- including the whole run(result) method of the unittest.TestCase -- so if you want normal unittest test result reporting, you must implement the same calls to result as unittest.TestCase.run.
Examples:
- The builtin isolate plugin implements prepareTestLoader but does not replace the test loader.
- The builtin profile plugin implements prepareTest and does replace the top-level test case by returning the case wrapped in the profiler function.
Watching or reporting on tests
To record information about tests or other modules imported during the testing process, output additional reports, or entirely change test report output, implement any of the methods outlined below that correspond to TextTestResult methods.
Examples:
- The builtin cover plugin implements begin and report to capture and report code coverage metrics for all or selected modules loaded during testing.
- The builtin profile plugin implements begin, prepareTest and report to record and output profiling information. In this case, the plugin's prepareTest method constructs a function that runs the test through the hotshot profiler's runcall() method.
Plugin Interface Methods
Plugin interface methods are described below, in alphabetical
order. Methods that are new in
this release are highlighted.
Deprecated methods are struck through. Methods that
may be generators are labeled with the icon . Methods that are chainable (that is, the
results of calling the method on one plugin are passed as
input to the next plugin) are labeled with the icon
.
Called when a deprecated test is seen. DO NOT return a value unless you want to stop other plugins from seeing the deprecated test.
Note
DEPRECATED -- check error class in addError instead
Parameters: |
|
---|
Called when a test raises an uncaught exception. DO NOT return a value unless you want to stop other plugins from seeing that the test has raised an error.
Parameters: |
|
---|
Called when a test fails. DO NOT return a value unless you want to stop other plugins from seeing that the test has failed.
Parameters: |
|
---|
Called to allow plugin to register command line options with the parser.
Do not return a value from this method unless you want to stop all other plugins from setting their options.
Note
DEPRECATED -- implement options instead.
Called when a test is skipped. DO NOT return a value unless you want to stop other plugins from seeing the skipped test.
Note
DEPRECATED -- check error class in addError instead
Parameters: |
|
---|
Called when a test passes. DO NOT return a value unless you want to stop other plugins from seeing the passing test.
Parameters: |
|
---|
Called to allow plugin to register command line options with the parser.
Do not return a value from this method unless you want to stop all other plugins from setting their options.
Note
DEPRECATED -- implement options instead.
Called after a context (generally a module) has been lazy-loaded, imported, setup, had its tests loaded and executed, and torn down.
Called after all tests have been loaded from directory at path and run.
Parameters: |
|
---|
Called after module is imported from filename. afterImport is called even if the import failed.
Parameters: |
|
---|
Called after the test has been run and the result recorded (after stopTest).
Parameters: |
|
---|
Called before a context (generally a module) is examined. Since the context is not yet loaded, plugins don't get to know what the context is; so any context operations should use a stack that is pushed in beforeContext and popped in afterContext to ensure they operate symmetrically.
beforeContext and afterContext are mainly useful for tracking and restoring global state around possible changes from within a context, whatever the context may be. If you need to operate on contexts themselves, see startContext and stopContext, which are passed the context in question, but are called after it has been loaded (imported in the module case).
Called before tests are loaded from directory at path.
Parameters: |
|
---|
Called before module is imported from filename.
Parameters: |
|
---|
Called before the test is run (before startTest).
Parameters: |
|
---|
Called before any tests are collected or run. Use this to perform any setup needed before testing begins.
Called after the command line has been parsed, with the parsed options and the config container. Here, implement any config storage or changes to state or operation that are set by command line options.
Do not return a value from this method unless you want to stop all other plugins from being configured.
Return a test description. Called by nose.case.Test.shortDescription.
Parameters: |
|
---|
Called after all report output, including output from all plugins, has been sent to the stream. Use this to print final test results or perform final cleanup. Return None to allow other plugins to continue printing, any other value to stop them.
Note
When tests are run under a test runner other than nose.core.TextTestRunner, for example when tests are run via python setup.py test, this method may be called before the default report output is sent.
Called in result.addError, before plugin.addError. If you want to replace or modify the error tuple, return a new error tuple.
Parameters: |
|
---|
Called in result.addFailure, before plugin.addFailure. If you want to replace or modify the error tuple, return a new error tuple. Since this method is chainable, you must return the test as well, so you you'll return something like:
return (test, err)
Parameters: |
|
---|
Called on addError. To handle the error yourself and prevent normal error processing, return a true value.
Parameters: |
|
---|
Called on addFailure. To handle the failure yourself and prevent normal failure processing, return a true value.
Parameters: |
|
---|
Return iterable of tests from a directory. May be a generator. Each item returned must be a runnable unittest.TestCase (or subclass) instance or suite instance. Return None if your plugin cannot collect any tests from directory.
Parameters: |
|
---|
Return tests in this file. Return None if you are not interested in loading any tests, or an iterable if you are and can load some. May be a generator. If you are interested in loading tests from the file and encounter no errors, but find no tests, yield False or return [False].
Note
This method replaces loadTestsFromPath from the 0.9 API.
Parameters: |
|
---|
Return iterable of tests in a module. May be a generator. Each item returned must be a runnable unittest.TestCase (or subclass) instance. Return None if your plugin cannot collect any tests from module.
Parameters: |
|
---|
Return tests in this file or module. Return None if you are not able to load any tests, or an iterable if you are. May be a generator.
Parameters: |
|
---|
Return a tuple of (tests loaded, remaining names). Return None if you are not able to load any tests. Multiple plugins may implement loadTestsFromNames; the remaining name list from each will be passed to the next as input.
Parameters: |
|
---|
Note
DEPRECATED -- use loadTestsFromFile instead
Return tests in this test case class. Return None if you are not able to load any tests, or an iterable if you are. May be a generator.
Parameters: |
|
---|
Return tests in this test class. Class will not be a unittest.TestCase subclass. Return None if you are not able to load any tests, an iterable if you are. May be a generator.
Parameters: |
|
---|
Given an object and its parent, return or yield one or more test cases. Each test must be a unittest.TestCase (or subclass) instance. This is called before default test loading to allow plugins to load an alternate test case or cases for an object. May be a generator.
Parameters: |
|
---|
Called to allow plugin to register command line options with the parser.
Do not return a value from this method unless you want to stop all other plugins from setting their options.
Called before the test is run by the test runner. Please note the article the in the previous sentence: prepareTest is called only once, and is passed the test case or test suite that the test runner will execute. It is not called for each individual test case. If you return a non-None value, that return value will be run as the test. Use this hook to wrap or decorate the test with another function. If you need to modify or wrap individual test cases, use prepareTestCase instead.
Parameters: |
|
---|
Prepare or wrap an individual test case. Called before execution of the test. The test passed here is a nose.case.Test instance; the case to be executed is in the test attribute of the passed case. To modify the test to be run, you should return a callable that takes one argument (the test result object) -- it is recommended that you do not side-effect the nose.case.Test instance you have been passed.
Keep in mind that when you replace the test callable you are replacing the run() method of the test case -- including the exception handling and result calls, etc.
Parameters: |
|
---|
Called before tests are loaded. To replace the test loader, return a test loader. To allow other plugins to process the test loader, return None. Only one plugin may replace the test loader. Only valid when using nose.TestProgram.
Parameters: |
|
---|
Called before the first test is run. To use a different test result handler for all tests than the given result, return a test result handler. NOTE however that this handler will only be seen by tests, that is, inside of the result proxy system. The TestRunner and TestProgram -- whether nose's or other -- will continue to see the original result handler. For this reason, it is usually better to monkeypatch the result (for instance, if you want to handle some exceptions in a unique way). Only one plugin may replace the result, but many may monkeypatch it. If you want to monkeypatch and stop other plugins from doing so, monkeypatch and return the patched result.
Parameters: |
|
---|
Called before tests are run. To replace the test runner, return a test runner. To allow other plugins to process the test runner, return None. Only valid when using nose.TestProgram.
Parameters: |
|
---|
Called after all error output has been printed. Print your plugin's report to the provided stream. Return None to allow other plugins to print reports, any other value to stop them.
Parameters: |
|
---|
Called before test output begins. To direct test output to a new stream, return a stream object, which must implement a write(msg) method. If you only want to note the stream, not capture or redirect it, then return None.
Parameters: |
|
---|
Called before context setup and the running of tests in the context. Note that tests have already been loaded from the context before this call.
Parameters: |
|
---|
Called before each test is run. DO NOT return a value unless you want to stop other plugins from seeing the test start.
Parameters: |
|
---|
Called after the tests in a context have run and the context has been torn down.
Parameters: |
|
---|
Called after each test is run. DO NOT return a value unless you want to stop other plugins from seeing that the test has stopped.
Parameters: |
|
---|
Return a short test name. Called by nose.case.Test.__str__.
Parameters: |
|
---|
Return true if you want the main test selector to collect tests from this class, false if you don't, and None if you don't care.
Parameters: |
|
---|
Return true if you want test collection to descend into this directory, false if you do not, and None if you don't care.
Parameters: |
|
---|
Return true if you want to collect tests from this file, false if you do not and None if you don't care.
Change from 0.9: The optional package parameter is no longer passed.
Parameters: |
|
---|
Return true to collect this function as a test, false to prevent it from being collected, and None if you don't care.
Parameters: |
|
---|
Return true to collect this method as a test, false to prevent it from being collected, and None if you don't care.
Parameters: |
|
---|
Return true if you want to collection to descend into this module, false to prevent the collector from descending into the module, and None if you don't care.
Parameters: |
|
---|
Note
DEPRECATED -- this method will not be called, it has been folded into wantModule.