Package zope :: Package testing :: Module doctest
[show private | hide private]
[frames | no frames]

Module zope.testing.doctest

Module doctest -- a framework for running examples in docstrings.

In simplest use, end each module M to be tested with:

def _test():
import doctest doctest.testmod()
if __name__ == "__main__":
_test()

Then running the module as a script will cause the examples in the docstrings to get executed and verified:

python M.py

This won't display anything unless an example fails, in which case the failing example(s) and the cause(s) of the failure(s) are printed to stdout (why not stderr? because stderr is a lame hack <0.2 wink>), and the final line of output is "Test failed.".

Run it with the -v switch instead:

python M.py -v

and a detailed report of all examples tried is printed to stdout, along with assorted summaries at the end.

You can force verbose mode by passing "verbose=True" to testmod, or prohibit it by passing "verbose=False". In either of those cases, sys.argv is not examined by testmod.

There are a variety of other ways to run doctests, including integration with the unittest framework, and support for running non-Python text files containing doctests. There are also many ways to override parts of doctest's default behaviors. See the Library Reference Manual for details.


Classes
DebugRunner Run doc tests but raise an exception as soon as there is a failure.
DocTest A collection of doctest examples that should be run in a single namespace.
DocTestFinder A class used to extract the DocTests that are relevant to a given object, from its docstring and the docstrings of its contained objects.
DocTestParser A class used to parse strings containing doctest examples.
DocTestRunner A class used to run DocTest test cases, and accumulate statistics.
Example A single doctest example, consisting of source code and expected output.
OutputChecker A class used to check the whether the actual output from a doctest example matches the expected output.
Tester  
_OutputRedirectingPdb A specialized version of the python debugger that redirects stdout to a given stream when interacting with the user.
_SpoofOut  
_TestClass A pointless class, for sanity-checking of docstring testing.
DocFileCase  
DocTestCase  

Exceptions
DocTestFailure A DocTest example has failed in debugging mode.
UnexpectedException A DocTest example has encountered an unexpected exception

Function Summary
  debug(module, name, pm)
Debug a single doctest docstring.
  debug_src(src, pm, globs)
Debug a single doctest docstring, in argument src'
  DocFileSuite(*paths, **kw)
A unittest suite for one or more doctest files.
  DocTestSuite(module, globs, extraglobs, test_finder, **options)
Convert doctest tests for a module to a unittest test suite.
  is_private(prefix, base)
prefix, base -> true iff name prefix + "." + base is "private".
  register_optionflag(name)
  run_docstring_examples(f, globs, verbose, name, compileflags, optionflags)
Test examples in the given object's docstring (f), using globs as globals.
  script_from_examples(s)
Extract script from text with examples.
  set_unittest_reportflags(flags)
Sets the unittest option flags.
  testfile(filename, module_relative, name, package, globs, verbose, report, optionflags, extraglobs, raise_on_error, parser)
Test examples in the given file.
  testmod(m, name, globs, verbose, isprivate, report, optionflags, extraglobs, raise_on_error, exclude_empty)
report=True, optionflags=0, extraglobs=None, raise_on_error=False, exclude_empty=False
  testsource(module, name)
Extract the test sources from a doctest docstring as a script.
  _comment_line(line)
Return a commented form of the given line
  _ellipsis_match(want, got)
Essentially the only subtle case: >>> _ellipsis_match('aa...aa', 'aaa') False
  _exception_traceback(exc_info)
Return a string containing a traceback message for the given exc_info tuple (as returned by sys.exc_info()).
  _extract_future_flags(globs)
Return the compiler-flags associated with the future features that have been imported into the given namespace (globs).
  _indent(s, indent)
Add the given number of space characters to the beginning every non-blank line in s, and return the result.
  _module_relative_path(module, path)
  _normalize_module(module, depth)
If module is a module, then return module.
  _test()
  _unittest_count(docstring)
  debug_script(src, pm, globs)
Debug a test script.
  DocFileTest(path, module_relative, package, globs, parser, **options)

Variable Summary
int COMPARISON_FLAGS = 31                                                                    
int DONT_ACCEPT_BLANKLINE = 2                                                                     
int DONT_ACCEPT_TRUE_FOR_1 = 1                                                                     
int ELLIPSIS = 8                                                                     
int IGNORE_EXCEPTION_DETAIL = 16                                                                    
int NORMALIZE_WHITESPACE = 4                                                                     
int REPORT_CDIFF = 64                                                                    
int REPORT_NDIFF = 128                                                                   
int REPORT_ONLY_FIRST_FAILURE = 256                                                                   
int REPORT_UDIFF = 32                                                                    
int REPORTING_FLAGS = 480                                                                   
dict __test__ = {'string': '\n                      Example o...
SRE_Pattern _para_re = \s*\n\s*\n\s*
int _unittest_reportflags = 0                                                                     
str BLANKLINE_MARKER = '<BLANKLINE>'
str ELLIPSIS_MARKER = '...'
NoneType master = None                                                                  
dict OPTIONFLAGS_BY_NAME = {'DONT_ACCEPT_BLANKLINE': 2, 'REPO...

Function Details

debug(module, name, pm=False)

Debug a single doctest docstring.

Provide the module (or dotted name of the module) containing the test to be debugged and the name (within the module) of the object with the docstring with tests to be debugged.

debug_src(src, pm=False, globs=None)

Debug a single doctest docstring, in argument src'

DocFileSuite(*paths, **kw)

A unittest suite for one or more doctest files.

The path to each doctest file is given as a string; the interpretation of that string depends on the keyword argument "module_relative".

A number of options may be provided as keyword arguments:

module_relative

If "module_relative" is True, then the given file paths are interpreted as os-independent module-relative paths. By default, these paths are relative to the calling module's directory; but if the "package" argument is specified, then they are relative to that package. To ensure os-independence, "filename" should use "/" characters to separate path segments, and may not be an absolute path (i.e., it may not begin with "/").

If "module_relative" is False, then the given file paths are interpreted as os-specific paths. These paths may be absolute or relative (to the current working directory).

package
A Python package or the name of a Python package whose directory should be used as the base directory for module relative paths. If "package" is not specified, then the calling module's directory is used as the base directory for module relative filenames. It is an error to specify "package" if "module_relative" is False.
setUp
A set-up function. This is called before running the tests in each file. The setUp function will be passed a DocTest object. The setUp function can access the test globals as the globs attribute of the test passed.
tearDown
A tear-down function. This is called after running the tests in each file. The tearDown function will be passed a DocTest object. The tearDown function can access the test globals as the globs attribute of the test passed.
globs
A dictionary containing initial global variables for the tests.
optionflags
A set of doctest option flags expressed as an integer.
parser
A DocTestParser (or subclass) that should be used to extract tests from the files.

DocTestSuite(module=None, globs=None, extraglobs=None, test_finder=None, **options)

Convert doctest tests for a module to a unittest test suite.

This converts each documentation string in a module that contains doctest tests to a unittest test case. If any of the tests in a doc string fail, then the test case fails. An exception is raised showing the name of the file containing the test and a (sometimes approximate) line number.

The module argument provides the module to be tested. The argument can be either a module or a module name.

If no argument is given, the calling module is used.

A number of options may be provided as keyword arguments:

setUp
A set-up function. This is called before running the tests in each file. The setUp function will be passed a DocTest object. The setUp function can access the test globals as the globs attribute of the test passed.
tearDown
A tear-down function. This is called after running the tests in each file. The tearDown function will be passed a DocTest object. The tearDown function can access the test globals as the globs attribute of the test passed.
globs
A dictionary containing initial global variables for the tests.
optionflags
A set of doctest option flags expressed as an integer.

is_private(prefix, base)

prefix, base -> true iff name prefix + "." + base is "private".

Prefix may be an empty string, and base does not contain a period. Prefix is ignored (although functions you write conforming to this protocol may make use of it). Return true iff base begins with an (at least one) underscore, but does not both begin and end with (at least) two underscores.

>>> is_private("a.b", "my_func")
False
>>> is_private("____", "_my_func")
True
>>> is_private("someclass", "__init__")
False
>>> is_private("sometypo", "__init_")
True
>>> is_private("x.y.z", "_")
True
>>> is_private("_x.y.z", "__")
False
>>> is_private("", "")  # senseless but consistent
False

run_docstring_examples(f, globs, verbose=False, name='NoName', compileflags=None, optionflags=0)

Test examples in the given object's docstring (f), using globs as globals. Optional argument name is used in failure messages. If the optional argument verbose is true, then generate output even if there are no failures.

compileflags gives the set of flags that should be used by the Python compiler when running the examples. If not specified, then it will default to the set of future-import flags that apply to globs.

Optional keyword arg optionflags specifies options for the testing and output. See the documentation for testmod for more information.

script_from_examples(s)

Extract script from text with examples.

Converts text with examples to a Python script. Example input is converted to regular code. Example output and all other words are converted to comments:

>>> text = '''
...       Here are examples of simple math.
...
...           Python has super accurate integer addition
...
...           >>> 2 + 2
...           5
...
...           And very friendly error messages:
...
...           >>> 1/0
...           To Infinity
...           And
...           Beyond
...
...           You can use logic if you want:
...
...           >>> if 0:
...           ...    blah
...           ...    blah
...           ...
...
...           Ho hum
...           '''
>>> print script_from_examples(text)
# Here are examples of simple math.
#
#     Python has super accurate integer addition
#
2 + 2
# Expected:
## 5
#
#     And very friendly error messages:
#
1/0
# Expected:
## To Infinity
## And
## Beyond
#
#     You can use logic if you want:
#
if 0:
   blah
   blah
#
#     Ho hum

set_unittest_reportflags(flags)

Sets the unittest option flags.

The old flag is returned so that a runner could restore the old value if it wished to:

>>> old = _unittest_reportflags
>>> set_unittest_reportflags(REPORT_NDIFF |
...                          REPORT_ONLY_FIRST_FAILURE) == old
True
>>> import doctest
>>> doctest._unittest_reportflags == (REPORT_NDIFF |
...                                   REPORT_ONLY_FIRST_FAILURE)
True

Only reporting flags can be set:

>>> set_unittest_reportflags(ELLIPSIS)
Traceback (most recent call last):
...
ValueError: ('Only reporting flags allowed', 8)
>>> set_unittest_reportflags(old) == (REPORT_NDIFF |
...                                   REPORT_ONLY_FIRST_FAILURE)
True

testfile(filename, module_relative=True, name=None, package=None, globs=None, verbose=None, report=True, optionflags=0, extraglobs=None, raise_on_error=False, parser=<zope.testing.doctest.DocTestParser instance at 0xb5f001ec>)

Test examples in the given file. Return (#failures, #tests).

Optional keyword arg "module_relative" specifies how filenames should be interpreted:

  • If "module_relative" is True (the default), then "filename"

    specifies a module-relative path. By default, this path is relative to the calling module's directory; but if the "package" argument is specified, then it is relative to that package. To ensure os-independence, "filename" should use "/" characters to separate path segments, and should not be an absolute path (i.e., it may not begin with "/").

  • If "module_relative" is False, then "filename" specifies an os-specific path. The path may be absolute or relative (to the current working directory).

Optional keyword arg "name" gives the name of the test; by default use the file's basename.

Optional keyword argument "package" is a Python package or the name of a Python package whose directory should be used as the base directory for a module relative filename. If no package is specified, then the calling module's directory is used as the base directory for module relative filenames. It is an error to specify "package" if "module_relative" is False.

Optional keyword arg "globs" gives a dict to be used as the globals when executing examples; by default, use {}. A copy of this dict is actually used for each docstring, so that each docstring's examples start with a clean slate.

Optional keyword arg "extraglobs" gives a dictionary that should be merged into the globals that are used to execute examples. By default, no extra globals are used.

Optional keyword arg "verbose" prints lots of stuff if true, prints only failures if false; by default, it's true iff "-v" is in sys.argv.

Optional keyword arg "report" prints a summary at the end when true, else prints nothing at the end. In verbose mode, the summary is detailed, else very brief (in fact, empty if all tests passed).

Optional keyword arg "optionflags" or's together module constants, and defaults to 0. Possible values (see the docs for details):

DONT_ACCEPT_TRUE_FOR_1 DONT_ACCEPT_BLANKLINE NORMALIZE_WHITESPACE ELLIPSIS IGNORE_EXCEPTION_DETAIL REPORT_UDIFF REPORT_CDIFF REPORT_NDIFF REPORT_ONLY_FIRST_FAILURE

Optional keyword arg "raise_on_error" raises an exception on the first unexpected exception or failure. This allows failures to be post-mortem debugged.

Optional keyword arg "parser" specifies a DocTestParser (or subclass) that should be used to extract tests from the files.

Advanced tomfoolery: testmod runs methods of a local instance of class doctest.Tester, then merges the results into (or creates) global Tester instance doctest.master. Methods of doctest.master can be called directly too, if you want to do something unusual. Passing report=0 to testmod is especially useful then, to delay displaying a summary. Invoke doctest.master.summarize(verbose) when you're done fiddling.

testmod(m=None, name=None, globs=None, verbose=None, isprivate=None, report=True, optionflags=0, extraglobs=None, raise_on_error=False, exclude_empty=False)

m=None, name=None, globs=None, verbose=None, isprivate=None,
report=True, optionflags=0, extraglobs=None, raise_on_error=False, exclude_empty=False

Test examples in docstrings in functions and classes reachable from module m (or the current module if m is not supplied), starting with m.__doc__. Unless isprivate is specified, private names are not skipped.

Also test examples reachable from dict m.__test__ if it exists and is not None. m.__test__ maps names to functions, classes and strings; function and class docstrings are tested even if the name is private; strings are tested directly, as if they were docstrings.

Return (#failures, #tests).

See doctest.__doc__ for an overview.

Optional keyword arg "name" gives the name of the module; by default use m.__name__.

Optional keyword arg "globs" gives a dict to be used as the globals when executing examples; by default, use m.__dict__. A copy of this dict is actually used for each docstring, so that each docstring's examples start with a clean slate.

Optional keyword arg "extraglobs" gives a dictionary that should be merged into the globals that are used to execute examples. By default, no extra globals are used. This is new in 2.4.

Optional keyword arg "verbose" prints lots of stuff if true, prints only failures if false; by default, it's true iff "-v" is in sys.argv.

Optional keyword arg "report" prints a summary at the end when true, else prints nothing at the end. In verbose mode, the summary is detailed, else very brief (in fact, empty if all tests passed).

Optional keyword arg "optionflags" or's together module constants, and defaults to 0. This is new in 2.3. Possible values (see the docs for details):

DONT_ACCEPT_TRUE_FOR_1 DONT_ACCEPT_BLANKLINE NORMALIZE_WHITESPACE ELLIPSIS IGNORE_EXCEPTION_DETAIL REPORT_UDIFF REPORT_CDIFF REPORT_NDIFF REPORT_ONLY_FIRST_FAILURE

Optional keyword arg "raise_on_error" raises an exception on the first unexpected exception or failure. This allows failures to be post-mortem debugged.

Deprecated in Python 2.4: Optional keyword arg "isprivate" specifies a function used to determine whether a name is private. The default function is treat all functions as public. Optionally, "isprivate" can be set to doctest.is_private to skip over functions marked as private using the underscore naming convention; see its docs for details.

Advanced tomfoolery: testmod runs methods of a local instance of class doctest.Tester, then merges the results into (or creates) global Tester instance doctest.master. Methods of doctest.master can be called directly too, if you want to do something unusual. Passing report=0 to testmod is especially useful then, to delay displaying a summary. Invoke doctest.master.summarize(verbose) when you're done fiddling.

testsource(module, name)

Extract the test sources from a doctest docstring as a script.

Provide the module (or dotted name of the module) containing the test to be debugged and the name (within the module) of the object with the doc string with tests to be debugged.

_comment_line(line)

Return a commented form of the given line

_ellipsis_match(want, got)

Essentially the only subtle case: >>> _ellipsis_match('aa...aa', 'aaa') False

_exception_traceback(exc_info)

Return a string containing a traceback message for the given exc_info tuple (as returned by sys.exc_info()).

_extract_future_flags(globs)

Return the compiler-flags associated with the future features that have been imported into the given namespace (globs).

_indent(s, indent=4)

Add the given number of space characters to the beginning every non-blank line in s, and return the result.

_normalize_module(module, depth=2)

Return the module specified by module. In particular:
  • If module is a module, then return module.
  • If module is a string, then import and return the module with that name.
  • If module is None, then return the calling module. The calling module is assumed to be the module of the stack frame at the given depth in the call stack.

debug_script(src, pm=False, globs=None)

Debug a test script. src is the script, as a string.


Variable Details

COMPARISON_FLAGS

Type:
int
Value:
31                                                                    

DONT_ACCEPT_BLANKLINE

Type:
int
Value:
2                                                                     

DONT_ACCEPT_TRUE_FOR_1

Type:
int
Value:
1                                                                     

ELLIPSIS

Type:
int
Value:
8                                                                     

IGNORE_EXCEPTION_DETAIL

Type:
int
Value:
16                                                                    

NORMALIZE_WHITESPACE

Type:
int
Value:
4                                                                     

REPORT_CDIFF

Type:
int
Value:
64                                                                    

REPORT_NDIFF

Type:
int
Value:
128                                                                   

REPORT_ONLY_FIRST_FAILURE

Type:
int
Value:
256                                                                   

REPORT_UDIFF

Type:
int
Value:
32                                                                    

REPORTING_FLAGS

Type:
int
Value:
480                                                                   

__test__

Type:
dict
Value:
{'_TestClass': <class zope.testing.doctest._TestClass at 0xb5eff5cc>,
 'blank lines': "\n                Blank lines can be marked with <BLA\
NKLINE>:\n                    >>> print 'foo\\n\\nbar\\n'\n           \
         foo\n                    <BLANKLINE>\n                    bar\
\n                    <BLANKLINE>\n            ",
 'bool-int equivalence': '\n                                    In 2.2\
, boolean expressions displayed\n                                    0\
 or 1.  By default, we still accept\n                                 \
...                                                                    

_para_re

Type:
SRE_Pattern
Value:
\s*\n\s*\n\s*                                                          

_unittest_reportflags

Type:
int
Value:
0                                                                     

BLANKLINE_MARKER

Type:
str
Value:
'<BLANKLINE>'                                                          

ELLIPSIS_MARKER

Type:
str
Value:
'...'                                                                  

master

Type:
NoneType
Value:
None                                                                  

OPTIONFLAGS_BY_NAME

Type:
dict
Value:
{'DONT_ACCEPT_BLANKLINE': 2,
 'DONT_ACCEPT_TRUE_FOR_1': 1,
 'ELLIPSIS': 8,
 'IGNORE_EXCEPTION_DETAIL': 16,
 'NORMALIZE_WHITESPACE': 4,
 'REPORT_CDIFF': 64,
 'REPORT_NDIFF': 128,
 'REPORT_ONLY_FIRST_FAILURE': 256,
...                                                                    

Generated by Epydoc 2.1 on Fri Jun 24 12:01:23 2005 http://epydoc.sf.net