Classes involved in doctesting¶
This module controls the various classes involved in doctesting.
AUTHORS:
David Roe (2012-03-27) – initial version, based on Robert Bradshaw’s code.
-
class
sage.doctest.control.
DocTestController
(options, args)¶ Bases:
sage.structure.sage_object.SageObject
This class controls doctesting of files.
After creating it with appropriate options, call the
run()
method to run the doctests.-
add_files
()¶ Checks for the flags ‘–all’ and ‘–new’.
For each one present, this function adds the appropriate directories and files to the todo list.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController sage: from sage.env import SAGE_SRC sage: import os sage: log_location = os.path.join(SAGE_TMP, 'control_dt_log.log') sage: DD = DocTestDefaults(all=True, logfile=log_location) sage: DC = DocTestController(DD, []) sage: DC.add_files() Doctesting entire Sage library. sage: os.path.join(SAGE_SRC, 'sage') in DC.files True
sage: DD = DocTestDefaults(new = True) sage: DC = DocTestController(DD, []) sage: DC.add_files() Doctesting ...
-
cleanup
(final=True)¶ Runs cleanup activities after actually running doctests.
In particular, saves the stats to disk and closes the logfile.
INPUT:
final
– whether to close the logfile
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController sage: from sage.env import SAGE_SRC sage: import os sage: dirname = os.path.join(SAGE_SRC, 'sage', 'rings', 'infinity.py') sage: DD = DocTestDefaults() sage: DC = DocTestController(DD, [dirname]) sage: DC.expand_files_into_sources() sage: DC.sources.sort(key=lambda s:s.basename) sage: for i, source in enumerate(DC.sources): ....: DC.stats[source.basename] = {'walltime': 0.1*(i+1)} ....: sage: DC.run() Running doctests with ID ... Doctesting 1 file. sage -t .../rings/infinity.py [... tests, ... s] ---------------------------------------------------------------------- All tests passed! ---------------------------------------------------------------------- Total time for all tests: ... seconds cpu time: ... seconds cumulative wall time: ... seconds 0 sage: DC.cleanup()
-
create_run_id
()¶ Creates the run id.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController sage: DC = DocTestController(DocTestDefaults(), []) sage: DC.create_run_id() Running doctests with ID ...
-
expand_files_into_sources
()¶ Expands
self.files
, which may include directories, into a list ofsage.doctest.FileDocTestSource
This function also handles the optional command line option.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController sage: from sage.env import SAGE_SRC sage: import os sage: dirname = os.path.join(SAGE_SRC, 'sage', 'doctest') sage: DD = DocTestDefaults(optional='all') sage: DC = DocTestController(DD, [dirname]) sage: DC.expand_files_into_sources() sage: len(DC.sources) 11 sage: DC.sources[0].options.optional True
sage: DD = DocTestDefaults(optional='magma,guava') sage: DC = DocTestController(DD, [dirname]) sage: DC.expand_files_into_sources() sage: all(t in DC.sources[0].options.optional for t in ['magma','guava']) True
We check that files are skipped appropriately:
sage: dirname = tmp_dir() sage: filename = os.path.join(dirname, 'not_tested.py') sage: with open(filename, 'w') as f: ....: _ = f.write("#"*80 + "\n\n\n\n## nodoctest\n sage: 1+1\n 4") sage: DC = DocTestController(DD, [dirname]) sage: DC.expand_files_into_sources() sage: DC.sources []
The directory
sage/doctest/tests
containsnodoctest.py
but the files should still be tested when that directory is explicitly given (as opposed to being recursed into):sage: DC = DocTestController(DD, [os.path.join(SAGE_SRC, 'sage', 'doctest', 'tests')]) sage: DC.expand_files_into_sources() sage: len(DC.sources) >= 10 True
-
filter_sources
()¶ EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController sage: from sage.env import SAGE_SRC sage: import os sage: dirname = os.path.join(SAGE_SRC, 'sage', 'doctest') sage: DD = DocTestDefaults(failed=True) sage: DC = DocTestController(DD, [dirname]) sage: DC.expand_files_into_sources() sage: for i, source in enumerate(DC.sources): ....: DC.stats[source.basename] = {'walltime': 0.1*(i+1)} sage: DC.stats['sage.doctest.control'] = {'failed':True,'walltime':1.0} sage: DC.filter_sources() Only doctesting files that failed last test. sage: len(DC.sources) 1
-
load_environment
()¶ Return the module that provides the global environment.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController sage: DC = DocTestController(DocTestDefaults(), []) sage: 'BipartiteGraph' in DC.load_environment().__dict__ True sage: DC = DocTestController(DocTestDefaults(environment='sage.doctest.all'), []) sage: 'BipartiteGraph' in DC.load_environment().__dict__ False sage: 'run_doctests' in DC.load_environment().__dict__ True
-
load_stats
(filename)¶ Load stats from the most recent run(s).
Stats are stored as a JSON file, and include information on which files failed tests and the walltime used for execution of the doctests.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController sage: DC = DocTestController(DocTestDefaults(), []) sage: import json sage: filename = tmp_filename() sage: with open(filename, 'w') as stats_file: ....: json.dump({'sage.doctest.control':{'walltime':1.0r}}, stats_file) sage: DC.load_stats(filename) sage: DC.stats['sage.doctest.control'] {u'walltime': 1.0}
If the file doesn’t exist, nothing happens. If there is an error, print a message. In any case, leave the stats alone:
sage: d = tmp_dir() sage: DC.load_stats(os.path.join(d)) # Cannot read a directory Error loading stats from ... sage: DC.load_stats(os.path.join(d, "no_such_file")) sage: DC.stats['sage.doctest.control'] {u'walltime': 1.0}
-
log
(s, end='\\n')¶ Log the string
s + end
(whereend
is a newline by default) to the logfile and print it to the standard output.EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController sage: DD = DocTestDefaults(logfile=tmp_filename()) sage: DC = DocTestController(DD, []) sage: DC.log("hello world") hello world sage: DC.logfile.close() sage: with open(DD.logfile) as f: ....: print(f.read()) hello world
In serial mode, check that logging works even if
stdout
is redirected:sage: DD = DocTestDefaults(logfile=tmp_filename(), serial=True) sage: DC = DocTestController(DD, []) sage: from sage.doctest.forker import SageSpoofInOut sage: with open(os.devnull, 'w') as devnull: ....: S = SageSpoofInOut(devnull) ....: S.start_spoofing() ....: DC.log("hello world") ....: S.stop_spoofing() hello world sage: DC.logfile.close() sage: with open(DD.logfile) as f: ....: print(f.read()) hello world
Check that no duplicate logs appear, even when forking (trac ticket #15244):
sage: DD = DocTestDefaults(logfile=tmp_filename()) sage: DC = DocTestController(DD, []) sage: DC.log("hello world") hello world sage: if os.fork() == 0: ....: DC.logfile.close() ....: os._exit(0) sage: DC.logfile.close() sage: with open(DD.logfile) as f: ....: print(f.read()) hello world
-
run
()¶ This function is called after initialization to set up and run all doctests.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController sage: from sage.env import SAGE_SRC sage: import os sage: DD = DocTestDefaults() sage: filename = os.path.join(SAGE_SRC, "sage", "sets", "non_negative_integers.py") sage: DC = DocTestController(DD, [filename]) sage: DC.run() Running doctests with ID ... Doctesting 1 file. sage -t .../sage/sets/non_negative_integers.py [... tests, ... s] ---------------------------------------------------------------------- All tests passed! ---------------------------------------------------------------------- Total time for all tests: ... seconds cpu time: ... seconds cumulative wall time: ... seconds 0
We check that trac ticket #25378 is fixed (testing external packages while providing a logfile does not raise a ValueError: I/O operation on closed file):
sage: logfile = tmp_filename(ext='.log') sage: DD = DocTestDefaults(optional=set(['sage', 'external']), logfile=logfile) sage: filename = tmp_filename(ext='.py') sage: DC = DocTestController(DD, [filename]) sage: DC.run() Running doctests with ID ... Using --optional=external,sage External software to be detected: ... Doctesting 1 file. sage -t ....py [0 tests, ... s] ---------------------------------------------------------------------- All tests passed! ---------------------------------------------------------------------- Total time for all tests: ... seconds cpu time: ... seconds cumulative wall time: ... seconds External software detected for doctesting:... 0
-
run_doctests
()¶ Actually runs the doctests.
This function is called by
run()
.EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController sage: from sage.env import SAGE_SRC sage: import os sage: dirname = os.path.join(SAGE_SRC, 'sage', 'rings', 'homset.py') sage: DD = DocTestDefaults() sage: DC = DocTestController(DD, [dirname]) sage: DC.expand_files_into_sources() sage: DC.run_doctests() Doctesting 1 file. sage -t .../sage/rings/homset.py [... tests, ... s] ---------------------------------------------------------------------- All tests passed! ---------------------------------------------------------------------- Total time for all tests: ... seconds cpu time: ... seconds cumulative wall time: ... seconds
-
run_val_gdb
(testing=False)¶ Spawns a subprocess to run tests under the control of gdb or valgrind.
INPUT:
testing
– boolean; if True then the command to be run will be printed rather than a subprocess started.
EXAMPLES:
Note that the command lines include unexpanded environment variables. It is safer to let the shell expand them than to expand them here and risk insufficient quoting.
sage: from sage.doctest.control import DocTestDefaults, DocTestController sage: DD = DocTestDefaults(gdb=True) sage: DC = DocTestController(DD, ["hello_world.py"]) sage: DC.run_val_gdb(testing=True) exec gdb -x "...sage-gdb-commands" --args sage-runtests --serial --timeout=0 hello_world.py
sage: DD = DocTestDefaults(valgrind=True, optional="all", timeout=172800) sage: DC = DocTestController(DD, ["hello_world.py"]) sage: DC.run_val_gdb(testing=True) exec valgrind --tool=memcheck --leak-resolution=high --leak-check=full --num-callers=25 --suppressions="...valgrind/pyalloc.supp" --suppressions="...valgrind/sage.supp" --suppressions="...valgrind/sage-additional.supp" --log-file=".../valgrind/sage-memcheck.%p" sage-runtests --serial --timeout=172800 --optional=all hello_world.py
-
save_stats
(filename)¶ Save stats from the most recent run as a JSON file.
WARNING: This function overwrites the file.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController sage: DC = DocTestController(DocTestDefaults(), []) sage: DC.stats['sage.doctest.control'] = {'walltime':1.0r} sage: filename = tmp_filename() sage: DC.save_stats(filename) sage: import json sage: with open(filename) as f: ....: D = json.load(f) sage: D['sage.doctest.control'] {u'walltime': 1.0}
-
second_on_modern_computer
()¶ Return the wall time equivalent of a second on a modern computer.
OUTPUT:
Float. The wall time on your computer that would be equivalent to one second on a modern computer. Unless you have kick-ass hardware this should always be >= 1.0. Raises a
RuntimeError
if there are no stored timings to use as benchmark.EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController sage: DC = DocTestController(DocTestDefaults(), []) sage: DC.second_on_modern_computer() # not tested
-
sort_sources
()¶ This function sorts the sources so that slower doctests are run first.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController sage: from sage.env import SAGE_SRC sage: import os sage: dirname = os.path.join(SAGE_SRC, 'sage', 'doctest') sage: DD = DocTestDefaults(nthreads=2) sage: DC = DocTestController(DD, [dirname]) sage: DC.expand_files_into_sources() sage: DC.sources.sort(key=lambda s:s.basename) sage: for i, source in enumerate(DC.sources): ....: DC.stats[source.basename] = {'walltime': 0.1*(i+1)} sage: DC.sort_sources() Sorting sources by runtime so that slower doctests are run first.... sage: print("\n".join(source.basename for source in DC.sources)) sage.doctest.util sage.doctest.test sage.doctest.sources sage.doctest.reporting sage.doctest.parsing sage.doctest.forker sage.doctest.fixtures sage.doctest.external sage.doctest.control sage.doctest.all sage.doctest
-
-
class
sage.doctest.control.
DocTestDefaults
(**kwds)¶ Bases:
sage.structure.sage_object.SageObject
This class is used for doctesting the Sage doctest module.
It fills in attributes to be the same as the defaults defined in
sage-runtests
, expect for a few places, which is mostly to make doctesting more predictable.EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults sage: D = DocTestDefaults() sage: D DocTestDefaults() sage: D.timeout -1
Keyword arguments become attributes:
sage: D = DocTestDefaults(timeout=100) sage: D DocTestDefaults(timeout=100) sage: D.timeout 100
-
class
sage.doctest.control.
Logger
(*files)¶ Bases:
object
File-like object which implements writing to multiple files at once.
EXAMPLES:
sage: from sage.doctest.control import Logger sage: with open(tmp_filename(), "w+") as t: ....: L = Logger(sys.stdout, t) ....: _ = L.write("hello world\n") ....: _ = t.seek(0) ....: t.read() hello world 'hello world\n'
-
flush
()¶ Flush all files.
-
write
(x)¶ Write
x
to all files.
-
-
sage.doctest.control.
run_doctests
(module, options=None)¶ Runs the doctests in a given file.
INPUT:
module
– a Sage module, a string, or a list of such.options
– a DocTestDefaults object or None.
EXAMPLES:
sage: run_doctests(sage.rings.infinity) Running doctests with ID ... Doctesting 1 file. sage -t .../sage/rings/infinity.py [... tests, ... s] ---------------------------------------------------------------------- All tests passed! ---------------------------------------------------------------------- Total time for all tests: ... seconds cpu time: ... seconds cumulative wall time: ... seconds
-
sage.doctest.control.
skipdir
(dirname)¶ Return True if and only if the directory
dirname
should not be doctested.EXAMPLES:
sage: from sage.doctest.control import skipdir sage: skipdir(sage.env.SAGE_SRC) False sage: skipdir(os.path.join(sage.env.SAGE_SRC, "sage", "doctest", "tests")) True
-
sage.doctest.control.
skipfile
(filename)¶ Return True if and only if the file
filename
should not be doctested.EXAMPLES:
sage: from sage.doctest.control import skipfile sage: skipfile("skipme.c") True sage: filename = tmp_filename(ext=".pyx") sage: skipfile(filename) False sage: with open(filename, "w") as f: ....: _ = f.write("# nodoctest") sage: skipfile(filename) True