Parsing docstrings - Sage's Doctesting Framework

Skip to content
Back to top
View this page
Edit this page

# Parsing docstrings

This module contains functions and classes that parse docstrings.

AUTHORS:

- David Roe (2012-03-27) – initial version, based on Robert Bradshaw’s code.

- Jeroen Demeyer(2014-08-28) – much improved handling of tolerances using interval arithmetic (Issue #16889).

class sage.doctest.parsing.OriginalSource(example)

  Bases: `object`

  Context swapping out the pre-parsed source with the original for better reporting.

  EXAMPLES:

  Sage

      sage: from sage.doctest.sources import FileDocTestSource
      sage: from sage.doctest.control import DocTestDefaults
      sage: filename = sage.doctest.forker.__file__
      sage: FDS = FileDocTestSource(filename, DocTestDefaults())
      sage: doctests, extras = FDS.create_doctests(globals())
      sage: ex = doctests[0].examples[0]
      sage: ex.sage_source
      'doctest_var = 42; doctest_var^2\n'
      sage: ex.source
      'doctest_var = Integer(42); doctest_var**Integer(2)\n'
      sage: from sage.doctest.parsing import OriginalSource
      sage: with OriginalSource(ex):
      ....:     ex.source
      'doctest_var = 42; doctest_var^2\n'

  Python

      >>> from sage.all import *
      >>> from sage.doctest.sources import FileDocTestSource
      >>> from sage.doctest.control import DocTestDefaults
      >>> filename = sage.doctest.forker.__file__
      >>> FDS = FileDocTestSource(filename, DocTestDefaults())
      >>> doctests, extras = FDS.create_doctests(globals())
      >>> ex = doctests[Integer(0)].examples[Integer(0)]
      >>> ex.sage_source
      'doctest_var = 42; doctest_var^2\n'
      >>> ex.source
      'doctest_var = Integer(42); doctest_var**Integer(2)\n'
      >>> from sage.doctest.parsing import OriginalSource
      >>> with OriginalSource(ex):
      ...     ex.source
      'doctest_var = 42; doctest_var^2\n'

class sage.doctest.parsing.SageDocTestParser(optional_tags=(), long=False, *, probed_tags=(), file_optional_tags=())

  Bases: `DocTestParser`

  A version of the standard doctest parser which handles Sage’s custom options and tolerances in floating point arithmetic.

  file_optional_tags: set[str]
  long: bool
  optional_only: bool
  optional_tags: bool | set[str]
  optionals: dict[str, int]
  parse(string, *args)

    A Sage specialization of `doctest.DocTestParser`.

    INPUT:

    - `string` – the string to parse

    - `name` – (optional) string giving the name identifying string, to be used in error messages

    OUTPUT:

    - A list consisting of strings and `doctest.Example` instances. There will be at least one string between successive examples (exactly one unless long or optional tests are removed), and it will begin and end with a string.

    EXAMPLES:

    Sage

        sage: from sage.doctest.parsing import SageDocTestParser
        sage: DTP = SageDocTestParser(('sage','magma','guava'))
        sage: example = 'Explanatory text::\n\n    sage: E = magma("EllipticCurve([1, 1, 1, -10, -10])") # optional: magma\n\nLater text'
        sage: parsed = DTP.parse(example)
        sage: parsed[0]
        'Explanatory text::\n\n'
        sage: parsed[1].sage_source
        'E = magma("EllipticCurve([1, 1, 1, -10, -10])") # optional: magma\n'
        sage: parsed[2]
        '\nLater text'

    Python

        >>> from sage.all import *
        >>> from sage.doctest.parsing import SageDocTestParser
        >>> DTP = SageDocTestParser(('sage','magma','guava'))
        >>> example = 'Explanatory text::\n\n    sage: E = magma("EllipticCurve([1, 1, 1, -10, -10])") # optional: magma\n\nLater text'
        >>> parsed = DTP.parse(example)
        >>> parsed[Integer(0)]
        'Explanatory text::\n\n'
        >>> parsed[Integer(1)].sage_source
        'E = magma("EllipticCurve([1, 1, 1, -10, -10])") # optional: magma\n'
        >>> parsed[Integer(2)]
        '\nLater text'

    If the doctest parser is not created to accept a given optional argument, the corresponding examples will just be removed:

    Sage

        sage: DTP2 = SageDocTestParser(('sage',))
        sage: parsed2 = DTP2.parse(example)
        sage: parsed2
        ['Explanatory text::\n\n', '\nLater text']

    [Python]

        >>> from sage.all import *
        >>> DTP2 = SageDocTestParser(('sage',))
        >>> parsed2 = DTP2.parse(example)
        >>> parsed2
        ['Explanatory text::\n\n', '\nLater text']

    You can mark doctests as having a particular tolerance:

    Sage

        sage: example2 = 'sage: gamma(1.6) # tol 2.0e-11\n0.893515349287690'
        sage: ex = DTP.parse(example2)[1]
        sage: ex.sage_source
        'gamma(1.6) # tol 2.0e-11\n'
        sage: ex.want
        '0.893515349287690\n'
        sage: type(ex.want)
        <class 'sage.doctest.marked_output.MarkedOutput'>
        sage: ex.want.tol
        2.000000000000000000...?e-11

    Python

        >>> from sage.all import *
        >>> example2 = 'sage: gamma(1.6) # tol 2.0e-11\n0.893515349287690'
        >>> ex = DTP.parse(example2)[Integer(1)]
        >>> ex.sage_source
        'gamma(1.6) # tol 2.0e-11\n'
        >>> ex.want
        '0.893515349287690\n'
        >>> type(ex.want)
        <class 'sage.doctest.marked_output.MarkedOutput'>
        >>> ex.want.tol
        2.000000000000000000...?e-11

    You can use continuation lines:

    Sage

        sage: s = "sage: for i in range(4):\n....:     print(i)\n....:\n"
        sage: ex = DTP2.parse(s)[1]
        sage: ex.source
        'for i in range(Integer(4)):\n    print(i)\n'

    [Python]

        >>> from sage.all import *
        >>> s = "sage: for i in range(4):\n....:     print(i)\n....:\n"
        >>> ex = DTP2.parse(s)[Integer(1)]
        >>> ex.source
        'for i in range(Integer(4)):\n    print(i)\n'

    Optional tags at the start of an example block persist to the end of the block (delimited by a blank line):

    Sage

        sage: # long time, needs sage.rings.number_field
        sage: QQbar(I)^10000
        1
        sage: QQbar(I)^10000  # not tested
        I

        sage: # needs sage.rings.finite_rings
        sage: GF(7)
        Finite Field of size 7
        sage: GF(10)
        Traceback (most recent call last):
        ...
        ValueError: the order of a finite field must be a prime power

    Python

        >>> from sage.all import *
        >>> # long time, needs sage.rings.number_field
        >>> QQbar(I)**Integer(10000)
        1
        >>> QQbar(I)**Integer(10000)  # not tested
        I

        >>> # needs sage.rings.finite_rings
        >>> GF(Integer(7))
        Finite Field of size 7
        >>> GF(Integer(10))
        Traceback (most recent call last):
        ...
        ValueError: the order of a finite field must be a prime power

    Test that Issue #26575 is resolved:

    Sage

        sage: example3 = 'sage: Zp(5,4,print_mode="digits")(5)\n...00010'
        sage: parsed3 = DTP.parse(example3)
        sage: dte = parsed3[1]
        sage: dte.sage_source
        'Zp(5,4,print_mode="digits")(5)\n'
        sage: dte.want
        '...00010\n'

    [Python]

        >>> from sage.all import *
        >>> example3 = 'sage: Zp(5,4,print_mode="digits")(5)\n...00010'
        >>> parsed3 = DTP.parse(example3)
        >>> dte = parsed3[Integer(1)]
        >>> dte.sage_source
        'Zp(5,4,print_mode="digits")(5)\n'
        >>> dte.want
        '...00010\n'

    Style warnings:

    Sage

        sage: def parse(test_string):
        ....:     return [x if isinstance(x, str)
        ....:               else (getattr(x, 'warnings', None), x.sage_source, x.source)
        ....:             for x in DTP.parse(test_string)]

        sage: parse('sage: 1 # optional guava mango\nsage: 2 # optional guava\nsage: 3 # optional guava\nsage: 4 # optional guava\nsage: 5 # optional guava\n\nsage: 11 # optional guava')
        ['',
         (["Consider using a block-scoped tag by inserting the line 'sage: # optional - guava' just before this line to avoid repeating the tag 5 times"],
          '1 # optional guava mango\n',
          'None  # virtual doctest'),
         '',
         (None, '2 # optional guava\n', 'Integer(2) # optional guava\n'),
         '',
         (None, '3 # optional guava\n', 'Integer(3) # optional guava\n'),
         '',
         (None, '4 # optional guava\n', 'Integer(4) # optional guava\n'),
         '',
         (None, '5 # optional guava\n', 'Integer(5) # optional guava\n'),
         '\n',
         (None, '11 # optional guava\n', 'Integer(11) # optional guava\n'),
         '']

        sage: parse('sage: 1 # optional guava\nsage: 2 # optional guava mango\nsage: 3 # optional guava\nsage: 4 # optional guava\nsage: 5 # optional guava\n')
        ['',
         (["Consider using a block-scoped tag by inserting the line 'sage: # optional - guava' just before this line to avoid repeating the tag 5 times"],
          '1 # optional guava\n',
          'Integer(1) # optional guava\n'),
         '',
         '',
         (None, '3 # optional guava\n', 'Integer(3) # optional guava\n'),
         '',
         (None, '4 # optional guava\n', 'Integer(4) # optional guava\n'),
         '',
         (None, '5 # optional guava\n', 'Integer(5) # optional guava\n'),
         '']

        sage: parse('sage: # optional mango\nsage: 1 # optional guava\nsage: 2 # optional guava mango\nsage: 3 # optional guava\nsage: 4 # optional guava\n sage: 5 # optional guava\n')  # optional - guava mango
        ['',
         (["Consider updating this block-scoped tag to 'sage: # optional - guava mango' to avoid repeating the tag 5 times"],
          '# optional mango\n',
          'None  # virtual doctest'),
         '',
         '',
         '',
         '',
         '',
         '']

        sage: parse('::\n\n    sage: 1 # optional guava\n    sage: 2 # optional guava mango\n    sage: 3 # optional guava\n\n::\n\n    sage: 4 # optional guava\n     sage: 5 # optional guava\n')
        ['::\n\n',
        (None, '1 # optional guava\n', 'Integer(1) # optional guava\n'),
        '',
        '',
        (None, '3 # optional guava\n', 'Integer(3) # optional guava\n'),
        '\n::\n\n',
        (None, '4 # optional guava\n', 'Integer(4) # optional guava\n'),
        '',
        (None, '5 # optional guava\n', 'Integer(5) # optional guava\n'),
        '']

    Python

        >>> from sage.all import *
        >>> def parse(test_string):
        ...     return [x if isinstance(x, str)
        ...               else (getattr(x, 'warnings', None), x.sage_source, x.source)
        ...             for x in DTP.parse(test_string)]

        >>> parse('sage: 1 # optional guava mango\nsage: 2 # optional guava\nsage: 3 # optional guava\nsage: 4 # optional guava\nsage: 5 # optional guava\n\nsage: 11 # optional guava')
        ['',
         (["Consider using a block-scoped tag by inserting the line 'sage: # optional - guava' just before this line to avoid repeating the tag 5 times"],
          '1 # optional guava mango\n',
          'None  # virtual doctest'),
         '',
         (None, '2 # optional guava\n', 'Integer(2) # optional guava\n'),
         '',
         (None, '3 # optional guava\n', 'Integer(3) # optional guava\n'),
         '',
         (None, '4 # optional guava\n', 'Integer(4) # optional guava\n'),
         '',
         (None, '5 # optional guava\n', 'Integer(5) # optional guava\n'),
         '\n',
         (None, '11 # optional guava\n', 'Integer(11) # optional guava\n'),
         '']

        >>> parse('sage: 1 # optional guava\nsage: 2 # optional guava mango\nsage: 3 # optional guava\nsage: 4 # optional guava\nsage: 5 # optional guava\n')
        ['',
         (["Consider using a block-scoped tag by inserting the line 'sage: # optional - guava' just before this line to avoid repeating the tag 5 times"],
          '1 # optional guava\n',
          'Integer(1) # optional guava\n'),
         '',
         '',
         (None, '3 # optional guava\n', 'Integer(3) # optional guava\n'),
         '',
         (None, '4 # optional guava\n', 'Integer(4) # optional guava\n'),
         '',
         (None, '5 # optional guava\n', 'Integer(5) # optional guava\n'),
         '']

        >>> parse('sage: # optional mango\nsage: 1 # optional guava\nsage: 2 # optional guava mango\nsage: 3 # optional guava\nsage: 4 # optional guava\n sage: 5 # optional guava\n')  # optional - guava mango
        ['',
         (["Consider updating this block-scoped tag to 'sage: # optional - guava mango' to avoid repeating the tag 5 times"],
          '# optional mango\n',
          'None  # virtual doctest'),
         '',
         '',
         '',
         '',
         '',
         '']

        >>> parse('::\n\n    sage: 1 # optional guava\n    sage: 2 # optional guava mango\n    sage: 3 # optional guava\n\n::\n\n    sage: 4 # optional guava\n     sage: 5 # optional guava\n')
        ['::\n\n',
        (None, '1 # optional guava\n', 'Integer(1) # optional guava\n'),
        '',
        '',
        (None, '3 # optional guava\n', 'Integer(3) # optional guava\n'),
        '\n::\n\n',
        (None, '4 # optional guava\n', 'Integer(4) # optional guava\n'),
        '',
        (None, '5 # optional guava\n', 'Integer(5) # optional guava\n'),
        '']

  probed_tags: bool | set[str]

class sage.doctest.parsing.SageOutputChecker

  Bases: `OutputChecker`

  A modification of the doctest OutputChecker that can check relative and absolute tolerance of answers.

  EXAMPLES:

  Sage

      sage: from sage.doctest.parsing import SageOutputChecker, MarkedOutput, SageDocTestParser
      sage: import doctest
      sage: optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
      sage: DTP = SageDocTestParser(('sage','magma','guava'))
      sage: OC = SageOutputChecker()
      sage: example2 = 'sage: gamma(1.6) # tol 2.0e-11\n0.893515349287690'
      sage: ex = DTP.parse(example2)[1]
      sage: ex.sage_source
      'gamma(1.6) # tol 2.0e-11\n'
      sage: ex.want
      '0.893515349287690\n'
      sage: type(ex.want)
      <class 'sage.doctest.marked_output.MarkedOutput'>
      sage: ex.want.tol
      2.000000000000000000...?e-11
      sage: OC.check_output(ex.want, '0.893515349287690', optflag)
      True
      sage: OC.check_output(ex.want, '0.8935153492877', optflag)
      True
      sage: OC.check_output(ex.want, '0', optflag)
      False
      sage: OC.check_output(ex.want, 'x + 0.8935153492877', optflag)
      False

  Python

      >>> from sage.all import *
      >>> from sage.doctest.parsing import SageOutputChecker, MarkedOutput, SageDocTestParser
      >>> import doctest
      >>> optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
      >>> DTP = SageDocTestParser(('sage','magma','guava'))
      >>> OC = SageOutputChecker()
      >>> example2 = 'sage: gamma(1.6) # tol 2.0e-11\n0.893515349287690'
      >>> ex = DTP.parse(example2)[Integer(1)]
      >>> ex.sage_source
      'gamma(1.6) # tol 2.0e-11\n'
      >>> ex.want
      '0.893515349287690\n'
      >>> type(ex.want)
      <class 'sage.doctest.marked_output.MarkedOutput'>
      >>> ex.want.tol
      2.000000000000000000...?e-11
      >>> OC.check_output(ex.want, '0.893515349287690', optflag)
      True
      >>> OC.check_output(ex.want, '0.8935153492877', optflag)
      True
      >>> OC.check_output(ex.want, '0', optflag)
      False
      >>> OC.check_output(ex.want, 'x + 0.8935153492877', optflag)
      False

  check_output(want, got, optionflags)

    Check to see if the output matches the desired output.

    If `want` is a `MarkedOutput` instance, takes into account the desired tolerance.

    INPUT:

    - `want` – string or `MarkedOutput`

    - `got` – string

    - `optionflags` – integer; passed down to `doctest.OutputChecker`

    OUTPUT: boolean; whether `got` matches `want` up to the specified tolerance

    EXAMPLES:

    Sage

        sage: from sage.doctest.parsing import MarkedOutput, SageOutputChecker
        sage: import doctest
        sage: optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
        sage: rndstr = MarkedOutput("I'm wrong!").update(random=True)
        sage: tentol = MarkedOutput("10.0").update(tol=.1)
        sage: tenabs = MarkedOutput("10.0").update(abs_tol=.1)
        sage: tenrel = MarkedOutput("10.0").update(rel_tol=.1)
        sage: zerotol = MarkedOutput("0.0").update(tol=.1)
        sage: zeroabs = MarkedOutput("0.0").update(abs_tol=.1)
        sage: zerorel = MarkedOutput("0.0").update(rel_tol=.1)
        sage: zero = "0.0"
        sage: nf = "9.5"
        sage: ten = "10.05"
        sage: eps = "-0.05"
        sage: OC = SageOutputChecker()

    Python

        >>> from sage.all import *
        >>> from sage.doctest.parsing import MarkedOutput, SageOutputChecker
        >>> import doctest
        >>> optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
        >>> rndstr = MarkedOutput("I'm wrong!").update(random=True)
        >>> tentol = MarkedOutput("10.0").update(tol=RealNumber('.1'))
        >>> tenabs = MarkedOutput("10.0").update(abs_tol=RealNumber('.1'))
        >>> tenrel = MarkedOutput("10.0").update(rel_tol=RealNumber('.1'))
        >>> zerotol = MarkedOutput("0.0").update(tol=RealNumber('.1'))
        >>> zeroabs = MarkedOutput("0.0").update(abs_tol=RealNumber('.1'))
        >>> zerorel = MarkedOutput("0.0").update(rel_tol=RealNumber('.1'))
        >>> zero = "0.0"
        >>> nf = "9.5"
        >>> ten = "10.05"
        >>> eps = "-0.05"
        >>> OC = SageOutputChecker()

    Sage

        sage: OC.check_output(rndstr,nf,optflag)
        True

        sage: OC.check_output(tentol,nf,optflag)
        True
        sage: OC.check_output(tentol,ten,optflag)
        True
        sage: OC.check_output(tentol,zero,optflag)
        False

        sage: OC.check_output(tenabs,nf,optflag)
        False
        sage: OC.check_output(tenabs,ten,optflag)
        True
        sage: OC.check_output(tenabs,zero,optflag)
        False

        sage: OC.check_output(tenrel,nf,optflag)
        True
        sage: OC.check_output(tenrel,ten,optflag)
        True
        sage: OC.check_output(tenrel,zero,optflag)
        False

        sage: OC.check_output(zerotol,zero,optflag)
        True
        sage: OC.check_output(zerotol,eps,optflag)
        True
        sage: OC.check_output(zerotol,ten,optflag)
        False

        sage: OC.check_output(zeroabs,zero,optflag)
        True
        sage: OC.check_output(zeroabs,eps,optflag)
        True
        sage: OC.check_output(zeroabs,ten,optflag)
        False

        sage: OC.check_output(zerorel,zero,optflag)
        True
        sage: OC.check_output(zerorel,eps,optflag)
        False
        sage: OC.check_output(zerorel,ten,optflag)
        False

    [Python]

        >>> from sage.all import *
        >>> OC.check_output(rndstr,nf,optflag)
        True

        >>> OC.check_output(tentol,nf,optflag)
        True
        >>> OC.check_output(tentol,ten,optflag)
        True
        >>> OC.check_output(tentol,zero,optflag)
        False

        >>> OC.check_output(tenabs,nf,optflag)
        False
        >>> OC.check_output(tenabs,ten,optflag)
        True
        >>> OC.check_output(tenabs,zero,optflag)
        False

        >>> OC.check_output(tenrel,nf,optflag)
        True
        >>> OC.check_output(tenrel,ten,optflag)
        True
        >>> OC.check_output(tenrel,zero,optflag)
        False

        >>> OC.check_output(zerotol,zero,optflag)
        True
        >>> OC.check_output(zerotol,eps,optflag)
        True
        >>> OC.check_output(zerotol,ten,optflag)
        False

        >>> OC.check_output(zeroabs,zero,optflag)
        True
        >>> OC.check_output(zeroabs,eps,optflag)
        True
        >>> OC.check_output(zeroabs,ten,optflag)
        False

        >>> OC.check_output(zerorel,zero,optflag)
        True
        >>> OC.check_output(zerorel,eps,optflag)
        False
        >>> OC.check_output(zerorel,ten,optflag)
        False

    More explicit tolerance checks:

    Sage

        sage: _ = x  # rel tol 1e10                                                 # needs sage.symbolic
        sage: raise RuntimeError   # rel tol 1e10
        Traceback (most recent call last):
        ...
        RuntimeError
        sage: 1  # abs tol 2
        -0.5
        sage: print("0.9999")    # rel tol 1e-4
        1.0
        sage: print("1.00001")   # abs tol 1e-5
        1.0
        sage: 0  # rel tol 1
        1

    Python

        >>> from sage.all import *
        >>> _ = x  # rel tol 1e10                                                 # needs sage.symbolic
        >>> raise RuntimeError   # rel tol 1e10
        Traceback (most recent call last):
        ...
        RuntimeError
        >>> Integer(1)  # abs tol 2
        -0.5
        >>> print("0.9999")    # rel tol 1e-4
        1.0
        >>> print("1.00001")   # abs tol 1e-5
        1.0
        >>> Integer(0)  # rel tol 1
        1

    Abs tol checks over the complex domain:

    Sage

        sage: [1, -1.3, -1.5 + 0.1*I, 0.5 - 0.1*I, -1.5*I]  # abs tol 1.0
        [1, -1, -1, 1, -I]

    [Python]

        >>> from sage.all import *
        >>> [Integer(1), -RealNumber('1.3'), -RealNumber('1.5') + RealNumber('0.1')*I, RealNumber('0.5') - RealNumber('0.1')*I, -RealNumber('1.5')*I]  # abs tol 1.0
        [1, -1, -1, 1, -I]

    Spaces before numbers or between the sign and number are ignored:

    Sage

        sage: print("[ - 1, 2]")  # abs tol 1e-10
        [-1,2]

    Python

        >>> from sage.all import *
        >>> print("[ - 1, 2]")  # abs tol 1e-10
        [-1,2]

    Tolerance on Python 3 for string results with unicode prefix:

    Sage

        sage: a = 'Cyrano'; a
        'Cyrano'
        sage: b = ['Fermat', 'Euler']; b
        ['Fermat',  'Euler']
        sage: c = 'you'; c
        'you'

    [Python]

        >>> from sage.all import *
        >>> a = 'Cyrano'; a
        'Cyrano'
        >>> b = ['Fermat', 'Euler']; b
        ['Fermat',  'Euler']
        >>> c = 'you'; c
        'you'

    This illustrates that Issue #33588 is fixed:

    Sage

        sage: from sage.doctest.parsing import SageOutputChecker, SageDocTestParser
        sage: import doctest
        sage: optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
        sage: DTP = SageDocTestParser(('sage','magma','guava'))
        sage: OC = SageOutputChecker()
        sage: example = "sage: 1.3090169943749475 # tol 1e-8\n1.3090169943749475"
        sage: ex = DTP.parse(example)[1]
        sage: OC.check_output(ex.want, '1.3090169943749475', optflag)
        True
        sage: OC.check_output(ex.want, 'ANYTHING1.3090169943749475', optflag)
        False
        sage: OC.check_output(ex.want, 'Long-step dual simplex will be used\n1.3090169943749475', optflag)
        True

    Python

        >>> from sage.all import *
        >>> from sage.doctest.parsing import SageOutputChecker, SageDocTestParser
        >>> import doctest
        >>> optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
        >>> DTP = SageDocTestParser(('sage','magma','guava'))
        >>> OC = SageOutputChecker()
        >>> example = "sage: 1.3090169943749475 # tol 1e-8\n1.3090169943749475"
        >>> ex = DTP.parse(example)[Integer(1)]
        >>> OC.check_output(ex.want, '1.3090169943749475', optflag)
        True
        >>> OC.check_output(ex.want, 'ANYTHING1.3090169943749475', optflag)
        False
        >>> OC.check_output(ex.want, 'Long-step dual simplex will be used\n1.3090169943749475', optflag)
        True

  do_fixup(want, got)

    Perform few changes to the strings `want` and `got`.

    For example, remove warnings to be ignored.

    INPUT:

    - `want` – string or `MarkedOutput`

    - `got` – string

    OUTPUT: a tuple:

    - boolean, `True` when some fixup were performed and `False` otherwise

    - string, edited wanted string

    - string, edited got string

    Note

    Currently, the code only possibly changes the string `got` while keeping `want` invariant. We keep open the possibility of adding a regular expression which would also change the `want` string. This is why `want` is an input and an output of the method even if currently kept invariant.

    EXAMPLES:

    Sage

        sage: from sage.doctest.parsing import SageOutputChecker
        sage: OC = SageOutputChecker()
        sage: OC.do_fixup('1.3090169943749475','1.3090169943749475')
        (False, '1.3090169943749475', '1.3090169943749475')
        sage: OC.do_fixup('1.3090169943749475','ANYTHING1.3090169943749475')
        (False, '1.3090169943749475', 'ANYTHING1.3090169943749475')
        sage: OC.do_fixup('1.3090169943749475','Long-step dual simplex will be used\n1.3090169943749475')
        (True, '1.3090169943749475', '\n1.3090169943749475')

    Python

        >>> from sage.all import *
        >>> from sage.doctest.parsing import SageOutputChecker
        >>> OC = SageOutputChecker()
        >>> OC.do_fixup('1.3090169943749475','1.3090169943749475')
        (False, '1.3090169943749475', '1.3090169943749475')
        >>> OC.do_fixup('1.3090169943749475','ANYTHING1.3090169943749475')
        (False, '1.3090169943749475', 'ANYTHING1.3090169943749475')
        >>> OC.do_fixup('1.3090169943749475','Long-step dual simplex will be used\n1.3090169943749475')
        (True, '1.3090169943749475', '\n1.3090169943749475')

    When `want` is an instance of class `MarkedOutput`:

    Sage

        sage: from sage.doctest.parsing import SageOutputChecker, SageDocTestParser
        sage: import doctest
        sage: optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
        sage: DTP = SageDocTestParser(('sage','magma','guava'))
        sage: OC = SageOutputChecker()
        sage: example = "sage: 1.3090169943749475\n1.3090169943749475"
        sage: ex = DTP.parse(example)[1]
        sage: ex.want
        '1.3090169943749475\n'
        sage: OC.do_fixup(ex.want,'1.3090169943749475')
        (False, '1.3090169943749475\n', '1.3090169943749475')
        sage: OC.do_fixup(ex.want,'ANYTHING1.3090169943749475')
        (False, '1.3090169943749475\n', 'ANYTHING1.3090169943749475')
        sage: OC.do_fixup(ex.want,'Long-step dual simplex will be used\n1.3090169943749475')
        (True, '1.3090169943749475\n', '\n1.3090169943749475')

    [Python]

        >>> from sage.all import *
        >>> from sage.doctest.parsing import SageOutputChecker, SageDocTestParser
        >>> import doctest
        >>> optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
        >>> DTP = SageDocTestParser(('sage','magma','guava'))
        >>> OC = SageOutputChecker()
        >>> example = "sage: 1.3090169943749475\n1.3090169943749475"
        >>> ex = DTP.parse(example)[Integer(1)]
        >>> ex.want
        '1.3090169943749475\n'
        >>> OC.do_fixup(ex.want,'1.3090169943749475')
        (False, '1.3090169943749475\n', '1.3090169943749475')
        >>> OC.do_fixup(ex.want,'ANYTHING1.3090169943749475')
        (False, '1.3090169943749475\n', 'ANYTHING1.3090169943749475')
        >>> OC.do_fixup(ex.want,'Long-step dual simplex will be used\n1.3090169943749475')
        (True, '1.3090169943749475\n', '\n1.3090169943749475')

  human_readable_escape_sequences(string)

    Make ANSI escape sequences human readable.

    EXAMPLES:

    Sage

        sage: print('This is \x1b[1mbold\x1b[0m text')
        This is <CSI-1m>bold<CSI-0m> text

    Python

        >>> from sage.all import *
        >>> print('This is \x1b[1mbold\x1b[0m text')
        This is <CSI-1m>bold<CSI-0m> text

  output_difference(example, got, optionflags)

    Report on the differences between the desired result and what was actually obtained.

    If `want` is a `MarkedOutput` instance, takes into account the desired tolerance.

    INPUT:

    - `example` – a `doctest.Example` instance

    - `got` – string

    - `optionflags` – integer; passed down to `doctest.OutputChecker`

    OUTPUT: string, describing how `got` fails to match `example.want`

    EXAMPLES:

    Sage

        sage: from sage.doctest.parsing import MarkedOutput, SageOutputChecker
        sage: import doctest
        sage: optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
        sage: tentol = doctest.Example('',MarkedOutput("10.0\n").update(tol=.1))
        sage: tenabs = doctest.Example('',MarkedOutput("10.0\n").update(abs_tol=.1))
        sage: tenrel = doctest.Example('',MarkedOutput("10.0\n").update(rel_tol=.1))
        sage: zerotol = doctest.Example('',MarkedOutput("0.0\n").update(tol=.1))
        sage: zeroabs = doctest.Example('',MarkedOutput("0.0\n").update(abs_tol=.1))
        sage: zerorel = doctest.Example('',MarkedOutput("0.0\n").update(rel_tol=.1))
        sage: tlist = doctest.Example('',MarkedOutput("[10.0, 10.0, 10.0, 10.0, 10.0, 10.0]\n").update(abs_tol=0.987))
        sage: zero = "0.0"
        sage: nf = "9.5"
        sage: ten = "10.05"
        sage: eps = "-0.05"
        sage: L = "[9.9, 8.7, 10.3, 11.2, 10.8, 10.0]"
        sage: OC = SageOutputChecker()

    Python

        >>> from sage.all import *
        >>> from sage.doctest.parsing import MarkedOutput, SageOutputChecker
        >>> import doctest
        >>> optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
        >>> tentol = doctest.Example('',MarkedOutput("10.0\n").update(tol=RealNumber('.1')))
        >>> tenabs = doctest.Example('',MarkedOutput("10.0\n").update(abs_tol=RealNumber('.1')))
        >>> tenrel = doctest.Example('',MarkedOutput("10.0\n").update(rel_tol=RealNumber('.1')))
        >>> zerotol = doctest.Example('',MarkedOutput("0.0\n").update(tol=RealNumber('.1')))
        >>> zeroabs = doctest.Example('',MarkedOutput("0.0\n").update(abs_tol=RealNumber('.1')))
        >>> zerorel = doctest.Example('',MarkedOutput("0.0\n").update(rel_tol=RealNumber('.1')))
        >>> tlist = doctest.Example('',MarkedOutput("[10.0, 10.0, 10.0, 10.0, 10.0, 10.0]\n").update(abs_tol=RealNumber('0.987')))
        >>> zero = "0.0"
        >>> nf = "9.5"
        >>> ten = "10.05"
        >>> eps = "-0.05"
        >>> L = "[9.9, 8.7, 10.3, 11.2, 10.8, 10.0]"
        >>> OC = SageOutputChecker()

    Sage

        sage: print(OC.output_difference(tenabs,nf,optflag))
        Expected:
            10.0
        Got:
            9.5
        Tolerance exceeded:
            10.0 vs 9.5, tolerance 5e-1 > 1e-1

        sage: print(OC.output_difference(tentol,zero,optflag))
        Expected:
            10.0
        Got:
            0.0
        Tolerance exceeded:
            10.0 vs 0.0, tolerance 1e0 > 1e-1

        sage: print(OC.output_difference(tentol,eps,optflag))
        Expected:
            10.0
        Got:
            -0.05
        Tolerance exceeded:
            10.0 vs -0.05, tolerance 2e0 > 1e-1

        sage: print(OC.output_difference(tlist,L,optflag))
        Expected:
            [10.0, 10.0, 10.0, 10.0, 10.0, 10.0]
        Got:
            [9.9, 8.7, 10.3, 11.2, 10.8, 10.0]
        Tolerance exceeded in 2 of 6:
            10.0 vs 8.7, tolerance 2e0 > 9.87e-1
            10.0 vs 11.2, tolerance 2e0 > 9.87e-1

    [Python]

        >>> from sage.all import *
        >>> print(OC.output_difference(tenabs,nf,optflag))
        Expected:
            10.0
        Got:
            9.5
        Tolerance exceeded:
            10.0 vs 9.5, tolerance 5e-1 > 1e-1

        >>> print(OC.output_difference(tentol,zero,optflag))
        Expected:
            10.0
        Got:
            0.0
        Tolerance exceeded:
            10.0 vs 0.0, tolerance 1e0 > 1e-1

        >>> print(OC.output_difference(tentol,eps,optflag))
        Expected:
            10.0
        Got:
            -0.05
        Tolerance exceeded:
            10.0 vs -0.05, tolerance 2e0 > 1e-1

        >>> print(OC.output_difference(tlist,L,optflag))
        Expected:
            [10.0, 10.0, 10.0, 10.0, 10.0, 10.0]
        Got:
            [9.9, 8.7, 10.3, 11.2, 10.8, 10.0]
        Tolerance exceeded in 2 of 6:
            10.0 vs 8.7, tolerance 2e0 > 9.87e-1
            10.0 vs 11.2, tolerance 2e0 > 9.87e-1

sage.doctest.parsing.get_source(example)

  Return the source with the leading ‘sage: ‘ stripped off.

  EXAMPLES:

  Sage

      sage: from sage.doctest.parsing import get_source
      sage: from sage.doctest.sources import DictAsObject
      sage: example = DictAsObject({})
      sage: example.sage_source = "2 + 2"
      sage: example.source = "sage: 2 + 2"
      sage: get_source(example)
      '2 + 2'
      sage: example = DictAsObject({})
      sage: example.source = "3 + 3"
      sage: get_source(example)
      '3 + 3'

  Python

      >>> from sage.all import *
      >>> from sage.doctest.parsing import get_source
      >>> from sage.doctest.sources import DictAsObject
      >>> example = DictAsObject({})
      >>> example.sage_source = "2 + 2"
      >>> example.source = "sage: 2 + 2"
      >>> get_source(example)
      '2 + 2'
      >>> example = DictAsObject({})
      >>> example.source = "3 + 3"
      >>> get_source(example)
      '3 + 3'

sage.doctest.parsing.parse_file_optional_tags(lines)

  Scan the first few lines for file-level doctest directives.

  INPUT:

  - `lines` – iterable of pairs `(lineno, line)`

  OUTPUT: dictionary whose keys are strings (tags); see `parse_optional_tags()`

  EXAMPLES:

  Sage

      sage: from sage.doctest.parsing import parse_file_optional_tags
      sage: filename = tmp_filename(ext='.pyx')
      sage: with open(filename, "r") as f:
      ....:     parse_file_optional_tags(enumerate(f))
      {}
      sage: with open(filename, "w") as f:
      ....:     _ = f.write("# nodoctest")
      sage: with open(filename, "r") as f:
      ....:     parse_file_optional_tags(enumerate(f))
      {'not tested': None}
      sage: with open(filename, "w") as f:
      ....:     _ = f.write("# sage.doctest: "    # broken in two source lines to avoid the pattern
      ....:                 "optional - xyz")     # of relint (multiline_doctest_comment)
      sage: with open(filename, "r") as f:
      ....:     parse_file_optional_tags(enumerate(f))
      {'xyz': None}

  Python

      >>> from sage.all import *
      >>> from sage.doctest.parsing import parse_file_optional_tags
      >>> filename = tmp_filename(ext='.pyx')
      >>> with open(filename, "r") as f:
      ...     parse_file_optional_tags(enumerate(f))
      {}
      >>> with open(filename, "w") as f:
      ...     _ = f.write("# nodoctest")
      >>> with open(filename, "r") as f:
      ...     parse_file_optional_tags(enumerate(f))
      {'not tested': None}
      >>> with open(filename, "w") as f:
      ...     _ = f.write("# sage.doctest: "    # broken in two source lines to avoid the pattern
      ...                 "optional - xyz")     # of relint (multiline_doctest_comment)
      >>> with open(filename, "r") as f:
      ...     parse_file_optional_tags(enumerate(f))
      {'xyz': None}

sage.doctest.parsing.parse_optional_tags(string: str) → dict[str, str | None]
sage.doctest.parsing.parse_optional_tags(string: str, *, return_string_sans_tags: Literal[True]) → tuple[dict[str, str | None], str, bool]

  Return a dictionary whose keys are optional tags from the following set that occur in a comment on the first line of the input string.

  - `'long time'`

  - `'not implemented'`

  - `'not tested'`

  - `'known bug'` (possible values are `None`, `linux` and `macos`)

  - `'py2'`

  - `'optional -- FEATURE...'` or `'needs FEATURE...'` – the dictionary will just have the key `'FEATURE'`

  The values, if non-`None`, are strings with optional explanations for a tag, which may appear in parentheses after the tag in `string`.

  INPUT:

  - `string` – string

  - `return_string_sans_tags` – boolean (default: `False`); whether to additionally return `string` with the optional tags removed but other comments kept and a boolean `is_persistent`

  EXAMPLES:

  Sage

      sage: from sage.doctest.parsing import parse_optional_tags
      sage: parse_optional_tags("sage: magma('2 + 2')# optional: magma")
      {'magma': None}
      sage: parse_optional_tags("sage: #optional -- mypkg")
      {'mypkg': None}
      sage: parse_optional_tags("sage: print(1)  # parentheses are optional here")
      {}
      sage: parse_optional_tags("sage: print(1)  # optional")
      {}
      sage: sorted(list(parse_optional_tags("sage: #optional -- foo bar, baz")))
      ['bar', 'foo']
      sage: parse_optional_tags("sage: #optional -- foo.bar, baz")
      {'foo.bar': None}
      sage: parse_optional_tags("sage: #needs foo.bar, baz")
      {'foo.bar': None}
      sage: sorted(list(parse_optional_tags("    sage: factor(10^(10^10) + 1) # LoNg TiME, NoT TeSTED; OptioNAL -- P4cka9e")))
      ['long time', 'not tested', 'p4cka9e']
      sage: parse_optional_tags("    sage: raise RuntimeError # known bug")
      {'bug': None}
      sage: sorted(list(parse_optional_tags("    sage: determine_meaning_of_life() # long time, not implemented")))
      ['long time', 'not implemented']

  Python

      >>> from sage.all import *
      >>> from sage.doctest.parsing import parse_optional_tags
      >>> parse_optional_tags("sage: magma('2 + 2')# optional: magma")
      {'magma': None}
      >>> parse_optional_tags("sage: #optional -- mypkg")
      {'mypkg': None}
      >>> parse_optional_tags("sage: print(1)  # parentheses are optional here")
      {}
      >>> parse_optional_tags("sage: print(1)  # optional")
      {}
      >>> sorted(list(parse_optional_tags("sage: #optional -- foo bar, baz")))
      ['bar', 'foo']
      >>> parse_optional_tags("sage: #optional -- foo.bar, baz")
      {'foo.bar': None}
      >>> parse_optional_tags("sage: #needs foo.bar, baz")
      {'foo.bar': None}
      >>> sorted(list(parse_optional_tags("    sage: factor(10^(10^10) + 1) # LoNg TiME, NoT TeSTED; OptioNAL -- P4cka9e")))
      ['long time', 'not tested', 'p4cka9e']
      >>> parse_optional_tags("    sage: raise RuntimeError # known bug")
      {'bug': None}
      >>> sorted(list(parse_optional_tags("    sage: determine_meaning_of_life() # long time, not implemented")))
      ['long time', 'not implemented']

  We don’t parse inside strings:

  Sage

      sage: parse_optional_tags("    sage: print('  # long time')")
      {}
      sage: parse_optional_tags("    sage: print('  # long time')  # not tested")
      {'not tested': None}

  [Python]

      >>> from sage.all import *
      >>> parse_optional_tags("    sage: print('  # long time')")
      {}
      >>> parse_optional_tags("    sage: print('  # long time')  # not tested")
      {'not tested': None}

  UTF-8 works:

  Sage

      sage: parse_optional_tags("'ěščřžýáíéďĎ'")
      {}

  Python

      >>> from sage.all import *
      >>> parse_optional_tags("'ěščřžýáíéďĎ'")
      {}

  Tags with parenthesized explanations:

  Sage

      sage: parse_optional_tags("    sage: 1 + 1  # long time (1 year, 2 months??), optional - bliss (because)")
      {'bliss': 'because', 'long time': '1 year, 2 months??'}

  [Python]

      >>> from sage.all import *
      >>> parse_optional_tags("    sage: 1 + 1  # long time (1 year, 2 months??), optional - bliss (because)")
      {'bliss': 'because', 'long time': '1 year, 2 months??'}

  With `return_string_sans_tags=True`:

  Sage

      sage: parse_optional_tags("sage: print(1)  # very important 1  # optional - foo",
      ....:                     return_string_sans_tags=True)
      ({'foo': None}, 'sage: print(1)  # very important 1  ', False)
      sage: parse_optional_tags("sage: print(    # very important too  # optional - foo\n....:     2)",
      ....:                     return_string_sans_tags=True)
      ({'foo': None}, 'sage: print(    # very important too  \n....:     2)', False)
      sage: parse_optional_tags("sage: #this is persistent #needs scipy",
      ....:                     return_string_sans_tags=True)
      ({'scipy': None}, 'sage: #this is persistent ', True)
      sage: parse_optional_tags("sage: #this is not #needs scipy\n....: import scipy",
      ....:                     return_string_sans_tags=True)
      ({'scipy': None}, 'sage: #this is not \n....: import scipy', False)

  Python

      >>> from sage.all import *
      >>> parse_optional_tags("sage: print(1)  # very important 1  # optional - foo",
      ...                     return_string_sans_tags=True)
      ({'foo': None}, 'sage: print(1)  # very important 1  ', False)
      >>> parse_optional_tags("sage: print(    # very important too  # optional - foo\n....:     2)",
      ...                     return_string_sans_tags=True)
      ({'foo': None}, 'sage: print(    # very important too  \n....:     2)', False)
      >>> parse_optional_tags("sage: #this is persistent #needs scipy",
      ...                     return_string_sans_tags=True)
      ({'scipy': None}, 'sage: #this is persistent ', True)
      >>> parse_optional_tags("sage: #this is not #needs scipy\n....: import scipy",
      ...                     return_string_sans_tags=True)
      ({'scipy': None}, 'sage: #this is not \n....: import scipy', False)

sage.doctest.parsing.parse_tolerance(source, want)

  Return a version of `want` marked up with the tolerance tags specified in `source`.

  INPUT:

  - `source` – string, the source of a doctest

  - `want` – string, the desired output of the doctest

  OUTPUT: `want` if there are no tolerance tags specified; a `MarkedOutput` version otherwise

  EXAMPLES:

  Sage

      sage: from sage.doctest.parsing import parse_tolerance
      sage: marked = parse_tolerance("sage: s.update(abs_tol = .0000001)", "")
      sage: type(marked)
      <class 'str'>
      sage: marked = parse_tolerance("sage: s.update(tol = 0.1); s.rel_tol # abs tol     0.01 ", "")
      sage: marked.tol
      0
      sage: marked.rel_tol
      0
      sage: marked.abs_tol
      0.010000000000000000000...?

  Python

      >>> from sage.all import *
      >>> from sage.doctest.parsing import parse_tolerance
      >>> marked = parse_tolerance("sage: s.update(abs_tol = .0000001)", "")
      >>> type(marked)
      <class 'str'>
      >>> marked = parse_tolerance("sage: s.update(tol = 0.1); s.rel_tol # abs tol     0.01 ", "")
      >>> marked.tol
      0
      >>> marked.rel_tol
      0
      >>> marked.abs_tol
      0.010000000000000000000...?

sage.doctest.parsing.pre_hash(s)

  Prepends a string with its length.

  EXAMPLES:

  Sage

      sage: from sage.doctest.parsing import pre_hash
      sage: pre_hash("abc")
      '3:abc'

  Python

      >>> from sage.all import *
      >>> from sage.doctest.parsing import pre_hash
      >>> pre_hash("abc")
      '3:abc'

sage.doctest.parsing.reduce_hex(fingerprints)

  Return a symmetric function of the arguments as hex strings.

  The arguments should be 32 character strings consisting of hex digits: 0-9 and a-f.

  EXAMPLES:

  Sage

      sage: from sage.doctest.parsing import reduce_hex
      sage: reduce_hex(["abc", "12399aedf"])
      '0000000000000000000000012399a463'
      sage: reduce_hex(["12399aedf","abc"])
      '0000000000000000000000012399a463'

  Python

      >>> from sage.all import *
      >>> from sage.doctest.parsing import reduce_hex
      >>> reduce_hex(["abc", "12399aedf"])
      '0000000000000000000000012399a463'
      >>> reduce_hex(["12399aedf","abc"])
      '0000000000000000000000012399a463'

sage.doctest.parsing.unparse_optional_tags(tags, prefix='# ')

  Return a comment string that sets `tags`.

  INPUT:

  - `tags` – dictionary or iterable of tags, as output by `parse_optional_tags()`

  - `prefix` – to be put before a nonempty string

  EXAMPLES:

  Sage

      sage: from sage.doctest.parsing import unparse_optional_tags
      sage: unparse_optional_tags({})
      ''
      sage: unparse_optional_tags({'magma': None})
      '# optional - magma'
      sage: unparse_optional_tags({'fictional_optional': None,
      ....:                        'sage.rings.number_field': None,
      ....:                        'scipy': 'just because',
      ....:                        'bliss': None})
      '# optional - bliss fictional_optional, needs scipy (just because) sage.rings.number_field'
      sage: unparse_optional_tags(['long time', 'not tested', 'p4cka9e'], prefix='')
      'long time, not tested, optional - p4cka9e'

  Python

      >>> from sage.all import *
      >>> from sage.doctest.parsing import unparse_optional_tags
      >>> unparse_optional_tags({})
      ''
      >>> unparse_optional_tags({'magma': None})
      '# optional - magma'
      >>> unparse_optional_tags({'fictional_optional': None,
      ...                        'sage.rings.number_field': None,
      ...                        'scipy': 'just because',
      ...                        'bliss': None})
      '# optional - bliss fictional_optional, needs scipy (just because) sage.rings.number_field'
      >>> unparse_optional_tags(['long time', 'not tested', 'p4cka9e'], prefix='')
      'long time, not tested, optional - p4cka9e'

sage.doctest.parsing.update_optional_tags(line, tags, add_tags, remove_tags, force_rewrite=None)

  Return the doctest `line` with tags changed.

  EXAMPLES:

  Sage

      sage: from sage.doctest.parsing import update_optional_tags, optional_tag_columns, standard_tag_columns
      sage: ruler = ''
      sage: for column in optional_tag_columns:
      ....:     ruler += ' ' * (column - len(ruler)) + 'V'
      sage: for column in standard_tag_columns:
      ....:     ruler += ' ' * (column - len(ruler)) + 'v'
      sage: def print_with_ruler(lines):
      ....:     print('|' + ruler)
      ....:     for line in lines:
      ....:         print('|' + line)
      sage: print_with_ruler([  # the tags are obscured in the source file to avoid relint warnings
      ....:     update_optional_tags('    sage: something()  # opt' 'ional - latte_int',
      ....:                          remove_tags=['latte_int', 'wasnt_even_there']),
      ....:     update_optional_tags('    sage: nothing_to_be_seen_here()',
      ....:                          tags=['scipy', 'long time']),
      ....:     update_optional_tags('    sage: nothing_to_be_seen_here(honestly=True)',
      ....:                          add_tags=['scipy', 'long time']),
      ....:     update_optional_tags('    sage: nothing_to_be_seen_here(honestly=True, very=True)',
      ....:                          add_tags=['scipy', 'long time']),
      ....:     update_optional_tags('    sage: no_there_is_absolutely_nothing_to_be_seen_here_i_am_serious()#opt' 'ional:bliss',
      ....:                          add_tags=['scipy', 'long time']),
      ....:     update_optional_tags('    sage: ntbsh()  # abbrv for above#opt' 'ional:bliss',
      ....:                          add_tags={'scipy': None, 'long time': '30s on the highest setting'}),
      ....:     update_optional_tags('    sage: no_there_is_absolutely_nothing_to_be_seen_here_i_am_serious()  # really, you can trust me here',
      ....:                          add_tags=['scipy']),
      ....: ])
      |                                                V       V       V       V       V   V   v           v                   v                                       v
      |    sage: something()
      |    sage: nothing_to_be_seen_here()             # long time                             # needs scipy
      |    sage: nothing_to_be_seen_here(honestly=True)        # long time                     # needs scipy
      |    sage: nothing_to_be_seen_here(honestly=True, very=True)     # long time             # needs scipy
      |    sage: no_there_is_absolutely_nothing_to_be_seen_here_i_am_serious()         # long time, optional - bliss, needs scipy
      |    sage: ntbsh()  # abbrv for above            # long time (30s on the highest setting), optional - bliss, needs scipy
      |    sage: no_there_is_absolutely_nothing_to_be_seen_here_i_am_serious()  # really, you can trust me here                # needs scipy

  Python

      >>> from sage.all import *
      >>> from sage.doctest.parsing import update_optional_tags, optional_tag_columns, standard_tag_columns
      >>> ruler = ''
      >>> for column in optional_tag_columns:
      ...     ruler += ' ' * (column - len(ruler)) + 'V'
      >>> for column in standard_tag_columns:
      ...     ruler += ' ' * (column - len(ruler)) + 'v'
      >>> def print_with_ruler(lines):
      ...     print('|' + ruler)
      ...     for line in lines:
      ...         print('|' + line)
      >>> print_with_ruler([  # the tags are obscured in the source file to avoid relint warnings
      ...     update_optional_tags('    sage: something()  # opt' 'ional - latte_int',
      ...                          remove_tags=['latte_int', 'wasnt_even_there']),
      ...     update_optional_tags('    sage: nothing_to_be_seen_here()',
      ...                          tags=['scipy', 'long time']),
      ...     update_optional_tags('    sage: nothing_to_be_seen_here(honestly=True)',
      ...                          add_tags=['scipy', 'long time']),
      ...     update_optional_tags('    sage: nothing_to_be_seen_here(honestly=True, very=True)',
      ...                          add_tags=['scipy', 'long time']),
      ...     update_optional_tags('    sage: no_there_is_absolutely_nothing_to_be_seen_here_i_am_serious()#opt' 'ional:bliss',
      ...                          add_tags=['scipy', 'long time']),
      ...     update_optional_tags('    sage: ntbsh()  # abbrv for above#opt' 'ional:bliss',
      ...                          add_tags={'scipy': None, 'long time': '30s on the highest setting'}),
      ...     update_optional_tags('    sage: no_there_is_absolutely_nothing_to_be_seen_here_i_am_serious()  # really, you can trust me here',
      ...                          add_tags=['scipy']),
      ... ])
      |                                                V       V       V       V       V   V   v           v                   v                                       v
      |    sage: something()
      |    sage: nothing_to_be_seen_here()             # long time                             # needs scipy
      |    sage: nothing_to_be_seen_here(honestly=True)        # long time                     # needs scipy
      |    sage: nothing_to_be_seen_here(honestly=True, very=True)     # long time             # needs scipy
      |    sage: no_there_is_absolutely_nothing_to_be_seen_here_i_am_serious()         # long time, optional - bliss, needs scipy
      |    sage: ntbsh()  # abbrv for above            # long time (30s on the highest setting), optional - bliss, needs scipy
      |    sage: no_there_is_absolutely_nothing_to_be_seen_here_i_am_serious()  # really, you can trust me here                # needs scipy

  When no tags are changed, by default, the unchanged input is returned. We can force a rewrite; unconditionally or whenever standard tags are involved. But even when forced, if comments are already aligned at one of the standard alignment columns, this alignment is kept even if we would normally realign farther to the left:

  Sage

      sage: print_with_ruler([
      ....:     update_optional_tags('    sage: unforced()       # opt' 'ional - latte_int'),
      ....:     update_optional_tags('    sage: unforced()  # opt' 'ional - latte_int',
      ....:                          add_tags=['latte_int']),
      ....:     update_optional_tags('    sage: forced()#opt' 'ional- latte_int',
      ....:                          force_rewrite=True),
      ....:     update_optional_tags('    sage: forced()  # opt' 'ional - scipy',
      ....:                          force_rewrite='standard'),
      ....:     update_optional_tags('    sage: aligned_with_below()                                  # opt' 'ional - 4ti2',
      ....:                          force_rewrite=True),
      ....:     update_optional_tags('    sage: aligned_with_above()                                  # opt' 'ional - 4ti2',
      ....:                          force_rewrite=True),
      ....:     update_optional_tags('    sage: also_already_aligned()                                                                                        # ne' 'eds scipy',
      ....:                          force_rewrite='standard'),
      ....:     update_optional_tags('    sage: two_columns_first_preserved()         # lo' 'ng time                             # ne' 'eds scipy',
      ....:                          force_rewrite='standard'),
      ....:     update_optional_tags('    sage: two_columns_first_preserved()                 # lo' 'ng time                                 # ne' 'eds scipy',
      ....:                          force_rewrite='standard'),
      ....: ])
      |                                                V       V       V       V       V   V   v           v                   v                                       v
      |    sage: unforced()       # optional - latte_int
      |    sage: unforced()  # optional - latte_int
      |    sage: forced()                              # optional - latte_int
      |    sage: forced()                                                                      # needs scipy
      |    sage: aligned_with_below()                                  # optional - 4ti2
      |    sage: aligned_with_above()                                  # optional - 4ti2
      |    sage: also_already_aligned()                                                                                        # needs scipy
      |    sage: two_columns_first_preserved()         # long time                             # needs scipy
      |    sage: two_columns_first_preserved()                 # long time                     # needs scipy

  [Python]

      >>> from sage.all import *
      >>> print_with_ruler([
      ...     update_optional_tags('    sage: unforced()       # opt' 'ional - latte_int'),
      ...     update_optional_tags('    sage: unforced()  # opt' 'ional - latte_int',
      ...                          add_tags=['latte_int']),
      ...     update_optional_tags('    sage: forced()#opt' 'ional- latte_int',
      ...                          force_rewrite=True),
      ...     update_optional_tags('    sage: forced()  # opt' 'ional - scipy',
      ...                          force_rewrite='standard'),
      ...     update_optional_tags('    sage: aligned_with_below()                                  # opt' 'ional - 4ti2',
      ...                          force_rewrite=True),
      ...     update_optional_tags('    sage: aligned_with_above()                                  # opt' 'ional - 4ti2',
      ...                          force_rewrite=True),
      ...     update_optional_tags('    sage: also_already_aligned()                                                                                        # ne' 'eds scipy',
      ...                          force_rewrite='standard'),
      ...     update_optional_tags('    sage: two_columns_first_preserved()         # lo' 'ng time                             # ne' 'eds scipy',
      ...                          force_rewrite='standard'),
      ...     update_optional_tags('    sage: two_columns_first_preserved()                 # lo' 'ng time                                 # ne' 'eds scipy',
      ...                          force_rewrite='standard'),
      ... ])
      |                                                V       V       V       V       V   V   v           v                   v                                       v
      |    sage: unforced()       # optional - latte_int
      |    sage: unforced()  # optional - latte_int
      |    sage: forced()                              # optional - latte_int
      |    sage: forced()                                                                      # needs scipy
      |    sage: aligned_with_below()                                  # optional - 4ti2
      |    sage: aligned_with_above()                                  # optional - 4ti2
      |    sage: also_already_aligned()                                                                                        # needs scipy
      |    sage: two_columns_first_preserved()         # long time                             # needs scipy
      |    sage: two_columns_first_preserved()                 # long time                     # needs scipy

  Rewriting a persistent (block-scoped) tag:

  Sage

      sage: print_with_ruler([
      ....:     update_optional_tags('    sage:    #opt' 'ional:magma sage.symbolic',
      ....:                          force_rewrite=True),
      ....: ])
      |                                                V       V       V       V       V   V   v           v                   v                                       v
      |    sage: # optional - magma, needs sage.symbolic

  Python

      >>> from sage.all import *
      >>> print_with_ruler([
      ...     update_optional_tags('    sage:    #opt' 'ional:magma sage.symbolic',
      ...                          force_rewrite=True),
      ... ])
      |                                                V       V       V       V       V   V   v           v                   v                                       v
      |    sage: # optional - magma, needs sage.symbolic