the last call to :c:func:`Py_Initialize`. Ideally, this frees all memory
allocated by the Python interpreter. This is a no-op when called for a second
time (without calling :c:func:`Py_Initialize` again first). Normally the
- return value is 0. If there were errors during finalization
- (flushing buffered data), -1 is returned.
+ return value is ``0``. If there were errors during finalization
+ (flushing buffered data), ``-1`` is returned.
This function is provided for a number of reasons. An embedding application
might want to restart Python without having to restart the application itself.
+------------------------------+--------------------------------------+
| Value of *what* | Meaning of *arg* |
+==============================+======================================+
- | :const:`PyTrace_CALL` | Always *NULL*. |
+ | :const:`PyTrace_CALL` | Always :c:data:`Py_None`. |
+------------------------------+--------------------------------------+
| :const:`PyTrace_EXCEPTION` | Exception information as returned by |
| | :func:`sys.exc_info`. |
+------------------------------+--------------------------------------+
- | :const:`PyTrace_LINE` | Always *NULL*. |
+ | :const:`PyTrace_LINE` | Always :c:data:`Py_None`. |
+------------------------------+--------------------------------------+
| :const:`PyTrace_RETURN` | Value being returned to the caller, |
| | or *NULL* if caused by an exception. |
.. c:var:: int PyTrace_RETURN
The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a
- call is returning without propagating an exception.
+ call is about to return.
.. c:var:: int PyTrace_C_CALL
function as its first parameter, and may be any Python object, or *NULL*. If
the profile function needs to maintain state, using a different value for *obj*
for each thread provides a convenient and thread-safe place to store it. The
- profile function is called for all monitored events except the line-number
- events.
+ profile function is called for all monitored events except :const:`PyTrace_LINE`
+ and :const:`PyTrace_EXCEPTION`.
.. c:function:: void PyEval_SetTrace(Py_tracefunc func, PyObject *obj)
Set the tracing function to *func*. This is similar to
:c:func:`PyEval_SetProfile`, except the tracing function does receive line-number
- events.
+ events and does not receive any event related to C function objects being called. Any
+ trace function registered using :c:func:`PyEval_SetTrace` will not receive
+ :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION` or :const:`PyTrace_C_RETURN`
+ as a value for the *what* parameter.
+
.. c:function:: PyObject* PyEval_GetCallStats(PyObject *self)
All integers are implemented as "long" integer objects of arbitrary size.
+On error, most ``PyLong_As*`` APIs return ``(return type)-1`` which cannot be
+distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate.
+
.. c:type:: PyLongObject
This subtype of :c:type:`PyObject` represents a Python integer object.
Raise :exc:`OverflowError` if the value of *obj* is out of range for a
:c:type:`long`.
+ Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate.
+
.. c:function:: long PyLong_AsLongAndOverflow(PyObject *obj, int *overflow)
return ``-1``; otherwise, set *\*overflow* to ``0``. If any other exception
occurs set *\*overflow* to ``0`` and return ``-1`` as usual.
+ Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate.
+
.. c:function:: long long PyLong_AsLongLong(PyObject *obj)
Raise :exc:`OverflowError` if the value of *obj* is out of range for a
:c:type:`long`.
+ Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate.
+
.. c:function:: long long PyLong_AsLongLongAndOverflow(PyObject *obj, int *overflow)
and return ``-1``; otherwise, set *\*overflow* to ``0``. If any other
exception occurs set *\*overflow* to ``0`` and return ``-1`` as usual.
+ Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate.
+
.. versionadded:: 3.2
Raise :exc:`OverflowError` if the value of *pylong* is out of range for a
:c:type:`Py_ssize_t`.
+ Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate.
+
.. c:function:: unsigned long PyLong_AsUnsignedLong(PyObject *pylong)
Raise :exc:`OverflowError` if the value of *pylong* is out of range for a
:c:type:`unsigned long`.
+ Returns ``(unsigned long)-1`` on error.
+ Use :c:func:`PyErr_Occurred` to disambiguate.
+
.. c:function:: size_t PyLong_AsSize_t(PyObject *pylong)
+ .. index::
+ single: SIZE_MAX
+ single: OverflowError (built-in exception)
+
Return a C :c:type:`size_t` representation of *pylong*. *pylong* must be
an instance of :c:type:`PyLongObject`.
Raise :exc:`OverflowError` if the value of *pylong* is out of range for a
:c:type:`size_t`.
+ Returns ``(size_t)-1`` on error.
+ Use :c:func:`PyErr_Occurred` to disambiguate.
+
.. c:function:: unsigned long long PyLong_AsUnsignedLongLong(PyObject *pylong)
Raise :exc:`OverflowError` if the value of *pylong* is out of range for an
:c:type:`unsigned long long`.
+ Returns ``(unsigned long long)-1`` on error.
+ Use :c:func:`PyErr_Occurred` to disambiguate.
+
.. versionchanged:: 3.1
A negative *pylong* now raises :exc:`OverflowError`, not :exc:`TypeError`.
If the value of *obj* is out of range for an :c:type:`unsigned long`,
return the reduction of that value modulo ``ULONG_MAX + 1``.
+ Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate.
+
.. c:function:: unsigned long long PyLong_AsUnsignedLongLongMask(PyObject *obj)
If the value of *obj* is out of range for an :c:type:`unsigned long long`,
return the reduction of that value modulo ``PY_ULLONG_MAX + 1``.
+ Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate.
+
.. c:function:: double PyLong_AsDouble(PyObject *pylong)
Raise :exc:`OverflowError` if the value of *pylong* is out of range for a
:c:type:`double`.
+ Returns ``-1.0`` on error. Use :c:func:`PyErr_Occurred` to disambiguate.
+
.. c:function:: void* PyLong_AsVoidPtr(PyObject *pylong)
If *pylong* cannot be converted, an :exc:`OverflowError` will be raised. This
is only assured to produce a usable :c:type:`void` pointer for values created
with :c:func:`PyLong_FromVoidPtr`.
+
+ Returns *NULL* on error. Use :c:func:`PyErr_Occurred` to disambiguate.
.. index:: builtin: len
- Returns the number of objects in sequence *o* on success, and ``-1`` on failure.
- For objects that do not provide sequence protocol, this is equivalent to the
- Python expression ``len(o)``.
+ Returns the number of objects in sequence *o* on success, and ``-1`` on
+ failure. This is equivalent to the Python expression ``len(o)``.
.. c:function:: PyObject* PySequence_Concat(PyObject *o1, PyObject *o2)
surrogate character, escape the bytes using the surrogateescape error
handler instead of decoding them.
+ Encoding, highest priority to lowest priority:
+
+ * ``UTF-8`` on macOS and Android;
+ * ``ASCII`` if the ``LC_CTYPE`` locale is ``"C"``,
+ ``nl_langinfo(CODESET)`` returns the ``ASCII`` encoding (or an alias),
+ and :c:func:`mbstowcs` and :c:func:`wcstombs` functions use the
+ ``ISO-8859-1`` encoding.
+ * the current locale encoding (``LC_CTYPE`` locale).
+
Return a pointer to a newly allocated wide character string, use
:c:func:`PyMem_RawFree` to free the memory. If size is not ``NULL``, write
- the number of wide characters excluding the null character into ``*size``
+ the number of wide characters excluding the null character into ``*size``.
Return ``NULL`` on decoding error or memory allocation error. If *size* is
not ``NULL``, ``*size`` is set to ``(size_t)-1`` on memory error or set to
:ref:`surrogateescape error handler <surrogateescape>`: surrogate characters
in the range U+DC80..U+DCFF are converted to bytes 0x80..0xFF.
+ Encoding, highest priority to lowest priority:
+
+ * ``UTF-8`` on macOS and Android;
+ * ``ASCII`` if the ``LC_CTYPE`` locale is ``"C"``,
+ ``nl_langinfo(CODESET)`` returns the ``ASCII`` encoding (or an alias),
+ and :c:func:`mbstowcs` and :c:func:`wcstombs` functions uses the
+ ``ISO-8859-1`` encoding.
+ * the current locale encoding.
+
Return a pointer to a newly allocated byte string, use :c:func:`PyMem_Free`
to free the memory. Return ``NULL`` on encoding error or memory allocation
error
.. versionadded:: 3.3
+ .. versionchanged:: 3.6.5
+ The function now also uses the current locale encoding for the
+ ``surrogateescape`` error handler. Previously, :c:func:`Py_DecodeLocale`
+ was used for the ``surrogateescape``, and the current locale encoding was
+ used for ``strict``.
+
.. c:function:: PyObject* PyUnicode_DecodeLocale(const char *str, const char *errors)
.. versionadded:: 3.3
+ .. versionchanged:: 3.6.5
+ The function now also uses the current locale encoding for the
+ ``surrogateescape`` error handler. Previously, :c:func:`Py_EncodeLocale`
+ was used for the ``surrogateescape``, and the current locale encoding was
+ used for ``strict``.
+
File System Encoding
""""""""""""""""""""
Python and this documentation is:
-Copyright © 2001-2017 Python Software Foundation. All rights reserved.
+Copyright © 2001-2018 Python Software Foundation. All rights reserved.
Copyright © 2000 BeOpen.com. All rights reserved.
can't guarantee that the initializer is called. The initializer isn't called
when unpickling objects and it can be overridden. Our initializer accepts
arguments to provide initial values for our instance. Initializers always accept
-positional and keyword arguments. Initializers should return either 0 on
-success or -1 on error.
+positional and keyword arguments. Initializers should return either ``0`` on
+success or ``-1`` on error.
Initializers can be called multiple times. Anyone can call the :meth:`__init__`
method on our objects. For this reason, we have to be extra careful when
<https://riverbankcomputing.com/software/sip/intro>`__, `CXX
<http://cxx.sourceforge.net/>`_ `Boost
<http://www.boost.org/libs/python/doc/index.html>`_, or `Weave
-<https://scipy.github.io/devdocs/tutorial/weave.html>`_ are also
+<https://github.com/scipy/weave>`_ are also
alternatives for wrapping C++ libraries.
The highest-level function to do this is :c:func:`PyRun_SimpleString` which takes
a single string argument to be executed in the context of the module
-``__main__`` and returns 0 for success and -1 when an exception occurred
+``__main__`` and returns ``0`` for success and ``-1`` when an exception occurred
(including ``SyntaxError``). If you want more control, use
:c:func:`PyRun_String`; see the source for :c:func:`PyRun_SimpleString` in
``Python/pythonrun.c``.
garbage collection
The process of freeing memory when it is not used anymore. Python
performs garbage collection via reference counting and a cyclic garbage
- collector that is able to detect and break reference cycles.
+ collector that is able to detect and break reference cycles. The
+ garbage collector can be controlled using the :mod:`gc` module.
.. index:: single: generator
-------------------
.. sectionauthor:: Doug Hellmann, Vinay Sajip (changes)
-.. (see <http://blog.doughellmann.com/2007/05/pymotw-logging.html>)
+.. (see <https://pymotw.com/3/logging/>)
Sometimes you want to let a log file grow to a certain size, then open a new
file and log to that. You may want to keep a certain number of these files, and
disadvantage which is the topic of the next section.
+.. _the-backslash-plague:
+
The Backslash Plague
--------------------
while ``"\n"`` is a one-character string containing a newline. Regular
expressions will often be written in Python code using this raw string notation.
+In addition, special escape sequences that are valid in regular expressions,
+but not valid as Python string literals, now result in a
+:exc:`DeprecationWarning` and will eventually become a :exc:`SyntaxError`,
+which means the sequences will be invalid if raw string notation or escaping
+the backslashes isn't used.
+
+
+-------------------+------------------+
| Regular String | Raw string |
+===================+==================+
Two pattern methods return all of the matches for a pattern.
:meth:`~re.pattern.findall` returns a list of matching strings::
- >>> p = re.compile('\d+')
+ >>> p = re.compile(r'\d+')
>>> p.findall('12 drummers drumming, 11 pipers piping, 10 lords a-leaping')
['12', '11', '10']
-:meth:`~re.pattern.findall` has to create the entire list before it can be returned as the
-result. The :meth:`~re.pattern.finditer` method returns a sequence of
+The ``r`` prefix, making the literal a raw string literal, is needed in this
+example because escape sequences in a normal "cooked" string literal that are
+not recognized by Python, as opposed to regular expressions, now result in a
+:exc:`DeprecationWarning` and will eventually become a :exc:`SyntaxError`. See
+:ref:`the-backslash-plague`.
+
+:meth:`~re.Pattern.findall` has to create the entire list before it can be returned as the
+result. The :meth:`~re.Pattern.finditer` method returns a sequence of
:ref:`match object <match-objects>` instances as an :term:`iterator`::
>>> iterator = p.finditer('12 drummers drumming, 11 ... 10 ...')
The module-level function :func:`re.split` adds the RE to be used as the first
argument, but is otherwise the same. ::
- >>> re.split('[\W]+', 'Words, words, words.')
+ >>> re.split(r'[\W]+', 'Words, words, words.')
['Words', 'words', 'words', '']
- >>> re.split('([\W]+)', 'Words, words, words.')
+ >>> re.split(r'([\W]+)', 'Words, words, words.')
['Words', ', ', 'words', ', ', 'words', '.', '']
- >>> re.split('[\W]+', 'Words, words, words.', 1)
+ >>> re.split(r'[\W]+', 'Words, words, words.', 1)
['Words', 'words, words.']
Arabic numerals::
import re
- p = re.compile('\d+')
+ p = re.compile(r'\d+')
s = "Over \u0e55\u0e57 57 flavours"
m = p.search(s)
print('From:', msg['from'])
print('Subject:', msg['subject'])
-# If we want to print a priview of the message content, we can extract whatever
+# If we want to print a preview of the message content, we can extract whatever
# the least formatted payload is and print the first three lines. Of course,
# if the message has no plain text part printing the first three lines of html
# is probably useless, but this is just a conceptual example.
-The :mod:`abc` module also provides the following decorators:
+The :mod:`abc` module also provides the following decorator:
.. decorator:: abstractmethod
multiple-inheritance.
+The :mod:`abc` module also supports the following legacy decorators:
+
.. decorator:: abstractclassmethod
+ .. versionadded:: 3.2
+ .. deprecated:: 3.3
+ It is now possible to use :class:`classmethod` with
+ :func:`abstractmethod`, making this decorator redundant.
+
A subclass of the built-in :func:`classmethod`, indicating an abstract
classmethod. Otherwise it is similar to :func:`abstractmethod`.
def my_abstract_classmethod(cls, ...):
...
+
+.. decorator:: abstractstaticmethod
+
.. versionadded:: 3.2
.. deprecated:: 3.3
- It is now possible to use :class:`classmethod` with
+ It is now possible to use :class:`staticmethod` with
:func:`abstractmethod`, making this decorator redundant.
-
-.. decorator:: abstractstaticmethod
-
A subclass of the built-in :func:`staticmethod`, indicating an abstract
staticmethod. Otherwise it is similar to :func:`abstractmethod`.
def my_abstract_staticmethod(...):
...
- .. versionadded:: 3.2
- .. deprecated:: 3.3
- It is now possible to use :class:`staticmethod` with
- :func:`abstractmethod`, making this decorator redundant.
-
.. decorator:: abstractproperty
+ .. deprecated:: 3.3
+ It is now possible to use :class:`property`, :meth:`property.getter`,
+ :meth:`property.setter` and :meth:`property.deleter` with
+ :func:`abstractmethod`, making this decorator redundant.
+
A subclass of the built-in :func:`property`, indicating an abstract
property.
- Using this function requires that the class's metaclass is :class:`ABCMeta`
- or is derived from it. A class that has a metaclass derived from
- :class:`ABCMeta` cannot be instantiated unless all of its abstract methods
- and properties are overridden. The abstract properties can be called using
- any of the normal 'super' call mechanisms.
-
This special case is deprecated, as the :func:`property` decorator
is now correctly identified as abstract when applied to an abstract
method::
...
- .. deprecated:: 3.3
- It is now possible to use :class:`property`, :meth:`property.getter`,
- :meth:`property.setter` and :meth:`property.deleter` with
- :func:`abstractmethod`, making this decorator redundant.
-
-
The :mod:`abc` module also provides the following functions:
.. function:: get_cache_token()
Parse the source into an AST node. Equivalent to ``compile(source,
filename, mode, ast.PyCF_ONLY_AST)``.
+ .. warning::
+ It is possible to crash the Python interpreter with a
+ sufficiently large/complex string due to stack depth limitations
+ in Python's AST compiler.
+
.. function:: literal_eval(node_or_string)
capable of evaluating arbitrarily complex expressions, for example involving
operators or indexing.
+ .. warning::
+ It is possible to crash the Python interpreter with a
+ sufficiently large/complex string due to stack depth limitations
+ in Python's AST compiler.
+
.. versionchanged:: 3.2
Now allows bytes and set literals.
This function is a :ref:`coroutine <coroutine>`.
-.. function:: shield(arg, \*, loop=None)
+.. coroutinefunction:: shield(arg, \*, loop=None)
Wait for a future, shielding it from cancellation.
.. versionadded:: 3.4
-.. note::
- Both Base85 and Ascii85 have an expansion factor of 5 to 4 (5 Base85 or
- Ascii85 characters can encode 4 binary bytes), while the better-known
- Base64 has an expansion factor of 6 to 4. They are therefore more
- efficient when space expensive. They differ by details such as the
- character map used for encoding.
-
-
The legacy interface:
.. function:: decode(input, output)
Some common encodings can bypass the codecs lookup machinery to
improve performance. These optimization opportunities are only
- recognized by CPython for a limited set of aliases: utf-8, utf8,
- latin-1, latin1, iso-8859-1, mbcs (Windows only), ascii, utf-16,
- and utf-32. Using alternative spellings for these encodings may
- result in slower execution.
+ recognized by CPython for a limited set of (case insensitive)
+ aliases: utf-8, utf8, latin-1, latin1, iso-8859-1, iso8859-1, mbcs
+ (Windows only), ascii, us-ascii, utf-16, utf16, utf-32, utf32, and
+ the same using underscores instead of dashes. Using alternative
+ aliases for these encodings may result in slower execution.
+
+ .. versionchanged:: 3.6
+ Optimization opportunity recognized for us-ascii.
Many of the character sets support the same languages. They vary in individual
characters (e.g. whether the EURO SIGN is supported or not), and in the
.. versionadded:: 3.2
- .. method:: rotate(n)
+ .. method:: rotate(n=1)
- Rotate the deque *n* steps to the right. If *n* is negative, rotate to
- the left. Rotating one step to the right is equivalent to:
- ``d.appendleft(d.pop())``.
+ Rotate the deque *n* steps to the right. If *n* is negative, rotate
+ to the left.
+
+ When the deque is not empty, rotating one step to the right is equivalent
+ to ``d.appendleft(d.pop())``, and rotating one step to the left is
+ equivalent to ``d.append(d.popleft())``.
Deque objects also provide one read-only attribute:
.. method:: map(func, *iterables, timeout=None, chunksize=1)
- Equivalent to :func:`map(func, *iterables) <map>` except *func* is executed
- asynchronously and several calls to *func* may be made concurrently. The
- returned iterator raises a :exc:`concurrent.futures.TimeoutError` if
- :meth:`~iterator.__next__` is called and the result isn't available
+ Similar to :func:`map(func, *iterables) <map>` except:
+
+ * the *iterables* are collected immediately rather than lazily;
+
+ * *func* is executed asynchronously and several calls to
+ *func* may be made concurrently.
+
+ The returned iterator raises a :exc:`concurrent.futures.TimeoutError`
+ if :meth:`~iterator.__next__` is called and the result isn't available
after *timeout* seconds from the original call to :meth:`Executor.map`.
*timeout* can be an int or a float. If *timeout* is not specified or
- ``None``, there is no limit to the wait time. If a call raises an
- exception, then that exception will be raised when its value is
- retrieved from the iterator. When using :class:`ProcessPoolExecutor`, this
- method chops *iterables* into a number of chunks which it submits to the
- pool as separate tasks. The (approximate) size of these chunks can be
- specified by setting *chunksize* to a positive integer. For very long
- iterables, using a large value for *chunksize* can significantly improve
- performance compared to the default size of 1. With :class:`ThreadPoolExecutor`,
- *chunksize* has no effect.
+ ``None``, there is no limit to the wait time.
+
+ If a *func* call raises an exception, then that exception will be
+ raised when its value is retrieved from the iterator.
+
+ When using :class:`ProcessPoolExecutor`, this method chops *iterables*
+ into a number of chunks which it submits to the pool as separate
+ tasks. The (approximate) size of these chunks can be specified by
+ setting *chunksize* to a positive integer. For very long iterables,
+ using a large value for *chunksize* can significantly improve
+ performance compared to the default size of 1. With
+ :class:`ThreadPoolExecutor`, *chunksize* has no effect.
.. versionchanged:: 3.5
Added the *chunksize* argument.
specified exit code.
.. data:: copyright
- license
credits
- Objects that when printed, print a message like "Type license() to see the
- full license text", and when called, display the corresponding text in a
+ Objects that when printed or called, print the text of copyright or
+ credits, respectively.
+
+.. data:: license
+
+ Object that when printed, prints the message "Type license() to see the
+ full license text", and when called, displays the full license text in a
pager-like fashion (one screen at a time).
Duplicates the reference on top of the stack.
+ .. versionadded:: 3.2
+
.. opcode:: DUP_TOP_TWO
Duplicates the two references on top of the stack, leaving them in the
same order.
+ .. versionadded:: 3.2
+
**Unary operations**
the CO_ITERABLE_COROUTINE flag, or resolves
``o.__await__``.
+ .. versionadded:: 3.5
+
.. opcode:: GET_AITER
Implements ``TOS = get_awaitable(TOS.__aiter__())``. See ``GET_AWAITABLE``
for details about ``get_awaitable``
+ .. versionadded:: 3.5
+
.. opcode:: GET_ANEXT
Implements ``PUSH(get_awaitable(TOS.__anext__()))``. See ``GET_AWAITABLE``
for details about ``get_awaitable``
+ .. versionadded:: 3.5
+
.. opcode:: BEFORE_ASYNC_WITH
Resolves ``__aenter__`` and ``__aexit__`` from the object on top of the
stack. Pushes ``__aexit__`` and result of ``__aenter__()`` to the stack.
+ .. versionadded:: 3.5
+
.. opcode:: SETUP_ASYNC_WITH
Creates a new frame object.
+ .. versionadded:: 3.5
+
**Miscellaneous opcodes**
Calls ``dict.setitem(TOS1[-i], TOS, TOS1)``. Used to implement dict
comprehensions.
+ .. versionadded:: 3.1
+
For all of the :opcode:`SET_ADD`, :opcode:`LIST_APPEND` and :opcode:`MAP_ADD`
instructions, while the added value or key/value pair is popped off, the
container object remains on the stack so that it is available for further
.. versionadded:: 3.3
+
.. opcode:: SETUP_ANNOTATIONS
Checks whether ``__annotations__`` is defined in ``locals()``, if not it is
.. versionadded:: 3.6
+
.. opcode:: IMPORT_STAR
Loads all symbols not starting with ``'_'`` directly from the module TOS to
store it in (a) variable(s) (:opcode:`STORE_FAST`, :opcode:`STORE_NAME`, or
:opcode:`UNPACK_SEQUENCE`).
+ .. versionadded:: 3.2
+
.. opcode:: WITH_CLEANUP_START
If TOS is true, sets the bytecode counter to *target*. TOS is popped.
+ .. versionadded:: 3.1
+
.. opcode:: POP_JUMP_IF_FALSE (target)
If TOS is false, sets the bytecode counter to *target*. TOS is popped.
+ .. versionadded:: 3.1
+
.. opcode:: JUMP_IF_TRUE_OR_POP (target)
If TOS is true, sets the bytecode counter to *target* and leaves TOS on the
stack. Otherwise (TOS is false), TOS is popped.
+ .. versionadded:: 3.1
+
.. opcode:: JUMP_IF_FALSE_OR_POP (target)
If TOS is false, sets the bytecode counter to *target* and leaves TOS on the
stack. Otherwise (TOS is true), TOS is popped.
+ .. versionadded:: 3.1
+
.. opcode:: JUMP_ABSOLUTE (target)
consulting the cell. This is used for loading free variables in class
bodies.
+ .. versionadded:: 3.4
+
.. opcode:: STORE_DEREF (i)
Empties the cell contained in slot *i* of the cell and free variable storage.
Used by the :keyword:`del` statement.
+ .. versionadded:: 3.2
+
.. opcode:: RAISE_VARARGS (argc)
Like :class:`IntFlag`, if a combination of :class:`Flag` members results in no
flags being set, the boolean evaluation is :data:`False`::
- >>> from enum import Flag
+ >>> from enum import Flag, auto
>>> class Color(Flag):
... RED = auto()
... BLUE = auto()
character. This is to facilitate detection of incomplete and complete
statements in the :mod:`code` module.
+ .. warning::
+
+ It is possible to crash the Python interpreter with a
+ sufficiently large/complex string when compiling to an AST
+ object due to stack depth limitations in Python's AST compiler.
+
.. versionchanged:: 3.2
Allowed use of Windows and Mac newlines. Also input in ``'exec'`` mode
does not have to end in a newline anymore. Added the *optimize* parameter.
Return an integer object constructed from a number or string *x*, or return
``0`` if no arguments are given. If *x* is a number, return
- :meth:`x.__int__() <object.__int__>`. For floating point numbers, this
- truncates towards zero.
+ :meth:`x.__int__() <object.__int__>`. If *x* defines
+ :meth:`x.__trunc__() <object.__trunc__>` but not
+ :meth:`x.__int__() <object.__int__>`, then return
+ if :meth:`x.__trunc__() <object.__trunc__>`. For floating point numbers,
+ this truncates towards zero.
If *x* is not a number or if *base* is given, then *x* must be a string,
:class:`bytes`, or :class:`bytearray` instance representing an :ref:`integer
a regular function and do something with its result. This is needed
in some cases where you need a reference to a function from a class
body and you want to avoid the automatic transformation to instance
- method. For these cases, use this idiom:
+ method. For these cases, use this idiom::
class C:
builtin_open = staticmethod(open)
Return the "login name" of the user.
This function checks the environment variables :envvar:`LOGNAME`,
- :envvar:`USER`, :envvar:`LNAME` and :envvar:`USERNAME`, in order, and returns
- the value of the first one which is set to a non-empty string. If none are set,
- the login name from the password database is returned on systems which support
- the :mod:`pwd` module, otherwise, an exception is raised.
+ :envvar:`USER`, :envvar:`LNAME` and :envvar:`USERNAME`, in order, and
+ returns the value of the first one which is set to a non-empty string. If
+ none are set, the login name from the password database is returned on
+ systems which support the :mod:`pwd` module, otherwise, an exception is
+ raised.
+ In general, this function should be preferred over :func:`os.getlogin()`.
Support for recursive globs using "``**``".
-.. function:: iglob(pathname, recursive=False)
+.. function:: iglob(pathname, *, recursive=False)
Return an :term:`iterator` which yields the same values as :func:`glob`
without actually storing them all simultaneously.
``writable``, and ``writelines``
:class:`RawIOBase` :class:`IOBase` ``readinto`` and Inherited :class:`IOBase` methods, ``read``,
``write`` and ``readall``
-:class:`BufferedIOBase` :class:`IOBase` ``detach``, ``read``, Inherited :class:`IOBase` methods, ``readinto``
- ``read1``, and ``write``
+:class:`BufferedIOBase` :class:`IOBase` ``detach``, ``read``, Inherited :class:`IOBase` methods, ``readinto``,
+ ``read1``, and ``write`` and ``readinto1``
:class:`TextIOBase` :class:`IOBase` ``detach``, ``read``, Inherited :class:`IOBase` methods, ``encoding``,
``readline``, and ``errors``, and ``newlines``
``write``
.. method:: read(size=-1)
Read up to *size* bytes from the object and return them. As a convenience,
- if *size* is unspecified or -1, :meth:`readall` is called. Otherwise,
- only one system call is ever made. Fewer than *size* bytes may be
- returned if the operating system call returns fewer than *size* bytes.
+ if *size* is unspecified or -1, all bytes until EOF are returned.
+ Otherwise, only one system call is ever made. Fewer than *size* bytes may
+ be returned if the operating system call returns fewer than *size* bytes.
If 0 bytes are returned, and *size* was not 0, this indicates end of file.
If the object is in non-blocking mode and no bytes are available,
``None`` is returned.
+ The default implementation defers to :meth:`readall` and
+ :meth:`readinto`.
+
.. method:: readall()
Read and return all the bytes from the stream until EOF, using multiple
groups consisting entirely of zeroes included.
- For the following attributes, see the corresponding documention of the
+ For the following attributes, see the corresponding documentation of the
:class:`IPv4Address` class:
.. attribute:: packed
.. attribute:: hostmask
- The host mask, as a string.
+ The host mask, as an :class:`IPv4Address` object.
+
+ .. attribute:: netmask
+
+ The net mask, as an :class:`IPv4Address` object.
.. attribute:: with_prefixlen
.. attribute:: compressed
1. A string consisting of an IP address and an optional mask, separated by
a slash (``/``). The IP address is the network address, and the mask
- can be either a single number, which means it's a *prefix*, or a string
- representation of an IPv6 address. If it's the latter, the mask is
- interpreted as a *net mask*. If no mask is provided, it's considered to
- be ``/128``.
+ is a single number, which represents a *prefix*. If no mask is provided,
+ it's considered to be ``/128``.
- For example, the following *address* specifications are equivalent:
- ``2001:db00::0/24`` and ``2001:db00::0/ffff:ff00::``.
+ Note that currently expanded netmasks are not supported. That means
+ ``2001:db00::0/24`` is a valid argument while ``2001:db00::0/ffff:ff00::``
+ not.
2. An integer that fits into 128 bits. This is equivalent to a
single-address network, with the network address being *address* and
.. attribute:: network_address
.. attribute:: broadcast_address
.. attribute:: hostmask
+ .. attribute:: netmask
.. attribute:: with_prefixlen
.. attribute:: compressed
.. attribute:: exploded
``sum(map(operator.mul, vector1, vector2))``.
-**Infinite Iterators:**
+**Infinite iterators:**
================== ================= ================================================= =========================================
Iterator Arguments Results Example
:func:`zip_longest` p, q, ... (p[0], q[0]), (p[1], q[1]), ... ``zip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-``
============================ ============================ ================================================= =============================================================
-**Combinatoric generators:**
+**Combinatoric iterators:**
============================================== ==================== =============================================================
Iterator Arguments Results
indices = sorted(random.randrange(n) for i in range(r))
return tuple(pool[i] for i in indices)
+ def nth_combination(iterable, r, index):
+ 'Equivalent to list(combinations(iterable, r))[index]'
+ pool = tuple(iterable)
+ n = len(pool)
+ if r < 0 or r > n:
+ raise ValueError
+ c = 1
+ k = min(r, n-r)
+ for i in range(1, k+1):
+ c = c * (n - k + i) // i
+ if index < 0:
+ index += c
+ if index < 0 or index >= c:
+ raise IndexError
+ result = []
+ while r:
+ c, n, r = c*r//n, n-1, r-1
+ while index >= c:
+ index -= c
+ c, n = c*(n-r)//n, n-1
+ result.append(pool[-1-n])
+ return tuple(result)
+
Note, many of the above recipes can be optimized by replacing global lookups
with local variables defined as default values. For example, the
*dotproduct* recipe can be written as::
| ``CHAR_MAX`` | Nothing is specified in this locale. |
+--------------+-----------------------------------------+
+ The function sets temporarily the ``LC_CTYPE`` locale to the ``LC_NUMERIC``
+ locale to decode ``decimal_point`` and ``thousands_sep`` byte strings if
+ they are non-ASCII or longer than 1 byte, and the ``LC_NUMERIC`` locale is
+ different than the ``LC_CTYPE`` locale. This temporary change affects other
+ threads.
+
+ .. versionchanged:: 3.6.5
+ The function now sets temporarily the ``LC_CTYPE`` locale to the
+ ``LC_NUMERIC`` locale in some cases.
+
.. function:: nl_langinfo(option)
.. class:: Logger
-.. attribute:: Logger.propagate
+ .. attribute:: Logger.propagate
- If this evaluates to true, events logged to this logger will be passed to the
- handlers of higher level (ancestor) loggers, in addition to any handlers
- attached to this logger. Messages are passed directly to the ancestor
- loggers' handlers - neither the level nor filters of the ancestor loggers in
- question are considered.
+ If this attribute evaluates to true, events logged to this logger will be
+ passed to the handlers of higher level (ancestor) loggers, in addition to
+ any handlers attached to this logger. Messages are passed directly to the
+ ancestor loggers' handlers - neither the level nor filters of the ancestor
+ loggers in question are considered.
- If this evaluates to false, logging messages are not passed to the handlers
- of ancestor loggers.
+ If this evaluates to false, logging messages are not passed to the handlers
+ of ancestor loggers.
- The constructor sets this attribute to ``True``.
+ The constructor sets this attribute to ``True``.
- .. note:: If you attach a handler to a logger *and* one or more of its
- ancestors, it may emit the same record multiple times. In general, you
- should not need to attach a handler to more than one logger - if you just
- attach it to the appropriate logger which is highest in the logger
- hierarchy, then it will see all events logged by all descendant loggers,
- provided that their propagate setting is left set to ``True``. A common
- scenario is to attach handlers only to the root logger, and to let
- propagation take care of the rest.
+ .. note:: If you attach a handler to a logger *and* one or more of its
+ ancestors, it may emit the same record multiple times. In general, you
+ should not need to attach a handler to more than one logger - if you just
+ attach it to the appropriate logger which is highest in the logger
+ hierarchy, then it will see all events logged by all descendant loggers,
+ provided that their propagate setting is left set to ``True``. A common
+ scenario is to attach handlers only to the root logger, and to let
+ propagation take care of the rest.
-.. method:: Logger.setLevel(lvl)
+ .. method:: Logger.setLevel(level)
- Sets the threshold for this logger to *lvl*. Logging messages which are less
- severe than *lvl* will be ignored. When a logger is created, the level is set to
- :const:`NOTSET` (which causes all messages to be processed when the logger is
- the root logger, or delegation to the parent when the logger is a non-root
- logger). Note that the root logger is created with level :const:`WARNING`.
+ Sets the threshold for this logger to *level*. Logging messages which are less
+ severe than *level* will be ignored; logging messages which have severity *level*
+ or higher will be emitted by whichever handler or handlers service this logger,
+ unless a handler's level has been set to a higher severity level than *level*.
- The term 'delegation to the parent' means that if a logger has a level of
- NOTSET, its chain of ancestor loggers is traversed until either an ancestor with
- a level other than NOTSET is found, or the root is reached.
+ When a logger is created, the level is set to :const:`NOTSET` (which causes
+ all messages to be processed when the logger is the root logger, or delegation
+ to the parent when the logger is a non-root logger). Note that the root logger
+ is created with level :const:`WARNING`.
- If an ancestor is found with a level other than NOTSET, then that ancestor's
- level is treated as the effective level of the logger where the ancestor search
- began, and is used to determine how a logging event is handled.
+ The term 'delegation to the parent' means that if a logger has a level of
+ NOTSET, its chain of ancestor loggers is traversed until either an ancestor with
+ a level other than NOTSET is found, or the root is reached.
- If the root is reached, and it has a level of NOTSET, then all messages will be
- processed. Otherwise, the root's level will be used as the effective level.
+ If an ancestor is found with a level other than NOTSET, then that ancestor's
+ level is treated as the effective level of the logger where the ancestor search
+ began, and is used to determine how a logging event is handled.
- See :ref:`levels` for a list of levels.
+ If the root is reached, and it has a level of NOTSET, then all messages will be
+ processed. Otherwise, the root's level will be used as the effective level.
- .. versionchanged:: 3.2
- The *lvl* parameter now accepts a string representation of the
- level such as 'INFO' as an alternative to the integer constants
- such as :const:`INFO`. Note, however, that levels are internally stored
- as integers, and methods such as e.g. :meth:`getEffectiveLevel` and
- :meth:`isEnabledFor` will return/expect to be passed integers.
+ See :ref:`levels` for a list of levels.
+ .. versionchanged:: 3.2
+ The *level* parameter now accepts a string representation of the
+ level such as 'INFO' as an alternative to the integer constants
+ such as :const:`INFO`. Note, however, that levels are internally stored
+ as integers, and methods such as e.g. :meth:`getEffectiveLevel` and
+ :meth:`isEnabledFor` will return/expect to be passed integers.
-.. method:: Logger.isEnabledFor(lvl)
- Indicates if a message of severity *lvl* would be processed by this logger.
- This method checks first the module-level level set by
- ``logging.disable(lvl)`` and then the logger's effective level as determined
- by :meth:`getEffectiveLevel`.
+ .. method:: Logger.isEnabledFor(lvl)
+ Indicates if a message of severity *lvl* would be processed by this logger.
+ This method checks first the module-level level set by
+ ``logging.disable(lvl)`` and then the logger's effective level as determined
+ by :meth:`getEffectiveLevel`.
-.. method:: Logger.getEffectiveLevel()
- Indicates the effective level for this logger. If a value other than
- :const:`NOTSET` has been set using :meth:`setLevel`, it is returned. Otherwise,
- the hierarchy is traversed towards the root until a value other than
- :const:`NOTSET` is found, and that value is returned. The value returned is
- an integer, typically one of :const:`logging.DEBUG`, :const:`logging.INFO`
- etc.
+ .. method:: Logger.getEffectiveLevel()
+ Indicates the effective level for this logger. If a value other than
+ :const:`NOTSET` has been set using :meth:`setLevel`, it is returned. Otherwise,
+ the hierarchy is traversed towards the root until a value other than
+ :const:`NOTSET` is found, and that value is returned. The value returned is
+ an integer, typically one of :const:`logging.DEBUG`, :const:`logging.INFO`
+ etc.
-.. method:: Logger.getChild(suffix)
- Returns a logger which is a descendant to this logger, as determined by the suffix.
- Thus, ``logging.getLogger('abc').getChild('def.ghi')`` would return the same
- logger as would be returned by ``logging.getLogger('abc.def.ghi')``. This is a
- convenience method, useful when the parent logger is named using e.g. ``__name__``
- rather than a literal string.
+ .. method:: Logger.getChild(suffix)
- .. versionadded:: 3.2
+ Returns a logger which is a descendant to this logger, as determined by the suffix.
+ Thus, ``logging.getLogger('abc').getChild('def.ghi')`` would return the same
+ logger as would be returned by ``logging.getLogger('abc.def.ghi')``. This is a
+ convenience method, useful when the parent logger is named using e.g. ``__name__``
+ rather than a literal string.
+ .. versionadded:: 3.2
-.. method:: Logger.debug(msg, *args, **kwargs)
- Logs a message with level :const:`DEBUG` on this logger. The *msg* is the
- message format string, and the *args* are the arguments which are merged into
- *msg* using the string formatting operator. (Note that this means that you can
- use keywords in the format string, together with a single dictionary argument.)
+ .. method:: Logger.debug(msg, *args, **kwargs)
- There are three keyword arguments in *kwargs* which are inspected:
- *exc_info*, *stack_info*, and *extra*.
+ Logs a message with level :const:`DEBUG` on this logger. The *msg* is the
+ message format string, and the *args* are the arguments which are merged into
+ *msg* using the string formatting operator. (Note that this means that you can
+ use keywords in the format string, together with a single dictionary argument.)
- If *exc_info* does not evaluate as false, it causes exception information to be
- added to the logging message. If an exception tuple (in the format returned by
- :func:`sys.exc_info`) or an exception instance is provided, it is used;
- otherwise, :func:`sys.exc_info` is called to get the exception information.
+ There are three keyword arguments in *kwargs* which are inspected:
+ *exc_info*, *stack_info*, and *extra*.
- The second optional keyword argument is *stack_info*, which defaults to
- ``False``. If true, stack information is added to the logging
- message, including the actual logging call. Note that this is not the same
- stack information as that displayed through specifying *exc_info*: The
- former is stack frames from the bottom of the stack up to the logging call
- in the current thread, whereas the latter is information about stack frames
- which have been unwound, following an exception, while searching for
- exception handlers.
+ If *exc_info* does not evaluate as false, it causes exception information to be
+ added to the logging message. If an exception tuple (in the format returned by
+ :func:`sys.exc_info`) or an exception instance is provided, it is used;
+ otherwise, :func:`sys.exc_info` is called to get the exception information.
- You can specify *stack_info* independently of *exc_info*, e.g. to just show
- how you got to a certain point in your code, even when no exceptions were
- raised. The stack frames are printed following a header line which says::
+ The second optional keyword argument is *stack_info*, which defaults to
+ ``False``. If true, stack information is added to the logging
+ message, including the actual logging call. Note that this is not the same
+ stack information as that displayed through specifying *exc_info*: The
+ former is stack frames from the bottom of the stack up to the logging call
+ in the current thread, whereas the latter is information about stack frames
+ which have been unwound, following an exception, while searching for
+ exception handlers.
- Stack (most recent call last):
+ You can specify *stack_info* independently of *exc_info*, e.g. to just show
+ how you got to a certain point in your code, even when no exceptions were
+ raised. The stack frames are printed following a header line which says::
- This mimics the ``Traceback (most recent call last):`` which is used when
- displaying exception frames.
+ Stack (most recent call last):
- The third keyword argument is *extra* which can be used to pass a
- dictionary which is used to populate the __dict__ of the LogRecord created for
- the logging event with user-defined attributes. These custom attributes can then
- be used as you like. For example, they could be incorporated into logged
- messages. For example::
+ This mimics the ``Traceback (most recent call last):`` which is used when
+ displaying exception frames.
- FORMAT = '%(asctime)-15s %(clientip)s %(user)-8s %(message)s'
- logging.basicConfig(format=FORMAT)
- d = {'clientip': '192.168.0.1', 'user': 'fbloggs'}
- logger = logging.getLogger('tcpserver')
- logger.warning('Protocol problem: %s', 'connection reset', extra=d)
+ The third keyword argument is *extra* which can be used to pass a
+ dictionary which is used to populate the __dict__ of the LogRecord created for
+ the logging event with user-defined attributes. These custom attributes can then
+ be used as you like. For example, they could be incorporated into logged
+ messages. For example::
- would print something like ::
+ FORMAT = '%(asctime)-15s %(clientip)s %(user)-8s %(message)s'
+ logging.basicConfig(format=FORMAT)
+ d = {'clientip': '192.168.0.1', 'user': 'fbloggs'}
+ logger = logging.getLogger('tcpserver')
+ logger.warning('Protocol problem: %s', 'connection reset', extra=d)
- 2006-02-08 22:20:02,165 192.168.0.1 fbloggs Protocol problem: connection reset
+ would print something like ::
- The keys in the dictionary passed in *extra* should not clash with the keys used
- by the logging system. (See the :class:`Formatter` documentation for more
- information on which keys are used by the logging system.)
+ 2006-02-08 22:20:02,165 192.168.0.1 fbloggs Protocol problem: connection reset
- If you choose to use these attributes in logged messages, you need to exercise
- some care. In the above example, for instance, the :class:`Formatter` has been
- set up with a format string which expects 'clientip' and 'user' in the attribute
- dictionary of the LogRecord. If these are missing, the message will not be
- logged because a string formatting exception will occur. So in this case, you
- always need to pass the *extra* dictionary with these keys.
+ The keys in the dictionary passed in *extra* should not clash with the keys used
+ by the logging system. (See the :class:`Formatter` documentation for more
+ information on which keys are used by the logging system.)
- While this might be annoying, this feature is intended for use in specialized
- circumstances, such as multi-threaded servers where the same code executes in
- many contexts, and interesting conditions which arise are dependent on this
- context (such as remote client IP address and authenticated user name, in the
- above example). In such circumstances, it is likely that specialized
- :class:`Formatter`\ s would be used with particular :class:`Handler`\ s.
+ If you choose to use these attributes in logged messages, you need to exercise
+ some care. In the above example, for instance, the :class:`Formatter` has been
+ set up with a format string which expects 'clientip' and 'user' in the attribute
+ dictionary of the LogRecord. If these are missing, the message will not be
+ logged because a string formatting exception will occur. So in this case, you
+ always need to pass the *extra* dictionary with these keys.
- .. versionadded:: 3.2
- The *stack_info* parameter was added.
+ While this might be annoying, this feature is intended for use in specialized
+ circumstances, such as multi-threaded servers where the same code executes in
+ many contexts, and interesting conditions which arise are dependent on this
+ context (such as remote client IP address and authenticated user name, in the
+ above example). In such circumstances, it is likely that specialized
+ :class:`Formatter`\ s would be used with particular :class:`Handler`\ s.
- .. versionchanged:: 3.5
- The *exc_info* parameter can now accept exception instances.
+ .. versionadded:: 3.2
+ The *stack_info* parameter was added.
+ .. versionchanged:: 3.5
+ The *exc_info* parameter can now accept exception instances.
-.. method:: Logger.info(msg, *args, **kwargs)
- Logs a message with level :const:`INFO` on this logger. The arguments are
- interpreted as for :meth:`debug`.
+ .. method:: Logger.info(msg, *args, **kwargs)
+ Logs a message with level :const:`INFO` on this logger. The arguments are
+ interpreted as for :meth:`debug`.
-.. method:: Logger.warning(msg, *args, **kwargs)
- Logs a message with level :const:`WARNING` on this logger. The arguments are
- interpreted as for :meth:`debug`.
+ .. method:: Logger.warning(msg, *args, **kwargs)
- .. note:: There is an obsolete method ``warn`` which is functionally
- identical to ``warning``. As ``warn`` is deprecated, please do not use
- it - use ``warning`` instead.
+ Logs a message with level :const:`WARNING` on this logger. The arguments are
+ interpreted as for :meth:`debug`.
-.. method:: Logger.error(msg, *args, **kwargs)
+ .. note:: There is an obsolete method ``warn`` which is functionally
+ identical to ``warning``. As ``warn`` is deprecated, please do not use
+ it - use ``warning`` instead.
- Logs a message with level :const:`ERROR` on this logger. The arguments are
- interpreted as for :meth:`debug`.
+ .. method:: Logger.error(msg, *args, **kwargs)
+ Logs a message with level :const:`ERROR` on this logger. The arguments are
+ interpreted as for :meth:`debug`.
-.. method:: Logger.critical(msg, *args, **kwargs)
- Logs a message with level :const:`CRITICAL` on this logger. The arguments are
- interpreted as for :meth:`debug`.
+ .. method:: Logger.critical(msg, *args, **kwargs)
+ Logs a message with level :const:`CRITICAL` on this logger. The arguments are
+ interpreted as for :meth:`debug`.
-.. method:: Logger.log(lvl, msg, *args, **kwargs)
- Logs a message with integer level *lvl* on this logger. The other arguments are
- interpreted as for :meth:`debug`.
+ .. method:: Logger.log(lvl, msg, *args, **kwargs)
+ Logs a message with integer level *lvl* on this logger. The other arguments are
+ interpreted as for :meth:`debug`.
-.. method:: Logger.exception(msg, *args, **kwargs)
- Logs a message with level :const:`ERROR` on this logger. The arguments are
- interpreted as for :meth:`debug`. Exception info is added to the logging
- message. This method should only be called from an exception handler.
+ .. method:: Logger.exception(msg, *args, **kwargs)
+ Logs a message with level :const:`ERROR` on this logger. The arguments are
+ interpreted as for :meth:`debug`. Exception info is added to the logging
+ message. This method should only be called from an exception handler.
-.. method:: Logger.addFilter(filt)
- Adds the specified filter *filt* to this logger.
+ .. method:: Logger.addFilter(filter)
+ Adds the specified filter *filter* to this logger.
-.. method:: Logger.removeFilter(filt)
- Removes the specified filter *filt* from this logger.
+ .. method:: Logger.removeFilter(filter)
+ Removes the specified filter *filter* from this logger.
-.. method:: Logger.filter(record)
- Applies this logger's filters to the record and returns a true value if the
- record is to be processed. The filters are consulted in turn, until one of
- them returns a false value. If none of them return a false value, the record
- will be processed (passed to handlers). If one returns a false value, no
- further processing of the record occurs.
+ .. method:: Logger.filter(record)
+ Applies this logger's filters to the record and returns a true value if the
+ record is to be processed. The filters are consulted in turn, until one of
+ them returns a false value. If none of them return a false value, the record
+ will be processed (passed to handlers). If one returns a false value, no
+ further processing of the record occurs.
-.. method:: Logger.addHandler(hdlr)
- Adds the specified handler *hdlr* to this logger.
+ .. method:: Logger.addHandler(hdlr)
+ Adds the specified handler *hdlr* to this logger.
-.. method:: Logger.removeHandler(hdlr)
- Removes the specified handler *hdlr* from this logger.
+ .. method:: Logger.removeHandler(hdlr)
+ Removes the specified handler *hdlr* from this logger.
-.. method:: Logger.findCaller(stack_info=False)
- Finds the caller's source filename and line number. Returns the filename, line
- number, function name and stack information as a 4-element tuple. The stack
- information is returned as ``None`` unless *stack_info* is ``True``.
+ .. method:: Logger.findCaller(stack_info=False)
+ Finds the caller's source filename and line number. Returns the filename, line
+ number, function name and stack information as a 4-element tuple. The stack
+ information is returned as ``None`` unless *stack_info* is ``True``.
-.. method:: Logger.handle(record)
- Handles a record by passing it to all handlers associated with this logger and
- its ancestors (until a false value of *propagate* is found). This method is used
- for unpickled records received from a socket, as well as those created locally.
- Logger-level filtering is applied using :meth:`~Logger.filter`.
+ .. method:: Logger.handle(record)
+ Handles a record by passing it to all handlers associated with this logger and
+ its ancestors (until a false value of *propagate* is found). This method is used
+ for unpickled records received from a socket, as well as those created locally.
+ Logger-level filtering is applied using :meth:`~Logger.filter`.
-.. method:: Logger.makeRecord(name, lvl, fn, lno, msg, args, exc_info, func=None, extra=None, sinfo=None)
- This is a factory method which can be overridden in subclasses to create
- specialized :class:`LogRecord` instances.
+ .. method:: Logger.makeRecord(name, lvl, fn, lno, msg, args, exc_info, func=None, extra=None, sinfo=None)
-.. method:: Logger.hasHandlers()
+ This is a factory method which can be overridden in subclasses to create
+ specialized :class:`LogRecord` instances.
- Checks to see if this logger has any handlers configured. This is done by
- looking for handlers in this logger and its parents in the logger hierarchy.
- Returns ``True`` if a handler was found, else ``False``. The method stops searching
- up the hierarchy whenever a logger with the 'propagate' attribute set to
- false is found - that will be the last logger which is checked for the
- existence of handlers.
+ .. method:: Logger.hasHandlers()
- .. versionadded:: 3.2
+ Checks to see if this logger has any handlers configured. This is done by
+ looking for handlers in this logger and its parents in the logger hierarchy.
+ Returns ``True`` if a handler was found, else ``False``. The method stops searching
+ up the hierarchy whenever a logger with the 'propagate' attribute set to
+ false is found - that will be the last logger which is checked for the
+ existence of handlers.
+ .. versionadded:: 3.2
+
+ .. versionchanged:: 3.7
+ Loggers can now be picked and unpickled.
.. _levels:
subclasses. However, the :meth:`__init__` method in subclasses needs to call
:meth:`Handler.__init__`.
+.. class:: Handler
-.. method:: Handler.__init__(level=NOTSET)
+ .. method:: Handler.__init__(level=NOTSET)
- Initializes the :class:`Handler` instance by setting its level, setting the list
- of filters to the empty list and creating a lock (using :meth:`createLock`) for
- serializing access to an I/O mechanism.
+ Initializes the :class:`Handler` instance by setting its level, setting the list
+ of filters to the empty list and creating a lock (using :meth:`createLock`) for
+ serializing access to an I/O mechanism.
-.. method:: Handler.createLock()
+ .. method:: Handler.createLock()
- Initializes a thread lock which can be used to serialize access to underlying
- I/O functionality which may not be threadsafe.
+ Initializes a thread lock which can be used to serialize access to underlying
+ I/O functionality which may not be threadsafe.
-.. method:: Handler.acquire()
+ .. method:: Handler.acquire()
- Acquires the thread lock created with :meth:`createLock`.
+ Acquires the thread lock created with :meth:`createLock`.
-.. method:: Handler.release()
+ .. method:: Handler.release()
- Releases the thread lock acquired with :meth:`acquire`.
+ Releases the thread lock acquired with :meth:`acquire`.
-.. method:: Handler.setLevel(lvl)
+ .. method:: Handler.setLevel(level)
- Sets the threshold for this handler to *lvl*. Logging messages which are less
- severe than *lvl* will be ignored. When a handler is created, the level is set
- to :const:`NOTSET` (which causes all messages to be processed).
+ Sets the threshold for this handler to *level*. Logging messages which are
+ less severe than *level* will be ignored. When a handler is created, the
+ level is set to :const:`NOTSET` (which causes all messages to be
+ processed).
- See :ref:`levels` for a list of levels.
+ See :ref:`levels` for a list of levels.
- .. versionchanged:: 3.2
- The *lvl* parameter now accepts a string representation of the
- level such as 'INFO' as an alternative to the integer constants
- such as :const:`INFO`.
+ .. versionchanged:: 3.2
+ The *level* parameter now accepts a string representation of the
+ level such as 'INFO' as an alternative to the integer constants
+ such as :const:`INFO`.
-.. method:: Handler.setFormatter(form)
+ .. method:: Handler.setFormatter(fmt)
- Sets the :class:`Formatter` for this handler to *form*.
+ Sets the :class:`Formatter` for this handler to *fmt*.
-.. method:: Handler.addFilter(filt)
+ .. method:: Handler.addFilter(filter)
- Adds the specified filter *filt* to this handler.
+ Adds the specified filter *filter* to this handler.
-.. method:: Handler.removeFilter(filt)
+ .. method:: Handler.removeFilter(filter)
- Removes the specified filter *filt* from this handler.
+ Removes the specified filter *filter* from this handler.
-.. method:: Handler.filter(record)
+ .. method:: Handler.filter(record)
- Applies this handler's filters to the record and returns a true value if the
- record is to be processed. The filters are consulted in turn, until one of
- them returns a false value. If none of them return a false value, the record
- will be emitted. If one returns a false value, the handler will not emit the
- record.
+ Applies this handler's filters to the record and returns a true value if the
+ record is to be processed. The filters are consulted in turn, until one of
+ them returns a false value. If none of them return a false value, the record
+ will be emitted. If one returns a false value, the handler will not emit the
+ record.
-.. method:: Handler.flush()
+ .. method:: Handler.flush()
- Ensure all logging output has been flushed. This version does nothing and is
- intended to be implemented by subclasses.
+ Ensure all logging output has been flushed. This version does nothing and is
+ intended to be implemented by subclasses.
-.. method:: Handler.close()
+ .. method:: Handler.close()
- Tidy up any resources used by the handler. This version does no output but
- removes the handler from an internal list of handlers which is closed when
- :func:`shutdown` is called. Subclasses should ensure that this gets called
- from overridden :meth:`close` methods.
+ Tidy up any resources used by the handler. This version does no output but
+ removes the handler from an internal list of handlers which is closed when
+ :func:`shutdown` is called. Subclasses should ensure that this gets called
+ from overridden :meth:`close` methods.
-.. method:: Handler.handle(record)
+ .. method:: Handler.handle(record)
- Conditionally emits the specified logging record, depending on filters which may
- have been added to the handler. Wraps the actual emission of the record with
- acquisition/release of the I/O thread lock.
+ Conditionally emits the specified logging record, depending on filters which may
+ have been added to the handler. Wraps the actual emission of the record with
+ acquisition/release of the I/O thread lock.
-.. method:: Handler.handleError(record)
+ .. method:: Handler.handleError(record)
- This method should be called from handlers when an exception is encountered
- during an :meth:`emit` call. If the module-level attribute
- ``raiseExceptions`` is ``False``, exceptions get silently ignored. This is
- what is mostly wanted for a logging system - most users will not care about
- errors in the logging system, they are more interested in application
- errors. You could, however, replace this with a custom handler if you wish.
- The specified record is the one which was being processed when the exception
- occurred. (The default value of ``raiseExceptions`` is ``True``, as that is
- more useful during development).
+ This method should be called from handlers when an exception is encountered
+ during an :meth:`emit` call. If the module-level attribute
+ ``raiseExceptions`` is ``False``, exceptions get silently ignored. This is
+ what is mostly wanted for a logging system - most users will not care about
+ errors in the logging system, they are more interested in application
+ errors. You could, however, replace this with a custom handler if you wish.
+ The specified record is the one which was being processed when the exception
+ occurred. (The default value of ``raiseExceptions`` is ``True``, as that is
+ more useful during development).
-.. method:: Handler.format(record)
+ .. method:: Handler.format(record)
- Do formatting for a record - if a formatter is set, use it. Otherwise, use the
- default formatter for the module.
+ Do formatting for a record - if a formatter is set, use it. Otherwise, use the
+ default formatter for the module.
-.. method:: Handler.emit(record)
+ .. method:: Handler.emit(record)
- Do whatever it takes to actually log the specified logging record. This version
- is intended to be implemented by subclasses and so raises a
- :exc:`NotImplementedError`.
+ Do whatever it takes to actually log the specified logging record. This version
+ is intended to be implemented by subclasses and so raises a
+ :exc:`NotImplementedError`.
For a list of handlers included as standard, see :mod:`logging.handlers`.
| lineno | ``%(lineno)d`` | Source line number where the logging call was |
| | | issued (if available). |
+----------------+-------------------------+-----------------------------------------------+
+| message | ``%(message)s`` | The logged message, computed as ``msg % |
+| | | args``. This is set when |
+| | | :meth:`Formatter.format` is invoked. |
++----------------+-------------------------+-----------------------------------------------+
| module | ``%(module)s`` | Module (name portion of ``filename``). |
+----------------+-------------------------+-----------------------------------------------+
| msecs | ``%(msecs)d`` | Millisecond portion of the time when the |
| | | :class:`LogRecord` was created. |
+----------------+-------------------------+-----------------------------------------------+
-| message | ``%(message)s`` | The logged message, computed as ``msg % |
-| | | args``. This is set when |
-| | | :meth:`Formatter.format` is invoked. |
-+----------------+-------------------------+-----------------------------------------------+
| msg | You shouldn't need to | The format string passed in the original |
| | format this yourself. | logging call. Merged with ``args`` to |
| | | produce ``message``, or an arbitrary object |
handlers being added multiple times to the root logger, which can in turn
lead to multiple messages for the same event.
-.. function:: disable(lvl)
+.. function:: disable(lvl=CRITICAL)
Provides an overriding level *lvl* for all loggers which takes precedence over
the logger's own level. When the need arises to temporarily throttle logging
overriding level, so that logging output again depends on the effective
levels of individual loggers.
+ Note that if you have defined any custom logging level higher than
+ ``CRITICAL`` (this is not recommended), you won't be able to rely on the
+ default value for the *lvl* parameter, but will have to explicitly supply a
+ suitable value.
+
+ .. versionchanged:: 3.7
+ The *lvl* parameter was defaulted to level ``CRITICAL``. See Issue
+ #28524 for more information about this change.
.. function:: addLevelName(lvl, levelName)
package available from this site is suitable for use with Python 1.5.2, 2.1.x
and 2.2.x, which do not include the :mod:`logging` package in the standard
library.
-
`Configuring Netscape Mail on Unix: Why The Content-Length Format is Bad <https://www.jwz.org/doc/content-length.html>`_
An argument for using the original mbox format rather than a variation.
- `"mbox" is a family of several mutually incompatible mailbox formats <http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/mail-mbox-formats.html>`_
+ `"mbox" is a family of several mutually incompatible mailbox formats <https://www.loc.gov/preservation/digital/formats/fdd/fdd000383.shtml>`_
A history of mbox variations.
Return the :class:`~numbers.Real` value *x* truncated to an
:class:`~numbers.Integral` (usually an integer). Delegates to
- ``x.__trunc__()``.
+ :meth:`x.__trunc__() <object.__trunc__>`.
Note that :func:`frexp` and :func:`modf` have a different call/return pattern
where
- * parser is the :class:`OptionParser` instance the group will be insterted in
+ * parser is the :class:`OptionParser` instance the group will be inserted in
to
* title is the group title
* description, optional, is a long description of the group
.. function:: isfile(path)
- Return ``True`` if *path* is an existing regular file. This follows symbolic
- links, so both :func:`islink` and :func:`isfile` can be true for the same path.
+ Return ``True`` if *path* is an :func:`existing <exists>` regular file.
+ This follows symbolic links, so both :func:`islink` and :func:`isfile` can
+ be true for the same path.
.. versionchanged:: 3.6
Accepts a :term:`path-like object`.
.. function:: isdir(path)
- Return ``True`` if *path* is an existing directory. This follows symbolic
- links, so both :func:`islink` and :func:`isdir` can be true for the same path.
+ Return ``True`` if *path* is an :func:`existing <exists>` directory. This
+ follows symbolic links, so both :func:`islink` and :func:`isdir` can be true
+ for the same path.
.. versionchanged:: 3.6
Accepts a :term:`path-like object`.
.. function:: islink(path)
- Return ``True`` if *path* refers to a directory entry that is a symbolic link.
- Always ``False`` if symbolic links are not supported by the Python runtime.
+ Return ``True`` if *path* refers to an :func:`existing <exists>` directory
+ entry that is a symbolic link. Always ``False`` if symbolic links are not
+ supported by the Python runtime.
.. versionchanged:: 3.6
Accepts a :term:`path-like object`.
.. function:: getlogin()
Return the name of the user logged in on the controlling terminal of the
- process. For most purposes, it is more useful to use the environment
- variables :envvar:`LOGNAME` or :envvar:`USERNAME` to find out who the user
- is, or ``pwd.getpwuid(os.getuid())[0]`` to get the login name of the current
- real user id.
+ process. For most purposes, it is more useful to use
+ :func:`getpass.getuser` since the latter checks the environment variables
+ :envvar:`LOGNAME` or :envvar:`USERNAME` to find out who the user is, and
+ falls back to ``pwd.getpwuid(os.getuid())[0]`` to get the login name of the
+ current real user id.
Availability: Unix, Windows.
.. Things will get more complicated for future Linux versions, since
ALSA is in the standard kernel as of 2.5.x. Presumably if you
use ALSA, you'll have to make sure its OSS compatibility layer
- is active to use ossaudiodev, but you're gonna need it for the vast
+ is active to use ossaudiodev, but you're going to need it for the vast
majority of Linux audio apps anyway.
Sounds like things are also complicated for other BSDs. In response
microphone input::
mixer.setrecsrc (1 << ossaudiodev.SOUND_MIXER_MIC)
-
Python 2 names to the new names used in Python 3. The *encoding* and
*errors* tell pickle how to decode 8-bit string instances pickled by Python
2; these default to 'ASCII' and 'strict', respectively. The *encoding* can
- be 'bytes' to read these ß8-bit string instances as bytes objects.
+ be 'bytes' to read these 8-bit string instances as bytes objects.
.. method:: load()
Encode the contents of the *input* file and write the resulting quoted-printable
data to the *output* file. *input* and *output* must be
- :term:`binary file objects <file object>`. *quotetabs*, a flag which controls
- whether to encode embedded spaces and tabs must be provideda and when true it
- encodes such embedded whitespace, and when false it leaves them unencoded.
+ :term:`binary file objects <file object>`. *quotetabs*, a
+ non-optional flag which controls whether to encode embedded spaces
+ and tabs; when true it encodes such embedded whitespace, and when
+ false it leaves them unencoded.
Note that spaces and tabs appearing at the end of lines are always encoded,
as per :rfc:`1521`. *header* is a flag which controls if spaces are encoded
as underscores as per :rfc:`1522`.
This example looks for a word following a hyphen:
- >>> m = re.search('(?<=-)\w+', 'spam-egg')
+ >>> m = re.search(r'(?<=-)\w+', 'spam-egg')
>>> m.group(0)
'egg'
Splitting on a pattern that could match an empty string now raises
a warning. Patterns that can only match empty strings are now rejected.
+
.. function:: findall(pattern, string, flags=0)
Return all non-overlapping matches of *pattern* in *string*, as a list of
strings. The *string* is scanned left-to-right, and matches are returned in
the order found. If one or more groups are present in the pattern, return a
list of groups; this will be a list of tuples if the pattern has more than
- one group. Empty matches are included in the result unless they touch the
- beginning of another match.
+ one group. Empty matches are included in the result.
+
+ .. note::
+
+ Due to the limitation of the current implementation the character
+ following an empty match is not included in a next match, so
+ ``findall(r'^|\w+', 'two words')`` returns ``['', 'wo', 'words']``
+ (note missed "t"). This is changed in Python 3.7.
.. function:: finditer(pattern, string, flags=0)
Return an :term:`iterator` yielding :ref:`match objects <match-objects>` over
all non-overlapping matches for the RE *pattern* in *string*. The *string*
is scanned left-to-right, and matches are returned in the order found. Empty
- matches are included in the result unless they touch the beginning of another
- match.
+ matches are included in the result. See also the note about :func:`findall`.
.. function:: sub(pattern, repl, string, count=0, flags=0)
Return disk usage statistics about the given path as a :term:`named tuple`
with the attributes *total*, *used* and *free*, which are the amount of
- total, used and free space, in bytes.
+ total, used and free space, in bytes. On Windows, *path* must be a
+ directory; on Unix, it can be a file or directory.
.. versionadded:: 3.3
``SO_DOMAIN``, ``SO_PROTOCOL``, ``SO_PEERSEC``, ``SO_PASSSEC``,
``TCP_USER_TIMEOUT``, ``TCP_CONGESTION`` were added.
+ .. versionchanged:: 3.6.5
+ On Windows, ``TCP_FASTOPEN``, ``TCP_KEEPCNT`` appear if run-time Windows
+ supports.
+
.. data:: AF_CAN
PF_CAN
SOL_CAN_*
The *capath* string, if present, is
the path to a directory containing several CA certificates in PEM format,
following an `OpenSSL specific layout
- <https://www.openssl.org/docs/ssl/SSL_CTX_load_verify_locations.html>`_.
+ <https://www.openssl.org/docs/man1.1.0/ssl/SSL_CTX_load_verify_locations.html>`_.
The *cadata* object, if present, is either an ASCII string of one or more
PEM-encoded certificates or a :term:`bytes-like object` of DER-encoded
.. method:: SSLContext.load_dh_params(dhfile)
- Load the key generation parameters for Diffie-Helman (DH) key exchange.
+ Load the key generation parameters for Diffie-Hellman (DH) key exchange.
Using DH key exchange improves forward secrecy at the expense of
computational resources (both on the server and on the client).
The *dhfile* parameter should be the path to a file containing DH
Get statistics about the SSL sessions created or managed by this context.
A dictionary is returned which maps the names of each `piece of information
- <https://www.openssl.org/docs/ssl/SSL_CTX_sess_number.html>`_ to their
+ <https://www.openssl.org/docs/man1.1.0/ssl/SSL_CTX_sess_number.html>`_ to their
numeric values. For example, here is the total number of hits and misses
in the session cache since the context was created::
import socket, ssl
- context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+ context = ssl.SSLContext()
context.verify_mode = ssl.CERT_REQUIRED
context.check_hostname = True
context.load_default_certs()
a context from scratch (but beware that you might not get the settings
right)::
- >>> context = ssl.SSLContext(ssl.PROTOCOL_TLS)
+ >>> context = ssl.SSLContext()
>>> context.verify_mode = ssl.CERT_REQUIRED
>>> context.check_hostname = True
>>> context.load_verify_locations("/etc/ssl/certs/ca-bundle.crt")
:func:`~ssl.RAND_pseudo_bytes` is sufficient.
+.. ssl-libressl:
+
+LibreSSL support
+----------------
+
+LibreSSL is a fork of OpenSSL 1.0.1. The ssl module has limited support for
+LibreSSL. Some features are not available when the ssl module is compiled
+with LibreSSL.
+
+* LibreSSL >= 2.6.1 no longer supports NPN. The methods
+ :meth:`SSLContext.set_npn_protocols` and
+ :meth:`SSLSocket.selected_npn_protocol` are not available.
+* :meth:`SSLContext.set_default_verify_paths` ignores the env vars
+ :envvar:`SSL_CERT_FILE` and :envvar:`SSL_CERT_PATH` although
+ :func:`get_default_verify_paths` still reports them.
+
+
.. seealso::
Class :class:`socket.socket`
Documentation of underlying :mod:`socket` class
`SSL/TLS Strong Encryption: An Introduction <https://httpd.apache.org/docs/trunk/en/ssl/ssl_intro.html>`_
- Intro from the Apache webserver documentation
+ Intro from the Apache HTTP Server documentation
`RFC 1422: Privacy Enhancement for Internet Electronic Mail: Part II: Certificate-Based Key Management <https://www.ietf.org/rfc/rfc1422>`_
Steve Kent
(8)
``index`` raises :exc:`ValueError` when *x* is not found in *s*.
- When supported, the additional arguments to the index method allow
- efficient searching of subsections of the sequence. Passing the extra
- arguments is roughly equivalent to using ``s[i:j].index(x)``, only
+ Not all implementations support passing the additional arguments *i* and *j*.
+ These arguments allow efficient searching of subsections of the sequence. Passing
+ the extra arguments is roughly equivalent to using ``s[i:j].index(x)``, only
without copying any data and with the returned index being relative to
the start of the sequence rather than the start of the slice.
See :ref:`formatstrings` for a description of the various formatting options
that can be specified in format strings.
+ .. note::
+ When formatting a number (:class:`int`, :class:`float`, :class:`float`
+ and subclasses) with the ``n`` type (ex: ``'{:n}'.format(1234)``), the
+ function sets temporarily the ``LC_CTYPE`` locale to the ``LC_NUMERIC``
+ locale to decode ``decimal_point`` and ``thousands_sep`` fields of
+ :c:func:`localeconv` if they are non-ASCII or longer than 1 byte, and the
+ ``LC_NUMERIC`` locale is different than the ``LC_CTYPE`` locale. This
+ temporary change affects other threads.
+
+ .. versionchanged:: 3.6.5
+ When formatting a number with the ``n`` type, the function sets
+ temporarily the ``LC_CTYPE`` locale to the ``LC_NUMERIC`` locale in some
+ cases.
+
.. method:: str.format_map(mapping)
.. productionlist:: sf
replacement_field: "{" [`field_name`] ["!" `conversion`] [":" `format_spec`] "}"
field_name: arg_name ("." `attribute_name` | "[" `element_index` "]")*
- arg_name: [`identifier` | `integer`]
+ arg_name: [`identifier` | `digit`+]
attribute_name: `identifier`
- element_index: `integer` | `index_string`
+ element_index: `digit`+ | `index_string`
index_string: <any source character except "]"> +
conversion: "r" | "s" | "a"
format_spec: <described in the next section>
fill: <any character>
align: "<" | ">" | "=" | "^"
sign: "+" | "-" | " "
- width: `integer`
+ width: `digit`+
grouping_option: "_" | ","
- precision: `integer`
+ precision: `digit`+
type: "b" | "c" | "d" | "e" | "E" | "f" | "F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%"
If a valid *align* value is specified, it can be preceded by a *fill*
.. versionchanged:: 3.4
Support for the *input* keyword argument was added.
+ .. versionchanged:: 3.6
+ *encoding* and *errors* were added. See :func:`run` for details.
+
.. _subprocess-replacements:
Replacing Older Functions with the :mod:`subprocess` Module
Set the system's profile function, which allows you to implement a Python source
code profiler in Python. See chapter :ref:`profile` for more information on the
Python profiler. The system's profile function is called similarly to the
- system's trace function (see :func:`settrace`), but it isn't called for each
- executed line of code (only on call and return, but the return event is reported
- even when an exception has been set). The function is thread-specific, but
- there is no way for the profiler to know about context switches between threads,
- so it does not make sense to use this in the presence of multiple threads. Also,
+ system's trace function (see :func:`settrace`), but it is called with different events,
+ for example it isn't called for each executed line of code (only on call and return,
+ but the return event is reported even when an exception has been set). The function is
+ thread-specific, but there is no way for the profiler to know about context switches between
+ threads, so it does not make sense to use this in the presence of multiple threads. Also,
its return value is not used, so it can simply return ``None``.
+ Profile functions should have three arguments: *frame*, *event*, and
+ *arg*. *frame* is the current stack frame. *event* is a string: ``'call'``,
+ ``'return'``, ``'c_call'``, ``'c_return'``, or ``'c_exception'``. *arg* depends
+ on the event type.
+
+ The events have the following meaning:
+
+ ``'call'``
+ A function is called (or some other code block entered). The
+ profile function is called; *arg* is ``None``.
+
+ ``'return'``
+ A function (or other code block) is about to return. The profile
+ function is called; *arg* is the value that will be returned, or ``None``
+ if the event is caused by an exception being raised.
+
+ ``'c_call'``
+ A C function is about to be called. This may be an extension function or
+ a built-in. *arg* is the C function object.
+
+ ``'c_return'``
+ A C function has returned. *arg* is the C function object.
+
+ ``'c_exception'``
+ A C function has raised an exception. *arg* is the C function object.
.. function:: setrecursionlimit(limit)
Trace functions should have three arguments: *frame*, *event*, and
*arg*. *frame* is the current stack frame. *event* is a string: ``'call'``,
- ``'line'``, ``'return'``, ``'exception'``, ``'c_call'``, ``'c_return'``, or
- ``'c_exception'``. *arg* depends on the event type.
+ ``'line'``, ``'return'`` or ``'exception'``. *arg* depends on
+ the event type.
The trace function is invoked (with *event* set to ``'call'``) whenever a new
local scope is entered; it should return a reference to a local trace
tuple ``(exception, value, traceback)``; the return value specifies the
new local trace function.
- ``'c_call'``
- A C function is about to be called. This may be an extension function or
- a built-in. *arg* is the C function object.
-
- ``'c_return'``
- A C function has returned. *arg* is the C function object.
-
- ``'c_exception'``
- A C function has raised an exception. *arg* is the C function object.
-
Note that as an exception is propagated down the chain of callers, an
``'exception'`` event is generated at each level.
Class used to record warnings for unit tests. See documentation of
:func:`check_warnings` above for more details.
+
+
+.. class:: FakePath(path)
+
+ Simple :term:`path-like object`. It implements the :meth:`__fspath__`
+ method which just returns the *path* argument. If *path* is an exception,
+ it will be raised in :meth:`!__fspath__`.
.. class:: Semaphore(value=1)
- This class implements semaphore objects. A semaphore manages a counter
- representing the number of :meth:`release` calls minus the number of
+ This class implements semaphore objects. A semaphore manages an atomic
+ counter representing the number of :meth:`release` calls minus the number of
:meth:`acquire` calls, plus an initial value. The :meth:`acquire` method
blocks if necessary until it can return without making the counter negative.
If not given, *value* defaults to 1.
Acquire a semaphore.
- When invoked without arguments: if the internal counter is larger than
- zero on entry, decrement it by one and return immediately. If it is zero
- on entry, block, waiting until some other thread has called
- :meth:`~Semaphore.release` to make it larger than zero. This is done
- with proper interlocking so that if multiple :meth:`acquire` calls are
- blocked, :meth:`~Semaphore.release` will wake exactly one of them up.
- The implementation may pick one at random, so the order in which
- blocked threads are awakened should not be relied on. Returns
- true (or blocks indefinitely).
+ When invoked without arguments:
+
+ * If the internal counter is larger than zero on entry, decrement it by
+ one and return true immediately.
+ * If the internal counter is zero on entry, block until awoken by a call to
+ :meth:`~Semaphore.release`. Once awoken (and the counter is greater
+ than 0), decrement the counter by 1 and return true. Exactly one
+ thread will be awoken by each call to :meth:`~Semaphore.release`. The
+ order in which threads are awoken should not be relied on.
When invoked with *blocking* set to false, do not block. If a call
- without an argument would block, return false immediately; otherwise,
- do the same thing as when called without arguments, and return true.
+ without an argument would block, return false immediately; otherwise, do
+ the same thing as when called without arguments, and return true.
When invoked with a *timeout* other than ``None``, it will block for at
most *timeout* seconds. If acquire does not complete successfully in
When called without arguments this function removes the control-c handler
if it has been installed. This function can also be used as a test decorator
- to temporarily remove the handler whilst the test is being executed::
+ to temporarily remove the handler while the test is being executed::
@unittest.removeHandler
def test_signal_handling(self):
with prefixes in the form ``prefix:sometag`` get expanded to
``{uri}sometag`` where the *prefix* is replaced by the full *URI*.
Also, if there is a `default namespace
-<https://www.w3.org/TR/2006/REC-xml-names-20060816/#defaulting>`__,
+<https://www.w3.org/TR/xml-names/#defaulting>`__,
that full URI gets prepended to all of the non-prefixed tags.
Here is an XML example that incorporates two namespaces, one with the
Compresses the bytes in *data*, returning a bytes object containing compressed data.
*level* is an integer from ``0`` to ``9`` or ``-1`` controlling the level of compression;
- ``1`` is fastest and produces the least compression, ``9`` is slowest and
- produces the most. ``0`` is no compression. The default value is ``-1``
- (Z_DEFAULT_COMPRESSION). Z_DEFAULT_COMPRESSION represents a default
+ ``1`` (Z_BEST_SPEED) is fastest and produces the least compression, ``9`` (Z_BEST_COMPRESSION)
+ is slowest and produces the most. ``0`` (Z_NO_COMPRESSION) is no compression.
+ The default value is ``-1`` (Z_DEFAULT_COMPRESSION). Z_DEFAULT_COMPRESSION represents a default
compromise between speed and compression (currently equivalent to level 6).
Raises the :exc:`error` exception if any error occurs.
*level* can now be used as a keyword parameter.
-.. function:: compressobj(level=-1, method=DEFLATED, wbits=15, memLevel=8, strategy=Z_DEFAULT_STRATEGY[, zdict])
+.. function:: compressobj(level=-1, method=DEFLATED, wbits=MAX_WBITS, memLevel=DEF_MEM_LEVEL, strategy=Z_DEFAULT_STRATEGY[, zdict])
Returns a compression object, to be used for compressing data streams that won't
fit into memory at once.
*level* is the compression level -- an integer from ``0`` to ``9`` or ``-1``.
- A value of ``1`` is fastest and produces the least compression, while a value of
- ``9`` is slowest and produces the most. ``0`` is no compression. The default
- value is ``-1`` (Z_DEFAULT_COMPRESSION). Z_DEFAULT_COMPRESSION represents a default
- compromise between speed and compression (currently equivalent to level 6).
+ A value of ``1`` (Z_BEST_SPEED) is fastest and produces the least compression,
+ while a value of ``9`` (Z_BEST_COMPRESSION) is slowest and produces the most.
+ ``0`` (Z_NO_COMPRESSION) is no compression. The default value is ``-1`` (Z_DEFAULT_COMPRESSION).
+ Z_DEFAULT_COMPRESSION represents a default compromise between speed and compression
+ (currently equivalent to level 6).
*method* is the compression algorithm. Currently, the only supported value is
- ``DEFLATED``.
+ :const:`DEFLATED`.
The *wbits* argument controls the size of the history buffer (or the
"window size") used when compressing data, and whether a header and
- trailer is included in the output. It can take several ranges of values:
+ trailer is included in the output. It can take several ranges of values,
+ defaulting to ``15`` (MAX_WBITS):
* +9 to +15: The base-two logarithm of the window size, which
therefore ranges between 512 and 32768. Larger values produce
Higher values use more memory, but are faster and produce smaller output.
*strategy* is used to tune the compression algorithm. Possible values are
- ``Z_DEFAULT_STRATEGY``, ``Z_FILTERED``, and ``Z_HUFFMAN_ONLY``.
+ :const:`Z_DEFAULT_STRATEGY`, :const:`Z_FILTERED`, :const:`Z_HUFFMAN_ONLY`,
+ :const:`Z_RLE` (zlib 1.2.0.1) and :const:`Z_FIXED` (zlib 1.2.2.2).
*zdict* is a predefined compression dictionary. This is a sequence of bytes
(such as a :class:`bytes` object) containing subsequences that are expected
.. versionchanged:: 3.6
*wbits* and *bufsize* can be used as keyword arguments.
-.. function:: decompressobj(wbits=15[, zdict])
+.. function:: decompressobj(wbits=MAX_WBITS[, zdict])
Returns a decompression object, to be used for decompressing data streams that
won't fit into memory at once.
All pending input is processed, and a bytes object containing the remaining compressed
output is returned. *mode* can be selected from the constants
- :const:`Z_SYNC_FLUSH`, :const:`Z_FULL_FLUSH`, or :const:`Z_FINISH`,
- defaulting to :const:`Z_FINISH`. :const:`Z_SYNC_FLUSH` and
- :const:`Z_FULL_FLUSH` allow compressing further bytestrings of data, while
- :const:`Z_FINISH` finishes the compressed stream and prevents compressing any
- more data. After calling :meth:`flush` with *mode* set to :const:`Z_FINISH`,
- the :meth:`compress` method cannot be called again; the only realistic action is
- to delete the object.
+ :const:`Z_NO_FLUSH`, :const:`Z_PARTIAL_FLUSH`, :const:`Z_SYNC_FLUSH`,
+ :const:`Z_FULL_FLUSH`, :const:`Z_BLOCK` (zlib 1.2.3.4), or :const:`Z_FINISH`,
+ defaulting to :const:`Z_FINISH`. Except :const:`Z_FINISH`, all constants
+ allow compressing further bytestrings of data, while :const:`Z_FINISH` finishes the
+ compressed stream and prevents compressing any more data. After calling :meth:`flush`
+ with *mode* set to :const:`Z_FINISH`, the :meth:`compress` method cannot be called again;
+ the only realistic action is to delete the object.
.. method:: Compress.copy()
analyze, test, perform and/or display publicly, prepare derivative works,
distribute, and otherwise use Python |release| alone or in any derivative
version, provided, however, that PSF's License Agreement and PSF's notice of
- copyright, i.e., "Copyright © 2001-2017 Python Software Foundation; All Rights
+ copyright, i.e., "Copyright © 2001-2018 Python Software Foundation; All Rights
Reserved" are retained in Python |release| alone or in any derivative version
prepared by Licensee.
test_epoll
----------
-The :mod:`test_epoll` contains the following notice::
+The :mod:`test_epoll` module contains the following notice::
Copyright (c) 2001-2006 Twisted Matrix Laboratories.
Select kqueue
-------------
-The :mod:`select` and contains the following notice for the kqueue interface::
+The :mod:`select` module contains the following notice for the kqueue
+interface::
Copyright (c) 2000 Doug White, 2006 James Knight, 2007 Christian Heimes
All rights reserved.
libmpdec
--------
-The :mod:`_decimal` Module is built using an included copy of the libmpdec
+The :mod:`_decimal` module is built using an included copy of the libmpdec
library unless the build is configured ``--with-system-libmpdec``::
Copyright (c) 2008-2016 Stefan Krah. All rights reserved.
.. index::
single: destructor
+ single: finalizer
statement: del
Called when the instance is about to be destroyed. This is also called a
- destructor. If a base class has a :meth:`__del__` method, the derived class's
- :meth:`__del__` method, if any, must explicitly call it to ensure proper
- deletion of the base class part of the instance. Note that it is possible
- (though not recommended!) for the :meth:`__del__` method to postpone destruction
- of the instance by creating a new reference to it. It may then be called at a
- later time when this new reference is deleted. It is not guaranteed that
- :meth:`__del__` methods are called for objects that still exist when the
- interpreter exits.
+ finalizer or (improperly) a destructor. If a base class has a
+ :meth:`__del__` method, the derived class's :meth:`__del__` method,
+ if any, must explicitly call it to ensure proper deletion of the base
+ class part of the instance.
+
+ It is possible (though not recommended!) for the :meth:`__del__` method
+ to postpone destruction of the instance by creating a new reference to
+ it. This is called object *resurrection*. It is implementation-dependent
+ whether :meth:`__del__` is called a second time when a resurrected object
+ is about to be destroyed; the current :term:`CPython` implementation
+ only calls it once.
+
+ It is not guaranteed that :meth:`__del__` methods are called for objects
+ that still exist when the interpreter exits.
.. note::
``del x`` doesn't directly call ``x.__del__()`` --- the former decrements
the reference count for ``x`` by one, and the latter is only called when
- ``x``'s reference count reaches zero. Some common situations that may
- prevent the reference count of an object from going to zero include:
- circular references between objects (e.g., a doubly-linked list or a tree
- data structure with parent and child pointers); a reference to the object
- on the stack frame of a function that caught an exception (the traceback
- stored in ``sys.exc_info()[2]`` keeps the stack frame alive); or a
- reference to the object on the stack frame that raised an unhandled
- exception in interactive mode (the traceback stored in
- ``sys.last_traceback`` keeps the stack frame alive). The first situation
- can only be remedied by explicitly breaking the cycles; the second can be
- resolved by freeing the reference to the traceback object when it is no
- longer useful, and the third can be resolved by storing ``None`` in
- ``sys.last_traceback``.
- Circular references which are garbage are detected and cleaned up when
- the cyclic garbage collector is enabled (it's on by default). Refer to the
- documentation for the :mod:`gc` module for more information about this
- topic.
+ ``x``'s reference count reaches zero.
+
+ .. impl-detail::
+ It is possible for a reference cycle to prevent the reference count
+ of an object from going to zero. In this case, the cycle will be
+ later detected and deleted by the :term:`cyclic garbage collector
+ <garbage collection>`. A common cause of reference cycles is when
+ an exception has been caught in a local variable. The frame's
+ locals then reference the exception, which references its own
+ traceback, which references the locals of all frames caught in the
+ traceback.
+
+ .. seealso::
+ Documentation for the :mod:`gc` module.
.. warning::
Due to the precarious circumstances under which :meth:`__del__` methods are
invoked, exceptions that occur during their execution are ignored, and a warning
- is printed to ``sys.stderr`` instead. Also, when :meth:`__del__` is invoked in
- response to a module being deleted (e.g., when execution of the program is
- done), other globals referenced by the :meth:`__del__` method may already have
- been deleted or in the process of being torn down (e.g. the import
- machinery shutting down). For this reason, :meth:`__del__` methods
- should do the absolute
- minimum needed to maintain external invariants. Starting with version 1.5,
- Python guarantees that globals whose name begins with a single underscore are
- deleted from their module before other globals are deleted; if no other
- references to such globals exist, this may help in assuring that imported
- modules are still available at the time when the :meth:`__del__` method is
- called.
+ is printed to ``sys.stderr`` instead. In particular:
- .. index::
- single: repr() (built-in function); __repr__() (object method)
+ * :meth:`__del__` can be invoked when arbitrary code is being executed,
+ including from any arbitrary thread. If :meth:`__del__` needs to take
+ a lock or invoke any other blocking resource, it may deadlock as
+ the resource may already be taken by the code that gets interrupted
+ to execute :meth:`__del__`.
+
+ * :meth:`__del__` can be executed during interpreter shutdown. As a
+ consequence, the global variables it needs to access (including other
+ modules) may already have been deleted or set to ``None``. Python
+ guarantees that globals whose name begins with a single underscore
+ are deleted from their module before other globals are deleted; if
+ no other references to such globals exist, this may help in assuring
+ that imported modules are still available at the time when the
+ :meth:`__del__` method is called.
+ .. index::
+ single: repr() (built-in function); __repr__() (object method)
+
.. method:: object.__repr__(self)
Called by the :func:`repr` built-in function to compute the "official" string
.. method:: object.__getattr__(self, name)
- Called when an attribute lookup has not found the attribute in the usual places
- (i.e. it is not an instance attribute nor is it found in the class tree for
- ``self``). ``name`` is the attribute name. This method should return the
- (computed) attribute value or raise an :exc:`AttributeError` exception.
+ Called when the default attribute access fails with an :exc:`AttributeError`
+ (either :meth:`__getattribute__` raises an :exc:`AttributeError` because
+ *name* is not an instance attribute or an attribute in the class tree
+ for ``self``; or :meth:`__get__` of a *name* property raises
+ :exc:`AttributeError`). This method should either return the (computed)
+ attribute value or raise an :exc:`AttributeError` exception.
Note that if the attribute is found through the normal mechanism,
:meth:`__getattr__` is not called. (This is an intentional asymmetry between
returned. :func:`dir` converts the returned sequence to a list and sorts it.
+Customizing module attribute access
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. index::
+ single: __class__ (module attribute)
+
+For a more fine grained customization of the module behavior (setting
+attributes, properties, etc.), one can set the ``__class__`` attribute of
+a module object to a subclass of :class:`types.ModuleType`. For example::
+
+ import sys
+ from types import ModuleType
+
+ class VerboseModule(ModuleType):
+ def __repr__(self):
+ return f'Verbose {self.__name__}'
+
+ def __setattr__(self, attr, value):
+ print(f'Setting {attr}...')
+ setattr(self, attr, value)
+
+ sys.modules[__name__].__class__ = VerboseModule
+
+.. note::
+ Setting module ``__class__`` only affects lookups made using the attribute
+ access syntax -- directly accessing the module globals (whether by code
+ within the module, or via a reference to the module's globals dictionary)
+ is unaffected.
+
+.. versionchanged:: 3.5
+ ``__class__`` module attribute is now writable.
+
+
.. _descriptors:
Implementing Descriptors
__slots__
^^^^^^^^^
-By default, instances of classes have a dictionary for attribute storage. This
-wastes space for objects having very few instance variables. The space
-consumption can become acute when creating large numbers of instances.
-
-The default can be overridden by defining *__slots__* in a class definition.
-The *__slots__* declaration takes a sequence of instance variables and reserves
-just enough space in each instance to hold a value for each variable. Space is
-saved because *__dict__* is not created for each instance.
+*__slots__* allow us to explicitly declare data members (like
+properties) and deny the creation of *__dict__* and *__weakref__*
+(unless explicitly declared in *__slots__* or available in a parent.)
+The space saved over using *__dict__* can be significant.
.. data:: object.__slots__
Notes on using *__slots__*
""""""""""""""""""""""""""
-* When inheriting from a class without *__slots__*, the *__dict__* attribute of
- that class will always be accessible, so a *__slots__* definition in the
- subclass is meaningless.
+* When inheriting from a class without *__slots__*, the *__dict__* and
+ *__weakref__* attribute of the instances will always be accessible.
* Without a *__dict__* variable, instances cannot be assigned new variables not
listed in the *__slots__* definition. Attempts to assign to an unlisted
*__slots__*; otherwise, the class attribute would overwrite the descriptor
assignment.
-* The action of a *__slots__* declaration is limited to the class where it is
- defined. As a result, subclasses will have a *__dict__* unless they also define
- *__slots__* (which must only contain names of any *additional* slots).
+* The action of a *__slots__* declaration is not limited to the class
+ where it is defined. *__slots__* declared in parents are available in
+ child classes. However, child subclasses will get a *__dict__* and
+ *__weakref__* unless they also define *__slots__* (which should only
+ contain names of any *additional* slots).
* If a class defines a slot also defined in a base class, the instance variable
defined by the base class slot is inaccessible (except by retrieving its
* *__class__* assignment works only if both classes have the same *__slots__*.
+* Multiple inheritance with multiple slotted parent classes can be used,
+ but only one parent is allowed to have attributes created by slots
+ (the other bases must have empty slot layouts) - violations raise
+ :exc:`TypeError`.
.. _class-customization:
.. method:: object.__complex__(self)
object.__int__(self)
object.__float__(self)
- object.__round__(self, [,n])
.. index::
builtin: complex
builtin: int
builtin: float
- builtin: round
Called to implement the built-in functions :func:`complex`,
- :func:`int`, :func:`float` and :func:`round`. Should return a value
+ :func:`int` and :func:`float`. Should return a value
of the appropriate type.
the same value.
+.. method:: object.__round__(self, [,ndigits])
+ object.__trunc__(self)
+ object.__floor__(self)
+ object.__ceil__(self)
+
+ .. index:: builtin: round
+
+ Called to implement the built-in function :func:`round` and :mod:`math`
+ functions :func:`~math.trunc`, :func:`~math.floor` and :func:`~math.ceil`.
+ Unless *ndigits* is passed to :meth:`!__round__` all these methods should
+ return the value of the object truncated to an :class:`~numbers.Integral`
+ (typically an :class:`int`).
+
+ If :meth:`__int__` is not defined then the built-in function :func:`int`
+ falls back to :meth:`__trunc__`.
+
+
.. _context-managers:
With Statement Context Managers
characters as part of the literal, *not* as a line continuation.
-.. _string-catenation:
+.. _string-concatenation:
String literal concatenation
----------------------------
format specifier is omitted. The formatted result is then included in
the final value of the whole string.
-Top-level format specifiers may include nested replacement fields.
-These nested fields may include their own conversion fields and
-format specifiers, but may not include more deeply-nested replacement fields.
+Top-level format specifiers may include nested replacement fields. These nested
+fields may include their own conversion fields and :ref:`format specifiers
+<formatspec>`, but may not include more deeply-nested replacement fields. The
+:ref:`format specifier mini-language <formatspec>` is the same as that used by
+the string .format() method.
Formatted string literals may be concatenated, but replacement fields
cannot be split across literals.
>>> f"result: {value:{width}.{precision}}" # nested fields
'result: 12.35'
>>> today = datetime(year=2017, month=1, day=27)
- >>> f"{today:%b %d, %Y}" # using date format specifier
+ >>> f"{today:%B %d, %Y}" # using date format specifier
'January 27, 2017'
>>> number = 1024
>>> f"{number:#0x}" # using integer format specifier
final_argument_whitespace = True
option_spec = {}
- _label = 'Deprecated since version %s, will be removed in version %s'
+ _label = 'Deprecated since version {deprecated}, will be removed in version {removed}'
def run(self):
node = addnodes.versionmodified()
node['type'] = 'deprecated-removed'
version = (self.arguments[0], self.arguments[1])
node['version'] = version
- text = self._label % version
+ label = translators['sphinx'].gettext(self._label)
+ text = label.format(deprecated=self.arguments[0], removed=self.arguments[1])
if len(self.arguments) == 3:
inodes, messages = self.state.inline_text(self.arguments[2],
self.lineno+1)
- para = nodes.paragraph(self.arguments[2], '', *inodes)
+ para = nodes.paragraph(self.arguments[2], '', *inodes, translatable=False)
node.append(para)
else:
messages = []
content.source = node[0].source
content.line = node[0].line
content += node[0].children
- node[0].replace_self(nodes.paragraph('', '', content))
+ node[0].replace_self(nodes.paragraph('', '', content, translatable=False))
node[0].insert(0, nodes.inline('', '%s: ' % text,
classes=['versionmodified']))
else:
para = nodes.paragraph('', '',
nodes.inline('', '%s.' % text,
- classes=['versionmodified']))
+ classes=['versionmodified']),
+ translatable=False)
node.append(para)
env = self.state.document.settings.env
env.note_versionchange('deprecated', version[0], node, self.lineno)
'(?:release/\\d.\\d[\\x\\d\\.]*)'];
var all_versions = {
- '3.7': 'dev (3.7)',
+ '3.8': 'dev (3.8)',
+ '3.7': 'pre (3.7)',
'3.6': '3.6',
'3.5': '3.5',
'2.7': '2.7',
In extensions/pyspecific.py:
{% trans %}CPython implementation detail:{% endtrans %}
+{% trans %}Deprecated since version {deprecated}, will be removed in version {removed}{% endtrans %}
<p><a href="{{ pathto('download') }}">{% trans %}Download these documents{% endtrans %}</a></p>
<h3>{% trans %}Docs for other versions{% endtrans %}</h3>
<ul>
- <li><a href="https://docs.python.org/3.7/">{% trans %}Python 3.7 (in development){% endtrans %}</a></li>
+ <li><a href="https://docs.python.org/3.8/">{% trans %}Python 3.8 (in development){% endtrans %}</a></li>
+ <li><a href="https://docs.python.org/3.7/">{% trans %}Python 3.7 (pre-release){% endtrans %}</a></li>
<li><a href="https://docs.python.org/3.5/">{% trans %}Python 3.5 (stable){% endtrans %}</a></li>
<li><a href="https://docs.python.org/2.7/">{% trans %}Python 2.7 (stable){% endtrans %}</a></li>
<li><a href="https://www.python.org/doc/versions/">{% trans %}Old versions{% endtrans %}</a></li>
=====================
Some simple generators can be coded succinctly as expressions using a syntax
-similar to list comprehensions but with parentheses instead of brackets. These
-expressions are designed for situations where the generator is used right away
-by an enclosing function. Generator expressions are more compact but less
+similar to list comprehensions but with parentheses instead of square brackets.
+These expressions are designed for situations where the generator is used right
+away by an enclosing function. Generator expressions are more compact but less
versatile than full generator definitions and tend to be more memory friendly
than equivalent list comprehensions.
increment (even negative; sometimes this is called the 'step')::
range(5, 10)
- 5 through 9
+ 5, 6, 7, 8, 9
range(0, 10, 3)
0, 3, 6, 9
10 100 1000
(Note that in the first example, one space between each column was added by the
-way :func:`print` works: it always adds spaces between its arguments.)
+way :func:`print` works: by default it adds spaces between its arguments.)
This example demonstrates the :meth:`str.rjust` method of string
objects, which right-justifies a string in a field of a given width by padding
frowned upon, since it often causes poorly readable code. However, it is okay to
use it to save typing in interactive sessions.
+If the module name is followed by :keyword:`as`, then the name
+following :keyword:`as` is bound directly to the imported module.
+
+::
+
+ >>> import fibo as fib
+ >>> fib.fib(500)
+ 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
+
+This is effectively importing the module in the same way that ``import fibo``
+will do, with the only difference of it being available as ``fib``.
+
+It can also be used when utilising :keyword:`from` with similar effects::
+
+ >>> from fibo import fib as fibonacci
+ >>> fibonacci(500)
+ 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
+
+
.. note::
For efficiency reasons, each module is only imported once per interpreter
.. cmdoption:: -O
- Turn on basic optimizations. See also :envvar:`PYTHONOPTIMIZE`.
+ Remove assert statements and any code conditional on the value of
+ :const:`__debug__`. Augment the filename for compiled
+ (:term:`bytecode`) files by adding ``.opt-1`` before the ``.pyc``
+ extension (see :pep:`488`). See also :envvar:`PYTHONOPTIMIZE`.
+
+ .. versionchanged:: 3.5
+ Modify ``.pyc`` filenames according to :pep:`488`.
.. cmdoption:: -OO
- Discard docstrings in addition to the :option:`-O` optimizations.
+ Do :option:`-O` and also discard docstrings. Augment the filename
+ for compiled (:term:`bytecode`) files by adding ``.opt-2`` before the
+ ``.pyc`` extension (see :pep:`488`).
+
+ .. versionchanged:: 3.5
+ Modify ``.pyc`` filenames according to :pep:`488`.
.. cmdoption:: -q
* FreeBSD users, to add the package use::
- pkg_add -r python
+ pkg install python3
+
+* OpenBSD users, to add the package use::
-* OpenBSD users use::
+ pkg_add -r python
pkg_add ftp://ftp.openbsd.org/pub/OpenBSD/4.2/packages/<insert your architecture here>/python-<version>.tgz
was run. It also creates a ``bin`` (or ``Scripts`` on Windows) subdirectory
containing a copy of the ``python`` binary (or binaries, in the case of
Windows). It also creates an (initially empty) ``lib/pythonX.Y/site-packages``
-subdirectory (on Windows, this is ``Lib\site-packages``).
+subdirectory (on Windows, this is ``Lib\site-packages``). If an existing
+directory is specified, it will be re-used.
.. deprecated:: 3.6
``pyvenv`` was the recommended tool for creating virtual environments for
.. versionchanged:: 3.4
In earlier versions, if the target directory already existed, an error was
- raised, unless the ``--clear`` or ``--upgrade`` option was provided. Now,
- if an existing directory is specified, its contents are removed and
- the directory is processed as if it had been newly created.
+ raised, unless the ``--clear`` or ``--upgrade`` option was provided.
The created ``pyvenv.cfg`` file also includes the
``include-system-site-packages`` key, set to ``true`` if ``venv`` is
:c:func:`PyUnicode_FromFormat()`, your code will automatically take
advantage of the new unicode representations.
-* :c:func:`PyImport_GetMagicNumber` now returns -1 upon failure.
+* :c:func:`PyImport_GetMagicNumber` now returns ``-1`` upon failure.
* As a negative value for the *level* argument to :func:`__import__` is no
longer valid, the same now holds for :c:func:`PyImport_ImportModuleLevel`.
This also means that the value of *level* used by
- :c:func:`PyImport_ImportModuleEx` is now 0 instead of -1.
+ :c:func:`PyImport_ImportModuleEx` is now ``0`` instead of ``-1``.
Building C extensions
any code relying on the presence of ``default_format`` may
need to be adapted. See :issue:`27819` for more details.
+The ``upload`` command now longer tries to change CR end-of-line characters
+to CRLF. This fixes a corruption issue with sdists that ended with a byte
+equivalent to CR.
+(Contributed by Bo Bayles in :issue:`32304`.)
+
email
-----
(Contributed by Victor Stinner in :issue:`23404`.)
.. versionchanged:: 3.6.2
+
+
+Notable changes in Python 3.6.5
+===============================
+
+The :func:`locale.localeconv` function now sets temporarily the ``LC_CTYPE``
+locale to the ``LC_NUMERIC`` locale in some cases.
+(Contributed by Victor Stinner in :issue:`31900`.)
#ifdef HAVE_UNISTD_H
#include <unistd.h>
#endif
+#ifdef HAVE_CRYPT_H
+#include <crypt.h>
+#endif
/* For size_t? */
#ifdef HAVE_STDDEF_H
#ifndef Py_LIMITED_API
+PyAPI_FUNC(wchar_t *) _Py_DecodeLocaleEx(
+ const char *arg,
+ size_t *size,
+ int current_locale);
+
+PyAPI_FUNC(char*) _Py_EncodeLocaleEx(
+ const wchar_t *text,
+ size_t *error_pos,
+ int current_locale);
+
PyAPI_FUNC(PyObject *) _Py_device_encoding(int);
#ifdef MS_WINDOWS
PyAPI_FUNC(int) _Py_set_inheritable(int fd, int inheritable,
int *atomic_flag_works);
+PyAPI_FUNC(int) _Py_set_inheritable_async_safe(int fd, int inheritable,
+ int *atomic_flag_works);
+
PyAPI_FUNC(int) _Py_dup(int fd);
#ifndef MS_WINDOWS
PyAPI_FUNC(int) _Py_set_blocking(int fd, int blocking);
#endif /* !MS_WINDOWS */
+PyAPI_FUNC(int) _Py_GetLocaleconvNumeric(
+ PyObject **decimal_point,
+ PyObject **thousands_sep,
+ const char **grouping);
+
#endif /* Py_LIMITED_API */
#ifdef __cplusplus
operator new), you must first allocate the object with your custom allocator,
then pass its pointer to PyObject_{Init, InitVar} for filling in its Python-
specific fields: reference count, type pointer, possibly others. You should
-be aware that Python no control over these objects because they don't
+be aware that Python has no control over these objects because they don't
cooperate with the Python memory manager. Such objects may not be eligible
for automatic garbage collection and you have to make sure that they are
released accordingly whenever their destructor gets called (cf. the specific
/* OrderedDict */
+/* This API is optional and mostly redundant. */
#ifndef Py_LIMITED_API
#define PyODict_CheckExact(op) (Py_TYPE(op) == &PyODict_Type)
#define PyODict_SIZE(op) ((PyDictObject *)op)->ma_used
-#endif /* Py_LIMITED_API */
-
-#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03050000
-
PyAPI_FUNC(PyObject *) PyODict_New(void);
PyAPI_FUNC(int) PyODict_SetItem(PyObject *od, PyObject *key, PyObject *item);
PyAPI_FUNC(int) PyODict_DelItem(PyObject *od, PyObject *key);
/*--start constants--*/
#define PY_MAJOR_VERSION 3
#define PY_MINOR_VERSION 6
-#define PY_MICRO_VERSION 4
+#define PY_MICRO_VERSION 5
#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_FINAL
#define PY_RELEASE_SERIAL 0
/* Version as a string */
-#define PY_VERSION "3.6.4"
+#define PY_VERSION "3.6.5"
/*--end constants--*/
/* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2.
distribute, and otherwise use Python alone or in any derivative version,
provided, however, that PSF's License Agreement and PSF's notice of copyright,
i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010,
-2011, 2012, 2013, 2014, 2015, 2016, 2017 Python Software Foundation; All Rights
-Reserved" are retained in Python alone or in any derivative version prepared by
-Licensee.
+2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018 Python Software Foundation; All
+Rights Reserved" are retained in Python alone or in any derivative version
+prepared by Licensee.
3. In the event Licensee prepares a derivative work that is based on
or incorporates Python or any part thereof, and wants to make
def index(self, value, start=0, stop=None):
'''S.index(value, [start, [stop]]) -> integer -- return first index of value.
Raises ValueError if the value is not present.
+
+ Supporting start and stop arguments is optional, but
+ recommended.
'''
if start is not None and start < 0:
start = max(len(self) + start, 0)
>>> class MyLocal(local):
... number = 2
- ... initialized = False
... def __init__(self, **kw):
- ... if self.initialized:
- ... raise SystemError('__init__ called too many times')
- ... self.initialized = True
... self.__dict__.update(kw)
... def squared(self):
... return self.number ** 2
>>> thread.start()
>>> thread.join()
>>> log
- [[('color', 'red'), ('initialized', True)], 11]
+ [[('color', 'red')], 11]
without affecting this thread's data:
"""Debug helper to print the ABC registry."""
print("Class: %s.%s" % (cls.__module__, cls.__qualname__), file=file)
print("Inv.counter: %s" % ABCMeta._abc_invalidation_counter, file=file)
- for name in sorted(cls.__dict__.keys()):
+ for name in sorted(cls.__dict__):
if name.startswith("_abc_"):
value = getattr(cls, name)
+ if isinstance(value, WeakSet):
+ value = set(value)
print("%s: %r" % (name, value), file=file)
def __instancecheck__(cls, instance):
else:
raise Error('not an AIFF or AIFF-C file')
self._comm_chunk_read = 0
+ self._ssnd_chunk = None
while 1:
self._ssnd_seek_needed = 1
try:
'SO_REUSEPORT defined but not implemented.')
-def _is_stream_socket(sock):
- # Linux's socket.type is a bitmask that can include extra info
- # about socket, therefore we can't do simple
- # `sock_type == socket.SOCK_STREAM`.
- return (sock.type & socket.SOCK_STREAM) == socket.SOCK_STREAM
+def _is_stream_socket(sock_type):
+ if hasattr(socket, 'SOCK_NONBLOCK'):
+ # Linux's socket.type is a bitmask that can include extra info
+ # about socket (like SOCK_NONBLOCK bit), therefore we can't do simple
+ # `sock_type == socket.SOCK_STREAM`, see
+ # https://github.com/torvalds/linux/blob/v4.13/include/linux/net.h#L77
+ # for more details.
+ return (sock_type & 0xF) == socket.SOCK_STREAM
+ else:
+ return sock_type == socket.SOCK_STREAM
-def _is_dgram_socket(sock):
- # Linux's socket.type is a bitmask that can include extra info
- # about socket, therefore we can't do simple
- # `sock_type == socket.SOCK_DGRAM`.
- return (sock.type & socket.SOCK_DGRAM) == socket.SOCK_DGRAM
+def _is_dgram_socket(sock_type):
+ if hasattr(socket, 'SOCK_NONBLOCK'):
+ # See the comment in `_is_stream_socket`.
+ return (sock_type & 0xF) == socket.SOCK_DGRAM
+ else:
+ return sock_type == socket.SOCK_DGRAM
def _ipaddr_info(host, port, family, type, proto):
host is None:
return None
- if type == socket.SOCK_STREAM:
- # Linux only:
- # getaddrinfo() can raise when socket.type is a bit mask.
- # So if socket.type is a bit mask of SOCK_STREAM, and say
- # SOCK_NONBLOCK, we simply return None, which will trigger
- # a call to getaddrinfo() letting it process this request.
+ if _is_stream_socket(type):
proto = socket.IPPROTO_TCP
- elif type == socket.SOCK_DGRAM:
+ elif _is_dgram_socket(type):
proto = socket.IPPROTO_UDP
else:
return None
if sock is None:
raise ValueError(
'host and port was not specified and no sock specified')
- if not _is_stream_socket(sock):
+ if not _is_stream_socket(sock.type):
# We allow AF_INET, AF_INET6, AF_UNIX as long as they
# are SOCK_STREAM.
# We support passing AF_UNIX sockets even though we have
allow_broadcast=None, sock=None):
"""Create datagram connection."""
if sock is not None:
- if not _is_dgram_socket(sock):
+ if not _is_dgram_socket(sock.type):
raise ValueError(
'A UDP Socket was expected, got {!r}'.format(sock))
if (local_addr or remote_addr or
else:
if sock is None:
raise ValueError('Neither host/port nor sock were specified')
- if not _is_stream_socket(sock):
+ if not _is_stream_socket(sock.type):
raise ValueError(
'A Stream Socket was expected, got {!r}'.format(sock))
sockets = [sock]
This method is a coroutine. When completed, the coroutine
returns a (transport, protocol) pair.
"""
- if not _is_stream_socket(sock):
+ if not _is_stream_socket(sock.type):
raise ValueError(
'A Stream Socket was expected, got {!r}'.format(sock))
fut = self._loop.create_future()
self._waiters.append(fut)
+
+ # Finally block should be called before the CancelledError
+ # handling as we don't want CancelledError to call
+ # _wake_up_first() and attempt to wake up itself.
try:
- yield from fut
- self._locked = True
- return True
+ try:
+ yield from fut
+ finally:
+ self._waiters.remove(fut)
except futures.CancelledError:
if not self._locked:
self._wake_up_first()
raise
- finally:
- self._waiters.remove(fut)
+
+ self._locked = True
+ return True
def release(self):
"""Release a lock.
raise RuntimeError('Lock is not acquired.')
def _wake_up_first(self):
- """Wake up the first waiter who isn't cancelled."""
- for fut in self._waiters:
- if not fut.done():
- fut.set_result(True)
- break
+ """Wake up the first waiter if it isn't done."""
+ try:
+ fut = next(iter(self._waiters))
+ except StopIteration:
+ return
+
+ # .done() necessarily means that a waiter will wake up later on and
+ # either take the lock, or, if it was cancelled and lock wasn't
+ # taken already, will hit this again and wake up a new waiter.
+ if not fut.done():
+ fut.set_result(True)
class Event:
finally:
# Must reacquire lock even if wait is cancelled
+ cancelled = False
while True:
try:
yield from self.acquire()
break
except futures.CancelledError:
- pass
+ cancelled = True
+
+ if cancelled:
+ raise futures.CancelledError
@coroutine
def wait_for(self, predicate):
if hasattr(socket, 'TCP_NODELAY'):
def _set_nodelay(sock):
if (sock.family in {socket.AF_INET, socket.AF_INET6} and
- sock.type == socket.SOCK_STREAM and
+ base_events._is_stream_socket(sock.type) and
sock.proto == socket.IPPROTO_TCP):
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
else:
The argument is a bytes object.
"""
+ if self._sslpipe is None:
+ # transport closing, sslpipe is destroyed
+ return
+
try:
ssldata, appdata = self._sslpipe.feed_ssldata(data)
except ssl.SSLError as e:
def _process_write_backlog(self):
# Try to make progress on the write backlog.
- if self._transport is None:
+ if self._transport is None or self._sslpipe is None:
return
try:
def close(self):
super().close()
- for sig in list(self._signal_handlers):
- self.remove_signal_handler(sig)
+ if not sys.is_finalizing():
+ for sig in list(self._signal_handlers):
+ self.remove_signal_handler(sig)
+ else:
+ if self._signal_handlers:
+ warnings.warn(f"Closing the loop {self!r} "
+ f"on interpreter shutdown "
+ f"stage, skipping signal handlers removal",
+ ResourceWarning,
+ source=self)
+ self._signal_handlers.clear()
def _process_self_data(self, data):
for signum in data:
if sock is None:
raise ValueError('no path and sock were specified')
if (sock.family != socket.AF_UNIX or
- not base_events._is_stream_socket(sock)):
+ not base_events._is_stream_socket(sock.type)):
raise ValueError(
'A UNIX Domain Stream Socket was expected, got {!r}'
.format(sock))
'path was not specified, and no sock specified')
if (sock.family != socket.AF_UNIX or
- not base_events._is_stream_socket(sock)):
+ not base_events._is_stream_socket(sock.type)):
raise ValueError(
'A UNIX Domain Stream Socket was expected, got {!r}'
.format(sock))
import fnmatch
import sys
import os
-from inspect import CO_GENERATOR
+from inspect import CO_GENERATOR, CO_COROUTINE, CO_ASYNC_GENERATOR
__all__ = ["BdbQuit", "Bdb", "Breakpoint"]
+GENERATOR_AND_COROUTINE_FLAGS = CO_GENERATOR | CO_COROUTINE | CO_ASYNC_GENERATOR
+
+
class BdbQuit(Exception):
"""Exception to give up completely."""
# No need to trace this function
return # None
# Ignore call events in generator except when stepping.
- if self.stopframe and frame.f_code.co_flags & CO_GENERATOR:
+ if self.stopframe and frame.f_code.co_flags & GENERATOR_AND_COROUTINE_FLAGS:
return self.trace_dispatch
self.user_call(frame, arg)
if self.quitting: raise BdbQuit
def dispatch_return(self, frame, arg):
if self.stop_here(frame) or frame == self.returnframe:
# Ignore return events in generator except when stepping.
- if self.stopframe and frame.f_code.co_flags & CO_GENERATOR:
+ if self.stopframe and frame.f_code.co_flags & GENERATOR_AND_COROUTINE_FLAGS:
return self.trace_dispatch
try:
self.frame_returning = frame
# When stepping with next/until/return in a generator frame, skip
# the internal StopIteration exception (with no traceback)
# triggered by a subiterator run with the 'yield from' statement.
- if not (frame.f_code.co_flags & CO_GENERATOR
+ if not (frame.f_code.co_flags & GENERATOR_AND_COROUTINE_FLAGS
and arg[0] is StopIteration and arg[2] is None):
self.user_exception(frame, arg)
if self.quitting: raise BdbQuit
# next/until command at the last statement in the generator before the
# exception.
elif (self.stopframe and frame is not self.stopframe
- and self.stopframe.f_code.co_flags & CO_GENERATOR
+ and self.stopframe.f_code.co_flags & GENERATOR_AND_COROUTINE_FLAGS
and arg[0] in (StopIteration, GeneratorExit)):
self.user_exception(frame, arg)
if self.quitting: raise BdbQuit
def set_return(self, frame):
"""Stop when returning from the given frame."""
- if frame.f_code.co_flags & CO_GENERATOR:
+ if frame.f_code.co_flags & GENERATOR_AND_COROUTINE_FLAGS:
self._set_stopinfo(frame, None, -1)
else:
self._set_stopinfo(frame.f_back, frame)
matches = []
for restr in (r'(?P<delim>[^\w\n"\'])(?P<space> ?)(?P<quote>["\']).*?(?P=quote)(?P=delim)', # ,".*?",
r'(?:^|\n)(?P<quote>["\']).*?(?P=quote)(?P<delim>[^\w\n"\'])(?P<space> ?)', # ".*?",
- r'(?P<delim>>[^\w\n"\'])(?P<space> ?)(?P<quote>["\']).*?(?P=quote)(?:$|\n)', # ,".*?"
+ r'(?P<delim>[^\w\n"\'])(?P<space> ?)(?P<quote>["\']).*?(?P=quote)(?:$|\n)', # ,".*?"
r'(?:^|\n)(?P<quote>["\']).*?(?P=quote)(?:$|\n)'): # ".*?" (no delim, no space)
regexp = re.compile(restr, re.DOTALL | re.MULTILINE)
matches = regexp.findall(data)
s2h = dll.ret_2h_func(self.wrap(inp))
self.assertEqual((s2h.x, s2h.y), (99*2, 88*3))
+ # Test also that the original struct was unmodified (i.e. was passed by
+ # value)
+ self.assertEqual((inp.x, inp.y), (99, 88))
+
def test_struct_return_8H(self):
class S8I(Structure):
_fields_ = [("a", c_int),
(PackedPoint, "B", (), PackedPoint),
(Point2, "T{<l:x:<l:y:}".replace('l', s_long), (), Point2),
(EmptyStruct, "T{}", (), EmptyStruct),
- # the pep does't support unions
+ # the pep doesn't support unions
(aUnion, "B", (), aUnion),
# structure with sub-arrays
(StructWithArrays, "T{(2,3)<l:x:(4)T{<l:x:<l:y:}:y:}".replace('l', s_long), (), StructWithArrays),
self.assertEqual(sizeof(XX), 0)
def test_fields(self):
- # test the offset and size attributes of Structure/Unoin fields.
+ # test the offset and size attributes of Structure/Union fields.
class X(Structure):
_fields_ = [("x", c_int),
("y", c_char)]
self.assertEqual(s.second, 0xcafebabe)
self.assertEqual(s.third, 0x0bad1dea)
+ def test_pass_by_value_in_register(self):
+ class X(Structure):
+ _fields_ = [
+ ('first', c_uint),
+ ('second', c_uint)
+ ]
+
+ s = X()
+ s.first = 0xdeadbeef
+ s.second = 0xcafebabe
+ dll = CDLL(_ctypes_test.__file__)
+ func = dll._testfunc_reg_struct_update_value
+ func.argtypes = (X,)
+ func.restype = None
+ func(s)
+ self.assertEqual(s.first, 0xdeadbeef)
+ self.assertEqual(s.second, 0xcafebabe)
+ got = X.in_dll(dll, "last_tfrsuv_arg")
+ self.assertEqual(s.first, got.first)
+ self.assertEqual(s.second, got.second)
+
+
class PointerMemberTestCase(unittest.TestCase):
def test(self):
import re
-def IS_LINE_JUNK(line, pat=re.compile(r"\s*#?\s*$").match):
+def IS_LINE_JUNK(line, pat=re.compile(r"\s*(?:#\s*)?$").match):
r"""
Return 1 for ignorable line: iff `line` is blank or contains a single '#'.
return best_version, best_dir
def _find_vc2017():
- import _findvs
+ import _distutils_findvs
import threading
best_version = 0, # tuple for full version comparisons
# initialize COM.
all_packages = []
def _getall():
- all_packages.extend(_findvs.findall())
+ all_packages.extend(_distutils_findvs.findall())
t = threading.Thread(target=_getall)
t.start()
t.join()
# cross-building, so assume the latest version
bv = '14.0'
else:
- bv = '.'.join(CRT_ASSEMBLY_VERSION.split('.', 2)[:2])
- if bv == '14.11':
- # v141 and v140 are binary compatible,
- # so keep using the 14.0 stub.
- bv = '14.0'
+ # as far as we know, CRT is binary compatible based on
+ # the first field, so assume 'x.0' until proven otherwise
+ major = CRT_ASSEMBLY_VERSION.partition('.')[0]
+ bv = major + '.0'
# wininst-x.y.exe is in the same directory as this file
def run(self):
if not self.distribution.dist_files:
- msg = "No dist file created in earlier command"
+ msg = ("Must create and upload files in one command "
+ "(e.g. setup.py sdist upload)")
raise DistutilsOptionError(msg)
for command, pyversion, filename in self.distribution.dist_files:
self.upload_file(command, pyversion, filename)
body.write(title.encode('utf-8'))
body.write(b"\r\n\r\n")
body.write(value)
- if value and value[-1:] == b'\r':
- body.write(b'\n') # write an extra newline (lurve Macs)
body.write(end_boundary)
body = body.getvalue()
results = self.get_logs(INFO)
self.assertEqual(results[-1], 75 * '-' + '\nxyzzy\n' + 75 * '-')
+ # bpo-32304: archives whose last byte was b'\r' were corrupted due to
+ # normalization intended for Mac OS 9.
+ def test_upload_correct_cr(self):
+ # content that ends with \r should not be modified.
+ tmp = self.mkdtemp()
+ path = os.path.join(tmp, 'xxx')
+ self.write_file(path, content='yy\r')
+ command, pyversion, filename = 'xxx', '2.6', path
+ dist_files = [(command, pyversion, filename)]
+ self.write_file(self.rc, PYPIRC_LONG_PASSWORD)
+
+ # other fields that ended with \r used to be modified, now are
+ # preserved.
+ pkg_dir, dist = self.create_dist(
+ dist_files=dist_files,
+ description='long description\r'
+ )
+ cmd = upload(dist)
+ cmd.show_response = 1
+ cmd.ensure_finalized()
+ cmd.run()
+
+ headers = dict(self.last_open.req.headers)
+ self.assertEqual(headers['Content-length'], '2172')
+ self.assertIn(b'long description\r', self.last_open.req.data)
+
def test_upload_fails(self):
self.next_msg = "Not Found"
self.next_code = 404
def addr_spec(self):
for x in self:
if x.token_type == 'addr-spec':
- return x.addr_spec
+ if x.local_part:
+ return x.addr_spec
+ else:
+ return quote_string(x.local_part) + x.addr_spec
else:
return '<>'
"expected '\"' but found '{}'".format(value))
bare_quoted_string = BareQuotedString()
value = value[1:]
+ if value[0] == '"':
+ token, value = get_qcontent(value)
+ bare_quoted_string.append(token)
while value and value[0] != '"':
if value[0] in WSP:
token, value = get_fws(value)
Using the decoded list of parameters and values, format them according to
the RFC rules, including using RFC2231 encoding if the value cannot be
- expressed in 'encoding' and/or the paramter+value is too long to fit within
- 'maxlen'.
+ expressed in 'encoding' and/or the parameter+value is too long to fit
+ within 'maxlen'.
"""
# Special case for RFC2231 encoding: start from decoded values and use
if not body:
return body
- # quote speacial characters
+ # quote special characters
body = body.translate(_QUOPRI_BODY_ENCODE_MAP)
soft_break = '=' + eol
__all__ = ["version", "bootstrap"]
-_SETUPTOOLS_VERSION = "28.8.0"
+_SETUPTOOLS_VERSION = "39.0.1"
-_PIP_VERSION = "9.0.1"
+_PIP_VERSION = "9.0.3"
_PROJECTS = [
("setuptools", _SETUPTOOLS_VERSION),
chunk_left = self.chunk_left
if not chunk_left: # Can be 0 or None
if chunk_left is not None:
- # We are at the end of chunk. dicard chunk end
+ # We are at the end of chunk, discard chunk end
self._safe_read(2) # toss the CRLF at the end of the chunk
try:
chunk_left = self._read_next_chunk_size()
-What's New in IDLE 3.6.3
-Released on 2017-09-25?
+What's New in IDLE 3.6.5
+Released on 2017-03-26?
+======================================
+
+
+bpo-32916: Change 'str' to 'code' in idlelib.pyparse and users.
+
+bpo-32905: Remove unused code in pyparse module.
+
+bpo-32874: IDLE - add pyparse tests with 97% coverage.
+
+bpo-32837: IDLE - require encoding argument for textview.view_file.
+Using the system and place-dependent default encoding for open()
+is a bad idea for IDLE's system and location-independent files.
+
+bpo-32826: Add "encoding=utf-8" to open() in IDLE's test_help_about.
+GUI test test_file_buttons() only looks at initial ascii-only lines,
+but failed on systems where open() defaults to 'ascii' because
+readline() internally reads and decodes far enough ahead to encounter
+a non-ascii character in CREDITS.txt.
+
+bpo-32765: Update configdialog General tab create page docstring.
+Add new widgets to the widget list.
+
+
+What's New in IDLE 3.6.4
+Released on 2017-12-19
========================
+bpo-32207: Improve tk event exception tracebacks in IDLE.
+When tk event handling is driven by IDLE's run loop, a confusing
+and distracting queue.EMPTY traceback context is no longer added
+to tk event exception tracebacks. The traceback is now the same
+as when event handling is driven by user code. Patch based on
+a suggestion by Serhiy Storchaka.
bpo-32164: Delete unused file idlelib/tabbedpages.py.
Use of TabbedPageSet in configdialog was replaced by ttk.Notebook.
re-arranged. The Font/Tabs help explains a bit about the additions.
Patch by Terry Jan Reedy
+
+What's New in IDLE 3.6.3
+Released on 2017-10-03
+========================
+
bpo-31460: Simplify the API of IDLE's Module Browser.
Passing a widget instead of an flist with a root widget opens the
option of creating a browser frame that is only part of a window.
IDLE calls tcl/tk update in the background in order to make live
interaction and experimentatin with tkinter applications much easier.
-bpo-bpo-31414: Fix tk entry box tests by deleting first.
+bpo-31414: Fix tk entry box tests by deleting first.
Adding to an int entry is not the same as deleting and inserting
because int('') will fail. Patch by Terry Jan Reedy.
What's New in IDLE 3.6.2
-Released on 2017-07-11
+Released on 2017-07-17
========================
bpo-15786: Fix several problems with IDLE's autocompletion box.
Update and augment description of the configuration system.
-What's New in IDLE 3.6.0
+What's New in IDLE 3.6.0 (since 3.5.0)
Released on 2016-12-23
-========================
+======================================
- Issue #15308: Add 'interrupt execution' (^C) to Shell menu.
Patch by Roger Serwy, updated by Bayard Randel.
acw.wm_geometry("+%d+%d" % (new_x, new_y))
if platform.system().startswith('Windows'):
- # See issue 15786. When on windows platform, Tk will misbehaive
+ # See issue 15786. When on windows platform, Tk will misbehave
# to call winconfig_event multiple times, we need to prevent this,
# otherwise mouse button double click will not be able to used.
acw.unbind(WINCONFIG_SEQUENCE, self.winconfigid)
Changing any of the font vars invokes var_changed_font, which
adds all 3 font options to changes and calls set_samples.
Set_samples applies a new font constructed from the font vars to
- font_sample and to highlight_sample on the hightlight page.
+ font_sample and to highlight_sample on the highlight page.
Tabs: Enable users to change spaces entered for indent tabs.
Changing indent_scale value with the mouse sets Var space_num,
Called on font initialization and change events.
Accesses font_name, font_size, and font_bold Variables.
- Updates font_sample and hightlight page highlight_sample.
+ Updates font_sample and highlight page highlight_sample.
"""
font_name = self.font_name.get()
font_weight = tkFont.BOLD if self.font_bold.get() else tkFont.NORMAL
(*)win_width_int: Entry - win_width
win_height_title: Label
(*)win_height_int: Entry - win_height
+ frame_autocomplete: Frame
+ auto_wait_title: Label
+ (*)auto_wait_int: Entry - autocomplete_wait
+ frame_paren1: Frame
+ paren_style_title: Label
+ (*)paren_style_type: OptionMenu - paren_style
+ frame_paren2: Frame
+ paren_time_title: Label
+ (*)paren_flash_time: Entry - flash_delay
+ (*)bell_on: Checkbutton - paren_bell
frame_editor: LabelFrame
frame_save: Frame
run_save_title: Label
(*)save_ask_on: Radiobutton - autosave
(*)save_auto_on: Radiobutton - autosave
+ frame_format: Frame
+ format_width_title: Label
+ (*)format_width_int: Entry - format_width
+ frame_context: Frame
+ context_title: Label
+ (*)context_int: Entry - context_lines
frame_help: LabelFrame
frame_helplist: Frame
frame_helplist_buttons: Frame
startat = max(lno - context, 1)
startatindex = repr(startat) + ".0"
rawtext = text.get(startatindex, "insert")
- y.set_str(rawtext)
+ y.set_code(rawtext)
bod = y.find_good_parse_start(
self.context_use_ps1,
self._build_char_in_string_func(startatindex))
else:
startatindex = "1.0"
rawtext = text.get(startatindex, "insert")
- y.set_str(rawtext)
+ y.set_code(rawtext)
y.set_lo(0)
c = y.get_continuation_type()
# at end. We add a space so that index won't be at end
# of line, so that its status will be the same as the
# char before it, if should.
- parser.set_str(text.get(startatindex, stopatindex)+' \n')
+ parser.set_code(text.get(startatindex, stopatindex)+' \n')
bod = parser.find_good_parse_start(
editwin._build_char_in_string_func(startatindex))
if bod is not None or startat == 1:
# We add the newline because PyParse requires it. We add a
# space so that index won't be at end of line, so that its
# status will be the same as the char before it, if should.
- parser.set_str(text.get(startatindex, stopatindex)+' \n')
+ parser.set_code(text.get(startatindex, stopatindex)+' \n')
parser.set_lo(0)
# We want what the parser has, minus the last newline and space.
- self.rawtext = parser.str[:-2]
- # Parser.str apparently preserves the statement we are in, so
+ self.rawtext = parser.code[:-2]
+ # Parser.code apparently preserves the statement we are in, so
# that stopatindex can be used to synchronize the string with
# the text box indices.
self.stopatindex = stopatindex
def test_printer_buttons(self):
"""Test buttons whose commands use printer function."""
dialog = self.dialog
- button_sources = [(dialog.py_license, license),
- (dialog.py_copyright, copyright),
- (dialog.py_credits, credits)]
-
- for button, printer in button_sources:
- printer._Printer__setup()
- button.invoke()
- get = dialog._current_textview.viewframe.textframe.text.get
- self.assertEqual(printer._Printer__lines[0], get('1.0', '1.end'))
- self.assertEqual(
- printer._Printer__lines[1], get('2.0', '2.end'))
- dialog._current_textview.destroy()
+ button_sources = [(dialog.py_license, license, 'license'),
+ (dialog.py_copyright, copyright, 'copyright'),
+ (dialog.py_credits, credits, 'credits')]
+
+ for button, printer, name in button_sources:
+ with self.subTest(name=name):
+ printer._Printer__setup()
+ button.invoke()
+ get = dialog._current_textview.viewframe.textframe.text.get
+ lines = printer._Printer__lines
+ self.assertEqual(lines[0], get('1.0', '1.end'))
+ self.assertEqual(lines[1], get('2.0', '2.end'))
+ dialog._current_textview.destroy()
def test_file_buttons(self):
"""Test buttons that display files."""
dialog = self.dialog
- button_sources = [(self.dialog.readme, 'README.txt'),
- (self.dialog.idle_news, 'NEWS.txt'),
- (self.dialog.idle_credits, 'CREDITS.txt')]
-
- for button, filename in button_sources:
- button.invoke()
- fn = findfile(filename, subdir='idlelib')
- get = dialog._current_textview.viewframe.textframe.text.get
- with open(fn) as f:
- self.assertEqual(f.readline().strip(), get('1.0', '1.end'))
- f.readline()
- self.assertEqual(f.readline().strip(), get('3.0', '3.end'))
- dialog._current_textview.destroy()
+ button_sources = [(self.dialog.readme, 'README.txt', 'readme'),
+ (self.dialog.idle_news, 'NEWS.txt', 'news'),
+ (self.dialog.idle_credits, 'CREDITS.txt', 'credits')]
+
+ for button, filename, name in button_sources:
+ with self.subTest(name=name):
+ button.invoke()
+ fn = findfile(filename, subdir='idlelib')
+ get = dialog._current_textview.viewframe.textframe.text.get
+ with open(fn, encoding='utf-8') as f:
+ self.assertEqual(f.readline().strip(), get('1.0', '1.end'))
+ f.readline()
+ self.assertEqual(f.readline().strip(), get('3.0', '3.end'))
+ dialog._current_textview.destroy()
class DefaultTitleTest(unittest.TestCase):
--- /dev/null
+"""Unittest for idlelib.pyparse.py.
+
+Coverage: 97%
+"""
+
+from collections import namedtuple
+import unittest
+from idlelib import pyparse
+
+
+class ParseMapTest(unittest.TestCase):
+
+ def test_parsemap(self):
+ keepwhite = {ord(c): ord(c) for c in ' \t\n\r'}
+ mapping = pyparse.ParseMap(keepwhite)
+ self.assertEqual(mapping[ord('\t')], ord('\t'))
+ self.assertEqual(mapping[ord('a')], ord('x'))
+ self.assertEqual(mapping[1000], ord('x'))
+
+ def test_trans(self):
+ # trans is the production instance of ParseMap, used in _study1
+ parser = pyparse.Parser(4, 4)
+ self.assertEqual('\t a([{b}])b"c\'d\n'.translate(pyparse.trans),
+ 'xxx(((x)))x"x\'x\n')
+
+
+class PyParseTest(unittest.TestCase):
+
+ @classmethod
+ def setUpClass(cls):
+ cls.parser = pyparse.Parser(indentwidth=4, tabwidth=4)
+
+ @classmethod
+ def tearDownClass(cls):
+ del cls.parser
+
+ def test_init(self):
+ self.assertEqual(self.parser.indentwidth, 4)
+ self.assertEqual(self.parser.tabwidth, 4)
+
+ def test_set_code(self):
+ eq = self.assertEqual
+ p = self.parser
+ setcode = p.set_code
+
+ # Not empty and doesn't end with newline.
+ with self.assertRaises(AssertionError):
+ setcode('a')
+
+ tests = ('',
+ 'a\n')
+
+ for string in tests:
+ with self.subTest(string=string):
+ setcode(string)
+ eq(p.code, string)
+ eq(p.study_level, 0)
+
+ def test_find_good_parse_start(self):
+ eq = self.assertEqual
+ p = self.parser
+ setcode = p.set_code
+ start = p.find_good_parse_start
+
+ # Split def across lines.
+ setcode('"""This is a module docstring"""\n'
+ 'class C():\n'
+ ' def __init__(self, a,\n'
+ ' b=True):\n'
+ ' pass\n'
+ )
+
+ # No value sent for is_char_in_string().
+ self.assertIsNone(start())
+
+ # Make text look like a string. This returns pos as the start
+ # position, but it's set to None.
+ self.assertIsNone(start(is_char_in_string=lambda index: True))
+
+ # Make all text look like it's not in a string. This means that it
+ # found a good start position.
+ eq(start(is_char_in_string=lambda index: False), 44)
+
+ # If the beginning of the def line is not in a string, then it
+ # returns that as the index.
+ eq(start(is_char_in_string=lambda index: index > 44), 44)
+ # If the beginning of the def line is in a string, then it
+ # looks for a previous index.
+ eq(start(is_char_in_string=lambda index: index >= 44), 33)
+ # If everything before the 'def' is in a string, then returns None.
+ # The non-continuation def line returns 44 (see below).
+ eq(start(is_char_in_string=lambda index: index < 44), None)
+
+ # Code without extra line break in def line - mostly returns the same
+ # values.
+ setcode('"""This is a module docstring"""\n'
+ 'class C():\n'
+ ' def __init__(self, a, b=True):\n'
+ ' pass\n'
+ )
+ eq(start(is_char_in_string=lambda index: False), 44)
+ eq(start(is_char_in_string=lambda index: index > 44), 44)
+ eq(start(is_char_in_string=lambda index: index >= 44), 33)
+ # When the def line isn't split, this returns which doesn't match the
+ # split line test.
+ eq(start(is_char_in_string=lambda index: index < 44), 44)
+
+ def test_set_lo(self):
+ code = (
+ '"""This is a module docstring"""\n'
+ 'class C():\n'
+ ' def __init__(self, a,\n'
+ ' b=True):\n'
+ ' pass\n'
+ )
+ p = self.parser
+ p.set_code(code)
+
+ # Previous character is not a newline.
+ with self.assertRaises(AssertionError):
+ p.set_lo(5)
+
+ # A value of 0 doesn't change self.code.
+ p.set_lo(0)
+ self.assertEqual(p.code, code)
+
+ # An index that is preceded by a newline.
+ p.set_lo(44)
+ self.assertEqual(p.code, code[44:])
+
+ def test_study1(self):
+ eq = self.assertEqual
+ p = self.parser
+ setcode = p.set_code
+ study = p._study1
+
+ (NONE, BACKSLASH, FIRST, NEXT, BRACKET) = range(5)
+ TestInfo = namedtuple('TestInfo', ['string', 'goodlines',
+ 'continuation'])
+ tests = (
+ TestInfo('', [0], NONE),
+ # Docstrings.
+ TestInfo('"""This is a complete docstring."""\n', [0, 1], NONE),
+ TestInfo("'''This is a complete docstring.'''\n", [0, 1], NONE),
+ TestInfo('"""This is a continued docstring.\n', [0, 1], FIRST),
+ TestInfo("'''This is a continued docstring.\n", [0, 1], FIRST),
+ TestInfo('"""Closing quote does not match."\n', [0, 1], FIRST),
+ TestInfo('"""Bracket in docstring [\n', [0, 1], FIRST),
+ TestInfo("'''Incomplete two line docstring.\n\n", [0, 2], NEXT),
+ # Single-quoted strings.
+ TestInfo('"This is a complete string."\n', [0, 1], NONE),
+ TestInfo('"This is an incomplete string.\n', [0, 1], NONE),
+ TestInfo("'This is more incomplete.\n\n", [0, 1, 2], NONE),
+ # Comment (backslash does not continue comments).
+ TestInfo('# Comment\\\n', [0, 1], NONE),
+ # Brackets.
+ TestInfo('("""Complete string in bracket"""\n', [0, 1], BRACKET),
+ TestInfo('("""Open string in bracket\n', [0, 1], FIRST),
+ TestInfo('a = (1 + 2) - 5 *\\\n', [0, 1], BACKSLASH), # No bracket.
+ TestInfo('\n def function1(self, a,\n b):\n',
+ [0, 1, 3], NONE),
+ TestInfo('\n def function1(self, a,\\\n', [0, 1, 2], BRACKET),
+ TestInfo('\n def function1(self, a,\n', [0, 1, 2], BRACKET),
+ TestInfo('())\n', [0, 1], NONE), # Extra closer.
+ TestInfo(')(\n', [0, 1], BRACKET), # Extra closer.
+ # For the mismatched example, it doesn't look like contination.
+ TestInfo('{)(]\n', [0, 1], NONE), # Mismatched.
+ )
+
+ for test in tests:
+ with self.subTest(string=test.string):
+ setcode(test.string) # resets study_level
+ study()
+ eq(p.study_level, 1)
+ eq(p.goodlines, test.goodlines)
+ eq(p.continuation, test.continuation)
+
+ # Called again, just returns without reprocessing.
+ self.assertIsNone(study())
+
+ def test_get_continuation_type(self):
+ eq = self.assertEqual
+ p = self.parser
+ setcode = p.set_code
+ gettype = p.get_continuation_type
+
+ (NONE, BACKSLASH, FIRST, NEXT, BRACKET) = range(5)
+ TestInfo = namedtuple('TestInfo', ['string', 'continuation'])
+ tests = (
+ TestInfo('', NONE),
+ TestInfo('"""This is a continuation docstring.\n', FIRST),
+ TestInfo("'''This is a multiline-continued docstring.\n\n", NEXT),
+ TestInfo('a = (1 + 2) - 5 *\\\n', BACKSLASH),
+ TestInfo('\n def function1(self, a,\\\n', BRACKET)
+ )
+
+ for test in tests:
+ with self.subTest(string=test.string):
+ setcode(test.string)
+ eq(gettype(), test.continuation)
+
+ def test_study2(self):
+ eq = self.assertEqual
+ p = self.parser
+ setcode = p.set_code
+ study = p._study2
+
+ TestInfo = namedtuple('TestInfo', ['string', 'start', 'end', 'lastch',
+ 'openbracket', 'bracketing'])
+ tests = (
+ TestInfo('', 0, 0, '', None, ((0, 0),)),
+ TestInfo("'''This is a multiline continutation docstring.\n\n",
+ 0, 49, "'", None, ((0, 0), (0, 1), (49, 0))),
+ TestInfo(' # Comment\\\n',
+ 0, 12, '', None, ((0, 0), (1, 1), (12, 0))),
+ # A comment without a space is a special case
+ TestInfo(' #Comment\\\n',
+ 0, 0, '', None, ((0, 0),)),
+ # Backslash continuation.
+ TestInfo('a = (1 + 2) - 5 *\\\n',
+ 0, 19, '*', None, ((0, 0), (4, 1), (11, 0))),
+ # Bracket continuation with close.
+ TestInfo('\n def function1(self, a,\n b):\n',
+ 1, 48, ':', None, ((1, 0), (17, 1), (46, 0))),
+ # Bracket continuation with unneeded backslash.
+ TestInfo('\n def function1(self, a,\\\n',
+ 1, 28, ',', 17, ((1, 0), (17, 1))),
+ # Bracket continuation.
+ TestInfo('\n def function1(self, a,\n',
+ 1, 27, ',', 17, ((1, 0), (17, 1))),
+ # Bracket continuation with comment at end of line with text.
+ TestInfo('\n def function1(self, a, # End of line comment.\n',
+ 1, 51, ',', 17, ((1, 0), (17, 1), (28, 2), (51, 1))),
+ # Multi-line statement with comment line in between code lines.
+ TestInfo(' a = ["first item",\n # Comment line\n "next item",\n',
+ 0, 55, ',', 6, ((0, 0), (6, 1), (7, 2), (19, 1),
+ (23, 2), (38, 1), (42, 2), (53, 1))),
+ TestInfo('())\n',
+ 0, 4, ')', None, ((0, 0), (0, 1), (2, 0), (3, 0))),
+ TestInfo(')(\n', 0, 3, '(', 1, ((0, 0), (1, 0), (1, 1))),
+ # Wrong closers still decrement stack level.
+ TestInfo('{)(]\n',
+ 0, 5, ']', None, ((0, 0), (0, 1), (2, 0), (2, 1), (4, 0))),
+ # Character after backslash.
+ TestInfo(':\\a\n', 0, 4, '\\a', None, ((0, 0),)),
+ TestInfo('\n', 0, 0, '', None, ((0, 0),)),
+ )
+
+ for test in tests:
+ with self.subTest(string=test.string):
+ setcode(test.string)
+ study()
+ eq(p.study_level, 2)
+ eq(p.stmt_start, test.start)
+ eq(p.stmt_end, test.end)
+ eq(p.lastch, test.lastch)
+ eq(p.lastopenbracketpos, test.openbracket)
+ eq(p.stmt_bracketing, test.bracketing)
+
+ # Called again, just returns without reprocessing.
+ self.assertIsNone(study())
+
+ def test_get_num_lines_in_stmt(self):
+ eq = self.assertEqual
+ p = self.parser
+ setcode = p.set_code
+ getlines = p.get_num_lines_in_stmt
+
+ TestInfo = namedtuple('TestInfo', ['string', 'lines'])
+ tests = (
+ TestInfo('[x for x in a]\n', 1), # Closed on one line.
+ TestInfo('[x\nfor x in a\n', 2), # Not closed.
+ TestInfo('[x\\\nfor x in a\\\n', 2), # "", uneeded backslashes.
+ TestInfo('[x\nfor x in a\n]\n', 3), # Closed on multi-line.
+ TestInfo('\n"""Docstring comment L1"""\nL2\nL3\nL4\n', 1),
+ TestInfo('\n"""Docstring comment L1\nL2"""\nL3\nL4\n', 1),
+ TestInfo('\n"""Docstring comment L1\\\nL2\\\nL3\\\nL4\\\n', 4),
+ TestInfo('\n\n"""Docstring comment L1\\\nL2\\\nL3\\\nL4\\\n"""\n', 5)
+ )
+
+ # Blank string doesn't have enough elements in goodlines.
+ setcode('')
+ with self.assertRaises(IndexError):
+ getlines()
+
+ for test in tests:
+ with self.subTest(string=test.string):
+ setcode(test.string)
+ eq(getlines(), test.lines)
+
+ def test_compute_bracket_indent(self):
+ eq = self.assertEqual
+ p = self.parser
+ setcode = p.set_code
+ indent = p.compute_bracket_indent
+
+ TestInfo = namedtuple('TestInfo', ['string', 'spaces'])
+ tests = (
+ TestInfo('def function1(self, a,\n', 14),
+ # Characters after bracket.
+ TestInfo('\n def function1(self, a,\n', 18),
+ TestInfo('\n\tdef function1(self, a,\n', 18),
+ # No characters after bracket.
+ TestInfo('\n def function1(\n', 8),
+ TestInfo('\n\tdef function1(\n', 8),
+ TestInfo('\n def function1( \n', 8), # Ignore extra spaces.
+ TestInfo('[\n"first item",\n # Comment line\n "next item",\n', 0),
+ TestInfo('[\n "first item",\n # Comment line\n "next item",\n', 2),
+ TestInfo('["first item",\n # Comment line\n "next item",\n', 1),
+ TestInfo('(\n', 4),
+ TestInfo('(a\n', 1),
+ )
+
+ # Must be C_BRACKET continuation type.
+ setcode('def function1(self, a, b):\n')
+ with self.assertRaises(AssertionError):
+ indent()
+
+ for test in tests:
+ setcode(test.string)
+ eq(indent(), test.spaces)
+
+ def test_compute_backslash_indent(self):
+ eq = self.assertEqual
+ p = self.parser
+ setcode = p.set_code
+ indent = p.compute_backslash_indent
+
+ # Must be C_BACKSLASH continuation type.
+ errors = (('def function1(self, a, b\\\n'), # Bracket.
+ (' """ (\\\n'), # Docstring.
+ ('a = #\\\n'), # Inline comment.
+ )
+ for string in errors:
+ with self.subTest(string=string):
+ setcode(string)
+ with self.assertRaises(AssertionError):
+ indent()
+
+ TestInfo = namedtuple('TestInfo', ('string', 'spaces'))
+ tests = (TestInfo('a = (1 + 2) - 5 *\\\n', 4),
+ TestInfo('a = 1 + 2 - 5 *\\\n', 4),
+ TestInfo(' a = 1 + 2 - 5 *\\\n', 8),
+ TestInfo(' a = "spam"\\\n', 6),
+ TestInfo(' a = \\\n"a"\\\n', 4),
+ TestInfo(' a = #\\\n"a"\\\n', 5),
+ TestInfo('a == \\\n', 2),
+ TestInfo('a != \\\n', 2),
+ # Difference between containing = and those not.
+ TestInfo('\\\n', 2),
+ TestInfo(' \\\n', 6),
+ TestInfo('\t\\\n', 6),
+ TestInfo('a\\\n', 3),
+ TestInfo('{}\\\n', 4),
+ TestInfo('(1 + 2) - 5 *\\\n', 3),
+ )
+ for test in tests:
+ with self.subTest(string=test.string):
+ setcode(test.string)
+ eq(indent(), test.spaces)
+
+ def test_get_base_indent_string(self):
+ eq = self.assertEqual
+ p = self.parser
+ setcode = p.set_code
+ baseindent = p.get_base_indent_string
+
+ TestInfo = namedtuple('TestInfo', ['string', 'indent'])
+ tests = (TestInfo('', ''),
+ TestInfo('def a():\n', ''),
+ TestInfo('\tdef a():\n', '\t'),
+ TestInfo(' def a():\n', ' '),
+ TestInfo(' def a(\n', ' '),
+ TestInfo('\t\n def a(\n', ' '),
+ TestInfo('\t\n # Comment.\n', ' '),
+ )
+
+ for test in tests:
+ with self.subTest(string=test.string):
+ setcode(test.string)
+ eq(baseindent(), test.indent)
+
+ def test_is_block_opener(self):
+ yes = self.assertTrue
+ no = self.assertFalse
+ p = self.parser
+ setcode = p.set_code
+ opener = p.is_block_opener
+
+ TestInfo = namedtuple('TestInfo', ['string', 'assert_'])
+ tests = (
+ TestInfo('def a():\n', yes),
+ TestInfo('\n def function1(self, a,\n b):\n', yes),
+ TestInfo(':\n', yes),
+ TestInfo('a:\n', yes),
+ TestInfo('):\n', yes),
+ TestInfo('(:\n', yes),
+ TestInfo('":\n', no),
+ TestInfo('\n def function1(self, a,\n', no),
+ TestInfo('def function1(self, a):\n pass\n', no),
+ TestInfo('# A comment:\n', no),
+ TestInfo('"""A docstring:\n', no),
+ TestInfo('"""A docstring:\n', no),
+ )
+
+ for test in tests:
+ with self.subTest(string=test.string):
+ setcode(test.string)
+ test.assert_(opener())
+
+ def test_is_block_closer(self):
+ yes = self.assertTrue
+ no = self.assertFalse
+ p = self.parser
+ setcode = p.set_code
+ closer = p.is_block_closer
+
+ TestInfo = namedtuple('TestInfo', ['string', 'assert_'])
+ tests = (
+ TestInfo('return\n', yes),
+ TestInfo('\tbreak\n', yes),
+ TestInfo(' continue\n', yes),
+ TestInfo(' raise\n', yes),
+ TestInfo('pass \n', yes),
+ TestInfo('pass\t\n', yes),
+ TestInfo('return #\n', yes),
+ TestInfo('raised\n', no),
+ TestInfo('returning\n', no),
+ TestInfo('# return\n', no),
+ TestInfo('"""break\n', no),
+ TestInfo('"continue\n', no),
+ TestInfo('def function1(self, a):\n pass\n', yes),
+ )
+
+ for test in tests:
+ with self.subTest(string=test.string):
+ setcode(test.string)
+ test.assert_(closer())
+
+ def test_get_last_stmt_bracketing(self):
+ eq = self.assertEqual
+ p = self.parser
+ setcode = p.set_code
+ bracketing = p.get_last_stmt_bracketing
+
+ TestInfo = namedtuple('TestInfo', ['string', 'bracket'])
+ tests = (
+ TestInfo('', ((0, 0),)),
+ TestInfo('a\n', ((0, 0),)),
+ TestInfo('()()\n', ((0, 0), (0, 1), (2, 0), (2, 1), (4, 0))),
+ TestInfo('(\n)()\n', ((0, 0), (0, 1), (3, 0), (3, 1), (5, 0))),
+ TestInfo('()\n()\n', ((3, 0), (3, 1), (5, 0))),
+ TestInfo('()(\n)\n', ((0, 0), (0, 1), (2, 0), (2, 1), (5, 0))),
+ TestInfo('(())\n', ((0, 0), (0, 1), (1, 2), (3, 1), (4, 0))),
+ TestInfo('(\n())\n', ((0, 0), (0, 1), (2, 2), (4, 1), (5, 0))),
+ # Same as matched test.
+ TestInfo('{)(]\n', ((0, 0), (0, 1), (2, 0), (2, 1), (4, 0))),
+ TestInfo('(((())\n',
+ ((0, 0), (0, 1), (1, 2), (2, 3), (3, 4), (5, 3), (6, 2))),
+ )
+
+ for test in tests:
+ with self.subTest(string=test.string):
+ setcode(test.string)
+ eq(bracketing(), test.bracket)
+
+
+if __name__ == '__main__':
+ unittest.main(verbosity=2)
# Path is widget entry, either '' or something.
# Func return is file dialog return, either '' or something.
# Func return should override widget entry.
- # We need all 4 combination to test all (most) code paths.
+ # We need all 4 combinations to test all (most) code paths.
for path, func, result in (
('', lambda a,b,c:'', ''),
('', lambda a,b,c: __file__, __file__),
"""Test SearchDialog class in idlelib.search.py"""
# Does not currently test the event handler wrappers.
-# A usage test should simulate clicks and check hilighting.
+# A usage test should simulate clicks and check highlighting.
# Tests need to be coordinated with SearchDialogBase tests
# to avoid duplication.
view.ok()
def test_view_file(self):
- view = tv.view_file(root, 'Title', __file__, modal=False)
+ view = tv.view_file(root, 'Title', __file__, 'ascii', modal=False)
self.assertIsInstance(view, tv.ViewWindow)
self.assertIsInstance(view.viewframe, tv.ViewFrame)
get = view.viewframe.textframe.text.get
def test_bad_file(self):
# Mock showerror will be used; view_file will return None.
- view = tv.view_file(root, 'Title', 'abc.xyz', modal=False)
+ view = tv.view_file(root, 'Title', 'abc.xyz', 'ascii', modal=False)
self.assertIsNone(view)
self.assertEqual(tv.showerror.title, 'File Load Error')
def test_view_file_bind_with_button(self):
def _command():
self.called = True
- self.view = tv.view_file(root, 'TITLE_FILE', __file__, _utest=True)
+ self.view = tv.view_file(root, 'TITLE_FILE', __file__,
+ encoding='ascii', _utest=True)
button = Button(root, text='BUTTON', command=_command)
button.invoke()
self.addCleanup(button.destroy)
-from collections.abc import Mapping
+"""Define partial Python code Parser used by editor and hyperparser.
+
+Instances of ParseMap are used with str.translate.
+
+The following bound search and match functions are defined:
+_synchre - start of popular statement;
+_junkre - whitespace or comment line;
+_match_stringre: string, possibly without closer;
+_itemre - line that may have bracket structure start;
+_closere - line that must be followed by dedent.
+_chew_ordinaryre - non-special characters.
+"""
import re
import sys
-# Reason last stmt is continued (or C_NONE if it's not).
+# Reason last statement is continued (or C_NONE if it's not).
(C_NONE, C_BACKSLASH, C_STRING_FIRST_LINE,
C_STRING_NEXT_LINES, C_BRACKET) = range(5)
-if 0: # for throwaway debugging output
- def dump(*stuff):
- sys.__stdout__.write(" ".join(map(str, stuff)) + "\n")
-
-# Find what looks like the start of a popular stmt.
+# Find what looks like the start of a popular statement.
_synchre = re.compile(r"""
^
[^\s#\\] # if we match, m.end()-1 is the interesting char
""", re.VERBOSE).match
-# Match start of stmts that should be followed by a dedent.
+# Match start of statements that should be followed by a dedent.
_closere = re.compile(r"""
\s*
""", re.VERBOSE).match
-class StringTranslatePseudoMapping(Mapping):
- r"""Utility class to be used with str.translate()
-
- This Mapping class wraps a given dict. When a value for a key is
- requested via __getitem__() or get(), the key is looked up in the
- given dict. If found there, the value from the dict is returned.
- Otherwise, the default value given upon initialization is returned.
+class ParseMap(dict):
+ r"""Dict subclass that maps anything not in dict to 'x'.
- This allows using str.translate() to make some replacements, and to
- replace all characters for which no replacement was specified with
- a given character instead of leaving them as-is.
+ This is designed to be used with str.translate in study1.
+ Anything not specifically mapped otherwise becomes 'x'.
+ Example: replace everything except whitespace with 'x'.
- For example, to replace everything except whitespace with 'x':
-
- >>> whitespace_chars = ' \t\n\r'
- >>> preserve_dict = {ord(c): ord(c) for c in whitespace_chars}
- >>> mapping = StringTranslatePseudoMapping(preserve_dict, ord('x'))
- >>> text = "a + b\tc\nd"
- >>> text.translate(mapping)
+ >>> keepwhite = ParseMap((ord(c), ord(c)) for c in ' \t\n\r')
+ >>> "a + b\tc\nd".translate(keepwhite)
'x x x\tx\nx'
"""
- def __init__(self, non_defaults, default_value):
- self._non_defaults = non_defaults
- self._default_value = default_value
-
- def _get(key, _get=non_defaults.get, _default=default_value):
- return _get(key, _default)
- self._get = _get
+ # Calling this triples access time; see bpo-32940
+ def __missing__(self, key):
+ return 120 # ord('x')
- def __getitem__(self, item):
- return self._get(item)
- def __len__(self):
- return len(self._non_defaults)
-
- def __iter__(self):
- return iter(self._non_defaults)
-
- def get(self, key, default=None):
- return self._get(key)
+# Map all ascii to 120 to avoid __missing__ call, then replace some.
+trans = ParseMap.fromkeys(range(128), 120)
+trans.update((ord(c), ord('(')) for c in "({[") # open brackets => '(';
+trans.update((ord(c), ord(')')) for c in ")}]") # close brackets => ')'.
+trans.update((ord(c), ord(c)) for c in "\"'\\\n#") # Keep these.
class Parser:
self.indentwidth = indentwidth
self.tabwidth = tabwidth
- def set_str(self, s):
+ def set_code(self, s):
assert len(s) == 0 or s[-1] == '\n'
- self.str = s
+ self.code = s
self.study_level = 0
- # Return index of a good place to begin parsing, as close to the
- # end of the string as possible. This will be the start of some
- # popular stmt like "if" or "def". Return None if none found:
- # the caller should pass more prior context then, if possible, or
- # if not (the entire program text up until the point of interest
- # has already been tried) pass 0 to set_lo.
- #
- # This will be reliable iff given a reliable is_char_in_string
- # function, meaning that when it says "no", it's absolutely
- # guaranteed that the char is not in a string.
-
def find_good_parse_start(self, is_char_in_string=None,
_synchre=_synchre):
- str, pos = self.str, None
+ """
+ Return index of a good place to begin parsing, as close to the
+ end of the string as possible. This will be the start of some
+ popular stmt like "if" or "def". Return None if none found:
+ the caller should pass more prior context then, if possible, or
+ if not (the entire program text up until the point of interest
+ has already been tried) pass 0 to set_lo().
+
+ This will be reliable iff given a reliable is_char_in_string()
+ function, meaning that when it says "no", it's absolutely
+ guaranteed that the char is not in a string.
+ """
+ code, pos = self.code, None
if not is_char_in_string:
# no clue -- make the caller pass everything
# Peek back from the end for a good place to start,
# but don't try too often; pos will be left None, or
# bumped to a legitimate synch point.
- limit = len(str)
+ limit = len(code)
for tries in range(5):
- i = str.rfind(":\n", 0, limit)
+ i = code.rfind(":\n", 0, limit)
if i < 0:
break
- i = str.rfind('\n', 0, i) + 1 # start of colon line
- m = _synchre(str, i, limit)
+ i = code.rfind('\n', 0, i) + 1 # start of colon line (-1+1=0)
+ m = _synchre(code, i, limit)
if m and not is_char_in_string(m.start()):
pos = m.start()
break
# going to have to parse the whole thing to be sure, so
# give it one last try from the start, but stop wasting
# time here regardless of the outcome.
- m = _synchre(str)
+ m = _synchre(code)
if m and not is_char_in_string(m.start()):
pos = m.start()
return pos
# matches.
i = pos + 1
while 1:
- m = _synchre(str, i)
+ m = _synchre(code, i)
if m:
s, i = m.span()
if not is_char_in_string(s):
break
return pos
- # Throw away the start of the string. Intended to be called with
- # find_good_parse_start's result.
-
def set_lo(self, lo):
- assert lo == 0 or self.str[lo-1] == '\n'
+ """ Throw away the start of the string.
+
+ Intended to be called with the result of find_good_parse_start().
+ """
+ assert lo == 0 or self.code[lo-1] == '\n'
if lo > 0:
- self.str = self.str[lo:]
-
- # Build a translation table to map uninteresting chars to 'x', open
- # brackets to '(', close brackets to ')' while preserving quotes,
- # backslashes, newlines and hashes. This is to be passed to
- # str.translate() in _study1().
- _tran = {}
- _tran.update((ord(c), ord('(')) for c in "({[")
- _tran.update((ord(c), ord(')')) for c in ")}]")
- _tran.update((ord(c), ord(c)) for c in "\"'\\\n#")
- _tran = StringTranslatePseudoMapping(_tran, default_value=ord('x'))
-
- # As quickly as humanly possible <wink>, find the line numbers (0-
- # based) of the non-continuation lines.
- # Creates self.{goodlines, continuation}.
+ self.code = self.code[lo:]
def _study1(self):
+ """Find the line numbers of non-continuation lines.
+
+ As quickly as humanly possible <wink>, find the line numbers (0-
+ based) of the non-continuation lines.
+ Creates self.{goodlines, continuation}.
+ """
if self.study_level >= 1:
return
self.study_level = 1
# to "(", all close brackets to ")", then collapse runs of
# uninteresting characters. This can cut the number of chars
# by a factor of 10-40, and so greatly speed the following loop.
- str = self.str
- str = str.translate(self._tran)
- str = str.replace('xxxxxxxx', 'x')
- str = str.replace('xxxx', 'x')
- str = str.replace('xx', 'x')
- str = str.replace('xx', 'x')
- str = str.replace('\nx', '\n')
- # note that replacing x\n with \n would be incorrect, because
- # x may be preceded by a backslash
+ code = self.code
+ code = code.translate(trans)
+ code = code.replace('xxxxxxxx', 'x')
+ code = code.replace('xxxx', 'x')
+ code = code.replace('xx', 'x')
+ code = code.replace('xx', 'x')
+ code = code.replace('\nx', '\n')
+ # Replacing x\n with \n would be incorrect because
+ # x may be preceded by a backslash.
# March over the squashed version of the program, accumulating
# the line numbers of non-continued stmts, and determining
level = lno = 0 # level is nesting level; lno is line number
self.goodlines = goodlines = [0]
push_good = goodlines.append
- i, n = 0, len(str)
+ i, n = 0, len(code)
while i < n:
- ch = str[i]
+ ch = code[i]
i = i+1
# cases are checked in decreasing order of frequency
if ch == '"' or ch == "'":
# consume the string
quote = ch
- if str[i-1:i+2] == quote * 3:
+ if code[i-1:i+2] == quote * 3:
quote = quote * 3
firstlno = lno
w = len(quote) - 1
i = i+w
while i < n:
- ch = str[i]
+ ch = code[i]
i = i+1
if ch == 'x':
continue
- if str[i-1:i+w] == quote:
+ if code[i-1:i+w] == quote:
i = i+w
break
if ch == '\\':
assert i < n
- if str[i] == '\n':
+ if code[i] == '\n':
lno = lno + 1
i = i+1
continue
# didn't break out of the loop, so we're still
# inside a string
if (lno - 1) == firstlno:
- # before the previous \n in str, we were in the first
+ # before the previous \n in code, we were in the first
# line of the string
continuation = C_STRING_FIRST_LINE
else:
if ch == '#':
# consume the comment
- i = str.find('\n', i)
+ i = code.find('\n', i)
assert i >= 0
continue
assert ch == '\\'
assert i < n
- if str[i] == '\n':
+ if code[i] == '\n':
lno = lno + 1
if i+1 == n:
continuation = C_BACKSLASH
self._study1()
return self.continuation
- # study1 was sufficient to determine the continuation status,
- # but doing more requires looking at every character. study2
- # does this for the last interesting statement in the block.
- # Creates:
- # self.stmt_start, stmt_end
- # slice indices of last interesting stmt
- # self.stmt_bracketing
- # the bracketing structure of the last interesting stmt;
- # for example, for the statement "say(boo) or die", stmt_bracketing
- # will be [(0, 0), (3, 1), (8, 0)]. Strings and comments are
- # treated as brackets, for the matter.
- # self.lastch
- # last non-whitespace character before optional trailing
- # comment
- # self.lastopenbracketpos
- # if continuation is C_BRACKET, index of last open bracket
-
def _study2(self):
+ """
+ study1 was sufficient to determine the continuation status,
+ but doing more requires looking at every character. study2
+ does this for the last interesting statement in the block.
+ Creates:
+ self.stmt_start, stmt_end
+ slice indices of last interesting stmt
+ self.stmt_bracketing
+ the bracketing structure of the last interesting stmt; for
+ example, for the statement "say(boo) or die",
+ stmt_bracketing will be ((0, 0), (0, 1), (2, 0), (2, 1),
+ (4, 0)). Strings and comments are treated as brackets, for
+ the matter.
+ self.lastch
+ last interesting character before optional trailing comment
+ self.lastopenbracketpos
+ if continuation is C_BRACKET, index of last open bracket
+ """
if self.study_level >= 2:
return
self._study1()
self.study_level = 2
# Set p and q to slice indices of last interesting stmt.
- str, goodlines = self.str, self.goodlines
- i = len(goodlines) - 1
- p = len(str) # index of newest line
+ code, goodlines = self.code, self.goodlines
+ i = len(goodlines) - 1 # Index of newest line.
+ p = len(code) # End of goodlines[i]
while i:
assert p
- # p is the index of the stmt at line number goodlines[i].
+ # Make p be the index of the stmt at line number goodlines[i].
# Move p back to the stmt at line number goodlines[i-1].
q = p
for nothing in range(goodlines[i-1], goodlines[i]):
# tricky: sets p to 0 if no preceding newline
- p = str.rfind('\n', 0, p-1) + 1
- # The stmt str[p:q] isn't a continuation, but may be blank
+ p = code.rfind('\n', 0, p-1) + 1
+ # The stmt code[p:q] isn't a continuation, but may be blank
# or a non-indenting comment line.
- if _junkre(str, p):
+ if _junkre(code, p):
i = i-1
else:
break
bracketing = [(p, 0)]
while p < q:
# suck up all except ()[]{}'"#\\
- m = _chew_ordinaryre(str, p, q)
+ m = _chew_ordinaryre(code, p, q)
if m:
# we skipped at least one boring char
newp = m.end()
# back up over totally boring whitespace
i = newp - 1 # index of last boring char
- while i >= p and str[i] in " \t\n":
+ while i >= p and code[i] in " \t\n":
i = i-1
if i >= p:
- lastch = str[i]
+ lastch = code[i]
p = newp
if p >= q:
break
- ch = str[p]
+ ch = code[p]
if ch in "([{":
push_stack(p)
# have to.
bracketing.append((p, len(stack)+1))
lastch = ch
- p = _match_stringre(str, p, q).end()
+ p = _match_stringre(code, p, q).end()
bracketing.append((p, len(stack)))
continue
if ch == '#':
# consume comment and trailing newline
bracketing.append((p, len(stack)+1))
- p = str.find('\n', p, q) + 1
+ p = code.find('\n', p, q) + 1
assert p > 0
bracketing.append((p, len(stack)))
continue
assert ch == '\\'
p = p+1 # beyond backslash
assert p < q
- if str[p] != '\n':
+ if code[p] != '\n':
# the program is invalid, but can't complain
- lastch = ch + str[p]
+ lastch = ch + code[p]
p = p+1 # beyond escaped char
# end while p < q:
self.lastch = lastch
- if stack:
- self.lastopenbracketpos = stack[-1]
+ self.lastopenbracketpos = stack[-1] if stack else None
self.stmt_bracketing = tuple(bracketing)
- # Assuming continuation is C_BRACKET, return the number
- # of spaces the next line should be indented.
-
def compute_bracket_indent(self):
+ """Return number of spaces the next line should be indented.
+
+ Line continuation must be C_BRACKET.
+ """
self._study2()
assert self.continuation == C_BRACKET
j = self.lastopenbracketpos
- str = self.str
- n = len(str)
- origi = i = str.rfind('\n', 0, j) + 1
+ code = self.code
+ n = len(code)
+ origi = i = code.rfind('\n', 0, j) + 1
j = j+1 # one beyond open bracket
# find first list item; set i to start of its line
while j < n:
- m = _itemre(str, j)
+ m = _itemre(code, j)
if m:
j = m.end() - 1 # index of first interesting char
extra = 0
break
else:
# this line is junk; advance to next line
- i = j = str.find('\n', j) + 1
+ i = j = code.find('\n', j) + 1
else:
# nothing interesting follows the bracket;
# reproduce the bracket line's indentation + a level
j = i = origi
- while str[j] in " \t":
+ while code[j] in " \t":
j = j+1
extra = self.indentwidth
- return len(str[i:j].expandtabs(self.tabwidth)) + extra
-
- # Return number of physical lines in last stmt (whether or not
- # it's an interesting stmt! this is intended to be called when
- # continuation is C_BACKSLASH).
+ return len(code[i:j].expandtabs(self.tabwidth)) + extra
def get_num_lines_in_stmt(self):
+ """Return number of physical lines in last stmt.
+
+ The statement doesn't have to be an interesting statement. This is
+ intended to be called when continuation is C_BACKSLASH.
+ """
self._study1()
goodlines = self.goodlines
return goodlines[-1] - goodlines[-2]
- # Assuming continuation is C_BACKSLASH, return the number of spaces
- # the next line should be indented. Also assuming the new line is
- # the first one following the initial line of the stmt.
-
def compute_backslash_indent(self):
+ """Return number of spaces the next line should be indented.
+
+ Line continuation must be C_BACKSLASH. Also assume that the new
+ line is the first one following the initial line of the stmt.
+ """
self._study2()
assert self.continuation == C_BACKSLASH
- str = self.str
+ code = self.code
i = self.stmt_start
- while str[i] in " \t":
+ while code[i] in " \t":
i = i+1
startpos = i
# See whether the initial line starts an assignment stmt; i.e.,
# look for an = operator
- endpos = str.find('\n', startpos) + 1
+ endpos = code.find('\n', startpos) + 1
found = level = 0
while i < endpos:
- ch = str[i]
+ ch = code[i]
if ch in "([{":
level = level + 1
i = i+1
level = level - 1
i = i+1
elif ch == '"' or ch == "'":
- i = _match_stringre(str, i, endpos).end()
+ i = _match_stringre(code, i, endpos).end()
elif ch == '#':
+ # This line is unreachable because the # makes a comment of
+ # everything after it.
break
elif level == 0 and ch == '=' and \
- (i == 0 or str[i-1] not in "=<>!") and \
- str[i+1] != '=':
+ (i == 0 or code[i-1] not in "=<>!") and \
+ code[i+1] != '=':
found = 1
break
else:
# found a legit =, but it may be the last interesting
# thing on the line
i = i+1 # move beyond the =
- found = re.match(r"\s*\\", str[i:endpos]) is None
+ found = re.match(r"\s*\\", code[i:endpos]) is None
if not found:
# oh well ... settle for moving beyond the first chunk
# of non-whitespace chars
i = startpos
- while str[i] not in " \t\n":
+ while code[i] not in " \t\n":
i = i+1
- return len(str[self.stmt_start:i].expandtabs(\
+ return len(code[self.stmt_start:i].expandtabs(\
self.tabwidth)) + 1
- # Return the leading whitespace on the initial line of the last
- # interesting stmt.
-
def get_base_indent_string(self):
+ """Return the leading whitespace on the initial line of the last
+ interesting stmt.
+ """
self._study2()
i, n = self.stmt_start, self.stmt_end
j = i
- str = self.str
- while j < n and str[j] in " \t":
+ code = self.code
+ while j < n and code[j] in " \t":
j = j + 1
- return str[i:j]
-
- # Did the last interesting stmt open a block?
+ return code[i:j]
def is_block_opener(self):
+ "Return True if the last interesting statemtent opens a block."
self._study2()
return self.lastch == ':'
- # Did the last interesting stmt close a block?
-
def is_block_closer(self):
+ "Return True if the last interesting statement closes a block."
self._study2()
- return _closere(self.str, self.stmt_start) is not None
+ return _closere(self.code, self.stmt_start) is not None
- # index of last open bracket ({[, or None if none
- lastopenbracketpos = None
+ def get_last_stmt_bracketing(self):
+ """Return bracketing structure of the last interesting statement.
- def get_last_open_bracket_pos(self):
+ The returned tuple is in the format defined in _study2().
+ """
self._study2()
- return self.lastopenbracketpos
+ return self.stmt_bracketing
- # the structure of the bracketing of the last interesting statement,
- # in the format defined in _study2, or None if the text didn't contain
- # anything
- stmt_bracketing = None
- def get_last_stmt_bracketing(self):
- self._study2()
- return self.stmt_bracketing
+if __name__ == '__main__': #pragma: nocover
+ import unittest
+ unittest.main('idlelib.idle_test.test_pyparse', verbosity=2)
if source is None:
with tokenize.open(filename) as fp:
source = fp.read()
+ if use_subprocess:
+ source = (f"__file__ = r'''{os.path.abspath(filename)}'''\n"
+ + source + "\ndel __file__")
try:
code = compile(source, filename, "exec")
except (OverflowError, SyntaxError):
return ViewWindow(parent, title, text, modal, _utest=_utest)
-def view_file(parent, title, filename, encoding=None, modal=True, _utest=False):
+def view_file(parent, title, filename, encoding, modal=True, _utest=False):
"""Create text viewer for text in filename.
Return error message if file cannot be read. Otherwise calls view_text
return sig
else:
sig_params = tuple(sig.parameters.values())
- assert first_wrapped_param is not sig_params[0]
+ assert (not sig_params or
+ first_wrapped_param is not sig_params[0])
new_params = (first_wrapped_param,) + sig_params
return sig.replace(parameters=new_params)
# Grammar for 2to3. This grammar supports Python 2.x and 3.x.
-# Note: Changing the grammar specified in this file will most likely
-# require corresponding changes in the parser module
-# (../Modules/parsermodule.c). If you can't make the changes to
-# that module yourself, please co-ordinate the required changes
-# with someone who can; ask around on python-dev for help. Fred
-# Drake <fdrake@acm.org> will probably be listening there.
-
-# NOTE WELL: You should also follow all the steps listed in PEP 306,
-# "How to Change Python's Grammar"
-
-# Commands for Kees Blom's railroad program
-#diagram:token NAME
-#diagram:token NUMBER
-#diagram:token STRING
-#diagram:token NEWLINE
-#diagram:token ENDMARKER
-#diagram:token INDENT
-#diagram:output\input python.bla
-#diagram:token DEDENT
-#diagram:output\textwidth 20.04cm\oddsidemargin 0.0cm\evensidemargin 0.0cm
-#diagram:rules
+# NOTE WELL: You should also follow all the steps listed at
+# https://devguide.python.org/grammar/
# Start symbols for the grammar:
# file_input is a module or sequence of commands read from an input file;
funcdef: 'def' NAME parameters ['->' test] ':' suite
parameters: '(' [typedargslist] ')'
typedargslist: ((tfpdef ['=' test] ',')*
- ('*' [tname] (',' tname ['=' test])* [',' '**' tname] | '**' tname)
+ ('*' [tname] (',' tname ['=' test])* [',' ['**' tname [',']]] | '**' tname [','])
| tfpdef ['=' test] (',' tfpdef ['=' test])* [','])
tname: NAME [':' test]
tfpdef: tname | '(' tfplist ')'
tfplist: tfpdef (',' tfpdef)* [',']
varargslist: ((vfpdef ['=' test] ',')*
- ('*' [vname] (',' vname ['=' test])* [',' '**' vname] | '**' vname)
+ ('*' [vname] (',' vname ['=' test])* [',' ['**' vname [',']]] | '**' vname [','])
| vfpdef ['=' test] (',' vfpdef ['=' test])* [','])
vname: NAME
vfpdef: vname | '(' vfplist ')'
# Python imports
import io
-import os
# Fairly local imports
from .pgen2 import driver, literals, token, tokenize, parse, grammar
from . import pytree
from . import pygram
-# The pattern grammar file
-_PATTERN_GRAMMAR_FILE = os.path.join(os.path.dirname(__file__),
- "PatternGrammar.txt")
-
class PatternSyntaxError(Exception):
pass
class PatternCompiler(object):
- def __init__(self, grammar_file=_PATTERN_GRAMMAR_FILE):
+ def __init__(self, grammar_file=None):
"""Initializer.
Takes an optional alternative filename for the pattern grammar.
"""
- self.grammar = driver.load_grammar(grammar_file)
- self.syms = pygram.Symbols(self.grammar)
+ if grammar_file is None:
+ self.grammar = pygram.pattern_grammar
+ self.syms = pygram.pattern_symbols
+ else:
+ self.grammar = driver.load_grammar(grammar_file)
+ self.syms = pygram.Symbols(self.grammar)
self.pygrammar = pygram.python_grammar
self.pysyms = pygram.python_symbols
self.driver = driver.Driver(self.grammar, convert=pattern_convert)
import io
import os
import logging
+import pkgutil
import sys
# Pgen imports
return os.path.getmtime(a) >= os.path.getmtime(b)
+def load_packaged_grammar(package, grammar_source):
+ """Normally, loads a pickled grammar by doing
+ pkgutil.get_data(package, pickled_grammar)
+ where *pickled_grammar* is computed from *grammar_source* by adding the
+ Python version and using a ``.pickle`` extension.
+
+ However, if *grammar_source* is an extant file, load_grammar(grammar_source)
+ is called instead. This facilitates using a packaged grammar file when needed
+ but preserves load_grammar's automatic regeneration behavior when possible.
+
+ """
+ if os.path.isfile(grammar_source):
+ return load_grammar(grammar_source)
+ pickled_name = _generate_pickle_name(os.path.basename(grammar_source))
+ data = pkgutil.get_data(package, pickled_name)
+ g = grammar.Grammar()
+ g.loads(data)
+ return g
+
+
def main(*args):
"""Main program, when run as a script: produce grammar pickle files.
d = pickle.load(f)
self.__dict__.update(d)
+ def loads(self, pkl):
+ """Load the grammar tables from a pickle bytes object."""
+ self.__dict__.update(pickle.loads(pkl))
+
def copy(self):
"""
Copy the grammar.
setattr(self, name, symbol)
-python_grammar = driver.load_grammar(_GRAMMAR_FILE)
+python_grammar = driver.load_packaged_grammar("lib2to3", _GRAMMAR_FILE)
python_symbols = Symbols(python_grammar)
python_grammar_no_print_statement = python_grammar.copy()
del python_grammar_no_print_statement.keywords["print"]
-pattern_grammar = driver.load_grammar(_PATTERN_GRAMMAR_FILE)
+pattern_grammar = driver.load_packaged_grammar("lib2to3", _PATTERN_GRAMMAR_FILE)
pattern_symbols = Symbols(pattern_grammar)
proj_dir = os.path.normpath(os.path.join(test_dir, ".."))
grammar_path = os.path.join(test_dir, "..", "Grammar.txt")
grammar = pgen2_driver.load_grammar(grammar_path)
+grammar_no_print_statement = pgen2_driver.load_grammar(grammar_path)
+del grammar_no_print_statement.keywords["print"]
driver = pgen2_driver.Driver(grammar, convert=pytree.convert)
+driver_no_print_statement = pgen2_driver.Driver(
+ grammar_no_print_statement,
+ convert=pytree.convert
+)
def parse_string(string):
return driver.parse_string(reformat(string), debug=True)
# Testing imports
from . import support
-from .support import driver
-from test.support import verbose
+from .support import driver, driver_no_print_statement
# Python imports
+import difflib
+import importlib
+import operator
import os
+import pickle
import shutil
import subprocess
import sys
import tempfile
import unittest
-import warnings
# Local imports
from lib2to3.pgen2 import driver as pgen2_driver
finally:
shutil.rmtree(tmpdir)
+ def test_load_packaged_grammar(self):
+ modname = __name__ + '.load_test'
+ class MyLoader:
+ def get_data(self, where):
+ return pickle.dumps({'elephant': 19})
+ class MyModule:
+ __file__ = 'parsertestmodule'
+ __spec__ = importlib.util.spec_from_loader(modname, MyLoader())
+ sys.modules[modname] = MyModule()
+ self.addCleanup(operator.delitem, sys.modules, modname)
+ g = pgen2_driver.load_packaged_grammar(modname, 'Grammar.txt')
+ self.assertEqual(g.elephant, 19)
class GrammarTest(support.TestCase):
def test_argument_unpacking_3(self):
self.validate("""f(2, *a, *b, **b, **c, **d)""")
+ def test_trailing_commas_1(self):
+ self.validate("def f(a, b): call(a, b)")
+ self.validate("def f(a, b,): call(a, b,)")
+
+ def test_trailing_commas_2(self):
+ self.validate("def f(a, *b): call(a, *b)")
+ self.validate("def f(a, *b,): call(a, *b,)")
+
+ def test_trailing_commas_3(self):
+ self.validate("def f(a, b=1): call(a, b=1)")
+ self.validate("def f(a, b=1,): call(a, b=1,)")
+
+ def test_trailing_commas_4(self):
+ self.validate("def f(a, **b): call(a, **b)")
+ self.validate("def f(a, **b,): call(a, **b,)")
+
+ def test_trailing_commas_5(self):
+ self.validate("def f(*a, b=1): call(*a, b=1)")
+ self.validate("def f(*a, b=1,): call(*a, b=1,)")
+
+ def test_trailing_commas_6(self):
+ self.validate("def f(*a, **b): call(*a, **b)")
+ self.validate("def f(*a, **b,): call(*a, **b,)")
+
+ def test_trailing_commas_7(self):
+ self.validate("def f(*, b=1): call(*b)")
+ self.validate("def f(*, b=1,): call(*b,)")
+
+ def test_trailing_commas_8(self):
+ self.validate("def f(a=1, b=2): call(a=1, b=2)")
+ self.validate("def f(a=1, b=2,): call(a=1, b=2,)")
+
+ def test_trailing_commas_9(self):
+ self.validate("def f(a=1, **b): call(a=1, **b)")
+ self.validate("def f(a=1, **b,): call(a=1, **b,)")
+
+ def test_trailing_commas_lambda_1(self):
+ self.validate("f = lambda a, b: call(a, b)")
+ self.validate("f = lambda a, b,: call(a, b,)")
+
+ def test_trailing_commas_lambda_2(self):
+ self.validate("f = lambda a, *b: call(a, *b)")
+ self.validate("f = lambda a, *b,: call(a, *b,)")
+
+ def test_trailing_commas_lambda_3(self):
+ self.validate("f = lambda a, b=1: call(a, b=1)")
+ self.validate("f = lambda a, b=1,: call(a, b=1,)")
+
+ def test_trailing_commas_lambda_4(self):
+ self.validate("f = lambda a, **b: call(a, **b)")
+ self.validate("f = lambda a, **b,: call(a, **b,)")
+
+ def test_trailing_commas_lambda_5(self):
+ self.validate("f = lambda *a, b=1: call(*a, b=1)")
+ self.validate("f = lambda *a, b=1,: call(*a, b=1,)")
+
+ def test_trailing_commas_lambda_6(self):
+ self.validate("f = lambda *a, **b: call(*a, **b)")
+ self.validate("f = lambda *a, **b,: call(*a, **b,)")
+
+ def test_trailing_commas_lambda_7(self):
+ self.validate("f = lambda *, b=1: call(*b)")
+ self.validate("f = lambda *, b=1,: call(*b,)")
+
+ def test_trailing_commas_lambda_8(self):
+ self.validate("f = lambda a=1, b=2: call(a=1, b=2)")
+ self.validate("f = lambda a=1, b=2,: call(a=1, b=2,)")
+
+ def test_trailing_commas_lambda_9(self):
+ self.validate("f = lambda a=1, **b: call(a=1, **b)")
+ self.validate("f = lambda a=1, **b,: call(a=1, **b,)")
+
# Adapted from Python 3's Lib/test/test_grammar.py:GrammarTests.testFuncdef
class TestFunctionAnnotations(GrammarTest):
*g:6, h:7, i=8, j:9=10, **k:11) -> 12: pass"""
self.validate(s)
+ def test_9(self):
+ s = """def f(
+ a: str,
+ b: int,
+ *,
+ c: bool = False,
+ **kwargs,
+ ) -> None:
+ call(c=c, **kwargs,)"""
+ self.validate(s)
+
+ def test_10(self):
+ s = """def f(
+ a: str,
+ ) -> None:
+ call(a,)"""
+ self.validate(s)
+
+ def test_11(self):
+ s = """def f(
+ a: str = '',
+ ) -> None:
+ call(a=a,)"""
+ self.validate(s)
+
+ def test_12(self):
+ s = """def f(
+ *args: str,
+ ) -> None:
+ call(*args,)"""
+ self.validate(s)
+
+ def test_13(self):
+ self.validate("def f(a: str, b: int) -> None: call(a, b)")
+ self.validate("def f(a: str, b: int,) -> None: call(a, b,)")
+
+ def test_14(self):
+ self.validate("def f(a: str, *b: int) -> None: call(a, *b)")
+ self.validate("def f(a: str, *b: int,) -> None: call(a, *b,)")
+
+ def test_15(self):
+ self.validate("def f(a: str, b: int=1) -> None: call(a, b=1)")
+ self.validate("def f(a: str, b: int=1,) -> None: call(a, b=1,)")
+
+ def test_16(self):
+ self.validate("def f(a: str, **b: int) -> None: call(a, **b)")
+ self.validate("def f(a: str, **b: int,) -> None: call(a, **b,)")
+
+ def test_17(self):
+ self.validate("def f(*a: str, b: int=1) -> None: call(*a, b=1)")
+ self.validate("def f(*a: str, b: int=1,) -> None: call(*a, b=1,)")
+
+ def test_18(self):
+ self.validate("def f(*a: str, **b: int) -> None: call(*a, **b)")
+ self.validate("def f(*a: str, **b: int,) -> None: call(*a, **b,)")
+
+ def test_19(self):
+ self.validate("def f(*, b: int=1) -> None: call(*b)")
+ self.validate("def f(*, b: int=1,) -> None: call(*b,)")
+
+ def test_20(self):
+ self.validate("def f(a: str='', b: int=2) -> None: call(a=a, b=2)")
+ self.validate("def f(a: str='', b: int=2,) -> None: call(a=a, b=2,)")
+
+ def test_21(self):
+ self.validate("def f(a: str='', **b: int) -> None: call(a=a, **b)")
+ self.validate("def f(a: str='', **b: int,) -> None: call(a=a, **b,)")
+
# Adapted from Python 3's Lib/test/test_grammar.py:GrammarTests.test_var_annot
class TestVarAnnotations(GrammarTest):
self.validate("class B(t, *args): pass")
self.validate("class B(t, **kwargs): pass")
self.validate("class B(t, *args, **kwargs): pass")
- self.validate("class B(t, y=9, *args, **kwargs): pass")
+ self.validate("class B(t, y=9, *args, **kwargs,): pass")
class TestParserIdempotency(support.TestCase):
"""A cut-down version of pytree_idempotency.py."""
- # Issue 13125
- @unittest.expectedFailure
def test_all_project_files(self):
for filepath in support.all_project_files():
with open(filepath, "rb") as fp:
source = fp.read()
try:
tree = driver.parse_string(source)
- except ParseError as err:
- if verbose > 0:
- warnings.warn('ParseError on file %s (%s)' % (filepath, err))
- continue
+ except ParseError:
+ try:
+ tree = driver_no_print_statement.parse_string(source)
+ except ParseError as err:
+ self.fail('ParseError on file %s (%s)' % (filepath, err))
new = str(tree)
- x = diff(filepath, new)
- if x:
+ if new != source:
+ print(diff_texts(source, new, filepath))
self.fail("Idempotency failed: %s" % filepath)
def test_extended_unpacking(self):
self.validate(s)
-def diff(fn, result):
- try:
- with open('@', 'w') as f:
- f.write(str(result))
- fn = fn.replace('"', '\\"')
- return subprocess.call(['diff', '-u', fn, '@'], stdout=(subprocess.DEVNULL if verbose < 1 else None))
- finally:
- try:
- os.remove("@")
- except OSError:
- pass
+def diff_texts(a, b, filename):
+ a = a.splitlines()
+ b = b.splitlines()
+ return difflib.unified_diff(a, b, filename, filename,
+ "(original)", "(reserialized)",
+ lineterm="")
"""Pathname and path-related operations for the Macintosh."""
+# strings representing various path-related bits and pieces
+# These are primarily for export; internally, they are hardcoded.
+# Should be set before imports for resolving cyclic dependency.
+curdir = ':'
+pardir = '::'
+extsep = '.'
+sep = ':'
+pathsep = '\n'
+defpath = ':'
+altsep = None
+devnull = 'Dev:Null'
+
import os
from stat import *
import genericpath
"curdir","pardir","sep","pathsep","defpath","altsep","extsep",
"devnull","realpath","supports_unicode_filenames"]
-# strings representing various path-related bits and pieces
-# These are primarily for export; internally, they are hardcoded.
-curdir = ':'
-pardir = '::'
-extsep = '.'
-sep = ':'
-pathsep = '\n'
-defpath = ':'
-altsep = None
-devnull = 'Dev:Null'
-
def _get_colon(path):
if isinstance(path, bytes):
return b':'
method = 'fork'
def __init__(self, process_obj):
- try:
- sys.stdout.flush()
- except (AttributeError, ValueError):
- pass
- try:
- sys.stderr.flush()
- except (AttributeError, ValueError):
- pass
+ util._flush_std_streams()
self.returncode = None
self._launch(process_obj)
traceback.print_exc()
finally:
util.info('process exiting with exitcode %d' % exitcode)
- sys.stdout.flush()
- sys.stderr.flush()
+ util._flush_std_streams()
return exitcode
except (OSError, ValueError):
pass
+#
+# Flush standard streams, if any
+#
+
+def _flush_std_streams():
+ try:
+ sys.stdout.flush()
+ except (AttributeError, ValueError):
+ pass
+ try:
+ sys.stderr.flush()
+ except (AttributeError, ValueError):
+ pass
+
#
# Start a program with only specified fds kept open
#
module as os.path.
"""
+# strings representing various path-related bits and pieces
+# These are primarily for export; internally, they are hardcoded.
+# Should be set before imports for resolving cyclic dependency.
+curdir = '.'
+pardir = '..'
+extsep = '.'
+sep = '\\'
+pathsep = ';'
+altsep = '/'
+defpath = '.;C:\\bin'
+devnull = 'nul'
+
import os
import sys
import stat
"extsep","devnull","realpath","supports_unicode_filenames","relpath",
"samefile", "sameopenfile", "samestat", "commonpath"]
-# strings representing various path-related bits and pieces
-# These are primarily for export; internally, they are hardcoded.
-curdir = '.'
-pardir = '..'
-extsep = '.'
-sep = '\\'
-pathsep = ';'
-altsep = '/'
-defpath = '.;C:\\bin'
-devnull = 'nul'
-
def _get_bothseps(path):
if isinstance(path, bytes):
return b'\\/'
def_op('BUILD_TUPLE', 102) # Number of tuple items
def_op('BUILD_LIST', 103) # Number of list items
def_op('BUILD_SET', 104) # Number of set items
-def_op('BUILD_MAP', 105) # Number of dict entries (upto 255)
+def_op('BUILD_MAP', 105) # Number of dict entries
name_op('LOAD_ATTR', 106) # Index in name list
def_op('COMPARE_OP', 107) # Comparison operator
hascompare.append(107)
otherwise return -SIG, where SIG is the signal that killed it. """
return _spawnvef(mode, file, args, env, execve)
- # Note: spawnvp[e] is't currently supported on Windows
+ # Note: spawnvp[e] isn't currently supported on Windows
def spawnvp(mode, file, args):
"""spawnvp(mode, file, args) -> integer
class PurePath(object):
- """PurePath represents a filesystem path and offers operations which
+ """Base class for manipulating paths without I/O.
+
+ PurePath represents a filesystem path and offers operations which
don't imply any actual filesystem I/O. Depending on your system,
instantiating a PurePath will return either a PurePosixPath or a
PureWindowsPath object. You can also instantiate either of these classes
class PurePosixPath(PurePath):
+ """PurePath subclass for non-Windows systems.
+
+ On a POSIX system, instantiating a PurePath should return this object.
+ However, you can also instantiate it directly on any system.
+ """
_flavour = _posix_flavour
__slots__ = ()
class PureWindowsPath(PurePath):
+ """PurePath subclass for Windows systems.
+
+ On a Windows system, instantiating a PurePath should return this object.
+ However, you can also instantiate it directly on any system.
+ """
_flavour = _windows_flavour
__slots__ = ()
class Path(PurePath):
+ """PurePath subclass that can make system calls.
+
+ Path represents a filesystem path but unlike PurePath, also offers
+ methods to do system calls on path objects. Depending on your system,
+ instantiating a Path will return either a PosixPath or a WindowsPath
+ object. You can also instantiate a PosixPath or WindowsPath directly,
+ but cannot instantiate a WindowsPath on a POSIX system or vice versa.
+ """
__slots__ = (
'_accessor',
'_closed',
class PosixPath(Path, PurePosixPath):
+ """Path subclass for non-Windows systems.
+
+ On a POSIX system, instantiating a Path should return this object.
+ """
__slots__ = ()
class WindowsPath(Path, PureWindowsPath):
+ """Path subclass for Windows systems.
+
+ On a Windows system, instantiating a Path should return this object.
+ """
__slots__ = ()
def owner(self):
return self._shortcmd('RPOP %s' % user)
- timestamp = re.compile(br'\+OK.*(<[^>]+>)')
+ timestamp = re.compile(br'\+OK.[^<]*(<.*>)')
def apop(self, user, password):
"""Authorisation
for manipulation of the pathname component of URLs.
"""
+# Strings representing various path-related bits and pieces.
+# These are primarily for export; internally, they are hardcoded.
+# Should be set before imports for resolving cyclic dependency.
+curdir = '.'
+pardir = '..'
+extsep = '.'
+sep = '/'
+pathsep = ':'
+defpath = ':/bin:/usr/bin'
+altsep = None
+devnull = '/dev/null'
+
import os
import sys
import stat
"devnull","realpath","supports_unicode_filenames","relpath",
"commonpath"]
-# Strings representing various path-related bits and pieces.
-# These are primarily for export; internally, they are hardcoded.
-curdir = '.'
-pardir = '..'
-extsep = '.'
-sep = '/'
-pathsep = ':'
-defpath = ':/bin:/usr/bin'
-altsep = None
-devnull = '/dev/null'
def _get_sep(path):
if isinstance(path, bytes):
Welcome to Python {0}'s help utility!
If this is your first time using Python, you should definitely check out
-the tutorial on the Internet at http://docs.python.org/{0}/tutorial/.
+the tutorial on the Internet at https://docs.python.org/{0}/tutorial/.
Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules. To quit this help utility and
# -*- coding: utf-8 -*-
-# Autogenerated by Sphinx on Tue Dec 5 03:11:02 2017
+# Autogenerated by Sphinx on Tue Mar 13 21:13:16 2018
topics = {'assert': 'The "assert" statement\n'
'**********************\n'
'\n'
'\n'
'object.__getattr__(self, name)\n'
'\n'
- ' Called when an attribute lookup has not found the '
- 'attribute in the\n'
- ' usual places (i.e. it is not an instance attribute '
- 'nor is it found\n'
- ' in the class tree for "self"). "name" is the '
- 'attribute name. This\n'
- ' method should return the (computed) attribute value '
- 'or raise an\n'
- ' "AttributeError" exception.\n'
+ ' Called when the default attribute access fails with '
+ 'an\n'
+ ' "AttributeError" (either "__getattribute__()" raises '
+ 'an\n'
+ ' "AttributeError" because *name* is not an instance '
+ 'attribute or an\n'
+ ' attribute in the class tree for "self"; or '
+ '"__get__()" of a *name*\n'
+ ' property raises "AttributeError"). This method '
+ 'should either\n'
+ ' return the (computed) attribute value or raise an '
+ '"AttributeError"\n'
+ ' exception.\n'
'\n'
' Note that if the attribute is found through the '
'normal mechanism,\n'
' sorts it.\n'
'\n'
'\n'
+ 'Customizing module attribute access\n'
+ '===================================\n'
+ '\n'
+ 'For a more fine grained customization of the module '
+ 'behavior (setting\n'
+ 'attributes, properties, etc.), one can set the '
+ '"__class__" attribute\n'
+ 'of a module object to a subclass of "types.ModuleType". '
+ 'For example:\n'
+ '\n'
+ ' import sys\n'
+ ' from types import ModuleType\n'
+ '\n'
+ ' class VerboseModule(ModuleType):\n'
+ ' def __repr__(self):\n'
+ " return f'Verbose {self.__name__}'\n"
+ '\n'
+ ' def __setattr__(self, attr, value):\n'
+ " print(f'Setting {attr}...')\n"
+ ' setattr(self, attr, value)\n'
+ '\n'
+ ' sys.modules[__name__].__class__ = VerboseModule\n'
+ '\n'
+ 'Note: Setting module "__class__" only affects lookups '
+ 'made using the\n'
+ ' attribute access syntax -- directly accessing the '
+ 'module globals\n'
+ ' (whether by code within the module, or via a reference '
+ 'to the\n'
+ " module's globals dictionary) is unaffected.\n"
+ '\n'
+ 'Changed in version 3.5: "__class__" module attribute is '
+ 'now writable.\n'
+ '\n'
+ '\n'
'Implementing Descriptors\n'
'========================\n'
'\n'
'__slots__\n'
'=========\n'
'\n'
- 'By default, instances of classes have a dictionary for '
- 'attribute\n'
- 'storage. This wastes space for objects having very few '
- 'instance\n'
- 'variables. The space consumption can become acute when '
- 'creating large\n'
- 'numbers of instances.\n'
+ '*__slots__* allow us to explicitly declare data members '
+ '(like\n'
+ 'properties) and deny the creation of *__dict__* and '
+ '*__weakref__*\n'
+ '(unless explicitly declared in *__slots__* or available '
+ 'in a parent.)\n'
'\n'
- 'The default can be overridden by defining *__slots__* in '
- 'a class\n'
- 'definition. The *__slots__* declaration takes a sequence '
- 'of instance\n'
- 'variables and reserves just enough space in each '
- 'instance to hold a\n'
- 'value for each variable. Space is saved because '
- '*__dict__* is not\n'
- 'created for each instance.\n'
+ 'The space saved over using *__dict__* can be '
+ 'significant.\n'
'\n'
'object.__slots__\n'
'\n'
'\n'
'* When inheriting from a class without *__slots__*, the '
'*__dict__*\n'
- ' attribute of that class will always be accessible, so '
- 'a *__slots__*\n'
- ' definition in the subclass is meaningless.\n'
+ ' and *__weakref__* attribute of the instances will '
+ 'always be\n'
+ ' accessible.\n'
'\n'
'* Without a *__dict__* variable, instances cannot be '
'assigned new\n'
'the class\n'
' attribute would overwrite the descriptor assignment.\n'
'\n'
- '* The action of a *__slots__* declaration is limited to '
- 'the class\n'
- ' where it is defined. As a result, subclasses will '
- 'have a *__dict__*\n'
- ' unless they also define *__slots__* (which must only '
- 'contain names\n'
- ' of any *additional* slots).\n'
+ '* The action of a *__slots__* declaration is not limited '
+ 'to the\n'
+ ' class where it is defined. *__slots__* declared in '
+ 'parents are\n'
+ ' available in child classes. However, child subclasses '
+ 'will get a\n'
+ ' *__dict__* and *__weakref__* unless they also define '
+ '*__slots__*\n'
+ ' (which should only contain names of any *additional* '
+ 'slots).\n'
'\n'
'* If a class defines a slot also defined in a base '
'class, the\n'
'\n'
'* *__class__* assignment works only if both classes have '
'the same\n'
- ' *__slots__*.\n',
+ ' *__slots__*.\n'
+ '\n'
+ '* Multiple inheritance with multiple slotted parent '
+ 'classes can be\n'
+ ' used, but only one parent is allowed to have '
+ 'attributes created by\n'
+ ' slots (the other bases must have empty slot layouts) - '
+ 'violations\n'
+ ' raise "TypeError".\n',
'attribute-references': 'Attribute references\n'
'********************\n'
'\n'
'\n'
' Called when the instance is about to be destroyed. This '
'is also\n'
- ' called a destructor. If a base class has a "__del__()" '
- 'method, the\n'
- ' derived class\'s "__del__()" method, if any, must '
- 'explicitly call it\n'
- ' to ensure proper deletion of the base class part of the '
- 'instance.\n'
- ' Note that it is possible (though not recommended!) for '
+ ' called a finalizer or (improperly) a destructor. If a '
+ 'base class\n'
+ ' has a "__del__()" method, the derived class\'s '
+ '"__del__()" method,\n'
+ ' if any, must explicitly call it to ensure proper '
+ 'deletion of the\n'
+ ' base class part of the instance.\n'
+ '\n'
+ ' It is possible (though not recommended!) for the '
+ '"__del__()" method\n'
+ ' to postpone destruction of the instance by creating a '
+ 'new reference\n'
+ ' to it. This is called object *resurrection*. It is\n'
+ ' implementation-dependent whether "__del__()" is called a '
+ 'second\n'
+ ' time when a resurrected object is about to be destroyed; '
'the\n'
- ' "__del__()" method to postpone destruction of the '
- 'instance by\n'
- ' creating a new reference to it. It may then be called '
- 'at a later\n'
- ' time when this new reference is deleted. It is not '
- 'guaranteed that\n'
- ' "__del__()" methods are called for objects that still '
- 'exist when\n'
- ' the interpreter exits.\n'
+ ' current *CPython* implementation only calls it once.\n'
+ '\n'
+ ' It is not guaranteed that "__del__()" methods are called '
+ 'for\n'
+ ' objects that still exist when the interpreter exits.\n'
'\n'
' Note: "del x" doesn\'t directly call "x.__del__()" --- '
'the former\n'
' decrements the reference count for "x" by one, and the '
'latter is\n'
- ' only called when "x"\'s reference count reaches zero. '
- 'Some common\n'
- ' situations that may prevent the reference count of an '
- 'object from\n'
- ' going to zero include: circular references between '
- 'objects (e.g.,\n'
- ' a doubly-linked list or a tree data structure with '
- 'parent and\n'
- ' child pointers); a reference to the object on the '
- 'stack frame of\n'
- ' a function that caught an exception (the traceback '
- 'stored in\n'
- ' "sys.exc_info()[2]" keeps the stack frame alive); or a '
+ ' only called when "x"\'s reference count reaches zero.\n'
+ '\n'
+ ' **CPython implementation detail:** It is possible for a '
'reference\n'
- ' to the object on the stack frame that raised an '
- 'unhandled\n'
- ' exception in interactive mode (the traceback stored '
- 'in\n'
- ' "sys.last_traceback" keeps the stack frame alive). '
- 'The first\n'
- ' situation can only be remedied by explicitly breaking '
- 'the cycles;\n'
- ' the second can be resolved by freeing the reference to '
- 'the\n'
- ' traceback object when it is no longer useful, and the '
- 'third can\n'
- ' be resolved by storing "None" in "sys.last_traceback". '
- 'Circular\n'
- ' references which are garbage are detected and cleaned '
- 'up when the\n'
- " cyclic garbage collector is enabled (it's on by "
- 'default). Refer\n'
- ' to the documentation for the "gc" module for more '
- 'information\n'
- ' about this topic.\n'
+ ' cycle to prevent the reference count of an object from '
+ 'going to\n'
+ ' zero. In this case, the cycle will be later detected '
+ 'and deleted\n'
+ ' by the *cyclic garbage collector*. A common cause of '
+ 'reference\n'
+ ' cycles is when an exception has been caught in a local '
+ 'variable.\n'
+ " The frame's locals then reference the exception, which "
+ 'references\n'
+ ' its own traceback, which references the locals of all '
+ 'frames caught\n'
+ ' in the traceback.\n'
+ '\n'
+ ' See also: Documentation for the "gc" module.\n'
'\n'
' Warning: Due to the precarious circumstances under '
'which\n'
'during\n'
' their execution are ignored, and a warning is printed '
'to\n'
- ' "sys.stderr" instead. Also, when "__del__()" is '
- 'invoked in\n'
- ' response to a module being deleted (e.g., when '
- 'execution of the\n'
- ' program is done), other globals referenced by the '
+ ' "sys.stderr" instead. In particular:\n'
+ '\n'
+ ' * "__del__()" can be invoked when arbitrary code is '
+ 'being\n'
+ ' executed, including from any arbitrary thread. If '
'"__del__()"\n'
- ' method may already have been deleted or in the process '
- 'of being\n'
- ' torn down (e.g. the import machinery shutting down). '
- 'For this\n'
- ' reason, "__del__()" methods should do the absolute '
- 'minimum needed\n'
- ' to maintain external invariants. Starting with '
- 'version 1.5,\n'
- ' Python guarantees that globals whose name begins with '
- 'a single\n'
- ' underscore are deleted from their module before other '
- 'globals are\n'
- ' deleted; if no other references to such globals exist, '
- 'this may\n'
- ' help in assuring that imported modules are still '
- 'available at the\n'
- ' time when the "__del__()" method is called.\n'
+ ' needs to take a lock or invoke any other blocking '
+ 'resource, it\n'
+ ' may deadlock as the resource may already be taken by '
+ 'the code\n'
+ ' that gets interrupted to execute "__del__()".\n'
+ '\n'
+ ' * "__del__()" can be executed during interpreter '
+ 'shutdown. As\n'
+ ' a consequence, the global variables it needs to '
+ 'access\n'
+ ' (including other modules) may already have been '
+ 'deleted or set\n'
+ ' to "None". Python guarantees that globals whose name '
+ 'begins\n'
+ ' with a single underscore are deleted from their '
+ 'module before\n'
+ ' other globals are deleted; if no other references to '
+ 'such\n'
+ ' globals exist, this may help in assuring that '
+ 'imported modules\n'
+ ' are still available at the time when the "__del__()" '
+ 'method is\n'
+ ' called.\n'
'\n'
'object.__repr__(self)\n'
'\n'
'conversion] [":" format_spec] "}"\n'
' field_name ::= arg_name ("." attribute_name | '
'"[" element_index "]")*\n'
- ' arg_name ::= [identifier | integer]\n'
+ ' arg_name ::= [identifier | digit+]\n'
' attribute_name ::= identifier\n'
- ' element_index ::= integer | index_string\n'
+ ' element_index ::= digit+ | index_string\n'
' index_string ::= <any source character except '
'"]"> +\n'
' conversion ::= "r" | "s" | "a"\n'
' fill ::= <any character>\n'
' align ::= "<" | ">" | "=" | "^"\n'
' sign ::= "+" | "-" | " "\n'
- ' width ::= integer\n'
+ ' width ::= digit+\n'
' grouping_option ::= "_" | ","\n'
- ' precision ::= integer\n'
+ ' precision ::= digit+\n'
' type ::= "b" | "c" | "d" | "e" | "E" | "f" | '
'"F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%"\n'
'\n'
'object.__complex__(self)\n'
'object.__int__(self)\n'
'object.__float__(self)\n'
- 'object.__round__(self[, n])\n'
'\n'
' Called to implement the built-in functions "complex()", '
- '"int()",\n'
- ' "float()" and "round()". Should return a value of the '
- 'appropriate\n'
- ' type.\n'
+ '"int()" and\n'
+ ' "float()". Should return a value of the appropriate '
+ 'type.\n'
'\n'
'object.__index__(self)\n'
'\n'
'when\n'
' "__index__()" is defined "__int__()" should also be '
'defined, and\n'
- ' both should return the same value.\n',
+ ' both should return the same value.\n'
+ '\n'
+ 'object.__round__(self[, ndigits])\n'
+ 'object.__trunc__(self)\n'
+ 'object.__floor__(self)\n'
+ 'object.__ceil__(self)\n'
+ '\n'
+ ' Called to implement the built-in function "round()" and '
+ '"math"\n'
+ ' functions "trunc()", "floor()" and "ceil()". Unless '
+ '*ndigits* is\n'
+ ' passed to "__round__()" all these methods should return '
+ 'the value\n'
+ ' of the object truncated to an "Integral" (typically an '
+ '"int").\n'
+ '\n'
+ ' If "__int__()" is not defined then the built-in function '
+ '"int()"\n'
+ ' falls back to "__trunc__()".\n',
'objects': 'Objects, values and types\n'
'*************************\n'
'\n'
'\n'
' Called when the instance is about to be destroyed. This '
'is also\n'
- ' called a destructor. If a base class has a "__del__()" '
- 'method, the\n'
- ' derived class\'s "__del__()" method, if any, must '
- 'explicitly call it\n'
- ' to ensure proper deletion of the base class part of the '
- 'instance.\n'
- ' Note that it is possible (though not recommended!) for '
+ ' called a finalizer or (improperly) a destructor. If a '
+ 'base class\n'
+ ' has a "__del__()" method, the derived class\'s '
+ '"__del__()" method,\n'
+ ' if any, must explicitly call it to ensure proper deletion '
+ 'of the\n'
+ ' base class part of the instance.\n'
+ '\n'
+ ' It is possible (though not recommended!) for the '
+ '"__del__()" method\n'
+ ' to postpone destruction of the instance by creating a new '
+ 'reference\n'
+ ' to it. This is called object *resurrection*. It is\n'
+ ' implementation-dependent whether "__del__()" is called a '
+ 'second\n'
+ ' time when a resurrected object is about to be destroyed; '
'the\n'
- ' "__del__()" method to postpone destruction of the '
- 'instance by\n'
- ' creating a new reference to it. It may then be called at '
- 'a later\n'
- ' time when this new reference is deleted. It is not '
- 'guaranteed that\n'
- ' "__del__()" methods are called for objects that still '
- 'exist when\n'
- ' the interpreter exits.\n'
+ ' current *CPython* implementation only calls it once.\n'
+ '\n'
+ ' It is not guaranteed that "__del__()" methods are called '
+ 'for\n'
+ ' objects that still exist when the interpreter exits.\n'
'\n'
' Note: "del x" doesn\'t directly call "x.__del__()" --- '
'the former\n'
' decrements the reference count for "x" by one, and the '
'latter is\n'
- ' only called when "x"\'s reference count reaches zero. '
- 'Some common\n'
- ' situations that may prevent the reference count of an '
- 'object from\n'
- ' going to zero include: circular references between '
- 'objects (e.g.,\n'
- ' a doubly-linked list or a tree data structure with '
- 'parent and\n'
- ' child pointers); a reference to the object on the stack '
- 'frame of\n'
- ' a function that caught an exception (the traceback '
- 'stored in\n'
- ' "sys.exc_info()[2]" keeps the stack frame alive); or a '
+ ' only called when "x"\'s reference count reaches zero.\n'
+ '\n'
+ ' **CPython implementation detail:** It is possible for a '
'reference\n'
- ' to the object on the stack frame that raised an '
- 'unhandled\n'
- ' exception in interactive mode (the traceback stored in\n'
- ' "sys.last_traceback" keeps the stack frame alive). The '
- 'first\n'
- ' situation can only be remedied by explicitly breaking '
- 'the cycles;\n'
- ' the second can be resolved by freeing the reference to '
- 'the\n'
- ' traceback object when it is no longer useful, and the '
- 'third can\n'
- ' be resolved by storing "None" in "sys.last_traceback". '
- 'Circular\n'
- ' references which are garbage are detected and cleaned '
- 'up when the\n'
- " cyclic garbage collector is enabled (it's on by "
- 'default). Refer\n'
- ' to the documentation for the "gc" module for more '
- 'information\n'
- ' about this topic.\n'
+ ' cycle to prevent the reference count of an object from '
+ 'going to\n'
+ ' zero. In this case, the cycle will be later detected and '
+ 'deleted\n'
+ ' by the *cyclic garbage collector*. A common cause of '
+ 'reference\n'
+ ' cycles is when an exception has been caught in a local '
+ 'variable.\n'
+ " The frame's locals then reference the exception, which "
+ 'references\n'
+ ' its own traceback, which references the locals of all '
+ 'frames caught\n'
+ ' in the traceback.\n'
+ '\n'
+ ' See also: Documentation for the "gc" module.\n'
'\n'
' Warning: Due to the precarious circumstances under which\n'
' "__del__()" methods are invoked, exceptions that occur '
'during\n'
' their execution are ignored, and a warning is printed '
'to\n'
- ' "sys.stderr" instead. Also, when "__del__()" is invoked '
- 'in\n'
- ' response to a module being deleted (e.g., when '
- 'execution of the\n'
- ' program is done), other globals referenced by the '
+ ' "sys.stderr" instead. In particular:\n'
+ '\n'
+ ' * "__del__()" can be invoked when arbitrary code is '
+ 'being\n'
+ ' executed, including from any arbitrary thread. If '
'"__del__()"\n'
- ' method may already have been deleted or in the process '
- 'of being\n'
- ' torn down (e.g. the import machinery shutting down). '
- 'For this\n'
- ' reason, "__del__()" methods should do the absolute '
- 'minimum needed\n'
- ' to maintain external invariants. Starting with version '
- '1.5,\n'
- ' Python guarantees that globals whose name begins with a '
- 'single\n'
- ' underscore are deleted from their module before other '
- 'globals are\n'
- ' deleted; if no other references to such globals exist, '
- 'this may\n'
- ' help in assuring that imported modules are still '
- 'available at the\n'
- ' time when the "__del__()" method is called.\n'
+ ' needs to take a lock or invoke any other blocking '
+ 'resource, it\n'
+ ' may deadlock as the resource may already be taken by '
+ 'the code\n'
+ ' that gets interrupted to execute "__del__()".\n'
+ '\n'
+ ' * "__del__()" can be executed during interpreter '
+ 'shutdown. As\n'
+ ' a consequence, the global variables it needs to '
+ 'access\n'
+ ' (including other modules) may already have been '
+ 'deleted or set\n'
+ ' to "None". Python guarantees that globals whose name '
+ 'begins\n'
+ ' with a single underscore are deleted from their '
+ 'module before\n'
+ ' other globals are deleted; if no other references to '
+ 'such\n'
+ ' globals exist, this may help in assuring that '
+ 'imported modules\n'
+ ' are still available at the time when the "__del__()" '
+ 'method is\n'
+ ' called.\n'
'\n'
'object.__repr__(self)\n'
'\n'
'\n'
'object.__getattr__(self, name)\n'
'\n'
- ' Called when an attribute lookup has not found the '
- 'attribute in the\n'
- ' usual places (i.e. it is not an instance attribute nor is '
- 'it found\n'
- ' in the class tree for "self"). "name" is the attribute '
- 'name. This\n'
- ' method should return the (computed) attribute value or '
- 'raise an\n'
- ' "AttributeError" exception.\n'
+ ' Called when the default attribute access fails with an\n'
+ ' "AttributeError" (either "__getattribute__()" raises an\n'
+ ' "AttributeError" because *name* is not an instance '
+ 'attribute or an\n'
+ ' attribute in the class tree for "self"; or "__get__()" of '
+ 'a *name*\n'
+ ' property raises "AttributeError"). This method should '
+ 'either\n'
+ ' return the (computed) attribute value or raise an '
+ '"AttributeError"\n'
+ ' exception.\n'
'\n'
' Note that if the attribute is found through the normal '
'mechanism,\n'
' sorts it.\n'
'\n'
'\n'
+ 'Customizing module attribute access\n'
+ '-----------------------------------\n'
+ '\n'
+ 'For a more fine grained customization of the module behavior '
+ '(setting\n'
+ 'attributes, properties, etc.), one can set the "__class__" '
+ 'attribute\n'
+ 'of a module object to a subclass of "types.ModuleType". For '
+ 'example:\n'
+ '\n'
+ ' import sys\n'
+ ' from types import ModuleType\n'
+ '\n'
+ ' class VerboseModule(ModuleType):\n'
+ ' def __repr__(self):\n'
+ " return f'Verbose {self.__name__}'\n"
+ '\n'
+ ' def __setattr__(self, attr, value):\n'
+ " print(f'Setting {attr}...')\n"
+ ' setattr(self, attr, value)\n'
+ '\n'
+ ' sys.modules[__name__].__class__ = VerboseModule\n'
+ '\n'
+ 'Note: Setting module "__class__" only affects lookups made '
+ 'using the\n'
+ ' attribute access syntax -- directly accessing the module '
+ 'globals\n'
+ ' (whether by code within the module, or via a reference to '
+ 'the\n'
+ " module's globals dictionary) is unaffected.\n"
+ '\n'
+ 'Changed in version 3.5: "__class__" module attribute is now '
+ 'writable.\n'
+ '\n'
+ '\n'
'Implementing Descriptors\n'
'------------------------\n'
'\n'
'__slots__\n'
'---------\n'
'\n'
- 'By default, instances of classes have a dictionary for '
- 'attribute\n'
- 'storage. This wastes space for objects having very few '
- 'instance\n'
- 'variables. The space consumption can become acute when '
- 'creating large\n'
- 'numbers of instances.\n'
+ '*__slots__* allow us to explicitly declare data members '
+ '(like\n'
+ 'properties) and deny the creation of *__dict__* and '
+ '*__weakref__*\n'
+ '(unless explicitly declared in *__slots__* or available in a '
+ 'parent.)\n'
'\n'
- 'The default can be overridden by defining *__slots__* in a '
- 'class\n'
- 'definition. The *__slots__* declaration takes a sequence of '
- 'instance\n'
- 'variables and reserves just enough space in each instance to '
- 'hold a\n'
- 'value for each variable. Space is saved because *__dict__* '
- 'is not\n'
- 'created for each instance.\n'
+ 'The space saved over using *__dict__* can be significant.\n'
'\n'
'object.__slots__\n'
'\n'
'\n'
'* When inheriting from a class without *__slots__*, the '
'*__dict__*\n'
- ' attribute of that class will always be accessible, so a '
- '*__slots__*\n'
- ' definition in the subclass is meaningless.\n'
+ ' and *__weakref__* attribute of the instances will always '
+ 'be\n'
+ ' accessible.\n'
'\n'
'* Without a *__dict__* variable, instances cannot be '
'assigned new\n'
'class\n'
' attribute would overwrite the descriptor assignment.\n'
'\n'
- '* The action of a *__slots__* declaration is limited to the '
- 'class\n'
- ' where it is defined. As a result, subclasses will have a '
- '*__dict__*\n'
- ' unless they also define *__slots__* (which must only '
- 'contain names\n'
- ' of any *additional* slots).\n'
+ '* The action of a *__slots__* declaration is not limited to '
+ 'the\n'
+ ' class where it is defined. *__slots__* declared in '
+ 'parents are\n'
+ ' available in child classes. However, child subclasses will '
+ 'get a\n'
+ ' *__dict__* and *__weakref__* unless they also define '
+ '*__slots__*\n'
+ ' (which should only contain names of any *additional* '
+ 'slots).\n'
'\n'
'* If a class defines a slot also defined in a base class, '
'the\n'
'same\n'
' *__slots__*.\n'
'\n'
+ '* Multiple inheritance with multiple slotted parent classes '
+ 'can be\n'
+ ' used, but only one parent is allowed to have attributes '
+ 'created by\n'
+ ' slots (the other bases must have empty slot layouts) - '
+ 'violations\n'
+ ' raise "TypeError".\n'
+ '\n'
'\n'
'Customizing class creation\n'
'==========================\n'
'object.__complex__(self)\n'
'object.__int__(self)\n'
'object.__float__(self)\n'
- 'object.__round__(self[, n])\n'
'\n'
' Called to implement the built-in functions "complex()", '
- '"int()",\n'
- ' "float()" and "round()". Should return a value of the '
- 'appropriate\n'
- ' type.\n'
+ '"int()" and\n'
+ ' "float()". Should return a value of the appropriate '
+ 'type.\n'
'\n'
'object.__index__(self)\n'
'\n'
'defined, and\n'
' both should return the same value.\n'
'\n'
+ 'object.__round__(self[, ndigits])\n'
+ 'object.__trunc__(self)\n'
+ 'object.__floor__(self)\n'
+ 'object.__ceil__(self)\n'
+ '\n'
+ ' Called to implement the built-in function "round()" and '
+ '"math"\n'
+ ' functions "trunc()", "floor()" and "ceil()". Unless '
+ '*ndigits* is\n'
+ ' passed to "__round__()" all these methods should return '
+ 'the value\n'
+ ' of the object truncated to an "Integral" (typically an '
+ '"int").\n'
+ '\n'
+ ' If "__int__()" is not defined then the built-in function '
+ '"int()"\n'
+ ' falls back to "__trunc__()".\n'
+ '\n'
'\n'
'With Statement Context Managers\n'
'===============================\n'
' formatting options that can be specified in format '
'strings.\n'
'\n'
+ ' Note: When formatting a number ("int", "float", "float" '
+ 'and\n'
+ ' subclasses) with the "n" type (ex: '
+ '"\'{:n}\'.format(1234)"), the\n'
+ ' function sets temporarily the "LC_CTYPE" locale to '
+ 'the\n'
+ ' "LC_NUMERIC" locale to decode "decimal_point" and '
+ '"thousands_sep"\n'
+ ' fields of "localeconv()" if they are non-ASCII or '
+ 'longer than 1\n'
+ ' byte, and the "LC_NUMERIC" locale is different than '
+ 'the\n'
+ ' "LC_CTYPE" locale. This temporary change affects '
+ 'other threads.\n'
+ '\n'
+ ' Changed in version 3.6.5: When formatting a number with '
+ 'the "n"\n'
+ ' type, the function sets temporarily the "LC_CTYPE" '
+ 'locale to the\n'
+ ' "LC_NUMERIC" locale in some cases.\n'
+ '\n'
'str.format_map(mapping)\n'
'\n'
' Similar to "str.format(**mapping)", except that '
' sequence concatenation or repetition.\n'
'\n'
'8. "index" raises "ValueError" when *x* is not found in *s*. '
- 'When\n'
- ' supported, the additional arguments to the index method '
- 'allow\n'
- ' efficient searching of subsections of the sequence. Passing '
- 'the\n'
- ' extra arguments is roughly equivalent to using '
- '"s[i:j].index(x)",\n'
- ' only without copying any data and with the returned index '
- 'being\n'
- ' relative to the start of the sequence rather than the start '
- 'of the\n'
- ' slice.\n'
+ 'Not\n'
+ ' all implementations support passing the additional arguments '
+ '*i*\n'
+ ' and *j*. These arguments allow efficient searching of '
+ 'subsections\n'
+ ' of the sequence. Passing the extra arguments is roughly '
+ 'equivalent\n'
+ ' to using "s[i:j].index(x)", only without copying any data and '
+ 'with\n'
+ ' the returned index being relative to the start of the '
+ 'sequence\n'
+ ' rather than the start of the slice.\n'
'\n'
'\n'
'Immutable Sequence Types\n'
from_addr = (msg[header_prefix + 'Sender']
if (header_prefix + 'Sender') in msg
else msg[header_prefix + 'From'])
+ from_addr = email.utils.getaddresses([from_addr])[0][1]
if to_addrs is None:
addr_fields = [f for f in (msg[header_prefix + 'To'],
msg[header_prefix + 'Bcc'],
# base-256 representation. This allows values up to (256**(digits-1))-1.
# A 0o200 byte indicates a positive number, a 0o377 byte a negative
# number.
+ n = int(n)
if 0 <= n < 8 ** (digits - 1):
- s = bytes("%0*o" % (digits - 1, int(n)), "ascii") + NUL
+ s = bytes("%0*o" % (digits - 1, n), "ascii") + NUL
elif format == GNU_FORMAT and -256 ** (digits - 1) <= n < 256 ** (digits - 1):
if n >= 0:
s = bytearray([0o200])
# The old GNU sparse format occupies some of the unused
# space in the buffer for up to 4 sparse structures.
- # Save the them for later processing in _proc_sparse().
+ # Save them for later processing in _proc_sparse().
if obj.type == GNUTYPE_SPARSE:
pos = 386
structs = []
close_queue(q)
@classmethod
- def _test_error_on_stdio_flush(self, evt):
+ def _test_error_on_stdio_flush(self, evt, break_std_streams={}):
+ for stream_name, action in break_std_streams.items():
+ if action == 'close':
+ stream = io.StringIO()
+ stream.close()
+ else:
+ assert action == 'remove'
+ stream = None
+ setattr(sys, stream_name, None)
evt.set()
- def test_error_on_stdio_flush(self):
+ def test_error_on_stdio_flush_1(self):
+ # Check that Process works with broken standard streams
streams = [io.StringIO(), None]
streams[0].close()
for stream_name in ('stdout', 'stderr'):
proc.start()
proc.join()
self.assertTrue(evt.is_set())
+ self.assertEqual(proc.exitcode, 0)
+ finally:
+ setattr(sys, stream_name, old_stream)
+
+ def test_error_on_stdio_flush_2(self):
+ # Same as test_error_on_stdio_flush_1(), but standard streams are
+ # broken by the child process
+ for stream_name in ('stdout', 'stderr'):
+ for action in ('close', 'remove'):
+ old_stream = getattr(sys, stream_name)
+ try:
+ evt = self.Event()
+ proc = self.Process(target=self._test_error_on_stdio_flush,
+ args=(evt, {stream_name: action}))
+ proc.start()
+ proc.join()
+ self.assertTrue(evt.is_set())
+ self.assertEqual(proc.exitcode, 0)
finally:
setattr(sys, stream_name, old_stream)
#
class TestForkAwareThreadLock(unittest.TestCase):
- # We recurisvely start processes. Issue #17555 meant that the
+ # We recursively start processes. Issue #17555 meant that the
# after fork registry would get duplicate entries for the same
# lock. The size of the registry at generation n was ~2**n.
'''
r, w = os.pipe()
p = subprocess.Popen([sys.executable,
- '-c', cmd % (w, w)],
+ '-E', '-c', cmd % (w, w)],
pass_fds=[w],
stderr=subprocess.PIPE)
os.close(w)
# Make sure comparison doesn't forget microseconds, and isn't done
# via comparing a float timestamp (an IEEE double doesn't have enough
- # precision to span microsecond resolution across years 1 thru 9999,
+ # precision to span microsecond resolution across years 1 through 9999,
# so comparing via timestamp necessarily calls some distinct values
# equal).
dt1 = self.theclass(MAXYEAR, 12, 31, 23, 59, 59, 999998)
+-- BufferError
+-- EOFError
+-- ImportError
- +-- ModuleNotFoundError
+ | +-- ModuleNotFoundError
+-- LookupError
| +-- IndexError
| +-- KeyError
if hasattr(module, '__path__'):
for index, path in enumerate(module.__path__):
module.__path__[index] = os.path.abspath(path)
- if hasattr(module, '__file__'):
+ if getattr(module, '__file__', None):
module.__file__ = os.path.abspath(module.__file__)
# MacOSX (a.k.a. Darwin) has a default stack size that is too small
self.assertEqual(str(a2), "[0, 1, 2, [...], 3]")
self.assertEqual(repr(a2), "[0, 1, 2, [...], 3]")
- l0 = []
+ def test_repr_deep(self):
+ a = self.type2test([])
for i in range(sys.getrecursionlimit() + 100):
- l0 = [l0]
- self.assertRaises(RecursionError, repr, l0)
+ a = self.type2test([a])
+ self.assertRaises(RecursionError, repr, a)
def test_print(self):
d = self.type2test(range(200))
sem = self.semtype(7)
sem.acquire()
N = 10
+ sem_results = []
results1 = []
results2 = []
phase_num = 0
def f():
- sem.acquire()
+ sem_results.append(sem.acquire())
results1.append(phase_num)
- sem.acquire()
+ sem_results.append(sem.acquire())
results2.append(phase_num)
b = Bunch(f, 10)
b.wait_for_started()
# Final release, to let the last thread finish
sem.release()
b.wait_for_finished()
+ self.assertEqual(sem_results, [True] * (6 + 7 + 6 + 1))
def test_try_acquire(self):
sem = self.semtype(2)
# tests common to dict and UserDict
import unittest
import collections
+import sys
class BasicTestMappingProtocol(unittest.TestCase):
d = self._full_mapping({1: BadRepr()})
self.assertRaises(Exc, repr, d)
+ def test_repr_deep(self):
+ d = self._empty_mapping()
+ for i in range(sys.getrecursionlimit() + 100):
+ d0 = d
+ d = self._empty_mapping()
+ d[1] = d0
+ self.assertRaises(RecursionError, repr, d)
+
def test_eq(self):
self.assertEqual(self._empty_mapping(), self._empty_mapping())
self.assertEqual(self._full_mapping({1: 2}),
"""
-Collect various informations about Python to help debugging test failures.
+Collect various information about Python to help debugging test failures.
"""
from __future__ import print_function
import errno
def get_infos(self):
"""
- Get informations as a key:value dictionary where values are strings.
+ Get information as a key:value dictionary where values are strings.
"""
return {key: str(value) for key, value in self.info.items()}
info_add(name, value)
+def copy_attr(info_add, name, mod, attr_name):
+ try:
+ value = getattr(mod, attr_name)
+ except AttributeError:
+ return
+ info_add(name, value)
+
+
def call_func(info_add, name, mod, func_name, *, formatter=None):
try:
func = getattr(mod, func_name)
info_add('locale.encoding', locale.getpreferredencoding(False))
+def collect_builtins(info_add):
+ info_add('builtins.float.float_format', float.__getformat__("float"))
+ info_add('builtins.float.double_format', float.__getformat__("double"))
+
+
def collect_os(info_add):
import os
)
copy_attributes(info_add, os, 'os.%s', attributes, formatter=format_attr)
- info_add("os.cwd", os.getcwd())
+ call_func(info_add, 'os.cwd', os, 'getcwd')
call_func(info_add, 'os.uid', os, 'getuid')
call_func(info_add, 'os.gid', os, 'getgid')
call_func(info_add, 'os.uname', os, 'uname')
- if hasattr(os, 'getgroups'):
- groups = os.getgroups()
- groups = map(str, groups)
- groups = ', '.join(groups)
- info_add("os.groups", groups)
+ def format_groups(groups):
+ return ', '.join(map(str, groups))
+
+ call_func(info_add, 'os.groups', os, 'getgroups', formatter=format_groups)
if hasattr(os, 'getlogin'):
try:
else:
info_add("os.login", login)
- if hasattr(os, 'cpu_count'):
- cpu_count = os.cpu_count()
- if cpu_count:
- info_add('os.cpu_count', cpu_count)
-
+ call_func(info_add, 'os.cpu_count', os, 'cpu_count')
call_func(info_add, 'os.loadavg', os, 'getloadavg')
# Get environment variables: filter to list
)
for name, value in os.environ.items():
uname = name.upper()
- if (uname in ENV_VARS or uname.startswith(("PYTHON", "LC_"))
+ if (uname in ENV_VARS
+ # Copy PYTHON* and LC_* variables
+ or uname.startswith(("PYTHON", "LC_"))
# Visual Studio: VS140COMNTOOLS
or (uname.startswith("VS") and uname.endswith("COMNTOOLS"))):
info_add('os.environ[%s]' % name, value)
def collect_time(info_add):
import time
+ info_add('time.time', time.time())
+
attributes = (
'altzone',
'daylight',
)
copy_attributes(info_add, time, 'time.%s', attributes)
- if not hasattr(time, 'get_clock_info'):
+ if hasattr(time, 'get_clock_info'):
+ for clock in ('time', 'perf_counter'):
+ tinfo = time.get_clock_info(clock)
+ info_add('time.get_clock_info(%s)' % clock, tinfo)
+
+
+def collect_datetime(info_add):
+ try:
+ import datetime
+ except ImportError:
return
- for clock in ('time', 'perf_counter'):
- tinfo = time.get_clock_info(clock)
- info_add('time.%s' % clock, tinfo)
+ info_add('datetime.datetime.now', datetime.datetime.now())
def collect_sysconfig(info_add):
'CCSHARED',
'CFLAGS',
'CFLAGSFORSHARED',
- 'PY_LDFLAGS',
'CONFIG_ARGS',
'HOST_GNU_TYPE',
'MACHDEP',
'OPT',
'PY_CFLAGS',
'PY_CFLAGS_NODIST',
+ 'PY_LDFLAGS',
'Py_DEBUG',
'Py_ENABLE_SHARED',
'SHELL',
copy_attributes(info_add, _decimal, '_decimal.%s', attributes)
+def collect_testcapi(info_add):
+ try:
+ import _testcapi
+ except ImportError:
+ return
+
+ call_func(info_add, 'pymem.allocator', _testcapi, 'pymem_getallocatorsname')
+ copy_attr(info_add, 'pymem.with_pymalloc', _testcapi, 'WITH_PYMALLOC')
+
+
+def collect_resource(info_add):
+ try:
+ import resource
+ except ImportError:
+ return
+
+ limits = [attr for attr in dir(resource) if attr.startswith('RLIMIT_')]
+ for name in limits:
+ key = getattr(resource, name)
+ value = resource.getrlimit(key)
+ info_add('resource.%s' % name, value)
+
+
+def collect_test_socket(info_add):
+ try:
+ from test import test_socket
+ except ImportError:
+ return
+
+ # all check attributes like HAVE_SOCKET_CAN
+ attributes = [name for name in dir(test_socket)
+ if name.startswith('HAVE_')]
+ copy_attributes(info_add, test_socket, 'test_socket.%s', attributes)
+
+
+def collect_test_support(info_add):
+ try:
+ from test import support
+ except ImportError:
+ return
+
+ attributes = ('IPV6_ENABLED',)
+ copy_attributes(info_add, support, 'test_support.%s', attributes)
+
+ call_func(info_add, 'test_support._is_gui_available', support, '_is_gui_available')
+ call_func(info_add, 'test_support.python_is_optimized', support, 'python_is_optimized')
+
+
def collect_info(info):
error = False
info_add = info.add
# collect_os() should be the first, to check the getrandom() status
collect_os,
+ collect_builtins,
collect_gdb,
collect_locale,
collect_platform,
collect_sys,
collect_sysconfig,
collect_time,
+ collect_datetime,
collect_tkinter,
collect_zlib,
collect_expat,
collect_decimal,
+ collect_testcapi,
+ collect_resource,
+
+ # Collecting from tests should be last as they have side effects.
+ collect_test_socket,
+ collect_test_support,
):
try:
collect_func(info_add)
raise
warnings.warn('tests may fail, unable to create temp dir: ' + path,
RuntimeWarning, stacklevel=3)
+ if dir_created:
+ pid = os.getpid()
try:
yield path
finally:
- if dir_created:
+ # In case the process forks, let only the parent remove the
+ # directory. The child has a diffent process id. (bpo-30028)
+ if dir_created and pid == os.getpid():
rmtree(path)
@contextlib.contextmanager
def restore(self):
for signum, handler in self.handlers.items():
self.signal.signal(signum, handler)
+
+
+class FakePath:
+ """Simple implementing of the path protocol.
+ """
+ def __init__(self, path):
+ self.path = path
+
+ def __repr__(self):
+ return f'<FakePath {self.path!r}>'
+
+ def __fspath__(self):
+ if (isinstance(self.path, BaseException) or
+ isinstance(self.path, type) and
+ issubclass(self.path, BaseException)):
+ raise self.path
+ else:
+ return self.path
"""
global __cached_interp_requires_environment
if __cached_interp_requires_environment is None:
+ # If PYTHONHOME is set, assume that we need it
+ if 'PYTHONHOME' in os.environ:
+ __cached_interp_requires_environment = True
+ return True
+
# Try running an interpreter with -E to see if it works or not.
try:
subprocess.check_call([sys.executable, '-E',
kw is extra keyword args to pass to subprocess.Popen. Returns a Popen
object.
"""
- cmd_line = [sys.executable, '-E']
+ cmd_line = [sys.executable]
+ if not interpreter_requires_environment():
+ cmd_line.append('-E')
cmd_line.extend(args)
# Under Fedora (?), GNU readline can output junk on stderr when initialized,
# depending on the TERM setting. Setting TERM=vt100 is supposed to disable
b = io.BytesIO(b'FORM' + struct.pack('>L', 4) + b'AIFF')
self.assertRaises(aifc.Error, aifc.open, b)
+ def test_read_no_ssnd_chunk(self):
+ b = b'FORM' + struct.pack('>L', 4) + b'AIFC'
+ b += b'COMM' + struct.pack('>LhlhhLL', 38, 0, 0, 0, 0, 0, 0)
+ b += b'NONE' + struct.pack('B', 14) + b'not compressed' + b'\x00'
+ with self.assertRaisesRegex(aifc.Error, 'COMM chunk and/or SSND chunk'
+ ' missing'):
+ aifc.open(io.BytesIO(b))
+
def test_read_wrong_compression_type(self):
b = b'FORM' + struct.pack('>L', 4) + b'AIFC'
b += b'COMM' + struct.pack('>LhlhhLL', 23, 0, 0, 0, 0, 0, 0)
self.assertIsNone(
base_events._ipaddr_info('::3%lo0', 1, INET6, STREAM, TCP))
- if hasattr(socket, 'SOCK_NONBLOCK'):
- self.assertEqual(
- None,
- base_events._ipaddr_info(
- '1.2.3.4', 1, INET, STREAM | socket.SOCK_NONBLOCK, TCP))
-
-
def test_port_parameter_types(self):
# Test obscure kinds of arguments for "port".
INET = socket.AF_INET
self.assertEqual(5, proto.nbytes)
os.close(slave)
+ proto.transport.close()
self.loop.run_until_complete(proto.done)
self.assertEqual(
['INITIAL', 'CONNECTED', 'EOF', 'CLOSED'], proto.state)
self.assertTrue(tb.cancelled())
self.assertTrue(tc.done())
+ def test_cancel_release_race(self):
+ # Issue 32734
+ # Acquire 4 locks, cancel second, release first
+ # and 2 locks are taken at once.
+ lock = asyncio.Lock(loop=self.loop)
+ lock_count = 0
+ call_count = 0
+
+ async def lockit():
+ nonlocal lock_count
+ nonlocal call_count
+ call_count += 1
+ await lock.acquire()
+ lock_count += 1
+
+ async def lockandtrigger():
+ await lock.acquire()
+ self.loop.call_soon(trigger)
+
+ def trigger():
+ t1.cancel()
+ lock.release()
+
+ t0 = self.loop.create_task(lockandtrigger())
+ t1 = self.loop.create_task(lockit())
+ t2 = self.loop.create_task(lockit())
+ t3 = self.loop.create_task(lockit())
+
+ # First loop acquires all
+ test_utils.run_briefly(self.loop)
+ self.assertTrue(t0.done())
+
+ # Second loop calls trigger
+ test_utils.run_briefly(self.loop)
+ # Third loop calls cancellation
+ test_utils.run_briefly(self.loop)
+
+ # Make sure only one lock was taken
+ self.assertEqual(lock_count, 1)
+ # While 3 calls were made to lockit()
+ self.assertEqual(call_count, 3)
+ self.assertTrue(t1.cancelled() and t2.done())
+
+ # Cleanup the task that is stuck on acquire.
+ t3.cancel()
+ test_utils.run_briefly(self.loop)
+ self.assertTrue(t3.cancelled())
+
def test_finished_waiter_cancelled(self):
lock = asyncio.Lock(loop=self.loop)
self.assertTrue(cond.locked())
+ def test_wait_cancel_after_notify(self):
+ # See bpo-32841
+ cond = asyncio.Condition(loop=self.loop)
+ waited = False
+
+ async def wait_on_cond():
+ nonlocal waited
+ async with cond:
+ waited = True # Make sure this area was reached
+ await cond.wait()
+
+ waiter = asyncio.ensure_future(wait_on_cond(), loop=self.loop)
+ test_utils.run_briefly(self.loop) # Start waiting
+
+ self.loop.run_until_complete(cond.acquire())
+ cond.notify()
+ test_utils.run_briefly(self.loop) # Get to acquire()
+ waiter.cancel()
+ test_utils.run_briefly(self.loop) # Activate cancellation
+ cond.release()
+ test_utils.run_briefly(self.loop) # Cancellation should occur
+
+ self.assertTrue(waiter.cancelled())
+ self.assertTrue(waited)
+
def test_wait_unacquired(self):
cond = asyncio.Condition(loop=self.loop)
self.assertRaises(
from asyncio.selector_events import _SelectorSslTransport
from asyncio.selector_events import _SelectorSocketTransport
from asyncio.selector_events import _SelectorDatagramTransport
+from asyncio.selector_events import _set_nodelay
MOCK_ANY = mock.ANY
'Fatal error on transport\nprotocol:.*\ntransport:.*'),
exc_info=(ConnectionRefusedError, MOCK_ANY, MOCK_ANY))
+
+class TestSelectorUtils(test_utils.TestCase):
+ def check_set_nodelay(self, sock):
+ opt = sock.getsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY)
+ self.assertFalse(opt)
+
+ _set_nodelay(sock)
+
+ opt = sock.getsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY)
+ self.assertTrue(opt)
+
+ @unittest.skipUnless(hasattr(socket, 'TCP_NODELAY'),
+ 'need socket.TCP_NODELAY')
+ def test_set_nodelay(self):
+ sock = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM,
+ proto=socket.IPPROTO_TCP)
+ with sock:
+ self.check_set_nodelay(sock)
+
+ sock = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM,
+ proto=socket.IPPROTO_TCP)
+ with sock:
+ sock.setblocking(False)
+ self.check_set_nodelay(sock)
+
+
if __name__ == '__main__':
unittest.main()
self.loop = asyncio.new_event_loop()
self.set_event_loop(self.loop)
- def ssl_protocol(self, waiter=None):
+ def ssl_protocol(self, *, waiter=None, proto=None):
sslcontext = test_utils.dummy_ssl_context()
- app_proto = asyncio.Protocol()
- proto = sslproto.SSLProtocol(self.loop, app_proto, sslcontext, waiter)
- self.assertIs(proto._app_transport.get_protocol(), app_proto)
- self.addCleanup(proto._app_transport.close)
- return proto
-
- def connection_made(self, ssl_proto, do_handshake=None):
+ if proto is None: # app protocol
+ proto = asyncio.Protocol()
+ ssl_proto = sslproto.SSLProtocol(self.loop, proto, sslcontext, waiter)
+ self.assertIs(ssl_proto._app_transport.get_protocol(), proto)
+ self.addCleanup(ssl_proto._app_transport.close)
+ return ssl_proto
+
+ def connection_made(self, ssl_proto, *, do_handshake=None):
transport = mock.Mock()
sslpipe = mock.Mock()
sslpipe.shutdown.return_value = b''
# Python issue #23197: cancelling a handshake must not raise an
# exception or log an error, even if the handshake failed
waiter = asyncio.Future(loop=self.loop)
- ssl_proto = self.ssl_protocol(waiter)
+ ssl_proto = self.ssl_protocol(waiter=waiter)
handshake_fut = asyncio.Future(loop=self.loop)
def do_handshake(callback):
return []
waiter.cancel()
- self.connection_made(ssl_proto, do_handshake)
+ self.connection_made(ssl_proto, do_handshake=do_handshake)
with test_utils.disable_logger():
self.loop.run_until_complete(handshake_fut)
def test_eof_received_waiter(self):
waiter = asyncio.Future(loop=self.loop)
- ssl_proto = self.ssl_protocol(waiter)
+ ssl_proto = self.ssl_protocol(waiter=waiter)
self.connection_made(ssl_proto)
ssl_proto.eof_received()
test_utils.run_briefly(self.loop)
# _fatal_error() generates a NameError if sslproto.py
# does not import base_events.
waiter = asyncio.Future(loop=self.loop)
- ssl_proto = self.ssl_protocol(waiter)
+ ssl_proto = self.ssl_protocol(waiter=waiter)
# Temporarily turn off error logging so as not to spoil test output.
log_level = log.logger.getEffectiveLevel()
log.logger.setLevel(logging.FATAL)
# From issue #472.
# yield from waiter hang if lost_connection was called.
waiter = asyncio.Future(loop=self.loop)
- ssl_proto = self.ssl_protocol(waiter)
+ ssl_proto = self.ssl_protocol(waiter=waiter)
self.connection_made(ssl_proto)
ssl_proto.connection_lost(ConnectionAbortedError)
test_utils.run_briefly(self.loop)
def test_close_during_handshake(self):
# bpo-29743 Closing transport during handshake process leaks socket
waiter = asyncio.Future(loop=self.loop)
- ssl_proto = self.ssl_protocol(waiter)
-
- def do_handshake(callback):
- return []
+ ssl_proto = self.ssl_protocol(waiter=waiter)
transport = self.connection_made(ssl_proto)
test_utils.run_briefly(self.loop)
def test_get_extra_info_on_closed_connection(self):
waiter = asyncio.Future(loop=self.loop)
- ssl_proto = self.ssl_protocol(waiter)
+ ssl_proto = self.ssl_protocol(waiter=waiter)
self.assertIsNone(ssl_proto._get_extra_info('socket'))
default = object()
self.assertIs(ssl_proto._get_extra_info('socket', default), default)
def test_set_new_app_protocol(self):
waiter = asyncio.Future(loop=self.loop)
- ssl_proto = self.ssl_protocol(waiter)
+ ssl_proto = self.ssl_protocol(waiter=waiter)
new_app_proto = asyncio.Protocol()
ssl_proto._app_transport.set_protocol(new_app_proto)
self.assertIs(ssl_proto._app_transport.get_protocol(), new_app_proto)
self.assertIs(ssl_proto._app_protocol, new_app_proto)
+ def test_data_received_after_closing(self):
+ ssl_proto = self.ssl_protocol()
+ self.connection_made(ssl_proto)
+ transp = ssl_proto._app_transport
+
+ transp.close()
+
+ # should not raise
+ self.assertIsNone(ssl_proto.data_received(b'data'))
+
+ def test_write_after_closing(self):
+ ssl_proto = self.ssl_protocol()
+ self.connection_made(ssl_proto)
+ transp = ssl_proto._app_transport
+ transp.close()
+
+ # should not raise
+ self.assertIsNone(transp.write(b'data'))
+
if __name__ == '__main__':
unittest.main()
Task = getattr(tasks, '_CTask', None)
Future = getattr(futures, '_CFuture', None)
+ @support.refcount_test
+ def test_refleaks_in_task___init__(self):
+ gettotalrefcount = support.get_attribute(sys, 'gettotalrefcount')
+ @asyncio.coroutine
+ def coro():
+ pass
+ task = self.new_task(self.loop, coro())
+ self.loop.run_until_complete(task)
+ refs_before = gettotalrefcount()
+ for i in range(100):
+ task.__init__(coro(), loop=self.loop)
+ self.loop.run_until_complete(task)
+ self.assertAlmostEqual(gettotalrefcount() - refs_before, 0, delta=10)
+
@unittest.skipUnless(hasattr(futures, '_CFuture'),
'requires the C _asyncio module')
self.assertEqual(len(self.loop._signal_handlers), 0)
m_signal.set_wakeup_fd.assert_called_once_with(-1)
+ @mock.patch('asyncio.unix_events.sys')
+ @mock.patch('asyncio.unix_events.signal')
+ def test_close_on_finalizing(self, m_signal, m_sys):
+ m_signal.NSIG = signal.NSIG
+ self.loop.add_signal_handler(signal.SIGHUP, lambda: True)
+
+ self.assertEqual(len(self.loop._signal_handlers), 1)
+ m_sys.is_finalizing.return_value = True
+ m_signal.signal.reset_mock()
+
+ with self.assertWarnsRegex(ResourceWarning,
+ "skipping signal handlers removal"):
+ self.loop.close()
+
+ self.assertEqual(len(self.loop._signal_handlers), 0)
+ self.assertFalse(m_signal.signal.called)
+
@unittest.skipUnless(hasattr(socket, 'AF_UNIX'),
'UNIX Sockets are not supported')
import io
import atexit
from test import support
+from test.support import script_helper
### helpers
def h1():
atexit._run_exitfuncs()
self.assertEqual(l, [5])
+ def test_shutdown(self):
+ # Actually test the shutdown mechanism in a subprocess
+ code = """if 1:
+ import atexit
+
+ def f(msg):
+ print(msg)
+
+ atexit.register(f, "one")
+ atexit.register(f, "two")
+ """
+ res = script_helper.assert_python_ok("-c", code)
+ self.assertEqual(res.out.decode().splitlines(), ["two", "one"])
+ self.assertFalse(res.err)
+
@support.cpython_only
class SubinterpreterTest(unittest.TestCase):
self.raise_fails("spam")
def test_catch_non_BaseException(self):
- # Tryinng to catch an object that does not inherit from BaseException
+ # Trying to catch an object that does not inherit from BaseException
# is not allowed.
class NonBaseException(object):
pass
try:
assert False
except AssertionError:
- return (True, f.__doc__)
+ return (True, f.__doc__, __debug__)
else:
- return (False, f.__doc__)
+ return (False, f.__doc__, __debug__)
'''
def f(): """doc"""
- values = [(-1, __debug__, f.__doc__),
- (0, True, 'doc'),
- (1, False, 'doc'),
- (2, False, None)]
- for optval, debugval, docstring in values:
+ values = [(-1, __debug__, f.__doc__, __debug__),
+ (0, True, 'doc', True),
+ (1, False, 'doc', False),
+ (2, False, None, False)]
+ for optval, *expected in values:
# test both direct compilation and compilation via AST
codeobjs = []
codeobjs.append(compile(codestr, "<test>", "exec", optimize=optval))
ns = {}
exec(code, ns)
rv = ns['f']()
- self.assertEqual(rv, (debugval, docstring))
+ self.assertEqual(rv, tuple(expected))
def test_delattr(self):
sys.spam = 1
import sys
from test.support import unlink
import _compression
+import sys
try:
import threading
# Previously, a second call could crash due to internal inconsistency
self.assertRaises(Exception, bzd.decompress, self.BAD_DATA * 30)
+ @support.refcount_test
+ def test_refleaks_in___init__(self):
+ gettotalrefcount = support.get_attribute(sys, 'gettotalrefcount')
+ bzd = BZ2Decompressor()
+ refs_before = gettotalrefcount()
+ for i in range(100):
+ bzd.__init__()
+ self.assertAlmostEqual(gettotalrefcount() - refs_before, 0, delta=10)
+
+
class CompressDecompressTest(BaseTest):
def testCompress(self):
data = bz2.compress(self.TEXT)
self.assertEqual(out, '')
self.assertEqual(err, '')
- def test_bpo20891(self):
- """
- bpo-20891: Calling PyGILState_Ensure in a non-Python thread before
- calling PyEval_InitThreads() must not crash. PyGILState_Ensure() must
- call PyEval_InitThreads() for us in this case.
- """
- out, err = self.run_embedded_interpreter("bpo20891")
- self.assertEqual(out, '')
- self.assertEqual(err, '')
-
class SkipitemTest(unittest.TestCase):
with self.assertRaises(TypeError):
type.__setattr__(A, b'x', None)
+
if __name__ == '__main__':
unittest.main()
Test for the function completedefault():
>>> mycmd.completedefault()
- This is the completedefault methode
+ This is the completedefault method
>>> mycmd.completenames("a")
['add']
print("Hello from postloop")
def completedefault(self, *ignored):
- print("This is the completedefault methode")
+ print("This is the completedefault method")
def complete_command(self):
print("complete command")
import tempfile
from test.support import script_helper, is_android
from test.support.script_helper import (spawn_python, kill_python, assert_python_ok,
- assert_python_failure)
+ assert_python_failure, interpreter_requires_environment)
# XXX (ncoghlan): Move to script_helper and make consistent with run_python
rc, out, err = assert_python_ok('-vv')
self.assertNotIn(b'stack overflow', err)
+ @unittest.skipIf(interpreter_requires_environment(),
+ 'Cannot run -E tests when PYTHON env vars are required.')
def test_xoptions(self):
def get_xoptions(*args):
# use subprocess module directly because test.support.script_helper adds
def test_displayhook_unencodable(self):
for encoding in ('ascii', 'latin-1', 'utf-8'):
- # We are testing a PYTHON environment variable here, so we can't
- # use -E, -I, or script_helper (which uses them). So instead we do
- # poor-man's isolation by deleting the PYTHON vars from env.
- env = {key:value for (key,value) in os.environ.copy().items()
- if not key.startswith('PYTHON')}
+ env = os.environ.copy()
env['PYTHONIOENCODING'] = encoding
p = subprocess.Popen(
[sys.executable, '-i'],
# Verify that sys.flags contains hash_randomization
code = 'import sys; print("random is", sys.flags.hash_randomization)'
- rc, out, err = assert_python_ok('-c', code)
- self.assertEqual(rc, 0)
+ rc, out, err = assert_python_ok('-c', code, PYTHONHASHSEED='')
+ self.assertIn(b'random is 1', out)
+
+ rc, out, err = assert_python_ok('-c', code, PYTHONHASHSEED='random')
self.assertIn(b'random is 1', out)
+ rc, out, err = assert_python_ok('-c', code, PYTHONHASHSEED='0')
+ self.assertIn(b'random is 0', out)
+
def test_del___main__(self):
# Issue #15001: PyRun_SimpleFileExFlags() did crash because it kept a
# borrowed reference to the dict of __main__ module and later modify
cwd=tmpdir)
self.assertEqual(out.strip(), b"ok")
+ @unittest.skipUnless(sys.platform == 'win32',
+ 'bpo-32457 only applies on Windows')
+ def test_argv0_normalization(self):
+ args = sys.executable, '-c', 'print(0)'
+ prefix, exe = os.path.split(sys.executable)
+ executable = prefix + '\\.\\.\\.\\' + exe
+
+ proc = subprocess.run(args, stdout=subprocess.PIPE,
+ executable=executable)
+ self.assertEqual(proc.returncode, 0, proc)
+ self.assertEqual(proc.stdout.strip(), b'0')
+
+
def test_main():
test.support.run_unittest(CmdLineTest)
test.support.reap_children()
def mutating(exc):
if isinstance(exc, UnicodeDecodeError):
- exc.object[:] = b""
+ exc.object = b""
return ("\u4242", 0)
else:
raise TypeError("don't know how to handle %r" % exc)
with test.support.check_warnings():
# unicode-internal has been deprecated
for (encoding, data) in baddata:
- with self.assertRaises(TypeError):
- data.decode(encoding, "test.replacing")
+ self.assertEqual(data.decode(encoding, "test.mutating"), "\u4242")
+
+ # issue32583
+ def test_crashing_decode_handler(self):
+ # better generating one more character to fill the extra space slot
+ # so in debug build it can steadily fail
+ def forward_shorter_than_end(exc):
+ if isinstance(exc, UnicodeDecodeError):
+ # size one character, 0 < forward < exc.end
+ return ('\ufffd', exc.start+1)
+ else:
+ raise TypeError("don't know how to handle %r" % exc)
+ codecs.register_error(
+ "test.forward_shorter_than_end", forward_shorter_than_end)
+
+ self.assertEqual(
+ b'\xd8\xd8\xd8\xd8\xd8\x00\x00\x00'.decode(
+ 'utf-16-le', 'test.forward_shorter_than_end'),
+ '\ufffd\ufffd\ufffd\ufffd\xd8\x00'
+ )
+ self.assertEqual(
+ b'\xd8\xd8\xd8\xd8\x00\xd8\x00\x00'.decode(
+ 'utf-16-be', 'test.forward_shorter_than_end'),
+ '\ufffd\ufffd\ufffd\ufffd\xd8\x00'
+ )
+ self.assertEqual(
+ b'\x11\x11\x11\x11\x11\x00\x00\x00\x00\x00\x00'.decode(
+ 'utf-32-le', 'test.forward_shorter_than_end'),
+ '\ufffd\ufffd\ufffd\u1111\x00'
+ )
+ self.assertEqual(
+ b'\x11\x11\x11\x00\x00\x11\x11\x00\x00\x00\x00'.decode(
+ 'utf-32-be', 'test.forward_shorter_than_end'),
+ '\ufffd\ufffd\ufffd\u1111\x00'
+ )
+
+ def replace_with_long(exc):
+ if isinstance(exc, UnicodeDecodeError):
+ exc.object = b"\x00" * 8
+ return ('\ufffd', exc.start)
+ else:
+ raise TypeError("don't know how to handle %r" % exc)
+ codecs.register_error("test.replace_with_long", replace_with_long)
+
+ self.assertEqual(
+ b'\x00'.decode('utf-16', 'test.replace_with_long'),
+ '\ufffd\x00\x00\x00\x00'
+ )
+ self.assertEqual(
+ b'\x00'.decode('utf-32', 'test.replace_with_long'),
+ '\ufffd\x00\x00'
+ )
+
def test_fake_error_class(self):
handlers = [
c = new_coro()
self.assertIsInstance(c, Awaitable)
- c.close() # awoid RuntimeWarning that coro() was not awaited
+ c.close() # avoid RuntimeWarning that coro() was not awaited
class CoroLike: pass
Coroutine.register(CoroLike)
c = new_coro()
self.assertIsInstance(c, Coroutine)
- c.close() # awoid RuntimeWarning that coro() was not awaited
+ c.close() # avoid RuntimeWarning that coro() was not awaited
class CoroLike:
def send(self, value):
'__len__', '__getitem__', '__setitem__', '__delitem__', 'insert')
def test_MutableSequence_mixins(self):
- # Test the mixins of MutableSequence by creating a miminal concrete
+ # Test the mixins of MutableSequence by creating a minimal concrete
# class inherited from it.
class MutableSequenceSubclass(MutableSequence):
def __init__(self):
import tempfile
import types
from test import support
-from test.support import script_helper
+from test.support import script_helper, FakePath
class TestSpecifics(unittest.TestCase):
import builtins
prev = builtins.__debug__
setattr(builtins, '__debug__', 'sure')
+ self.assertEqual(__debug__, prev)
setattr(builtins, '__debug__', prev)
def test_argument_handling(self):
def test_path_like_objects(self):
# An implicit test for PyUnicode_FSDecoder().
- class PathLike:
- def __init__(self, path):
- self._path = path
- def __fspath__(self):
- return self._path
-
- compile("42", PathLike("test_compile_pathlike"), "single")
+ compile("42", FakePath("test_compile_pathlike"), "single")
class TestStackSize(unittest.TestCase):
with TemporaryFile("w+", newline='') as fileobj:
writer = csv.writer(fileobj)
self.assertRaises(TypeError, writer.writerows, None)
- writer.writerows([['a','b'],['c','d']])
+ writer.writerows([['a', 'b'], ['c', 'd']])
fileobj.seek(0)
self.assertEqual(fileobj.read(), "a,b\r\nc,d\r\n")
+ def test_writerows_with_none(self):
+ with TemporaryFile("w+", newline='') as fileobj:
+ writer = csv.writer(fileobj)
+ writer.writerows([['a', None], [None, 'd']])
+ fileobj.seek(0)
+ self.assertEqual(fileobj.read(), "a,\r\n,d\r\n")
+
+ with TemporaryFile("w+", newline='') as fileobj:
+ writer = csv.writer(fileobj)
+ writer.writerows([[None], ['a']])
+ fileobj.seek(0)
+ self.assertEqual(fileobj.read(), '""\r\na\r\n')
+
+ with TemporaryFile("w+", newline='') as fileobj:
+ writer = csv.writer(fileobj)
+ writer.writerows([['a'], [None]])
+ fileobj.seek(0)
+ self.assertEqual(fileobj.read(), 'a\r\n""\r\n')
+
@support.cpython_only
def test_writerows_legacy_strings(self):
import _testcapi
self.assertEqual(sniffer.has_header(self.header2 + self.sample8),
True)
+ def test_guess_quote_and_delimiter(self):
+ sniffer = csv.Sniffer()
+ for header in (";'123;4';", "'123;4';", ";'123;4'", "'123;4'"):
+ with self.subTest(header):
+ dialect = sniffer.sniff(header, ",;")
+ self.assertEqual(dialect.delimiter, ';')
+ self.assertEqual(dialect.quotechar, "'")
+ self.assertIs(dialect.doublequote, False)
+ self.assertIs(dialect.skipinitialspace, False)
+
def test_sniff(self):
sniffer = csv.Sniffer()
dialect = sniffer.sniff(self.sample1)
self.assertEqual(sorted(f.keys()), sorted(self._dict))
f.close() # don't write
+ @unittest.skipUnless(support.TESTFN_NONASCII,
+ 'requires OS support of non-ASCII encodings')
+ def test_nonascii_filename(self):
+ filename = support.TESTFN_NONASCII
+ for suffix in ['.dir', '.dat', '.bak']:
+ self.addCleanup(support.unlink, filename + suffix)
+ with dumbdbm.open(filename, 'c') as db:
+ db[b'key'] = b'value'
+ self.assertTrue(os.path.exists(filename + '.dat'))
+ self.assertTrue(os.path.exists(filename + '.dir'))
+ with dumbdbm.open(filename, 'r') as db:
+ self.assertEqual(list(db.keys()), [b'key'])
+ self.assertTrue(b'key' in db)
+ self.assertEqual(db[b'key'], b'value')
+
def tearDown(self):
_delete_files()
gdbm = support.import_module("dbm.gnu") #skip if not supported
import unittest
import os
-from test.support import TESTFN, unlink
+from test.support import TESTFN, TESTFN_NONASCII, unlink
filename = TESTFN
self.assertEqual(str(cm.exception),
"GDBM object has already been closed")
+ def test_bytes(self):
+ with gdbm.open(filename, 'c') as db:
+ db[b'bytes key \xbd'] = b'bytes value \xbd'
+ with gdbm.open(filename, 'r') as db:
+ self.assertEqual(list(db.keys()), [b'bytes key \xbd'])
+ self.assertTrue(b'bytes key \xbd' in db)
+ self.assertEqual(db[b'bytes key \xbd'], b'bytes value \xbd')
+
+ def test_unicode(self):
+ with gdbm.open(filename, 'c') as db:
+ db['Unicode key \U0001f40d'] = 'Unicode value \U0001f40d'
+ with gdbm.open(filename, 'r') as db:
+ self.assertEqual(list(db.keys()), ['Unicode key \U0001f40d'.encode()])
+ self.assertTrue('Unicode key \U0001f40d'.encode() in db)
+ self.assertTrue('Unicode key \U0001f40d' in db)
+ self.assertEqual(db['Unicode key \U0001f40d'.encode()],
+ 'Unicode value \U0001f40d'.encode())
+ self.assertEqual(db['Unicode key \U0001f40d'],
+ 'Unicode value \U0001f40d'.encode())
+
+ @unittest.skipUnless(TESTFN_NONASCII,
+ 'requires OS support of non-ASCII encodings')
+ def test_nonascii_filename(self):
+ filename = TESTFN_NONASCII
+ self.addCleanup(unlink, filename)
+ with gdbm.open(filename, 'c') as db:
+ db[b'key'] = b'value'
+ self.assertTrue(os.path.exists(filename))
+ with gdbm.open(filename, 'r') as db:
+ self.assertEqual(list(db.keys()), [b'key'])
+ self.assertTrue(b'key' in db)
+ self.assertEqual(db[b'key'], b'value')
+
+
if __name__ == '__main__':
unittest.main()
from test import support
support.import_module("dbm.ndbm") #skip if not supported
+import os
import unittest
import dbm.ndbm
from dbm.ndbm import error
self.assertEqual(str(cm.exception),
"DBM object has already been closed")
+ def test_bytes(self):
+ with dbm.ndbm.open(self.filename, 'c') as db:
+ db[b'bytes key \xbd'] = b'bytes value \xbd'
+ with dbm.ndbm.open(self.filename, 'r') as db:
+ self.assertEqual(list(db.keys()), [b'bytes key \xbd'])
+ self.assertTrue(b'bytes key \xbd' in db)
+ self.assertEqual(db[b'bytes key \xbd'], b'bytes value \xbd')
+
+ def test_unicode(self):
+ with dbm.ndbm.open(self.filename, 'c') as db:
+ db['Unicode key \U0001f40d'] = 'Unicode value \U0001f40d'
+ with dbm.ndbm.open(self.filename, 'r') as db:
+ self.assertEqual(list(db.keys()), ['Unicode key \U0001f40d'.encode()])
+ self.assertTrue('Unicode key \U0001f40d'.encode() in db)
+ self.assertTrue('Unicode key \U0001f40d' in db)
+ self.assertEqual(db['Unicode key \U0001f40d'.encode()],
+ 'Unicode value \U0001f40d'.encode())
+ self.assertEqual(db['Unicode key \U0001f40d'],
+ 'Unicode value \U0001f40d'.encode())
+
+ @unittest.skipUnless(support.TESTFN_NONASCII,
+ 'requires OS support of non-ASCII encodings')
+ def test_nonascii_filename(self):
+ filename = support.TESTFN_NONASCII
+ for suffix in ['', '.pag', '.dir', '.db']:
+ self.addCleanup(support.unlink, filename + suffix)
+ with dbm.ndbm.open(filename, 'c') as db:
+ db[b'key'] = b'value'
+ self.assertTrue(any(os.path.exists(filename + suffix)
+ for suffix in ['', '.pag', '.dir', '.db']))
+ with dbm.ndbm.open(filename, 'r') as db:
+ self.assertEqual(list(db.keys()), [b'key'])
+ self.assertTrue(b'key' in db)
+ self.assertEqual(db[b'key'], b'value')
+
+
if __name__ == '__main__':
unittest.main()
del cm.x
self.assertNotHasAttr(cm, "x")
+ @support.refcount_test
+ def test_refleaks_in_classmethod___init__(self):
+ gettotalrefcount = support.get_attribute(sys, 'gettotalrefcount')
+ cm = classmethod(None)
+ refs_before = gettotalrefcount()
+ for i in range(100):
+ cm.__init__(None)
+ self.assertAlmostEqual(gettotalrefcount() - refs_before, 0, delta=10)
+
@support.impl_detail("the module 'xxsubtype' is internal")
def test_classmethods_in_c(self):
# Testing C-based class methods...
del sm.x
self.assertNotHasAttr(sm, "x")
+ @support.refcount_test
+ def test_refleaks_in_staticmethod___init__(self):
+ gettotalrefcount = support.get_attribute(sys, 'gettotalrefcount')
+ sm = staticmethod(None)
+ refs_before = gettotalrefcount()
+ for i in range(100):
+ sm.__init__(None)
+ self.assertAlmostEqual(gettotalrefcount() - refs_before, 0, delta=10)
+
@support.impl_detail("the module 'xxsubtype' is internal")
def test_staticmethods_in_c(self):
# Testing C-based static methods...
d = {1: BadRepr()}
self.assertRaises(Exc, repr, d)
+ def test_repr_deep(self):
+ d = {}
+ for i in range(sys.getrecursionlimit() + 100):
+ d = {1: d}
+ self.assertRaises(RecursionError, repr, d)
+
def test_eq(self):
self.assertEqual({}, {})
self.assertEqual({1: 2}, {1: 2})
import collections
import copy
import pickle
+import sys
import unittest
class DictSetTest(unittest.TestCase):
def test_recursive_repr(self):
d = {}
d[42] = d.values()
+ r = repr(d)
+ # Cannot perform a stronger test, as the contents of the repr
+ # are implementation-dependent. All we can say is that we
+ # want a str result, not an exception of any sort.
+ self.assertIsInstance(r, str)
+ d[42] = d.items()
+ r = repr(d)
+ # Again.
+ self.assertIsInstance(r, str)
+
+ def test_deeply_nested_repr(self):
+ d = {}
+ for i in range(sys.getrecursionlimit() + 100):
+ d = {42: d.values()}
self.assertRaises(RecursionError, repr, d)
def test_copy(self):
list(generator(*args))
self.assertEqual(msg, str(ctx.exception))
+class TestJunkAPIs(unittest.TestCase):
+ def test_is_line_junk_true(self):
+ for line in ['#', ' ', ' #', '# ', ' # ', '']:
+ self.assertTrue(difflib.IS_LINE_JUNK(line), repr(line))
+
+ def test_is_line_junk_false(self):
+ for line in ['##', ' ##', '## ', 'abc ', 'abc #', 'Mr. Moose is up!']:
+ self.assertFalse(difflib.IS_LINE_JUNK(line), repr(line))
+
+ def test_is_line_junk_REDOS(self):
+ evil_input = ('\t' * 1000000) + '##'
+ self.assertFalse(difflib.IS_LINE_JUNK(evil_input))
+
+ def test_is_character_junk_true(self):
+ for char in [' ', '\t']:
+ self.assertTrue(difflib.IS_CHARACTER_JUNK(char), repr(char))
+
+ def test_is_character_junk_false(self):
+ for char in ['a', '#', '\n', '\f', '\r', '\v']:
+ self.assertFalse(difflib.IS_CHARACTER_JUNK(char), repr(char))
def test_main():
difflib.HtmlDiff._default_prefix = 0
Doctests = doctest.DocTestSuite(difflib)
run_unittest(
TestWithAscii, TestAutojunk, TestSFpatches, TestSFbugs,
- TestOutputFormat, TestBytes, Doctests)
+ TestOutputFormat, TestBytes, TestJunkAPIs, Doctests)
if __name__ == '__main__':
test_main()
try:
output = self.trace(abspath("assert_usable" + self.EXTENSION))
output = output.strip()
- except (FileNotFoundError, PermissionError) as fnfe:
+ except (FileNotFoundError, NotADirectoryError, PermissionError) as fnfe:
output = str(fnfe)
if output != "probe: success":
raise unittest.SkipTest(
with self.assertRaises(errors.HeaderParseError):
parser.get_bare_quoted_string(' "foo"')
+ def test_get_bare_quoted_string_only_quotes(self):
+ self._test_get_x(parser.get_bare_quoted_string,
+ '""', '""', '', [], '')
+
def test_get_bare_quoted_string_following_wsp_preserved(self):
self._test_get_x(parser.get_bare_quoted_string,
'"foo"\t bar', '"foo"', 'foo', [], '\t bar')
self.assertIsNone(angle_addr.route)
self.assertEqual(angle_addr.addr_spec, '<>')
+ def test_get_angle_addr_qs_only_quotes(self):
+ angle_addr = self._test_get_x(parser.get_angle_addr,
+ '<""@example.com>',
+ '<""@example.com>',
+ '<""@example.com>',
+ [],
+ '')
+ self.assertEqual(angle_addr.token_type, 'angle-addr')
+ self.assertEqual(angle_addr.local_part, '')
+ self.assertEqual(angle_addr.domain, 'example.com')
+ self.assertIsNone(angle_addr.route)
+ self.assertEqual(angle_addr.addr_spec, '""@example.com')
+
def test_get_angle_addr_with_cfws(self):
angle_addr = self._test_get_x(parser.get_angle_addr,
' (foo) <dinsdale@example.com>(bar)',
self.client.auth()
self.assertRaises(ValueError, self.client.auth)
finally:
- self.client.ssl_version = ssl.PROTOCOL_TLSv1
+ self.client.ssl_version = ssl.PROTOCOL_TLS
def test_context(self):
self.client.quit()
- ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+ ctx = ssl.SSLContext(ssl.PROTOCOL_TLS)
self.assertRaises(ValueError, ftplib.FTP_TLS, keyfile=CERTFILE,
context=ctx)
self.assertRaises(ValueError, ftplib.FTP_TLS, certfile=CERTFILE,
def test_check_hostname(self):
self.client.quit()
- ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+ ctx = ssl.SSLContext(ssl.PROTOCOL_TLS)
ctx.verify_mode = ssl.CERT_REQUIRED
ctx.check_hostname = True
ctx.load_verify_locations(CAFILE)
p.keywords[self] = ['sth2']
return 'astr'
- # Raplacing the value during key formatting should keep the original
+ # Replacing the value during key formatting should keep the original
# value alive (at least long enough).
p.keywords[MutatesYourDict()] = ['sth']
r = repr(p)
# If we create a square with one exit, we must visit it next;
# else somebody else will have to visit it, and since there's
# only one adjacent, there won't be a way to leave it again.
- # Finelly, if we create more than one free square with a
+ # Finally, if we create more than one free square with a
# single exit, we can only move to one of them next, leaving
# the other one a dead end.
ne0 = ne1 = 0
succs[final].remove(corner)
add_to_successors(this)
- # Generate moves 3 thru m*n-1.
+ # Generate moves 3 through m*n-1.
def advance(len=len):
# If some successor has only one exit, must take it.
# Else favor successors with fewer exits.
yield i
add_to_successors(i)
- # Generate moves 3 thru m*n-1. Alternative version using a
+ # Generate moves 3 through m*n-1. Alternative version using a
# stronger (but more expensive) heuristic to order successors.
# Since the # of backtracking levels is m*n, a poor move early on
# can take eons to undo. Smallest square board for which this
import unittest
import warnings
from test import support
+from test.support.script_helper import assert_python_ok
+from test.support import FakePath
def create_file(filename, data=b'foo'):
with self.assertRaisesRegex(TypeError, 'bytearray'):
self.pathmodule.relpath(bytearray(b'foo'), bytearray(b'bar'))
+ def test_import(self):
+ assert_python_ok('-S', '-c', 'import ' + self.pathmodule.__name__)
-class PathLikeTests(unittest.TestCase):
- class PathLike:
- def __init__(self, path=''):
- self.path = path
- def __fspath__(self):
- if isinstance(self.path, BaseException):
- raise self.path
- else:
- return self.path
+class PathLikeTests(unittest.TestCase):
def setUp(self):
self.file_name = support.TESTFN.lower()
- self.file_path = self.PathLike(support.TESTFN)
+ self.file_path = FakePath(support.TESTFN)
self.addCleanup(support.unlink, self.file_name)
create_file(self.file_name, b"test_genericpath.PathLikeTests")
pattern = os.path.join(*parts)
p = os.path.join(self.tempdir, pattern)
res = glob.glob(p, **kwargs)
- self.assertEqual(list(glob.iglob(p, **kwargs)), res)
+ self.assertCountEqual(glob.iglob(p, **kwargs), res)
bres = [os.fsencode(x) for x in res]
- self.assertEqual(glob.glob(os.fsencode(p), **kwargs), bres)
- self.assertEqual(list(glob.iglob(os.fsencode(p), **kwargs)), bres)
+ self.assertCountEqual(glob.glob(os.fsencode(p), **kwargs), bres)
+ self.assertCountEqual(glob.iglob(os.fsencode(p), **kwargs), bres)
return res
def assertSequencesEqual_noorder(self, l1, l2):
self.assertEqual(f(spam='fried', **{'eggs':'scrambled'}),
((), {'eggs':'scrambled', 'spam':'fried'}))
+ # Check ast errors in *args and *kwargs
+ check_syntax_error(self, "f(*g(1=2))")
+ check_syntax_error(self, "f(**g(1=2))")
+
# argument annotation tests
def f(x) -> list: pass
self.assertEqual(f.__annotations__, {'return': list})
def f(*, k=1): return closure
def f() -> int: return closure
- # Check ast errors in *args and *kwargs
- check_syntax_error(self, "f(*g(1=2))")
- check_syntax_error(self, "f(**g(1=2))")
-
# Check trailing commas are permitted in funcdef argument list
def f(a,): pass
def f(*args,): pass
x = g2()
check_syntax_error(self, "class foo:return 1")
+ def test_break_in_finally(self):
+ count = 0
+ while count < 2:
+ count += 1
+ try:
+ pass
+ finally:
+ break
+ self.assertEqual(count, 1)
+
+ count = 0
+ while count < 2:
+ count += 1
+ try:
+ continue
+ finally:
+ break
+ self.assertEqual(count, 1)
+
+ count = 0
+ while count < 2:
+ count += 1
+ try:
+ 1/0
+ finally:
+ break
+ self.assertEqual(count, 1)
+
+ for count in [0, 1]:
+ self.assertEqual(count, 0)
+ try:
+ pass
+ finally:
+ break
+ self.assertEqual(count, 0)
+
+ for count in [0, 1]:
+ self.assertEqual(count, 0)
+ try:
+ continue
+ finally:
+ break
+ self.assertEqual(count, 0)
+
+ for count in [0, 1]:
+ self.assertEqual(count, 0)
+ try:
+ 1/0
+ finally:
+ break
+ self.assertEqual(count, 0)
+
+ def test_return_in_finally(self):
+ def g1():
+ try:
+ pass
+ finally:
+ return 1
+ self.assertEqual(g1(), 1)
+
+ def g2():
+ try:
+ return 2
+ finally:
+ return 3
+ self.assertEqual(g2(), 3)
+
+ def g3():
+ try:
+ 1/0
+ finally:
+ return 4
+ self.assertEqual(g3(), 4)
+
def test_yield(self):
# Allowed as standalone statement
def g(): yield 1
try: 1/0
except EOFError: pass
except TypeError as msg: pass
- except RuntimeError as msg: pass
except: pass
else: pass
try: 1/0
d[1,2] = 3
d[1,2,3] = 4
L = list(d)
- L.sort(key=lambda x: x if isinstance(x, tuple) else ())
+ L.sort(key=lambda x: (type(x).__name__, x))
self.assertEqual(str(L), '[1, (1,), (1, 2), (1, 2, 3)]')
def test_atoms(self):
import array
from binascii import unhexlify
import hashlib
+import importlib
import itertools
import os
import sys
def _conditional_import_module(self, module_name):
"""Import a module and return a reference to it or None on failure."""
try:
- exec('import '+module_name)
- except ImportError as error:
+ return importlib.import_module(module_name)
+ except ModuleNotFoundError as error:
if self._warn_on_extension_import:
warnings.warn('Did a C extension fail to compile? %s' % error)
- return locals().get(module_name)
+ return None
def __init__(self, *args, **kwargs):
algorithms = set()
constructors = self.constructors_to_test.values()
return itertools.chain.from_iterable(constructors)
+ @support.refcount_test
+ @unittest.skipIf(c_hashlib is None, 'Require _hashlib module')
+ def test_refleaks_in_hash___init__(self):
+ gettotalrefcount = support.get_attribute(sys, 'gettotalrefcount')
+ sha1_hash = c_hashlib.new('sha1')
+ refs_before = gettotalrefcount()
+ for i in range(100):
+ sha1_hash.__init__('sha1')
+ self.assertAlmostEqual(gettotalrefcount() - refs_before, 0, delta=10)
+
def test_hash_array(self):
a = array.array("b", range(10))
for cons in self.hash_constructors:
try:
import _md5
except ImportError:
- pass
+ self.skipTest("_md5 module not available")
# This forces an ImportError for "import _md5" statements
sys.modules['_md5'] = None
# clear the cache
import ssl
support.requires('network')
with support.transient_internet('self-signed.pythontest.net'):
- context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+ context = ssl.SSLContext(ssl.PROTOCOL_TLS)
context.verify_mode = ssl.CERT_REQUIRED
context.load_verify_locations(CERT_selfsigned_pythontestdotnet)
h = client.HTTPSConnection('self-signed.pythontest.net', 443, context=context)
import ssl
support.requires('network')
with support.transient_internet('self-signed.pythontest.net'):
- context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+ context = ssl.SSLContext(ssl.PROTOCOL_TLS)
context.verify_mode = ssl.CERT_REQUIRED
context.load_verify_locations(CERT_localhost)
h = client.HTTPSConnection('self-signed.pythontest.net', 443, context=context)
# The (valid) cert validates the HTTP hostname
import ssl
server = self.make_server(CERT_localhost)
- context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+ context = ssl.SSLContext(ssl.PROTOCOL_TLS)
context.verify_mode = ssl.CERT_REQUIRED
context.load_verify_locations(CERT_localhost)
h = client.HTTPSConnection('localhost', server.port, context=context)
# The (valid) cert doesn't validate the HTTP hostname
import ssl
server = self.make_server(CERT_fakehostname)
- context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+ context = ssl.SSLContext(ssl.PROTOCOL_TLS)
context.verify_mode = ssl.CERT_REQUIRED
context.check_hostname = True
context.load_verify_locations(CERT_fakehostname)
# Built-in modules cannot be a package.
test_package = None
- # Built-in modules cannobt be in a package.
+ # Built-in modules cannot be in a package.
test_module_in_package = None
# Built-in modules cannot be a package.
foo.__dict__['d'] = 1
self.assertEqual(inspect.getattr_static(foo, 'd'), 1)
- # if the descriptor is a data-desciptor we should return the
+ # if the descriptor is a data-descriptor we should return the
# descriptor
descriptor.__set__ = lambda s, i, v: None
self.assertEqual(inspect.getattr_static(foo, 'd'), Foo.__dict__['d'])
('c', 1, ..., 'keyword_only')),
'spam'))
+ class Spam:
+ def test(self: 'anno', x):
+ pass
+
+ g = partialmethod(test, 1)
+
+ self.assertEqual(self.signature(Spam.g),
+ ((('self', ..., 'anno', 'positional_or_keyword'),),
+ ...))
+
def test_signature_on_fake_partialmethod(self):
def foo(a): pass
foo._partialmethod = 'spam'
from itertools import cycle, count
from test import support
from test.support.script_helper import assert_python_ok, run_python_until_end
+from test.support import FakePath
import codecs
import io # C implementation of io
self.read_ops(f, True)
def test_large_file_ops(self):
- # On Windows and Mac OSX this test comsumes large resources; It takes
- # a long time to build the >2GB file and takes >2GB of disk space
+ # On Windows and Mac OSX this test consumes large resources; It takes
+ # a long time to build the >2 GiB file and takes >2 GiB of disk space
# therefore the resource must be enabled to run this test.
if sys.platform[:3] == 'win' or sys.platform == 'darwin':
support.requires(
self.assertRaises(ValueError, f.flush)
def test_RawIOBase_read(self):
- # Exercise the default RawIOBase.read() implementation (which calls
- # readinto() internally).
+ # Exercise the default limited RawIOBase.read(n) implementation (which
+ # calls readinto() internally).
rawio = self.MockRawIOWithoutRead((b"abc", b"d", None, b"efg", None))
self.assertEqual(rawio.read(2), b"ab")
self.assertEqual(rawio.read(2), b"c")
self.assertEqual(bytes(buffer), b"12345")
def test_fspath_support(self):
- class PathLike:
- def __init__(self, path):
- self.path = path
-
- def __fspath__(self):
- return self.path
-
def check_path_succeeds(path):
with self.open(path, "w") as f:
f.write("egg\n")
with self.open(path, "r") as f:
self.assertEqual(f.read(), "egg\n")
- check_path_succeeds(PathLike(support.TESTFN))
- check_path_succeeds(PathLike(support.TESTFN.encode('utf-8')))
+ check_path_succeeds(FakePath(support.TESTFN))
+ check_path_succeeds(FakePath(support.TESTFN.encode('utf-8')))
+
+ with self.open(support.TESTFN, "w") as f:
+ bad_path = FakePath(f.fileno())
+ with self.assertRaises(TypeError):
+ self.open(bad_path, 'w')
- bad_path = PathLike(TypeError)
+ bad_path = FakePath(None)
with self.assertRaises(TypeError):
self.open(bad_path, 'w')
+ bad_path = FakePath(FloatingPointError)
+ with self.assertRaises(FloatingPointError):
+ self.open(bad_path, 'w')
+
# ensure that refcounting is correct with some error conditions
with self.assertRaisesRegex(ValueError, 'read/write/append mode'):
- self.open(PathLike(support.TESTFN), 'rwxa')
+ self.open(FakePath(support.TESTFN), 'rwxa')
+
+ def test_RawIOBase_readall(self):
+ # Exercise the default unlimited RawIOBase.read() and readall()
+ # implementations.
+ rawio = self.MockRawIOWithoutRead((b"abc", b"d", b"efg"))
+ self.assertEqual(rawio.read(), b"abcdefg")
+ rawio = self.MockRawIOWithoutRead((b"abc", b"d", b"efg"))
+ self.assertEqual(rawio.readall(), b"abcdefg")
+
+ def test_BufferedIOBase_readinto(self):
+ # Exercise the default BufferedIOBase.readinto() and readinto1()
+ # implementations (which call read() or read1() internally).
+ class Reader(self.BufferedIOBase):
+ def __init__(self, avail):
+ self.avail = avail
+ def read(self, size):
+ result = self.avail[:size]
+ self.avail = self.avail[size:]
+ return result
+ def read1(self, size):
+ """Returns no more than 5 bytes at once"""
+ return self.read(min(size, 5))
+ tests = (
+ # (test method, total data available, read buffer size, expected
+ # read size)
+ ("readinto", 10, 5, 5),
+ ("readinto", 10, 6, 6), # More than read1() can return
+ ("readinto", 5, 6, 5), # Buffer larger than total available
+ ("readinto", 6, 7, 6),
+ ("readinto", 10, 0, 0), # Empty buffer
+ ("readinto1", 10, 5, 5), # Result limited to single read1() call
+ ("readinto1", 10, 6, 5), # Buffer larger than read1() can return
+ ("readinto1", 5, 6, 5), # Buffer larger than total available
+ ("readinto1", 6, 7, 5),
+ ("readinto1", 10, 0, 0), # Empty buffer
+ )
+ UNUSED_BYTE = 0x81
+ for test in tests:
+ with self.subTest(test):
+ method, avail, request, result = test
+ reader = Reader(bytes(range(avail)))
+ buffer = bytearray((UNUSED_BYTE,) * request)
+ method = getattr(reader, method)
+ self.assertEqual(method(buffer), result)
+ self.assertEqual(len(buffer), request)
+ self.assertSequenceEqual(buffer[:result], range(result))
+ unused = (UNUSED_BYTE,) * (request - result)
+ self.assertSequenceEqual(buffer[result:], unused)
+ self.assertEqual(len(reader.avail), avail - result)
class CIOTest(IOTest):
with self.open(support.TESTFN, "rb", buffering=0) as f:
self.assertEqual(f.read(), b"abc")
+ def test_truncate_after_write(self):
+ # Ensure that truncate preserves the file position after
+ # writes longer than the buffer size.
+ # Issue: https://bugs.python.org/issue32228
+ with self.open(support.TESTFN, "wb") as f:
+ # Fill with some buffer
+ f.write(b'\x00' * 10000)
+ buffer_sizes = [8192, 4096, 200]
+ for buffer_size in buffer_sizes:
+ with self.open(support.TESTFN, "r+b", buffering=buffer_size) as f:
+ f.write(b'\x00' * (buffer_size + 1))
+ # After write write_pos and write_end are set to 0
+ f.read(1)
+ # read operation makes sure that pos != raw_pos
+ f.truncate()
+ self.assertEqual(f.tell(), buffer_size + 2)
+
@unittest.skipUnless(threading, 'Threading required for this test.')
@support.requires_resource('cpu')
def test_threads(self):
... # first_true([a,b], x, f) --> a if f(a) else b if f(b) else x
... return next(filter(pred, iterable), default)
+>>> def nth_combination(iterable, r, index):
+... 'Equivalent to list(combinations(iterable, r))[index]'
+... pool = tuple(iterable)
+... n = len(pool)
+... if r < 0 or r > n:
+... raise ValueError
+... c = 1
+... k = min(r, n-r)
+... for i in range(1, k+1):
+... c = c * (n - k + i) // i
+... if index < 0:
+... index += c
+... if index < 0 or index >= c:
+... raise IndexError
+... result = []
+... while r:
+... c, n, r = c*r//n, n-1, r-1
+... while index >= c:
+... index -= c
+... c, n = c*(n-r)//n, n-1
+... result.append(pool[-1-n])
+... return tuple(result)
+
+
This is not part of the examples but it tests to make sure the definitions
perform as purported.
>>> first_true('ABC0DEF1', '9', str.isdigit)
'0'
+>>> population = 'ABCDEFGH'
+>>> for r in range(len(population) + 1):
+... seq = list(combinations(population, r))
+... for i in range(len(seq)):
+... assert nth_combination(population, r, i) == seq[i]
+... for i in range(-len(seq), 0):
+... assert nth_combination(population, r, i) == seq[i]
+
+
"""
__test__ = {'libreftest' : libreftest}
except (ImportError, AttributeError):
pass
- # On Windows and Mac OSX this test comsumes large resources; It
- # takes a long time to build the >2GB file and takes >2GB of disk
+ # On Windows and Mac OSX this test consumes large resources; It
+ # takes a long time to build the >2 GiB file and takes >2 GiB of disk
# space therefore the resource must be enabled to run this test.
# If not, nothing after this line stanza will be executed.
if sys.platform[:3] == 'win' or sys.platform == 'darwin':
"""Test builtin levels and their inheritance."""
def test_flat(self):
- #Logging levels in a flat logger namespace.
+ # Logging levels in a flat logger namespace.
m = self.next_message
ERR = logging.getLogger("ERR")
])
def test_nested_inherited(self):
- #Logging levels in a nested namespace, inherited from parent loggers.
+ # Logging levels in a nested namespace, inherited from parent loggers.
m = self.next_message
INF = logging.getLogger("INF")
def test_encoding_cyrillic_unicode(self):
log = logging.getLogger("test")
- #Get a message in Unicode: Do svidanya in Cyrillic (meaning goodbye)
+ # Get a message in Unicode: Do svidanya in Cyrillic (meaning goodbye)
message = '\u0434\u043e \u0441\u0432\u0438\u0434\u0430\u043d\u0438\u044f'
- #Ensure it's written in a Cyrillic encoding
+ # Ensure it's written in a Cyrillic encoding
writer_class = codecs.getwriter('cp1251')
writer_class.encoding = 'cp1251'
stream = io.BytesIO()
handler.close()
# check we wrote exactly those bytes, ignoring trailing \n etc
s = stream.getvalue()
- #Compare against what the data should be when encoded in CP-1251
+ # Compare against what the data should be when encoded in CP-1251
self.assertEqual(s, b'\xe4\xee \xf1\xe2\xe8\xe4\xe0\xed\xe8\xff\n')
h.close()
self.assertGreater(s.find("UserWarning: I'm warning you...\n"), 0)
- #See if an explicit file uses the original implementation
+ # See if an explicit file uses the original implementation
a_file = io.StringIO()
warnings.showwarning("Explicit", UserWarning, "dummy.py", 42,
a_file, "Dummy line")
},
}
- #As config1 but with a misspelt level on a handler
+ # As config1 but with a misspelt level on a handler
config2a = {
'version': 1,
'formatters': {
}
- #As config1 but with a misspelt level on a logger
+ # As config1 but with a misspelt level on a logger
config2b = {
'version': 1,
'formatters': {
},
}
- #config 7 does not define compiler.parser but defines compiler.lexer
- #so compiler.parser should be disabled after applying it
+ # config 7 does not define compiler.parser but defines compiler.lexer
+ # so compiler.parser should be disabled after applying it
config7 = {
'version': 1,
'formatters': {
},
}
- #As config1 but with a filter added
+ # As config1 but with a filter added
config10 = {
'version': 1,
'formatters': {
},
}
- #As config1 but using cfg:// references
+ # As config1 but using cfg:// references
config11 = {
'version': 1,
'true_formatters': {
},
}
- #As config11 but missing the version key
+ # As config11 but missing the version key
config12 = {
'true_formatters': {
'form1' : {
},
}
- #As config11 but using an unsupported version
+ # As config11 but using an unsupported version
config13 = {
'version': 2,
'true_formatters': {
# Original logger output is empty.
self.assert_log_lines([])
- #Same as test_config_7_ok but don't disable old loggers.
+ # Same as test_config_7_ok but don't disable old loggers.
def test_config_8_ok(self):
with support.captured_stdout() as output:
self.apply_config(self.config1)
with support.captured_stdout() as output:
self.apply_config(self.config9)
logger = logging.getLogger("compiler.parser")
- #Nothing will be output since both handler and logger are set to WARNING
+ # Nothing will be output since both handler and logger are set to WARNING
logger.info(self.next_message())
self.assert_log_lines([], stream=output)
self.apply_config(self.config9a)
- #Nothing will be output since both handler is still set to WARNING
+ # Nothing will be output since handler is still set to WARNING
logger.info(self.next_message())
self.assert_log_lines([], stream=output)
self.apply_config(self.config9b)
- #Message should now be output
+ # Message should now be output
logger.info(self.next_message())
self.assert_log_lines([
('INFO', '3'),
logger = logging.getLogger("compiler.parser")
logger.warning(self.next_message())
logger = logging.getLogger('compiler')
- #Not output, because filtered
+ # Not output, because filtered
logger.warning(self.next_message())
logger = logging.getLogger('compiler.lexer')
- #Not output, because filtered
+ # Not output, because filtered
logger.warning(self.next_message())
logger = logging.getLogger("compiler.parser.codegen")
- #Output, as not filtered
+ # Output, as not filtered
logger.error(self.next_message())
self.assert_log_lines([
('WARNING', '1'),
logger = logging.getLogger("compiler.parser")
logger.warning(self.next_message())
logger = logging.getLogger('compiler')
- #Not output, because filtered
+ # Not output, because filtered
logger.warning(self.next_message())
logger = logging.getLogger('compiler.lexer')
- #Not output, because filtered
+ # Not output, because filtered
logger.warning(self.next_message())
logger = logging.getLogger("compiler.parser.codegen")
- #Output, as not filtered
+ # Output, as not filtered
logger.error(self.next_message())
self.assert_log_lines([
('WARNING', '1'),
break
msg = 'No rotated files found, went back %d seconds' % GO_BACK
if not found:
- #print additional diagnostics
+ # print additional diagnostics
dn, fn = os.path.split(self.fn)
files = [f for f in os.listdir(dn) if f.startswith(fn)]
print('Test time: %s' % now.strftime("%Y-%m-%d %H-%M-%S"), file=sys.stderr)
import pathlib
import pickle
import random
+import sys
+from test import support
import unittest
from test.support import (
with self.assertRaises(TypeError):
pickle.dumps(LZMADecompressor(), proto)
+ @support.refcount_test
+ def test_refleaks_in_decompressor___init__(self):
+ gettotalrefcount = support.get_attribute(sys, 'gettotalrefcount')
+ lzd = LZMADecompressor()
+ refs_before = gettotalrefcount()
+ for i in range(100):
+ lzd.__init__()
+ self.assertAlmostEqual(gettotalrefcount() - refs_before, 0, delta=10)
+
class CompressDecompressFunctionTestCase(unittest.TestCase):
class IsCloseTests(unittest.TestCase):
- isclose = math.isclose # sublcasses should override this
+ isclose = math.isclose # subclasses should override this
def assertIsClose(self, a, b, *args, **kwargs):
self.assertTrue(self.isclose(a, b, *args, **kwargs),
check = self.check_sizeof
self.assertEqual(object.__sizeof__(io.BytesIO()), basesize)
check(io.BytesIO(), basesize )
- check(io.BytesIO(b'a' * 1000), basesize + sys.getsizeof(b'a' * 1000))
+ n = 1000 # use a variable to prevent constant folding
+ check(io.BytesIO(b'a' * n), basesize + sys.getsizeof(b'a' * n))
# Various tests of copy-on-write behaviour for BytesIO.
import sys
import unittest
import warnings
-from test.support import TestFailed
+from test.support import TestFailed, FakePath
from test import support, test_genericpath
from tempfile import TemporaryFile
path = ntpath
- class PathLike:
- def __init__(self, path=''):
- self.path = path
- def __fspath__(self):
- if isinstance(self.path, BaseException):
- raise self.path
- else:
- return self.path
-
def setUp(self):
self.file_name = support.TESTFN.lower()
- self.file_path = self.PathLike(support.TESTFN)
+ self.file_path = FakePath(support.TESTFN)
self.addCleanup(support.unlink, self.file_name)
with open(self.file_name, 'xb', 0) as file:
file.write(b"test_ntpath.PathLikeTests")
self.assertPathEqual(self.path.isabs)
def test_path_join(self):
- self.assertEqual(self.path.join('a', self.PathLike('b'), 'c'),
+ self.assertEqual(self.path.join('a', FakePath('b'), 'c'),
self.path.join('a', 'b', 'c'))
def test_path_split(self):
self.assertEqual(repr(od),
"OrderedDict([('a', None), ('b', None), ('c', None), ('x', ...)])")
+ def test_repr_recursive_values(self):
+ OrderedDict = self.OrderedDict
+ od = OrderedDict()
+ od[42] = od.values()
+ r = repr(od)
+ # Cannot perform a stronger test, as the contents of the repr
+ # are implementation-dependent. All we can say is that we
+ # want a str result, not an exception of any sort.
+ self.assertIsInstance(r, str)
+ od[42] = od.items()
+ r = repr(od)
+ # Again.
+ self.assertIsInstance(r, str)
+
def test_setdefault(self):
OrderedDict = self.OrderedDict
pairs = [('c', 1), ('b', 2), ('a', 3), ('d', 4), ('e', 5), ('f', 6)]
INT_MAX = PY_SSIZE_T_MAX = sys.maxsize
from test.support.script_helper import assert_python_ok
-from test.support import unix_shell
+from test.support import unix_shell, FakePath
root_in_posix = False
return unittest.skipUnless(hasattr(os, name), 'requires os.%s' % name)
-class _PathLike(os.PathLike):
-
- def __init__(self, path=""):
- self.path = path
-
- def __str__(self):
- return str(self.path)
-
- def __fspath__(self):
- if isinstance(self.path, BaseException):
- raise self.path
- else:
- return self.path
-
-
def create_file(filename, content=b'content'):
with open(filename, "xb", 0) as fp:
fp.write(content)
dirs.remove('SUB1')
self.assertEqual(len(all), 2)
- self.assertEqual(all[0],
- (str(walk_path), ["SUB2"], ["tmp1"]))
+ self.assertEqual(all[0], (self.walk_path, ["SUB2"], ["tmp1"]))
all[1][-1].sort()
all[1][1].sort()
self.assertEqual(all[1], self.sub2_tree)
def test_file_like_path(self):
- self.test_walk_prune(_PathLike(self.walk_path))
+ self.test_walk_prune(FakePath(self.walk_path))
def test_walk_bottom_up(self):
# Walk bottom-up.
finally:
os.chdir(orig_dir)
+ @unittest.skipUnless(os.path.lexists(r'C:\Users\All Users')
+ and os.path.exists(r'C:\ProgramData'),
+ 'Test directories not found')
+ def test_29248(self):
+ # os.symlink() calls CreateSymbolicLink, which creates
+ # the reparse data buffer with the print name stored
+ # first, so the offset is always 0. CreateSymbolicLink
+ # stores the "PrintName" DOS path (e.g. "C:\") first,
+ # with an offset of 0, followed by the "SubstituteName"
+ # NT path (e.g. "\??\C:\"). The "All Users" link, on
+ # the other hand, seems to have been created manually
+ # with an inverted order.
+ target = os.readlink(r'C:\Users\All Users')
+ self.assertTrue(os.path.samefile(target, r'C:\ProgramData'))
+
+ def test_buffer_overflow(self):
+ # Older versions would have a buffer overflow when detecting
+ # whether a link source was a directory. This test ensures we
+ # no longer crash, but does not otherwise validate the behavior
+ segment = 'X' * 27
+ path = os.path.join(*[segment] * 10)
+ test_cases = [
+ # overflow with absolute src
+ ('\\' + path, segment),
+ # overflow dest with relative src
+ (segment, path),
+ # overflow when joining src
+ (path[:180], path[:180]),
+ ]
+ for src, dest in test_cases:
+ try:
+ os.symlink(src, dest)
+ except FileNotFoundError:
+ pass
+ else:
+ try:
+ os.remove(dest)
+ except OSError:
+ pass
+ # Also test with bytes, since that is a separate code path.
+ try:
+ os.symlink(os.fsencode(src), os.fsencode(dest))
+ except FileNotFoundError:
+ pass
+ else:
+ try:
+ os.remove(dest)
+ except OSError:
+ pass
+
@unittest.skipUnless(sys.platform == "win32", "Win32 specific tests")
class Win32JunctionTests(unittest.TestCase):
def test_waitpid(self):
args = [sys.executable, '-c', 'pass']
# Add an implicit test for PyUnicode_FSConverter().
- pid = os.spawnv(os.P_NOWAIT, _PathLike(args[0]), args)
+ pid = os.spawnv(os.P_NOWAIT, FakePath(args[0]), args)
status = os.waitpid(pid, 0)
self.assertEqual(status, (pid, 0))
bytes_fspath = bytes_filename = None
else:
bytes_filename = support.TESTFN.encode('ascii')
- bytes_fspath = _PathLike(bytes_filename)
- fd = os.open(_PathLike(str_filename), os.O_WRONLY|os.O_CREAT)
+ bytes_fspath = FakePath(bytes_filename)
+ fd = os.open(FakePath(str_filename), os.O_WRONLY|os.O_CREAT)
self.addCleanup(support.unlink, support.TESTFN)
self.addCleanup(os.close, fd)
- int_fspath = _PathLike(fd)
- str_fspath = _PathLike(str_filename)
+ int_fspath = FakePath(fd)
+ str_fspath = FakePath(str_filename)
for name, allow_fd, extra_args, cleanup_fn in self.functions:
with self.subTest(name=name):
def test_fsencode_fsdecode(self):
for p in "path/like/object", b"path/like/object":
- pathlike = _PathLike(p)
+ pathlike = FakePath(p)
self.assertEqual(p, self.fspath(pathlike))
self.assertEqual(b"path/like/object", os.fsencode(pathlike))
self.assertEqual("path/like/object", os.fsdecode(pathlike))
def test_pathlike(self):
- self.assertEqual('#feelthegil', self.fspath(_PathLike('#feelthegil')))
- self.assertTrue(issubclass(_PathLike, os.PathLike))
- self.assertTrue(isinstance(_PathLike(), os.PathLike))
+ self.assertEqual('#feelthegil', self.fspath(FakePath('#feelthegil')))
+ self.assertTrue(issubclass(FakePath, os.PathLike))
+ self.assertTrue(isinstance(FakePath('x'), os.PathLike))
def test_garbage_in_exception_out(self):
vapor = type('blah', (), {})
def test_bad_pathlike(self):
# __fspath__ returns a value other than str or bytes.
- self.assertRaises(TypeError, self.fspath, _PathLike(42))
+ self.assertRaises(TypeError, self.fspath, FakePath(42))
# __fspath__ attribute that is not callable.
c = type('foo', (), {})
c.__fspath__ = 1
self.assertRaises(TypeError, self.fspath, c())
# __fspath__ raises an exception.
self.assertRaises(ZeroDivisionError, self.fspath,
- _PathLike(ZeroDivisionError()))
+ FakePath(ZeroDivisionError()))
# Only test if the C version is provided, otherwise TestPEP519 already tested
# the pure Python implementation.
self.roundtrip(parser.expr, s)
def test_flags_passed(self):
- # The unicode literals flags has to be passed from the paser to AST
+ # The unicode literals flags has to be passed from the parser to AST
# generation.
suite = parser.suite("from __future__ import unicode_literals; x = ''")
code = suite.compile()
from unittest import mock
from test import support
-TESTFN = support.TESTFN
+from test.support import TESTFN, FakePath
try:
import grp, pwd
P = self.cls
p = P('a')
self.assertIsInstance(p, P)
- class PathLike:
- def __fspath__(self):
- return "a/b/c"
P('a', 'b', 'c')
P('/a', 'b', 'c')
P('a/b/c')
P('/a/b/c')
- P(PathLike())
+ P(FakePath("a/b/c"))
self.assertEqual(P(P('a')), P('a'))
self.assertEqual(P(P('a'), 'b'), P('a/b'))
self.assertEqual(P(P('a'), P('b')), P('a/b'))
- self.assertEqual(P(P('a'), P('b'), P('c')), P(PathLike()))
+ self.assertEqual(P(P('a'), P('b'), P('c')), P(FakePath("a/b/c")))
def _check_str_subclass(self, *args):
# Issue #21127: it should be possible to construct a PurePath object
finished
"""
+def test_pdb_next_command_for_coroutine():
+ """Testing skip unwindng stack on yield for coroutines for "next" command
+
+ >>> import asyncio
+
+ >>> async def test_coro():
+ ... await asyncio.sleep(0)
+ ... await asyncio.sleep(0)
+ ... await asyncio.sleep(0)
+
+ >>> async def test_main():
+ ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace()
+ ... await test_coro()
+
+ >>> def test_function():
+ ... loop = asyncio.new_event_loop()
+ ... loop.run_until_complete(test_main())
+ ... loop.close()
+ ... print("finished")
+
+ >>> with PdbTestInput(['step',
+ ... 'step',
+ ... 'next',
+ ... 'next',
+ ... 'next',
+ ... 'step',
+ ... 'continue']):
+ ... test_function()
+ > <doctest test.test_pdb.test_pdb_next_command_for_coroutine[2]>(3)test_main()
+ -> await test_coro()
+ (Pdb) step
+ --Call--
+ > <doctest test.test_pdb.test_pdb_next_command_for_coroutine[1]>(1)test_coro()
+ -> async def test_coro():
+ (Pdb) step
+ > <doctest test.test_pdb.test_pdb_next_command_for_coroutine[1]>(2)test_coro()
+ -> await asyncio.sleep(0)
+ (Pdb) next
+ > <doctest test.test_pdb.test_pdb_next_command_for_coroutine[1]>(3)test_coro()
+ -> await asyncio.sleep(0)
+ (Pdb) next
+ > <doctest test.test_pdb.test_pdb_next_command_for_coroutine[1]>(4)test_coro()
+ -> await asyncio.sleep(0)
+ (Pdb) next
+ Internal StopIteration
+ > <doctest test.test_pdb.test_pdb_next_command_for_coroutine[2]>(3)test_main()
+ -> await test_coro()
+ (Pdb) step
+ --Return--
+ > <doctest test.test_pdb.test_pdb_next_command_for_coroutine[2]>(3)test_main()->None
+ -> await test_coro()
+ (Pdb) continue
+ finished
+ """
+
+def test_pdb_next_command_for_asyncgen():
+ """Testing skip unwindng stack on yield for coroutines for "next" command
+
+ >>> import asyncio
+
+ >>> async def agen():
+ ... yield 1
+ ... await asyncio.sleep(0)
+ ... yield 2
+
+ >>> async def test_coro():
+ ... async for x in agen():
+ ... print(x)
+
+ >>> async def test_main():
+ ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace()
+ ... await test_coro()
+
+ >>> def test_function():
+ ... loop = asyncio.new_event_loop()
+ ... loop.run_until_complete(test_main())
+ ... loop.close()
+ ... print("finished")
+
+ >>> with PdbTestInput(['step',
+ ... 'step',
+ ... 'next',
+ ... 'next',
+ ... 'step',
+ ... 'next',
+ ... 'continue']):
+ ... test_function()
+ > <doctest test.test_pdb.test_pdb_next_command_for_asyncgen[3]>(3)test_main()
+ -> await test_coro()
+ (Pdb) step
+ --Call--
+ > <doctest test.test_pdb.test_pdb_next_command_for_asyncgen[2]>(1)test_coro()
+ -> async def test_coro():
+ (Pdb) step
+ > <doctest test.test_pdb.test_pdb_next_command_for_asyncgen[2]>(2)test_coro()
+ -> async for x in agen():
+ (Pdb) next
+ > <doctest test.test_pdb.test_pdb_next_command_for_asyncgen[2]>(3)test_coro()
+ -> print(x)
+ (Pdb) next
+ 1
+ > <doctest test.test_pdb.test_pdb_next_command_for_asyncgen[2]>(2)test_coro()
+ -> async for x in agen():
+ (Pdb) step
+ --Call--
+ > <doctest test.test_pdb.test_pdb_next_command_for_asyncgen[1]>(2)agen()
+ -> yield 1
+ (Pdb) next
+ > <doctest test.test_pdb.test_pdb_next_command_for_asyncgen[1]>(3)agen()
+ -> await asyncio.sleep(0)
+ (Pdb) continue
+ 2
+ finished
+ """
+
def test_pdb_return_command_for_generator():
"""Testing no unwindng stack on yield for generators
for "return" command
finished
"""
+def test_pdb_return_command_for_coroutine():
+ """Testing no unwindng stack on yield for coroutines for "return" command
+
+ >>> import asyncio
+
+ >>> async def test_coro():
+ ... await asyncio.sleep(0)
+ ... await asyncio.sleep(0)
+ ... await asyncio.sleep(0)
+
+ >>> async def test_main():
+ ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace()
+ ... await test_coro()
+
+ >>> def test_function():
+ ... loop = asyncio.new_event_loop()
+ ... loop.run_until_complete(test_main())
+ ... loop.close()
+ ... print("finished")
+
+ >>> with PdbTestInput(['step',
+ ... 'step',
+ ... 'next',
+ ... 'continue']):
+ ... test_function()
+ > <doctest test.test_pdb.test_pdb_return_command_for_coroutine[2]>(3)test_main()
+ -> await test_coro()
+ (Pdb) step
+ --Call--
+ > <doctest test.test_pdb.test_pdb_return_command_for_coroutine[1]>(1)test_coro()
+ -> async def test_coro():
+ (Pdb) step
+ > <doctest test.test_pdb.test_pdb_return_command_for_coroutine[1]>(2)test_coro()
+ -> await asyncio.sleep(0)
+ (Pdb) next
+ > <doctest test.test_pdb.test_pdb_return_command_for_coroutine[1]>(3)test_coro()
+ -> await asyncio.sleep(0)
+ (Pdb) continue
+ finished
+ """
+
def test_pdb_until_command_for_generator():
"""Testing no unwindng stack on yield for generators
for "until" command if target breakpoing is not reached
finished
"""
+def test_pdb_until_command_for_coroutine():
+ """Testing no unwindng stack for coroutines
+ for "until" command if target breakpoing is not reached
+
+ >>> import asyncio
+
+ >>> async def test_coro():
+ ... print(0)
+ ... await asyncio.sleep(0)
+ ... print(1)
+ ... await asyncio.sleep(0)
+ ... print(2)
+ ... await asyncio.sleep(0)
+ ... print(3)
+
+ >>> async def test_main():
+ ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace()
+ ... await test_coro()
+
+ >>> def test_function():
+ ... loop = asyncio.new_event_loop()
+ ... loop.run_until_complete(test_main())
+ ... loop.close()
+ ... print("finished")
+
+ >>> with PdbTestInput(['step',
+ ... 'until 8',
+ ... 'continue']):
+ ... test_function()
+ > <doctest test.test_pdb.test_pdb_until_command_for_coroutine[2]>(3)test_main()
+ -> await test_coro()
+ (Pdb) step
+ --Call--
+ > <doctest test.test_pdb.test_pdb_until_command_for_coroutine[1]>(1)test_coro()
+ -> async def test_coro():
+ (Pdb) until 8
+ 0
+ 1
+ 2
+ > <doctest test.test_pdb.test_pdb_until_command_for_coroutine[1]>(8)test_coro()
+ -> print(3)
+ (Pdb) continue
+ 3
+ finished
+ """
+
def test_pdb_next_command_in_generator_for_loop():
"""The next command on returning from a generator controlled by a for loop.
self.assertInBytecode(code, 'LOAD_CONST', 'b')
# Verify that large sequences do not result from folding
- code = compile('a="x"*1000', '', 'single')
+ code = compile('a="x"*10000', '', 'single')
+ self.assertInBytecode(code, 'LOAD_CONST', 10000)
+ self.assertNotIn("x"*10000, code.co_consts)
+ code = compile('a=1<<1000', '', 'single')
self.assertInBytecode(code, 'LOAD_CONST', 1000)
+ self.assertNotIn(1<<1000, code.co_consts)
+ code = compile('a=2**1000', '', 'single')
+ self.assertInBytecode(code, 'LOAD_CONST', 1000)
+ self.assertNotIn(2**1000, code.co_consts)
def test_binary_subscr_on_unicode(self):
# valid code get optimized
'second': [1, 2],
'third': [3, 4],
})
- self.assertIsNot(pl2['first'], pl2['second'])
+ if fmt != plistlib.FMT_BINARY:
+ self.assertIsNot(pl2['first'], pl2['second'])
def test_list_members(self):
pl = {
def test_rpop(self):
self.assertOK(self.client.rpop('foo'))
- def test_apop(self):
+ def test_apop_normal(self):
self.assertOK(self.client.apop('foo', 'dummypassword'))
+ def test_apop_REDOS(self):
+ # Replace welcome with very long evil welcome.
+ # NB The upper bound on welcome length is currently 2048.
+ # At this length, evil input makes each apop call take
+ # on the order of milliseconds instead of microseconds.
+ evil_welcome = b'+OK' + (b'<' * 1000000)
+ with test_support.swap_attr(self.client, 'welcome', evil_welcome):
+ # The evil welcome is invalid, so apop should throw.
+ self.assertRaises(poplib.error_proto, self.client.apop, 'a', 'kb')
+
def test_top(self):
expected = (b'+OK 116 bytes',
[b'From: postmaster@python.org', b'Content-Type: text/plain',
@requires_ssl
def test_stls_context(self):
expected = b'+OK Begin TLS negotiation'
- ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+ ctx = ssl.SSLContext(ssl.PROTOCOL_TLS)
ctx.load_verify_locations(CAFILE)
ctx.verify_mode = ssl.CERT_REQUIRED
ctx.check_hostname = True
self.assertIn('POP3_SSL', poplib.__all__)
def test_context(self):
- ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+ ctx = ssl.SSLContext(ssl.PROTOCOL_TLS)
self.assertRaises(ValueError, poplib.POP3_SSL, self.server.host,
self.server.port, keyfile=CERTFILE, context=ctx)
self.assertRaises(ValueError, poplib.POP3_SSL, self.server.host,
finally:
os.close(fd)
+ # issue31106 - posix_fallocate() does not set error in errno.
+ @unittest.skipUnless(hasattr(posix, 'posix_fallocate'),
+ "test needs posix.posix_fallocate()")
+ def test_posix_fallocate_errno(self):
+ try:
+ posix.posix_fallocate(-42, 0, 10)
+ except OSError as inst:
+ if inst.errno != errno.EBADF:
+ raise
+
@unittest.skipUnless(hasattr(posix, 'posix_fadvise'),
"test needs posix.posix_fadvise()")
def test_posix_fadvise(self):
finally:
os.close(fd)
+ @unittest.skipUnless(hasattr(posix, 'posix_fadvise'),
+ "test needs posix.posix_fadvise()")
+ def test_posix_fadvise_errno(self):
+ try:
+ posix.posix_fadvise(-42, 0, 0, posix.POSIX_FADV_WILLNEED)
+ except OSError as inst:
+ if inst.errno != errno.EBADF:
+ raise
+
@unittest.skipUnless(os.utime in os.supports_fd, "test needs fd support in os.utime")
def test_utime_with_fd(self):
now = time.time()
import warnings
from posixpath import realpath, abspath, dirname, basename
from test import support, test_genericpath
+from test.support import FakePath
try:
import posix
path = posixpath
- class PathLike:
- def __init__(self, path=''):
- self.path = path
- def __fspath__(self):
- if isinstance(self.path, BaseException):
- raise self.path
- else:
- return self.path
-
def setUp(self):
self.file_name = support.TESTFN.lower()
- self.file_path = self.PathLike(support.TESTFN)
+ self.file_path = FakePath(support.TESTFN)
self.addCleanup(support.unlink, self.file_name)
with open(self.file_name, 'xb', 0) as file:
file.write(b"test_posixpath.PathLikeTests")
self.assertPathEqual(self.path.isabs)
def test_path_join(self):
- self.assertEqual(self.path.join('a', self.PathLike('b'), 'c'),
+ self.assertEqual(self.path.join('a', FakePath('b'), 'c'),
self.path.join('a', 'b', 'c'))
def test_path_split(self):
self.assertIn('print("Hello World", end=" ")', str(context.exception))
+ def test_string_with_leading_whitespace(self):
+ python2_print_str = '''if 1:
+ print "Hello World"
+ '''
+ with self.assertRaises(SyntaxError) as context:
+ exec(python2_print_str)
+
+ self.assertIn('print("Hello World")', str(context.exception))
+
+ # bpo-32685: Suggestions for print statement should be proper when
+ # it is in the same line as the header of a compound statement
+ # and/or followed by a semicolon
+ def test_string_with_semicolon(self):
+ python2_print_str = 'print p;'
+ with self.assertRaises(SyntaxError) as context:
+ exec(python2_print_str)
+
+ self.assertIn('print(p)', str(context.exception))
+
+ def test_string_in_loop_on_same_line(self):
+ python2_print_str = 'for i in s: print i'
+ with self.assertRaises(SyntaxError) as context:
+ exec(python2_print_str)
+
+ self.assertIn('print(i)', str(context.exception))
+
def test_stream_redirection_hint_for_py2_migration(self):
# Test correct hint produced for Py2 redirection syntax
with self.assertRaises(TypeError) as context:
import sys
import unittest
+from test import support
class PropertyBase(Exception):
pass
sub.__class__.spam.__doc__ = 'Spam'
self.assertEqual(sub.__class__.spam.__doc__, 'Spam')
+ @support.refcount_test
+ def test_refleaks_in___init__(self):
+ gettotalrefcount = support.get_attribute(sys, 'gettotalrefcount')
+ fake_prop = property('fget', 'fset', 'fdel', 'doc')
+ refs_before = gettotalrefcount()
+ for i in range(100):
+ fake_prop.__init__('fget', 'fset', 'fdel', 'doc')
+ self.assertAlmostEqual(gettotalrefcount() - refs_before, 0, delta=10)
+
+
# Issue 5890: subclasses of property do not preserve method __doc__ strings
class PropertySub(property):
"""This is a subclass of property"""
self.gen.seed()
state1 = self.gen.getstate()
time.sleep(0.1)
- self.gen.seed() # diffent seeds at different times
+ self.gen.seed() # different seeds at different times
state2 = self.gen.getstate()
self.assertNotEqual(state1, state2)
s = r"\1\1"
self.assertEqual(re.sub('(.)', s, 'x'), 'xx')
- self.assertEqual(re.sub('(.)', re.escape(s), 'x'), s)
+ self.assertEqual(re.sub('(.)', s.replace('\\', r'\\'), 'x'), s)
self.assertEqual(re.sub('(.)', lambda m: s, 'x'), s)
self.assertEqual(re.sub('(?P<a>x)', r'\g<a>\g<a>', 'xx'), 'xxxx')
"""
from contextlib import ExitStack
from errno import EIO
+import locale
import os
import selectors
import subprocess
self.assertIn(b"History length: 0\r\n", output)
def test_nonascii(self):
+ loc = locale.setlocale(locale.LC_CTYPE, None)
+ if loc in ('C', 'POSIX'):
+ # bpo-29240: On FreeBSD, if the LC_CTYPE locale is C or POSIX,
+ # writing and reading non-ASCII bytes into/from a TTY works, but
+ # readline or ncurses ignores non-ASCII bytes on read.
+ self.skipTest(f"the LC_CTYPE locale is {loc!r}")
+
try:
readline.add_history("\xEB\xEF")
except UnicodeEncodeError as err:
import os.path
import errno
import functools
+import pathlib
import subprocess
from contextlib import ExitStack
from shutil import (make_archive,
import tarfile
import zipfile
import warnings
+import pathlib
from test import support
-from test.support import TESTFN, check_warnings, captured_stdout
+from test.support import TESTFN, FakePath
TESTFN2 = TESTFN + "2"
self.assertNotIn('xxx', formats)
def check_unpack_archive(self, format):
+ self.check_unpack_archive_with_converter(format, lambda path: path)
+ self.check_unpack_archive_with_converter(format, pathlib.Path)
+ self.check_unpack_archive_with_converter(format, FakePath)
+
+ def check_unpack_archive_with_converter(self, format, converter):
root_dir, base_dir = self._create_files()
expected = rlistdir(root_dir)
expected.remove('outer')
'site-packages')
self.assertEqual(dirs[1], wanted)
elif os.sep == '/':
- # OS X non-framwework builds, Linux, FreeBSD, etc
+ # OS X non-framework builds, Linux, FreeBSD, etc
self.assertEqual(len(dirs), 1)
wanted = os.path.join('xoxo', 'lib',
'python%d.%d' % sys.version_info[:2],
def __init__(self, *args, **kw):
self._extra_features = []
+ self._addresses = {}
smtpd.SMTPServer.__init__(self, *args, **kw)
def handle_accepted(self, conn, addr):
decode_data=self._decode_data)
def process_message(self, peer, mailfrom, rcpttos, data):
- pass
+ self._addresses['from'] = mailfrom
+ self._addresses['tos'] = rcpttos
def add_feature(self, feature):
self._extra_features.append(feature)
self.assertRaises(UnicodeEncodeError, smtp.sendmail, 'Alice', 'Böb', '')
self.assertRaises(UnicodeEncodeError, smtp.mail, 'Älice')
+ def test_name_field_not_included_in_envelop_addresses(self):
+ smtp = smtplib.SMTP(
+ HOST, self.port, local_hostname='localhost', timeout=3
+ )
+ self.addCleanup(smtp.close)
+
+ message = EmailMessage()
+ message['From'] = email.utils.formataddr(('Michaël', 'michael@example.com'))
+ message['To'] = email.utils.formataddr(('René', 'rene@example.com'))
+
+ self.assertDictEqual(smtp.send_message(message), {})
+
+ self.assertEqual(self.serv._addresses['from'], 'michael@example.com')
+ self.assertEqual(self.serv._addresses['tos'], ['rene@example.com'])
+
class SimSMTPUTF8Server(SimSMTPServer):
socket.gethostbyname(domain)
socket.gethostbyname_ex(domain)
socket.getaddrinfo(domain,0,socket.AF_UNSPEC,socket.SOCK_STREAM)
- # this may not work if the forward lookup choses the IPv6 address, as that doesn't
+ # this may not work if the forward lookup chooses the IPv6 address, as that doesn't
# have a reverse entry yet
# socket.gethostbyaddr('испытание.python.org')
self.write_file.write(self.write_msg)
self.write_file.flush()
self.evt2.set()
- # Avoid cloding the socket before the server test has finished,
+ # Avoid closing the socket before the server test has finished,
# otherwise system recv() will return 0 instead of EWOULDBLOCK.
self.serv_finished.wait(5.0)
with self.assertRaises(TypeError):
sock.sendmsg_afalg(op=socket.ALG_OP_ENCRYPT, assoclen=-1)
+@unittest.skipUnless(sys.platform.startswith("win"), "requires Windows")
+class TestMSWindowsTCPFlags(unittest.TestCase):
+ knownTCPFlags = {
+ # avaliable since long time ago
+ 'TCP_MAXSEG',
+ 'TCP_NODELAY',
+ # available starting with Windows 10 1607
+ 'TCP_FASTOPEN',
+ # available starting with Windows 10 1703
+ 'TCP_KEEPCNT',
+ }
+
+ def test_new_tcp_flags(self):
+ provided = [s for s in dir(socket) if s.startswith('TCP')]
+ unknown = [s for s in provided if s not in self.knownTCPFlags]
+
+ self.assertEqual([], unknown,
+ "New TCP flags were discovered. See bpo-32394 for more information")
def test_main():
tests = [GeneralModuleTests, BasicTCPTest, TCPCloserTest, TCPTimeoutTest,
SendfileUsingSendTest,
SendfileUsingSendfileTest,
])
+ tests.append(TestMSWindowsTCPFlags)
thread_info = support.threading_setup()
support.run_unittest(*tests)
self.cert_time_ok("Jan 5 09:34:61 2018 GMT", 1515144901)
self.cert_time_fail("Jan 5 09:34:62 2018 GMT") # invalid seconds
- # no special treatement for the special value:
+ # no special treatment for the special value:
# 99991231235959Z (rfc 5280)
self.cert_time_ok("Dec 31 23:59:59 9999 GMT", 253402300799.0)
@needs_sni
def test_context_setget(self):
# Check that the context of a connected socket can be replaced.
- ctx1 = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+ ctx1 = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
ctx2 = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
s = socket.socket(socket.AF_INET)
with ctx1.wrap_socket(s) as ss:
else:
self.context = ssl.SSLContext(ssl_version
if ssl_version is not None
- else ssl.PROTOCOL_TLSv1)
+ else ssl.PROTOCOL_TLS)
self.context.verify_mode = (certreqs if certreqs is not None
else ssl.CERT_NONE)
if cacerts:
import shutil
import gc
import textwrap
+from test.support import FakePath
try:
import ctypes
SETBINARY = ''
NONEXISTING_CMD = ('nonexisting_i_hope',)
+# Ignore errors that indicate the command was not found
+NONEXISTING_ERRORS = (FileNotFoundError, NotADirectoryError, PermissionError)
class BaseTestCase(unittest.TestCase):
# Verify first that the call succeeds without the executable arg.
pre_args = [sys.executable, "-c"]
self._assert_python(pre_args)
- self.assertRaises((FileNotFoundError, PermissionError),
+ self.assertRaises(NONEXISTING_ERRORS,
self._assert_python, pre_args,
- executable="doesnotexist")
+ executable=NONEXISTING_CMD[0])
@unittest.skipIf(mswindows, "executable argument replaces shell")
def test_executable_replaces_shell(self):
def test_cwd_with_pathlike(self):
temp_dir = tempfile.gettempdir()
temp_dir = self._normalize_cwd(temp_dir)
-
- class _PathLikeObj:
- def __fspath__(self):
- return temp_dir
-
- self._assert_cwd(temp_dir, sys.executable, cwd=_PathLikeObj())
+ self._assert_cwd(temp_dir, sys.executable, cwd=FakePath(temp_dir))
@unittest.skipIf(mswindows, "pending resolution of issue #15533")
def test_cwd_with_relative_arg(self):
# value for that limit, but Windows has 2048, so we loop
# 1024 times (each call leaked two fds).
for i in range(1024):
- with self.assertRaises(OSError) as c:
+ with self.assertRaises(NONEXISTING_ERRORS):
subprocess.Popen(NONEXISTING_CMD,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
- # ignore errors that indicate the command was not found
- if c.exception.errno not in (errno.ENOENT, errno.EACCES):
- raise c.exception
def test_nonexisting_with_pipes(self):
# bpo-30121: Popen with pipes must close properly pipes on error.
msvcrt.CrtSetReportFile(report_type, msvcrt.CRTDBG_FILE_STDERR)
try:
- subprocess.Popen([cmd],
+ subprocess.Popen(cmd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
except OSError:
# let some time for the process to exit, and create a new Popen: this
# should trigger the wait() of p
time.sleep(0.2)
- with self.assertRaises(OSError) as c:
+ with self.assertRaises(OSError):
with subprocess.Popen(NONEXISTING_CMD,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE) as proc:
self.assertEqual(proc.returncode, 1)
def test_invalid_args(self):
- with self.assertRaises((FileNotFoundError, PermissionError)) as c:
+ with self.assertRaises(NONEXISTING_ERRORS):
with subprocess.Popen(NONEXISTING_CMD,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE) as proc:
import unittest
import socket
import tempfile
+import textwrap
import errno
from test import support
+from test.support import script_helper
TESTFN = support.TESTFN
expected = ['tests may fail, unable to create temp dir: ' + path]
self.assertEqual(warnings, expected)
+ @unittest.skipUnless(hasattr(os, "fork"), "test requires os.fork")
+ def test_temp_dir__forked_child(self):
+ """Test that a forked child process does not remove the directory."""
+ # See bpo-30028 for details.
+ # Run the test as an external script, because it uses fork.
+ script_helper.assert_python_ok("-c", textwrap.dedent("""
+ import os
+ from test import support
+ with support.temp_cwd() as temp_path:
+ pid = os.fork()
+ if pid != 0:
+ # parent process (child has pid == 0)
+
+ # wait for the child to terminate
+ (pid, status) = os.waitpid(pid, 0)
+ if status != 0:
+ raise AssertionError(f"Child process failed with exit "
+ f"status indication 0x{status:x}.")
+
+ # Make sure that temp_path is still present. When the child
+ # process leaves the 'temp_cwd'-context, the __exit__()-
+ # method of the context must not remove the temporary
+ # directory.
+ if not os.path.isdir(temp_path):
+ raise AssertionError("Child removed temp_path.")
+ """))
+
# Tests for change_cwd()
def test_change_cwd(self):
"positional argument follows keyword argument")
def test_kwargs_last2(self):
- self._check_error("int(**{base: 10}, '2')",
+ self._check_error("int(**{'base': 10}, '2')",
"positional argument follows "
"keyword argument unpacking")
def test_kwargs_last3(self):
- self._check_error("int(**{base: 10}, *['2'])",
+ self._check_error("int(**{'base': 10}, *['2'])",
"iterable argument unpacking follows "
"keyword argument unpacking")
import sys
import difflib
import gc
+from functools import wraps
+
+class tracecontext:
+ """Contex manager that traces its enter and exit."""
+ def __init__(self, output, value):
+ self.output = output
+ self.value = value
+
+ def __enter__(self):
+ self.output.append(self.value)
+
+ def __exit__(self, *exc_info):
+ self.output.append(-self.value)
# A very basic example. If this fails, we're in deep trouble.
def basic():
# Many of the tests below are tricky because they involve pass statements.
# If there is implicit control flow around a pass statement (in an except
-# clause or else caluse) under what conditions do you set a line number
+# clause or else clause) under what conditions do you set a line number
# following that clause?
# command (aka. "Set next statement").
class JumpTracer:
- """Defines a trace function that jumps from one place to another,
- with the source and destination lines of the jump being defined by
- the 'jump' property of the function under test."""
-
- def __init__(self, function):
- self.function = function
- self.jumpFrom = function.jump[0]
- self.jumpTo = function.jump[1]
+ """Defines a trace function that jumps from one place to another."""
+
+ def __init__(self, function, jumpFrom, jumpTo, event='line',
+ decorated=False):
+ self.code = function.__code__
+ self.jumpFrom = jumpFrom
+ self.jumpTo = jumpTo
+ self.event = event
+ self.firstLine = None if decorated else self.code.co_firstlineno
self.done = False
def trace(self, frame, event, arg):
- if not self.done and frame.f_code == self.function.__code__:
- firstLine = frame.f_code.co_firstlineno
- if event == 'line' and frame.f_lineno == firstLine + self.jumpFrom:
+ if self.done:
+ return
+ # frame.f_code.co_firstlineno is the first line of the decorator when
+ # 'function' is decorated and the decorator may be written using
+ # multiple physical lines when it is too long. Use the first line
+ # trace event in 'function' to find the first line of 'function'.
+ if (self.firstLine is None and frame.f_code == self.code and
+ event == 'line'):
+ self.firstLine = frame.f_lineno - 1
+ if (event == self.event and self.firstLine and
+ frame.f_lineno == self.firstLine + self.jumpFrom):
+ f = frame
+ while f is not None and f.f_code != self.code:
+ f = f.f_back
+ if f is not None:
# Cope with non-integer self.jumpTo (because of
# no_jump_to_non_integers below).
try:
- frame.f_lineno = firstLine + self.jumpTo
+ frame.f_lineno = self.firstLine + self.jumpTo
except TypeError:
frame.f_lineno = self.jumpTo
self.done = True
return self.trace
-# The first set of 'jump' tests are for things that are allowed:
+# This verifies the line-numbers-must-be-integers rule.
+def no_jump_to_non_integers(output):
+ try:
+ output.append(2)
+ except ValueError as e:
+ output.append('integer' in str(e))
-def jump_simple_forwards(output):
- output.append(1)
- output.append(2)
- output.append(3)
+# This verifies that you can't set f_lineno via _getframe or similar
+# trickery.
+def no_jump_without_trace_function():
+ try:
+ previous_frame = sys._getframe().f_back
+ previous_frame.f_lineno = previous_frame.f_lineno
+ except ValueError as e:
+ # This is the exception we wanted; make sure the error message
+ # talks about trace functions.
+ if 'trace' not in str(e):
+ raise
+ else:
+ # Something's wrong - the expected exception wasn't raised.
+ raise AssertionError("Trace-function-less jump failed to fail")
-jump_simple_forwards.jump = (1, 3)
-jump_simple_forwards.output = [3]
-def jump_simple_backwards(output):
- output.append(1)
- output.append(2)
+class JumpTestCase(unittest.TestCase):
+ def setUp(self):
+ self.addCleanup(sys.settrace, sys.gettrace())
+ sys.settrace(None)
-jump_simple_backwards.jump = (2, 1)
-jump_simple_backwards.output = [1, 1, 2]
+ def compare_jump_output(self, expected, received):
+ if received != expected:
+ self.fail( "Outputs don't match:\n" +
+ "Expected: " + repr(expected) + "\n" +
+ "Received: " + repr(received))
+
+ def run_test(self, func, jumpFrom, jumpTo, expected, error=None,
+ event='line', decorated=False):
+ tracer = JumpTracer(func, jumpFrom, jumpTo, event, decorated)
+ sys.settrace(tracer.trace)
+ output = []
+ if error is None:
+ func(output)
+ else:
+ with self.assertRaisesRegex(*error):
+ func(output)
+ sys.settrace(None)
+ self.compare_jump_output(expected, output)
+
+ def jump_test(jumpFrom, jumpTo, expected, error=None, event='line'):
+ """Decorator that creates a test that makes a jump
+ from one place to another in the following code.
+ """
+ def decorator(func):
+ @wraps(func)
+ def test(self):
+ self.run_test(func, jumpFrom, jumpTo, expected,
+ error=error, event=event, decorated=True)
+ return test
+ return decorator
+
+ ## The first set of 'jump' tests are for things that are allowed:
+
+ @jump_test(1, 3, [3])
+ def test_jump_simple_forwards(output):
+ output.append(1)
+ output.append(2)
+ output.append(3)
+
+ @jump_test(2, 1, [1, 1, 2])
+ def test_jump_simple_backwards(output):
+ output.append(1)
+ output.append(2)
+
+ @jump_test(3, 5, [2, 5])
+ def test_jump_out_of_block_forwards(output):
+ for i in 1, 2:
+ output.append(2)
+ for j in [3]: # Also tests jumping over a block
+ output.append(4)
+ output.append(5)
+
+ @jump_test(6, 1, [1, 3, 5, 1, 3, 5, 6, 7])
+ def test_jump_out_of_block_backwards(output):
+ output.append(1)
+ for i in [1]:
+ output.append(3)
+ for j in [2]: # Also tests jumping over a block
+ output.append(5)
+ output.append(6)
+ output.append(7)
+
+ @jump_test(1, 2, [3])
+ def test_jump_to_codeless_line(output):
+ output.append(1)
+ # Jumping to this line should skip to the next one.
+ output.append(3)
-def jump_out_of_block_forwards(output):
- for i in 1, 2:
+ @jump_test(2, 2, [1, 2, 3])
+ def test_jump_to_same_line(output):
+ output.append(1)
output.append(2)
- for j in [3]: # Also tests jumping over a block
+ output.append(3)
+
+ # Tests jumping within a finally block, and over one.
+ @jump_test(4, 9, [2, 9])
+ def test_jump_in_nested_finally(output):
+ try:
+ output.append(2)
+ finally:
output.append(4)
- output.append(5)
+ try:
+ output.append(6)
+ finally:
+ output.append(8)
+ output.append(9)
-jump_out_of_block_forwards.jump = (3, 5)
-jump_out_of_block_forwards.output = [2, 5]
+ @jump_test(6, 7, [2, 7], (ZeroDivisionError, ''))
+ def test_jump_in_nested_finally_2(output):
+ try:
+ output.append(2)
+ 1/0
+ return
+ finally:
+ output.append(6)
+ output.append(7)
+ output.append(8)
-def jump_out_of_block_backwards(output):
- output.append(1)
- for i in [1]:
+ @jump_test(6, 11, [2, 11], (ZeroDivisionError, ''))
+ def test_jump_in_nested_finally_3(output):
+ try:
+ output.append(2)
+ 1/0
+ return
+ finally:
+ output.append(6)
+ try:
+ output.append(8)
+ finally:
+ output.append(10)
+ output.append(11)
+ output.append(12)
+
+ @jump_test(3, 4, [1, 4])
+ def test_jump_infinite_while_loop(output):
+ output.append(1)
+ while True:
+ output.append(3)
+ output.append(4)
+
+ @jump_test(2, 3, [1, 3])
+ def test_jump_forwards_out_of_with_block(output):
+ with tracecontext(output, 1):
+ output.append(2)
output.append(3)
- for j in [2]: # Also tests jumping over a block
- output.append(5)
- output.append(6)
- output.append(7)
-jump_out_of_block_backwards.jump = (6, 1)
-jump_out_of_block_backwards.output = [1, 3, 5, 1, 3, 5, 6, 7]
+ @jump_test(3, 1, [1, 2, 1, 2, 3, -2])
+ def test_jump_backwards_out_of_with_block(output):
+ output.append(1)
+ with tracecontext(output, 2):
+ output.append(3)
-def jump_to_codeless_line(output):
- output.append(1)
- # Jumping to this line should skip to the next one.
- output.append(3)
+ @jump_test(2, 5, [5])
+ def test_jump_forwards_out_of_try_finally_block(output):
+ try:
+ output.append(2)
+ finally:
+ output.append(4)
+ output.append(5)
-jump_to_codeless_line.jump = (1, 2)
-jump_to_codeless_line.output = [3]
+ @jump_test(3, 1, [1, 1, 3, 5])
+ def test_jump_backwards_out_of_try_finally_block(output):
+ output.append(1)
+ try:
+ output.append(3)
+ finally:
+ output.append(5)
-def jump_to_same_line(output):
- output.append(1)
- output.append(2)
- output.append(3)
+ @jump_test(2, 6, [6])
+ def test_jump_forwards_out_of_try_except_block(output):
+ try:
+ output.append(2)
+ except:
+ output.append(4)
+ raise
+ output.append(6)
-jump_to_same_line.jump = (2, 2)
-jump_to_same_line.output = [1, 2, 3]
+ @jump_test(3, 1, [1, 1, 3])
+ def test_jump_backwards_out_of_try_except_block(output):
+ output.append(1)
+ try:
+ output.append(3)
+ except:
+ output.append(5)
+ raise
-# Tests jumping within a finally block, and over one.
-def jump_in_nested_finally(output):
- try:
- output.append(2)
- finally:
- output.append(4)
+ @jump_test(5, 7, [4, 7, 8])
+ def test_jump_between_except_blocks(output):
+ try:
+ 1/0
+ except ZeroDivisionError:
+ output.append(4)
+ output.append(5)
+ except FloatingPointError:
+ output.append(7)
+ output.append(8)
+
+ @jump_test(5, 6, [4, 6, 7])
+ def test_jump_within_except_block(output):
try:
+ 1/0
+ except:
+ output.append(4)
+ output.append(5)
output.append(6)
- finally:
- output.append(8)
- output.append(9)
+ output.append(7)
-jump_in_nested_finally.jump = (4, 9)
-jump_in_nested_finally.output = [2, 9]
+ @jump_test(2, 4, [1, 4, 5, -4])
+ def test_jump_across_with(output):
+ output.append(1)
+ with tracecontext(output, 2):
+ output.append(3)
+ with tracecontext(output, 4):
+ output.append(5)
-def jump_infinite_while_loop(output):
- output.append(1)
- while 1:
- output.append(2)
- output.append(3)
+ @jump_test(4, 5, [1, 3, 5, 6])
+ def test_jump_out_of_with_block_within_for_block(output):
+ output.append(1)
+ for i in [1]:
+ with tracecontext(output, 3):
+ output.append(4)
+ output.append(5)
+ output.append(6)
-jump_infinite_while_loop.jump = (3, 4)
-jump_infinite_while_loop.output = [1, 3]
+ @jump_test(4, 5, [1, 2, 3, 5, -2, 6])
+ def test_jump_out_of_with_block_within_with_block(output):
+ output.append(1)
+ with tracecontext(output, 2):
+ with tracecontext(output, 3):
+ output.append(4)
+ output.append(5)
+ output.append(6)
-# The second set of 'jump' tests are for things that are not allowed:
+ @jump_test(5, 6, [2, 4, 6, 7])
+ def test_jump_out_of_with_block_within_finally_block(output):
+ try:
+ output.append(2)
+ finally:
+ with tracecontext(output, 4):
+ output.append(5)
+ output.append(6)
+ output.append(7)
-def no_jump_too_far_forwards(output):
- try:
- output.append(2)
- output.append(3)
- except ValueError as e:
- output.append('after' in str(e))
+ @jump_test(8, 11, [1, 3, 5, 11, 12])
+ def test_jump_out_of_complex_nested_blocks(output):
+ output.append(1)
+ for i in [1]:
+ output.append(3)
+ for j in [1, 2]:
+ output.append(5)
+ try:
+ for k in [1, 2]:
+ output.append(8)
+ finally:
+ output.append(10)
+ output.append(11)
+ output.append(12)
+
+ @jump_test(3, 5, [1, 2, 5])
+ def test_jump_out_of_with_assignment(output):
+ output.append(1)
+ with tracecontext(output, 2) \
+ as x:
+ output.append(4)
+ output.append(5)
-no_jump_too_far_forwards.jump = (3, 6)
-no_jump_too_far_forwards.output = [2, True]
+ @jump_test(3, 6, [1, 6, 8, 9])
+ def test_jump_over_return_in_try_finally_block(output):
+ output.append(1)
+ try:
+ output.append(3)
+ if not output: # always false
+ return
+ output.append(6)
+ finally:
+ output.append(8)
+ output.append(9)
-def no_jump_too_far_backwards(output):
- try:
- output.append(2)
- output.append(3)
- except ValueError as e:
- output.append('before' in str(e))
+ @jump_test(5, 8, [1, 3, 8, 10, 11, 13])
+ def test_jump_over_break_in_try_finally_block(output):
+ output.append(1)
+ while True:
+ output.append(3)
+ try:
+ output.append(5)
+ if not output: # always false
+ break
+ output.append(8)
+ finally:
+ output.append(10)
+ output.append(11)
+ break
+ output.append(13)
+
+ @jump_test(1, 7, [7, 8])
+ def test_jump_over_for_block_before_else(output):
+ output.append(1)
+ if not output: # always false
+ for i in [3]:
+ output.append(4)
+ else:
+ output.append(6)
+ output.append(7)
+ output.append(8)
-no_jump_too_far_backwards.jump = (3, -1)
-no_jump_too_far_backwards.output = [2, True]
+ # The second set of 'jump' tests are for things that are not allowed:
-# Test each kind of 'except' line.
-def no_jump_to_except_1(output):
- try:
+ @jump_test(2, 3, [1], (ValueError, 'after'))
+ def test_no_jump_too_far_forwards(output):
+ output.append(1)
output.append(2)
- except:
- e = sys.exc_info()[1]
- output.append('except' in str(e))
-
-no_jump_to_except_1.jump = (2, 3)
-no_jump_to_except_1.output = [True]
-def no_jump_to_except_2(output):
- try:
+ @jump_test(2, -2, [1], (ValueError, 'before'))
+ def test_no_jump_too_far_backwards(output):
+ output.append(1)
output.append(2)
- except ValueError:
- e = sys.exc_info()[1]
- output.append('except' in str(e))
-no_jump_to_except_2.jump = (2, 3)
-no_jump_to_except_2.output = [True]
+ # Test each kind of 'except' line.
+ @jump_test(2, 3, [4], (ValueError, 'except'))
+ def test_no_jump_to_except_1(output):
+ try:
+ output.append(2)
+ except:
+ output.append(4)
+ raise
-def no_jump_to_except_3(output):
- try:
- output.append(2)
- except ValueError as e:
- output.append('except' in str(e))
+ @jump_test(2, 3, [4], (ValueError, 'except'))
+ def test_no_jump_to_except_2(output):
+ try:
+ output.append(2)
+ except ValueError:
+ output.append(4)
+ raise
-no_jump_to_except_3.jump = (2, 3)
-no_jump_to_except_3.output = [True]
+ @jump_test(2, 3, [4], (ValueError, 'except'))
+ def test_no_jump_to_except_3(output):
+ try:
+ output.append(2)
+ except ValueError as e:
+ output.append(4)
+ raise e
-def no_jump_to_except_4(output):
- try:
- output.append(2)
- except (ValueError, RuntimeError) as e:
- output.append('except' in str(e))
+ @jump_test(2, 3, [4], (ValueError, 'except'))
+ def test_no_jump_to_except_4(output):
+ try:
+ output.append(2)
+ except (ValueError, RuntimeError) as e:
+ output.append(4)
+ raise e
-no_jump_to_except_4.jump = (2, 3)
-no_jump_to_except_4.output = [True]
+ @jump_test(1, 3, [], (ValueError, 'into'))
+ def test_no_jump_forwards_into_for_block(output):
+ output.append(1)
+ for i in 1, 2:
+ output.append(3)
-def no_jump_forwards_into_block(output):
- try:
- output.append(2)
+ @jump_test(3, 2, [2, 2], (ValueError, 'into'))
+ def test_no_jump_backwards_into_for_block(output):
for i in 1, 2:
+ output.append(2)
+ output.append(3)
+
+ @jump_test(2, 4, [], (ValueError, 'into'))
+ def test_no_jump_forwards_into_while_block(output):
+ i = 1
+ output.append(2)
+ while i <= 2:
output.append(4)
- except ValueError as e:
- output.append('into' in str(e))
+ i += 1
-no_jump_forwards_into_block.jump = (2, 4)
-no_jump_forwards_into_block.output = [True]
+ @jump_test(5, 3, [3, 3], (ValueError, 'into'))
+ def test_no_jump_backwards_into_while_block(output):
+ i = 1
+ while i <= 2:
+ output.append(3)
+ i += 1
+ output.append(5)
-def no_jump_backwards_into_block(output):
- try:
- for i in 1, 2:
+ @jump_test(1, 3, [], (ValueError, 'into'))
+ def test_no_jump_forwards_into_with_block(output):
+ output.append(1)
+ with tracecontext(output, 2):
output.append(3)
- output.append(4)
- except ValueError as e:
- output.append('into' in str(e))
-no_jump_backwards_into_block.jump = (4, 3)
-no_jump_backwards_into_block.output = [3, 3, True]
+ @jump_test(3, 2, [1, 2, -1], (ValueError, 'into'))
+ def test_no_jump_backwards_into_with_block(output):
+ with tracecontext(output, 1):
+ output.append(2)
+ output.append(3)
-def no_jump_into_finally_block(output):
- try:
+ @jump_test(1, 3, [], (ValueError, 'into'))
+ def test_no_jump_forwards_into_try_finally_block(output):
+ output.append(1)
try:
output.append(3)
- x = 1
finally:
- output.append(6)
- except ValueError as e:
- output.append('finally' in str(e))
+ output.append(5)
-no_jump_into_finally_block.jump = (4, 6)
-no_jump_into_finally_block.output = [3, 6, True] # The 'finally' still runs
+ @jump_test(5, 2, [2, 4], (ValueError, 'into'))
+ def test_no_jump_backwards_into_try_finally_block(output):
+ try:
+ output.append(2)
+ finally:
+ output.append(4)
+ output.append(5)
-def no_jump_out_of_finally_block(output):
- try:
+ @jump_test(1, 3, [], (ValueError, 'into'))
+ def test_no_jump_forwards_into_try_except_block(output):
+ output.append(1)
try:
output.append(3)
- finally:
+ except:
output.append(5)
- output.append(6)
- except ValueError as e:
- output.append('finally' in str(e))
+ raise
-no_jump_out_of_finally_block.jump = (5, 1)
-no_jump_out_of_finally_block.output = [3, True]
+ @jump_test(6, 2, [2], (ValueError, 'into'))
+ def test_no_jump_backwards_into_try_except_block(output):
+ try:
+ output.append(2)
+ except:
+ output.append(4)
+ raise
+ output.append(6)
-# This verifies the line-numbers-must-be-integers rule.
-def no_jump_to_non_integers(output):
- try:
- output.append(2)
- except ValueError as e:
- output.append('integer' in str(e))
+ # 'except' with a variable creates an implicit finally block
+ @jump_test(5, 7, [4], (ValueError, 'into'))
+ def test_no_jump_between_except_blocks_2(output):
+ try:
+ 1/0
+ except ZeroDivisionError:
+ output.append(4)
+ output.append(5)
+ except FloatingPointError as e:
+ output.append(7)
+ output.append(8)
-no_jump_to_non_integers.jump = (2, "Spam")
-no_jump_to_non_integers.output = [True]
+ @jump_test(3, 6, [2, 5, 6], (ValueError, 'finally'))
+ def test_no_jump_into_finally_block(output):
+ try:
+ output.append(2)
+ output.append(3)
+ finally: # still executed if the jump is failed
+ output.append(5)
+ output.append(6)
+ output.append(7)
-def jump_across_with(output):
- with open(support.TESTFN, "wb") as fp:
- pass
- with open(support.TESTFN, "wb") as fp:
- pass
-jump_across_with.jump = (1, 3)
-jump_across_with.output = []
+ @jump_test(1, 5, [], (ValueError, 'finally'))
+ def test_no_jump_into_finally_block_2(output):
+ output.append(1)
+ try:
+ output.append(3)
+ finally:
+ output.append(5)
-# This verifies that you can't set f_lineno via _getframe or similar
-# trickery.
-def no_jump_without_trace_function():
- try:
- previous_frame = sys._getframe().f_back
- previous_frame.f_lineno = previous_frame.f_lineno
- except ValueError as e:
- # This is the exception we wanted; make sure the error message
- # talks about trace functions.
- if 'trace' not in str(e):
- raise
- else:
- # Something's wrong - the expected exception wasn't raised.
- raise RuntimeError("Trace-function-less jump failed to fail")
+ @jump_test(5, 1, [1, 3], (ValueError, 'finally'))
+ def test_no_jump_out_of_finally_block(output):
+ output.append(1)
+ try:
+ output.append(3)
+ finally:
+ output.append(5)
+ @jump_test(3, 5, [1, 2, -2], (ValueError, 'into'))
+ def test_no_jump_between_with_blocks(output):
+ output.append(1)
+ with tracecontext(output, 2):
+ output.append(3)
+ with tracecontext(output, 4):
+ output.append(5)
-class JumpTestCase(unittest.TestCase):
- def setUp(self):
- self.addCleanup(sys.settrace, sys.gettrace())
- sys.settrace(None)
+ @jump_test(7, 4, [1, 6], (ValueError, 'into'))
+ def test_no_jump_into_for_block_before_else(output):
+ output.append(1)
+ if not output: # always false
+ for i in [3]:
+ output.append(4)
+ else:
+ output.append(6)
+ output.append(7)
+ output.append(8)
- def compare_jump_output(self, expected, received):
- if received != expected:
- self.fail( "Outputs don't match:\n" +
- "Expected: " + repr(expected) + "\n" +
- "Received: " + repr(received))
+ def test_no_jump_to_non_integers(self):
+ self.run_test(no_jump_to_non_integers, 2, "Spam", [True])
- def run_test(self, func):
- tracer = JumpTracer(func)
- sys.settrace(tracer.trace)
- output = []
- func(output)
- sys.settrace(None)
- self.compare_jump_output(func.output, output)
-
- def test_01_jump_simple_forwards(self):
- self.run_test(jump_simple_forwards)
- def test_02_jump_simple_backwards(self):
- self.run_test(jump_simple_backwards)
- def test_03_jump_out_of_block_forwards(self):
- self.run_test(jump_out_of_block_forwards)
- def test_04_jump_out_of_block_backwards(self):
- self.run_test(jump_out_of_block_backwards)
- def test_05_jump_to_codeless_line(self):
- self.run_test(jump_to_codeless_line)
- def test_06_jump_to_same_line(self):
- self.run_test(jump_to_same_line)
- def test_07_jump_in_nested_finally(self):
- self.run_test(jump_in_nested_finally)
- def test_jump_infinite_while_loop(self):
- self.run_test(jump_infinite_while_loop)
- def test_08_no_jump_too_far_forwards(self):
- self.run_test(no_jump_too_far_forwards)
- def test_09_no_jump_too_far_backwards(self):
- self.run_test(no_jump_too_far_backwards)
- def test_10_no_jump_to_except_1(self):
- self.run_test(no_jump_to_except_1)
- def test_11_no_jump_to_except_2(self):
- self.run_test(no_jump_to_except_2)
- def test_12_no_jump_to_except_3(self):
- self.run_test(no_jump_to_except_3)
- def test_13_no_jump_to_except_4(self):
- self.run_test(no_jump_to_except_4)
- def test_14_no_jump_forwards_into_block(self):
- self.run_test(no_jump_forwards_into_block)
- def test_15_no_jump_backwards_into_block(self):
- self.run_test(no_jump_backwards_into_block)
- def test_16_no_jump_into_finally_block(self):
- self.run_test(no_jump_into_finally_block)
- def test_17_no_jump_out_of_finally_block(self):
- self.run_test(no_jump_out_of_finally_block)
- def test_18_no_jump_to_non_integers(self):
- self.run_test(no_jump_to_non_integers)
- def test_19_no_jump_without_trace_function(self):
+ def test_no_jump_without_trace_function(self):
# Must set sys.settrace(None) in setUp(), else condition is not
# triggered.
no_jump_without_trace_function()
- def test_jump_across_with(self):
- self.addCleanup(support.unlink, support.TESTFN)
- self.run_test(jump_across_with)
- def test_20_large_function(self):
+ def test_large_function(self):
d = {}
exec("""def f(output): # line 0
x = 0 # line 1
output.append(x) # line 1007
return""" % ('\n' * 1000,), d)
f = d['f']
-
- f.jump = (2, 1007)
- f.output = [0]
- self.run_test(f)
+ self.run_test(f, 2, 1007, [0])
def test_jump_to_firstlineno(self):
# This tests that PDB can jump back to the first line in a
""", "<fake module>", "exec")
class fake_function:
__code__ = code
- jump = (2, 0)
- tracer = JumpTracer(fake_function)
+ tracer = JumpTracer(fake_function, 2, 0)
sys.settrace(tracer.trace)
namespace = {"output": []}
exec(code, namespace)
sys.settrace(None)
self.compare_jump_output([2, 3, 2, 3, 4], namespace["output"])
+ @jump_test(2, 3, [1], event='call', error=(ValueError, "can't jump from"
+ " the 'call' trace event of a new frame"))
+ def test_no_jump_from_call(output):
+ output.append(1)
+ def nested():
+ output.append(3)
+ nested()
+ output.append(5)
+
+ @jump_test(2, 1, [1], event='return', error=(ValueError,
+ "can only jump from a 'line' trace event"))
+ def test_no_jump_from_return_event(output):
+ output.append(1)
+ return
+
+ @jump_test(2, 1, [1], event='exception', error=(ValueError,
+ "can only jump from a 'line' trace event"))
+ def test_no_jump_from_exception_event(output):
+ output.append(1)
+ 1 / 0
+
+ @jump_test(3, 2, [2], event='return', error=(ValueError,
+ "can't jump from a yield statement"))
+ def test_no_jump_from_yield(output):
+ def gen():
+ output.append(2)
+ yield 3
+ next(gen())
+ output.append(5)
-def test_main():
- support.run_unittest(
- TraceTestCase,
- RaisingTraceFuncTestCase,
- JumpTestCase
- )
if __name__ == "__main__":
- test_main()
+ unittest.main()
self.assertEqual(tarfile.itn(-0x100000000000000),
b"\xff\x00\x00\x00\x00\x00\x00\x00")
+ # Issue 32713: Test if itn() supports float values outside the
+ # non-GNU format range
+ self.assertEqual(tarfile.itn(-100.0, format=tarfile.GNU_FORMAT),
+ b"\xff\xff\xff\xff\xff\xff\xff\x9c")
+ self.assertEqual(tarfile.itn(8 ** 12 + 0.0, format=tarfile.GNU_FORMAT),
+ b"\x80\x00\x00\x10\x00\x00\x00\x00")
+ self.assertEqual(tarfile.nti(tarfile.itn(-0.1, format=tarfile.GNU_FORMAT)), 0)
+
def test_number_field_limits(self):
with self.assertRaises(ValueError):
tarfile.itn(-1, 8, tarfile.USTAR_FORMAT)
import os
import unittest
+import textwrap
from test.support.script_helper import assert_python_ok
from test.test_tools import skip_if_missing, toolsdir
headers[key] = val.strip()
return headers
+ def get_msgids(self, data):
+ """ utility: return all msgids in .po file as a list of strings """
+ msgids = []
+ reading_msgid = False
+ cur_msgid = []
+ for line in data.split('\n'):
+ if reading_msgid:
+ if line.startswith('"'):
+ cur_msgid.append(line.strip('"'))
+ else:
+ msgids.append('\n'.join(cur_msgid))
+ cur_msgid = []
+ reading_msgid = False
+ continue
+ if line.startswith('msgid '):
+ line = line[len('msgid '):]
+ cur_msgid.append(line.strip('"'))
+ reading_msgid = True
+ else:
+ if reading_msgid:
+ msgids.append('\n'.join(cur_msgid))
+
+ return msgids
+
+ def extract_docstrings_from_str(self, module_content):
+ """ utility: return all msgids extracted from module_content """
+ filename = 'test_docstrings.py'
+ with temp_cwd(None) as cwd:
+ with open(filename, 'w') as fp:
+ fp.write(module_content)
+ assert_python_ok(self.script, '-D', filename)
+ with open('messages.pot') as fp:
+ data = fp.read()
+ return self.get_msgids(data)
+
def test_header(self):
"""Make sure the required fields are in the header, according to:
http://www.gnu.org/software/gettext/manual/gettext.html#Header-Entry
# This will raise if the date format does not exactly match.
datetime.strptime(creationDate, '%Y-%m-%d %H:%M%z')
+
+ def test_funcdocstring_annotated_args(self):
+ """ Test docstrings for functions with annotated args """
+ msgids = self.extract_docstrings_from_str(textwrap.dedent('''\
+ def foo(bar: str):
+ """doc"""
+ '''))
+ self.assertIn('doc', msgids)
+
+ def test_funcdocstring_annotated_return(self):
+ """ Test docstrings for functions with annotated return type """
+ msgids = self.extract_docstrings_from_str(textwrap.dedent('''\
+ def foo(bar) -> str:
+ """doc"""
+ '''))
+ self.assertIn('doc', msgids)
+
+ def test_funcdocstring_defvalue_args(self):
+ """ Test docstring for functions with default arg values """
+ msgids = self.extract_docstrings_from_str(textwrap.dedent('''\
+ def foo(bar=()):
+ """doc"""
+ '''))
+ self.assertIn('doc', msgids)
+
+ def test_funcdocstring_multiple_funcs(self):
+ """ Test docstring extraction for multiple functions combining
+ annotated args, annotated return types and default arg values
+ """
+ msgids = self.extract_docstrings_from_str(textwrap.dedent('''\
+ def foo1(bar: tuple=()) -> str:
+ """doc1"""
+
+ def foo2(bar: List[1:2]) -> (lambda x: x):
+ """doc2"""
+
+ def foo3(bar: 'func'=lambda x: x) -> {1: 2}:
+ """doc3"""
+ '''))
+ self.assertIn('doc1', msgids)
+ self.assertIn('doc2', msgids)
+ self.assertIn('doc3', msgids)
+
+ def test_classdocstring_early_colon(self):
+ """ Test docstring extraction for a class with colons occuring within
+ the parentheses.
+ """
+ msgids = self.extract_docstrings_from_str(textwrap.dedent('''\
+ class D(L[1:2], F({1: 2}), metaclass=M(lambda x: x)):
+ """doc"""
+ '''))
+ self.assertIn('doc', msgids)
status, stdout, stderr = assert_python_ok('-m', 'trace', '-l', TESTFN)
self.assertIn(b'functions called:', stdout)
+ def test_sys_argv_list(self):
+ with open(TESTFN, 'w') as fd:
+ self.addCleanup(unlink, TESTFN)
+ fd.write("import sys\n")
+ fd.write("print(type(sys.argv))\n")
+
+ status, direct_stdout, stderr = assert_python_ok(TESTFN)
+ status, trace_stdout, stderr = assert_python_ok('-m', 'trace', '-l', TESTFN)
+ self.assertIn(direct_stdout.strip(), trace_stdout)
+
if __name__ == '__main__':
unittest.main()
def cb_sni(ssl_sock, server_name, initial_context):
nonlocal sni_name
sni_name = server_name
- context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+ context = ssl.SSLContext(ssl.PROTOCOL_TLS)
context.set_servername_callback(cb_sni)
handler = self.start_https_server(context=context, certfile=CERT_localhost)
context = ssl.create_default_context(cafile=CERT_localhost)
self.assertEqual(result.url, defrag)
self.assertEqual(result.fragment, frag)
+ def test_urlsplit_scoped_IPv6(self):
+ p = urllib.parse.urlsplit('http://[FE80::822a:a8ff:fe49:470c%tESt]:1234')
+ self.assertEqual(p.hostname, "fe80::822a:a8ff:fe49:470c%tESt")
+ self.assertEqual(p.netloc, '[FE80::822a:a8ff:fe49:470c%tESt]:1234')
+
+ p = urllib.parse.urlsplit(b'http://[FE80::822a:a8ff:fe49:470c%tESt]:1234')
+ self.assertEqual(p.hostname, b"fe80::822a:a8ff:fe49:470c%tESt")
+ self.assertEqual(p.netloc, b'[FE80::822a:a8ff:fe49:470c%tESt]:1234')
+
def test_urlsplit_attributes(self):
url = "HTTP://WWW.PYTHON.ORG/doc/#frag"
p = urllib.parse.urlsplit(url)
node2 = uuid.getnode()
self.assertEqual(node1, node2, '%012x != %012x' % (node1, node2))
+ # bpo-32502: UUID1 requires a 48-bit identifier, but hardware identifiers
+ # need not necessarily be 48 bits (e.g., EUI-64).
+ def test_uuid1_eui64(self):
+ # Confirm that uuid.getnode ignores hardware addresses larger than 48
+ # bits. Mock out each platform's *_getnode helper functions to return
+ # something just larger than 48 bits to test. This will cause
+ # uuid.getnode to fall back on uuid._random_getnode, which will
+ # generate a valid value.
+ too_large_getter = lambda: 1 << 48
+ with unittest.mock.patch.multiple(
+ uuid,
+ _node=None, # Ignore any cached node value.
+ _NODE_GETTERS_WIN32=[too_large_getter],
+ _NODE_GETTERS_UNIX=[too_large_getter],
+ ):
+ node = uuid.getnode()
+ self.assertTrue(0 < node < (1 << 48), '%012x' % node)
+
+ # Confirm that uuid1 can use the generated node, i.e., the that
+ # uuid.getnode fell back on uuid._random_getnode() rather than using
+ # the value from too_large_getter above.
+ try:
+ uuid.uuid1(node=node)
+ except ValueError as e:
+ self.fail('uuid1 was given an invalid node ID')
+
@unittest.skipUnless(importable('ctypes'), 'requires ctypes')
def test_uuid1(self):
equal = self.assertEqual
out, err = p.communicate()
self.assertEqual(out.strip(), envpy.encode())
+ @unittest.skipUnless(os.name == 'nt', 'only relevant on Windows')
+ def test_unicode_in_batch_file(self):
+ """
+ Test isolation from system site-packages
+ """
+ rmtree(self.env_dir)
+ env_dir = os.path.join(os.path.realpath(self.env_dir), 'ϼўТλФЙ')
+ builder = venv.EnvBuilder(clear=True)
+ builder.create(env_dir)
+ activate = os.path.join(env_dir, self.bindir, 'activate.bat')
+ envpy = os.path.join(env_dir, self.bindir, self.exe)
+ cmd = [activate, '&', self.exe, '-c', 'print(0)']
+ p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE, encoding='oem',
+ shell=True)
+ out, err = p.communicate()
+ print(err)
+ self.assertEqual(out.strip(), '0')
@skipInVenv
class EnsurePipTest(BaseTest):
# has to keep looping to find the first object we delete.
objs.reverse()
- # Turn on mutation in C.__eq__. The first time thru the loop,
+ # Turn on mutation in C.__eq__. The first time through the loop,
# under the iterkeys() business the first comparison will delete
# the last item iterkeys() would see, and that causes a
# RuntimeError: dictionary changed size during iteration
else:
self.assertNotIsInstance(f, ConIO)
+ def test_write_empty_data(self):
+ with ConIO('CONOUT$', 'w') as f:
+ self.assertEqual(f.write(b''), 0)
+
def assertStdinRoundTrip(self, text):
stdin = open('CONIN$', 'r')
old_stdin = sys.stdin
sourcefile = serialize(doc, to_string=False)
self.assertEqual(next(ET.iterparse(sourcefile))[0], 'end')
- # With an explitit parser too (issue #9708)
+ # With an explicit parser too (issue #9708)
sourcefile = serialize(doc, to_string=False)
parser = ET.XMLParser(target=ET.TreeBuilder())
self.assertEqual(next(ET.iterparse(sourcefile, parser=parser))[0],
args = [str(original), '-o', str(original)]
with self.assertRaises(SystemExit) as cm:
zipapp.main(args)
- # Program should exit with a non-zero returm code.
+ # Program should exit with a non-zero return code.
self.assertTrue(cm.exception.code)
def test_cmdline_copy_change_main(self):
args = [str(original), '-o', str(target), '-m', 'foo:bar']
with self.assertRaises(SystemExit) as cm:
zipapp.main(args)
- # Program should exit with a non-zero returm code.
+ # Program should exit with a non-zero return code.
self.assertTrue(cm.exception.code)
@patch('sys.stdout', new_callable=io.StringIO)
args = [str(target), '--info']
with self.assertRaises(SystemExit) as cm:
zipapp.main(args)
- # Program should exit with a zero returm code.
+ # Program should exit with a zero return code.
self.assertEqual(cm.exception.code, 0)
self.assertEqual(mock_stdout.getvalue(), "Interpreter: <none>\n")
args = [str(target), '--info']
with self.assertRaises(SystemExit) as cm:
zipapp.main(args)
- # Program should exit with a non-zero returm code.
+ # Program should exit with a non-zero return code.
self.assertTrue(cm.exception.code)
def test_flushes(self):
# Test flush() with the various options, using all the
# different levels in order to provide more variations.
- sync_opt = ['Z_NO_FLUSH', 'Z_SYNC_FLUSH', 'Z_FULL_FLUSH']
+ sync_opt = ['Z_NO_FLUSH', 'Z_SYNC_FLUSH', 'Z_FULL_FLUSH',
+ 'Z_PARTIAL_FLUSH', 'Z_BLOCK']
sync_opt = [getattr(zlib, opt) for opt in sync_opt
if hasattr(zlib, opt)]
data = HAMLET_SCENE * 8
def test_wbits(self):
# wbits=0 only supported since zlib v1.2.3.5
# Register "1.2.3" as "1.2.3.0"
- v = (zlib.ZLIB_RUNTIME_VERSION + ".0").split(".", 4)
- supports_wbits_0 = int(v[0]) > 1 or int(v[0]) == 1 \
- and (int(v[1]) > 2 or int(v[1]) == 2
- and (int(v[2]) > 3 or int(v[2]) == 3 and int(v[3]) >= 5))
+ # or "1.2.0-linux","1.2.0.f","1.2.0.f-linux"
+ v = zlib.ZLIB_RUNTIME_VERSION.split('-', 1)[0].split('.')
+ if len(v) < 4:
+ v.append('0')
+ elif not v[-1].isnumeric():
+ v[-1] = '0'
+
+ v = tuple(map(int, v))
+ supports_wbits_0 = v >= (1, 2, 3, 5)
co = zlib.compressobj(level=1, wbits=15)
zlib15 = co.compress(HAMLET_SCENE) + co.flush()
if not func:
# I'd rather use time.sleep(ms*0.001)
self.tk.call('after', ms)
+ return None
else:
def callit():
try:
"""Cancel scheduling of function identified with ID.
Identifier returned by after or after_idle must be
- given as first parameter."""
+ given as first parameter.
+ """
+ if not id:
+ raise ValueError('id must be a valid identifier returned from '
+ 'after or after_idle')
try:
data = self.tk.call('after', 'info', id)
- # In Tk 8.3, splitlist returns: (script, type)
- # In Tk 8.4, splitlist may return (script, type) or (script,)
script = self.tk.splitlist(data)[0]
self.deletecommand(script)
except TclError:
'^must specify a background color$',
root.tk_setPalette, highlightColor='blue')
+ def test_after(self):
+ root = self.root
+
+ def callback(start=0, step=1):
+ nonlocal count
+ count = start + step
+
+ # Without function, sleeps for ms.
+ self.assertIsNone(root.after(1))
+
+ # Set up with callback with no args.
+ count = 0
+ timer1 = root.after(0, callback)
+ self.assertIn(timer1, root.tk.call('after', 'info'))
+ (script, _) = root.tk.splitlist(root.tk.call('after', 'info', timer1))
+ root.update() # Process all pending events.
+ self.assertEqual(count, 1)
+ with self.assertRaises(tkinter.TclError):
+ root.tk.call(script)
+
+ # Set up with callback with args.
+ count = 0
+ timer1 = root.after(0, callback, 42, 11)
+ root.update() # Process all pending events.
+ self.assertEqual(count, 53)
+
+ # Cancel before called.
+ timer1 = root.after(1000, callback)
+ self.assertIn(timer1, root.tk.call('after', 'info'))
+ (script, _) = root.tk.splitlist(root.tk.call('after', 'info', timer1))
+ root.after_cancel(timer1) # Cancel this event.
+ self.assertEqual(count, 53)
+ with self.assertRaises(tkinter.TclError):
+ root.tk.call(script)
+
+ def test_after_idle(self):
+ root = self.root
+
+ def callback(start=0, step=1):
+ nonlocal count
+ count = start + step
+
+ # Set up with callback with no args.
+ count = 0
+ idle1 = root.after_idle(callback)
+ self.assertIn(idle1, root.tk.call('after', 'info'))
+ (script, _) = root.tk.splitlist(root.tk.call('after', 'info', idle1))
+ root.update_idletasks() # Process all pending events.
+ self.assertEqual(count, 1)
+ with self.assertRaises(tkinter.TclError):
+ root.tk.call(script)
+
+ # Set up with callback with args.
+ count = 0
+ idle1 = root.after_idle(callback, 42, 11)
+ root.update_idletasks() # Process all pending events.
+ self.assertEqual(count, 53)
+
+ # Cancel before called.
+ idle1 = root.after_idle(callback)
+ self.assertIn(idle1, root.tk.call('after', 'info'))
+ (script, _) = root.tk.splitlist(root.tk.call('after', 'info', idle1))
+ root.after_cancel(idle1) # Cancel this event.
+ self.assertEqual(count, 53)
+ with self.assertRaises(tkinter.TclError):
+ root.tk.call(script)
+
+ def test_after_cancel(self):
+ root = self.root
+
+ def callback():
+ nonlocal count
+ count += 1
+
+ timer1 = root.after(5000, callback)
+ idle1 = root.after_idle(callback)
+
+ # No value for id raises a ValueError.
+ with self.assertRaises(ValueError):
+ root.after_cancel(None)
+
+ # Cancel timer event.
+ count = 0
+ (script, _) = root.tk.splitlist(root.tk.call('after', 'info', timer1))
+ root.tk.call(script)
+ self.assertEqual(count, 1)
+ root.after_cancel(timer1)
+ with self.assertRaises(tkinter.TclError):
+ root.tk.call(script)
+ self.assertEqual(count, 1)
+ with self.assertRaises(tkinter.TclError):
+ root.tk.call('after', 'info', timer1)
+
+ # Cancel same event - nothing happens.
+ root.after_cancel(timer1)
+
+ # Cancel idle event.
+ count = 0
+ (script, _) = root.tk.splitlist(root.tk.call('after', 'info', idle1))
+ root.tk.call(script)
+ self.assertEqual(count, 1)
+ root.after_cancel(idle1)
+ with self.assertRaises(tkinter.TclError):
+ root.tk.call(script)
+ self.assertEqual(count, 1)
+ with self.assertRaises(tkinter.TclError):
+ root.tk.call('after', 'info', idle1)
+
tests_gui = (MiscTest, )
# but when passing a single state, it can be anything
valid = {'opt': [[1, 'value']]}
self.assertEqual(ttk._format_mapdict(valid), ('-opt', '1 value'))
- # special attention to single states which evalute to False
+ # special attention to single states which evaluate to False
for stateval in (None, 0, False, '', set()): # just some samples
valid = {'opt': [(stateval, 'value')]}
self.assertEqual(ttk._format_mapdict(valid),
self.assertEqual(ttk._format_layoutlist([])[0], '')
# _format_layoutlist always expects the second item (in every item)
- # to act like a dict (except when the value evalutes to False).
+ # to act like a dict (except when the value evaluates to False).
self.assertRaises(AttributeError,
ttk._format_layoutlist, [('a', 'b')])
# ['active selected', 'grey', 'focus', [1, 2, 3, 4]]
opt_val = []
for *state, val in items:
- # hacks for bakward compatibility
+ # hacks for backward compatibility
state[0] # raise IndexError if empty
if len(state) == 1:
# if it is empty (something that evaluates to False), then
if opts.filename is None:
parser.error('filename is missing: required with the main options')
- sys.argv = opts.filename, *opts.arguments
+ sys.argv = [opts.filename, *opts.arguments]
sys.path[0] = os.path.dirname(opts.filename)
t = Trace(opts.count, opts.trace, countfuncs=opts.listfuncs,
return 'break'
def update_mousewheel(self, event):
- # For wheel up, event.delte = 120 on Windows, -1 on darwin.
+ # For wheel up, event.delta = 120 on Windows, -1 on darwin.
# X-11 sends Control-Button-4 event instead.
if (event.delta < 0) == (not darwin):
return self.decrease_size()
['a_directory', 'test_directory', 'test_directory2'])
# load_tests should have been called once with loader, tests and pattern
- # (but there are no tests in our stub module itself, so thats [] at the
- # time of call.
+ # (but there are no tests in our stub module itself, so that is [] at
+ # the time of call).
self.assertEqual(Module.load_tests_args,
[(loader, [], 'test*')])
self.assertIsInstance(bar_direct, MagicMock)
# http://bugs.python.org/issue23310
- # Check if you can change behaviour of magic methds in MagicMock init
+ # Check if you can change behaviour of magic methods in MagicMock init
def test_magic_in_initialization(self):
m = MagicMock(**{'__str__.return_value': "12"})
self.assertEqual(str(m), "12")
def hostname(self):
hostname = self._hostinfo[0]
if not hostname:
- hostname = None
- elif hostname is not None:
- hostname = hostname.lower()
- return hostname
+ return None
+ # Scoped IPv6 address may have zone info, which must not be lowercased
+ # like http://[fe80::822a:a8ff:fe49:470c%tESt]:1234/keys
+ separator = '%' if isinstance(hostname, str) else b'%'
+ hostname, percent, zone = hostname.partition(separator)
+ return hostname.lower() + percent + zone
@property
def port(self):
def _ip_getnode():
"""Get the hardware address on Unix by running ip."""
# This works on Linux with iproute2.
- mac = _find_mac('ip', 'link list', [b'link/ether'], lambda i: i+1)
+ mac = _find_mac('ip', 'link', [b'link/ether'], lambda i: i+1)
if mac:
return mac
def _ipconfig_getnode():
"""Get the hardware address on Windows by running ipconfig.exe."""
- import os, re
+ import os, re, subprocess
dirs = ['', r'c:\windows\system32', r'c:\winnt\system32']
try:
import ctypes
pass
for dir in dirs:
try:
- pipe = os.popen(os.path.join(dir, 'ipconfig') + ' /all')
+ proc = subprocess.Popen([os.path.join(dir, 'ipconfig'), '/all'],
+ stdout=subprocess.PIPE,
+ encoding="oem")
except OSError:
continue
- with pipe:
- for line in pipe:
+ with proc:
+ for line in proc.stdout:
value = line.split(':')[-1].strip().lower()
if re.match('([0-9a-f][0-9a-f]-){5}[0-9a-f][0-9a-f]', value):
return int(value.replace('-', ''), 16)
_node = None
+_NODE_GETTERS_WIN32 = [_windll_getnode, _netbios_getnode, _ipconfig_getnode]
+
+_NODE_GETTERS_UNIX = [_unixdll_getnode, _ifconfig_getnode, _ip_getnode,
+ _arp_getnode, _lanscan_getnode, _netstat_getnode]
+
def getnode():
"""Get the hardware address as a 48-bit positive integer.
import sys
if sys.platform == 'win32':
- getters = [_windll_getnode, _netbios_getnode, _ipconfig_getnode]
+ getters = _NODE_GETTERS_WIN32
else:
- getters = [_unixdll_getnode, _ifconfig_getnode, _ip_getnode,
- _arp_getnode, _lanscan_getnode, _netstat_getnode]
+ getters = _NODE_GETTERS_UNIX
for getter in getters + [_random_getnode]:
try:
_node = getter()
except:
continue
- if _node is not None:
+ if (_node is not None) and (0 <= _node < (1 << 48)):
return _node
+ assert False, '_random_getnode() returned invalid value: {}'.format(_node)
_last_timestamp = None
@echo off\r
+\r
+rem This file is UTF-8 encoded, so we need to update the current code page while executing it\r
+for /f "tokens=2 delims=:" %%a in ('"%SystemRoot%\System32\chcp.com"') do (\r
+ set "_OLD_CODEPAGE=%%a"\r
+)\r
+if defined _OLD_CODEPAGE (\r
+ "%SystemRoot%\System32\chcp.com" 65001 > nul\r
+)\r
+\r
set "VIRTUAL_ENV=__VENV_DIR__"\r
\r
if not defined PROMPT (\r
set "PATH=%VIRTUAL_ENV%\__VENV_BIN_NAME__;%PATH%"\r
\r
:END\r
+if defined _OLD_CODEPAGE (\r
+ "%SystemRoot%\System32\chcp.com" %_OLD_CODEPAGE% > nul\r
+ set "_OLD_CODEPAGE="\r
+)\r
# escape character data
try:
# it's worth avoiding do-nothing calls for strings that are
- # shorter than 500 character, or so. assume that's, by far,
+ # shorter than 500 characters, or so. assume that's, by far,
# the most common case in most applications.
if "&" in text:
text = text.replace("&", "&")
accept_gzip_encoding = True
# if positive, encode request using gzip if it exceeds this threshold
- # note that many server will get confused, so only use it if you know
+ # note that many servers will get confused, so only use it if you know
# that they can decode such a request
encode_threshold = None #None = don't encode
# Send HTTP request.
#
# @param host Host descriptor (URL or (URL, x509 info) tuple).
- # @param handler Targer RPC handler (a path relative to host)
+ # @param handler Target RPC handler (a path relative to host)
# @param request_body The XML-RPC request body
# @param debug Enable debugging if debug is true.
# @return An HTTPConnection.
#!/usr/bin/env python
"""
-This script is used to build "official" universal installers on Mac OS X.
-It requires at least Mac OS X 10.5, Xcode 3, and the 10.4u SDK for
-32-bit builds. 64-bit or four-way universal builds require at least
-OS X 10.5 and the 10.5 SDK.
+This script is used to build "official" universal installers on macOS.
+
+NEW for 3.6.5:
+- support Intel 64-bit-only () and 32-bit-only installer builds
+- build and link with private Tcl/Tk 8.6 for 10.9+ builds
+- deprecate use of explicit SDK (--sdk-path=) since all but the oldest
+ versions of Xcode support implicit setting of an SDK via environment
+ variables (SDKROOT and friends, see the xcrun man page for more info).
+ The SDK stuff was primarily needed for building universal installers
+ for 10.4; so as of 3.6.5, building installers for 10.4 is no longer
+ supported with build-installer.
+- use generic "gcc" as compiler (CC env var) rather than "gcc-4.2"
Please ensure that this script keeps working with Python 2.5, to avoid
-bootstrap issues (/usr/bin/python is Python 2.5 on OSX 10.5). Sphinx,
-which is used to build the documentation, currently requires at least
-Python 2.4. However, as of Python 3.4.1, Doc builds require an external
-sphinx-build and the current versions of Sphinx now require at least
-Python 2.6.
-
-In addition to what is supplied with OS X 10.5+ and Xcode 3+, the script
-requires an installed third-party version of
-Tcl/Tk 8.4 (for OS X 10.4 and 10.5 deployment targets) or Tcl/TK 8.5
+bootstrap issues (/usr/bin/python is Python 2.5 on OSX 10.5). Doc builds
+use current versions of Sphinx and require a reasonably current python3.
+Sphinx and dependencies are installed into a venv using the python3's pip
+so will fetch them from PyPI if necessary. Since python3 is now used for
+Sphinx, build-installer.py should also be converted to use python3!
+
+For 10.9 or greater deployment targets, build-installer builds and links
+with its own copy of Tcl/Tk 8.5 and the rest of this paragraph does not
+apply. Otherwise, build-installer requires an installed third-party version
+of Tcl/Tk 8.4 (for OS X 10.4 and 10.5 deployment targets) or Tcl/TK 8.5
(for 10.6 or later) installed in /Library/Frameworks. When installed,
the Python built by this script will attempt to dynamically link first to
Tcl and Tk frameworks in /Library/Frameworks if available otherwise fall
back to the ones in /System/Library/Framework. For the build, we recommend
-installing the most recent ActiveTcl 8.4 or 8.5 version.
-
-32-bit-only installer builds are still possible on OS X 10.4 with Xcode 2.5
-and the installation of additional components, such as a newer Python
-(2.5 is needed for Python parser updates) and for the documentation
-build either svn (pre-3.4.1) or sphinx-build (3.4.1 and later).
+installing the most recent ActiveTcl 8.5 or 8.4 version, depending
+on the deployment target. The actual version linked to depends on the
+path of /Library/Frameworks/{Tcl,Tk}.framework/Versions/Current.
Usage: see USAGE variable in the script.
"""
DEPSRC = os.path.join(WORKDIR, 'third-party')
DEPSRC = os.path.expanduser('~/Universal/other-sources')
-# Location of the preferred SDK
-
-### There are some issues with the SDK selection below here,
-### The resulting binary doesn't work on all platforms that
-### it should. Always default to the 10.4u SDK until that
-### issue is resolved.
-###
-##if int(os.uname()[2].split('.')[0]) == 8:
-## # Explicitly use the 10.4u (universal) SDK when
-## # building on 10.4, the system headers are not
-## # useable for a universal build
-## SDKPATH = "/Developer/SDKs/MacOSX10.4u.sdk"
-##else:
-## SDKPATH = "/"
-
-SDKPATH = "/Developer/SDKs/MacOSX10.4u.sdk"
-
universal_opts_map = { '32-bit': ('i386', 'ppc',),
'64-bit': ('x86_64', 'ppc64',),
'intel': ('i386', 'x86_64'),
+ 'intel-32': ('i386',),
+ 'intel-64': ('x86_64',),
'3-way': ('ppc', 'i386', 'x86_64'),
'all': ('i386', 'ppc', 'x86_64', 'ppc64',) }
default_target_map = {
'64-bit': '10.5',
'3-way': '10.5',
'intel': '10.5',
+ 'intel-32': '10.4',
+ 'intel-64': '10.5',
'all': '10.5',
}
))))
# $MACOSX_DEPLOYMENT_TARGET -> minimum OS X level
-DEPTARGET = '10.3'
+DEPTARGET = '10.5'
def getDeptargetTuple():
return tuple([int(n) for n in DEPTARGET.split('.')[0:2]])
def getTargetCompilers():
target_cc_map = {
- '10.3': ('gcc-4.0', 'g++-4.0'),
'10.4': ('gcc-4.0', 'g++-4.0'),
- '10.5': ('gcc-4.2', 'g++-4.2'),
- '10.6': ('gcc-4.2', 'g++-4.2'),
+ '10.5': ('gcc', 'g++'),
+ '10.6': ('gcc', 'g++'),
}
- return target_cc_map.get(DEPTARGET, ('clang', 'clang++') )
+ return target_cc_map.get(DEPTARGET, ('gcc', 'g++') )
CC, CXX = getTargetCompilers()
-b DIR
--build-dir=DIR: Create build here (default: %(WORKDIR)r)
--third-party=DIR: Store third-party sources here (default: %(DEPSRC)r)
- --sdk-path=DIR: Location of the SDK (default: %(SDKPATH)r)
+ --sdk-path=DIR: Location of the SDK (deprecated, use SDKROOT env variable)
--src-dir=DIR: Location of the Python sources (default: %(SRCDIR)r)
- --dep-target=10.n OS X deployment target (default: %(DEPTARGET)r)
+ --dep-target=10.n macOS deployment target (default: %(DEPTARGET)r)
--universal-archs=x universal architectures (options: %(UNIVERSALOPTS)r, default: %(UNIVERSALARCHS)r)
""")% globals()
# '/Library/Frameworks/Tk.framework/Versions/8.5/Tk']
EXPECTED_SHARED_LIBS = {}
+# Are we building and linking with our own copy of Tcl/TK?
+# For now, do so if deployment target is 10.9+.
+def internalTk():
+ return getDeptargetTuple() >= (10, 9)
+
# List of names of third party software built with this installer.
# The names will be inserted into the rtf version of the License.
THIRD_PARTY_LIBS = []
result.extend([
dict(
- name="OpenSSL 1.0.2m",
- url="https://www.openssl.org/source/openssl-1.0.2m.tar.gz",
- checksum='10e9e37f492094b9ef296f68f24a7666',
- patches=[
- "openssl_sdk_makedepend.patch",
- ],
+ name="OpenSSL 1.0.2n",
+ url="https://www.openssl.org/source/openssl-1.0.2n.tar.gz",
+ checksum='13bdc1b1d1ff39b6fd42a255e74676a4',
buildrecipe=build_universal_openssl,
configure=None,
install=None,
),
])
-# Disable for now
- if False: # if getDeptargetTuple() > (10, 5):
+ if internalTk():
result.extend([
dict(
- name="Tcl 8.5.15",
- url="ftp://ftp.tcl.tk/pub/tcl//tcl8_5/tcl8.5.15-src.tar.gz",
- checksum='f3df162f92c69b254079c4d0af7a690f',
+ name="Tcl 8.6.8",
+ url="ftp://ftp.tcl.tk/pub/tcl//tcl8_6/tcl8.6.8-src.tar.gz",
+ checksum='81656d3367af032e0ae6157eff134f89',
buildDir="unix",
configure_pre=[
'--enable-shared',
useLDFlags=False,
install='make TCL_LIBRARY=%(TCL_LIBRARY)s && make install TCL_LIBRARY=%(TCL_LIBRARY)s DESTDIR=%(DESTDIR)s'%{
"DESTDIR": shellQuote(os.path.join(WORKDIR, 'libraries')),
- "TCL_LIBRARY": shellQuote('/Library/Frameworks/Python.framework/Versions/%s/lib/tcl8.5'%(getVersion())),
+ "TCL_LIBRARY": shellQuote('/Library/Frameworks/Python.framework/Versions/%s/lib/tcl8.6'%(getVersion())),
},
),
dict(
- name="Tk 8.5.15",
- url="ftp://ftp.tcl.tk/pub/tcl//tcl8_5/tk8.5.15-src.tar.gz",
- checksum='55b8e33f903210a4e1c8bce0f820657f',
+ name="Tk 8.6.8",
+ url="ftp://ftp.tcl.tk/pub/tcl//tcl8_6/tk8.6.8-src.tar.gz",
+ checksum='5e0faecba458ee1386078fb228d008ba',
patches=[
- "issue19373_tk_8_5_15_source.patch",
+ "tk868_on_10_8_10_9.patch",
],
buildDir="unix",
configure_pre=[
useLDFlags=False,
install='make TCL_LIBRARY=%(TCL_LIBRARY)s TK_LIBRARY=%(TK_LIBRARY)s && make install TCL_LIBRARY=%(TCL_LIBRARY)s TK_LIBRARY=%(TK_LIBRARY)s DESTDIR=%(DESTDIR)s'%{
"DESTDIR": shellQuote(os.path.join(WORKDIR, 'libraries')),
- "TCL_LIBRARY": shellQuote('/Library/Frameworks/Python.framework/Versions/%s/lib/tcl8.5'%(getVersion())),
- "TK_LIBRARY": shellQuote('/Library/Frameworks/Python.framework/Versions/%s/lib/tk8.5'%(getVersion())),
+ "TCL_LIBRARY": shellQuote('/Library/Frameworks/Python.framework/Versions/%s/lib/tcl8.6'%(getVersion())),
+ "TK_LIBRARY": shellQuote('/Library/Frameworks/Python.framework/Versions/%s/lib/tk8.6'%(getVersion())),
},
),
])
if PYTHON_3:
result.extend([
dict(
- name="XZ 5.2.2",
- url="http://tukaani.org/xz/xz-5.2.2.tar.gz",
- checksum='7cf6a8544a7dae8e8106fdf7addfa28c',
+ name="XZ 5.2.3",
+ url="http://tukaani.org/xz/xz-5.2.3.tar.gz",
+ checksum='ef68674fb47a8b8e741b34e429d86e9d',
configure_pre=[
'--disable-dependency-tracking',
]
),
),
dict(
- name="SQLite 3.21.0",
- url="https://www.sqlite.org/2017/sqlite-autoconf-3210000.tar.gz",
- checksum='7913de4c3126ba3c24689cb7a199ea31',
+ name="SQLite 3.22.0",
+ url="https://www.sqlite.org/2018/sqlite-autoconf-3220000.tar.gz",
+ checksum='96b5648d542e8afa6ab7ffb8db8ddc3d',
extra_cflags=('-Os '
'-DSQLITE_ENABLE_FTS5 '
'-DSQLITE_ENABLE_FTS4 '
'-DSQLITE_ENABLE_FTS3_PARENTHESIS '
+ '-DSQLITE_ENABLE_JSON1 '
'-DSQLITE_ENABLE_RTREE '
'-DSQLITE_TCL=0 '
'%s' % ('','-DSQLITE_WITHOUT_ZONEMALLOC ')[LT_10_5]),
url="http://bzip.org/1.0.6/bzip2-1.0.6.tar.gz",
checksum='00b516f4704d4a7cb50a1d97e6e8e15b',
configure=None,
- install='make install CC=%s CXX=%s, PREFIX=%s/usr/local/ CFLAGS="-arch %s -isysroot %s"'%(
+ install='make install CC=%s CXX=%s, PREFIX=%s/usr/local/ CFLAGS="-arch %s"'%(
CC, CXX,
shellQuote(os.path.join(WORKDIR, 'libraries')),
' -arch '.join(ARCHLIST),
- SDKPATH,
),
),
dict(
url="http://www.gzip.org/zlib/zlib-1.2.3.tar.gz",
checksum='debc62758716a169df9f62e6ab2bc634',
configure=None,
- install='make install CC=%s CXX=%s, prefix=%s/usr/local/ CFLAGS="-arch %s -isysroot %s"'%(
+ install='make install CC=%s CXX=%s, prefix=%s/usr/local/ CFLAGS="-arch %s"'%(
CC, CXX,
shellQuote(os.path.join(WORKDIR, 'libraries')),
' -arch '.join(ARCHLIST),
- SDKPATH,
),
),
dict(
source="/Library/Frameworks/Python.framework",
readme="""\
This package installs Python.framework, that is the python
- interpreter and the standard library. This also includes Python
- wrappers for lots of Mac OS X API's.
+ interpreter and the standard library.
""",
postflight="scripts/postflight.framework",
selected='selected',
),
]
- if getDeptargetTuple() < (10, 4) and not PYTHON_3:
- result.append(
- dict(
- name="PythonSystemFixes",
- long_name="Fix system Python",
- readme="""\
- This package updates the system python installation on
- Mac OS X 10.3 to ensure that you can build new python extensions
- using that copy of python after installing this version.
- """,
- postflight="../Tools/fixapplepython23.py",
- topdir="/Library/Frameworks/Python.framework",
- source="/empty-dir",
- required=False,
- selected=unselected_for_python3,
- )
- )
-
return result
def fatal(msg):
Check that we're running on a supported system.
"""
- if sys.version_info[0:2] < (2, 4):
- fatal("This script must be run with Python 2.4 or later")
+ if sys.version_info[0:2] < (2, 5):
+ fatal("This script must be run with Python 2.5 (or later)")
if platform.system() != 'Darwin':
- fatal("This script should be run on a Mac OS X 10.4 (or later) system")
+ fatal("This script should be run on a macOS 10.5 (or later) system")
if int(platform.release().split('.')[0]) < 8:
- fatal("This script should be run on a Mac OS X 10.4 (or later) system")
-
- if not os.path.exists(SDKPATH):
- fatal("Please install the latest version of Xcode and the %s SDK"%(
- os.path.basename(SDKPATH[:-4])))
+ fatal("This script should be run on a macOS 10.5 (or later) system")
# Because we only support dynamic load of only one major/minor version of
+ # Tcl/Tk, if we are not using building and using our own private copy of
# Tcl/Tk, ensure:
- # 1. there are no user-installed frameworks of Tcl/Tk with version
- # higher than the Apple-supplied system version in
- # SDKROOT/System/Library/Frameworks
- # 2. there is a user-installed framework (usually ActiveTcl) in (or linked
- # in) SDKROOT/Library/Frameworks with the same version as the system
- # version. This allows users to choose to install a newer patch level.
-
- frameworks = {}
- for framework in ['Tcl', 'Tk']:
- fwpth = 'Library/Frameworks/%s.framework/Versions/Current' % framework
- sysfw = os.path.join(SDKPATH, 'System', fwpth)
- libfw = os.path.join(SDKPATH, fwpth)
- usrfw = os.path.join(os.getenv('HOME'), fwpth)
- frameworks[framework] = os.readlink(sysfw)
- if not os.path.exists(libfw):
- fatal("Please install a link to a current %s %s as %s so "
- "the user can override the system framework."
- % (framework, frameworks[framework], libfw))
- if os.readlink(libfw) != os.readlink(sysfw):
- fatal("Version of %s must match %s" % (libfw, sysfw) )
- if os.path.exists(usrfw):
- fatal("Please rename %s to avoid possible dynamic load issues."
- % usrfw)
-
- if frameworks['Tcl'] != frameworks['Tk']:
- fatal("The Tcl and Tk frameworks are not the same version.")
-
- # add files to check after build
- EXPECTED_SHARED_LIBS['_tkinter.so'] = [
- "/Library/Frameworks/Tcl.framework/Versions/%s/Tcl"
- % frameworks['Tcl'],
- "/Library/Frameworks/Tk.framework/Versions/%s/Tk"
- % frameworks['Tk'],
- ]
+ # 1. there is a user-installed framework (usually ActiveTcl) in (or linked
+ # in) SDKROOT/Library/Frameworks. As of Python 3.6.5, we no longer
+ # enforce that the version of the user-installed framework also
+ # exists in the system-supplied Tcl/Tk frameworks. Time to support
+ # Tcl/Tk 8.6 even if Apple does not.
+ if not internalTk():
+ frameworks = {}
+ for framework in ['Tcl', 'Tk']:
+ fwpth = 'Library/Frameworks/%s.framework/Versions/Current' % framework
+ libfw = os.path.join('/', fwpth)
+ usrfw = os.path.join(os.getenv('HOME'), fwpth)
+ frameworks[framework] = os.readlink(libfw)
+ if not os.path.exists(libfw):
+ fatal("Please install a link to a current %s %s as %s so "
+ "the user can override the system framework."
+ % (framework, frameworks[framework], libfw))
+ if os.path.exists(usrfw):
+ fatal("Please rename %s to avoid possible dynamic load issues."
+ % usrfw)
+
+ if frameworks['Tcl'] != frameworks['Tk']:
+ fatal("The Tcl and Tk frameworks are not the same version.")
+
+ print(" -- Building with external Tcl/Tk %s frameworks"
+ % frameworks['Tk'])
+
+ # add files to check after build
+ EXPECTED_SHARED_LIBS['_tkinter.so'] = [
+ "/Library/Frameworks/Tcl.framework/Versions/%s/Tcl"
+ % frameworks['Tcl'],
+ "/Library/Frameworks/Tk.framework/Versions/%s/Tk"
+ % frameworks['Tk'],
+ ]
+ else:
+ print(" -- Building private copy of Tcl/Tk")
+ print("")
# Remove inherited environment variables which might influence build
environ_var_prefixes = ['CPATH', 'C_INCLUDE_', 'DYLD_', 'LANG', 'LC_',
"""
Parse arguments and update global settings.
"""
- global WORKDIR, DEPSRC, SDKPATH, SRCDIR, DEPTARGET
+ global WORKDIR, DEPSRC, SRCDIR, DEPTARGET
global UNIVERSALOPTS, UNIVERSALARCHS, ARCHLIST, CC, CXX
global FW_VERSION_PREFIX
global FW_SSL_DIRECTORY
DEPSRC=v
elif k in ('--sdk-path',):
- SDKPATH=v
+ print(" WARNING: --sdk-path is no longer supported")
elif k in ('--src-dir',):
SRCDIR=v
if deptarget is None:
# Select alternate default deployment
# target
- DEPTARGET = default_target_map.get(v, '10.3')
+ DEPTARGET = default_target_map.get(v, '10.5')
else:
raise NotImplementedError(v)
SRCDIR=os.path.abspath(SRCDIR)
WORKDIR=os.path.abspath(WORKDIR)
- SDKPATH=os.path.abspath(SDKPATH)
DEPSRC=os.path.abspath(DEPSRC)
CC, CXX = getTargetCompilers()
print("-- Settings:")
print(" * Source directory: %s" % SRCDIR)
print(" * Build directory: %s" % WORKDIR)
- print(" * SDK location: %s" % SDKPATH)
print(" * Third-party source: %s" % DEPSRC)
print(" * Deployment target: %s" % DEPTARGET)
print(" * Universal archs: %s" % str(ARCHLIST))
configure_opts.append("no-asm")
runCommand(" ".join(["perl", "Configure"]
+ arch_opts[arch] + configure_opts))
- runCommand("make depend OSX_SDK=%s" % SDKPATH)
- runCommand("make all OSX_SDK=%s" % SDKPATH)
- runCommand("make install_sw OSX_SDK=%s" % SDKPATH)
+ runCommand("make depend")
+ runCommand("make all")
+ runCommand("make install_sw")
# runCommand("make test")
return
if recipe.get('useLDFlags', 1):
configure_args.extend([
- "CFLAGS=%s-mmacosx-version-min=%s -arch %s -isysroot %s "
+ "CFLAGS=%s-mmacosx-version-min=%s -arch %s "
"-I%s/usr/local/include"%(
recipe.get('extra_cflags', ''),
DEPTARGET,
' -arch '.join(archList),
- shellQuote(SDKPATH)[1:-1],
shellQuote(basedir)[1:-1],),
- "LDFLAGS=-mmacosx-version-min=%s -isysroot %s -L%s/usr/local/lib -arch %s"%(
+ "LDFLAGS=-mmacosx-version-min=%s -L%s/usr/local/lib -arch %s"%(
DEPTARGET,
- shellQuote(SDKPATH)[1:-1],
shellQuote(basedir)[1:-1],
' -arch '.join(archList)),
])
else:
configure_args.extend([
- "CFLAGS=%s-mmacosx-version-min=%s -arch %s -isysroot %s "
+ "CFLAGS=%s-mmacosx-version-min=%s -arch %s "
"-I%s/usr/local/include"%(
recipe.get('extra_cflags', ''),
DEPTARGET,
' -arch '.join(archList),
- shellQuote(SDKPATH)[1:-1],
shellQuote(basedir)[1:-1],),
])
def buildPythonDocs():
# This stores the documentation as Resources/English.lproj/Documentation
- # inside the framwork. pydoc and IDLE will pick it up there.
+ # inside the framework. pydoc and IDLE will pick it up there.
print("Install python documentation")
rootDir = os.path.join(WORKDIR, '_root')
buildDir = os.path.join('../../Doc')
curdir = os.getcwd()
os.chdir(buildDir)
- # Not sure if this is still needed, the original build script
- # claims that parts of the install assume python.exe exists.
- os.symlink('python', os.path.join(buildDir, 'python.exe'))
-
# Extract the version from the configure file, needed to calculate
# several paths.
version = getVersion()
os.environ['DYLD_LIBRARY_PATH'] = os.path.join(WORKDIR,
'libraries', 'usr', 'local', 'lib')
print("Running configure...")
- runCommand("%s -C --enable-framework --enable-universalsdk=%s "
+ runCommand("%s -C --enable-framework --enable-universalsdk=/ "
"--with-universal-archs=%s "
"%s "
"%s "
+ "%s "
+ "%s "
"LDFLAGS='-g -L%s/libraries/usr/local/lib' "
"CFLAGS='-g -I%s/libraries/usr/local/include' 2>&1"%(
- shellQuote(os.path.join(SRCDIR, 'configure')), shellQuote(SDKPATH),
+ shellQuote(os.path.join(SRCDIR, 'configure')),
UNIVERSALARCHS,
(' ', '--with-computed-gotos ')[PYTHON_3],
(' ', '--without-ensurepip ')[PYTHON_3],
+ (' ', "--with-tcltk-includes='-I%s/libraries/usr/local/include'"%(
+ shellQuote(WORKDIR)[1:-1],))[internalTk()],
+ (' ', "--with-tcltk-libs='-L%s/libraries/usr/local/lib -ltcl8.6 -ltk8.6'"%(
+ shellQuote(WORKDIR)[1:-1],))[internalTk()],
shellQuote(WORKDIR)[1:-1],
shellQuote(WORKDIR)[1:-1]))
del os.environ['DYLD_LIBRARY_PATH']
print("Copying required shared libraries")
if os.path.exists(os.path.join(WORKDIR, 'libraries', 'Library')):
- runCommand("mv %s/* %s"%(
- shellQuote(os.path.join(
+ build_lib_dir = os.path.join(
WORKDIR, 'libraries', 'Library', 'Frameworks',
- 'Python.framework', 'Versions', getVersion(),
- 'lib')),
- shellQuote(os.path.join(WORKDIR, '_root', 'Library', 'Frameworks',
- 'Python.framework', 'Versions', getVersion(),
- 'lib'))))
+ 'Python.framework', 'Versions', getVersion(), 'lib')
+ fw_lib_dir = os.path.join(
+ WORKDIR, '_root', 'Library', 'Frameworks',
+ 'Python.framework', 'Versions', getVersion(), 'lib')
+ if internalTk():
+ # move Tcl and Tk pkgconfig files
+ runCommand("mv %s/pkgconfig/* %s/pkgconfig"%(
+ shellQuote(build_lib_dir),
+ shellQuote(fw_lib_dir) ))
+ runCommand("rm -r %s/pkgconfig"%(
+ shellQuote(build_lib_dir), ))
+ runCommand("mv %s/* %s"%(
+ shellQuote(build_lib_dir),
+ shellQuote(fw_lib_dir) ))
frmDir = os.path.join(rootDir, 'Library', 'Frameworks', 'Python.framework')
frmDirVersioned = os.path.join(frmDir, 'Versions', version)
+++ /dev/null
-Issue #19373: Patch to Tk 8.5.15 to correct refresh problem on OS x 10.9.
-From upstream checkin https://core.tcl.tk/tk/info/5a5abf71f9
-
---- tk8.5.15/macosx/tkMacOSXDraw.c 2013-09-16 09:41:21.000000000 -0700
-+++ Tk_Source_Code-5a5abf71f9fdb0da/macosx/tkMacOSXDraw.c 2013-10-27 13:27:00.000000000 -0700
-@@ -1688,6 +1688,7 @@
- {
- if (dcPtr->context) {
- CGContextSynchronize(dcPtr->context);
-+ [[dcPtr->view window] setViewsNeedDisplay:YES];
- [[dcPtr->view window] enableFlushWindow];
- if (dcPtr->focusLocked) {
- [dcPtr->view unlockFocus];
+++ /dev/null
-# HG changeset patch
-#
-# using openssl 1.0.2k
-#
-# - support building with an OS X SDK
-
-diff Configure
-
-diff --git a/Configure b/Configure
---- a/Configure
-+++ b/Configure
-@@ -642,12 +642,12 @@
-
- ##### MacOS X (a.k.a. Rhapsody or Darwin) setup
- "rhapsody-ppc-cc","cc:-O3 -DB_ENDIAN::(unknown):MACOSX_RHAPSODY::BN_LLONG RC4_CHAR RC4_CHUNK DES_UNROLL BF_PTR:${no_asm}::",
--"darwin-ppc-cc","cc:-arch ppc -O3 -DB_ENDIAN -Wa,-force_cpusubtype_ALL::-D_REENTRANT:MACOSX:-Wl,-search_paths_first%:BN_LLONG RC4_CHAR RC4_CHUNK DES_UNROLL BF_PTR:${ppc32_asm}:osx32:dlfcn:darwin-shared:-fPIC -fno-common:-arch ppc -dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib",
--"darwin64-ppc-cc","cc:-arch ppc64 -O3 -DB_ENDIAN::-D_REENTRANT:MACOSX:-Wl,-search_paths_first%:SIXTY_FOUR_BIT_LONG RC4_CHAR RC4_CHUNK DES_UNROLL BF_PTR:${ppc64_asm}:osx64:dlfcn:darwin-shared:-fPIC -fno-common:-arch ppc64 -dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib",
--"darwin-i386-cc","cc:-arch i386 -O3 -fomit-frame-pointer -DL_ENDIAN::-D_REENTRANT:MACOSX:-Wl,-search_paths_first%:BN_LLONG RC4_INT RC4_CHUNK DES_UNROLL BF_PTR:".eval{my $asm=$x86_asm;$asm=~s/cast\-586\.o//;$asm}.":macosx:dlfcn:darwin-shared:-fPIC -fno-common:-arch i386 -dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib",
--"debug-darwin-i386-cc","cc:-arch i386 -g3 -DL_ENDIAN::-D_REENTRANT:MACOSX:-Wl,-search_paths_first%:BN_LLONG RC4_INT RC4_CHUNK DES_UNROLL BF_PTR:${x86_asm}:macosx:dlfcn:darwin-shared:-fPIC -fno-common:-arch i386 -dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib",
--"darwin64-x86_64-cc","cc:-arch x86_64 -O3 -DL_ENDIAN -Wall::-D_REENTRANT:MACOSX:-Wl,-search_paths_first%:SIXTY_FOUR_BIT_LONG RC4_CHUNK DES_INT DES_UNROLL:".eval{my $asm=$x86_64_asm;$asm=~s/rc4\-[^:]+//;$asm}.":macosx:dlfcn:darwin-shared:-fPIC -fno-common:-arch x86_64 -dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib",
--"debug-darwin64-x86_64-cc","cc:-arch x86_64 -ggdb -g2 -O0 -DL_ENDIAN -Wall::-D_REENTRANT:MACOSX:-Wl,-search_paths_first%:SIXTY_FOUR_BIT_LONG RC4_CHUNK DES_INT DES_UNROLL:".eval{my $asm=$x86_64_asm;$asm=~s/rc4\-[^:]+//;$asm}.":macosx:dlfcn:darwin-shared:-fPIC -fno-common:-arch x86_64 -dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib",
-+"darwin-ppc-cc","cc:-arch ppc -isysroot \$(OSX_SDK) -O3 -DB_ENDIAN -Wa,-force_cpusubtype_ALL::-D_REENTRANT:MACOSX:-Wl,-search_paths_first%:BN_LLONG RC4_CHAR RC4_CHUNK DES_UNROLL BF_PTR:${ppc32_asm}:osx32:dlfcn:darwin-shared:-fPIC -fno-common:-arch ppc -dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib",
-+"darwin64-ppc-cc","cc:-arch ppc64 -isysroot \$(OSX_SDK) -O3 -DB_ENDIAN::-D_REENTRANT:MACOSX:-Wl,-search_paths_first%:SIXTY_FOUR_BIT_LONG RC4_CHAR RC4_CHUNK DES_UNROLL BF_PTR:${ppc64_asm}:osx64:dlfcn:darwin-shared:-fPIC -fno-common:-arch ppc64 -dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib",
-+"darwin-i386-cc","cc:-arch i386 -isysroot \$(OSX_SDK) -O3 -fomit-frame-pointer -DL_ENDIAN::-D_REENTRANT:MACOSX:-Wl,-search_paths_first%:BN_LLONG RC4_INT RC4_CHUNK DES_UNROLL BF_PTR:".eval{my $asm=$x86_asm;$asm=~s/cast\-586\.o//;$asm}.":macosx:dlfcn:darwin-shared:-fPIC -fno-common:-arch i386 -dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib",
-+"debug-darwin-i386-cc","cc:-arch i386 -isysroot \$(OSX_SDK) -g3 -DL_ENDIAN::-D_REENTRANT:MACOSX:-Wl,-search_paths_first%:BN_LLONG RC4_INT RC4_CHUNK DES_UNROLL BF_PTR:${x86_asm}:macosx:dlfcn:darwin-shared:-fPIC -fno-common:-arch i386 -dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib",
-+"darwin64-x86_64-cc","cc:-arch x86_64 -isysroot \$(OSX_SDK) -O3 -DL_ENDIAN -Wall::-D_REENTRANT:MACOSX:-Wl,-search_paths_first%:SIXTY_FOUR_BIT_LONG RC4_CHUNK DES_INT DES_UNROLL:".eval{my $asm=$x86_64_asm;$asm=~s/rc4\-[^:]+//;$asm}.":macosx:dlfcn:darwin-shared:-fPIC -fno-common:-arch x86_64 -dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib",
-+"debug-darwin64-x86_64-cc","cc:-arch x86_64 -isysroot \$(OSX_SDK) -ggdb -g2 -O0 -DL_ENDIAN -Wall::-D_REENTRANT:MACOSX:-Wl,-search_paths_first%:SIXTY_FOUR_BIT_LONG RC4_CHUNK DES_INT DES_UNROLL:".eval{my $asm=$x86_64_asm;$asm=~s/rc4\-[^:]+//;$asm}.":macosx:dlfcn:darwin-shared:-fPIC -fno-common:-arch x86_64 -dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib",
- "debug-darwin-ppc-cc","cc:-DBN_DEBUG -DREF_CHECK -DCONF_DEBUG -DCRYPTO_MDEBUG -DB_ENDIAN -g -Wall -O::-D_REENTRANT:MACOSX::BN_LLONG RC4_CHAR RC4_CHUNK DES_UNROLL BF_PTR:${ppc32_asm}:osx32:dlfcn:darwin-shared:-fPIC:-dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib",
- # iPhoneOS/iOS
- "iphoneos-cross","llvm-gcc:-O3 -isysroot \$(CROSS_TOP)/SDKs/\$(CROSS_SDK) -fomit-frame-pointer -fno-common::-D_REENTRANT:iOS:-Wl,-search_paths_first%:BN_LLONG RC4_CHAR RC4_CHUNK DES_UNROLL BF_PTR:${no_asm}:dlfcn:darwin-shared:-fPIC -fno-common:-dynamiclib:.\$(SHLIB_MAJOR).\$(SHLIB_MINOR).dylib",
-@@ -1728,8 +1728,7 @@
- s/^AR=\s*ar/AR= $ar/;
- s/^RANLIB=.*/RANLIB= $ranlib/;
- s/^RC=.*/RC= $windres/;
-- s/^MAKEDEPPROG=.*$/MAKEDEPPROG= $cc/ if $cc eq "gcc";
-- s/^MAKEDEPPROG=.*$/MAKEDEPPROG= $cc/ if $ecc eq "gcc" || $ecc eq "clang";
-+ s/^MAKEDEPPROG=.*$/MAKEDEPPROG= $cc/;
- }
- s/^CFLAG=.*$/CFLAG= $cflags/;
- s/^DEPFLAG=.*$/DEPFLAG=$depflags/;
--- /dev/null
+{\rtf1\ansi\ansicpg1252\cocoartf1561\cocoasubrtf200
+{\fonttbl\f0\fswiss\fcharset0 Helvetica;\f1\fnil\fcharset0 LucidaGrande-Bold;\f2\fnil\fcharset0 LucidaGrande;
+\f3\fnil\fcharset0 Monaco;}
+{\colortbl;\red255\green255\blue255;}
+{\*\expandedcolortbl;;}
+\margl1440\margr1440\vieww10540\viewh8400\viewkind0
+\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
+
+\f0\fs28 \cf0 Congratulations!
+\fs24
+\f1\b\fs28 Python $FULL_VERSION for macOS $MACOSX_DEPLOYMENT_TARGET
+\f2\b0 was successfully installed.
+\fs24 \
+\
+One more thing: to verify the identity of secure network connections, this Python needs a set of SSL root certificates. You can download and install a current curated set from {\field{\*\fldinst{HYPERLINK "https://pypi.org/project/certifi/"}}{\fldrslt the Certifi project}} by double-clicking on the
+\f3 Install Certificates
+\f2 icon in {\field{\*\fldinst{HYPERLINK "file://localhost/Applications/Python%20$VERSION/"}}{\fldrslt the Finder window}}. See {\field{\*\fldinst{HYPERLINK "file://localhost/Applications/Python%20$VERSION/ReadMe.rtf"}}{\fldrslt the
+\f3 ReadMe
+\f2 file}} for more information.\
+}
\ No newline at end of file
-{\rtf1\ansi\ansicpg1252\cocoartf1504\cocoasubrtf750
+{\rtf1\ansi\ansicpg1252\cocoartf1561\cocoasubrtf200
{\fonttbl\f0\fswiss\fcharset0 Helvetica;\f1\fmodern\fcharset0 CourierNewPSMT;}
{\colortbl;\red255\green255\blue255;}
{\*\expandedcolortbl;;}
\margl1440\margr1440\vieww13380\viewh14600\viewkind0
\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
-\f0\fs24 \cf0 This package will install Python $FULL_VERSION for Mac OS X $MACOSX_DEPLOYMENT_TARGET for the following architecture(s): $ARCHITECTURES.\
+\f0\fs24 \cf0 This package will install Python $FULL_VERSION for macOS $MACOSX_DEPLOYMENT_TARGET for the following architecture(s): $ARCHITECTURES.\
\
\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
-\b \cf0 \ul \ulc0 Which installer variant should I use?
+\b \cf0 \ul \ulc0 Which installer variant should I use? [CHANGED in 3.6.5]
\b0 \ulnone \
\
\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
\b \cf0 **NEW**
-\b0 For Python 3.6, the python.org website now provides only one installer variant for download: one that installs a
+\b0 With Python 3.6.5, the python.org website now provides two installer variants for download: one that installs a
+\i 64-bit-only
+\i0 Python capable of running on
+\i macOS 10.9 (Mavericks)
+\i0 or later; and one that installs a
\i 64-bit/32-bit Intel
\i0 Python capable of running on
-\i Mac OS X 10.6 (Snow Leopard)
-\i0 or later. This ReadMe was installed with the
+\i macOS 10.6 (Snow Leopard)
+\i0 or later. (This ReadMe was installed with the
\i $MACOSX_DEPLOYMENT_TARGET
-\i0 variant. By default, Python will automatically run in 64-bit mode if your system supports it. The Python installed by this installer is built with private copies of some third-party libraries not included with or newer than those in OS X itself. The list of these libraries is included at the end of the License.rtf file.
-\b \ul \
-\
-Certificate verification and OpenSSL\
+\i0 variant.) Previous Python 3.6.x releases only provided the 10.6 or later installer. If you are running on macOS 10.9 or later and if you have no need for compatibility with older systems, use the 10.9 variant. Use the 10.6 variant if you are running on macOS 10.6 through 10.8, if you need to maintain compatibility with previous 3.6.x releases, or if you want to produce standalone applications that can run on systems from 10.6. The Pythons installed by these installers are built with private copies of some third-party libraries not included with or newer than those in macOS itself. The list of these libraries varies by installer variant and is included at the end of the License.rtf file.\
\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
-\b0 \cf0 \ulnone \
-\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
+\b \cf0 \ul \ulc0 \
+Certificate verification and OpenSSL\
-\b \cf0 **NEW**
-\b0 This variant of Python 3.6 now includes its own private copy of OpenSSL 1.0.2. Unlike previous releases, the deprecated Apple-supplied OpenSSL libraries are no longer used. This also means that the trust certificates in system and user keychains managed by the
+\b0 \ulnone \
+This variant of Python 3.6 now includes its own private copy of OpenSSL 1.0.2. Unlike previous releases, the deprecated Apple-supplied OpenSSL libraries are no longer used. This also means that the trust certificates in system and user keychains managed by the
\i Keychain Access
\i0 application and the
\i security
\i0 command line utility are no longer used as defaults by the Python
\f1 ssl
-\f0 module. For 3.6.0, a sample command script is included in
+\f0 module. A sample command script is included in
\f1 /Applications/Python 3.6
\f0 to install a curated bundle of default root certificates from the third-party
\f1 certifi
\f1 pip
\f0 included with the Python 3.6 installer has its own default certificate store for verifying download connections.\
\
-\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
-\b \cf0 \ul Update your version of Tcl/Tk to use IDLE or other Tk applications
+\b \ul Using IDLE or other Tk applications [NEW/CHANGED in 3.6.5]
\b0 \ulnone \
\
-To use IDLE or other programs that use the Tkinter graphical user interface toolkit, you need to install a newer third-party version of the
+The 10.9+ installer variant comes with its own private version of Tcl/Tk 8.6. It does not use system-supplied or third-party supplied versions of Tcl/Tk.\
+\
+For the 10.6+ variant, you continue to need to install a newer third-party version of the
\i Tcl/Tk
-\i0 frameworks. Visit {\field{\*\fldinst{HYPERLINK "https://www.python.org/download/mac/tcltk/"}}{\fldrslt https://www.python.org/download/mac/tcltk/}} for current information about supported and recommended versions of
+\i0 8.5 (not 8.6) frameworks to use IDLE or other programs that use the Tkinter graphical user interface toolkit. Visit {\field{\*\fldinst{HYPERLINK "https://www.python.org/download/mac/tcltk/"}}{\fldrslt https://www.python.org/download/mac/tcltk/}} for current information about supported and recommended versions of
\i Tcl/Tk
-\i0 for this version of Python and of Mac OS X. For the initial release of Python 3.6, the installer is still linked with Tcl/Tk 8.5.\
+\i0 for this version of Python and of macOS.\
\b \ul \
Other changes\
-{\rtf1\ansi\ansicpg1252\cocoartf1504
+{\rtf1\ansi\ansicpg1252\cocoartf1561\cocoasubrtf200
\cocoascreenfonts1{\fonttbl\f0\fswiss\fcharset0 Helvetica;}
{\colortbl;\red255\green255\blue255;}
-{\*\expandedcolortbl;\csgray\c100000;}
+{\*\expandedcolortbl;;}
\paperw11905\paperh16837\margl1440\margr1440\vieww12200\viewh10880\viewkind0
-\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\partightenfactor0
+\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
\f0\fs24 \cf0 This package will install
\b Python $FULL_VERSION
\b0 for
-\b Mac OS X $MACOSX_DEPLOYMENT_TARGET
+\b macOS $MACOSX_DEPLOYMENT_TARGET
\b0 .\
-\
+\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\partightenfactor0
+\cf0 \
-\b Python for Mac OS X
-\b0 consists of the Python programming language interpreter, plus a set of programs to allow easy access to it for Mac OS X users including an integrated development environment
+\b Python for macOS
+\b0 consists of the Python programming language interpreter, plus a set of programs to allow easy access to it for macOS users including an integrated development environment
\b IDLE
\b0 .\
\
-\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\partightenfactor0
-
-\b \cf0 NEW:
-\b0 There are important changes in this release regarding network security and trust certificates. Please see the ReadMe for more details.\
-\
-\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\partightenfactor0
-\b \cf0 IMPORTANT:
-\b0
-\b IDLE
-\b0 and other programs using the
-\b tkinter
-\b0 graphical user interface toolkit require specific versions of the
-\b Tcl/Tk
-\b0 platform independent windowing toolkit. Visit {\field{\*\fldinst{HYPERLINK "https://www.python.org/download/mac/tcltk/"}}{\fldrslt https://www.python.org/download/mac/tcltk/}} for current information on supported and recommended versions of Tcl/Tk for this version of Python and Mac OS X.}
\ No newline at end of file
+\b NEW in 3.6.5:
+\b0 two installer variants (10.9+ 64-bit-only, 10.6+ 64-/32-bit), built-in Tcl/Tk 8.6 support in the 10.9+ variant (no additional third-party downloads!)\
+}
\ No newline at end of file
# make link in /Applications/Python m.n/ for Finder users
if [ -d "${APPDIR}" ]; then
ln -fhs "${FWK_DOCDIR}/index.html" "${APPDIR}/Python Documentation.html"
+ open "${APPDIR}" || true # open the applications folder
fi
# make share/doc link in framework for command line users
--- /dev/null
+Fix build failure with +quartz variant on OS X 10.8 and 10.9.
+Even though Gestalt was deprecated in OS X 10.8, it should work fine
+through OS X 10.9, and its replacement NSOperatingSystemVersion was
+not introduced until OS X 10.10.
+
+Patch from MacPorts project and reported upstream:
+https://trac.macports.org/ticket/55649
+--- tk8.6.8/macosx/tkMacOSXXStubs.c.orig 2017-12-06 09:25:08.000000000 -0600
++++ tk8.6.8-patched/macosx/tkMacOSXXStubs.c 2018-01-06 19:34:17.000000000 -0600
+@@ -175,7 +175,7 @@
+ {
+ int major, minor, patch;
+
+-#if MAC_OS_X_VERSION_MIN_REQUIRED < 1080
++#if MAC_OS_X_VERSION_MIN_REQUIRED < 101000
+ Gestalt(gestaltSystemVersionMajor, (SInt32*)&major);
+ Gestalt(gestaltSystemVersionMinor, (SInt32*)&minor);
+ Gestalt(gestaltSystemVersionBugFix, (SInt32*)&patch);
<key>CFBundleExecutable</key>
<string>IDLE</string>
<key>CFBundleGetInfoString</key>
- <string>%version%, © 2001-2017 Python Software Foundation</string>
+ <string>%version%, © 2001-2018 Python Software Foundation</string>
<key>CFBundleIconFile</key>
<string>IDLE.icns</string>
<key>CFBundleIdentifier</key>
<key>CFBundleExecutable</key>
<string>Python Launcher</string>
<key>CFBundleGetInfoString</key>
- <string>%VERSION%, © 2001-2017 Python Software Foundation</string>
+ <string>%VERSION%, © 2001-2018 Python Software Foundation</string>
<key>CFBundleIconFile</key>
<string>PythonLauncher.icns</string>
<key>CFBundleIdentifier</key>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundleLongVersionString</key>
- <string>%version%, (c) 2001-2017 Python Software Foundation.</string>
+ <string>%version%, (c) 2001-2018 Python Software Foundation.</string>
<key>CFBundleName</key>
<string>Python</string>
<key>CFBundlePackageType</key>
<key>CFBundlePackageType</key>
<string>FMWK</string>
<key>CFBundleShortVersionString</key>
- <string>%VERSION%, (c) 2001-2017 Python Software Foundation.</string>
+ <string>%VERSION%, (c) 2001-2018 Python Software Foundation.</string>
<key>CFBundleLongVersionString</key>
- <string>%VERSION%, (c) 2001-2017 Python Software Foundation.</string>
+ <string>%VERSION%, (c) 2001-2018 Python Software Foundation.</string>
<key>CFBundleSignature</key>
<string>????</string>
<key>CFBundleVersion</key>
$(MAKE) coverage-lcov
# Run "Argument Clinic" over all source files
-# (depends on python having already been built)
.PHONY=clinic
-clinic: check-clean-src $(BUILDPYTHON) $(srcdir)/Modules/_blake2/blake2s_impl.c
- $(RUNSHARED) $(PYTHON_FOR_BUILD) ./Tools/clinic/clinic.py --make
+clinic: check-clean-src $(srcdir)/Modules/_blake2/blake2s_impl.c
+ $(PYTHON_FOR_REGEN) ./Tools/clinic/clinic.py --make
# Build the interpreter
$(BUILDPYTHON): Programs/python.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY)
$(CC) -c $(CCSHARED) $(PY_CORE_CFLAGS) -o $@ $<
# blake2s is auto-generated from blake2b
-$(srcdir)/Modules/_blake2/blake2s_impl.c: $(BUILDPYTHON) $(srcdir)/Modules/_blake2/blake2b_impl.c $(srcdir)/Modules/_blake2/blake2b2s.py
- $(RUNSHARED) $(PYTHON_FOR_BUILD) $(srcdir)/Modules/_blake2/blake2b2s.py
- $(RUNSHARED) $(PYTHON_FOR_BUILD) $(srcdir)/Tools/clinic/clinic.py -f $@
+$(srcdir)/Modules/_blake2/blake2s_impl.c: $(srcdir)/Modules/_blake2/blake2b_impl.c $(srcdir)/Modules/_blake2/blake2b2s.py
+ $(PYTHON_FOR_REGEN) $(srcdir)/Modules/_blake2/blake2b2s.py
+ $(PYTHON_FOR_REGEN) $(srcdir)/Tools/clinic/clinic.py -f $@
# Build the shared modules
# Under GNU make, MAKEFLAGS are sorted and normalized; the 's' for
############################################################################
# Regenerate all generated files
-regen-all: regen-opcode regen-opcode-targets regen-typeslots regen-grammar regen-ast regen-importlib
+regen-all: regen-opcode regen-opcode-targets regen-typeslots regen-grammar \
+ regen-ast regen-importlib clinic
############################################################################
# Special rules for object files
Jonathan Dasteel
Pierre-Yves David
A. Jesse Jiryu Davis
+Jake Davis
+Jamie (James C.) Davis
Merlijn van Deen
John DeGood
Ned Deily
Stephen Hansen
Barry Hantman
Lynda Hardman
+Bar Harel
Derek Harland
Jason Harper
David Harrigan
Malte Helmert
Lance Finn Helsten
Jonathan Hendry
+Nathan Henrie
Michael Henry
James Henstridge
Kasun Herath
Hannu Krosing
Andrej Krpic
Ivan Krstić
+Anselm Kruis
Steven Kryskalla
Andrew Kuchling
Dave Kuhlman
Per Spilling
Joshua Spoerri
Noah Spurrier
+Zackery Spytz
Nathan Srebro
RajGopal Srinivasan
Tage Stabell-Kulo
Arfrever Frehtes Taifersar Arahesis
Hideaki Takahashi
Takase Arihiro
+Licht Takeuchi
Indra Talip
Neil Tallim
Geoff Talvola
Python News
+++++++++++
+What's New in Python 3.6.5 final?
+=================================
+
+*Release date: 2018-03-28*
+
+Tests
+-----
+
+- bpo-32872: Avoid regrtest compatibility issue with namespace packages.
+
+Build
+-----
+
+- bpo-33163: Upgrade pip to 9.0.3 and setuptools to v39.0.1.
+
+
+What's New in Python 3.6.5 release candidate 1?
+===============================================
+
+*Release date: 2018-03-13*
+
+Security
+--------
+
+- bpo-33001: Minimal fix to prevent buffer overrun in os.symlink on Windows
+
+- bpo-32981: Regexes in difflib and poplib were vulnerable to catastrophic
+ backtracking. These regexes formed potential DOS vectors (REDOS). They
+ have been refactored. This resolves CVE-2018-1060 and CVE-2018-1061. Patch
+ by Jamie Davis.
+
+Core and Builtins
+-----------------
+
+- bpo-33026: Fixed jumping out of "with" block by setting f_lineno.
+
+- bpo-17288: Prevent jumps from 'return' and 'exception' trace events.
+
+- bpo-32889: Update Valgrind suppression list to account for the rename of
+ ``Py_ADDRESS_IN_RANG`` to ``address_in_range``.
+
+- bpo-32650: Pdb and other debuggers dependent on bdb.py will correctly step
+ over (next command) native coroutines. Patch by Pablo Galindo.
+
+- bpo-32685: Improve suggestion when the Python 2 form of print statement is
+ either present on the same line as the header of a compound statement or
+ else terminated by a semi-colon instead of a newline. Patch by Nitish
+ Chandra.
+
+- bpo-32583: Fix possible crashing in builtin Unicode decoders caused by
+ write out-of- bound errors when using customized decode error handlers.
+
+- bpo-26163: Improved frozenset() hash to create more distinct hash values
+ when faced with datasets containing many similar values.
+
+- bpo-27169: The ``__debug__`` constant is now optimized out at compile
+ time. This fixes also bpo-22091.
+
+- bpo-32329: ``sys.flags.hash_randomization`` is now properly set to 0 when
+ hash randomization is turned off by ``PYTHONHASHSEED=0``.
+
+- bpo-30416: The optimizer is now protected from spending much time doing
+ complex calculations and consuming much memory for creating large
+ constants in constant folding.
+
+- bpo-18533: ``repr()`` on a dict containing its own ``values()`` or
+ ``items()`` no longer raises ``RecursionError``; OrderedDict similarly.
+ Instead, use ``...``, as for other recursive structures. Patch by Ben
+ North.
+
+- bpo-32028: Leading whitespace is now correctly ignored when generating
+ suggestions for converting Py2 print statements to Py3 builtin print
+ function calls. Patch by Sanyam Khurana.
+
+- bpo-32137: The repr of deeply nested dict now raises a RecursionError
+ instead of crashing due to a stack overflow.
+
+Library
+-------
+
+- bpo-33064: lib2to3 now properly supports trailing commas after ``*args``
+ and ``**kwargs`` in function signatures.
+
+- bpo-31804: Avoid failing in multiprocessing.Process if the standard
+ streams are closed or None at exit.
+
+- bpo-33037: Skip sending/receiving data after SSL transport closing.
+
+- bpo-30353: Fix ctypes pass-by-value for structs on 64-bit Cygwin/MinGW.
+
+- bpo-33009: Fix inspect.signature() for single-parameter partialmethods.
+
+- bpo-32969: Expose several missing constants in zlib and fix corresponding
+ documentation.
+
+- bpo-32713: Fixed tarfile.itn handling of out-of-bounds float values. Patch
+ by Joffrey Fuhrer.
+
+- bpo-30622: The ssl module now detects missing NPN support in LibreSSL.
+
+- bpo-32922: dbm.open() now encodes filename with the filesystem encoding
+ rather than default encoding.
+
+- bpo-32859: In ``os.dup2``, don't check every call whether the ``dup3``
+ syscall exists or not.
+
+- bpo-21060: Rewrite confusing message from setup.py upload from "No dist
+ file created in earlier command" to the more helpful "Must create and
+ upload files in one command".
+
+- bpo-32857: In :mod:`tkinter`, ``after_cancel(None)`` now raises a
+ :exc:`ValueError` instead of canceling the first scheduled function.
+ Patch by Cheryl Sabella.
+
+- bpo-32852: Make sure sys.argv remains as a list when running trace.
+
+- bpo-32841: Fixed `asyncio.Condition` issue which silently ignored
+ cancellation after notifying and cancelling a conditional lock. Patch by
+ Bar Harel.
+
+- bpo-31787: Fixed refleaks of ``__init__()`` methods in various modules.
+ (Contributed by Oren Milman)
+
+- bpo-30157: Fixed guessing quote and delimiter in csv.Sniffer.sniff() when
+ only the last field is quoted. Patch by Jake Davis.
+
+- bpo-32394: socket: Remove TCP_FASTOPEN, TCP_KEEPCNT flags on older version
+ Windows during run-time.
+
+- bpo-32777: Fix a rare but potential pre-exec child process deadlock in
+ subprocess on POSIX systems when marking file descriptors inheritable on
+ exec in the child process. This bug appears to have been introduced in
+ 3.4.
+
+- bpo-32647: The ctypes module used to depend on indirect linking for
+ dlopen. The shared extension is now explicitly linked against libdl on
+ platforms with dl.
+
+- bpo-32734: Fixed ``asyncio.Lock()`` safety issue which allowed acquiring
+ and locking the same lock multiple times, without it being free. Patch by
+ Bar Harel.
+
+- bpo-32727: Do not include name field in SMTP envelope from address. Patch
+ by Stéphane Wirtel
+
+- bpo-27931: Fix email address header parsing error when the username is an
+ empty quoted string. Patch by Xiang Zhang.
+
+- bpo-32304: distutils' upload command no longer corrupts tar files ending
+ with a CR byte, and no longer tries to convert CR to CRLF in any of the
+ upload text fields.
+
+- bpo-32502: uuid.uuid1 no longer raises an exception if a 64-bit hardware
+ address is encountered.
+
+- bpo-31848: Fix the error handling in Aifc_read.initfp() when the SSND
+ chunk is not found. Patch by Zackery Spytz.
+
+- bpo-32555: On FreeBSD and Solaris, os.strerror() now always decode the
+ byte string from the current locale encoding, rather than using
+ ASCII/surrogateescape in some cases.
+
+- bpo-32521: The nis module is now compatible with new libnsl and headers
+ location.
+
+- bpo-32473: Improve ABCMeta._dump_registry() output readability
+
+- bpo-32521: glibc has removed Sun RPC. Use replacement libtirpc headers and
+ library in nis module.
+
+- bpo-32228: Ensure that ``truncate()`` preserves the file position (as
+ reported by ``tell()``) after writes longer than the buffer size.
+
+- bpo-26133: Don't unsubscribe signals in asyncio UNIX event loop on
+ interpreter shutdown.
+
+- bpo-32185: The SSL module no longer sends IP addresses in SNI TLS
+ extension on platforms with OpenSSL 1.0.2+ or inet_pton.
+
+- bpo-32323: :func:`urllib.parse.urlsplit()` does not convert zone-id
+ (scope) to lower case for scoped IPv6 addresses in hostnames now.
+
+- bpo-32302: Fix bdist_wininst of distutils for CRT v142: it binary
+ compatible with CRT v140.
+
+- bpo-32255: A single empty field is now always quoted when written into a
+ CSV file. This allows to distinguish an empty row from a row consisting of
+ a single empty field. Patch by Licht Takeuchi.
+
+- bpo-32277: Raise ``NotImplementedError`` instead of ``SystemError`` on
+ platforms where ``chmod(..., follow_symlinks=False)`` is not supported.
+ Patch by Anthony Sottile.
+
+- bpo-32199: The getnode() ip getter now uses 'ip link' instead of 'ip link
+ list'.
+
+- bpo-27456: Ensure TCP_NODELAY is set on Linux. Tests by Victor Stinner.
+
+- bpo-31900: The :func:`locale.localeconv` function now sets temporarily the
+ ``LC_CTYPE`` locale to the ``LC_NUMERIC`` locale to decode
+ ``decimal_point`` and ``thousands_sep`` byte strings if they are non-ASCII
+ or longer than 1 byte, and the ``LC_NUMERIC`` locale is different than the
+ ``LC_CTYPE`` locale. This temporary change affects other threads.
+
+ Same change for the :meth:`str.format` method when formatting a number
+ (:class:`int`, :class:`float`, :class:`float` and subclasses) with the
+ ``n`` type (ex: ``'{:n}'.format(1234)``).
+
+- bpo-31802: Importing native path module (``posixpath``, ``ntpath``) now
+ works even if the ``os`` module still is not imported.
+
+Documentation
+-------------
+
+- bpo-17232: Clarify docs for -O and -OO. Patch by Terry Reedy.
+
+- bpo-32800: Update link to w3c doc for xml default namespaces.
+
+- bpo-8722: Document :meth:`__getattr__` behavior when property :meth:`get`
+ method raises :exc:`AttributeError`.
+
+- bpo-32614: Modify RE examples in documentation to use raw strings to
+ prevent :exc:`DeprecationWarning` and add text to REGEX HOWTO to highlight
+ the deprecation.
+
+- bpo-31972: Improve docstrings for `pathlib.PurePath` subclasses.
+
+- bpo-17799: Explain real behaviour of sys.settrace and sys.setprofile and
+ their C-API counterparts regarding which type of events are received in
+ each function. Patch by Pablo Galindo Salgado.
+
+Tests
+-----
+
+- bpo-32517: Fix failing ``test_asyncio`` on macOS 10.12.2+ due to transport
+ of ``KqueueSelector`` loop was not being closed.
+
+- bpo-32721: Fix test_hashlib to not fail if the _md5 module is not built.
+
+- bpo-32252: Fix faulthandler_suppress_crash_report() used to prevent core
+ dump files when testing crashes. getrlimit() returns zero on success.
+
+- bpo-31518: Debian Unstable has disabled TLS 1.0 and 1.1 for
+ SSLv23_METHOD(). Change TLS/SSL protocol of some tests to PROTOCOL_TLS or
+ PROTOCOL_TLSv1_2 to make them pass on Debian.
+
+Build
+-----
+
+- bpo-32635: Fix segfault of the crypt module when libxcrypt is provided
+ instead of libcrypt at the system.
+
+Windows
+-------
+
+- bpo-33016: Fix potential use of uninitialized memory in
+ nt._getfinalpathname
+
+- bpo-32903: Fix a memory leak in os.chdir() on Windows if the current
+ directory is set to a UNC path.
+
+- bpo-31966: Fixed WindowsConsoleIO.write() for writing empty data.
+
+- bpo-32409: Ensures activate.bat can handle Unicode contents.
+
+- bpo-32457: Improves handling of denormalized executable path when
+ launching Python.
+
+- bpo-32370: Use the correct encoding for ipconfig output in the uuid
+ module. Patch by Segev Finer.
+
+- bpo-29248: Fix :func:`os.readlink` on Windows, which was mistakenly
+ treating the ``PrintNameOffset`` field of the reparse data buffer as a
+ number of characters instead of bytes. Patch by Craig Holmquist and SSE4.
+
+- bpo-32588: Create standalone _distutils_findvs module.
+
+macOS
+-----
+
+- bpo-32726: Provide an additional, more modern macOS installer variant that
+ supports macOS 10.9+ systems in 64-bit mode only. Upgrade the supplied
+ third-party libraries to OpenSSL 1.0.2n, XZ 5.2.3, and SQLite 3.22.0. The
+ 10.9+ installer now links with and supplies its own copy of Tcl/Tk 8.6.8.
+
+IDLE
+----
+
+- bpo-32984: Set ``__file__`` while running a startup file. Like Python,
+ IDLE optionally runs one startup file in the Shell window before
+ presenting the first interactive input prompt. For IDLE, ``-s`` runs a
+ file named in environmental variable :envvar:`IDLESTARTUP` or
+ :envvar:`PYTHONSTARTUP`; ``-r file`` runs ``file``. Python sets
+ ``__file__`` to the startup file name before running the file and unsets
+ it before the first prompt. IDLE now does the same when run normally,
+ without the ``-n`` option.
+
+- bpo-32940: Simplify and rename StringTranslatePseudoMapping in pyparse.
+
+- bpo-32916: Change ``str`` to ``code`` in pyparse.
+
+- bpo-32905: Remove unused code in pyparse module.
+
+- bpo-32874: Add tests for pyparse.
+
+- bpo-32837: Using the system and place-dependent default encoding for
+ open() is a bad idea for IDLE's system and location-independent files.
+
+- bpo-32826: Add "encoding=utf-8" to open() in IDLE's test_help_about. GUI
+ test test_file_buttons() only looks at initial ascii-only lines, but
+ failed on systems where open() defaults to 'ascii' because readline()
+ internally reads and decodes far enough ahead to encounter a non-ascii
+ character in CREDITS.txt.
+
+- bpo-32765: Update configdialog General tab docstring to add new widgets to
+ the widget list.
+
+Tools/Demos
+-----------
+
+- bpo-24960: 2to3 and lib2to3 can now read pickled grammar files using
+ pkgutil.get_data() rather than probing the filesystem. This lets 2to3 and
+ lib2to3 work when run from a zipfile.
+
+- bpo-32222: Fix pygettext not extracting docstrings for functions with type
+ annotated arguments. Patch by Toby Harradine.
+
+C API
+-----
+
+- bpo-29084: Undocumented C API for OrderedDict has been excluded from the
+ limited C API. It was added by mistake and actually never worked in the
+ limited C API.
+
+
What's New in Python 3.6.4 final?
=================================
file as a script.
.TP
.B \-O
-Turn on basic optimizations. Given twice, causes docstrings to be discarded.
+Remove assert statements and any code conditional on the value of
+__debug__; augment the filename for compiled (bytecode) files by
+adding .opt-1 before the .pyc extension.
.TP
.B \-OO
-Discard docstrings in addition to the \fB-O\fP optimizations.
+Do \fB-O\fP and also discard docstrings; change the filename for
+compiled (bytecode) files by adding .opt-2 before the .pyc extension.
.TP
.B \-q
Do not print the version and copyright messages. These messages are
# ./python -E ./Lib/test/regrtest.py -u gui,network
#
# You must edit Objects/obmalloc.c and uncomment Py_USING_MEMORY_DEBUGGER
-# to use the preferred suppressions with Py_ADDRESS_IN_RANGE.
+# to use the preferred suppressions with address_in_range.
#
# If you do not want to recompile Python, you can uncomment
# suppressions for PyObject_Free and PyObject_Realloc.
{
ADDRESS_IN_RANGE/Invalid read of size 4
Memcheck:Addr4
- fun:Py_ADDRESS_IN_RANGE
+ fun:address_in_range
}
{
ADDRESS_IN_RANGE/Invalid read of size 4
Memcheck:Value4
- fun:Py_ADDRESS_IN_RANGE
+ fun:address_in_range
}
{
ADDRESS_IN_RANGE/Invalid read of size 8 (x86_64 aka amd64)
Memcheck:Value8
- fun:Py_ADDRESS_IN_RANGE
+ fun:address_in_range
}
{
ADDRESS_IN_RANGE/Conditional jump or move depends on uninitialised value
Memcheck:Cond
- fun:Py_ADDRESS_IN_RANGE
+ fun:address_in_range
}
#
return 0;
}
+
static int
future_init(FutureObj *fut, PyObject *loop)
{
int is_true;
_Py_IDENTIFIER(get_debug);
+ // Same to FutureObj_clear() but not clearing fut->dict
+ Py_CLEAR(fut->fut_loop);
+ Py_CLEAR(fut->fut_callbacks);
+ Py_CLEAR(fut->fut_result);
+ Py_CLEAR(fut->fut_exception);
+ Py_CLEAR(fut->fut_source_tb);
+
+ fut->fut_state = STATE_PENDING;
+ fut->fut_log_tb = 0;
+ fut->fut_blocking = 0;
+
if (loop == Py_None) {
loop = _PyObject_CallNoArg(asyncio_get_event_loop);
if (loop == NULL) {
else {
Py_INCREF(loop);
}
- Py_XSETREF(fut->fut_loop, loop);
+ fut->fut_loop = loop;
res = _PyObject_CallMethodId(fut->fut_loop, &PyId_get_debug, NULL);
if (res == NULL) {
return -1;
}
if (is_true) {
- Py_XSETREF(fut->fut_source_tb, _PyObject_CallNoArg(traceback_extract_stack));
+ fut->fut_source_tb = _PyObject_CallNoArg(traceback_extract_stack);
if (fut->fut_source_tb == NULL) {
return -1;
}
}
- Py_XSETREF(fut->fut_callbacks, PyList_New(0));
+ fut->fut_callbacks = PyList_New(0);
if (fut->fut_callbacks == NULL) {
return -1;
}
return -1;
}
- self->task_fut_waiter = NULL;
+ Py_CLEAR(self->task_fut_waiter);
self->task_must_cancel = 0;
self->task_log_destroy_pending = 1;
Py_INCREF(coro);
- self->task_coro = coro;
+ Py_XSETREF(self->task_coro, coro);
if (task_call_step_soon(self, NULL)) {
return -1;
self->bzs_avail_in_real = 0;
self->input_buffer = NULL;
self->input_buffer_size = 0;
- self->unused_data = PyBytes_FromStringAndSize(NULL, 0);
+ Py_XSETREF(self->unused_data, PyBytes_FromStringAndSize(NULL, 0));
if (self->unused_data == NULL)
goto error;
if (PyErr_Occurred())
return NULL;
- if (self->num_fields > 0 && self->rec_size == 0) {
+ if (self->num_fields > 0 && self->rec_len == 0) {
if (dialect->quoting == QUOTE_NONE) {
PyErr_Format(_csvstate_global->error_obj,
"single empty field record must be quoted");
((volatile Test *)&in)->third = 0x0badf00d;
}
+typedef struct {
+ unsigned int first;
+ unsigned int second;
+} TestReg;
+
+
+EXPORT(TestReg) last_tfrsuv_arg;
+
+
+EXPORT(void)
+_testfunc_reg_struct_update_value(TestReg in)
+{
+ last_tfrsuv_arg = in;
+ ((volatile TestReg *)&in)->first = 0x0badf00d;
+ ((volatile TestReg *)&in)->second = 0x0badf00d;
+}
+
+
EXPORT(void)testfunc_array(int values[4])
{
printf("testfunc_array %d %d %d %d\n",
}
#endif
+#if (defined(__x86_64__) && (defined(__MINGW64__) || defined(__CYGWIN__))) || \
+ defined(__aarch64__)
+#define CTYPES_PASS_BY_REF_HACK
+#define POW2(x) (((x & ~(x - 1)) == x) ? x : 0)
+#define IS_PASS_BY_REF(x) (x > 8 || !POW2(x))
+#endif
+
/*
* Requirements, must be ensured by the caller:
* - argtuple is tuple of arguments
}
for (i = 0; i < argcount; ++i) {
atypes[i] = args[i].ffi_type;
- if (atypes[i]->type == FFI_TYPE_STRUCT
- )
+#ifdef CTYPES_PASS_BY_REF_HACK
+ size_t size = atypes[i]->size;
+ if (IS_PASS_BY_REF(size)) {
+ void *tmp = alloca(size);
+ if (atypes[i]->type == FFI_TYPE_STRUCT)
+ memcpy(tmp, args[i].value.p, size);
+ else
+ memcpy(tmp, (void*)&args[i].value, size);
+
+ avalues[i] = tmp;
+ }
+ else
+#endif
+ if (atypes[i]->type == FFI_TYPE_STRUCT)
avalues[i] = (void *)args[i].value.p;
else
avalues[i] = (void *)&args[i].value;
use_xy = TRUE;
break;
default:
- PyErr_SetString(PyExc_TypeError, "insch requires 1 or 4 arguments");
+ PyErr_SetString(PyExc_TypeError, "insch requires 1 to 4 arguments");
return NULL;
}
rtn = mvwinch(self->win,y,x);
break;
default:
- PyErr_SetString(PyExc_TypeError, "inch requires 0 or 2 arguments");
+ PyErr_SetString(PyExc_TypeError, "inch requires 0 to 2 arguments");
return NULL;
}
return PyLong_FromUnsignedLong(rtn);
PyErr_SetString(PyExc_ValueError, "fromutc: tz.dst() gave"
"inconsistent results; cannot convert");
- /* fall thru to failure */
+ /* fall through to failure */
Fail:
Py_XDECREF(off);
Py_XDECREF(dst);
_dbm.open as dbmopen
- filename: str
+ filename: unicode
The filename to open.
flags: str="r"
[clinic start generated code]*/
static PyObject *
-dbmopen_impl(PyObject *module, const char *filename, const char *flags,
+dbmopen_impl(PyObject *module, PyObject *filename, const char *flags,
int mode)
-/*[clinic end generated code: output=5fade8cf16e0755f input=226334bade5764e6]*/
+/*[clinic end generated code: output=9527750f5df90764 input=376a9d903a50df59]*/
{
int iflags;
"arg 2 to open should be 'r', 'w', 'c', or 'n'");
return NULL;
}
- return newdbmobject(filename, iflags, mode);
+
+ PyObject *filenamebytes = PyUnicode_EncodeFSDefault(filename);
+ if (filenamebytes == NULL) {
+ return NULL;
+ }
+ const char *name = PyBytes_AS_STRING(filenamebytes);
+ if (strlen(name) != (size_t)PyBytes_GET_SIZE(filenamebytes)) {
+ Py_DECREF(filenamebytes);
+ PyErr_SetString(PyExc_ValueError, "embedded null character");
+ return NULL;
+ }
+ PyObject *self = newdbmobject(name, iflags, mode);
+ Py_DECREF(filenamebytes);
+ return self;
}
static PyMethodDef dbmmodule_methods[] = {
/*[clinic input]
_gdbm.open as dbmopen
- filename as name: str
+ filename: unicode
flags: str="r"
mode: int(py_default="0o666") = 0o666
/
[clinic start generated code]*/
static PyObject *
-dbmopen_impl(PyObject *module, const char *name, const char *flags, int mode)
-/*[clinic end generated code: output=31aa1bafdf5da688 input=55563cd60e51984a]*/
+dbmopen_impl(PyObject *module, PyObject *filename, const char *flags,
+ int mode)
+/*[clinic end generated code: output=9527750f5df90764 input=3be0b0875974b928]*/
{
int iflags;
}
}
- return newdbmobject(name, iflags, mode);
+ PyObject *filenamebytes = PyUnicode_EncodeFSDefault(filename);
+ if (filenamebytes == NULL) {
+ return NULL;
+ }
+ const char *name = PyBytes_AS_STRING(filenamebytes);
+ if (strlen(name) != (size_t)PyBytes_GET_SIZE(filenamebytes)) {
+ Py_DECREF(filenamebytes);
+ PyErr_SetString(PyExc_ValueError, "embedded null character");
+ return NULL;
+ }
+ PyObject *self = newdbmobject(name, iflags, mode);
+ Py_DECREF(filenamebytes);
+ return self;
}
static const char dbmmodule_open_flags[] = "rwcn"
return -1;
}
- self->name = name_obj;
- Py_INCREF(self->name);
+ Py_INCREF(name_obj);
+ Py_XSETREF(self->name, name_obj);
if (data_obj) {
if (view.len >= HASHLIB_GIL_MINSIZE) {
* This macro generates constructor function definitions for specific
* hash algorithms. These constructors are much faster than calling
* the generic one passing it a python string and are noticeably
- * faster than calling a python new() wrapper. Thats important for
+ * faster than calling a python new() wrapper. That is important for
* code that wants to make hashes of a bunch of small strings.
* The first call will lazy-initialize, which reports an exception
* if initialization fails.
if (res == NULL)
goto end;
Py_CLEAR(res);
- _bufferedwriter_reset_buf(self);
}
/* TODO: align on block boundary and read buffer if needed? */
return n;
}
-/* `restore_pos` is 1 if we need to restore the raw stream position at
- the end, 0 otherwise. */
static PyObject *
_bufferedwriter_flush_unlocked(buffered *self)
{
goto error;
}
- _bufferedwriter_reset_buf(self);
end:
+ /* This ensures that after return from this function,
+ VALID_WRITE_BUFFER(self) returns false.
+
+ This is a required condition because when a tell() is called
+ after flushing and if VALID_READ_BUFFER(self) is false, we need
+ VALID_WRITE_BUFFER(self) to be false to have
+ RAW_OFFSET(self) == 0.
+
+ Issue: https://bugs.python.org/issue32228 */
+ _bufferedwriter_reset_buf(self);
Py_RETURN_NONE;
error:
if (!self->writable)
return err_mode("writing");
+ if (!b->len) {
+ return PyLong_FromLong(0);
+ }
if (b->len > BUFMAX)
len = BUFMAX;
else
RESULT(#i, x); \
} while (0)
- /* Numeric information */
- RESULT_STRING(decimal_point);
- RESULT_STRING(thousands_sep);
- x = copy_grouping(l->grouping);
- RESULT("grouping", x);
-
/* Monetary information */
RESULT_STRING(int_curr_symbol);
RESULT_STRING(currency_symbol);
RESULT_INT(n_sep_by_space);
RESULT_INT(p_sign_posn);
RESULT_INT(n_sign_posn);
+
+ /* Numeric information */
+ PyObject *decimal_point, *thousands_sep;
+ const char *grouping;
+ if (_Py_GetLocaleconvNumeric(&decimal_point,
+ &thousands_sep,
+ &grouping) < 0) {
+ goto failed;
+ }
+
+ if (PyDict_SetItemString(result, "decimal_point", decimal_point) < 0) {
+ Py_DECREF(decimal_point);
+ Py_DECREF(thousands_sep);
+ goto failed;
+ }
+ Py_DECREF(decimal_point);
+
+ if (PyDict_SetItemString(result, "thousands_sep", thousands_sep) < 0) {
+ Py_DECREF(thousands_sep);
+ goto failed;
+ }
+ Py_DECREF(thousands_sep);
+
+ x = copy_grouping(grouping);
+ RESULT("grouping", x);
+
return result;
failed:
self->needs_input = 1;
self->input_buffer = NULL;
self->input_buffer_size = 0;
- self->unused_data = PyBytes_FromStringAndSize(NULL, 0);
+ Py_XSETREF(self->unused_data, PyBytes_FromStringAndSize(NULL, 0));
if (self->unused_data == NULL)
goto error;
called. */
continue;
}
- if (_Py_set_inheritable((int)fd, 1, NULL) < 0)
+ if (_Py_set_inheritable_async_safe((int)fd, 1, NULL) < 0)
return -1;
}
return 0;
dup2() removes the CLOEXEC flag but we must do it ourselves if dup2()
would be a no-op (issue #10806). */
if (p2cread == 0) {
- if (_Py_set_inheritable(p2cread, 1, NULL) < 0)
+ if (_Py_set_inheritable_async_safe(p2cread, 1, NULL) < 0)
goto error;
}
else if (p2cread != -1)
POSIX_CALL(dup2(p2cread, 0)); /* stdin */
if (c2pwrite == 1) {
- if (_Py_set_inheritable(c2pwrite, 1, NULL) < 0)
+ if (_Py_set_inheritable_async_safe(c2pwrite, 1, NULL) < 0)
goto error;
}
else if (c2pwrite != -1)
POSIX_CALL(dup2(c2pwrite, 1)); /* stdout */
if (errwrite == 2) {
- if (_Py_set_inheritable(errwrite, 1, NULL) < 0)
+ if (_Py_set_inheritable_async_safe(errwrite, 1, NULL) < 0)
goto error;
}
else if (errwrite != -1)
#include <sys/poll.h>
#endif
+#ifndef MS_WINDOWS
+/* inet_pton */
+#include <arpa/inet.h>
+#endif
+
/* Don't warn about deprecated functions */
#ifdef __GNUC__
#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
#endif
#ifdef TLSEXT_TYPE_application_layer_protocol_negotiation
-# define HAVE_ALPN
+# define HAVE_ALPN 1
+#else
+# define HAVE_ALPN 0
+#endif
+
+/* We cannot rely on OPENSSL_NO_NEXTPROTONEG because LibreSSL 2.6.1 dropped
+ * NPN support but did not set OPENSSL_NO_NEXTPROTONEG for compatibility
+ * reasons. The check for TLSEXT_TYPE_next_proto_neg works with
+ * OpenSSL 1.0.1+ and LibreSSL.
+ * OpenSSL 1.1.1-pre1 dropped NPN but still has TLSEXT_TYPE_next_proto_neg.
+ */
+#ifdef OPENSSL_NO_NEXTPROTONEG
+# define HAVE_NPN 0
+#elif (OPENSSL_VERSION_NUMBER >= 0x10101000L) && !defined(LIBRESSL_VERSION_NUMBER)
+# define HAVE_NPN 0
+#elif defined(TLSEXT_TYPE_next_proto_neg)
+# define HAVE_NPN 1
+#else
+# define HAVE_NPN 0
#endif
#ifndef INVALID_SOCKET /* MS defines this */
typedef struct {
PyObject_HEAD
SSL_CTX *ctx;
-#if defined(OPENSSL_NPN_NEGOTIATED) && !defined(OPENSSL_NO_NEXTPROTONEG)
+#if HAVE_NPN
unsigned char *npn_protocols;
int npn_protocols_len;
#endif
-#ifdef HAVE_ALPN
+#if HAVE_ALPN
unsigned char *alpn_protocols;
int alpn_protocols_len;
#endif
SSL_set_mode(self->ssl, mode);
#if HAVE_SNI
- if (server_hostname != NULL)
- SSL_set_tlsext_host_name(self->ssl, server_hostname);
+ if (server_hostname != NULL) {
+/* Don't send SNI for IP addresses. We cannot simply use inet_aton() and
+ * inet_pton() here. inet_aton() may be linked weakly and inet_pton() isn't
+ * available on all platforms. Use OpenSSL's IP address parser. It's
+ * available since 1.0.2 and LibreSSL since at least 2.3.0. */
+ int send_sni = 1;
+#if OPENSSL_VERSION_NUMBER >= 0x10200000L
+ ASN1_OCTET_STRING *ip = a2i_IPADDRESS(server_hostname);
+ if (ip == NULL) {
+ send_sni = 1;
+ ERR_clear_error();
+ } else {
+ send_sni = 0;
+ ASN1_OCTET_STRING_free(ip);
+ }
+#elif defined(HAVE_INET_PTON)
+#ifdef ENABLE_IPV6
+ char packed[Py_MAX(sizeof(struct in_addr), sizeof(struct in6_addr))];
+#else
+ char packed[sizeof(struct in_addr)];
+#endif /* ENABLE_IPV6 */
+ if (inet_pton(AF_INET, server_hostname, packed)) {
+ send_sni = 0;
+#ifdef ENABLE_IPV6
+ } else if(inet_pton(AF_INET6, server_hostname, packed)) {
+ send_sni = 0;
+#endif /* ENABLE_IPV6 */
+ } else {
+ send_sni = 1;
+ }
+#endif /* HAVE_INET_PTON */
+ if (send_sni) {
+ SSL_set_tlsext_host_name(self->ssl, server_hostname);
+ }
+ }
#endif
/* If the socket is in non-blocking mode or timeout mode, set the BIO
return PyUnicode_FromString(version);
}
-#if defined(OPENSSL_NPN_NEGOTIATED) && !defined(OPENSSL_NO_NEXTPROTONEG)
+#if HAVE_NPN
/*[clinic input]
_ssl._SSLSocket.selected_npn_protocol
[clinic start generated code]*/
}
#endif
-#ifdef HAVE_ALPN
+#if HAVE_ALPN
/*[clinic input]
_ssl._SSLSocket.selected_alpn_protocol
[clinic start generated code]*/
return NULL;
}
self->ctx = ctx;
-#if defined(OPENSSL_NPN_NEGOTIATED) && !defined(OPENSSL_NO_NEXTPROTONEG)
+#if HAVE_NPN
self->npn_protocols = NULL;
#endif
-#ifdef HAVE_ALPN
+#if HAVE_ALPN
self->alpn_protocols = NULL;
#endif
#ifndef OPENSSL_NO_TLSEXT
PyObject_GC_UnTrack(self);
context_clear(self);
SSL_CTX_free(self->ctx);
-#if defined(OPENSSL_NPN_NEGOTIATED) && !defined(OPENSSL_NO_NEXTPROTONEG)
+#if HAVE_NPN
PyMem_FREE(self->npn_protocols);
#endif
-#ifdef HAVE_ALPN
+#if HAVE_ALPN
PyMem_FREE(self->alpn_protocols);
#endif
Py_TYPE(self)->tp_free(self);
#endif
-#if defined(OPENSSL_NPN_NEGOTIATED) && !defined(OPENSSL_NO_NEXTPROTONEG) || defined(HAVE_ALPN)
+#if HAVE_NPN || HAVE_ALPN
static int
do_protocol_selection(int alpn, unsigned char **out, unsigned char *outlen,
const unsigned char *server_protocols, unsigned int server_protocols_len,
}
#endif
-#if defined(OPENSSL_NPN_NEGOTIATED) && !defined(OPENSSL_NO_NEXTPROTONEG)
+#if HAVE_NPN
/* this callback gets passed to SSL_CTX_set_next_protos_advertise_cb */
static int
_advertiseNPN_cb(SSL *s,
Py_buffer *protos)
/*[clinic end generated code: output=72b002c3324390c6 input=319fcb66abf95bd7]*/
{
-#if defined(OPENSSL_NPN_NEGOTIATED) && !defined(OPENSSL_NO_NEXTPROTONEG)
+#if HAVE_NPN
PyMem_Free(self->npn_protocols);
self->npn_protocols = PyMem_Malloc(protos->len);
if (self->npn_protocols == NULL)
#endif
}
-#ifdef HAVE_ALPN
+#if HAVE_ALPN
static int
_selectALPN_cb(SSL *s,
const unsigned char **out, unsigned char *outlen,
Py_buffer *protos)
/*[clinic end generated code: output=87599a7f76651a9b input=9bba964595d519be]*/
{
-#ifdef HAVE_ALPN
+#if HAVE_ALPN
if ((size_t)protos->len > UINT_MAX) {
PyErr_Format(PyExc_OverflowError,
"protocols longer than %d bytes", UINT_MAX);
Py_INCREF(r);
PyModule_AddObject(m, "HAS_ECDH", r);
-#if defined(OPENSSL_NPN_NEGOTIATED) && !defined(OPENSSL_NO_NEXTPROTONEG)
+#if HAVE_NPN
r = Py_True;
#else
r = Py_False;
Py_INCREF(r);
PyModule_AddObject(m, "HAS_NPN", r);
-#ifdef HAVE_ALPN
+#if HAVE_ALPN
r = Py_True;
#else
r = Py_False;
{"open", (PyCFunction)dbmopen, METH_VARARGS, dbmopen__doc__},
static PyObject *
-dbmopen_impl(PyObject *module, const char *filename, const char *flags,
+dbmopen_impl(PyObject *module, PyObject *filename, const char *flags,
int mode);
static PyObject *
dbmopen(PyObject *module, PyObject *args)
{
PyObject *return_value = NULL;
- const char *filename;
+ PyObject *filename;
const char *flags = "r";
int mode = 438;
- if (!PyArg_ParseTuple(args, "s|si:open",
+ if (!PyArg_ParseTuple(args, "U|si:open",
&filename, &flags, &mode)) {
goto exit;
}
exit:
return return_value;
}
-/*[clinic end generated code: output=99adf966ef0475ff input=a9049054013a1b77]*/
+/*[clinic end generated code: output=001fabffcecb99f1 input=a9049054013a1b77]*/
{"open", (PyCFunction)dbmopen, METH_VARARGS, dbmopen__doc__},
static PyObject *
-dbmopen_impl(PyObject *module, const char *name, const char *flags, int mode);
+dbmopen_impl(PyObject *module, PyObject *filename, const char *flags,
+ int mode);
static PyObject *
dbmopen(PyObject *module, PyObject *args)
{
PyObject *return_value = NULL;
- const char *name;
+ PyObject *filename;
const char *flags = "r";
int mode = 438;
- if (!PyArg_ParseTuple(args, "s|si:open",
- &name, &flags, &mode)) {
+ if (!PyArg_ParseTuple(args, "U|si:open",
+ &filename, &flags, &mode)) {
goto exit;
}
- return_value = dbmopen_impl(module, name, flags, mode);
+ return_value = dbmopen_impl(module, filename, flags, mode);
exit:
return return_value;
}
-/*[clinic end generated code: output=ed0f5d4e3d79b80c input=a9049054013a1b77]*/
+/*[clinic end generated code: output=afb99364ac420d10 input=a9049054013a1b77]*/
return _ssl__SSLSocket_version_impl(self);
}
-#if (defined(OPENSSL_NPN_NEGOTIATED) && !defined(OPENSSL_NO_NEXTPROTONEG))
+#if (HAVE_NPN)
PyDoc_STRVAR(_ssl__SSLSocket_selected_npn_protocol__doc__,
"selected_npn_protocol($self, /)\n"
return _ssl__SSLSocket_selected_npn_protocol_impl(self);
}
-#endif /* (defined(OPENSSL_NPN_NEGOTIATED) && !defined(OPENSSL_NO_NEXTPROTONEG)) */
+#endif /* (HAVE_NPN) */
-#if defined(HAVE_ALPN)
+#if (HAVE_ALPN)
PyDoc_STRVAR(_ssl__SSLSocket_selected_alpn_protocol__doc__,
"selected_alpn_protocol($self, /)\n"
return _ssl__SSLSocket_selected_alpn_protocol_impl(self);
}
-#endif /* defined(HAVE_ALPN) */
+#endif /* (HAVE_ALPN) */
PyDoc_STRVAR(_ssl__SSLSocket_compression__doc__,
"compression($self, /)\n"
#ifndef _SSL_ENUM_CRLS_METHODDEF
#define _SSL_ENUM_CRLS_METHODDEF
#endif /* !defined(_SSL_ENUM_CRLS_METHODDEF) */
-/*[clinic end generated code: output=a8b184655068c238 input=a9049054013a1b77]*/
+/*[clinic end generated code: output=c79fb0dfd3c90784 input=a9049054013a1b77]*/
struct rlimit rl;
/* Disable creation of core dump */
- if (getrlimit(RLIMIT_CORE, &rl) != 0) {
+ if (getrlimit(RLIMIT_CORE, &rl) == 0) {
rl.rlim_cur = 0;
setrlimit(RLIMIT_CORE, &rl);
}
}
/* wait until the thread completes: it will never occur, since Py_FatalError()
- exits the process immedialty. */
+ exits the process immediately. */
PyThread_acquire_lock(lock, WAIT_LOCK);
PyThread_release_lock(lock);
PyThread_free_lock(lock);
if stdin does not appear to be a terminal; also PYTHONINSPECT=x\n\
-I : isolate Python from the user's environment (implies -E and -s)\n\
-m mod : run library module as a script (terminates option list)\n\
--O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x\n\
--OO : remove doc-strings in addition to the -O optimizations\n\
+-O : remove assert and __debug__-dependent statements; add .opt-1 before\n\
+ .pyc extension; also PYTHONOPTIMIZE=x\n\
+-OO : do -O changes and also discard docstrings; add .opt-2 before\n\
+ .pyc extension\n\
-q : don't print version and copyright messages on interactive startup\n\
-s : don't add user site directory to sys.path; also PYTHONNOUSERSITE\n\
-S : don't imply 'import site' on initialization\n\
exit(1);
}
- Py_HashRandomizationFlag = 1;
_PyRandom_Init();
PySys_ResetWarnOptions();
md5->curlen = 0;
}
- /* pad upto 56 bytes of zeroes */
+ /* pad up to 56 bytes of zeroes */
while (md5->curlen < 56) {
md5->buf[md5->curlen++] = (unsigned char)0;
}
#ifdef HAVE_PROCESS_H
#include <process.h>
#endif
-#ifndef VOLUME_NAME_DOS
-#define VOLUME_NAME_DOS 0x0
-#endif
-#ifndef VOLUME_NAME_NT
-#define VOLUME_NAME_NT 0x2
-#endif
#ifndef IO_REPARSE_TAG_SYMLINK
#define IO_REPARSE_TAG_SYMLINK (0xA000000CL)
#endif
return FALSE;
}
}
- if (wcsncmp(new_path, L"\\\\", 2) == 0 ||
- wcsncmp(new_path, L"//", 2) == 0)
- /* UNC path, nothing to do. */
- return TRUE;
- env[1] = new_path[0];
- result = SetEnvironmentVariableW(env, new_path);
+ int is_unc_like_path = (wcsncmp(new_path, L"\\\\", 2) == 0 ||
+ wcsncmp(new_path, L"//", 2) == 0);
+ if (!is_unc_like_path) {
+ env[1] = new_path[0];
+ result = SetEnvironmentVariableW(env, new_path);
+ }
if (new_path != path_buf)
PyMem_RawFree(new_path);
- return result;
+ return result ? TRUE : FALSE;
}
#endif
dir_fd, follow_symlinks);
else
follow_symlinks_specified("chmod", follow_symlinks);
+ return NULL;
}
else
#endif
/*[clinic end generated code: output=9bd78d0e52782e75 input=71d5e89334891bf4]*/
{
HANDLE hFile;
- int buf_size;
- wchar_t *target_path;
+ wchar_t buf[MAXPATHLEN], *target_path = buf;
+ int buf_size = Py_ARRAY_LENGTH(buf);
int result_length;
PyObject *result;
const wchar_t *path_wchar;
if (path_wchar == NULL)
return NULL;
+ Py_BEGIN_ALLOW_THREADS
hFile = CreateFileW(
path_wchar,
0, /* desired access */
/* FILE_FLAG_BACKUP_SEMANTICS is required to open a directory */
FILE_FLAG_BACKUP_SEMANTICS,
NULL);
+ Py_END_ALLOW_THREADS
if(hFile == INVALID_HANDLE_VALUE)
return win32_error_object("CreateFileW", path);
/* We have a good handle to the target, use it to determine the
target path name. */
- buf_size = GetFinalPathNameByHandleW(hFile, 0, 0, VOLUME_NAME_NT);
+ while (1) {
+ Py_BEGIN_ALLOW_THREADS
+ result_length = GetFinalPathNameByHandleW(hFile, target_path,
+ buf_size, VOLUME_NAME_DOS);
+ Py_END_ALLOW_THREADS
- if(!buf_size)
- return win32_error_object("GetFinalPathNameByHandle", path);
+ if (!result_length) {
+ result = win32_error_object("GetFinalPathNameByHandleW", path);
+ goto cleanup;
+ }
- target_path = PyMem_New(wchar_t, buf_size+1);
- if(!target_path)
- return PyErr_NoMemory();
+ if (result_length < buf_size) {
+ break;
+ }
- result_length = GetFinalPathNameByHandleW(hFile, target_path,
- buf_size, VOLUME_NAME_DOS);
- if(!result_length)
- return win32_error_object("GetFinalPathNamyByHandle", path);
+ wchar_t *tmp;
+ tmp = PyMem_Realloc(target_path != buf ? target_path : NULL,
+ result_length * sizeof(*tmp));
+ if (!tmp) {
+ result = PyErr_NoMemory();
+ goto cleanup;
+ }
- if(!CloseHandle(hFile))
- return win32_error_object("CloseHandle", path);
+ buf_size = result_length;
+ target_path = tmp;
+ }
- target_path[result_length] = 0;
result = PyUnicode_FromWideChar(target_path, result_length);
- PyMem_Free(target_path);
+
+cleanup:
+ if (target_path != buf) {
+ PyMem_Free(target_path);
+ }
+ CloseHandle(hFile);
return result;
}
#ifdef HAVE_STROPTS_H
#include <stropts.h>
#endif
-#endif /* defined(HAVE_OPENPTY) || defined(HAVE_FORKPTY) || defined(HAVE_DEV_PTMX */
+#endif /* defined(HAVE_OPENPTY) || defined(HAVE_FORKPTY) || defined(HAVE_DEV_PTMX) */
#if defined(HAVE_OPENPTY) || defined(HAVE__GETPTY) || defined(HAVE_DEV_PTMX)
"not a symbolic link");
return NULL;
}
- print_name = rdb->SymbolicLinkReparseBuffer.PathBuffer +
- rdb->SymbolicLinkReparseBuffer.PrintNameOffset;
+ print_name = (wchar_t *)((char*)rdb->SymbolicLinkReparseBuffer.PathBuffer +
+ rdb->SymbolicLinkReparseBuffer.PrintNameOffset);
result = PyUnicode_FromWideChar(print_name,
- rdb->SymbolicLinkReparseBuffer.PrintNameLength/2);
+ rdb->SymbolicLinkReparseBuffer.PrintNameLength / sizeof(wchar_t));
return result;
}
#if defined(MS_WINDOWS)
/* Grab CreateSymbolicLinkW dynamically from kernel32 */
-static DWORD (CALLBACK *Py_CreateSymbolicLinkW)(LPCWSTR, LPCWSTR, DWORD) = NULL;
+static BOOLEAN (CALLBACK *Py_CreateSymbolicLinkW)(LPCWSTR, LPCWSTR, DWORD) = NULL;
static int
check_CreateSymbolicLink(void)
return Py_CreateSymbolicLinkW != NULL;
}
-/* Remove the last portion of the path */
-static void
+/* Remove the last portion of the path - return 0 on success */
+static int
_dirnameW(WCHAR *path)
{
WCHAR *ptr;
+ size_t length = wcsnlen_s(path, MAX_PATH);
+ if (length == MAX_PATH) {
+ return -1;
+ }
/* walk the path from the end until a backslash is encountered */
- for(ptr = path + wcslen(path); ptr != path; ptr--) {
- if (*ptr == L'\\' || *ptr == L'/')
+ for(ptr = path + length; ptr != path; ptr--) {
+ if (*ptr == L'\\' || *ptr == L'/') {
break;
+ }
}
*ptr = 0;
+ return 0;
}
/* Is this path absolute? */
static int
_is_absW(const WCHAR *path)
{
- return path[0] == L'\\' || path[0] == L'/' || path[1] == L':';
-
+ return path[0] == L'\\' || path[0] == L'/' ||
+ (path[0] && path[1] == L':');
}
-/* join root and rest with a backslash */
-static void
+/* join root and rest with a backslash - return 0 on success */
+static int
_joinW(WCHAR *dest_path, const WCHAR *root, const WCHAR *rest)
{
- size_t root_len;
-
if (_is_absW(rest)) {
- wcscpy(dest_path, rest);
- return;
+ return wcscpy_s(dest_path, MAX_PATH, rest);
}
- root_len = wcslen(root);
+ if (wcscpy_s(dest_path, MAX_PATH, root)) {
+ return -1;
+ }
- wcscpy(dest_path, root);
- if(root_len) {
- dest_path[root_len] = L'\\';
- root_len++;
+ if (dest_path[0] && wcscat_s(dest_path, MAX_PATH, L"\\")) {
+ return -1;
}
- wcscpy(dest_path+root_len, rest);
+
+ return wcscat_s(dest_path, MAX_PATH, rest);
}
/* Return True if the path at src relative to dest is a directory */
WCHAR src_resolved[MAX_PATH] = L"";
/* dest_parent = os.path.dirname(dest) */
- wcscpy(dest_parent, dest);
- _dirnameW(dest_parent);
+ if (wcscpy_s(dest_parent, MAX_PATH, dest) ||
+ _dirnameW(dest_parent)) {
+ return 0;
+ }
/* src_resolved = os.path.join(dest_parent, src) */
- _joinW(src_resolved, dest_parent, src);
+ if (_joinW(src_resolved, dest_parent, src)) {
+ return 0;
+ }
return (
GetFileAttributesExW(src_resolved, GetFileExInfoStandard, &src_info)
&& src_info.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY
}
#endif
- if ((src->narrow && dst->wide) || (src->wide && dst->narrow)) {
- PyErr_SetString(PyExc_ValueError,
- "symlink: src and dst must be the same type");
- return NULL;
- }
-
#ifdef MS_WINDOWS
Py_BEGIN_ALLOW_THREADS
+ _Py_BEGIN_SUPPRESS_IPH
/* if src is a directory, ensure target_is_directory==1 */
target_is_directory |= _check_dirW(src->wide, dst->wide);
result = Py_CreateSymbolicLinkW(dst->wide, src->wide,
target_is_directory);
+ _Py_END_SUPPRESS_IPH
Py_END_ALLOW_THREADS
if (!result)
#else
+ if ((src->narrow && dst->wide) || (src->wide && dst->narrow)) {
+ PyErr_SetString(PyExc_ValueError,
+ "symlink: src and dst must be the same type");
+ return NULL;
+ }
+
Py_BEGIN_ALLOW_THREADS
#if HAVE_SYMLINKAT
if (dir_fd != DEFAULT_DIR_FD)
#if defined(HAVE_DUP3) && \
!(defined(HAVE_FCNTL_H) && defined(F_DUP2FD_CLOEXEC))
/* dup3() is available on Linux 2.6.27+ and glibc 2.9 */
- int dup3_works = -1;
+ static int dup3_works = -1;
#endif
if (fd < 0 || fd2 < 0)
Py_BEGIN_ALLOW_THREADS
result = posix_fallocate(fd, offset, length);
Py_END_ALLOW_THREADS
- } while (result != 0 && errno == EINTR &&
- !(async_err = PyErr_CheckSignals()));
- if (result != 0)
- return (!async_err) ? posix_error() : NULL;
- Py_RETURN_NONE;
+ } while (result == EINTR && !(async_err = PyErr_CheckSignals()));
+
+ if (result == 0)
+ Py_RETURN_NONE;
+
+ if (async_err)
+ return NULL;
+
+ errno = result;
+ return posix_error();
}
#endif /* HAVE_POSIX_FALLOCATE) && !POSIX_FADVISE_AIX_BUG */
Py_BEGIN_ALLOW_THREADS
result = posix_fadvise(fd, offset, length, advice);
Py_END_ALLOW_THREADS
- } while (result != 0 && errno == EINTR &&
- !(async_err = PyErr_CheckSignals()));
- if (result != 0)
- return (!async_err) ? posix_error() : NULL;
- Py_RETURN_NONE;
+ } while (result == EINTR && !(async_err = PyErr_CheckSignals()));
+
+ if (result == 0)
+ Py_RETURN_NONE;
+
+ if (async_err)
+ return NULL;
+
+ errno = result;
+ return posix_error();
}
#endif /* HAVE_POSIX_FADVISE && !POSIX_FADVISE_AIX_BUG */
# include <ctype.h>
#endif
-#if defined(__APPLE__) || defined(__NetBSD__)
+#if defined(__APPLE__) || defined(__CYGWIN__) || defined(__NetBSD__)
# include <sys/ioctl.h>
#endif
#include <VersionHelpers.h>
#endif
+/* remove some flags on older version Windows during run-time.
+ https://msdn.microsoft.com/en-us/library/windows/desktop/ms738596.aspx */
+typedef struct {
+ DWORD build_number; /* available starting with this Win10 BuildNumber */
+ const char flag_name[20];
+} FlagRuntimeInfo;
+
+/* IMPORTANT: make sure the list ordered by descending build_number */
+static FlagRuntimeInfo win_runtime_flags[] = {
+ /* available starting with Windows 10 1703 */
+ {15063, "TCP_KEEPCNT"},
+ /* available starting with Windows 10 1607 */
+ {14393, "TCP_FASTOPEN"}
+};
+
+static void
+remove_unusable_flags(PyObject *m)
+{
+ PyObject *dict;
+ OSVERSIONINFOEX info;
+ DWORDLONG dwlConditionMask;
+
+ dict = PyModule_GetDict(m);
+ if (dict == NULL) {
+ return;
+ }
+
+ /* set to Windows 10, except BuildNumber. */
+ memset(&info, 0, sizeof(info));
+ info.dwOSVersionInfoSize = sizeof(info);
+ info.dwMajorVersion = 10;
+ info.dwMinorVersion = 0;
+
+ /* set Condition Mask */
+ dwlConditionMask = 0;
+ VER_SET_CONDITION(dwlConditionMask, VER_MAJORVERSION, VER_GREATER_EQUAL);
+ VER_SET_CONDITION(dwlConditionMask, VER_MINORVERSION, VER_GREATER_EQUAL);
+ VER_SET_CONDITION(dwlConditionMask, VER_BUILDNUMBER, VER_GREATER_EQUAL);
+
+ for (int i=0; i<sizeof(win_runtime_flags)/sizeof(FlagRuntimeInfo); i++) {
+ info.dwBuildNumber = win_runtime_flags[i].build_number;
+ /* greater than or equal to the specified version?
+ Compatibility Mode will not cheat VerifyVersionInfo(...) */
+ if (VerifyVersionInfo(
+ &info,
+ VER_MAJORVERSION|VER_MINORVERSION|VER_BUILDNUMBER,
+ dwlConditionMask)) {
+ break;
+ }
+ else {
+ if (PyDict_GetItemString(
+ dict,
+ win_runtime_flags[i].flag_name) != NULL) {
+ PyDict_DelItemString(
+ dict,
+ win_runtime_flags[i].flag_name);
+ }
+ }
+ }
+}
+
#endif
#include <stddef.h>
}
#if defined(__APPLE__) && defined(AI_NUMERICSERV)
if ((flags & AI_NUMERICSERV) && (pptr == NULL || (pptr[0] == '0' && pptr[1] == 0))) {
- /* On OSX upto at least OSX 10.8 getaddrinfo crashes
+ /* On OSX up to at least OSX 10.8 getaddrinfo crashes
* if AI_NUMERICSERV is set and the servname is NULL or "0".
* This workaround avoids a segfault in libsystem.
*/
#if defined(USE_GETHOSTBYNAME_LOCK) || defined(USE_GETADDRINFO_LOCK)
netdb_lock = PyThread_allocate_lock();
#endif
+
+#ifdef MS_WINDOWS
+ /* removes some flags on older version Windows during run-time */
+ remove_unusable_flags(m);
+#endif
+
return m;
}
PyModule_AddIntMacro(m, DEFLATED);
PyModule_AddIntMacro(m, DEF_MEM_LEVEL);
PyModule_AddIntMacro(m, DEF_BUF_SIZE);
+ // compression levels
+ PyModule_AddIntMacro(m, Z_NO_COMPRESSION);
PyModule_AddIntMacro(m, Z_BEST_SPEED);
PyModule_AddIntMacro(m, Z_BEST_COMPRESSION);
PyModule_AddIntMacro(m, Z_DEFAULT_COMPRESSION);
+ // compression strategies
PyModule_AddIntMacro(m, Z_FILTERED);
PyModule_AddIntMacro(m, Z_HUFFMAN_ONLY);
+#ifdef Z_RLE // 1.2.0.1
+ PyModule_AddIntMacro(m, Z_RLE);
+#endif
+#ifdef Z_FIXED // 1.2.2.2
+ PyModule_AddIntMacro(m, Z_FIXED);
+#endif
PyModule_AddIntMacro(m, Z_DEFAULT_STRATEGY);
-
- PyModule_AddIntMacro(m, Z_FINISH);
+ // allowed flush values
PyModule_AddIntMacro(m, Z_NO_FLUSH);
+ PyModule_AddIntMacro(m, Z_PARTIAL_FLUSH);
PyModule_AddIntMacro(m, Z_SYNC_FLUSH);
PyModule_AddIntMacro(m, Z_FULL_FLUSH);
-
+ PyModule_AddIntMacro(m, Z_FINISH);
+#ifdef Z_BLOCK // 1.2.0.5 for inflate, 1.2.3.4 for deflate
+ PyModule_AddIntMacro(m, Z_BLOCK);
+#endif
+#ifdef Z_TREES // 1.2.3.4, only for inflate
+ PyModule_AddIntMacro(m, Z_TREES);
+#endif
ver = PyUnicode_FromString(ZLIB_VERSION);
if (ver != NULL)
PyModule_AddObject(m, "ZLIB_VERSION", ver);
Py_XINCREF(del);
Py_XINCREF(doc);
- prop->prop_get = get;
- prop->prop_set = set;
- prop->prop_del = del;
- prop->prop_doc = doc;
+ Py_XSETREF(prop->prop_get, get);
+ Py_XSETREF(prop->prop_set, set);
+ Py_XSETREF(prop->prop_del, del);
+ Py_XSETREF(prop->prop_doc, doc);
prop->getter_doc = 0;
/* if no docstring given and the getter has one, use that one */
dictview_repr(_PyDictViewObject *dv)
{
PyObject *seq;
- PyObject *result;
+ PyObject *result = NULL;
+ Py_ssize_t rc;
+ rc = Py_ReprEnter((PyObject *)dv);
+ if (rc != 0) {
+ return rc > 0 ? PyUnicode_FromString("...") : NULL;
+ }
seq = PySequence_List((PyObject *)dv);
- if (seq == NULL)
- return NULL;
-
+ if (seq == NULL) {
+ goto Done;
+ }
result = PyUnicode_FromFormat("%s(%R)", Py_TYPE(dv)->tp_name, seq);
Py_DECREF(seq);
+
+Done:
+ Py_ReprLeave((PyObject *)dv);
return result;
}
static int
_set_legacy_print_statement_msg(PySyntaxErrorObject *self, Py_ssize_t start)
{
- PyObject *strip_sep_obj = PyUnicode_FromString(" \t\r\n");
- if (strip_sep_obj == NULL)
- return -1;
-
- // PRINT_OFFSET is to remove `print ` word from the data.
+ // PRINT_OFFSET is to remove the `print ` prefix from the data.
const int PRINT_OFFSET = 6;
+ const int STRIP_BOTH = 2;
+ Py_ssize_t start_pos = start + PRINT_OFFSET;
Py_ssize_t text_len = PyUnicode_GET_LENGTH(self->text);
- PyObject *data = PyUnicode_Substring(self->text, PRINT_OFFSET, text_len);
+ Py_UCS4 semicolon = ';';
+ Py_ssize_t end_pos = PyUnicode_FindChar(self->text, semicolon,
+ start_pos, text_len, 1);
+ if (end_pos < -1) {
+ return -1;
+ } else if (end_pos == -1) {
+ end_pos = text_len;
+ }
+ PyObject *data = PyUnicode_Substring(self->text, start_pos, end_pos);
if (data == NULL) {
- Py_DECREF(strip_sep_obj);
return -1;
}
- PyObject *new_data = _PyUnicode_XStrip(data, 2, strip_sep_obj);
+
+ PyObject *strip_sep_obj = PyUnicode_FromString(" \t\r\n");
+ if (strip_sep_obj == NULL) {
+ Py_DECREF(data);
+ return -1;
+ }
+
+ PyObject *new_data = _PyUnicode_XStrip(data, STRIP_BOTH, strip_sep_obj);
Py_DECREF(data);
Py_DECREF(strip_sep_obj);
-
if (new_data == NULL) {
return -1;
}
* o 'try'/'for'/'while' blocks can't be jumped into because the blockstack
* needs to be set up before their code runs, and for 'for' loops the
* iterator needs to be on the stack.
+ * o Jumps cannot be made from within a trace function invoked with a
+ * 'return' or 'exception' event since the eval loop has been exited at
+ * that time.
*/
static int
frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno)
return -1;
}
+ /* Upon the 'call' trace event of a new frame, f->f_lasti is -1 and
+ * f->f_trace is NULL, check first on the first condition.
+ * Forbidding jumps from the 'call' event of a new frame is a side effect
+ * of allowing to set f_lineno only from trace functions. */
+ if (f->f_lasti == -1) {
+ PyErr_Format(PyExc_ValueError,
+ "can't jump from the 'call' trace event of a new frame");
+ return -1;
+ }
+
/* You can only do this from within a trace function, not via
* _getframe or similar hackery. */
- if (!f->f_trace)
- {
+ if (!f->f_trace) {
PyErr_Format(PyExc_ValueError,
- "f_lineno can only be set by a"
- " line trace function");
+ "f_lineno can only be set by a trace function");
+ return -1;
+ }
+
+ /* Forbid jumps upon a 'return' trace event (except after executing a
+ * YIELD_VALUE or YIELD_FROM opcode, f_stacktop is not NULL in that case)
+ * and upon an 'exception' trace event.
+ * Jumps from 'call' trace events have already been forbidden above for new
+ * frames, so this check does not change anything for 'call' events. */
+ if (f->f_stacktop == NULL) {
+ PyErr_SetString(PyExc_ValueError,
+ "can only jump from a 'line' trace event");
return -1;
}
/* We're now ready to look at the bytecode. */
PyBytes_AsStringAndSize(f->f_code->co_code, (char **)&code, &code_len);
+
+ /* The trace function is called with a 'return' trace event after the
+ * execution of a yield statement. */
+ assert(f->f_lasti != -1);
+ if (code[f->f_lasti] == YIELD_VALUE || code[f->f_lasti] == YIELD_FROM) {
+ PyErr_SetString(PyExc_ValueError,
+ "can't jump from a yield statement");
+ return -1;
+ }
+
min_addr = Py_MIN(new_lasti, f->f_lasti);
max_addr = Py_MAX(new_lasti, f->f_lasti);
PyObject *v = (*--f->f_stacktop);
Py_DECREF(v);
}
+ if (b->b_type == SETUP_FINALLY &&
+ code[b->b_handler] == WITH_CLEANUP_START)
+ {
+ /* Pop the exit function. */
+ PyObject *v = (*--f->f_stacktop);
+ Py_DECREF(v);
+ }
}
/* Finally set the new f_lineno and f_lasti and return OK. */
if (!_PyArg_NoKeywords("classmethod", kwds))
return -1;
Py_INCREF(callable);
- cm->cm_callable = callable;
+ Py_XSETREF(cm->cm_callable, callable);
return 0;
}
if (!_PyArg_NoKeywords("staticmethod", kwds))
return -1;
Py_INCREF(callable);
- sm->sm_callable = callable;
+ Py_XSETREF(sm->sm_callable, callable);
return 0;
}
goto error;
}
- if (Py_EnterRecursiveCall(" while getting the repr of a list"))
- goto error;
s = PyObject_Repr(v->ob_item[i]);
- Py_LeaveRecursiveCall();
if (s == NULL)
goto error;
assert(!PyErr_Occurred());
#endif
+ /* It is possible for a type to have a tp_repr representation that loops
+ infinitely. */
+ if (Py_EnterRecursiveCall(" while getting the repr of an object"))
+ return NULL;
res = (*v->ob_type->tp_repr)(v);
+ Py_LeaveRecursiveCall();
if (res == NULL)
return NULL;
if (!PyUnicode_Check(res)) {
hash ^= ((Py_uhash_t)PySet_GET_SIZE(self) + 1) * 1927868237UL;
/* Disperse patterns arising in nested frozensets */
+ hash ^= (hash >> 11) ^ (hash >> 25);
hash = hash * 69069U + 907133923UL;
/* -1 is reserved as an error code */
goto error;
}
- if (Py_EnterRecursiveCall(" while getting the repr of a tuple"))
- goto error;
s = PyObject_Repr(v->ob_item[i]);
- Py_LeaveRecursiveCall();
if (s == NULL)
goto error;
/* The addend 82520, was selected from the range(0, 1000000) for
generating the greatest number of prime multipliers for tuples
- upto length eight:
+ up to length eight:
1082527, 1165049, 1082531, 1165057, 1247581, 1330103, 1082533,
1330111, 1412633, 1165069, 1247599, 1495177, 1577699
}
/* SF bug 475327 -- if that didn't trigger, we need 3
- arguments. but PyArg_ParseTupleAndKeywords below may give
+ arguments. but PyArg_ParseTuple below may give
a msg saying type() needs exactly 3. */
if (nargs != 3) {
PyErr_SetString(PyExc_TypeError,
/* PyType_Ready is the closest thing we have to a choke point
* for type objects, so is the best place I can think of to try
* to get type objects into the doubly-linked list of all objects.
- * Still, not all type objects go thru PyType_Ready.
+ * Still, not all type objects go through PyType_Ready.
*/
_Py_AddToAllObjects((PyObject *)type, 0);
#endif
}
}
-PyObject *
-PyUnicode_EncodeLocale(PyObject *unicode, const char *errors)
+static PyObject *
+unicode_encode_locale(PyObject *unicode, const char *errors,
+ int current_locale)
{
Py_ssize_t wlen, wlen2;
wchar_t *wstr;
/* "surrogateescape" error handler */
char *str;
- str = Py_EncodeLocale(wstr, &error_pos);
+ str = _Py_EncodeLocaleEx(wstr, &error_pos, current_locale);
if (str == NULL) {
if (error_pos == (size_t)-1) {
PyErr_NoMemory();
return NULL;
}
+PyObject *
+PyUnicode_EncodeLocale(PyObject *unicode, const char *errors)
+{
+ return unicode_encode_locale(unicode, errors, 1);
+}
+
PyObject *
PyUnicode_EncodeFSDefault(PyObject *unicode)
{
Py_FileSystemDefaultEncodeErrors);
}
else {
- return PyUnicode_EncodeLocale(unicode, Py_FileSystemDefaultEncodeErrors);
+ return unicode_encode_locale(unicode,
+ Py_FileSystemDefaultEncodeErrors, 0);
}
#endif
}
return 0;
}
-PyObject*
-PyUnicode_DecodeLocaleAndSize(const char *str, Py_ssize_t len,
- const char *errors)
+static PyObject*
+unicode_decode_locale(const char *str, Py_ssize_t len,
+ const char *errors, int current_locale)
{
wchar_t smallbuf[256];
size_t smallbuf_len = Py_ARRAY_LENGTH(smallbuf);
if (surrogateescape) {
/* "surrogateescape" error handler */
- wstr = Py_DecodeLocale(str, &wlen);
+ wstr = _Py_DecodeLocaleEx(str, &wlen, current_locale);
if (wstr == NULL) {
if (wlen == (size_t)-1)
PyErr_NoMemory();
return NULL;
}
+PyObject*
+PyUnicode_DecodeLocaleAndSize(const char *str, Py_ssize_t size,
+ const char *errors)
+{
+ return unicode_decode_locale(str, size, errors, 1);
+}
+
PyObject*
PyUnicode_DecodeLocale(const char *str, const char *errors)
{
Py_ssize_t size = (Py_ssize_t)strlen(str);
- return PyUnicode_DecodeLocaleAndSize(str, size, errors);
+ return unicode_decode_locale(str, size, errors, 1);
}
Py_FileSystemDefaultEncodeErrors);
}
else {
- return PyUnicode_DecodeLocaleAndSize(s, size, Py_FileSystemDefaultEncodeErrors);
+ return unicode_decode_locale(s, size,
+ Py_FileSystemDefaultEncodeErrors, 0);
}
#endif
}
Py_ssize_t insize;
Py_ssize_t newpos;
Py_ssize_t replen;
+ Py_ssize_t remain;
PyObject *inputobj = NULL;
+ int need_to_grow = 0;
+ const char *new_inptr;
if (*errorHandler == NULL) {
*errorHandler = PyCodec_LookupError(errors);
if (!PyBytes_Check(inputobj)) {
PyErr_Format(PyExc_TypeError, "exception attribute object must be bytes");
}
+ remain = *inend - *input - *endinpos;
*input = PyBytes_AS_STRING(inputobj);
insize = PyBytes_GET_SIZE(inputobj);
*inend = *input + insize;
replen = PyUnicode_GET_LENGTH(repunicode);
if (replen > 1) {
writer->min_length += replen - 1;
+ need_to_grow = 1;
+ }
+ new_inptr = *input + newpos;
+ if (*inend - new_inptr > remain) {
+ /* We don't know the decoding algorithm here so we make the worst
+ assumption that one byte decodes to one unicode character.
+ If unfortunately one byte could decode to more unicode characters,
+ the decoder may write out-of-bound then. Is it possible for the
+ algorithms using this function? */
+ writer->min_length += *inend - new_inptr - remain;
+ need_to_grow = 1;
+ }
+ if (need_to_grow) {
writer->overallocate = 1;
- if (_PyUnicodeWriter_Prepare(writer, writer->min_length,
+ if (_PyUnicodeWriter_Prepare(writer, writer->min_length - writer->pos,
PyUnicode_MAX_CHAR_VALUE(repunicode)) == -1)
goto onError;
}
goto onError;
*endinpos = newpos;
- *inptr = *input + newpos;
+ *inptr = new_inptr;
/* we made it! */
Py_XDECREF(restuple);
#endif
/* Note: size will always be longer than the resulting Unicode
- character count */
+ character count normally. Error handler will take care of
+ resizing when needed. */
_PyUnicodeWriter_Init(&writer);
writer.min_length = (e - q + 1) / 2;
if (_PyUnicodeWriter_Prepare(&writer, writer.min_length, 127) == -1)
&writer)) {
goto onError;
}
- if (_PyUnicodeWriter_Prepare(&writer, writer.min_length, 127) < 0) {
- goto onError;
- }
+ assert(end - s <= writer.size - writer.pos);
#undef WRITE_ASCII_CHAR
#undef WRITE_CHAR
&writer)) {
goto onError;
}
- if (_PyUnicodeWriter_Prepare(&writer, writer.min_length, 127) < 0) {
- goto onError;
- }
+ assert(end - s <= writer.size - writer.pos);
#undef WRITE_CHAR
}
if (PyUnicode_CHECK_INTERNED(left))
return 0;
- assert(_PyUnicode_HASH(right_uni) != 1);
+ assert(_PyUnicode_HASH(right_uni) != -1);
hash = _PyUnicode_HASH(left);
if (hash != -1 && hash != _PyUnicode_HASH(right_uni))
return 0;
#include <Windows.h>
#include <Strsafe.h>
#include "external\include\Setup.Configuration.h"
-#pragma comment(lib, "ole32.lib")
-#pragma comment(lib, "oleaut32.lib")
-#pragma comment(lib, "version.lib")
-#pragma comment(lib, "Microsoft.VisualStudio.Setup.Configuration.Native.lib")
#include <Python.h>
return 0; // success
}
-PyDoc_STRVAR(findvs_doc, "The _findvs helper module");
+PyDoc_STRVAR(findvs_doc, "The _distutils_findvs helper module");
static PyModuleDef_Slot findvs_slots[] = {
{ Py_mod_exec, exec_findvs },
static PyModuleDef findvs_def = {
PyModuleDef_HEAD_INIT,
- "_findvs",
+ "_distutils_findvs",
findvs_doc,
0, // m_size
NULL, // m_methods
};
extern "C" {
- PyMODINIT_FUNC PyInit__findvs(void)
+ PyMODINIT_FUNC PyInit__distutils_findvs(void)
{
return PyModuleDef_Init(&findvs_def);
}
-}
\ No newline at end of file
+}
extern PyObject* PyInit__string(void);
extern PyObject* PyInit__stat(void);
extern PyObject* PyInit__opcode(void);
-extern PyObject* PyInit__findvs(void);
/* tools/freeze/makeconfig.py marker for additional "extern" */
/* -- ADDMODULE MARKER 1 -- */
{"_stat", PyInit__stat},
{"_opcode", PyInit__opcode},
- {"_findvs", PyInit__findvs},
-
/* Sentinel */
{0, 0}
};
}
}
+static int _PathCchCanonicalizeEx_Initialized = 0;
+typedef HRESULT(__stdcall *PPathCchCanonicalizeEx) (PWSTR pszPathOut, size_t cchPathOut,
+ PCWSTR pszPathIn, unsigned long dwFlags);
+static PPathCchCanonicalizeEx _PathCchCanonicalizeEx;
+
+static void canonicalize(wchar_t *buffer, const wchar_t *path)
+{
+ if (_PathCchCanonicalizeEx_Initialized == 0) {
+ HMODULE pathapi = LoadLibraryW(L"api-ms-win-core-path-l1-1-0.dll");
+ if (pathapi) {
+ _PathCchCanonicalizeEx = (PPathCchCanonicalizeEx)GetProcAddress(pathapi, "PathCchCanonicalizeEx");
+ }
+ else {
+ _PathCchCanonicalizeEx = NULL;
+ }
+ _PathCchCanonicalizeEx_Initialized = 1;
+ }
+
+ if (_PathCchCanonicalizeEx) {
+ if (FAILED(_PathCchCanonicalizeEx(buffer, MAXPATHLEN + 1, path, 0))) {
+ Py_FatalError("buffer overflow in getpathp.c's canonicalize()");
+ }
+ }
+ else {
+ if (!PathCanonicalizeW(buffer, path)) {
+ Py_FatalError("buffer overflow in getpathp.c's canonicalize()");
+ }
+ }
+}
+
/* gotlandmark only called by search_for_prefix, which ensures
'prefix' is null terminated in bounds. join() ensures
'landmark' can not overflow prefix if too long.
get_progpath(void)
{
extern wchar_t *Py_GetProgramName(void);
+ wchar_t modulepath[MAXPATHLEN];
wchar_t *path = _wgetenv(L"PATH");
wchar_t *prog = Py_GetProgramName();
#else
dllpath[0] = 0;
#endif
- if (GetModuleFileNameW(NULL, progpath, MAXPATHLEN))
+ if (GetModuleFileNameW(NULL, modulepath, MAXPATHLEN)) {
+ canonicalize(progpath, modulepath);
return;
+ }
if (prog == NULL || *prog == '\0')
prog = L"python";
--- /dev/null
+<?xml version="1.0" encoding="utf-8"?>\r
+<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">\r
+ <ItemGroup Label="ProjectConfigurations">\r
+ <ProjectConfiguration Include="Debug|Win32">\r
+ <Configuration>Debug</Configuration>\r
+ <Platform>Win32</Platform>\r
+ </ProjectConfiguration>\r
+ <ProjectConfiguration Include="Debug|x64">\r
+ <Configuration>Debug</Configuration>\r
+ <Platform>x64</Platform>\r
+ </ProjectConfiguration>\r
+ <ProjectConfiguration Include="PGInstrument|Win32">\r
+ <Configuration>PGInstrument</Configuration>\r
+ <Platform>Win32</Platform>\r
+ </ProjectConfiguration>\r
+ <ProjectConfiguration Include="PGInstrument|x64">\r
+ <Configuration>PGInstrument</Configuration>\r
+ <Platform>x64</Platform>\r
+ </ProjectConfiguration>\r
+ <ProjectConfiguration Include="PGUpdate|Win32">\r
+ <Configuration>PGUpdate</Configuration>\r
+ <Platform>Win32</Platform>\r
+ </ProjectConfiguration>\r
+ <ProjectConfiguration Include="PGUpdate|x64">\r
+ <Configuration>PGUpdate</Configuration>\r
+ <Platform>x64</Platform>\r
+ </ProjectConfiguration>\r
+ <ProjectConfiguration Include="Release|Win32">\r
+ <Configuration>Release</Configuration>\r
+ <Platform>Win32</Platform>\r
+ </ProjectConfiguration>\r
+ <ProjectConfiguration Include="Release|x64">\r
+ <Configuration>Release</Configuration>\r
+ <Platform>x64</Platform>\r
+ </ProjectConfiguration>\r
+ </ItemGroup>\r
+ <PropertyGroup Label="Globals">\r
+ <ProjectGuid>{41ADEDF9-11D8-474E-B4D7-BB82332C878E}</ProjectGuid>\r
+ <RootNamespace>_distutils_findvs</RootNamespace>\r
+ <Keyword>Win32Proj</Keyword>\r
+ </PropertyGroup>\r
+ <Import Project="python.props" />\r
+ <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />\r
+ <PropertyGroup Label="Configuration">\r
+ <ConfigurationType>DynamicLibrary</ConfigurationType>\r
+ <CharacterSet>NotSet</CharacterSet>\r
+ </PropertyGroup>\r
+ <Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />\r
+ <PropertyGroup>\r
+ <TargetExt>.pyd</TargetExt>\r
+ </PropertyGroup>\r
+ <ImportGroup Label="ExtensionSettings">\r
+ </ImportGroup>\r
+ <ImportGroup Label="PropertySheets">\r
+ <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />\r
+ <Import Project="pyproject.props" />\r
+ </ImportGroup>\r
+ <PropertyGroup Label="UserMacros" />\r
+ <PropertyGroup>\r
+ <_ProjectFileVersion>10.0.30319.1</_ProjectFileVersion>\r
+ </PropertyGroup>\r
+ <ItemDefinitionGroup>\r
+ <Link>\r
+ <AdditionalDependencies>version.lib;ole32.lib;oleaut32.lib;Microsoft.VisualStudio.Setup.Configuration.Native.lib;%(AdditionalDependencies)</AdditionalDependencies>\r
+ <AdditionalLibraryDirectories>%(AdditionalLibraryDirectories);$(PySourcePath)PC\external\$(PlatformToolset)\$(ArchName)</AdditionalLibraryDirectories>\r
+ </Link>\r
+ </ItemDefinitionGroup>\r
+ <ItemGroup>\r
+ <ClCompile Include="..\PC\_findvs.cpp" />\r
+ </ItemGroup>\r
+ <ItemGroup>\r
+ <ResourceCompile Include="..\PC\python_nt.rc" />\r
+ </ItemGroup>\r
+ <ItemGroup>\r
+ <ProjectReference Include="pythoncore.vcxproj">\r
+ <Project>{cf7ac3d1-e2df-41d2-bea6-1e2556cdea26}</Project>\r
+ <ReferenceOutputAssembly>false</ReferenceOutputAssembly>\r
+ </ProjectReference>\r
+ </ItemGroup>\r
+ <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />\r
+ <ImportGroup Label="ExtensionTargets">\r
+ </ImportGroup>\r
+</Project>\r
--- /dev/null
+<?xml version="1.0" encoding="utf-8"?>\r
+<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">\r
+ <ItemGroup>\r
+ <ResourceCompile Include="..\PC\python_nt.rc" />\r
+ </ItemGroup>\r
+ <ItemGroup>\r
+ <Filter Include="Source Files">\r
+ <UniqueIdentifier>{c56a5dd3-7838-48e9-a781-855d8be7370f}</UniqueIdentifier>\r
+ </Filter>\r
+ </ItemGroup>\r
+ <ItemGroup>\r
+ <ClCompile Include="..\PC\_findvs.cpp">\r
+ <Filter>Source Files</Filter>\r
+ </ClCompile>\r
+ </ItemGroup>\r
+</Project>
\ No newline at end of file
@if exist "%_Py_EXTERNALS_DIR%\pythonx86\tools\python.exe" (set PYTHON="%_Py_EXTERNALS_DIR%\pythonx86\tools\python.exe") & (set _Py_Python_Source=found in externals directory) & goto :found\r
\r
@rem If HOST_PYTHON is recent enough, use that\r
-@if NOT "%HOST_PYTHON%"=="" @%HOST_PYTHON% -c "import sys; assert sys.version_info[:2] >= (3, 6)" >nul 2>nul && (set PYTHON="%HOST_PYTHON%") && (set _Py_Python_Source=found as HOST_PYTHON) && goto :found\r
+@if NOT "%HOST_PYTHON%"=="" @%HOST_PYTHON% -Ec "import sys; assert sys.version_info[:2] >= (3, 6)" >nul 2>nul && (set PYTHON="%HOST_PYTHON%") && (set _Py_Python_Source=found as HOST_PYTHON) && goto :found\r
\r
@rem If py.exe finds a recent enough version, use that one\r
-@py -3.6 -V >nul 2>&1 && (set PYTHON=py -3.6) && (set _Py_Python_Source=found with py.exe) && goto :found\r
+@py -3.6 -EV >nul 2>&1 && (set PYTHON=py -3.6) && (set _Py_Python_Source=found with py.exe) && goto :found\r
\r
@if NOT exist "%_Py_EXTERNALS_DIR%" mkdir "%_Py_EXTERNALS_DIR%"\r
@set _Py_NUGET=%NUGET%\r
@rem If it fails, retry with any available copy of Python\r
@powershell.exe -Command Invoke-WebRequest %_Py_NUGET_URL% -OutFile '%_Py_NUGET%'\r
@if errorlevel 1 (\r
- @%_Py_HOST_PYTHON% "%~dp0\urlretrieve.py" "%_Py_NUGET_URL%" "%_Py_NUGET%"\r
+ @%_Py_HOST_PYTHON% -E "%~dp0\urlretrieve.py" "%_Py_NUGET_URL%" "%_Py_NUGET%"\r
)\r
)\r
@echo Installing Python via nuget...\r
\r
if "%ORG%"=="" (set ORG=python)\r
\r
-call "%PCBUILD%find_python.bat" "%PYTHON%"\r
+call "%PCBUILD%\find_python.bat" "%PYTHON%"\r
\r
if "%PYTHON%"=="" (\r
where /Q git || echo Python 3.6 could not be found or installed, and git.exe is not on your PATH && exit /B 1\r
git clone --depth 1 https://github.com/%ORG%/cpython-source-deps --branch %%e "%EXTERNALS_DIR%\%%e"\r
) else (\r
echo.Fetching %%e...\r
- %PYTHON% "%PCBUILD%get_external.py" -O %ORG% %%e\r
+ %PYTHON% -E "%PCBUILD%\get_external.py" -O %ORG% -e "%EXTERNALS_DIR%" %%e\r
)\r
)\r
\r
git clone --depth 1 https://github.com/%ORG%/cpython-bin-deps --branch %%b "%EXTERNALS_DIR%\%%b"\r
) else (\r
echo.Fetching %%b...\r
- %PYTHON% "%PCBUILD%get_external.py" -b -O %ORG% %%b\r
+ %PYTHON% -E "%PCBUILD%\get_external.py" -b -O %ORG% -e "%EXTERNALS_DIR%" %%b\r
)\r
)\r
\r
<!-- pyshellext.dll -->\r
<Projects Include="pyshellext.vcxproj" />\r
<!-- Extension modules -->\r
- <ExtensionModules Include="_asyncio;_ctypes;_decimal;_elementtree;_msi;_multiprocessing;_overlapped;pyexpat;select;unicodedata;winsound" />\r
+ <ExtensionModules Include="_asyncio;_ctypes;_decimal;_distutils_findvs;_elementtree;_msi;_multiprocessing;_overlapped;pyexpat;select;unicodedata;winsound" />\r
<!-- Extension modules that require external sources -->\r
<ExternalModules Include="_bz2;_lzma;_sqlite3" />\r
<!-- _ssl will build _socket as well, which may cause conflicts in parallel builds -->\r
EndProject\r
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "liblzma", "liblzma.vcxproj", "{12728250-16EC-4DC6-94D7-E21DD88947F8}"\r
EndProject\r
+Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "_distutils_findvs", "_distutils_findvs.vcxproj", "{41ADEDF9-11D8-474E-B4D7-BB82332C878E}"\r
+EndProject\r
Global\r
GlobalSection(SolutionConfigurationPlatforms) = preSolution\r
Debug|Win32 = Debug|Win32\r
{12728250-16EC-4DC6-94D7-E21DD88947F8}.Release|Win32.Build.0 = Release|Win32\r
{12728250-16EC-4DC6-94D7-E21DD88947F8}.Release|x64.ActiveCfg = Release|x64\r
{12728250-16EC-4DC6-94D7-E21DD88947F8}.Release|x64.Build.0 = Release|x64\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.Debug|Win32.ActiveCfg = Debug|Win32\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.Debug|Win32.Build.0 = Debug|Win32\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.Debug|x64.ActiveCfg = Debug|x64\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.Debug|x64.Build.0 = Debug|x64\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.PGInstrument|Win32.ActiveCfg = PGInstrument|Win32\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.PGInstrument|Win32.Build.0 = PGInstrument|Win32\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.PGInstrument|x64.ActiveCfg = PGInstrument|x64\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.PGInstrument|x64.Build.0 = PGInstrument|x64\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.PGUpdate|Win32.ActiveCfg = PGUpdate|Win32\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.PGUpdate|Win32.Build.0 = PGUpdate|Win32\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.PGUpdate|x64.ActiveCfg = PGUpdate|x64\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.PGUpdate|x64.Build.0 = PGUpdate|x64\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.Release|Win32.ActiveCfg = Release|Win32\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.Release|Win32.Build.0 = Release|Win32\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.Release|x64.ActiveCfg = Release|x64\r
+ {41ADEDF9-11D8-474E-B4D7-BB82332C878E}.Release|x64.Build.0 = Release|x64\r
EndGlobalSection\r
GlobalSection(SolutionProperties) = preSolution\r
HideSolutionNode = FALSE\r
EndGlobalSection\r
+ GlobalSection(ExtensibilityGlobals) = postSolution\r
+ SolutionGuid = {E8690CFE-326A-430E-82D4-B4CE667CC1BA}\r
+ EndGlobalSection\r
EndGlobal\r
<BuildPath Condition="$(Configuration) == 'PGInstrument'">$(BuildPath)instrumented\</BuildPath>\r
\r
<!-- Directories of external projects. tcltk is handled in tcltk.props -->\r
- <ExternalsDir>$([System.IO.Path]::GetFullPath(`$(PySourcePath)externals\`))</ExternalsDir>\r
+ <ExternalsDir>$(EXTERNALS_DIR)</ExternalsDir>\r
+ <ExternalsDir Condition="$(ExternalsDir) == ''">$([System.IO.Path]::GetFullPath(`$(PySourcePath)externals`))</ExternalsDir>\r
+ <ExternalsDir Condition="!HasTrailingSlash($(ExternalsDir))">$(ExternalsDir)\</ExternalsDir>\r
<sqlite3Dir>$(ExternalsDir)sqlite-3.21.0.0\</sqlite3Dir>\r
<bz2Dir>$(ExternalsDir)bzip2-1.0.6\</bz2Dir>\r
<lzmaDir>$(ExternalsDir)xz-5.2.2\</lzmaDir>\r
</ClCompile>\r
<Link>\r
<AdditionalDependencies>version.lib;shlwapi.lib;ws2_32.lib;%(AdditionalDependencies)</AdditionalDependencies>\r
- <AdditionalLibraryDirectories>%(AdditionalLibraryDirectories);$(PySourcePath)PC\external\$(PlatformToolset)\$(ArchName)</AdditionalLibraryDirectories>\r
<BaseAddress>0x1e000000</BaseAddress>\r
</Link>\r
</ItemDefinitionGroup>\r
<ClCompile Include="..\PC\config.c" />\r
<ClCompile Include="..\PC\getpathp.c" />\r
<ClCompile Include="..\PC\msvcrtmodule.c" />\r
- <ClCompile Include="..\PC\_findvs.cpp" />\r
<ClCompile Include="..\Python\pyhash.c" />\r
<ClCompile Include="..\Python\random.c" />\r
<ClCompile Include="..\Python\_warnings.c" />\r
return 0;
}
-static void bpo20891_thread(void *lockp)
-{
- PyThread_type_lock lock = *((PyThread_type_lock*)lockp);
-
- PyGILState_STATE state = PyGILState_Ensure();
- if (!PyGILState_Check()) {
- fprintf(stderr, "PyGILState_Check failed!");
- abort();
- }
-
- PyGILState_Release(state);
-
- PyThread_release_lock(lock);
-
- PyThread_exit_thread();
-}
-
-static int test_bpo20891(void)
-{
- /* bpo-20891: Calling PyGILState_Ensure in a non-Python thread before
- calling PyEval_InitThreads() must not crash. PyGILState_Ensure() must
- call PyEval_InitThreads() for us in this case. */
- PyThread_type_lock lock = PyThread_allocate_lock();
- if (!lock) {
- fprintf(stderr, "PyThread_allocate_lock failed!");
- return 1;
- }
-
- _testembed_Py_Initialize();
-
- long thrd = PyThread_start_new_thread(bpo20891_thread, &lock);
- if (thrd == -1) {
- fprintf(stderr, "PyThread_start_new_thread failed!");
- return 1;
- }
- PyThread_acquire_lock(lock, WAIT_LOCK);
-
- Py_BEGIN_ALLOW_THREADS
- /* wait until the thread exit */
- PyThread_acquire_lock(lock, WAIT_LOCK);
- Py_END_ALLOW_THREADS
-
- PyThread_free_lock(lock);
-
- return 0;
-}
-
/* *********************************************************
* List of test cases and the function that implements it.
{ "forced_io_encoding", test_forced_io_encoding },
{ "repeated_init_and_subinterpreters", test_repeated_init_and_subinterpreters },
{ "pre_initialization_api", test_pre_initialization_api },
- { "bpo20891", test_bpo20891 },
{ NULL, NULL }
};
"__import__(name, globals=None, locals=None, fromlist=(), level=0) -> module\n\
\n\
Import a module. Because this function is meant for use by the Python\n\
-interpreter and not for general use it is better to use\n\
+interpreter and not for general use, it is better to use\n\
importlib.import_module() to programmatically import a module.\n\
\n\
The globals argument is only used to determine the context;\n\
empty list to emulate ``import name''.\n\
When importing a module from a package, note that __import__('A.B', ...)\n\
returns package A when fromlist is empty, but its submodule B when\n\
-fromlist is not empty. Level is used to determine whether to perform \n\
-absolute or relative imports. 0 is absolute while a positive number\n\
+fromlist is not empty. The level argument is used to determine whether to\n\
+perform absolute or relative imports: 0 is absolute, while a positive number\n\
is the number of parent directories to search relative to the current module.");
case Ellipsis_kind:
case NameConstant_kind:
return 1;
+ case Name_kind:
+ return _PyUnicode_EqualToASCIIString(e->v.Name.id, "__debug__");
default:
return 0;
}
}
static PyObject *
-get_const_value(expr_ty e)
+get_const_value(struct compiler *c, expr_ty e)
{
switch (e->kind) {
case Constant_kind:
return Py_Ellipsis;
case NameConstant_kind:
return e->v.NameConstant.value;
+ case Name_kind:
+ assert(_PyUnicode_EqualToASCIIString(e->v.Name.id, "__debug__"));
+ return c->c_optimize ? Py_False : Py_True;
default:
assert(!is_const(e));
return NULL;
_Py_IDENTIFIER(StopAsyncIteration);
basicblock *try, *except, *end, *after_try, *try_cleanup,
- *after_loop, *after_loop_else;
+ *after_loop_else;
PyObject *stop_aiter_error = _PyUnicode_FromId(&PyId_StopAsyncIteration);
if (stop_aiter_error == NULL) {
end = compiler_new_block(c);
after_try = compiler_new_block(c);
try_cleanup = compiler_new_block(c);
- after_loop = compiler_new_block(c);
after_loop_else = compiler_new_block(c);
if (try == NULL || except == NULL || end == NULL
- || after_try == NULL || try_cleanup == NULL)
+ || after_try == NULL || try_cleanup == NULL
+ || after_loop_else == NULL)
return 0;
- ADDOP_JREL(c, SETUP_LOOP, after_loop);
+ ADDOP_JREL(c, SETUP_LOOP, end);
if (!compiler_push_fblock(c, LOOP, try))
return 0;
ADDOP(c, POP_BLOCK); /* for SETUP_LOOP */
compiler_pop_fblock(c, LOOP, try);
- compiler_use_next_block(c, after_loop);
- ADDOP_JABS(c, JUMP_ABSOLUTE, end);
-
compiler_use_next_block(c, after_loop_else);
VISIT_SEQ(c, stmt, s->v.For.orelse);
PyObject *mangled;
/* XXX AugStore isn't used anywhere! */
- mangled = _Py_Mangle(c->u->u_private, name);
- if (!mangled)
- return 0;
-
assert(!_PyUnicode_EqualToASCIIString(name, "None") &&
!_PyUnicode_EqualToASCIIString(name, "True") &&
!_PyUnicode_EqualToASCIIString(name, "False"));
+ if (ctx == Load && _PyUnicode_EqualToASCIIString(name, "__debug__")) {
+ ADDOP_O(c, LOAD_CONST, c->c_optimize ? Py_False : Py_True, consts);
+ return 1;
+ }
+
+ mangled = _Py_Mangle(c->u->u_private, name);
+ if (!mangled)
+ return 0;
+
op = 0;
optype = OP_NAME;
scope = PyST_GetScope(c->u->u_ste, mangled);
return 0;
}
for (i = begin; i < end; i++) {
- key = get_const_value((expr_ty)asdl_seq_GET(e->v.Dict.keys, i));
+ key = get_const_value(c, (expr_ty)asdl_seq_GET(e->v.Dict.keys, i));
Py_INCREF(key);
PyTuple_SET_ITEM(keys, i - begin, key);
}
_Py_IDENTIFIER(StopAsyncIteration);
comprehension_ty gen;
- basicblock *anchor, *skip, *if_cleanup, *try,
+ basicblock *anchor, *if_cleanup, *try,
*after_try, *except, *try_cleanup;
Py_ssize_t i, n;
after_try = compiler_new_block(c);
try_cleanup = compiler_new_block(c);
except = compiler_new_block(c);
- skip = compiler_new_block(c);
if_cleanup = compiler_new_block(c);
anchor = compiler_new_block(c);
- if (skip == NULL || if_cleanup == NULL || anchor == NULL ||
+ if (if_cleanup == NULL || anchor == NULL ||
try == NULL || after_try == NULL ||
- except == NULL || after_try == NULL) {
+ except == NULL || try_cleanup == NULL) {
return 0;
}
default:
return 0;
}
-
- compiler_use_next_block(c, skip);
}
compiler_use_next_block(c, if_cleanup);
ADDOP_JABS(c, JUMP_ABSOLUTE, try);
static int
expr_constant(struct compiler *c, expr_ty e)
{
- char *id;
- switch (e->kind) {
- case Ellipsis_kind:
- return 1;
- case Constant_kind:
- return PyObject_IsTrue(e->v.Constant.value);
- case Num_kind:
- return PyObject_IsTrue(e->v.Num.n);
- case Str_kind:
- return PyObject_IsTrue(e->v.Str.s);
- case Name_kind:
- /* optimize away names that can't be reassigned */
- id = PyUnicode_AsUTF8(e->v.Name.id);
- if (id && strcmp(id, "__debug__") == 0)
- return !c->c_optimize;
- return -1;
- case NameConstant_kind: {
- PyObject *o = e->v.NameConstant.value;
- if (o == Py_None)
- return 0;
- else if (o == Py_True)
- return 1;
- else if (o == Py_False)
- return 0;
- }
- /* fall through */
- default:
- return -1;
+ if (is_const(e)) {
+ return PyObject_IsTrue(get_const_value(c, e));
}
+ return -1;
}
{
PySTEntryObject *ste = c->u->u_ste;
int flags = 0;
- Py_ssize_t n;
if (ste->ste_type == FunctionBlock) {
flags |= CO_NEWLOCALS | CO_OPTIMIZED;
if (ste->ste_nested)
Py_RETURN_NONE;
}
-#if !defined(__APPLE__) && !defined(MS_WINDOWS)
+#if !defined(__APPLE__) && !defined(__ANDROID__) && !defined(MS_WINDOWS)
+
+#define USE_FORCE_ASCII
+
extern int _Py_normalize_encoding(const char *, char *, size_t);
/* Workaround FreeBSD and OpenIndiana locale encoding issue with the C locale.
}
#endif /* !defined(__APPLE__) && !defined(MS_WINDOWS) */
-#if !defined(__APPLE__) && (!defined(MS_WINDOWS) || !defined(HAVE_MBRTOWC))
+#if !defined(HAVE_MBRTOWC) || defined(USE_FORCE_ASCII)
static wchar_t*
decode_ascii_surrogateescape(const char *arg, size_t *size)
{
#endif
-/* Decode a byte string from the locale encoding with the
- surrogateescape error handler: undecodable bytes are decoded as characters
- in range U+DC80..U+DCFF. If a byte sequence can be decoded as a surrogate
- character, escape the bytes using the surrogateescape error handler instead
- of decoding them.
-
- Return a pointer to a newly allocated wide character string, use
- PyMem_RawFree() to free the memory. If size is not NULL, write the number of
- wide characters excluding the null character into *size
-
- Return NULL on decoding error or memory allocation error. If *size* is not
- NULL, *size is set to (size_t)-1 on memory error or set to (size_t)-2 on
- decoding error.
-
- Decoding errors should never happen, unless there is a bug in the C
- library.
-
- Use the Py_EncodeLocale() function to encode the character string back to a
- byte string. */
-wchar_t*
-Py_DecodeLocale(const char* arg, size_t *size)
+static wchar_t*
+decode_current_locale(const char* arg, size_t *size)
{
-#if defined(__APPLE__) || defined(__ANDROID__)
- wchar_t *wstr;
- wstr = _Py_DecodeUTF8_surrogateescape(arg, strlen(arg));
- if (size != NULL) {
- if (wstr != NULL)
- *size = wcslen(wstr);
- else
- *size = (size_t)-1;
- }
- return wstr;
-#else
wchar_t *res;
size_t argsize;
size_t count;
mbstate_t mbs;
#endif
-#ifndef MS_WINDOWS
- if (force_ascii == -1)
- force_ascii = check_force_ascii();
-
- if (force_ascii) {
- /* force ASCII encoding to workaround mbstowcs() issue */
- res = decode_ascii_surrogateescape(arg, size);
- if (res == NULL)
- goto oom;
- return res;
- }
-#endif
-
#ifdef HAVE_BROKEN_MBSTOWCS
/* Some platforms have a broken implementation of
* mbstowcs which does not count the characters that
goto oom;
#endif /* HAVE_MBRTOWC */
return res;
+
oom:
if (size != NULL)
*size = (size_t)-1;
return NULL;
+}
+
+
+static wchar_t*
+decode_locale(const char* arg, size_t *size, int current_locale)
+{
+ if (current_locale) {
+ return decode_current_locale(arg, size);
+ }
+
+#if defined(__APPLE__) || defined(__ANDROID__)
+ wchar_t *wstr;
+ wstr = _Py_DecodeUTF8_surrogateescape(arg, strlen(arg));
+ if (size != NULL) {
+ if (wstr != NULL)
+ *size = wcslen(wstr);
+ else
+ *size = (size_t)-1;
+ }
+ return wstr;
+#else
+
+#ifdef USE_FORCE_ASCII
+ if (force_ascii == -1) {
+ force_ascii = check_force_ascii();
+ }
+
+ if (force_ascii) {
+ /* force ASCII encoding to workaround mbstowcs() issue */
+ wchar_t *res = decode_ascii_surrogateescape(arg, size);
+ if (res == NULL) {
+ if (size != NULL)
+ *size = (size_t)-1;
+ return NULL;
+ }
+ return res;
+ }
+#endif
+
+ return decode_current_locale(arg, size);
#endif /* __APPLE__ or __ANDROID__ */
}
-/* Encode a wide character string to the locale encoding with the
- surrogateescape error handler: surrogate characters in the range
- U+DC80..U+DCFF are converted to bytes 0x80..0xFF.
- Return a pointer to a newly allocated byte string, use PyMem_Free() to free
- the memory. Return NULL on encoding or memory allocation error.
+/* Decode a byte string from the locale encoding with the
+ surrogateescape error handler: undecodable bytes are decoded as characters
+ in range U+DC80..U+DCFF. If a byte sequence can be decoded as a surrogate
+ character, escape the bytes using the surrogateescape error handler instead
+ of decoding them.
- If error_pos is not NULL, *error_pos is set to the index of the invalid
- character on encoding error, or set to (size_t)-1 otherwise.
+ Return a pointer to a newly allocated wide character string, use
+ PyMem_RawFree() to free the memory. If size is not NULL, write the number of
+ wide characters excluding the null character into *size
- Use the Py_DecodeLocale() function to decode the bytes string back to a wide
- character string. */
-char*
-Py_EncodeLocale(const wchar_t *text, size_t *error_pos)
+ Return NULL on decoding error or memory allocation error. If *size* is not
+ NULL, *size is set to (size_t)-1 on memory error or set to (size_t)-2 on
+ decoding error.
+
+ Decoding errors should never happen, unless there is a bug in the C
+ library.
+
+ Use the Py_EncodeLocale() function to encode the character string back to a
+ byte string. */
+wchar_t*
+Py_DecodeLocale(const char* arg, size_t *size)
{
-#if defined(__APPLE__) || defined(__ANDROID__)
- Py_ssize_t len;
- PyObject *unicode, *bytes = NULL;
- char *cpath;
+ return decode_locale(arg, size, 0);
+}
- unicode = PyUnicode_FromWideChar(text, wcslen(text));
- if (unicode == NULL)
- return NULL;
- bytes = _PyUnicode_AsUTF8String(unicode, "surrogateescape");
- Py_DECREF(unicode);
- if (bytes == NULL) {
- PyErr_Clear();
- if (error_pos != NULL)
- *error_pos = (size_t)-1;
- return NULL;
- }
+wchar_t*
+_Py_DecodeLocaleEx(const char* arg, size_t *size, int current_locale)
+{
+ return decode_locale(arg, size, current_locale);
+}
- len = PyBytes_GET_SIZE(bytes);
- cpath = PyMem_Malloc(len+1);
- if (cpath == NULL) {
- PyErr_Clear();
- Py_DECREF(bytes);
- if (error_pos != NULL)
- *error_pos = (size_t)-1;
- return NULL;
- }
- memcpy(cpath, PyBytes_AsString(bytes), len + 1);
- Py_DECREF(bytes);
- return cpath;
-#else /* __APPLE__ */
+
+static char*
+encode_current_locale(const wchar_t *text, size_t *error_pos)
+{
const size_t len = wcslen(text);
char *result = NULL, *bytes = NULL;
size_t i, size, converted;
wchar_t c, buf[2];
-#ifndef MS_WINDOWS
- if (force_ascii == -1)
- force_ascii = check_force_ascii();
-
- if (force_ascii)
- return encode_ascii_surrogateescape(text, error_pos);
-#endif
-
/* The function works in two steps:
1. compute the length of the output buffer in bytes (size)
2. outputs the bytes */
bytes = result;
}
return result;
+}
+
+
+static char*
+encode_locale(const wchar_t *text, size_t *error_pos, int current_locale)
+{
+ if (current_locale) {
+ return encode_current_locale(text, error_pos);
+ }
+
+#if defined(__APPLE__) || defined(__ANDROID__)
+ Py_ssize_t len;
+ PyObject *unicode, *bytes = NULL;
+ char *cpath;
+
+ unicode = PyUnicode_FromWideChar(text, wcslen(text));
+ if (unicode == NULL)
+ return NULL;
+
+ bytes = _PyUnicode_AsUTF8String(unicode, "surrogateescape");
+ Py_DECREF(unicode);
+ if (bytes == NULL) {
+ PyErr_Clear();
+ if (error_pos != NULL)
+ *error_pos = (size_t)-1;
+ return NULL;
+ }
+
+ len = PyBytes_GET_SIZE(bytes);
+ cpath = PyMem_Malloc(len+1);
+ if (cpath == NULL) {
+ PyErr_Clear();
+ Py_DECREF(bytes);
+ if (error_pos != NULL)
+ *error_pos = (size_t)-1;
+ return NULL;
+ }
+ memcpy(cpath, PyBytes_AsString(bytes), len + 1);
+ Py_DECREF(bytes);
+ return cpath;
+#else /* __APPLE__ */
+
+#ifdef USE_FORCE_ASCII
+ if (force_ascii == -1) {
+ force_ascii = check_force_ascii();
+ }
+
+ if (force_ascii) {
+ return encode_ascii_surrogateescape(text, error_pos);
+ }
+#endif
+
+ return encode_current_locale(text, error_pos);
#endif /* __APPLE__ or __ANDROID__ */
}
+/* Encode a wide character string to the locale encoding with the
+ surrogateescape error handler: surrogate characters in the range
+ U+DC80..U+DCFF are converted to bytes 0x80..0xFF.
+
+ Return a pointer to a newly allocated byte string, use PyMem_Free() to free
+ the memory. Return NULL on encoding or memory allocation error.
+
+ If error_pos is not NULL, *error_pos is set to the index of the invalid
+ character on encoding error, or set to (size_t)-1 otherwise.
+
+ Use the Py_DecodeLocale() function to decode the bytes string back to a wide
+ character string. */
+char*
+Py_EncodeLocale(const wchar_t *text, size_t *error_pos)
+{
+ return encode_locale(text, error_pos, 0);
+}
+
+
+char*
+_Py_EncodeLocaleEx(const wchar_t *text, size_t *error_pos, int current_locale)
+{
+ return encode_locale(text, error_pos, current_locale);
+}
+
+
#ifdef MS_WINDOWS
static __int64 secs_between_epochs = 11644473600; /* Seconds between 1.1.1601 and 1.1.1970 */
}
+/* This function MUST be kept async-signal-safe on POSIX when raise=0. */
static int
get_inheritable(int fd, int raise)
{
return get_inheritable(fd, 1);
}
+
+/* This function MUST be kept async-signal-safe on POSIX when raise=0. */
static int
set_inheritable(int fd, int inheritable, int raise, int *atomic_flag_works)
{
#else
#if defined(HAVE_SYS_IOCTL_H) && defined(FIOCLEX) && defined(FIONCLEX)
- if (ioctl_works != 0) {
+ if (ioctl_works != 0 && raise != 0) {
/* fast-path: ioctl() only requires one syscall */
+ /* caveat: raise=0 is an indicator that we must be async-signal-safe
+ * thus avoid using ioctl() so we skip the fast-path. */
if (inheritable)
request = FIONCLEX;
else
}
/* Set the inheritable flag of the specified file descriptor.
- On success: return 0, on error: raise an exception if raise is nonzero
- and return -1.
+ On success: return 0, on error: raise an exception and return -1.
If atomic_flag_works is not NULL:
return set_inheritable(fd, inheritable, 1, atomic_flag_works);
}
+/* Same as _Py_set_inheritable() but on error, set errno and
+ don't raise an exception.
+ This function is async-signal-safe. */
+int
+_Py_set_inheritable_async_safe(int fd, int inheritable, int *atomic_flag_works)
+{
+ return set_inheritable(fd, inheritable, 0, atomic_flag_works);
+}
+
static int
_Py_open_impl(const char *pathname, int flags, int gil_held)
{
if (wpath == NULL)
return NULL;
- usize = MultiByteToWideChar(CP_ACP, 0, mode, -1, wmode, sizeof(wmode));
+ usize = MultiByteToWideChar(CP_ACP, 0, mode, -1,
+ wmode, Py_ARRAY_LENGTH(wmode));
if (usize == 0) {
PyErr_SetFromWindowsErr(0);
return NULL;
return -1;
}
#endif
+
+
+int
+_Py_GetLocaleconvNumeric(PyObject **decimal_point, PyObject **thousands_sep,
+ const char **grouping)
+{
+ int res = -1;
+
+ struct lconv *lc = localeconv();
+
+ int change_locale = 0;
+ if (decimal_point != NULL &&
+ (strlen(lc->decimal_point) > 1 || ((unsigned char)lc->decimal_point[0]) > 127))
+ {
+ change_locale = 1;
+ }
+ if (thousands_sep != NULL &&
+ (strlen(lc->thousands_sep) > 1 || ((unsigned char)lc->thousands_sep[0]) > 127))
+ {
+ change_locale = 1;
+ }
+
+ /* Keep a copy of the LC_CTYPE locale */
+ char *oldloc = NULL, *loc = NULL;
+ if (change_locale) {
+ oldloc = setlocale(LC_CTYPE, NULL);
+ if (!oldloc) {
+ PyErr_SetString(PyExc_RuntimeWarning, "faild to get LC_CTYPE locale");
+ return -1;
+ }
+
+ oldloc = _PyMem_Strdup(oldloc);
+ if (!oldloc) {
+ PyErr_NoMemory();
+ return -1;
+ }
+
+ loc = setlocale(LC_NUMERIC, NULL);
+ if (loc != NULL && strcmp(loc, oldloc) == 0) {
+ loc = NULL;
+ }
+
+ if (loc != NULL) {
+ /* Only set the locale temporarilty the LC_CTYPE locale
+ if LC_NUMERIC locale is different than LC_CTYPE locale and
+ decimal_point and/or thousands_sep are non-ASCII or longer than
+ 1 byte */
+ setlocale(LC_CTYPE, loc);
+ }
+ }
+
+ if (decimal_point != NULL) {
+ *decimal_point = PyUnicode_DecodeLocale(lc->decimal_point, NULL);
+ if (*decimal_point == NULL) {
+ goto error;
+ }
+ }
+ if (thousands_sep != NULL) {
+ *thousands_sep = PyUnicode_DecodeLocale(lc->thousands_sep, NULL);
+ if (*thousands_sep == NULL) {
+ goto error;
+ }
+ }
+
+ if (grouping != NULL) {
+ *grouping = lc->grouping;
+ }
+
+ res = 0;
+
+error:
+ if (loc != NULL) {
+ setlocale(LC_CTYPE, oldloc);
+ }
+ PyMem_Free(oldloc);
+ return res;
+}
{
switch (type) {
case LT_CURRENT_LOCALE: {
- struct lconv *locale_data = localeconv();
- locale_info->decimal_point = PyUnicode_DecodeLocale(
- locale_data->decimal_point,
- NULL);
- if (locale_info->decimal_point == NULL)
+ if (_Py_GetLocaleconvNumeric(&locale_info->decimal_point,
+ &locale_info->thousands_sep,
+ &locale_info->grouping) < 0) {
return -1;
- locale_info->thousands_sep = PyUnicode_DecodeLocale(
- locale_data->thousands_sep,
- NULL);
- if (locale_info->thousands_sep == NULL)
- return -1;
- locale_info->grouping = locale_data->grouping;
+ }
break;
}
case LT_DEFAULT_LOCALE:
}
}
- /* We are into optional args, skip thru to any remaining
+ /* We are into optional args, skip through to any remaining
* keyword args */
msg = skipitem(&format, p_va, flags);
if (msg) {
return cleanreturn(1, &freelist);
}
- /* We are into optional args, skip thru to any remaining
+ /* We are into optional args, skip through to any remaining
* keyword args */
msg = skipitem(&format, p_va, flags);
assert(msg == NULL);
static const char cprt[] =
"\
-Copyright (c) 2001-2017 Python Software Foundation.\n\
+Copyright (c) 2001-2018 Python Software Foundation.\n\
All Rights Reserved.\n\
\n\
Copyright (c) 2000 BeOpen.com.\n\
return maxi - 1;
}
+/* Check whether a collection doesn't containing too much items (including
+ subcollections). This protects from creating a constant that needs
+ too much time for calculating a hash.
+ "limit" is the maximal number of items.
+ Returns the negative number if the total number of items exceeds the
+ limit. Otherwise returns the limit minus the total number of items.
+*/
+
+static Py_ssize_t
+check_complexity(PyObject *obj, Py_ssize_t limit)
+{
+ if (PyTuple_Check(obj)) {
+ Py_ssize_t i;
+ limit -= PyTuple_GET_SIZE(obj);
+ for (i = 0; limit >= 0 && i < PyTuple_GET_SIZE(obj); i++) {
+ limit = check_complexity(PyTuple_GET_ITEM(obj, i), limit);
+ }
+ return limit;
+ }
+ else if (PyFrozenSet_Check(obj)) {
+ Py_ssize_t i = 0;
+ PyObject *item;
+ Py_hash_t hash;
+ limit -= PySet_GET_SIZE(obj);
+ while (limit >= 0 && _PySet_NextEntry(obj, &i, &item, &hash)) {
+ limit = check_complexity(item, limit);
+ }
+ }
+ return limit;
+}
+
/* Replace LOAD_CONST c1, LOAD_CONST c2 ... LOAD_CONST cn, BUILD_TUPLE n
with LOAD_CONST (c1, c2, ... cn).
The consts table must still be in list form so that the
return copy_op_arg(codestr, c_start, LOAD_CONST, len_consts, opcode_end);
}
+#define MAX_INT_SIZE 128 /* bits */
+#define MAX_COLLECTION_SIZE 20 /* items */
+#define MAX_STR_SIZE 20 /* characters */
+#define MAX_TOTAL_ITEMS 1024 /* including nested collections */
+
+static PyObject *
+safe_multiply(PyObject *v, PyObject *w)
+{
+ if (PyLong_Check(v) && PyLong_Check(w) && Py_SIZE(v) && Py_SIZE(w)) {
+ size_t vbits = _PyLong_NumBits(v);
+ size_t wbits = _PyLong_NumBits(w);
+ if (vbits == (size_t)-1 || wbits == (size_t)-1) {
+ return NULL;
+ }
+ if (vbits + wbits > MAX_INT_SIZE) {
+ return NULL;
+ }
+ }
+ else if (PyLong_Check(v) && (PyTuple_Check(w) || PyFrozenSet_Check(w))) {
+ Py_ssize_t size = PyTuple_Check(w) ? PyTuple_GET_SIZE(w) :
+ PySet_GET_SIZE(w);
+ if (size) {
+ long n = PyLong_AsLong(v);
+ if (n < 0 || n > MAX_COLLECTION_SIZE / size) {
+ return NULL;
+ }
+ if (n && check_complexity(w, MAX_TOTAL_ITEMS / n) < 0) {
+ return NULL;
+ }
+ }
+ }
+ else if (PyLong_Check(v) && (PyUnicode_Check(w) || PyBytes_Check(w))) {
+ Py_ssize_t size = PyUnicode_Check(w) ? PyUnicode_GET_LENGTH(w) :
+ PyBytes_GET_SIZE(w);
+ if (size) {
+ long n = PyLong_AsLong(v);
+ if (n < 0 || n > MAX_STR_SIZE / size) {
+ return NULL;
+ }
+ }
+ }
+ else if (PyLong_Check(w) &&
+ (PyTuple_Check(v) || PyFrozenSet_Check(v) ||
+ PyUnicode_Check(v) || PyBytes_Check(v)))
+ {
+ return safe_multiply(w, v);
+ }
+
+ return PyNumber_Multiply(v, w);
+}
+
+static PyObject *
+safe_power(PyObject *v, PyObject *w)
+{
+ if (PyLong_Check(v) && PyLong_Check(w) && Py_SIZE(v) && Py_SIZE(w) > 0) {
+ size_t vbits = _PyLong_NumBits(v);
+ size_t wbits = PyLong_AsSize_t(w);
+ if (vbits == (size_t)-1 || wbits == (size_t)-1) {
+ return NULL;
+ }
+ if (vbits > MAX_INT_SIZE / wbits) {
+ return NULL;
+ }
+ }
+
+ return PyNumber_Power(v, w, Py_None);
+}
+
+static PyObject *
+safe_lshift(PyObject *v, PyObject *w)
+{
+ if (PyLong_Check(v) && PyLong_Check(w) && Py_SIZE(v) && Py_SIZE(w)) {
+ size_t vbits = _PyLong_NumBits(v);
+ size_t wbits = PyLong_AsSize_t(w);
+ if (vbits == (size_t)-1 || wbits == (size_t)-1) {
+ return NULL;
+ }
+ if (wbits > MAX_INT_SIZE || vbits > MAX_INT_SIZE - wbits) {
+ return NULL;
+ }
+ }
+
+ return PyNumber_Lshift(v, w);
+}
+
+static PyObject *
+safe_mod(PyObject *v, PyObject *w)
+{
+ if (PyUnicode_Check(v) || PyBytes_Check(v)) {
+ return NULL;
+ }
+
+ return PyNumber_Remainder(v, w);
+}
+
/* Replace LOAD_CONST c1, LOAD_CONST c2, BINOP
with LOAD_CONST binop(c1,c2)
The consts table must still be in list form so that the
PyObject *consts, PyObject **objs)
{
PyObject *newconst, *v, *w;
- Py_ssize_t len_consts, size;
+ Py_ssize_t len_consts;
/* Pre-conditions */
assert(PyList_CheckExact(consts));
w = objs[1];
switch (opcode) {
case BINARY_POWER:
- newconst = PyNumber_Power(v, w, Py_None);
+ newconst = safe_power(v, w);
break;
case BINARY_MULTIPLY:
- newconst = PyNumber_Multiply(v, w);
+ newconst = safe_multiply(v, w);
break;
case BINARY_TRUE_DIVIDE:
newconst = PyNumber_TrueDivide(v, w);
newconst = PyNumber_FloorDivide(v, w);
break;
case BINARY_MODULO:
- newconst = PyNumber_Remainder(v, w);
+ newconst = safe_mod(v, w);
break;
case BINARY_ADD:
newconst = PyNumber_Add(v, w);
newconst = PyObject_GetItem(v, w);
break;
case BINARY_LSHIFT:
- newconst = PyNumber_Lshift(v, w);
+ newconst = safe_lshift(v, w);
break;
case BINARY_RSHIFT:
newconst = PyNumber_Rshift(v, w);
}
return -1;
}
- size = PyObject_Size(newconst);
- if (size == -1) {
- if (PyErr_ExceptionMatches(PyExc_KeyboardInterrupt)) {
- return -1;
- }
- PyErr_Clear();
- } else if (size > 20) {
- Py_DECREF(newconst);
- return -1;
- }
/* Append folded constant into consts table */
if (PyList_Append(consts, newconst)) {
Py_OptimizeFlag = add_flag(Py_OptimizeFlag, p);
if ((p = Py_GETENV("PYTHONDONTWRITEBYTECODE")) && *p != '\0')
Py_DontWriteBytecodeFlag = add_flag(Py_DontWriteBytecodeFlag, p);
- /* The variable is only tested for existence here; _PyRandom_Init will
- check its value further. */
- if ((p = Py_GETENV("PYTHONHASHSEED")) && *p != '\0')
- Py_HashRandomizationFlag = add_flag(Py_HashRandomizationFlag, p);
#ifdef MS_WINDOWS
if ((p = Py_GETENV("PYTHONLEGACYWINDOWSFSENCODING")) && *p != '\0')
Py_LegacyWindowsFSEncodingFlag = add_flag(Py_LegacyWindowsFSEncodingFlag, p);
if (seed == 0) {
/* disable the randomized hash */
memset(secret, 0, secret_size);
+ Py_HashRandomizationFlag = 0;
}
else {
lcg_urandom(seed, secret, secret_size);
+ Py_HashRandomizationFlag = 1;
}
}
else {
if (res < 0) {
Py_FatalError("failed to get random numbers to initialize Python");
}
+ Py_HashRandomizationFlag = 1;
}
}
}
/*
- * Lock support. It has too be implemented as semaphores.
+ * Lock support. It has to be implemented as semaphores.
* I [Dag] tried to implement it with mutex but I could find a way to
* tell whether a thread already own the lock or not.
*/
-This is Python version 3.6.4
+This is Python version 3.6.5
============================
.. image:: https://travis-ci.org/python/cpython.svg?branch=3.6
:target: https://codecov.io/gh/python/cpython
Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011,
-2012, 2013, 2014, 2015, 2016, 2017 Python Software Foundation. All rights
+2012, 2013, 2014, 2015, 2016, 2017, 2018 Python Software Foundation. All rights
reserved.
See the end of this file for further copyright and license information.
---------------------------------
Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011,
-2012, 2013, 2014, 2015, 2016, 2017 Python Software Foundation. All rights
+2012, 2013, 2014, 2015, 2016, 2017, 2018 Python Software Foundation. All rights
reserved.
Copyright (c) 2000 BeOpen.com. All rights reserved.
self.__lineno = -1
self.__freshmodule = 1
self.__curfile = None
+ self.__enclosurecount = 0
def __call__(self, ttype, tstring, stup, etup, line):
# dispatch
elif ttype not in (tokenize.COMMENT, tokenize.NL):
self.__freshmodule = 0
return
- # class docstring?
+ # class or func/method docstring?
if ttype == tokenize.NAME and tstring in ('class', 'def'):
self.__state = self.__suiteseen
return
self.__state = self.__keywordseen
def __suiteseen(self, ttype, tstring, lineno):
- # ignore anything until we see the colon
- if ttype == tokenize.OP and tstring == ':':
- self.__state = self.__suitedocstring
+ # skip over any enclosure pairs until we see the colon
+ if ttype == tokenize.OP:
+ if tstring == ':' and self.__enclosurecount == 0:
+ # we see a colon and we're not in an enclosure: end of def
+ self.__state = self.__suitedocstring
+ elif tstring in '([{':
+ self.__enclosurecount += 1
+ elif tstring in ')]}':
+ self.__enclosurecount -= 1
def __suitedocstring(self, ttype, tstring, lineno):
# ignore any intervening noise
<?xml version="1.0" encoding="UTF-8"?>
<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi">
- <?define exts=pyexpat;select;unicodedata;winsound;_bz2;_elementtree;_socket;_ssl;_msi;_ctypes;_hashlib;_multiprocessing;_lzma;_decimal;_overlapped;_sqlite3;_asyncio ?>
+ <?define exts=pyexpat;select;unicodedata;winsound;_bz2;_elementtree;_socket;_ssl;_msi;_ctypes;_hashlib;_multiprocessing;_lzma;_decimal;_overlapped;_sqlite3;_asyncio;_distutils_findvs ?>
<Fragment>
<ComponentGroup Id="lib_extensions">
<?foreach ext in $(var.exts)?>
--without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no)
--with-universal-archs=ARCH
select architectures for universal build ("32-bit",
- "64-bit", "3-way", "intel", "intel-32", or "all")
+ "64-bit", "3-way", "intel", "intel-32", "intel-64",
+ or "all")
--with-framework-name=FRAMEWORK
specify an alternate name of the framework built
with --enable-framework
LIPO_32BIT_FLAGS=""
ARCH_RUN_32BIT=""
;;
+ intel-64)
+ UNIVERSAL_ARCH_FLAGS="-arch x86_64"
+ LIPO_32BIT_FLAGS=""
+ ARCH_RUN_32BIT="true"
+ ;;
3-way)
UNIVERSAL_ARCH_FLAGS="-arch i386 -arch ppc -arch x86_64"
LIPO_32BIT_FLAGS="-extract ppc7400 -extract i386"
;;
esac
- CFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${CFLAGS}"
- LDFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${LDFLAGS}"
if test "${UNIVERSALSDK}" != "/"
then
+ CFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${CFLAGS}"
+ LDFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${LDFLAGS}"
CPPFLAGS="-isysroot ${UNIVERSALSDK} ${CPPFLAGS}"
+ else
+ CFLAGS="${UNIVERSAL_ARCH_FLAGS} ${CFLAGS}"
+ LDFLAGS="${UNIVERSAL_ARCH_FLAGS} ${LDFLAGS}"
fi
fi
fi
-for ac_header in asm/types.h conio.h direct.h dlfcn.h errno.h \
+for ac_header in asm/types.h crypt.h conio.h direct.h dlfcn.h errno.h \
fcntl.h grp.h \
ieeefp.h io.h langinfo.h libintl.h process.h pthread.h \
sched.h shadow.h signal.h stropts.h termios.h \
AC_SUBST(LIPO_32BIT_FLAGS)
AC_MSG_CHECKING(for --with-universal-archs)
AC_ARG_WITH(universal-archs,
- AS_HELP_STRING([--with-universal-archs=ARCH], [select architectures for universal build ("32-bit", "64-bit", "3-way", "intel", "intel-32", or "all")]),
+ AS_HELP_STRING([--with-universal-archs=ARCH], [select architectures for universal build ("32-bit", "64-bit", "3-way", "intel", "intel-32", "intel-64", or "all")]),
[
UNIVERSAL_ARCHS="$withval"
],
LIPO_32BIT_FLAGS=""
ARCH_RUN_32BIT=""
;;
+ intel-64)
+ UNIVERSAL_ARCH_FLAGS="-arch x86_64"
+ LIPO_32BIT_FLAGS=""
+ ARCH_RUN_32BIT="true"
+ ;;
3-way)
UNIVERSAL_ARCH_FLAGS="-arch i386 -arch ppc -arch x86_64"
LIPO_32BIT_FLAGS="-extract ppc7400 -extract i386"
;;
esac
- CFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${CFLAGS}"
- LDFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${LDFLAGS}"
if test "${UNIVERSALSDK}" != "/"
then
+ CFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${CFLAGS}"
+ LDFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${LDFLAGS}"
CPPFLAGS="-isysroot ${UNIVERSALSDK} ${CPPFLAGS}"
+ else
+ CFLAGS="${UNIVERSAL_ARCH_FLAGS} ${CFLAGS}"
+ LDFLAGS="${UNIVERSAL_ARCH_FLAGS} ${LDFLAGS}"
fi
fi
# checks for header files
AC_HEADER_STDC
-AC_CHECK_HEADERS(asm/types.h conio.h direct.h dlfcn.h errno.h \
+AC_CHECK_HEADERS(asm/types.h crypt.h conio.h direct.h dlfcn.h errno.h \
fcntl.h grp.h \
ieeefp.h io.h langinfo.h libintl.h process.h pthread.h \
sched.h shadow.h signal.h stropts.h termios.h \
/* Define to 1 if you have the `copysign' function. */
#undef HAVE_COPYSIGN
+/* Define to 1 if you have the <crypt.h> header file. */
+#undef HAVE_CRYPT_H
+
/* Define to 1 if you have the `ctermid' function. */
#undef HAVE_CTERMID
exts.append( Extension('termios', ['termios.c']) )
# Jeremy Hylton's rlimit interface
exts.append( Extension('resource', ['resource.c']) )
+ else:
+ missing.extend(['resource', 'termios'])
- # Sun yellow pages. Some systems have the functions in libc.
- if (host_platform not in ['cygwin', 'qnx6'] and
- find_file('rpcsvc/yp_prot.h', inc_dirs, []) is not None):
- if (self.compiler.find_library_file(lib_dirs, 'nsl')):
- libs = ['nsl']
- else:
- libs = []
- exts.append( Extension('nis', ['nismodule.c'],
- libraries = libs) )
- else:
- missing.append('nis')
+ nis = self._detect_nis(inc_dirs, lib_dirs)
+ if nis is not None:
+ exts.append(nis)
else:
- missing.extend(['nis', 'resource', 'termios'])
+ missing.append('nis')
# Curses support, requiring the System V version of curses, often
# provided by the ncurses library.
if host_platform == 'darwin':
# On OS X, there is no separate /usr/lib/libncursesw nor
# libpanelw. If we are here, we found a locally-supplied
- # version of libncursesw. There should be also be a
+ # version of libncursesw. There should also be a
# libpanelw. _XOPEN_SOURCE defines are usually excluded
# for OS X but we need _XOPEN_SOURCE_EXTENDED here for
# ncurses wide char support
ext.libraries.append(ffi_lib)
self.use_system_libffi = True
+ if sysconfig.get_config_var('HAVE_LIBDL'):
+ # for dlopen, see bpo-32647
+ ext.libraries.append('dl')
+
def _decimal_ext(self):
extra_compile_args = []
undef_macros = []
)
return ext
+ def _detect_nis(self, inc_dirs, lib_dirs):
+ if host_platform in {'win32', 'cygwin', 'qnx6'}:
+ return None
+
+ libs = []
+ library_dirs = []
+ includes_dirs = []
+
+ # bpo-32521: glibc has deprecated Sun RPC for some time. Fedora 28
+ # moved headers and libraries to libtirpc and libnsl. The headers
+ # are in tircp and nsl sub directories.
+ rpcsvc_inc = find_file(
+ 'rpcsvc/yp_prot.h', inc_dirs,
+ [os.path.join(inc_dir, 'nsl') for inc_dir in inc_dirs]
+ )
+ rpc_inc = find_file(
+ 'rpc/rpc.h', inc_dirs,
+ [os.path.join(inc_dir, 'tirpc') for inc_dir in inc_dirs]
+ )
+ if rpcsvc_inc is None or rpc_inc is None:
+ # not found
+ return None
+ includes_dirs.extend(rpcsvc_inc)
+ includes_dirs.extend(rpc_inc)
+
+ if self.compiler.find_library_file(lib_dirs, 'nsl'):
+ libs.append('nsl')
+ else:
+ # libnsl-devel: check for libnsl in nsl/ subdirectory
+ nsl_dirs = [os.path.join(lib_dir, 'nsl') for lib_dir in lib_dirs]
+ libnsl = self.compiler.find_library_file(nsl_dirs, 'nsl')
+ if libnsl is not None:
+ library_dirs.append(os.path.dirname(libnsl))
+ libs.append('nsl')
+
+ if self.compiler.find_library_file(lib_dirs, 'tirpc'):
+ libs.append('tirpc')
+
+ return Extension(
+ 'nis', ['nismodule.c'],
+ libraries=libs,
+ library_dirs=library_dirs,
+ include_dirs=includes_dirs
+ )
+
+
class PyBuildInstall(install):
# Suppress the warning about installation into the lib_dynload
# directory, which is not in sys.path when running Python during
projects / platform / upstream / python3.git / commitdiff