From: Hyunjee Kim Date: Wed, 20 Mar 2019 01:53:40 +0000 (+0900) Subject: Imported Upstream version 3.6.5 X-Git-Tag: upstream/3.7.3~7 X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=d361874b526245a28217f9f1ea52de837efb0a65;p=platform%2Fupstream%2Fpython3.git Imported Upstream version 3.6.5 --- diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 2965bc93..f6014762 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -64,8 +64,8 @@ Initializing and finalizing the interpreter the last call to :c:func:`Py_Initialize`. Ideally, this frees all memory allocated by the Python interpreter. This is a no-op when called for a second time (without calling :c:func:`Py_Initialize` again first). Normally the - return value is 0. If there were errors during finalization - (flushing buffered data), -1 is returned. + return value is ``0``. If there were errors during finalization + (flushing buffered data), ``-1`` is returned. This function is provided for a number of reasons. An embedding application might want to restart Python without having to restart the application itself. @@ -1063,12 +1063,12 @@ Python-level trace functions in previous versions. +------------------------------+--------------------------------------+ | Value of *what* | Meaning of *arg* | +==============================+======================================+ - | :const:`PyTrace_CALL` | Always *NULL*. | + | :const:`PyTrace_CALL` | Always :c:data:`Py_None`. | +------------------------------+--------------------------------------+ | :const:`PyTrace_EXCEPTION` | Exception information as returned by | | | :func:`sys.exc_info`. | +------------------------------+--------------------------------------+ - | :const:`PyTrace_LINE` | Always *NULL*. | + | :const:`PyTrace_LINE` | Always :c:data:`Py_None`. | +------------------------------+--------------------------------------+ | :const:`PyTrace_RETURN` | Value being returned to the caller, | | | or *NULL* if caused by an exception. | @@ -1110,7 +1110,7 @@ Python-level trace functions in previous versions. .. c:var:: int PyTrace_RETURN The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a - call is returning without propagating an exception. + call is about to return. .. c:var:: int PyTrace_C_CALL @@ -1137,15 +1137,19 @@ Python-level trace functions in previous versions. function as its first parameter, and may be any Python object, or *NULL*. If the profile function needs to maintain state, using a different value for *obj* for each thread provides a convenient and thread-safe place to store it. The - profile function is called for all monitored events except the line-number - events. + profile function is called for all monitored events except :const:`PyTrace_LINE` + and :const:`PyTrace_EXCEPTION`. .. c:function:: void PyEval_SetTrace(Py_tracefunc func, PyObject *obj) Set the tracing function to *func*. This is similar to :c:func:`PyEval_SetProfile`, except the tracing function does receive line-number - events. + events and does not receive any event related to C function objects being called. Any + trace function registered using :c:func:`PyEval_SetTrace` will not receive + :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION` or :const:`PyTrace_C_RETURN` + as a value for the *what* parameter. + .. c:function:: PyObject* PyEval_GetCallStats(PyObject *self) diff --git a/Doc/c-api/long.rst b/Doc/c-api/long.rst index f50680b3..5b1f386f 100644 --- a/Doc/c-api/long.rst +++ b/Doc/c-api/long.rst @@ -10,6 +10,9 @@ Integer Objects All integers are implemented as "long" integer objects of arbitrary size. +On error, most ``PyLong_As*`` APIs return ``(return type)-1`` which cannot be +distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate. + .. c:type:: PyLongObject This subtype of :c:type:`PyObject` represents a Python integer object. @@ -134,6 +137,8 @@ All integers are implemented as "long" integer objects of arbitrary size. Raise :exc:`OverflowError` if the value of *obj* is out of range for a :c:type:`long`. + Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. + .. c:function:: long PyLong_AsLongAndOverflow(PyObject *obj, int *overflow) @@ -146,6 +151,8 @@ All integers are implemented as "long" integer objects of arbitrary size. return ``-1``; otherwise, set *\*overflow* to ``0``. If any other exception occurs set *\*overflow* to ``0`` and return ``-1`` as usual. + Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. + .. c:function:: long long PyLong_AsLongLong(PyObject *obj) @@ -159,6 +166,8 @@ All integers are implemented as "long" integer objects of arbitrary size. Raise :exc:`OverflowError` if the value of *obj* is out of range for a :c:type:`long`. + Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. + .. c:function:: long long PyLong_AsLongLongAndOverflow(PyObject *obj, int *overflow) @@ -171,6 +180,8 @@ All integers are implemented as "long" integer objects of arbitrary size. and return ``-1``; otherwise, set *\*overflow* to ``0``. If any other exception occurs set *\*overflow* to ``0`` and return ``-1`` as usual. + Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. + .. versionadded:: 3.2 @@ -186,6 +197,8 @@ All integers are implemented as "long" integer objects of arbitrary size. Raise :exc:`OverflowError` if the value of *pylong* is out of range for a :c:type:`Py_ssize_t`. + Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. + .. c:function:: unsigned long PyLong_AsUnsignedLong(PyObject *pylong) @@ -199,15 +212,25 @@ All integers are implemented as "long" integer objects of arbitrary size. Raise :exc:`OverflowError` if the value of *pylong* is out of range for a :c:type:`unsigned long`. + Returns ``(unsigned long)-1`` on error. + Use :c:func:`PyErr_Occurred` to disambiguate. + .. c:function:: size_t PyLong_AsSize_t(PyObject *pylong) + .. index:: + single: SIZE_MAX + single: OverflowError (built-in exception) + Return a C :c:type:`size_t` representation of *pylong*. *pylong* must be an instance of :c:type:`PyLongObject`. Raise :exc:`OverflowError` if the value of *pylong* is out of range for a :c:type:`size_t`. + Returns ``(size_t)-1`` on error. + Use :c:func:`PyErr_Occurred` to disambiguate. + .. c:function:: unsigned long long PyLong_AsUnsignedLongLong(PyObject *pylong) @@ -220,6 +243,9 @@ All integers are implemented as "long" integer objects of arbitrary size. Raise :exc:`OverflowError` if the value of *pylong* is out of range for an :c:type:`unsigned long long`. + Returns ``(unsigned long long)-1`` on error. + Use :c:func:`PyErr_Occurred` to disambiguate. + .. versionchanged:: 3.1 A negative *pylong* now raises :exc:`OverflowError`, not :exc:`TypeError`. @@ -233,6 +259,8 @@ All integers are implemented as "long" integer objects of arbitrary size. If the value of *obj* is out of range for an :c:type:`unsigned long`, return the reduction of that value modulo ``ULONG_MAX + 1``. + Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. + .. c:function:: unsigned long long PyLong_AsUnsignedLongLongMask(PyObject *obj) @@ -243,6 +271,8 @@ All integers are implemented as "long" integer objects of arbitrary size. If the value of *obj* is out of range for an :c:type:`unsigned long long`, return the reduction of that value modulo ``PY_ULLONG_MAX + 1``. + Returns ``-1`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. + .. c:function:: double PyLong_AsDouble(PyObject *pylong) @@ -252,6 +282,8 @@ All integers are implemented as "long" integer objects of arbitrary size. Raise :exc:`OverflowError` if the value of *pylong* is out of range for a :c:type:`double`. + Returns ``-1.0`` on error. Use :c:func:`PyErr_Occurred` to disambiguate. + .. c:function:: void* PyLong_AsVoidPtr(PyObject *pylong) @@ -259,3 +291,5 @@ All integers are implemented as "long" integer objects of arbitrary size. If *pylong* cannot be converted, an :exc:`OverflowError` will be raised. This is only assured to produce a usable :c:type:`void` pointer for values created with :c:func:`PyLong_FromVoidPtr`. + + Returns *NULL* on error. Use :c:func:`PyErr_Occurred` to disambiguate. diff --git a/Doc/c-api/sequence.rst b/Doc/c-api/sequence.rst index f1825f07..81f8557e 100644 --- a/Doc/c-api/sequence.rst +++ b/Doc/c-api/sequence.rst @@ -17,9 +17,8 @@ Sequence Protocol .. index:: builtin: len - Returns the number of objects in sequence *o* on success, and ``-1`` on failure. - For objects that do not provide sequence protocol, this is equivalent to the - Python expression ``len(o)``. + Returns the number of objects in sequence *o* on success, and ``-1`` on + failure. This is equivalent to the Python expression ``len(o)``. .. c:function:: PyObject* PySequence_Concat(PyObject *o1, PyObject *o2) diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index 035cdc16..48e2b2bb 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -66,9 +66,18 @@ Operating System Utilities surrogate character, escape the bytes using the surrogateescape error handler instead of decoding them. + Encoding, highest priority to lowest priority: + + * ``UTF-8`` on macOS and Android; + * ``ASCII`` if the ``LC_CTYPE`` locale is ``"C"``, + ``nl_langinfo(CODESET)`` returns the ``ASCII`` encoding (or an alias), + and :c:func:`mbstowcs` and :c:func:`wcstombs` functions use the + ``ISO-8859-1`` encoding. + * the current locale encoding (``LC_CTYPE`` locale). + Return a pointer to a newly allocated wide character string, use :c:func:`PyMem_RawFree` to free the memory. If size is not ``NULL``, write - the number of wide characters excluding the null character into ``*size`` + the number of wide characters excluding the null character into ``*size``. Return ``NULL`` on decoding error or memory allocation error. If *size* is not ``NULL``, ``*size`` is set to ``(size_t)-1`` on memory error or set to @@ -94,6 +103,15 @@ Operating System Utilities :ref:`surrogateescape error handler `: surrogate characters in the range U+DC80..U+DCFF are converted to bytes 0x80..0xFF. + Encoding, highest priority to lowest priority: + + * ``UTF-8`` on macOS and Android; + * ``ASCII`` if the ``LC_CTYPE`` locale is ``"C"``, + ``nl_langinfo(CODESET)`` returns the ``ASCII`` encoding (or an alias), + and :c:func:`mbstowcs` and :c:func:`wcstombs` functions uses the + ``ISO-8859-1`` encoding. + * the current locale encoding. + Return a pointer to a newly allocated byte string, use :c:func:`PyMem_Free` to free the memory. Return ``NULL`` on encoding error or memory allocation error diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index 6e91576e..b9acaec9 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -773,6 +773,12 @@ system. .. versionadded:: 3.3 + .. versionchanged:: 3.6.5 + The function now also uses the current locale encoding for the + ``surrogateescape`` error handler. Previously, :c:func:`Py_DecodeLocale` + was used for the ``surrogateescape``, and the current locale encoding was + used for ``strict``. + .. c:function:: PyObject* PyUnicode_DecodeLocale(const char *str, const char *errors) @@ -800,6 +806,12 @@ system. .. versionadded:: 3.3 + .. versionchanged:: 3.6.5 + The function now also uses the current locale encoding for the + ``surrogateescape`` error handler. Previously, :c:func:`Py_EncodeLocale` + was used for the ``surrogateescape``, and the current locale encoding was + used for ``strict``. + File System Encoding """""""""""""""""""" diff --git a/Doc/copyright.rst b/Doc/copyright.rst index 2b5400cd..540ff5ef 100644 --- a/Doc/copyright.rst +++ b/Doc/copyright.rst @@ -4,7 +4,7 @@ Copyright Python and this documentation is: -Copyright © 2001-2017 Python Software Foundation. All rights reserved. +Copyright © 2001-2018 Python Software Foundation. All rights reserved. Copyright © 2000 BeOpen.com. All rights reserved. diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst index d28d224c..edba57e4 100644 --- a/Doc/extending/newtypes.rst +++ b/Doc/extending/newtypes.rst @@ -391,8 +391,8 @@ is used to initialize an object after it's created. Unlike the new method, we can't guarantee that the initializer is called. The initializer isn't called when unpickling objects and it can be overridden. Our initializer accepts arguments to provide initial values for our instance. Initializers always accept -positional and keyword arguments. Initializers should return either 0 on -success or -1 on error. +positional and keyword arguments. Initializers should return either ``0`` on +success or ``-1`` on error. Initializers can be called multiple times. Anyone can call the :meth:`__init__` method on our objects. For this reason, we have to be extra careful when diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst index 3eafdf19..fd04a83d 100644 --- a/Doc/faq/extending.rst +++ b/Doc/faq/extending.rst @@ -53,7 +53,7 @@ with a tool such as `SWIG `_. `SIP `__, `CXX `_ `Boost `_, or `Weave -`_ are also +`_ are also alternatives for wrapping C++ libraries. @@ -62,7 +62,7 @@ How can I execute arbitrary Python statements from C? The highest-level function to do this is :c:func:`PyRun_SimpleString` which takes a single string argument to be executed in the context of the module -``__main__`` and returns 0 for success and -1 when an exception occurred +``__main__`` and returns ``0`` for success and ``-1`` when an exception occurred (including ``SyntaxError``). If you want more control, use :c:func:`PyRun_String`; see the source for :c:func:`PyRun_SimpleString` in ``Python/pythonrun.c``. diff --git a/Doc/glossary.rst b/Doc/glossary.rst index b947520b..aa5f47ed 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -391,7 +391,8 @@ Glossary garbage collection The process of freeing memory when it is not used anymore. Python performs garbage collection via reference counting and a cyclic garbage - collector that is able to detect and break reference cycles. + collector that is able to detect and break reference cycles. The + garbage collector can be controlled using the :mod:`gc` module. .. index:: single: generator diff --git a/Doc/howto/logging-cookbook.rst b/Doc/howto/logging-cookbook.rst index 44f38e03..598df539 100644 --- a/Doc/howto/logging-cookbook.rst +++ b/Doc/howto/logging-cookbook.rst @@ -941,7 +941,7 @@ Using file rotation ------------------- .. sectionauthor:: Doug Hellmann, Vinay Sajip (changes) -.. (see ) +.. (see ) Sometimes you want to let a log file grow to a certain size, then open a new file and log to that. You may want to keep a certain number of these files, and diff --git a/Doc/howto/regex.rst b/Doc/howto/regex.rst index eef63478..a3a65534 100644 --- a/Doc/howto/regex.rst +++ b/Doc/howto/regex.rst @@ -289,6 +289,8 @@ Putting REs in strings keeps the Python language simpler, but has one disadvantage which is the topic of the next section. +.. _the-backslash-plague: + The Backslash Plague -------------------- @@ -327,6 +329,13 @@ backslashes are not handled in any special way in a string literal prefixed with while ``"\n"`` is a one-character string containing a newline. Regular expressions will often be written in Python code using this raw string notation. +In addition, special escape sequences that are valid in regular expressions, +but not valid as Python string literals, now result in a +:exc:`DeprecationWarning` and will eventually become a :exc:`SyntaxError`, +which means the sequences will be invalid if raw string notation or escaping +the backslashes isn't used. + + +-------------------+------------------+ | Regular String | Raw string | +===================+==================+ @@ -457,12 +466,18 @@ In actual programs, the most common style is to store the Two pattern methods return all of the matches for a pattern. :meth:`~re.pattern.findall` returns a list of matching strings:: - >>> p = re.compile('\d+') + >>> p = re.compile(r'\d+') >>> p.findall('12 drummers drumming, 11 pipers piping, 10 lords a-leaping') ['12', '11', '10'] -:meth:`~re.pattern.findall` has to create the entire list before it can be returned as the -result. The :meth:`~re.pattern.finditer` method returns a sequence of +The ``r`` prefix, making the literal a raw string literal, is needed in this +example because escape sequences in a normal "cooked" string literal that are +not recognized by Python, as opposed to regular expressions, now result in a +:exc:`DeprecationWarning` and will eventually become a :exc:`SyntaxError`. See +:ref:`the-backslash-plague`. + +:meth:`~re.Pattern.findall` has to create the entire list before it can be returned as the +result. The :meth:`~re.Pattern.finditer` method returns a sequence of :ref:`match object ` instances as an :term:`iterator`:: >>> iterator = p.finditer('12 drummers drumming, 11 ... 10 ...') @@ -1096,11 +1111,11 @@ following calls:: The module-level function :func:`re.split` adds the RE to be used as the first argument, but is otherwise the same. :: - >>> re.split('[\W]+', 'Words, words, words.') + >>> re.split(r'[\W]+', 'Words, words, words.') ['Words', 'words', 'words', ''] - >>> re.split('([\W]+)', 'Words, words, words.') + >>> re.split(r'([\W]+)', 'Words, words, words.') ['Words', ', ', 'words', ', ', 'words', '.', ''] - >>> re.split('[\W]+', 'Words, words, words.', 1) + >>> re.split(r'[\W]+', 'Words, words, words.', 1) ['Words', 'words, words.'] diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst index 9649b9c6..b54e1507 100644 --- a/Doc/howto/unicode.rst +++ b/Doc/howto/unicode.rst @@ -463,7 +463,7 @@ The string in this example has the number 57 written in both Thai and Arabic numerals:: import re - p = re.compile('\d+') + p = re.compile(r'\d+') s = "Over \u0e55\u0e57 57 flavours" m = p.search(s) diff --git a/Doc/includes/email-read-alternative.py b/Doc/includes/email-read-alternative.py index 3f5ab24c..5ea84e62 100644 --- a/Doc/includes/email-read-alternative.py +++ b/Doc/includes/email-read-alternative.py @@ -21,7 +21,7 @@ print('To:', msg['to']) print('From:', msg['from']) print('Subject:', msg['subject']) -# If we want to print a priview of the message content, we can extract whatever +# If we want to print a preview of the message content, we can extract whatever # the least formatted payload is and print the first three lines. Of course, # if the message has no plain text part printing the first three lines of html # is probably useless, but this is just a conceptual example. diff --git a/Doc/library/abc.rst b/Doc/library/abc.rst index 9522dd62..70710761 100644 --- a/Doc/library/abc.rst +++ b/Doc/library/abc.rst @@ -160,7 +160,7 @@ a helper class :class:`ABC` to alternatively define ABCs through inheritance: -The :mod:`abc` module also provides the following decorators: +The :mod:`abc` module also provides the following decorator: .. decorator:: abstractmethod @@ -236,8 +236,15 @@ The :mod:`abc` module also provides the following decorators: multiple-inheritance. +The :mod:`abc` module also supports the following legacy decorators: + .. decorator:: abstractclassmethod + .. versionadded:: 3.2 + .. deprecated:: 3.3 + It is now possible to use :class:`classmethod` with + :func:`abstractmethod`, making this decorator redundant. + A subclass of the built-in :func:`classmethod`, indicating an abstract classmethod. Otherwise it is similar to :func:`abstractmethod`. @@ -251,14 +258,14 @@ The :mod:`abc` module also provides the following decorators: def my_abstract_classmethod(cls, ...): ... + +.. decorator:: abstractstaticmethod + .. versionadded:: 3.2 .. deprecated:: 3.3 - It is now possible to use :class:`classmethod` with + It is now possible to use :class:`staticmethod` with :func:`abstractmethod`, making this decorator redundant. - -.. decorator:: abstractstaticmethod - A subclass of the built-in :func:`staticmethod`, indicating an abstract staticmethod. Otherwise it is similar to :func:`abstractmethod`. @@ -272,23 +279,17 @@ The :mod:`abc` module also provides the following decorators: def my_abstract_staticmethod(...): ... - .. versionadded:: 3.2 - .. deprecated:: 3.3 - It is now possible to use :class:`staticmethod` with - :func:`abstractmethod`, making this decorator redundant. - .. decorator:: abstractproperty + .. deprecated:: 3.3 + It is now possible to use :class:`property`, :meth:`property.getter`, + :meth:`property.setter` and :meth:`property.deleter` with + :func:`abstractmethod`, making this decorator redundant. + A subclass of the built-in :func:`property`, indicating an abstract property. - Using this function requires that the class's metaclass is :class:`ABCMeta` - or is derived from it. A class that has a metaclass derived from - :class:`ABCMeta` cannot be instantiated unless all of its abstract methods - and properties are overridden. The abstract properties can be called using - any of the normal 'super' call mechanisms. - This special case is deprecated, as the :func:`property` decorator is now correctly identified as abstract when applied to an abstract method:: @@ -322,12 +323,6 @@ The :mod:`abc` module also provides the following decorators: ... - .. deprecated:: 3.3 - It is now possible to use :class:`property`, :meth:`property.getter`, - :meth:`property.setter` and :meth:`property.deleter` with - :func:`abstractmethod`, making this decorator redundant. - - The :mod:`abc` module also provides the following functions: .. function:: get_cache_token() diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index 8d4ae2cc..6376f5fe 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -113,6 +113,11 @@ and classes for traversing abstract syntax trees: Parse the source into an AST node. Equivalent to ``compile(source, filename, mode, ast.PyCF_ONLY_AST)``. + .. warning:: + It is possible to crash the Python interpreter with a + sufficiently large/complex string due to stack depth limitations + in Python's AST compiler. + .. function:: literal_eval(node_or_string) @@ -126,6 +131,11 @@ and classes for traversing abstract syntax trees: capable of evaluating arbitrarily complex expressions, for example involving operators or indexing. + .. warning:: + It is possible to crash the Python interpreter with a + sufficiently large/complex string due to stack depth limitations + in Python's AST compiler. + .. versionchanged:: 3.2 Now allows bytes and set literals. diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index cc8fffb0..5b801aaf 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -630,7 +630,7 @@ Task functions This function is a :ref:`coroutine `. -.. function:: shield(arg, \*, loop=None) +.. coroutinefunction:: shield(arg, \*, loop=None) Wait for a future, shielding it from cancellation. diff --git a/Doc/library/base64.rst b/Doc/library/base64.rst index ceecf17c..ad9f5f58 100644 --- a/Doc/library/base64.rst +++ b/Doc/library/base64.rst @@ -218,14 +218,6 @@ The modern interface provides: .. versionadded:: 3.4 -.. note:: - Both Base85 and Ascii85 have an expansion factor of 5 to 4 (5 Base85 or - Ascii85 characters can encode 4 binary bytes), while the better-known - Base64 has an expansion factor of 6 to 4. They are therefore more - efficient when space expensive. They differ by details such as the - character map used for encoding. - - The legacy interface: .. function:: decode(input, output) diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst index 6e249ecf..74b24e10 100644 --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -977,10 +977,14 @@ e.g. ``'utf-8'`` is a valid alias for the ``'utf_8'`` codec. Some common encodings can bypass the codecs lookup machinery to improve performance. These optimization opportunities are only - recognized by CPython for a limited set of aliases: utf-8, utf8, - latin-1, latin1, iso-8859-1, mbcs (Windows only), ascii, utf-16, - and utf-32. Using alternative spellings for these encodings may - result in slower execution. + recognized by CPython for a limited set of (case insensitive) + aliases: utf-8, utf8, latin-1, latin1, iso-8859-1, iso8859-1, mbcs + (Windows only), ascii, us-ascii, utf-16, utf16, utf-32, utf32, and + the same using underscores instead of dashes. Using alternative + aliases for these encodings may result in slower execution. + + .. versionchanged:: 3.6 + Optimization opportunity recognized for us-ascii. Many of the character sets support the same languages. They vary in individual characters (e.g. whether the EURO SIGN is supported or not), and in the diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index d6d2056d..82ba0573 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -509,11 +509,14 @@ or subtracting from an empty counter. .. versionadded:: 3.2 - .. method:: rotate(n) + .. method:: rotate(n=1) - Rotate the deque *n* steps to the right. If *n* is negative, rotate to - the left. Rotating one step to the right is equivalent to: - ``d.appendleft(d.pop())``. + Rotate the deque *n* steps to the right. If *n* is negative, rotate + to the left. + + When the deque is not empty, rotating one step to the right is equivalent + to ``d.appendleft(d.pop())``, and rotating one step to the left is + equivalent to ``d.append(d.popleft())``. Deque objects also provide one read-only attribute: diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst index d85576b8..9794e735 100644 --- a/Doc/library/concurrent.futures.rst +++ b/Doc/library/concurrent.futures.rst @@ -40,21 +40,29 @@ Executor Objects .. method:: map(func, *iterables, timeout=None, chunksize=1) - Equivalent to :func:`map(func, *iterables) ` except *func* is executed - asynchronously and several calls to *func* may be made concurrently. The - returned iterator raises a :exc:`concurrent.futures.TimeoutError` if - :meth:`~iterator.__next__` is called and the result isn't available + Similar to :func:`map(func, *iterables) ` except: + + * the *iterables* are collected immediately rather than lazily; + + * *func* is executed asynchronously and several calls to + *func* may be made concurrently. + + The returned iterator raises a :exc:`concurrent.futures.TimeoutError` + if :meth:`~iterator.__next__` is called and the result isn't available after *timeout* seconds from the original call to :meth:`Executor.map`. *timeout* can be an int or a float. If *timeout* is not specified or - ``None``, there is no limit to the wait time. If a call raises an - exception, then that exception will be raised when its value is - retrieved from the iterator. When using :class:`ProcessPoolExecutor`, this - method chops *iterables* into a number of chunks which it submits to the - pool as separate tasks. The (approximate) size of these chunks can be - specified by setting *chunksize* to a positive integer. For very long - iterables, using a large value for *chunksize* can significantly improve - performance compared to the default size of 1. With :class:`ThreadPoolExecutor`, - *chunksize* has no effect. + ``None``, there is no limit to the wait time. + + If a *func* call raises an exception, then that exception will be + raised when its value is retrieved from the iterator. + + When using :class:`ProcessPoolExecutor`, this method chops *iterables* + into a number of chunks which it submits to the pool as separate + tasks. The (approximate) size of these chunks can be specified by + setting *chunksize* to a positive integer. For very long iterables, + using a large value for *chunksize* can significantly improve + performance compared to the default size of 1. With + :class:`ThreadPoolExecutor`, *chunksize* has no effect. .. versionchanged:: 3.5 Added the *chunksize* argument. diff --git a/Doc/library/constants.rst b/Doc/library/constants.rst index 469a3eed..78f16196 100644 --- a/Doc/library/constants.rst +++ b/Doc/library/constants.rst @@ -86,10 +86,14 @@ should not be used in programs. specified exit code. .. data:: copyright - license credits - Objects that when printed, print a message like "Type license() to see the - full license text", and when called, display the corresponding text in a + Objects that when printed or called, print the text of copyright or + credits, respectively. + +.. data:: license + + Object that when printed, prints the message "Type license() to see the + full license text", and when called, displays the full license text in a pager-like fashion (one screen at a time). diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index c68b15a9..3f615dbd 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -318,12 +318,16 @@ The Python compiler currently generates the following bytecode instructions. Duplicates the reference on top of the stack. + .. versionadded:: 3.2 + .. opcode:: DUP_TOP_TWO Duplicates the two references on top of the stack, leaving them in the same order. + .. versionadded:: 3.2 + **Unary operations** @@ -534,29 +538,39 @@ the original TOS1. the CO_ITERABLE_COROUTINE flag, or resolves ``o.__await__``. + .. versionadded:: 3.5 + .. opcode:: GET_AITER Implements ``TOS = get_awaitable(TOS.__aiter__())``. See ``GET_AWAITABLE`` for details about ``get_awaitable`` + .. versionadded:: 3.5 + .. opcode:: GET_ANEXT Implements ``PUSH(get_awaitable(TOS.__anext__()))``. See ``GET_AWAITABLE`` for details about ``get_awaitable`` + .. versionadded:: 3.5 + .. opcode:: BEFORE_ASYNC_WITH Resolves ``__aenter__`` and ``__aexit__`` from the object on top of the stack. Pushes ``__aexit__`` and result of ``__aenter__()`` to the stack. + .. versionadded:: 3.5 + .. opcode:: SETUP_ASYNC_WITH Creates a new frame object. + .. versionadded:: 3.5 + **Miscellaneous opcodes** @@ -594,6 +608,8 @@ the original TOS1. Calls ``dict.setitem(TOS1[-i], TOS, TOS1)``. Used to implement dict comprehensions. + .. versionadded:: 3.1 + For all of the :opcode:`SET_ADD`, :opcode:`LIST_APPEND` and :opcode:`MAP_ADD` instructions, while the added value or key/value pair is popped off, the container object remains on the stack so that it is available for further @@ -616,6 +632,7 @@ iterations of the loop. .. versionadded:: 3.3 + .. opcode:: SETUP_ANNOTATIONS Checks whether ``__annotations__`` is defined in ``locals()``, if not it is @@ -625,6 +642,7 @@ iterations of the loop. .. versionadded:: 3.6 + .. opcode:: IMPORT_STAR Loads all symbols not starting with ``'_'`` directly from the module TOS to @@ -670,6 +688,8 @@ iterations of the loop. store it in (a) variable(s) (:opcode:`STORE_FAST`, :opcode:`STORE_NAME`, or :opcode:`UNPACK_SEQUENCE`). + .. versionadded:: 3.2 + .. opcode:: WITH_CLEANUP_START @@ -900,23 +920,31 @@ All of the following opcodes use their arguments. If TOS is true, sets the bytecode counter to *target*. TOS is popped. + .. versionadded:: 3.1 + .. opcode:: POP_JUMP_IF_FALSE (target) If TOS is false, sets the bytecode counter to *target*. TOS is popped. + .. versionadded:: 3.1 + .. opcode:: JUMP_IF_TRUE_OR_POP (target) If TOS is true, sets the bytecode counter to *target* and leaves TOS on the stack. Otherwise (TOS is false), TOS is popped. + .. versionadded:: 3.1 + .. opcode:: JUMP_IF_FALSE_OR_POP (target) If TOS is false, sets the bytecode counter to *target* and leaves TOS on the stack. Otherwise (TOS is true), TOS is popped. + .. versionadded:: 3.1 + .. opcode:: JUMP_ABSOLUTE (target) @@ -996,6 +1024,8 @@ All of the following opcodes use their arguments. consulting the cell. This is used for loading free variables in class bodies. + .. versionadded:: 3.4 + .. opcode:: STORE_DEREF (i) @@ -1008,6 +1038,8 @@ All of the following opcodes use their arguments. Empties the cell contained in slot *i* of the cell and free variable storage. Used by the :keyword:`del` statement. + .. versionadded:: 3.2 + .. opcode:: RAISE_VARARGS (argc) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 6548adf7..5c1b226e 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -654,7 +654,7 @@ value and let :class:`Flag` select an appropriate value. Like :class:`IntFlag`, if a combination of :class:`Flag` members results in no flags being set, the boolean evaluation is :data:`False`:: - >>> from enum import Flag + >>> from enum import Flag, auto >>> class Color(Flag): ... RED = auto() ... BLUE = auto() diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 7a7a84d1..9cb6b0e1 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -260,6 +260,12 @@ are always available. They are listed here in alphabetical order. character. This is to facilitate detection of incomplete and complete statements in the :mod:`code` module. + .. warning:: + + It is possible to crash the Python interpreter with a + sufficiently large/complex string when compiling to an AST + object due to stack depth limitations in Python's AST compiler. + .. versionchanged:: 3.2 Allowed use of Windows and Mac newlines. Also input in ``'exec'`` mode does not have to end in a newline anymore. Added the *optimize* parameter. @@ -711,8 +717,11 @@ are always available. They are listed here in alphabetical order. Return an integer object constructed from a number or string *x*, or return ``0`` if no arguments are given. If *x* is a number, return - :meth:`x.__int__() `. For floating point numbers, this - truncates towards zero. + :meth:`x.__int__() `. If *x* defines + :meth:`x.__trunc__() ` but not + :meth:`x.__int__() `, then return + if :meth:`x.__trunc__() `. For floating point numbers, + this truncates towards zero. If *x* is not a number or if *base* is given, then *x* must be a string, :class:`bytes`, or :class:`bytearray` instance representing an :ref:`integer @@ -1409,7 +1418,7 @@ are always available. They are listed here in alphabetical order. a regular function and do something with its result. This is needed in some cases where you need a reference to a function from a class body and you want to avoid the automatic transformation to instance - method. For these cases, use this idiom: + method. For these cases, use this idiom:: class C: builtin_open = staticmethod(open) diff --git a/Doc/library/getpass.rst b/Doc/library/getpass.rst index 5eb9f04a..82b11919 100644 --- a/Doc/library/getpass.rst +++ b/Doc/library/getpass.rst @@ -42,8 +42,10 @@ The :mod:`getpass` module provides two functions: Return the "login name" of the user. This function checks the environment variables :envvar:`LOGNAME`, - :envvar:`USER`, :envvar:`LNAME` and :envvar:`USERNAME`, in order, and returns - the value of the first one which is set to a non-empty string. If none are set, - the login name from the password database is returned on systems which support - the :mod:`pwd` module, otherwise, an exception is raised. + :envvar:`USER`, :envvar:`LNAME` and :envvar:`USERNAME`, in order, and + returns the value of the first one which is set to a non-empty string. If + none are set, the login name from the password database is returned on + systems which support the :mod:`pwd` module, otherwise, an exception is + raised. + In general, this function should be preferred over :func:`os.getlogin()`. diff --git a/Doc/library/glob.rst b/Doc/library/glob.rst index a8a5a500..25bd4b7f 100644 --- a/Doc/library/glob.rst +++ b/Doc/library/glob.rst @@ -48,7 +48,7 @@ For example, ``'[?]'`` matches the character ``'?'``. Support for recursive globs using "``**``". -.. function:: iglob(pathname, recursive=False) +.. function:: iglob(pathname, *, recursive=False) Return an :term:`iterator` which yields the same values as :func:`glob` without actually storing them all simultaneously. diff --git a/Doc/library/io.rst b/Doc/library/io.rst index 4da6e095..af38fd72 100644 --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -205,8 +205,8 @@ ABC Inherits Stub Methods Mixin M ``writable``, and ``writelines`` :class:`RawIOBase` :class:`IOBase` ``readinto`` and Inherited :class:`IOBase` methods, ``read``, ``write`` and ``readall`` -:class:`BufferedIOBase` :class:`IOBase` ``detach``, ``read``, Inherited :class:`IOBase` methods, ``readinto`` - ``read1``, and ``write`` +:class:`BufferedIOBase` :class:`IOBase` ``detach``, ``read``, Inherited :class:`IOBase` methods, ``readinto``, + ``read1``, and ``write`` and ``readinto1`` :class:`TextIOBase` :class:`IOBase` ``detach``, ``read``, Inherited :class:`IOBase` methods, ``encoding``, ``readline``, and ``errors``, and ``newlines`` ``write`` @@ -385,14 +385,17 @@ I/O Base Classes .. method:: read(size=-1) Read up to *size* bytes from the object and return them. As a convenience, - if *size* is unspecified or -1, :meth:`readall` is called. Otherwise, - only one system call is ever made. Fewer than *size* bytes may be - returned if the operating system call returns fewer than *size* bytes. + if *size* is unspecified or -1, all bytes until EOF are returned. + Otherwise, only one system call is ever made. Fewer than *size* bytes may + be returned if the operating system call returns fewer than *size* bytes. If 0 bytes are returned, and *size* was not 0, this indicates end of file. If the object is in non-blocking mode and no bytes are available, ``None`` is returned. + The default implementation defers to :meth:`readall` and + :meth:`readinto`. + .. method:: readall() Read and return all the bytes from the stream until EOF, using multiple diff --git a/Doc/library/ipaddress.rst b/Doc/library/ipaddress.rst index 50fb778d..b3c691e4 100644 --- a/Doc/library/ipaddress.rst +++ b/Doc/library/ipaddress.rst @@ -234,7 +234,7 @@ write code that handles both IP versions correctly. groups consisting entirely of zeroes included. - For the following attributes, see the corresponding documention of the + For the following attributes, see the corresponding documentation of the :class:`IPv4Address` class: .. attribute:: packed @@ -440,7 +440,11 @@ so to avoid duplication they are only documented for :class:`IPv4Network`. .. attribute:: hostmask - The host mask, as a string. + The host mask, as an :class:`IPv4Address` object. + + .. attribute:: netmask + + The net mask, as an :class:`IPv4Address` object. .. attribute:: with_prefixlen .. attribute:: compressed @@ -561,13 +565,12 @@ so to avoid duplication they are only documented for :class:`IPv4Network`. 1. A string consisting of an IP address and an optional mask, separated by a slash (``/``). The IP address is the network address, and the mask - can be either a single number, which means it's a *prefix*, or a string - representation of an IPv6 address. If it's the latter, the mask is - interpreted as a *net mask*. If no mask is provided, it's considered to - be ``/128``. + is a single number, which represents a *prefix*. If no mask is provided, + it's considered to be ``/128``. - For example, the following *address* specifications are equivalent: - ``2001:db00::0/24`` and ``2001:db00::0/ffff:ff00::``. + Note that currently expanded netmasks are not supported. That means + ``2001:db00::0/24`` is a valid argument while ``2001:db00::0/ffff:ff00::`` + not. 2. An integer that fits into 128 bits. This is equivalent to a single-address network, with the network address being *address* and @@ -604,6 +607,7 @@ so to avoid duplication they are only documented for :class:`IPv4Network`. .. attribute:: network_address .. attribute:: broadcast_address .. attribute:: hostmask + .. attribute:: netmask .. attribute:: with_prefixlen .. attribute:: compressed .. attribute:: exploded diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 594af39f..700a13a0 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -32,7 +32,7 @@ operator can be mapped across two vectors to form an efficient dot-product: ``sum(map(operator.mul, vector1, vector2))``. -**Infinite Iterators:** +**Infinite iterators:** ================== ================= ================================================= ========================================= Iterator Arguments Results Example @@ -61,7 +61,7 @@ Iterator Arguments Results :func:`zip_longest` p, q, ... (p[0], q[0]), (p[1], q[1]), ... ``zip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-`` ============================ ============================ ================================================= ============================================================= -**Combinatoric generators:** +**Combinatoric iterators:** ============================================== ==================== ============================================================= Iterator Arguments Results @@ -859,6 +859,29 @@ which incur interpreter overhead. indices = sorted(random.randrange(n) for i in range(r)) return tuple(pool[i] for i in indices) + def nth_combination(iterable, r, index): + 'Equivalent to list(combinations(iterable, r))[index]' + pool = tuple(iterable) + n = len(pool) + if r < 0 or r > n: + raise ValueError + c = 1 + k = min(r, n-r) + for i in range(1, k+1): + c = c * (n - k + i) // i + if index < 0: + index += c + if index < 0 or index >= c: + raise IndexError + result = [] + while r: + c, n, r = c*r//n, n-1, r-1 + while index >= c: + index -= c + c, n = c*(n-r)//n, n-1 + result.append(pool[-1-n]) + return tuple(result) + Note, many of the above recipes can be optimized by replacing global lookups with local variables defined as default values. For example, the *dotproduct* recipe can be written as:: diff --git a/Doc/library/locale.rst b/Doc/library/locale.rst index b04442bc..9a0c5705 100644 --- a/Doc/library/locale.rst +++ b/Doc/library/locale.rst @@ -147,6 +147,16 @@ The :mod:`locale` module defines the following exception and functions: | ``CHAR_MAX`` | Nothing is specified in this locale. | +--------------+-----------------------------------------+ + The function sets temporarily the ``LC_CTYPE`` locale to the ``LC_NUMERIC`` + locale to decode ``decimal_point`` and ``thousands_sep`` byte strings if + they are non-ASCII or longer than 1 byte, and the ``LC_NUMERIC`` locale is + different than the ``LC_CTYPE`` locale. This temporary change affects other + threads. + + .. versionchanged:: 3.6.5 + The function now sets temporarily the ``LC_CTYPE`` locale to the + ``LC_NUMERIC`` locale in some cases. + .. function:: nl_langinfo(option) diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst index 6098878c..1ed129c0 100644 --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -69,260 +69,266 @@ is the module's name in the Python package namespace. .. class:: Logger -.. attribute:: Logger.propagate + .. attribute:: Logger.propagate - If this evaluates to true, events logged to this logger will be passed to the - handlers of higher level (ancestor) loggers, in addition to any handlers - attached to this logger. Messages are passed directly to the ancestor - loggers' handlers - neither the level nor filters of the ancestor loggers in - question are considered. + If this attribute evaluates to true, events logged to this logger will be + passed to the handlers of higher level (ancestor) loggers, in addition to + any handlers attached to this logger. Messages are passed directly to the + ancestor loggers' handlers - neither the level nor filters of the ancestor + loggers in question are considered. - If this evaluates to false, logging messages are not passed to the handlers - of ancestor loggers. + If this evaluates to false, logging messages are not passed to the handlers + of ancestor loggers. - The constructor sets this attribute to ``True``. + The constructor sets this attribute to ``True``. - .. note:: If you attach a handler to a logger *and* one or more of its - ancestors, it may emit the same record multiple times. In general, you - should not need to attach a handler to more than one logger - if you just - attach it to the appropriate logger which is highest in the logger - hierarchy, then it will see all events logged by all descendant loggers, - provided that their propagate setting is left set to ``True``. A common - scenario is to attach handlers only to the root logger, and to let - propagation take care of the rest. + .. note:: If you attach a handler to a logger *and* one or more of its + ancestors, it may emit the same record multiple times. In general, you + should not need to attach a handler to more than one logger - if you just + attach it to the appropriate logger which is highest in the logger + hierarchy, then it will see all events logged by all descendant loggers, + provided that their propagate setting is left set to ``True``. A common + scenario is to attach handlers only to the root logger, and to let + propagation take care of the rest. -.. method:: Logger.setLevel(lvl) + .. method:: Logger.setLevel(level) - Sets the threshold for this logger to *lvl*. Logging messages which are less - severe than *lvl* will be ignored. When a logger is created, the level is set to - :const:`NOTSET` (which causes all messages to be processed when the logger is - the root logger, or delegation to the parent when the logger is a non-root - logger). Note that the root logger is created with level :const:`WARNING`. + Sets the threshold for this logger to *level*. Logging messages which are less + severe than *level* will be ignored; logging messages which have severity *level* + or higher will be emitted by whichever handler or handlers service this logger, + unless a handler's level has been set to a higher severity level than *level*. - The term 'delegation to the parent' means that if a logger has a level of - NOTSET, its chain of ancestor loggers is traversed until either an ancestor with - a level other than NOTSET is found, or the root is reached. + When a logger is created, the level is set to :const:`NOTSET` (which causes + all messages to be processed when the logger is the root logger, or delegation + to the parent when the logger is a non-root logger). Note that the root logger + is created with level :const:`WARNING`. - If an ancestor is found with a level other than NOTSET, then that ancestor's - level is treated as the effective level of the logger where the ancestor search - began, and is used to determine how a logging event is handled. + The term 'delegation to the parent' means that if a logger has a level of + NOTSET, its chain of ancestor loggers is traversed until either an ancestor with + a level other than NOTSET is found, or the root is reached. - If the root is reached, and it has a level of NOTSET, then all messages will be - processed. Otherwise, the root's level will be used as the effective level. + If an ancestor is found with a level other than NOTSET, then that ancestor's + level is treated as the effective level of the logger where the ancestor search + began, and is used to determine how a logging event is handled. - See :ref:`levels` for a list of levels. + If the root is reached, and it has a level of NOTSET, then all messages will be + processed. Otherwise, the root's level will be used as the effective level. - .. versionchanged:: 3.2 - The *lvl* parameter now accepts a string representation of the - level such as 'INFO' as an alternative to the integer constants - such as :const:`INFO`. Note, however, that levels are internally stored - as integers, and methods such as e.g. :meth:`getEffectiveLevel` and - :meth:`isEnabledFor` will return/expect to be passed integers. + See :ref:`levels` for a list of levels. + .. versionchanged:: 3.2 + The *level* parameter now accepts a string representation of the + level such as 'INFO' as an alternative to the integer constants + such as :const:`INFO`. Note, however, that levels are internally stored + as integers, and methods such as e.g. :meth:`getEffectiveLevel` and + :meth:`isEnabledFor` will return/expect to be passed integers. -.. method:: Logger.isEnabledFor(lvl) - Indicates if a message of severity *lvl* would be processed by this logger. - This method checks first the module-level level set by - ``logging.disable(lvl)`` and then the logger's effective level as determined - by :meth:`getEffectiveLevel`. + .. method:: Logger.isEnabledFor(lvl) + Indicates if a message of severity *lvl* would be processed by this logger. + This method checks first the module-level level set by + ``logging.disable(lvl)`` and then the logger's effective level as determined + by :meth:`getEffectiveLevel`. -.. method:: Logger.getEffectiveLevel() - Indicates the effective level for this logger. If a value other than - :const:`NOTSET` has been set using :meth:`setLevel`, it is returned. Otherwise, - the hierarchy is traversed towards the root until a value other than - :const:`NOTSET` is found, and that value is returned. The value returned is - an integer, typically one of :const:`logging.DEBUG`, :const:`logging.INFO` - etc. + .. method:: Logger.getEffectiveLevel() + Indicates the effective level for this logger. If a value other than + :const:`NOTSET` has been set using :meth:`setLevel`, it is returned. Otherwise, + the hierarchy is traversed towards the root until a value other than + :const:`NOTSET` is found, and that value is returned. The value returned is + an integer, typically one of :const:`logging.DEBUG`, :const:`logging.INFO` + etc. -.. method:: Logger.getChild(suffix) - Returns a logger which is a descendant to this logger, as determined by the suffix. - Thus, ``logging.getLogger('abc').getChild('def.ghi')`` would return the same - logger as would be returned by ``logging.getLogger('abc.def.ghi')``. This is a - convenience method, useful when the parent logger is named using e.g. ``__name__`` - rather than a literal string. + .. method:: Logger.getChild(suffix) - .. versionadded:: 3.2 + Returns a logger which is a descendant to this logger, as determined by the suffix. + Thus, ``logging.getLogger('abc').getChild('def.ghi')`` would return the same + logger as would be returned by ``logging.getLogger('abc.def.ghi')``. This is a + convenience method, useful when the parent logger is named using e.g. ``__name__`` + rather than a literal string. + .. versionadded:: 3.2 -.. method:: Logger.debug(msg, *args, **kwargs) - Logs a message with level :const:`DEBUG` on this logger. The *msg* is the - message format string, and the *args* are the arguments which are merged into - *msg* using the string formatting operator. (Note that this means that you can - use keywords in the format string, together with a single dictionary argument.) + .. method:: Logger.debug(msg, *args, **kwargs) - There are three keyword arguments in *kwargs* which are inspected: - *exc_info*, *stack_info*, and *extra*. + Logs a message with level :const:`DEBUG` on this logger. The *msg* is the + message format string, and the *args* are the arguments which are merged into + *msg* using the string formatting operator. (Note that this means that you can + use keywords in the format string, together with a single dictionary argument.) - If *exc_info* does not evaluate as false, it causes exception information to be - added to the logging message. If an exception tuple (in the format returned by - :func:`sys.exc_info`) or an exception instance is provided, it is used; - otherwise, :func:`sys.exc_info` is called to get the exception information. + There are three keyword arguments in *kwargs* which are inspected: + *exc_info*, *stack_info*, and *extra*. - The second optional keyword argument is *stack_info*, which defaults to - ``False``. If true, stack information is added to the logging - message, including the actual logging call. Note that this is not the same - stack information as that displayed through specifying *exc_info*: The - former is stack frames from the bottom of the stack up to the logging call - in the current thread, whereas the latter is information about stack frames - which have been unwound, following an exception, while searching for - exception handlers. + If *exc_info* does not evaluate as false, it causes exception information to be + added to the logging message. If an exception tuple (in the format returned by + :func:`sys.exc_info`) or an exception instance is provided, it is used; + otherwise, :func:`sys.exc_info` is called to get the exception information. - You can specify *stack_info* independently of *exc_info*, e.g. to just show - how you got to a certain point in your code, even when no exceptions were - raised. The stack frames are printed following a header line which says:: + The second optional keyword argument is *stack_info*, which defaults to + ``False``. If true, stack information is added to the logging + message, including the actual logging call. Note that this is not the same + stack information as that displayed through specifying *exc_info*: The + former is stack frames from the bottom of the stack up to the logging call + in the current thread, whereas the latter is information about stack frames + which have been unwound, following an exception, while searching for + exception handlers. - Stack (most recent call last): + You can specify *stack_info* independently of *exc_info*, e.g. to just show + how you got to a certain point in your code, even when no exceptions were + raised. The stack frames are printed following a header line which says:: - This mimics the ``Traceback (most recent call last):`` which is used when - displaying exception frames. + Stack (most recent call last): - The third keyword argument is *extra* which can be used to pass a - dictionary which is used to populate the __dict__ of the LogRecord created for - the logging event with user-defined attributes. These custom attributes can then - be used as you like. For example, they could be incorporated into logged - messages. For example:: + This mimics the ``Traceback (most recent call last):`` which is used when + displaying exception frames. - FORMAT = '%(asctime)-15s %(clientip)s %(user)-8s %(message)s' - logging.basicConfig(format=FORMAT) - d = {'clientip': '192.168.0.1', 'user': 'fbloggs'} - logger = logging.getLogger('tcpserver') - logger.warning('Protocol problem: %s', 'connection reset', extra=d) + The third keyword argument is *extra* which can be used to pass a + dictionary which is used to populate the __dict__ of the LogRecord created for + the logging event with user-defined attributes. These custom attributes can then + be used as you like. For example, they could be incorporated into logged + messages. For example:: - would print something like :: + FORMAT = '%(asctime)-15s %(clientip)s %(user)-8s %(message)s' + logging.basicConfig(format=FORMAT) + d = {'clientip': '192.168.0.1', 'user': 'fbloggs'} + logger = logging.getLogger('tcpserver') + logger.warning('Protocol problem: %s', 'connection reset', extra=d) - 2006-02-08 22:20:02,165 192.168.0.1 fbloggs Protocol problem: connection reset + would print something like :: - The keys in the dictionary passed in *extra* should not clash with the keys used - by the logging system. (See the :class:`Formatter` documentation for more - information on which keys are used by the logging system.) + 2006-02-08 22:20:02,165 192.168.0.1 fbloggs Protocol problem: connection reset - If you choose to use these attributes in logged messages, you need to exercise - some care. In the above example, for instance, the :class:`Formatter` has been - set up with a format string which expects 'clientip' and 'user' in the attribute - dictionary of the LogRecord. If these are missing, the message will not be - logged because a string formatting exception will occur. So in this case, you - always need to pass the *extra* dictionary with these keys. + The keys in the dictionary passed in *extra* should not clash with the keys used + by the logging system. (See the :class:`Formatter` documentation for more + information on which keys are used by the logging system.) - While this might be annoying, this feature is intended for use in specialized - circumstances, such as multi-threaded servers where the same code executes in - many contexts, and interesting conditions which arise are dependent on this - context (such as remote client IP address and authenticated user name, in the - above example). In such circumstances, it is likely that specialized - :class:`Formatter`\ s would be used with particular :class:`Handler`\ s. + If you choose to use these attributes in logged messages, you need to exercise + some care. In the above example, for instance, the :class:`Formatter` has been + set up with a format string which expects 'clientip' and 'user' in the attribute + dictionary of the LogRecord. If these are missing, the message will not be + logged because a string formatting exception will occur. So in this case, you + always need to pass the *extra* dictionary with these keys. - .. versionadded:: 3.2 - The *stack_info* parameter was added. + While this might be annoying, this feature is intended for use in specialized + circumstances, such as multi-threaded servers where the same code executes in + many contexts, and interesting conditions which arise are dependent on this + context (such as remote client IP address and authenticated user name, in the + above example). In such circumstances, it is likely that specialized + :class:`Formatter`\ s would be used with particular :class:`Handler`\ s. - .. versionchanged:: 3.5 - The *exc_info* parameter can now accept exception instances. + .. versionadded:: 3.2 + The *stack_info* parameter was added. + .. versionchanged:: 3.5 + The *exc_info* parameter can now accept exception instances. -.. method:: Logger.info(msg, *args, **kwargs) - Logs a message with level :const:`INFO` on this logger. The arguments are - interpreted as for :meth:`debug`. + .. method:: Logger.info(msg, *args, **kwargs) + Logs a message with level :const:`INFO` on this logger. The arguments are + interpreted as for :meth:`debug`. -.. method:: Logger.warning(msg, *args, **kwargs) - Logs a message with level :const:`WARNING` on this logger. The arguments are - interpreted as for :meth:`debug`. + .. method:: Logger.warning(msg, *args, **kwargs) - .. note:: There is an obsolete method ``warn`` which is functionally - identical to ``warning``. As ``warn`` is deprecated, please do not use - it - use ``warning`` instead. + Logs a message with level :const:`WARNING` on this logger. The arguments are + interpreted as for :meth:`debug`. -.. method:: Logger.error(msg, *args, **kwargs) + .. note:: There is an obsolete method ``warn`` which is functionally + identical to ``warning``. As ``warn`` is deprecated, please do not use + it - use ``warning`` instead. - Logs a message with level :const:`ERROR` on this logger. The arguments are - interpreted as for :meth:`debug`. + .. method:: Logger.error(msg, *args, **kwargs) + Logs a message with level :const:`ERROR` on this logger. The arguments are + interpreted as for :meth:`debug`. -.. method:: Logger.critical(msg, *args, **kwargs) - Logs a message with level :const:`CRITICAL` on this logger. The arguments are - interpreted as for :meth:`debug`. + .. method:: Logger.critical(msg, *args, **kwargs) + Logs a message with level :const:`CRITICAL` on this logger. The arguments are + interpreted as for :meth:`debug`. -.. method:: Logger.log(lvl, msg, *args, **kwargs) - Logs a message with integer level *lvl* on this logger. The other arguments are - interpreted as for :meth:`debug`. + .. method:: Logger.log(lvl, msg, *args, **kwargs) + Logs a message with integer level *lvl* on this logger. The other arguments are + interpreted as for :meth:`debug`. -.. method:: Logger.exception(msg, *args, **kwargs) - Logs a message with level :const:`ERROR` on this logger. The arguments are - interpreted as for :meth:`debug`. Exception info is added to the logging - message. This method should only be called from an exception handler. + .. method:: Logger.exception(msg, *args, **kwargs) + Logs a message with level :const:`ERROR` on this logger. The arguments are + interpreted as for :meth:`debug`. Exception info is added to the logging + message. This method should only be called from an exception handler. -.. method:: Logger.addFilter(filt) - Adds the specified filter *filt* to this logger. + .. method:: Logger.addFilter(filter) + Adds the specified filter *filter* to this logger. -.. method:: Logger.removeFilter(filt) - Removes the specified filter *filt* from this logger. + .. method:: Logger.removeFilter(filter) + Removes the specified filter *filter* from this logger. -.. method:: Logger.filter(record) - Applies this logger's filters to the record and returns a true value if the - record is to be processed. The filters are consulted in turn, until one of - them returns a false value. If none of them return a false value, the record - will be processed (passed to handlers). If one returns a false value, no - further processing of the record occurs. + .. method:: Logger.filter(record) + Applies this logger's filters to the record and returns a true value if the + record is to be processed. The filters are consulted in turn, until one of + them returns a false value. If none of them return a false value, the record + will be processed (passed to handlers). If one returns a false value, no + further processing of the record occurs. -.. method:: Logger.addHandler(hdlr) - Adds the specified handler *hdlr* to this logger. + .. method:: Logger.addHandler(hdlr) + Adds the specified handler *hdlr* to this logger. -.. method:: Logger.removeHandler(hdlr) - Removes the specified handler *hdlr* from this logger. + .. method:: Logger.removeHandler(hdlr) + Removes the specified handler *hdlr* from this logger. -.. method:: Logger.findCaller(stack_info=False) - Finds the caller's source filename and line number. Returns the filename, line - number, function name and stack information as a 4-element tuple. The stack - information is returned as ``None`` unless *stack_info* is ``True``. + .. method:: Logger.findCaller(stack_info=False) + Finds the caller's source filename and line number. Returns the filename, line + number, function name and stack information as a 4-element tuple. The stack + information is returned as ``None`` unless *stack_info* is ``True``. -.. method:: Logger.handle(record) - Handles a record by passing it to all handlers associated with this logger and - its ancestors (until a false value of *propagate* is found). This method is used - for unpickled records received from a socket, as well as those created locally. - Logger-level filtering is applied using :meth:`~Logger.filter`. + .. method:: Logger.handle(record) + Handles a record by passing it to all handlers associated with this logger and + its ancestors (until a false value of *propagate* is found). This method is used + for unpickled records received from a socket, as well as those created locally. + Logger-level filtering is applied using :meth:`~Logger.filter`. -.. method:: Logger.makeRecord(name, lvl, fn, lno, msg, args, exc_info, func=None, extra=None, sinfo=None) - This is a factory method which can be overridden in subclasses to create - specialized :class:`LogRecord` instances. + .. method:: Logger.makeRecord(name, lvl, fn, lno, msg, args, exc_info, func=None, extra=None, sinfo=None) -.. method:: Logger.hasHandlers() + This is a factory method which can be overridden in subclasses to create + specialized :class:`LogRecord` instances. - Checks to see if this logger has any handlers configured. This is done by - looking for handlers in this logger and its parents in the logger hierarchy. - Returns ``True`` if a handler was found, else ``False``. The method stops searching - up the hierarchy whenever a logger with the 'propagate' attribute set to - false is found - that will be the last logger which is checked for the - existence of handlers. + .. method:: Logger.hasHandlers() - .. versionadded:: 3.2 + Checks to see if this logger has any handlers configured. This is done by + looking for handlers in this logger and its parents in the logger hierarchy. + Returns ``True`` if a handler was found, else ``False``. The method stops searching + up the hierarchy whenever a logger with the 'propagate' attribute set to + false is found - that will be the last logger which is checked for the + existence of handlers. + .. versionadded:: 3.2 + + .. versionchanged:: 3.7 + Loggers can now be picked and unpickled. .. _levels: @@ -362,113 +368,115 @@ is never instantiated directly; this class acts as a base for more useful subclasses. However, the :meth:`__init__` method in subclasses needs to call :meth:`Handler.__init__`. +.. class:: Handler -.. method:: Handler.__init__(level=NOTSET) + .. method:: Handler.__init__(level=NOTSET) - Initializes the :class:`Handler` instance by setting its level, setting the list - of filters to the empty list and creating a lock (using :meth:`createLock`) for - serializing access to an I/O mechanism. + Initializes the :class:`Handler` instance by setting its level, setting the list + of filters to the empty list and creating a lock (using :meth:`createLock`) for + serializing access to an I/O mechanism. -.. method:: Handler.createLock() + .. method:: Handler.createLock() - Initializes a thread lock which can be used to serialize access to underlying - I/O functionality which may not be threadsafe. + Initializes a thread lock which can be used to serialize access to underlying + I/O functionality which may not be threadsafe. -.. method:: Handler.acquire() + .. method:: Handler.acquire() - Acquires the thread lock created with :meth:`createLock`. + Acquires the thread lock created with :meth:`createLock`. -.. method:: Handler.release() + .. method:: Handler.release() - Releases the thread lock acquired with :meth:`acquire`. + Releases the thread lock acquired with :meth:`acquire`. -.. method:: Handler.setLevel(lvl) + .. method:: Handler.setLevel(level) - Sets the threshold for this handler to *lvl*. Logging messages which are less - severe than *lvl* will be ignored. When a handler is created, the level is set - to :const:`NOTSET` (which causes all messages to be processed). + Sets the threshold for this handler to *level*. Logging messages which are + less severe than *level* will be ignored. When a handler is created, the + level is set to :const:`NOTSET` (which causes all messages to be + processed). - See :ref:`levels` for a list of levels. + See :ref:`levels` for a list of levels. - .. versionchanged:: 3.2 - The *lvl* parameter now accepts a string representation of the - level such as 'INFO' as an alternative to the integer constants - such as :const:`INFO`. + .. versionchanged:: 3.2 + The *level* parameter now accepts a string representation of the + level such as 'INFO' as an alternative to the integer constants + such as :const:`INFO`. -.. method:: Handler.setFormatter(form) + .. method:: Handler.setFormatter(fmt) - Sets the :class:`Formatter` for this handler to *form*. + Sets the :class:`Formatter` for this handler to *fmt*. -.. method:: Handler.addFilter(filt) + .. method:: Handler.addFilter(filter) - Adds the specified filter *filt* to this handler. + Adds the specified filter *filter* to this handler. -.. method:: Handler.removeFilter(filt) + .. method:: Handler.removeFilter(filter) - Removes the specified filter *filt* from this handler. + Removes the specified filter *filter* from this handler. -.. method:: Handler.filter(record) + .. method:: Handler.filter(record) - Applies this handler's filters to the record and returns a true value if the - record is to be processed. The filters are consulted in turn, until one of - them returns a false value. If none of them return a false value, the record - will be emitted. If one returns a false value, the handler will not emit the - record. + Applies this handler's filters to the record and returns a true value if the + record is to be processed. The filters are consulted in turn, until one of + them returns a false value. If none of them return a false value, the record + will be emitted. If one returns a false value, the handler will not emit the + record. -.. method:: Handler.flush() + .. method:: Handler.flush() - Ensure all logging output has been flushed. This version does nothing and is - intended to be implemented by subclasses. + Ensure all logging output has been flushed. This version does nothing and is + intended to be implemented by subclasses. -.. method:: Handler.close() + .. method:: Handler.close() - Tidy up any resources used by the handler. This version does no output but - removes the handler from an internal list of handlers which is closed when - :func:`shutdown` is called. Subclasses should ensure that this gets called - from overridden :meth:`close` methods. + Tidy up any resources used by the handler. This version does no output but + removes the handler from an internal list of handlers which is closed when + :func:`shutdown` is called. Subclasses should ensure that this gets called + from overridden :meth:`close` methods. -.. method:: Handler.handle(record) + .. method:: Handler.handle(record) - Conditionally emits the specified logging record, depending on filters which may - have been added to the handler. Wraps the actual emission of the record with - acquisition/release of the I/O thread lock. + Conditionally emits the specified logging record, depending on filters which may + have been added to the handler. Wraps the actual emission of the record with + acquisition/release of the I/O thread lock. -.. method:: Handler.handleError(record) + .. method:: Handler.handleError(record) - This method should be called from handlers when an exception is encountered - during an :meth:`emit` call. If the module-level attribute - ``raiseExceptions`` is ``False``, exceptions get silently ignored. This is - what is mostly wanted for a logging system - most users will not care about - errors in the logging system, they are more interested in application - errors. You could, however, replace this with a custom handler if you wish. - The specified record is the one which was being processed when the exception - occurred. (The default value of ``raiseExceptions`` is ``True``, as that is - more useful during development). + This method should be called from handlers when an exception is encountered + during an :meth:`emit` call. If the module-level attribute + ``raiseExceptions`` is ``False``, exceptions get silently ignored. This is + what is mostly wanted for a logging system - most users will not care about + errors in the logging system, they are more interested in application + errors. You could, however, replace this with a custom handler if you wish. + The specified record is the one which was being processed when the exception + occurred. (The default value of ``raiseExceptions`` is ``True``, as that is + more useful during development). -.. method:: Handler.format(record) + .. method:: Handler.format(record) - Do formatting for a record - if a formatter is set, use it. Otherwise, use the - default formatter for the module. + Do formatting for a record - if a formatter is set, use it. Otherwise, use the + default formatter for the module. -.. method:: Handler.emit(record) + .. method:: Handler.emit(record) - Do whatever it takes to actually log the specified logging record. This version - is intended to be implemented by subclasses and so raises a - :exc:`NotImplementedError`. + Do whatever it takes to actually log the specified logging record. This version + is intended to be implemented by subclasses and so raises a + :exc:`NotImplementedError`. For a list of handlers included as standard, see :mod:`logging.handlers`. @@ -772,15 +780,15 @@ the options available to you. | lineno | ``%(lineno)d`` | Source line number where the logging call was | | | | issued (if available). | +----------------+-------------------------+-----------------------------------------------+ +| message | ``%(message)s`` | The logged message, computed as ``msg % | +| | | args``. This is set when | +| | | :meth:`Formatter.format` is invoked. | ++----------------+-------------------------+-----------------------------------------------+ | module | ``%(module)s`` | Module (name portion of ``filename``). | +----------------+-------------------------+-----------------------------------------------+ | msecs | ``%(msecs)d`` | Millisecond portion of the time when the | | | | :class:`LogRecord` was created. | +----------------+-------------------------+-----------------------------------------------+ -| message | ``%(message)s`` | The logged message, computed as ``msg % | -| | | args``. This is set when | -| | | :meth:`Formatter.format` is invoked. | -+----------------+-------------------------+-----------------------------------------------+ | msg | You shouldn't need to | The format string passed in the original | | | format this yourself. | logging call. Merged with ``args`` to | | | | produce ``message``, or an arbitrary object | @@ -1023,7 +1031,7 @@ functions. handlers being added multiple times to the root logger, which can in turn lead to multiple messages for the same event. -.. function:: disable(lvl) +.. function:: disable(lvl=CRITICAL) Provides an overriding level *lvl* for all loggers which takes precedence over the logger's own level. When the need arises to temporarily throttle logging @@ -1036,6 +1044,14 @@ functions. overriding level, so that logging output again depends on the effective levels of individual loggers. + Note that if you have defined any custom logging level higher than + ``CRITICAL`` (this is not recommended), you won't be able to rely on the + default value for the *lvl* parameter, but will have to explicitly supply a + suitable value. + + .. versionchanged:: 3.7 + The *lvl* parameter was defaulted to level ``CRITICAL``. See Issue + #28524 for more information about this change. .. function:: addLevelName(lvl, levelName) @@ -1248,4 +1264,3 @@ with the :mod:`warnings` module. package available from this site is suitable for use with Python 1.5.2, 2.1.x and 2.2.x, which do not include the :mod:`logging` package in the standard library. - diff --git a/Doc/library/mailbox.rst b/Doc/library/mailbox.rst index 81244c2e..48d76a83 100644 --- a/Doc/library/mailbox.rst +++ b/Doc/library/mailbox.rst @@ -491,7 +491,7 @@ Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF. `Configuring Netscape Mail on Unix: Why The Content-Length Format is Bad `_ An argument for using the original mbox format rather than a variation. - `"mbox" is a family of several mutually incompatible mailbox formats `_ + `"mbox" is a family of several mutually incompatible mailbox formats `_ A history of mbox variations. diff --git a/Doc/library/math.rst b/Doc/library/math.rst index da2b8cc5..0f36a3fa 100644 --- a/Doc/library/math.rst +++ b/Doc/library/math.rst @@ -179,7 +179,7 @@ Number-theoretic and representation functions Return the :class:`~numbers.Real` value *x* truncated to an :class:`~numbers.Integral` (usually an integer). Delegates to - ``x.__trunc__()``. + :meth:`x.__trunc__() `. Note that :func:`frexp` and :func:`modf` have a different call/return pattern diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst index 2ef187db..337c7c29 100644 --- a/Doc/library/optparse.rst +++ b/Doc/library/optparse.rst @@ -567,7 +567,7 @@ An option group is obtained using the class :class:`OptionGroup`: where - * parser is the :class:`OptionParser` instance the group will be insterted in + * parser is the :class:`OptionParser` instance the group will be inserted in to * title is the group title * description, optional, is a long description of the group diff --git a/Doc/library/os.path.rst b/Doc/library/os.path.rst index 406054e5..06493f95 100644 --- a/Doc/library/os.path.rst +++ b/Doc/library/os.path.rst @@ -246,8 +246,9 @@ the :mod:`glob` module.) .. function:: isfile(path) - Return ``True`` if *path* is an existing regular file. This follows symbolic - links, so both :func:`islink` and :func:`isfile` can be true for the same path. + Return ``True`` if *path* is an :func:`existing ` regular file. + This follows symbolic links, so both :func:`islink` and :func:`isfile` can + be true for the same path. .. versionchanged:: 3.6 Accepts a :term:`path-like object`. @@ -255,8 +256,9 @@ the :mod:`glob` module.) .. function:: isdir(path) - Return ``True`` if *path* is an existing directory. This follows symbolic - links, so both :func:`islink` and :func:`isdir` can be true for the same path. + Return ``True`` if *path* is an :func:`existing ` directory. This + follows symbolic links, so both :func:`islink` and :func:`isdir` can be true + for the same path. .. versionchanged:: 3.6 Accepts a :term:`path-like object`. @@ -264,8 +266,9 @@ the :mod:`glob` module.) .. function:: islink(path) - Return ``True`` if *path* refers to a directory entry that is a symbolic link. - Always ``False`` if symbolic links are not supported by the Python runtime. + Return ``True`` if *path* refers to an :func:`existing ` directory + entry that is a symbolic link. Always ``False`` if symbolic links are not + supported by the Python runtime. .. versionchanged:: 3.6 Accepts a :term:`path-like object`. diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 974ab2d4..b2722168 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -325,10 +325,11 @@ process and user. .. function:: getlogin() Return the name of the user logged in on the controlling terminal of the - process. For most purposes, it is more useful to use the environment - variables :envvar:`LOGNAME` or :envvar:`USERNAME` to find out who the user - is, or ``pwd.getpwuid(os.getuid())[0]`` to get the login name of the current - real user id. + process. For most purposes, it is more useful to use + :func:`getpass.getuser` since the latter checks the environment variables + :envvar:`LOGNAME` or :envvar:`USERNAME` to find out who the user is, and + falls back to ``pwd.getpwuid(os.getuid())[0]`` to get the login name of the + current real user id. Availability: Unix, Windows. diff --git a/Doc/library/ossaudiodev.rst b/Doc/library/ossaudiodev.rst index ec40c0b9..522bb7e0 100644 --- a/Doc/library/ossaudiodev.rst +++ b/Doc/library/ossaudiodev.rst @@ -14,7 +14,7 @@ the standard audio interface for Linux and recent versions of FreeBSD. .. Things will get more complicated for future Linux versions, since ALSA is in the standard kernel as of 2.5.x. Presumably if you use ALSA, you'll have to make sure its OSS compatibility layer - is active to use ossaudiodev, but you're gonna need it for the vast + is active to use ossaudiodev, but you're going to need it for the vast majority of Linux audio apps anyway. Sounds like things are also complicated for other BSDs. In response @@ -447,4 +447,3 @@ The remaining methods are specific to audio mixing: microphone input:: mixer.setrecsrc (1 << ossaudiodev.SOUND_MIXER_MIC) - diff --git a/Doc/library/pickle.rst b/Doc/library/pickle.rst index 6e8430fa..d0c4cf93 100644 --- a/Doc/library/pickle.rst +++ b/Doc/library/pickle.rst @@ -370,7 +370,7 @@ The :mod:`pickle` module exports two classes, :class:`Pickler` and Python 2 names to the new names used in Python 3. The *encoding* and *errors* tell pickle how to decode 8-bit string instances pickled by Python 2; these default to 'ASCII' and 'strict', respectively. The *encoding* can - be 'bytes' to read these ß8-bit string instances as bytes objects. + be 'bytes' to read these 8-bit string instances as bytes objects. .. method:: load() diff --git a/Doc/library/quopri.rst b/Doc/library/quopri.rst index a3f94a0a..86717c00 100644 --- a/Doc/library/quopri.rst +++ b/Doc/library/quopri.rst @@ -34,9 +34,10 @@ sending a graphics file. Encode the contents of the *input* file and write the resulting quoted-printable data to the *output* file. *input* and *output* must be - :term:`binary file objects `. *quotetabs*, a flag which controls - whether to encode embedded spaces and tabs must be provideda and when true it - encodes such embedded whitespace, and when false it leaves them unencoded. + :term:`binary file objects `. *quotetabs*, a + non-optional flag which controls whether to encode embedded spaces + and tabs; when true it encodes such embedded whitespace, and when + false it leaves them unencoded. Note that spaces and tabs appearing at the end of lines are always encoded, as per :rfc:`1521`. *header* is a flag which controls if spaces are encoded as underscores as per :rfc:`1522`. diff --git a/Doc/library/re.rst b/Doc/library/re.rst index fae8945f..db92c480 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -315,7 +315,7 @@ The special characters are: This example looks for a word following a hyphen: - >>> m = re.search('(?<=-)\w+', 'spam-egg') + >>> m = re.search(r'(?<=-)\w+', 'spam-egg') >>> m.group(0) 'egg' @@ -719,14 +719,21 @@ form. Splitting on a pattern that could match an empty string now raises a warning. Patterns that can only match empty strings are now rejected. + .. function:: findall(pattern, string, flags=0) Return all non-overlapping matches of *pattern* in *string*, as a list of strings. The *string* is scanned left-to-right, and matches are returned in the order found. If one or more groups are present in the pattern, return a list of groups; this will be a list of tuples if the pattern has more than - one group. Empty matches are included in the result unless they touch the - beginning of another match. + one group. Empty matches are included in the result. + + .. note:: + + Due to the limitation of the current implementation the character + following an empty match is not included in a next match, so + ``findall(r'^|\w+', 'two words')`` returns ``['', 'wo', 'words']`` + (note missed "t"). This is changed in Python 3.7. .. function:: finditer(pattern, string, flags=0) @@ -734,8 +741,7 @@ form. Return an :term:`iterator` yielding :ref:`match objects ` over all non-overlapping matches for the RE *pattern* in *string*. The *string* is scanned left-to-right, and matches are returned in the order found. Empty - matches are included in the result unless they touch the beginning of another - match. + matches are included in the result. See also the note about :func:`findall`. .. function:: sub(pattern, repl, string, count=0, flags=0) diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst index 41e5bafa..3d8cb030 100644 --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -318,7 +318,8 @@ Directory and files operations Return disk usage statistics about the given path as a :term:`named tuple` with the attributes *total*, *used* and *free*, which are the amount of - total, used and free space, in bytes. + total, used and free space, in bytes. On Windows, *path* must be a + directory; on Unix, it can be a file or directory. .. versionadded:: 3.3 diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst index 678e32ce..512c38e7 100644 --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -303,6 +303,10 @@ Constants ``SO_DOMAIN``, ``SO_PROTOCOL``, ``SO_PEERSEC``, ``SO_PASSSEC``, ``TCP_USER_TIMEOUT``, ``TCP_CONGESTION`` were added. + .. versionchanged:: 3.6.5 + On Windows, ``TCP_FASTOPEN``, ``TCP_KEEPCNT`` appear if run-time Windows + supports. + .. data:: AF_CAN PF_CAN SOL_CAN_* diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 495cd590..677e9453 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -1351,7 +1351,7 @@ to speed up repeated connections from the same clients. The *capath* string, if present, is the path to a directory containing several CA certificates in PEM format, following an `OpenSSL specific layout - `_. + `_. The *cadata* object, if present, is either an ASCII string of one or more PEM-encoded certificates or a :term:`bytes-like object` of DER-encoded @@ -1539,7 +1539,7 @@ to speed up repeated connections from the same clients. .. method:: SSLContext.load_dh_params(dhfile) - Load the key generation parameters for Diffie-Helman (DH) key exchange. + Load the key generation parameters for Diffie-Hellman (DH) key exchange. Using DH key exchange improves forward secrecy at the expense of computational resources (both on the server and on the client). The *dhfile* parameter should be the path to a file containing DH @@ -1614,7 +1614,7 @@ to speed up repeated connections from the same clients. Get statistics about the SSL sessions created or managed by this context. A dictionary is returned which maps the names of each `piece of information - `_ to their + `_ to their numeric values. For example, here is the total number of hits and misses in the session cache since the context was created:: @@ -1634,7 +1634,7 @@ to speed up repeated connections from the same clients. import socket, ssl - context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) + context = ssl.SSLContext() context.verify_mode = ssl.CERT_REQUIRED context.check_hostname = True context.load_default_certs() @@ -1861,7 +1861,7 @@ If you prefer to tune security settings yourself, you might create a context from scratch (but beware that you might not get the settings right):: - >>> context = ssl.SSLContext(ssl.PROTOCOL_TLS) + >>> context = ssl.SSLContext() >>> context.verify_mode = ssl.CERT_REQUIRED >>> context.check_hostname = True >>> context.load_verify_locations("/etc/ssl/certs/ca-bundle.crt") @@ -2320,13 +2320,30 @@ successful call of :func:`~ssl.RAND_add`, :func:`~ssl.RAND_bytes` or :func:`~ssl.RAND_pseudo_bytes` is sufficient. +.. ssl-libressl: + +LibreSSL support +---------------- + +LibreSSL is a fork of OpenSSL 1.0.1. The ssl module has limited support for +LibreSSL. Some features are not available when the ssl module is compiled +with LibreSSL. + +* LibreSSL >= 2.6.1 no longer supports NPN. The methods + :meth:`SSLContext.set_npn_protocols` and + :meth:`SSLSocket.selected_npn_protocol` are not available. +* :meth:`SSLContext.set_default_verify_paths` ignores the env vars + :envvar:`SSL_CERT_FILE` and :envvar:`SSL_CERT_PATH` although + :func:`get_default_verify_paths` still reports them. + + .. seealso:: Class :class:`socket.socket` Documentation of underlying :mod:`socket` class `SSL/TLS Strong Encryption: An Introduction `_ - Intro from the Apache webserver documentation + Intro from the Apache HTTP Server documentation `RFC 1422: Privacy Enhancement for Internet Electronic Mail: Part II: Certificate-Based Key Management `_ Steve Kent diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 75e97b9d..d8a1647e 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -973,9 +973,9 @@ Notes: (8) ``index`` raises :exc:`ValueError` when *x* is not found in *s*. - When supported, the additional arguments to the index method allow - efficient searching of subsections of the sequence. Passing the extra - arguments is roughly equivalent to using ``s[i:j].index(x)``, only + Not all implementations support passing the additional arguments *i* and *j*. + These arguments allow efficient searching of subsections of the sequence. Passing + the extra arguments is roughly equivalent to using ``s[i:j].index(x)``, only without copying any data and with the returned index being relative to the start of the sequence rather than the start of the slice. @@ -1599,6 +1599,20 @@ expression support in the :mod:`re` module). See :ref:`formatstrings` for a description of the various formatting options that can be specified in format strings. + .. note:: + When formatting a number (:class:`int`, :class:`float`, :class:`float` + and subclasses) with the ``n`` type (ex: ``'{:n}'.format(1234)``), the + function sets temporarily the ``LC_CTYPE`` locale to the ``LC_NUMERIC`` + locale to decode ``decimal_point`` and ``thousands_sep`` fields of + :c:func:`localeconv` if they are non-ASCII or longer than 1 byte, and the + ``LC_NUMERIC`` locale is different than the ``LC_CTYPE`` locale. This + temporary change affects other threads. + + .. versionchanged:: 3.6.5 + When formatting a number with the ``n`` type, the function sets + temporarily the ``LC_CTYPE`` locale to the ``LC_NUMERIC`` locale in some + cases. + .. method:: str.format_map(mapping) diff --git a/Doc/library/string.rst b/Doc/library/string.rst index 7a9fcc38..706d5e1f 100644 --- a/Doc/library/string.rst +++ b/Doc/library/string.rst @@ -202,9 +202,9 @@ The grammar for a replacement field is as follows: .. productionlist:: sf replacement_field: "{" [`field_name`] ["!" `conversion`] [":" `format_spec`] "}" field_name: arg_name ("." `attribute_name` | "[" `element_index` "]")* - arg_name: [`identifier` | `integer`] + arg_name: [`identifier` | `digit`+] attribute_name: `identifier` - element_index: `integer` | `index_string` + element_index: `digit`+ | `index_string` index_string: + conversion: "r" | "s" | "a" format_spec: @@ -304,9 +304,9 @@ The general form of a *standard format specifier* is: fill: align: "<" | ">" | "=" | "^" sign: "+" | "-" | " " - width: `integer` + width: `digit`+ grouping_option: "_" | "," - precision: `integer` + precision: `digit`+ type: "b" | "c" | "d" | "e" | "E" | "f" | "F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%" If a valid *align* value is specified, it can be preceded by a *fill* diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index ea7e664f..6a5f2760 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -956,6 +956,9 @@ calls these functions. .. versionchanged:: 3.4 Support for the *input* keyword argument was added. + .. versionchanged:: 3.6 + *encoding* and *errors* were added. See :func:`run` for details. + .. _subprocess-replacements: Replacing Older Functions with the :mod:`subprocess` Module diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index dd51ffd5..68521df3 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1005,13 +1005,38 @@ always available. Set the system's profile function, which allows you to implement a Python source code profiler in Python. See chapter :ref:`profile` for more information on the Python profiler. The system's profile function is called similarly to the - system's trace function (see :func:`settrace`), but it isn't called for each - executed line of code (only on call and return, but the return event is reported - even when an exception has been set). The function is thread-specific, but - there is no way for the profiler to know about context switches between threads, - so it does not make sense to use this in the presence of multiple threads. Also, + system's trace function (see :func:`settrace`), but it is called with different events, + for example it isn't called for each executed line of code (only on call and return, + but the return event is reported even when an exception has been set). The function is + thread-specific, but there is no way for the profiler to know about context switches between + threads, so it does not make sense to use this in the presence of multiple threads. Also, its return value is not used, so it can simply return ``None``. + Profile functions should have three arguments: *frame*, *event*, and + *arg*. *frame* is the current stack frame. *event* is a string: ``'call'``, + ``'return'``, ``'c_call'``, ``'c_return'``, or ``'c_exception'``. *arg* depends + on the event type. + + The events have the following meaning: + + ``'call'`` + A function is called (or some other code block entered). The + profile function is called; *arg* is ``None``. + + ``'return'`` + A function (or other code block) is about to return. The profile + function is called; *arg* is the value that will be returned, or ``None`` + if the event is caused by an exception being raised. + + ``'c_call'`` + A C function is about to be called. This may be an extension function or + a built-in. *arg* is the C function object. + + ``'c_return'`` + A C function has returned. *arg* is the C function object. + + ``'c_exception'`` + A C function has raised an exception. *arg* is the C function object. .. function:: setrecursionlimit(limit) @@ -1058,8 +1083,8 @@ always available. Trace functions should have three arguments: *frame*, *event*, and *arg*. *frame* is the current stack frame. *event* is a string: ``'call'``, - ``'line'``, ``'return'``, ``'exception'``, ``'c_call'``, ``'c_return'``, or - ``'c_exception'``. *arg* depends on the event type. + ``'line'``, ``'return'`` or ``'exception'``. *arg* depends on + the event type. The trace function is invoked (with *event* set to ``'call'``) whenever a new local scope is entered; it should return a reference to a local trace @@ -1094,16 +1119,6 @@ always available. tuple ``(exception, value, traceback)``; the return value specifies the new local trace function. - ``'c_call'`` - A C function is about to be called. This may be an extension function or - a built-in. *arg* is the C function object. - - ``'c_return'`` - A C function has returned. *arg* is the C function object. - - ``'c_exception'`` - A C function has raised an exception. *arg* is the C function object. - Note that as an exception is propagated down the chain of callers, an ``'exception'`` event is generated at each level. diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 1a3f8f9c..04d6cd87 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -677,3 +677,10 @@ The :mod:`test.support` module defines the following classes: Class used to record warnings for unit tests. See documentation of :func:`check_warnings` above for more details. + + +.. class:: FakePath(path) + + Simple :term:`path-like object`. It implements the :meth:`__fspath__` + method which just returns the *path* argument. If *path* is an exception, + it will be raised in :meth:`!__fspath__`. diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst index c3757546..26e6a35b 100644 --- a/Doc/library/threading.rst +++ b/Doc/library/threading.rst @@ -684,8 +684,8 @@ Semaphores also support the :ref:`context management protocol `. .. class:: Semaphore(value=1) - This class implements semaphore objects. A semaphore manages a counter - representing the number of :meth:`release` calls minus the number of + This class implements semaphore objects. A semaphore manages an atomic + counter representing the number of :meth:`release` calls minus the number of :meth:`acquire` calls, plus an initial value. The :meth:`acquire` method blocks if necessary until it can return without making the counter negative. If not given, *value* defaults to 1. @@ -701,19 +701,19 @@ Semaphores also support the :ref:`context management protocol `. Acquire a semaphore. - When invoked without arguments: if the internal counter is larger than - zero on entry, decrement it by one and return immediately. If it is zero - on entry, block, waiting until some other thread has called - :meth:`~Semaphore.release` to make it larger than zero. This is done - with proper interlocking so that if multiple :meth:`acquire` calls are - blocked, :meth:`~Semaphore.release` will wake exactly one of them up. - The implementation may pick one at random, so the order in which - blocked threads are awakened should not be relied on. Returns - true (or blocks indefinitely). + When invoked without arguments: + + * If the internal counter is larger than zero on entry, decrement it by + one and return true immediately. + * If the internal counter is zero on entry, block until awoken by a call to + :meth:`~Semaphore.release`. Once awoken (and the counter is greater + than 0), decrement the counter by 1 and return true. Exactly one + thread will be awoken by each call to :meth:`~Semaphore.release`. The + order in which threads are awoken should not be relied on. When invoked with *blocking* set to false, do not block. If a call - without an argument would block, return false immediately; otherwise, - do the same thing as when called without arguments, and return true. + without an argument would block, return false immediately; otherwise, do + the same thing as when called without arguments, and return true. When invoked with a *timeout* other than ``None``, it will block for at most *timeout* seconds. If acquire does not complete successfully in diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index e52f1400..a4c4c5c6 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -2282,7 +2282,7 @@ handling functionality within test frameworks. When called without arguments this function removes the control-c handler if it has been installed. This function can also be used as a test decorator - to temporarily remove the handler whilst the test is being executed:: + to temporarily remove the handler while the test is being executed:: @unittest.removeHandler def test_signal_handling(self): diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst index 7d814ad4..80505ea6 100644 --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -297,7 +297,7 @@ If the XML input has `namespaces with prefixes in the form ``prefix:sometag`` get expanded to ``{uri}sometag`` where the *prefix* is replaced by the full *URI*. Also, if there is a `default namespace -`__, +`__, that full URI gets prepended to all of the non-prefixed tags. Here is an XML example that incorporates two namespaces, one with the diff --git a/Doc/library/zlib.rst b/Doc/library/zlib.rst index 3d742ab3..8a531c92 100644 --- a/Doc/library/zlib.rst +++ b/Doc/library/zlib.rst @@ -51,9 +51,9 @@ The available exception and functions in this module are: Compresses the bytes in *data*, returning a bytes object containing compressed data. *level* is an integer from ``0`` to ``9`` or ``-1`` controlling the level of compression; - ``1`` is fastest and produces the least compression, ``9`` is slowest and - produces the most. ``0`` is no compression. The default value is ``-1`` - (Z_DEFAULT_COMPRESSION). Z_DEFAULT_COMPRESSION represents a default + ``1`` (Z_BEST_SPEED) is fastest and produces the least compression, ``9`` (Z_BEST_COMPRESSION) + is slowest and produces the most. ``0`` (Z_NO_COMPRESSION) is no compression. + The default value is ``-1`` (Z_DEFAULT_COMPRESSION). Z_DEFAULT_COMPRESSION represents a default compromise between speed and compression (currently equivalent to level 6). Raises the :exc:`error` exception if any error occurs. @@ -61,23 +61,25 @@ The available exception and functions in this module are: *level* can now be used as a keyword parameter. -.. function:: compressobj(level=-1, method=DEFLATED, wbits=15, memLevel=8, strategy=Z_DEFAULT_STRATEGY[, zdict]) +.. function:: compressobj(level=-1, method=DEFLATED, wbits=MAX_WBITS, memLevel=DEF_MEM_LEVEL, strategy=Z_DEFAULT_STRATEGY[, zdict]) Returns a compression object, to be used for compressing data streams that won't fit into memory at once. *level* is the compression level -- an integer from ``0`` to ``9`` or ``-1``. - A value of ``1`` is fastest and produces the least compression, while a value of - ``9`` is slowest and produces the most. ``0`` is no compression. The default - value is ``-1`` (Z_DEFAULT_COMPRESSION). Z_DEFAULT_COMPRESSION represents a default - compromise between speed and compression (currently equivalent to level 6). + A value of ``1`` (Z_BEST_SPEED) is fastest and produces the least compression, + while a value of ``9`` (Z_BEST_COMPRESSION) is slowest and produces the most. + ``0`` (Z_NO_COMPRESSION) is no compression. The default value is ``-1`` (Z_DEFAULT_COMPRESSION). + Z_DEFAULT_COMPRESSION represents a default compromise between speed and compression + (currently equivalent to level 6). *method* is the compression algorithm. Currently, the only supported value is - ``DEFLATED``. + :const:`DEFLATED`. The *wbits* argument controls the size of the history buffer (or the "window size") used when compressing data, and whether a header and - trailer is included in the output. It can take several ranges of values: + trailer is included in the output. It can take several ranges of values, + defaulting to ``15`` (MAX_WBITS): * +9 to +15: The base-two logarithm of the window size, which therefore ranges between 512 and 32768. Larger values produce @@ -97,7 +99,8 @@ The available exception and functions in this module are: Higher values use more memory, but are faster and produce smaller output. *strategy* is used to tune the compression algorithm. Possible values are - ``Z_DEFAULT_STRATEGY``, ``Z_FILTERED``, and ``Z_HUFFMAN_ONLY``. + :const:`Z_DEFAULT_STRATEGY`, :const:`Z_FILTERED`, :const:`Z_HUFFMAN_ONLY`, + :const:`Z_RLE` (zlib 1.2.0.1) and :const:`Z_FIXED` (zlib 1.2.2.2). *zdict* is a predefined compression dictionary. This is a sequence of bytes (such as a :class:`bytes` object) containing subsequences that are expected @@ -175,7 +178,7 @@ The available exception and functions in this module are: .. versionchanged:: 3.6 *wbits* and *bufsize* can be used as keyword arguments. -.. function:: decompressobj(wbits=15[, zdict]) +.. function:: decompressobj(wbits=MAX_WBITS[, zdict]) Returns a decompression object, to be used for decompressing data streams that won't fit into memory at once. @@ -213,13 +216,13 @@ Compression objects support the following methods: All pending input is processed, and a bytes object containing the remaining compressed output is returned. *mode* can be selected from the constants - :const:`Z_SYNC_FLUSH`, :const:`Z_FULL_FLUSH`, or :const:`Z_FINISH`, - defaulting to :const:`Z_FINISH`. :const:`Z_SYNC_FLUSH` and - :const:`Z_FULL_FLUSH` allow compressing further bytestrings of data, while - :const:`Z_FINISH` finishes the compressed stream and prevents compressing any - more data. After calling :meth:`flush` with *mode* set to :const:`Z_FINISH`, - the :meth:`compress` method cannot be called again; the only realistic action is - to delete the object. + :const:`Z_NO_FLUSH`, :const:`Z_PARTIAL_FLUSH`, :const:`Z_SYNC_FLUSH`, + :const:`Z_FULL_FLUSH`, :const:`Z_BLOCK` (zlib 1.2.3.4), or :const:`Z_FINISH`, + defaulting to :const:`Z_FINISH`. Except :const:`Z_FINISH`, all constants + allow compressing further bytestrings of data, while :const:`Z_FINISH` finishes the + compressed stream and prevents compressing any more data. After calling :meth:`flush` + with *mode* set to :const:`Z_FINISH`, the :meth:`compress` method cannot be called again; + the only realistic action is to delete the object. .. method:: Compress.copy() diff --git a/Doc/license.rst b/Doc/license.rst index 49d29cee..aa4b1450 100644 --- a/Doc/license.rst +++ b/Doc/license.rst @@ -87,7 +87,7 @@ PSF LICENSE AGREEMENT FOR PYTHON |release| analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python |release| alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of - copyright, i.e., "Copyright © 2001-2017 Python Software Foundation; All Rights + copyright, i.e., "Copyright © 2001-2018 Python Software Foundation; All Rights Reserved" are retained in Python |release| alone or in any derivative version prepared by Licensee. @@ -545,7 +545,7 @@ The :mod:`xmlrpc.client` module contains the following notice:: test_epoll ---------- -The :mod:`test_epoll` contains the following notice:: +The :mod:`test_epoll` module contains the following notice:: Copyright (c) 2001-2006 Twisted Matrix Laboratories. @@ -571,7 +571,8 @@ The :mod:`test_epoll` contains the following notice:: Select kqueue ------------- -The :mod:`select` and contains the following notice for the kqueue interface:: +The :mod:`select` module contains the following notice for the kqueue +interface:: Copyright (c) 2000 Doug White, 2006 James Knight, 2007 Christian Heimes All rights reserved. @@ -927,7 +928,7 @@ on the cfuhash project:: libmpdec -------- -The :mod:`_decimal` Module is built using an included copy of the libmpdec +The :mod:`_decimal` module is built using an included copy of the libmpdec library unless the build is configured ``--with-system-libmpdec``:: Copyright (c) 2008-2016 Stefan Krah. All rights reserved. diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 230caf84..c08986f6 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1157,61 +1157,69 @@ Basic customization .. index:: single: destructor + single: finalizer statement: del Called when the instance is about to be destroyed. This is also called a - destructor. If a base class has a :meth:`__del__` method, the derived class's - :meth:`__del__` method, if any, must explicitly call it to ensure proper - deletion of the base class part of the instance. Note that it is possible - (though not recommended!) for the :meth:`__del__` method to postpone destruction - of the instance by creating a new reference to it. It may then be called at a - later time when this new reference is deleted. It is not guaranteed that - :meth:`__del__` methods are called for objects that still exist when the - interpreter exits. + finalizer or (improperly) a destructor. If a base class has a + :meth:`__del__` method, the derived class's :meth:`__del__` method, + if any, must explicitly call it to ensure proper deletion of the base + class part of the instance. + + It is possible (though not recommended!) for the :meth:`__del__` method + to postpone destruction of the instance by creating a new reference to + it. This is called object *resurrection*. It is implementation-dependent + whether :meth:`__del__` is called a second time when a resurrected object + is about to be destroyed; the current :term:`CPython` implementation + only calls it once. + + It is not guaranteed that :meth:`__del__` methods are called for objects + that still exist when the interpreter exits. .. note:: ``del x`` doesn't directly call ``x.__del__()`` --- the former decrements the reference count for ``x`` by one, and the latter is only called when - ``x``'s reference count reaches zero. Some common situations that may - prevent the reference count of an object from going to zero include: - circular references between objects (e.g., a doubly-linked list or a tree - data structure with parent and child pointers); a reference to the object - on the stack frame of a function that caught an exception (the traceback - stored in ``sys.exc_info()[2]`` keeps the stack frame alive); or a - reference to the object on the stack frame that raised an unhandled - exception in interactive mode (the traceback stored in - ``sys.last_traceback`` keeps the stack frame alive). The first situation - can only be remedied by explicitly breaking the cycles; the second can be - resolved by freeing the reference to the traceback object when it is no - longer useful, and the third can be resolved by storing ``None`` in - ``sys.last_traceback``. - Circular references which are garbage are detected and cleaned up when - the cyclic garbage collector is enabled (it's on by default). Refer to the - documentation for the :mod:`gc` module for more information about this - topic. + ``x``'s reference count reaches zero. + + .. impl-detail:: + It is possible for a reference cycle to prevent the reference count + of an object from going to zero. In this case, the cycle will be + later detected and deleted by the :term:`cyclic garbage collector + `. A common cause of reference cycles is when + an exception has been caught in a local variable. The frame's + locals then reference the exception, which references its own + traceback, which references the locals of all frames caught in the + traceback. + + .. seealso:: + Documentation for the :mod:`gc` module. .. warning:: Due to the precarious circumstances under which :meth:`__del__` methods are invoked, exceptions that occur during their execution are ignored, and a warning - is printed to ``sys.stderr`` instead. Also, when :meth:`__del__` is invoked in - response to a module being deleted (e.g., when execution of the program is - done), other globals referenced by the :meth:`__del__` method may already have - been deleted or in the process of being torn down (e.g. the import - machinery shutting down). For this reason, :meth:`__del__` methods - should do the absolute - minimum needed to maintain external invariants. Starting with version 1.5, - Python guarantees that globals whose name begins with a single underscore are - deleted from their module before other globals are deleted; if no other - references to such globals exist, this may help in assuring that imported - modules are still available at the time when the :meth:`__del__` method is - called. + is printed to ``sys.stderr`` instead. In particular: - .. index:: - single: repr() (built-in function); __repr__() (object method) + * :meth:`__del__` can be invoked when arbitrary code is being executed, + including from any arbitrary thread. If :meth:`__del__` needs to take + a lock or invoke any other blocking resource, it may deadlock as + the resource may already be taken by the code that gets interrupted + to execute :meth:`__del__`. + + * :meth:`__del__` can be executed during interpreter shutdown. As a + consequence, the global variables it needs to access (including other + modules) may already have been deleted or set to ``None``. Python + guarantees that globals whose name begins with a single underscore + are deleted from their module before other globals are deleted; if + no other references to such globals exist, this may help in assuring + that imported modules are still available at the time when the + :meth:`__del__` method is called. + .. index:: + single: repr() (built-in function); __repr__() (object method) + .. method:: object.__repr__(self) Called by the :func:`repr` built-in function to compute the "official" string @@ -1435,10 +1443,12 @@ access (use of, assignment to, or deletion of ``x.name``) for class instances. .. method:: object.__getattr__(self, name) - Called when an attribute lookup has not found the attribute in the usual places - (i.e. it is not an instance attribute nor is it found in the class tree for - ``self``). ``name`` is the attribute name. This method should return the - (computed) attribute value or raise an :exc:`AttributeError` exception. + Called when the default attribute access fails with an :exc:`AttributeError` + (either :meth:`__getattribute__` raises an :exc:`AttributeError` because + *name* is not an instance attribute or an attribute in the class tree + for ``self``; or :meth:`__get__` of a *name* property raises + :exc:`AttributeError`). This method should either return the (computed) + attribute value or raise an :exc:`AttributeError` exception. Note that if the attribute is found through the normal mechanism, :meth:`__getattr__` is not called. (This is an intentional asymmetry between @@ -1492,6 +1502,39 @@ access (use of, assignment to, or deletion of ``x.name``) for class instances. returned. :func:`dir` converts the returned sequence to a list and sorts it. +Customizing module attribute access +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. index:: + single: __class__ (module attribute) + +For a more fine grained customization of the module behavior (setting +attributes, properties, etc.), one can set the ``__class__`` attribute of +a module object to a subclass of :class:`types.ModuleType`. For example:: + + import sys + from types import ModuleType + + class VerboseModule(ModuleType): + def __repr__(self): + return f'Verbose {self.__name__}' + + def __setattr__(self, attr, value): + print(f'Setting {attr}...') + setattr(self, attr, value) + + sys.modules[__name__].__class__ = VerboseModule + +.. note:: + Setting module ``__class__`` only affects lookups made using the attribute + access syntax -- directly accessing the module globals (whether by code + within the module, or via a reference to the module's globals dictionary) + is unaffected. + +.. versionchanged:: 3.5 + ``__class__`` module attribute is now writable. + + .. _descriptors: Implementing Descriptors @@ -1610,15 +1653,11 @@ instances cannot override the behavior of a property. __slots__ ^^^^^^^^^ -By default, instances of classes have a dictionary for attribute storage. This -wastes space for objects having very few instance variables. The space -consumption can become acute when creating large numbers of instances. - -The default can be overridden by defining *__slots__* in a class definition. -The *__slots__* declaration takes a sequence of instance variables and reserves -just enough space in each instance to hold a value for each variable. Space is -saved because *__dict__* is not created for each instance. +*__slots__* allow us to explicitly declare data members (like +properties) and deny the creation of *__dict__* and *__weakref__* +(unless explicitly declared in *__slots__* or available in a parent.) +The space saved over using *__dict__* can be significant. .. data:: object.__slots__ @@ -1631,9 +1670,8 @@ saved because *__dict__* is not created for each instance. Notes on using *__slots__* """""""""""""""""""""""""" -* When inheriting from a class without *__slots__*, the *__dict__* attribute of - that class will always be accessible, so a *__slots__* definition in the - subclass is meaningless. +* When inheriting from a class without *__slots__*, the *__dict__* and + *__weakref__* attribute of the instances will always be accessible. * Without a *__dict__* variable, instances cannot be assigned new variables not listed in the *__slots__* definition. Attempts to assign to an unlisted @@ -1652,9 +1690,11 @@ Notes on using *__slots__* *__slots__*; otherwise, the class attribute would overwrite the descriptor assignment. -* The action of a *__slots__* declaration is limited to the class where it is - defined. As a result, subclasses will have a *__dict__* unless they also define - *__slots__* (which must only contain names of any *additional* slots). +* The action of a *__slots__* declaration is not limited to the class + where it is defined. *__slots__* declared in parents are available in + child classes. However, child subclasses will get a *__dict__* and + *__weakref__* unless they also define *__slots__* (which should only + contain names of any *additional* slots). * If a class defines a slot also defined in a base class, the instance variable defined by the base class slot is inaccessible (except by retrieving its @@ -1670,6 +1710,10 @@ Notes on using *__slots__* * *__class__* assignment works only if both classes have the same *__slots__*. +* Multiple inheritance with multiple slotted parent classes can be used, + but only one parent is allowed to have attributes created by slots + (the other bases must have empty slot layouts) - violations raise + :exc:`TypeError`. .. _class-customization: @@ -2255,16 +2299,14 @@ left undefined. .. method:: object.__complex__(self) object.__int__(self) object.__float__(self) - object.__round__(self, [,n]) .. index:: builtin: complex builtin: int builtin: float - builtin: round Called to implement the built-in functions :func:`complex`, - :func:`int`, :func:`float` and :func:`round`. Should return a value + :func:`int` and :func:`float`. Should return a value of the appropriate type. @@ -2283,6 +2325,23 @@ left undefined. the same value. +.. method:: object.__round__(self, [,ndigits]) + object.__trunc__(self) + object.__floor__(self) + object.__ceil__(self) + + .. index:: builtin: round + + Called to implement the built-in function :func:`round` and :mod:`math` + functions :func:`~math.trunc`, :func:`~math.floor` and :func:`~math.ceil`. + Unless *ndigits* is passed to :meth:`!__round__` all these methods should + return the value of the object truncated to an :class:`~numbers.Integral` + (typically an :class:`int`). + + If :meth:`__int__` is not defined then the built-in function :func:`int` + falls back to :meth:`__trunc__`. + + .. _context-managers: With Statement Context Managers diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index 4a5abf62..6d093dcc 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -571,7 +571,7 @@ that a single backslash followed by a newline is interpreted as those two characters as part of the literal, *not* as a line continuation. -.. _string-catenation: +.. _string-concatenation: String literal concatenation ---------------------------- @@ -654,9 +654,11 @@ expression or conversion result. An empty string is passed when the format specifier is omitted. The formatted result is then included in the final value of the whole string. -Top-level format specifiers may include nested replacement fields. -These nested fields may include their own conversion fields and -format specifiers, but may not include more deeply-nested replacement fields. +Top-level format specifiers may include nested replacement fields. These nested +fields may include their own conversion fields and :ref:`format specifiers +`, but may not include more deeply-nested replacement fields. The +:ref:`format specifier mini-language ` is the same as that used by +the string .format() method. Formatted string literals may be concatenated, but replacement fields cannot be split across literals. @@ -674,7 +676,7 @@ Some examples of formatted string literals:: >>> f"result: {value:{width}.{precision}}" # nested fields 'result: 12.35' >>> today = datetime(year=2017, month=1, day=27) - >>> f"{today:%b %d, %Y}" # using date format specifier + >>> f"{today:%B %d, %Y}" # using date format specifier 'January 27, 2017' >>> number = 1024 >>> f"{number:#0x}" # using integer format specifier diff --git a/Doc/tools/extensions/pyspecific.py b/Doc/tools/extensions/pyspecific.py index 8f507e46..f533ac48 100644 --- a/Doc/tools/extensions/pyspecific.py +++ b/Doc/tools/extensions/pyspecific.py @@ -196,7 +196,7 @@ class DeprecatedRemoved(Directive): final_argument_whitespace = True option_spec = {} - _label = 'Deprecated since version %s, will be removed in version %s' + _label = 'Deprecated since version {deprecated}, will be removed in version {removed}' def run(self): node = addnodes.versionmodified() @@ -204,11 +204,12 @@ class DeprecatedRemoved(Directive): node['type'] = 'deprecated-removed' version = (self.arguments[0], self.arguments[1]) node['version'] = version - text = self._label % version + label = translators['sphinx'].gettext(self._label) + text = label.format(deprecated=self.arguments[0], removed=self.arguments[1]) if len(self.arguments) == 3: inodes, messages = self.state.inline_text(self.arguments[2], self.lineno+1) - para = nodes.paragraph(self.arguments[2], '', *inodes) + para = nodes.paragraph(self.arguments[2], '', *inodes, translatable=False) node.append(para) else: messages = [] @@ -220,13 +221,14 @@ class DeprecatedRemoved(Directive): content.source = node[0].source content.line = node[0].line content += node[0].children - node[0].replace_self(nodes.paragraph('', '', content)) + node[0].replace_self(nodes.paragraph('', '', content, translatable=False)) node[0].insert(0, nodes.inline('', '%s: ' % text, classes=['versionmodified'])) else: para = nodes.paragraph('', '', nodes.inline('', '%s.' % text, - classes=['versionmodified'])) + classes=['versionmodified']), + translatable=False) node.append(para) env = self.state.document.settings.env env.note_versionchange('deprecated', version[0], node, self.lineno) diff --git a/Doc/tools/static/switchers.js b/Doc/tools/static/switchers.js index c450f5ea..8e0c5ea0 100644 --- a/Doc/tools/static/switchers.js +++ b/Doc/tools/static/switchers.js @@ -10,7 +10,8 @@ '(?:release/\\d.\\d[\\x\\d\\.]*)']; var all_versions = { - '3.7': 'dev (3.7)', + '3.8': 'dev (3.8)', + '3.7': 'pre (3.7)', '3.6': '3.6', '3.5': '3.5', '2.7': '2.7', diff --git a/Doc/tools/templates/dummy.html b/Doc/tools/templates/dummy.html index 6e43be23..8d94137b 100644 --- a/Doc/tools/templates/dummy.html +++ b/Doc/tools/templates/dummy.html @@ -4,3 +4,4 @@ texts in extensions to sphinx.pot file. In extensions/pyspecific.py: {% trans %}CPython implementation detail:{% endtrans %} +{% trans %}Deprecated since version {deprecated}, will be removed in version {removed}{% endtrans %} diff --git a/Doc/tools/templates/indexsidebar.html b/Doc/tools/templates/indexsidebar.html index c73c4064..20e66e0c 100644 --- a/Doc/tools/templates/indexsidebar.html +++ b/Doc/tools/templates/indexsidebar.html @@ -2,7 +2,8 @@

{% trans %}Download these documents{% endtrans %}

{% trans %}Docs for other versions{% endtrans %}