From: DongHun Kwak Date: Tue, 25 Jan 2022 23:27:44 +0000 (+0900) Subject: Imported Upstream version 3.9.4 X-Git-Tag: upstream/3.9.4^0 X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=170d6c9e2e1cfbedcf22e35f6a18873185618c0c;p=platform%2Fupstream%2Fpython3.git Imported Upstream version 3.9.4 --- diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 8df62c56..fd59f680 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1702,6 +1702,93 @@ to the object: 13891296 +When can I rely on identity tests with the *is* operator? +--------------------------------------------------------- + +The ``is`` operator tests for object identity. The test ``a is b`` is +equivalent to ``id(a) == id(b)``. + +The most important property of an identity test is that an object is always +identical to itself, ``a is a`` always returns ``True``. Identity tests are +usually faster than equality tests. And unlike equality tests, identity tests +are guaranteed to return a boolean ``True`` or ``False``. + +However, identity tests can *only* be substituted for equality tests when +object identity is assured. Generally, there are three circumstances where +identity is guaranteed: + +1) Assignments create new names but do not change object identity. After the +assignment ``new = old``, it is guaranteed that ``new is old``. + +2) Putting an object in a container that stores object references does not +change object identity. After the list assignment ``s[0] = x``, it is +guaranteed that ``s[0] is x``. + +3) If an object is a singleton, it means that only one instance of that object +can exist. After the assignments ``a = None`` and ``b = None``, it is +guaranteed that ``a is b`` because ``None`` is a singleton. + +In most other circumstances, identity tests are inadvisable and equality tests +are preferred. In particular, identity tests should not be used to check +constants such as :class:`int` and :class:`str` which aren't guaranteed to be +singletons:: + + >>> a = 1000 + >>> b = 500 + >>> c = b + 500 + >>> a is c + False + + >>> a = 'Python' + >>> b = 'Py' + >>> c = b + 'thon' + >>> a is c + False + +Likewise, new instances of mutable containers are never identical:: + + >>> a = [] + >>> b = [] + >>> a is b + False + +In the standard library code, you will see several common patterns for +correctly using identity tests: + +1) As recommended by :pep:`8`, an identity test is the preferred way to check +for ``None``. This reads like plain English in code and avoids confusion with +other objects that may have boolean values that evaluate to false. + +2) Detecting optional arguments can be tricky when ``None`` is a valid input +value. In those situations, you can create an singleton sentinel object +guaranteed to be distinct from other objects. For example, here is how +to implement a method that behaves like :meth:`dict.pop`:: + + _sentinel = object() + + def pop(self, key, default=_sentinel): + if key in self: + value = self[key] + del self[key] + return value + if default is _sentinel: + raise KeyError(key) + return default + +3) Container implementations sometimes need to augment equality tests with +identity tests. This prevents the code from being confused by objects such as +``float('NaN')`` that are not equal to themselves. + +For example, here is the implementation of +:meth:`collections.abc.Sequence.__contains__`:: + + def __contains__(self, value): + for v in self: + if v is value or v == value: + return True + return False + + Modules ======= diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index c7035bad..93dd3544 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -115,9 +115,9 @@ different, updated answers each time:: 20 >>> g.size # The games directory has three files 3 - >>> open('games/newfile').close() # Add a fourth file to the directory + >>> os.remove('games/chess') # Delete a game >>> g.size # File count is automatically updated - 4 + 2 Besides showing how descriptors can run computations, this example also reveals the purpose of the parameters to :meth:`__get__`. The *self* @@ -281,7 +281,9 @@ The new class now logs access to both *name* and *age*: INFO:root:Updating 'name' to 'Catherine C' INFO:root:Updating 'age' to 20 -The two *Person* instances contain only the private names:: +The two *Person* instances contain only the private names: + +.. doctest:: >>> vars(pete) {'_name': 'Peter P', '_age': 10} @@ -710,6 +712,38 @@ perform attribute lookup by way of a helper function: raise return type(obj).__getattr__(obj, name) # __getattr__ +.. doctest:: + :hide: + + + >>> class ClassWithGetAttr: + ... x = 123 + ... def __getattr__(self, attr): + ... return attr.upper() + ... + >>> cw = ClassWithGetAttr() + >>> cw.y = 456 + >>> getattr_hook(cw, 'x') + 123 + >>> getattr_hook(cw, 'y') + 456 + >>> getattr_hook(cw, 'z') + 'Z' + + >>> class ClassWithoutGetAttr: + ... x = 123 + ... + >>> cwo = ClassWithoutGetAttr() + >>> cwo.y = 456 + >>> getattr_hook(cwo, 'x') + 123 + >>> getattr_hook(cwo, 'y') + 456 + >>> getattr_hook(cwo, 'z') + Traceback (most recent call last): + ... + AttributeError: 'ClassWithoutGetAttr' object has no attribute 'z' + So if :meth:`__getattr__` exists, it is called whenever :meth:`__getattribute__` raises :exc:`AttributeError` (either directly or in one of the descriptor calls). @@ -1129,8 +1163,8 @@ If you have ever wondered where *self* comes from in regular methods or where *cls* comes from in class methods, this is it! -Other kinds of methods ----------------------- +Kinds of methods +---------------- Non-data descriptors provide a simple mechanism for variations on the usual patterns of binding functions into methods. @@ -1183,19 +1217,19 @@ example calls are unexciting: class E: @staticmethod def f(x): - print(x) + return x * 10 .. doctest:: >>> E.f(3) - 3 + 30 >>> E().f(3) - 3 + 30 Using the non-data descriptor protocol, a pure Python version of :func:`staticmethod` would look like this: -.. doctest:: +.. testcode:: class StaticMethod: "Emulate PyStaticMethod_Type() in Objects/funcobject.c" @@ -1206,6 +1240,22 @@ Using the non-data descriptor protocol, a pure Python version of def __get__(self, obj, objtype=None): return self.f +.. testcode:: + :hide: + + class E_sim: + @StaticMethod + def f(x): + return x * 10 + +.. doctest:: + :hide: + + >>> E_sim.f(3) + 30 + >>> E_sim().f(3) + 30 + Class methods ------------- diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst index df2cda9d..28d1ab15 100644 --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -397,9 +397,25 @@ The following exceptions are the exceptions that are usually raised. or :func:`eval`, or when reading the initial script or standard input (also interactively). - Instances of this class have attributes :attr:`filename`, :attr:`lineno`, - :attr:`offset` and :attr:`text` for easier access to the details. :func:`str` - of the exception instance returns only the message. + The :func:`str` of the exception instance returns only the error message. + + .. attribute:: filename + + The name of the file the syntax error occurred in. + + .. attribute:: lineno + + Which line number in the file the error occurred in. This is + 1-indexed: the first line in the file has a ``lineno`` of 1. + + .. attribute:: offset + + The column in the line where the error occurred. This is + 1-indexed: the first character in the line has an ``offset`` of 1. + + .. attribute:: text + + The source code text involved in the error. .. exception:: IndentationError diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst index 5c6b65f0..e42b380d 100644 --- a/Doc/tutorial/datastructures.rst +++ b/Doc/tutorial/datastructures.rst @@ -661,9 +661,8 @@ operators, not just comparisons. The comparison operators ``in`` and ``not in`` check whether a value occurs (does not occur) in a sequence. The operators ``is`` and ``is not`` compare -whether two objects are really the same object; this only matters for mutable -objects like lists. All comparison operators have the same priority, which is -lower than that of all numerical operators. +whether two objects are really the same object. All comparison operators have +the same priority, which is lower than that of all numerical operators. Comparisons can be chained. For example, ``a < b == c`` tests whether ``a`` is less than ``b`` and moreover ``b`` equals ``c``. diff --git a/Include/cpython/pystate.h b/Include/cpython/pystate.h index 29f6bf5a..f292da1d 100644 --- a/Include/cpython/pystate.h +++ b/Include/cpython/pystate.h @@ -58,7 +58,8 @@ struct _ts { /* Borrowed reference to the current frame (it can be NULL) */ PyFrameObject *frame; int recursion_depth; - int recursion_headroom; /* Allow 50 more calls to handle any errors. */ + char overflowed; /* The stack has overflowed. Allow 50 more calls + to handle the runtime error. */ char recursion_critical; /* The current calls must not cause a stack overflow. */ int stackcheck_counter; diff --git a/Include/internal/pycore_ceval.h b/Include/internal/pycore_ceval.h index e7ace9bd..18c8f027 100644 --- a/Include/internal/pycore_ceval.h +++ b/Include/internal/pycore_ceval.h @@ -90,8 +90,24 @@ static inline int _Py_EnterRecursiveCall_inline(const char *where) { #define Py_EnterRecursiveCall(where) _Py_EnterRecursiveCall_inline(where) +/* Compute the "lower-water mark" for a recursion limit. When + * Py_LeaveRecursiveCall() is called with a recursion depth below this mark, + * the overflowed flag is reset to 0. */ +static inline int _Py_RecursionLimitLowerWaterMark(int limit) { + if (limit > 200) { + return (limit - 50); + } + else { + return (3 * (limit >> 2)); + } +} + static inline void _Py_LeaveRecursiveCall(PyThreadState *tstate) { tstate->recursion_depth--; + int limit = tstate->interp->ceval.recursion_limit; + if (tstate->recursion_depth < _Py_RecursionLimitLowerWaterMark(limit)) { + tstate->overflowed = 0; + } } static inline void _Py_LeaveRecursiveCall_inline(void) { diff --git a/Include/patchlevel.h b/Include/patchlevel.h index 093ea5da..cf76d874 100644 --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -18,12 +18,12 @@ /*--start constants--*/ #define PY_MAJOR_VERSION 3 #define PY_MINOR_VERSION 9 -#define PY_MICRO_VERSION 3 +#define PY_MICRO_VERSION 4 #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_FINAL #define PY_RELEASE_SERIAL 0 /* Version as a string */ -#define PY_VERSION "3.9.3" +#define PY_VERSION "3.9.4" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/pdb.py b/Lib/pdb.py index 7a5192cb..98dc975e 100755 --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -1708,7 +1708,7 @@ def main(): print("The program finished and will be restarted") except Restart: print("Restarting", mainpyfile, "with arguments:") - print("\t" + " ".join(args)) + print("\t" + " ".join(sys.argv[1:])) except SystemExit: # In most cases SystemExit does not warrant a post-mortem session. print("The program exited via sys.exit(). Exit status:", end=' ') diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py index eb5b8936..e3b6f14e 100644 --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -1,5 +1,5 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Fri Apr 2 11:48:03 2021 +# Autogenerated by Sphinx on Sun Apr 4 14:53:49 2021 topics = {'assert': 'The "assert" statement\n' '**********************\n' '\n' diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py index 9c09bdc3..8d125b57 100644 --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -1043,7 +1043,7 @@ class ExceptionTests(unittest.TestCase): # tstate->recursion_depth is equal to (recursion_limit - 1) # and is equal to recursion_limit when _gen_throw() calls # PyErr_NormalizeException(). - recurse(setrecursionlimit(depth + 2) - depth) + recurse(setrecursionlimit(depth + 2) - depth - 1) finally: sys.setrecursionlimit(recursionlimit) print('Done.') @@ -1073,54 +1073,6 @@ class ExceptionTests(unittest.TestCase): b'while normalizing an exception', err) self.assertIn(b'Done.', out) - - def test_recursion_in_except_handler(self): - - def set_relative_recursion_limit(n): - depth = 1 - while True: - try: - sys.setrecursionlimit(depth) - except RecursionError: - depth += 1 - else: - break - sys.setrecursionlimit(depth+n) - - def recurse_in_except(): - try: - 1/0 - except: - recurse_in_except() - - def recurse_after_except(): - try: - 1/0 - except: - pass - recurse_after_except() - - def recurse_in_body_and_except(): - try: - recurse_in_body_and_except() - except: - recurse_in_body_and_except() - - recursionlimit = sys.getrecursionlimit() - try: - set_relative_recursion_limit(10) - for func in (recurse_in_except, recurse_after_except, recurse_in_body_and_except): - with self.subTest(func=func): - try: - func() - except RecursionError: - pass - else: - self.fail("Should have raised a RecursionError") - finally: - sys.setrecursionlimit(recursionlimit) - - @cpython_only def test_recursion_normalizing_with_no_memory(self): # Issue #30697. Test that in the abort that occurs when there is no @@ -1157,7 +1109,7 @@ class ExceptionTests(unittest.TestCase): except MemoryError as e: tb = e.__traceback__ else: - self.fail("Should have raised a MemoryError") + self.fail("Should have raises a MemoryError") return traceback.format_tb(tb) tb1 = raiseMemError() diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py index 6c4eaf31..0da449e3 100644 --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -1443,6 +1443,19 @@ def bœr(): 'Fail to handle a syntax error in the debuggee.' .format(expected, stdout)) + def test_issue26053(self): + # run command of pdb prompt echoes the correct args + script = "print('hello')" + commands = """ + continue + run a b c + run d e f + quit + """ + stdout, stderr = self.run_pdb_script(script, commands) + res = '\n'.join([x.strip() for x in stdout.splitlines()]) + self.assertRegex(res, "Restarting .* with arguments:\na b c") + self.assertRegex(res, "Restarting .* with arguments:\nd e f") def test_readrc_kwarg(self): script = textwrap.dedent(""" diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py index fce6b982..ed9b1770 100644 --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -219,7 +219,7 @@ class SysModuleTest(unittest.TestCase): def f(): f() try: - for depth in (50, 75, 100, 250, 1000): + for depth in (10, 25, 50, 75, 100, 250, 1000): try: sys.setrecursionlimit(depth) except RecursionError: @@ -229,17 +229,17 @@ class SysModuleTest(unittest.TestCase): # Issue #5392: test stack overflow after hitting recursion # limit twice - with self.assertRaises(RecursionError): - f() - with self.assertRaises(RecursionError): - f() + self.assertRaises(RecursionError, f) + self.assertRaises(RecursionError, f) finally: sys.setrecursionlimit(oldlimit) @test.support.cpython_only def test_setrecursionlimit_recursion_depth(self): # Issue #25274: Setting a low recursion limit must be blocked if the - # current recursion depth is already higher than limit. + # current recursion depth is already higher than the "lower-water + # mark". Otherwise, it may not be possible anymore to + # reset the overflowed flag to 0. from _testinternalcapi import get_recursion_depth @@ -260,10 +260,42 @@ class SysModuleTest(unittest.TestCase): sys.setrecursionlimit(1000) for limit in (10, 25, 50, 75, 100, 150, 200): - set_recursion_limit_at_depth(limit, limit) + # formula extracted from _Py_RecursionLimitLowerWaterMark() + if limit > 200: + depth = limit - 50 + else: + depth = limit * 3 // 4 + set_recursion_limit_at_depth(depth, limit) finally: sys.setrecursionlimit(oldlimit) + # The error message is specific to CPython + @test.support.cpython_only + def test_recursionlimit_fatalerror(self): + # A fatal error occurs if a second recursion limit is hit when recovering + # from a first one. + code = textwrap.dedent(""" + import sys + + def f(): + try: + f() + except RecursionError: + f() + + sys.setrecursionlimit(%d) + f()""") + with test.support.SuppressCrashReport(): + for i in (50, 1000): + sub = subprocess.Popen([sys.executable, '-c', code % i], + stderr=subprocess.PIPE) + err = sub.communicate()[1] + self.assertTrue(sub.returncode, sub.returncode) + self.assertIn( + b"Fatal Python error: _Py_CheckRecursiveCall: " + b"Cannot recover from stack overflow", + err) + def test_getwindowsversion(self): # Raise SkipTest if sys doesn't have getwindowsversion attribute test.support.get_attribute(sys, "getwindowsversion") diff --git a/Misc/NEWS b/Misc/NEWS index ba371386..944c3293 100644 --- a/Misc/NEWS +++ b/Misc/NEWS @@ -2,6 +2,27 @@ Python News +++++++++++ +What's New in Python 3.9.4 final? +================================= + +*Release date: 2021-04-04* + +Core and Builtins +----------------- + +- bpo-43710: Reverted the fix for https://bugs.python.org/issue42500 as it + changed the PyThreadState struct size and broke the 3.9.x ABI in the 3.9.3 + release (visible on 32-bit platforms using binaries compiled using an + earlier version of Python 3.9.x headers). + +Library +------- + +- bpo-26053: Fixed bug where the :mod:`pdb` interactive run command echoed + the args from the shell command line, even if those have been overridden + at the pdb prompt. + + What's New in Python 3.9.3 final? ================================= diff --git a/Python/ceval.c b/Python/ceval.c index b7176dc0..91e879e8 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -793,22 +793,23 @@ _Py_CheckRecursiveCall(PyThreadState *tstate, const char *where) _Py_CheckRecursionLimit = recursion_limit; } #endif - if (tstate->recursion_headroom) { + if (tstate->recursion_critical) + /* Somebody asked that we don't check for recursion. */ + return 0; + if (tstate->overflowed) { if (tstate->recursion_depth > recursion_limit + 50) { /* Overflowing while handling an overflow. Give up. */ Py_FatalError("Cannot recover from stack overflow."); } + return 0; } - else { - if (tstate->recursion_depth > recursion_limit) { - tstate->recursion_headroom++; - _PyErr_Format(tstate, PyExc_RecursionError, - "maximum recursion depth exceeded%s", - where); - tstate->recursion_headroom--; - --tstate->recursion_depth; - return -1; - } + if (tstate->recursion_depth > recursion_limit) { + --tstate->recursion_depth; + tstate->overflowed = 1; + _PyErr_Format(tstate, PyExc_RecursionError, + "maximum recursion depth exceeded%s", + where); + return -1; } return 0; } diff --git a/Python/errors.c b/Python/errors.c index d8c2d8b9..87af39d5 100644 --- a/Python/errors.c +++ b/Python/errors.c @@ -290,14 +290,12 @@ _PyErr_NormalizeException(PyThreadState *tstate, PyObject **exc, PyObject **val, PyObject **tb) { int recursion_depth = 0; - tstate->recursion_headroom++; PyObject *type, *value, *initial_tb; restart: type = *exc; if (type == NULL) { /* There was no exception, so nothing to do. */ - tstate->recursion_headroom--; return; } @@ -349,7 +347,6 @@ _PyErr_NormalizeException(PyThreadState *tstate, PyObject **exc, } *exc = type; *val = value; - tstate->recursion_headroom--; return; error: diff --git a/Python/pystate.c b/Python/pystate.c index 71aac6f6..9beefa8e 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -576,7 +576,7 @@ new_threadstate(PyInterpreterState *interp, int init) tstate->frame = NULL; tstate->recursion_depth = 0; - tstate->recursion_headroom = 0; + tstate->overflowed = 0; tstate->recursion_critical = 0; tstate->stackcheck_counter = 0; tstate->tracing = 0; diff --git a/Python/sysmodule.c b/Python/sysmodule.c index 4c7f7b67..3e4115fe 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -1160,6 +1160,7 @@ static PyObject * sys_setrecursionlimit_impl(PyObject *module, int new_limit) /*[clinic end generated code: output=35e1c64754800ace input=b0f7a23393924af3]*/ { + int mark; PyThreadState *tstate = _PyThreadState_GET(); if (new_limit < 1) { @@ -1177,7 +1178,8 @@ sys_setrecursionlimit_impl(PyObject *module, int new_limit) Reject too low new limit if the current recursion depth is higher than the new low-water mark. Otherwise it may not be possible anymore to reset the overflowed flag to 0. */ - if (tstate->recursion_depth >= new_limit) { + mark = _Py_RecursionLimitLowerWaterMark(new_limit); + if (tstate->recursion_depth >= mark) { _PyErr_Format(tstate, PyExc_RecursionError, "cannot set the recursion limit to %i at " "the recursion depth %i: the limit is too low", diff --git a/README.rst b/README.rst index ae7f264e..f62e3b36 100644 --- a/README.rst +++ b/README.rst @@ -1,4 +1,4 @@ -This is Python version 3.9.3 +This is Python version 3.9.4 ============================ .. image:: https://travis-ci.org/python/cpython.svg?branch=3.9