Tim Teichmann <t.teichmann@dashdos.com> tteichmann <44259103+tteichmann@users.noreply.github.com>
Tom Boyd <pezcore@users.noreply.github.com> pezcore <pezcore@users.noreply.github.com>
Tom Poole <t.b.poole@gmail.com> tpoole <t.b.poole@gmail.com>
+Tony LaTorre <tlatorre@uchicago.edu> tlatorre <tlatorre@uchicago.edu>
Travis Oliphant <travis@continuum.io> Travis E. Oliphant <teoliphant@gmail.com>
Travis Oliphant <travis@continuum.io> Travis Oliphant <oliphant@enthought.com>
Valentin Haenel <valentin@haenel.co> Valentin Haenel <valentin.haenel@gmx.de>
jobs:
- job: Linux_Python_36_32bit_full_with_asserts
pool:
- vmIMage: 'ubuntu-16.04'
+ vmImage: 'ubuntu-16.04'
steps:
- script: |
docker pull i386/ubuntu:bionic
# the docs even though i.e., numba uses another in their
# azure config for mac os -- Microsoft has indicated
# they will patch this issue
- vmIMage: macOS-10.13
+ vmImage: macOS-10.13
steps:
# the @0 refers to the (major) version of the *task* on Microsoft's
# end, not the order in the build matrix nor anything to do
testRunTitle: 'Publish test results for Python $(python.version)'
- job: Windows
pool:
- vmIMage: 'VS2017-Win2016'
+ vmImage: 'VS2017-Win2016'
variables:
# openblas URLs from numpy-wheels
# appveyor / Windows config
pip install $_.FullName
}
displayName: 'Build NumPy'
+ - bash: |
+ pushd . && cd .. && target=$(python -c "import numpy, os; print(os.path.abspath(os.path.join(os.path.dirname(numpy.__file__), '.libs')))") && popd
+ pip download -d destination --only-binary --no-deps numpy==1.14
+ cd destination && unzip numpy*.whl && cp numpy/.libs/*.dll $target
+ ls $target
+ displayName: 'Add extraneous & older DLL to numpy/.libs to probe DLL handling robustness'
+ condition: eq(variables['PYTHON_VERSION'], '3.6')
+ - script: pushd . && cd .. && python -c "from ctypes import windll; windll.kernel32.SetDefaultDllDirectories(0x00000800); import numpy" && popd
+ displayName: 'For gh-12667; Windows DLL resolution'
- script: python runtests.py -n --show-build-log --mode=$(TEST_MODE) -- -rsx --junitxml=junit/test-results.xml
displayName: 'Run NumPy Test Suite'
- task: PublishTestResults@2
--- /dev/null
+
+Contributors
+============
+
+A total of 5 people contributed to this release. People with a "+" by their
+names contributed a patch for the first time.
+
+* Charles Harris
+* Eric Wieser
+* Matti Picus
+* Tyler Reddy
+* Tony LaTorre +
+
+Pull requests merged
+====================
+
+A total of 7 pull requests were merged for this release.
+
+* `#12909 <https://github.com/numpy/numpy/pull/12909>`__: TST: fix vmImage dispatch in Azure
+* `#12923 <https://github.com/numpy/numpy/pull/12923>`__: MAINT: remove complicated test of multiarray import failure mode
+* `#13020 <https://github.com/numpy/numpy/pull/13020>`__: BUG: fix signed zero behavior in npy_divmod
+* `#13026 <https://github.com/numpy/numpy/pull/13026>`__: MAINT: Add functions to parse shell-strings in the platform-native...
+* `#13028 <https://github.com/numpy/numpy/pull/13028>`__: BUG: Fix regression in parsing of F90 and F77 environment variables
+* `#13038 <https://github.com/numpy/numpy/pull/13038>`__: BUG: parse shell escaping in extra_compile_args and extra_link_args
+* `#13041 <https://github.com/numpy/numpy/pull/13041>`__: BLD: Windows absolute path DLL loading
--- /dev/null
+==========================
+NumPy 1.16.2 Release Notes
+==========================
+
+NumPy 1.16.2 is a quick release fixing several problems encountered on Windows.
+The Python versions supported are 2.7 and 3.5-3.7. The Windows problems
+addressed are:
+
+- DLL load problems for NumPy wheels on Windows,
+- distutils command line parsing on Windows.
+
+There is also a regression fix correcting signed zeros produced by divmod, see
+below for details.
+
+Downstream developers building this release should use Cython >= 0.29.2 and, if
+using OpenBLAS, OpenBLAS > v0.3.4.
+
+If you are installing using pip, you may encounter a problem with older
+installed versions of NumPy that pip did not delete becoming mixed with the
+current version, resulting in an ``ImportError``. That problem is particularly
+common on Debian derived distributions due to a modified pip. The fix is to
+make sure all previous NumPy versions installed by pip have been removed. See
+`#12736 <https://github.com/numpy/numpy/issues/12736>`__ for discussion of the
+issue.
+
+
+Compatibility notes
+===================
+
+Signed zero when using divmod
+-----------------------------
+Starting in version 1.12.0, numpy incorrectly returned a negatively signed zero
+when using the ``divmod`` and ``floor_divide`` functions when the result was
+zero. For example::
+
+ >>> np.zeros(10)//1
+ array([-0., -0., -0., -0., -0., -0., -0., -0., -0., -0.])
+
+With this release, the result is correctly returned as a positively signed
+zero::
+
+ >>> np.zeros(10)//1
+ array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
+
+
+Contributors
+============
+
+A total of 5 people contributed to this release. People with a "+" by their
+names contributed a patch for the first time.
+
+* Charles Harris
+* Eric Wieser
+* Matti Picus
+* Tyler Reddy
+* Tony LaTorre +
+
+
+Pull requests merged
+====================
+
+A total of 7 pull requests were merged for this release.
+
+* `#12909 <https://github.com/numpy/numpy/pull/12909>`__: TST: fix vmImage dispatch in Azure
+* `#12923 <https://github.com/numpy/numpy/pull/12923>`__: MAINT: remove complicated test of multiarray import failure mode
+* `#13020 <https://github.com/numpy/numpy/pull/13020>`__: BUG: fix signed zero behavior in npy_divmod
+* `#13026 <https://github.com/numpy/numpy/pull/13026>`__: MAINT: Add functions to parse shell-strings in the platform-native...
+* `#13028 <https://github.com/numpy/numpy/pull/13028>`__: BUG: Fix regression in parsing of F90 and F77 environment variables
+* `#13038 <https://github.com/numpy/numpy/pull/13038>`__: BUG: parse shell escaping in extra_compile_args and extra_link_args
+* `#13041 <https://github.com/numpy/numpy/pull/13041>`__: BLD: Windows absolute path DLL loading
from .info import __doc__
from numpy.version import version as __version__
+import os
+
+# on Windows NumPy loads an important OpenBLAS-related DLL
+# and the code below aims to alleviate issues with DLL
+# path resolution portability with an absolute path DLL load
+if os.name == 'nt':
+ from ctypes import WinDLL
+ import glob
+ # convention for storing / loading the DLL from
+ # numpy/.libs/, if present
+ libs_path = os.path.abspath(os.path.join(os.path.dirname(__file__),
+ '..', '.libs'))
+ DLL_filenames = []
+ if os.path.isdir(libs_path):
+ for filename in glob.glob(os.path.join(libs_path, '*openblas*dll')):
+ # NOTE: would it change behavior to load ALL
+ # DLLs at this path vs. the name restriction?
+ WinDLL(os.path.abspath(filename))
+ DLL_filenames.append(filename)
+ if len(DLL_filenames) > 1:
+ import warnings
+ warnings.warn("loaded more than 1 DLL from .libs:\n%s" %
+ "\n".join(DLL_filenames),
+ stacklevel=1)
+
# disables OpenBLAS affinity setting of the main thread that limits
# python threads or processes to one core
-import os
env_added = []
for envkey in ['OPENBLAS_MAIN_FREE', 'GOTOBLAS_MAIN_FREE']:
if envkey not in os.environ:
config.add_extension('_operand_flag_tests',
sources=[join('src', 'umath', '_operand_flag_tests.c.src')])
- #######################################################################
- # _multiarray_module_test module #
- #######################################################################
-
- config.add_extension('_multiarray_module_test',
- sources=[join('src', 'multiarray',
- '_multiarray_module_test.c')])
-
config.add_data_dir('tests')
config.add_data_dir('tests/data')
+++ /dev/null
-#include "Python.h"
-
-/*
- * This is a dummy module. It will be used to ruin the import of multiarray
- * during testing. It exports two entry points, one to make the build happy,
- * and a multiarray one for the actual test. The content of the module is
- * irrelevant to the test.
- *
- * The code is from
- * https://docs.python.org/3/howto/cporting.html
- * or
- * https://github.com/python/cpython/blob/v3.7.0/Doc/howto/cporting.rst
- */
-
-#if defined _WIN32 || defined __CYGWIN__ || defined __MINGW32__
- #if defined __GNUC__ || defined __clang__
- #define DLL_PUBLIC __attribute__ ((dllexport))
- #else
- #define DLL_PUBLIC __declspec(dllexport)
- #endif
-#elif defined __GNUC__ || defined __clang__
- #define DLL_PUBLIC __attribute__ ((visibility ("default")))
-#else
- /* Enhancement: error now instead ? */
- #define DLL_PUBLIC
-#endif
-
-struct module_state {
- PyObject *error;
-};
-
-#if PY_MAJOR_VERSION >= 3
-#define GETSTATE(m) ((struct module_state*)PyModule_GetState(m))
-#else
-#define GETSTATE(m) (&_state)
-static struct module_state _state;
-#endif
-
-static PyObject *
-error_out(PyObject *m) {
- struct module_state *st = GETSTATE(m);
- PyErr_SetString(st->error, "something bad happened");
- return NULL;
-}
-
-static PyMethodDef multiarray_methods[] = {
- {"error_out", (PyCFunction)error_out, METH_NOARGS, NULL},
- {NULL, NULL}
-};
-
-#if PY_MAJOR_VERSION >= 3
-
-static int multiarray_traverse(PyObject *m, visitproc visit, void *arg) {
- Py_VISIT(GETSTATE(m)->error);
- return 0;
-}
-
-static int multiarray_clear(PyObject *m) {
- Py_CLEAR(GETSTATE(m)->error);
- return 0;
-}
-
-
-static struct PyModuleDef moduledef = {
- PyModuleDef_HEAD_INIT,
- "multiarray",
- NULL,
- sizeof(struct module_state),
- multiarray_methods,
- NULL,
- multiarray_traverse,
- multiarray_clear,
- NULL
-};
-
-#define INITERROR return NULL
-
-DLL_PUBLIC PyObject *
-PyInit_multiarray(void)
-
-#else
-#define INITERROR return
-
-void
-DLL_PUBLIC initmultiarray(void)
-#endif
-{
-#if PY_MAJOR_VERSION >= 3
- PyObject *module = PyModule_Create(&moduledef);
-#else
- PyObject *module = Py_InitModule("multiarray", multiarray_methods);
-#endif
- struct module_state *st;
- if (module == NULL)
- INITERROR;
- st = GETSTATE(module);
-
- st->error = PyErr_NewException("multiarray.Error", NULL, NULL);
- if (st->error == NULL) {
- Py_DECREF(module);
- INITERROR;
- }
-
-#if PY_MAJOR_VERSION >= 3
- return module;
-#endif
-}
-
-/*
- * Define a dummy entry point to make MSVC happy
- * Python's build system will export this function automatically
- */
-#if PY_MAJOR_VERSION >= 3
-
-PyObject *
-PyInit__multiarray_module_test(void)
-{
- return PyInit_multiarray();
-}
-
-#else
-
-void
-init_multiarray_module_test(void)
-{
- initmultiarray();
-}
-
-#endif
}
else {
/* if mod is zero ensure correct sign */
- mod = (b > 0) ? 0.0@c@ : -0.0@c@;
+ mod = npy_copysign@c@(0, b);
}
/* snap quotient to nearest integral value */
}
else {
/* if div is zero ensure correct sign */
- floordiv = (a / b > 0) ? 0.0@c@ : -0.0@c@;
+ floordiv = npy_copysign@c@(0, a/b);
}
*modulus = mod;
pytest.raises(ValueError, a.getfield, 'uint8', -1)
pytest.raises(ValueError, a.getfield, 'uint8', 16)
pytest.raises(ValueError, a.getfield, 'uint64', 0)
-
-def test_multiarray_module():
- # gh-12736
- # numpy 1.16 replaced the multiarray and umath c-extension modules with
- # a single _multiarray_umath one. For backward compatibility, it added a
- # pure-python multiarray.py and umath.py shim so people can still do
- # from numpy.core.multirarray import something-public-api
- # It turns out pip can leave old pieces of previous versions of numpy
- # around when installing a newer version. If the old c-extension modules
- # are found, they will be given precedence over the new pure-python ones.
- #
- # This test copies a multiarray c-extension in parallel with the pure-
- # python one, and starts another python interpreter to load multiarray.
- # The expectation is that import will fail.
- import subprocess, shutil
- core_dir = os.path.dirname(np.core.multiarray.__file__)
- cextension = np.core._multiarray_umath.__file__
- testfile = cextension.replace('_multiarray_umath', '_multiarray_module_test')
- badfile = cextension.replace('_multiarray_umath', 'multiarray')
- assert not os.path.exists(badfile), '%s exists, this numpy ' \
- 'installation is faulty' % badfile
- try:
- shutil.copy(testfile, badfile)
- p = subprocess.Popen([sys.executable, '-c', 'import numpy' ],
- stdout=subprocess.PIPE, stderr=subprocess.PIPE,
- env=os.environ.copy())
- stdout, stderr = p.communicate()
- r = p.wait()
- #print(stdout.decode())
- #print(stderr.decode())
- assert r != 0
- assert b'ImportError' in stderr
- finally:
- if os.path.exists(badfile):
- try:
- # can this fail?
- os.remove(badfile)
- except:
- print("Could not remove %s, remove it by hand" % badfile)
- raise
y = np.floor_divide(x**2, x)
assert_equal(y, [1.e+110, 0], err_msg=msg)
+ def test_floor_division_signed_zero(self):
+ # Check that the sign bit is correctly set when dividing positive and
+ # negative zero by one.
+ x = np.zeros(10)
+ assert_equal(np.signbit(x//1), 0)
+ assert_equal(np.signbit((-x)//1), 1)
def floor_divide_and_remainder(x, y):
return (np.floor_divide(x, y), np.remainder(x, y))
--- /dev/null
+"""
+Helper functions for interacting with the shell, and consuming shell-style
+parameters provided in config files.
+"""
+import os
+import shlex
+import subprocess
+try:
+ from shlex import quote
+except ImportError:
+ from pipes import quote
+
+__all__ = ['WindowsParser', 'PosixParser', 'NativeParser']
+
+
+class CommandLineParser:
+ """
+ An object that knows how to split and join command-line arguments.
+
+ It must be true that ``argv == split(join(argv))`` for all ``argv``.
+ The reverse neednt be true - `join(split(cmd))` may result in the addition
+ or removal of unnecessary escaping.
+ """
+ @staticmethod
+ def join(argv):
+ """ Join a list of arguments into a command line string """
+ raise NotImplemented
+
+ @staticmethod
+ def split(cmd):
+ """ Split a command line string into a list of arguments """
+ raise NotImplemented
+
+
+class WindowsParser:
+ """
+ The parsing behavior used by `subprocess.call("string")` on Windows, which
+ matches the Microsoft C/C++ runtime.
+
+ Note that this is _not_ the behavior of cmd.
+ """
+ @staticmethod
+ def join(argv):
+ # note that list2cmdline is specific to the windows syntax
+ return subprocess.list2cmdline(argv)
+
+ @staticmethod
+ def split(cmd):
+ import ctypes # guarded import for systems without ctypes
+ try:
+ ctypes.windll
+ except AttributeError:
+ raise NotImplementedError
+
+ # Windows has special parsing rules for the executable (no quotes),
+ # that we do not care about - insert a dummy element
+ if not cmd:
+ return []
+ cmd = 'dummy ' + cmd
+
+ CommandLineToArgvW = ctypes.windll.shell32.CommandLineToArgvW
+ CommandLineToArgvW.restype = ctypes.POINTER(ctypes.c_wchar_p)
+ CommandLineToArgvW.argtypes = (ctypes.c_wchar_p, ctypes.POINTER(ctypes.c_int))
+
+ nargs = ctypes.c_int()
+ lpargs = CommandLineToArgvW(cmd, ctypes.byref(nargs))
+ args = [lpargs[i] for i in range(nargs.value)]
+ assert not ctypes.windll.kernel32.LocalFree(lpargs)
+
+ # strip the element we inserted
+ assert args[0] == "dummy"
+ return args[1:]
+
+
+class PosixParser:
+ """
+ The parsing behavior used by `subprocess.call("string", shell=True)` on Posix.
+ """
+ @staticmethod
+ def join(argv):
+ return ' '.join(quote(arg) for arg in argv)
+
+ @staticmethod
+ def split(cmd):
+ return shlex.split(cmd, posix=True)
+
+
+if os.name == 'nt':
+ NativeParser = WindowsParser
+elif os.name == 'posix':
+ NativeParser = PosixParser
import sys
import re
import types
-import shlex
from numpy.compat import open_latin1
make_temp_file, get_shared_lib_extension
from numpy.distutils.exec_command import find_executable
from numpy.distutils.compat import get_exception
+from numpy.distutils import _shell_utils
from .environment import EnvironmentConfig
fixflags = []
if f77:
- f77 = shlex.split(f77, posix=(os.name == 'posix'))
+ f77 = _shell_utils.NativeParser.split(f77)
f77flags = self.flag_vars.f77
if f90:
- f90 = shlex.split(f90, posix=(os.name == 'posix'))
+ f90 = _shell_utils.NativeParser.split(f90)
f90flags = self.flag_vars.f90
freeflags = self.flag_vars.free
# XXX Assuming that free format is default for f90 compiler.
# should perhaps eventually be more throughly tested and more
# robustly handled
if fix:
- fix = shlex.split(fix, posix=(os.name == 'posix'))
+ fix = _shell_utils.NativeParser.split(fix)
fixflags = self.flag_vars.fix + f90flags
oflags, aflags, dflags = [], [], []
from numpy.distutils.command.config import config as cmd_config
from numpy.distutils.compat import get_exception
from numpy.distutils import customized_ccompiler
+from numpy.distutils import _shell_utils
import distutils.ccompiler
import tempfile
import shutil
for key in ['extra_compile_args', 'extra_link_args']:
# Get values
opt = self.cp.get(self.section, key)
+ opt = _shell_utils.NativeParser.split(opt)
if opt:
- tmp = {key : [opt]}
+ tmp = {key: opt}
dict_append(info, **tmp)
return info
--- /dev/null
+from __future__ import division, absolute_import, print_function
+
+import pytest
+import subprocess
+import os
+import json
+import sys
+
+from numpy.distutils import _shell_utils
+
+argv_cases = [
+ [r'exe'],
+ [r'path/exe'],
+ [r'path\exe'],
+ [r'\\server\path\exe'],
+ [r'path to/exe'],
+ [r'path to\exe'],
+
+ [r'exe', '--flag'],
+ [r'path/exe', '--flag'],
+ [r'path\exe', '--flag'],
+ [r'path to/exe', '--flag'],
+ [r'path to\exe', '--flag'],
+
+ # flags containing literal quotes in their name
+ [r'path to/exe', '--flag-"quoted"'],
+ [r'path to\exe', '--flag-"quoted"'],
+ [r'path to/exe', '"--flag-quoted"'],
+ [r'path to\exe', '"--flag-quoted"'],
+]
+
+
+@pytest.fixture(params=[
+ _shell_utils.WindowsParser,
+ _shell_utils.PosixParser
+])
+def Parser(request):
+ return request.param
+
+
+@pytest.fixture
+def runner(Parser):
+ if Parser != _shell_utils.NativeParser:
+ pytest.skip('Unable to run with non-native parser')
+
+ if Parser == _shell_utils.WindowsParser:
+ return lambda cmd: subprocess.check_output(cmd)
+ elif Parser == _shell_utils.PosixParser:
+ # posix has no non-shell string parsing
+ return lambda cmd: subprocess.check_output(cmd, shell=True)
+ else:
+ raise NotImplementedError
+
+
+@pytest.mark.parametrize('argv', argv_cases)
+def test_join_matches_subprocess(Parser, runner, argv):
+ """
+ Test that join produces strings understood by subprocess
+ """
+ # invoke python to return its arguments as json
+ cmd = [
+ sys.executable, '-c',
+ 'import json, sys; print(json.dumps(sys.argv[1:]))'
+ ]
+ joined = Parser.join(cmd + argv)
+ json_out = runner(joined).decode()
+ assert json.loads(json_out) == argv
+
+
+@pytest.mark.parametrize('argv', argv_cases)
+def test_roundtrip(Parser, argv):
+ """
+ Test that split is the inverse operation of join
+ """
+ try:
+ joined = Parser.join(argv)
+ assert argv == Parser.split(joined)
+ except NotImplementedError:
+ pytest.skip("Not implemented")
from numpy.testing import assert_, assert_equal
from numpy.distutils.system_info import system_info, ConfigParser
from numpy.distutils.system_info import default_lib_dirs, default_include_dirs
+from numpy.distutils import _shell_utils
def get_class(name, notfound_action=1):
[ALL]
library_dirs = {dir1:s}{pathsep:s}{dir2:s}
libraries = {lib1:s},{lib2:s}
-extra_compile_args = -I/fake/directory
+extra_compile_args = -I/fake/directory -I"/path with/spaces" -Os
runtime_library_dirs = {dir1:s}
[temp1]
[temp2]
library_dirs = {dir2:s}
libraries = {lib2:s}
-extra_link_args = -Wl,-rpath={lib2:s}
+extra_link_args = -Wl,-rpath={lib2_escaped:s}
rpath = {dir2:s}
"""
site_cfg = simple_site
'lib1': self._lib1,
'dir2': self._dir2,
'lib2': self._lib2,
- 'pathsep': os.pathsep
+ 'pathsep': os.pathsep,
+ 'lib2_escaped': _shell_utils.NativeParser.join([self._lib2])
})
# Write site.cfg
fd, self._sitecfg = mkstemp()
assert_equal(tsi.get_libraries(), [self._lib1, self._lib2])
assert_equal(tsi.get_runtime_lib_dirs(), [self._dir1])
extra = tsi.calc_extra_info()
- assert_equal(extra['extra_compile_args'], ['-I/fake/directory'])
+ assert_equal(extra['extra_compile_args'], ['-I/fake/directory', '-I/path with/spaces', '-Os'])
def test_temp1(self):
# Read in all information in the temp1 block
#-----------------------------------
# Path to the release notes
-RELEASE_NOTES = 'doc/release/1.16.1-notes.rst'
+RELEASE_NOTES = 'doc/release/1.16.2-notes.rst'
#-------------------------------------------------------
MAJOR = 1
MINOR = 16
-MICRO = 1
+MICRO = 2
ISRELEASED = True
VERSION = '%d.%d.%d' % (MAJOR, MINOR, MICRO)
#
# extra_compile_args
# Add additional arguments to the compilation of sources.
-# Simple variable with no parsing done.
+# Split into arguments in a platform-appropriate way.
# Provide a single line with all complete flags.
# extra_compile_args = -g -ftree-vectorize
#
# extra_link_args
# Add additional arguments when libraries/executables
# are linked.
-# Simple variable with no parsing done.
+# Split into arguments in a platform-appropriate way.
# Provide a single line with all complete flags.
# extra_link_args = -lgfortran
#
projects / platform / upstream / python3-numpy.git / commitdiff