_ _ ____ _ ___| | | | _ \| | / __| | | | |_) | | | (__| |_| | _ <| |___ \___|\___/|_| \_\_____| Changelog Version 7.59.0 (13 Mar 2018) Daniel Stenberg (13 Mar 2018) - release: 7.59.0 Kamil Dudka (13 Mar 2018) - tests/.../spnego.py: fix identifier typo Detected by Coverity Analysis: Error: IDENTIFIER_TYPO: curl-7.58.0/tests/python_dependencies/impacket/spnego.py:229: identifier_typo: Using "SuportedMech" appears to be a typo: * Identifier "SuportedMech" is only known to be referenced here, or in copies of this code. * Identifier "SupportedMech" is referenced elsewhere at least 4 times. curl-7.58.0/tests/python_dependencies/impacket/smbserver.py:2651: identifier_use: Example 1: Using identifier "SupportedMech". curl-7.58.0/tests/python_dependencies/impacket/smbserver.py:2308: identifier_use: Example 2: Using identifier "SupportedMech". curl-7.58.0/tests/python_dependencies/impacket/spnego.py:252: identifier_use: Example 3: Using identifier "SupportedMech" (2 total uses in this function). curl-7.58.0/tests/python_dependencies/impacket/spnego.py:229: remediation: Should identifier "SuportedMech" be replaced by "SupportedMech"? Closes #2379 Daniel Stenberg (13 Mar 2018) - CURLOPT_COOKIEFILE.3: "-" as file name means stdin Reported-by: Aron Bergman Bug: https://curl.haxx.se/mail/lib-2018-03/0049.html [ci skip] - Revert "hostip: fix compiler warning: 'variable set but not used'" This reverts commit a577059f92fc65bd6b81717f0737f897a5b34248. The assignment really needs to be there or we risk working with an uninitialized pointer. Michael Kaufmann (12 Mar 2018) - limit-rate: fix compiler warning follow-up to 72a0f62 Viktor Szakats (12 Mar 2018) - checksrc.pl: add -i and -m options To sync it with changes made for the libssh2 project. Also cleanup some whitespace. - curl-openssl.m4: fix spelling [ci skip] - FAQ: fix a broken URL [ci skip] Daniel Stenberg (12 Mar 2018) - http2: mark the connection for close on GOAWAY ... don't consider it an error! Assisted-by: Jay Satiro Reported-by: Łukasz Domeradzki Fixes #2365 Closes #2375 - credits: Viktor prefers without accent - openldap: white space changes, fixed up the copyright years - openldap: check ldap_get_attribute_ber() results for NULL before using CVE-2018-1000121 Reported-by: Dario Weisser Bug: https://curl.haxx.se/docs/adv_2018-97a2.html - FTP: reject path components with control codes Refuse to operate when given path components featuring byte values lower than 32. Previously, inserting a %00 sequence early in the directory part when using the 'singlecwd' ftp method could make curl write a zero byte outside of the allocated buffer. Test case 340 verifies. CVE-2018-1000120 Reported-by: Duy Phan Thanh Bug: https://curl.haxx.se/docs/adv_2018-9cd6.html - readwrite: make sure excess reads don't go beyond buffer end CVE-2018-1000122 Bug: https://curl.haxx.se/docs/adv_2018-b047.html Detected by OSS-fuzz - BUGS: updated link to security process - limit-rate: kick in even before "limit" data has been received ... and make sure to avoid integer overflows with really large values. Reported-by: 刘佩东 Fixes #2371 Closes #2373 - docs/SECURITY.md -> docs/SECURITY-PROCESS.md - SECURITY.md: call it the security process Michael Kaufmann (11 Mar 2018) - Curl_range: fix FTP-only and FILE-only builds follow-up to e04417d - hostip: fix compiler warning: 'variable set but not used' Daniel Stenberg (11 Mar 2018) - HTTP: allow "header;" to replace an internal header with a blank one Reported-by: Michael Kaufmann Fixes #2357 Closes #2362 - http2: verbose output new MAX_CONCURRENT_STREAMS values ... as it is interesting for many users. - SECURITY: distros' max embargo time is 14 days now Patrick Monnerat (8 Mar 2018) - curl tool: accept --compressed also if Brotli is enabled and zlib is not. Daniel Stenberg (5 Mar 2018) - THANKS + mailmap: remove duplicates, fixup full names - [sergii.kavunenko brought this change] WolfSSL: adding TLSv1.3 Closes #2349 - RELEASE-NOTES/THANKS: synced with cc1d4c505 - [Richard Alcock brought this change] winbuild: prefer documented zlib library names Check for existence of import and static libraries with documented names and use them if they do. Fallback to previous names. According to https://github.com/madler/zlib/blob/master/win32/README-WIN32.txt on Windows, the names of the import library is "zdll.lib" and static library is "zlib.lib". closes #2354 Marcel Raad (4 Mar 2018) - krb5: use nondeprecated functions gss_seal/gss_unseal have been deprecated in favor of gss_wrap/gss_unwrap with GSS-API v2 from January 1997 [1]. The first version of "The Kerberos Version 5 GSS-API Mechanism" [2] from June 1996 already says "GSS_Wrap() (formerly GSS_Seal())" and "GSS_Unwrap() (formerly GSS_Unseal())". Use the nondeprecated functions to avoid deprecation warnings. [1] https://tools.ietf.org/html/rfc2078 [2] https://tools.ietf.org/html/rfc1964 Closes https://github.com/curl/curl/pull/2356 Daniel Stenberg (4 Mar 2018) - curl.1: mention how to add numerical IP addresses in NO_PROXY - CURLOPT_NOPROXY.3: mention how to list numerical IPv6 addresses - NO_PROXY: fix for IPv6 numericals in the URL Added test 1265 that verifies. Reported-by: steelman on github Fixes #2353 Closes #2355 - build: get CFLAGS (including -werror) used for examples and tests ... so that the CI and more detects compiler warnings/errors properly! Closes #2337 Marcel Raad (3 Mar 2018) - curl_ctype: fix macro redefinition warnings On MinGW and Cygwin, GCC and clang have been complaining about macro redefinitions since 4272a0b0fc49a1ac0ceab5c4a365c9f6ab8bf8e2. Fix this by undefining the macros before redefining them as suggested in https://github.com/curl/curl/pull/2269. Suggested-by: Daniel Stenberg Dan Fandrich (2 Mar 2018) - unit1307: proper cleanup on OOM to fix torture tests Marcel Raad (28 Feb 2018) - unit1309: fix warning on Windows x64 When targeting x64, MinGW-w64 complains about conversions between 32-bit long and 64-bit pointers. Fix this by reusing the GNUTLS_POINTER_TO_SOCKET_CAST / GNUTLS_SOCKET_TO_POINTER_CAST logic from gtls.c, moving it to warnless.h as CURLX_POINTER_TO_INTEGER_CAST / CURLX_INTEGER_TO_POINTER_CAST. Closes https://github.com/curl/curl/pull/2341 - travis: update compiler versions Update clang to version 3.9 and GCC to version 6. Closes https://github.com/curl/curl/pull/2345 Daniel Stenberg (26 Feb 2018) - docs/MANUAL: formfind.pl is not accessible on the site anymore Fixes #2342 Jay Satiro (24 Feb 2018) - curl-openssl.m4: Fix version check for OpenSSL 1.1.1 - Add OpenSSL 1.1.1 to the header/library version lists. - Detect OpenSSL 1.1.1 library using its function ERR_clear_last_mark, which was added in that version. Prior to this change an erroneous header/library mismatch was caused by lack of OpenSSL 1.1.1 detection. I tested using openssl-1.1.1-pre1. Viktor Szakats (23 Feb 2018) - lib655: silence compiler warning Closes https://github.com/curl/curl/pull/2335 - spelling fixes Detected using the `codespell` tool. Also contains one URL protocol upgrade. Closes https://github.com/curl/curl/pull/2334 Daniel Stenberg (24 Feb 2018) - projects/README: remove reference to dead IDN link/package Reported-by: Stefan Kanthak and Rod Widdowson Fixes #2325 Jay Satiro (23 Feb 2018) - [Rod Widdowson brought this change] winbuild: Use macros for the names of some build utilities - Add macros to the top of the makefile for rc and mt utilities so that it is easier to change their locations. Bug: https://curl.haxx.se/mail/lib-2018-02/0075.html Reported-by: Stefan Kanthak Closes https://github.com/curl/curl/issues/2329 Daniel Stenberg (23 Feb 2018) - TODO: remove "sha-256 digest", added in 2b5b37cb9109e7c2 - curl_share_setopt.3: connection cache is shared within multi handles Jay Satiro (22 Feb 2018) - [Rod Widdowson brought this change] winbuild: Use CALL to run batch scripts Co-authored-by: Stefan Kanthak Closes https://github.com/curl/curl/issues/2330 Closes https://github.com/curl/curl/pull/2331 Patrick Monnerat (22 Feb 2018) - os400: add curl_resolver_start_callback type to ILE/RPG binding Daniel Stenberg (22 Feb 2018) - form.d: rephrased somewhat, added two example command lines Jay Satiro (21 Feb 2018) - [Francisco Sedano brought this change] url: Add option CURLOPT_RESOLVER_START_FUNCTION - Add new option CURLOPT_RESOLVER_START_FUNCTION to set a callback that will be called every time before a new resolve request is started (ie before a host is resolved) with a pointer to backend-specific resolver data. Currently this is only useful for ares. - Add new option CURLOPT_RESOLVER_START_DATA to set a user pointer to pass to the resolver start callback. Closes https://github.com/curl/curl/pull/2311 - lib: CURLOPT_HAPPY_EYEBALLS_TIMEOUT => CURLOPT_HAPPY_EYEBALLS_TIMEOUT_MS - In keeping with the naming of our other connect timeout options rename CURLOPT_HAPPY_EYEBALLS_TIMEOUT to CURLOPT_HAPPY_EYEBALLS_TIMEOUT_MS. This change adds the _MS suffix since the option expects milliseconds. This is more intuitive for our users since other connect timeout options that expect milliseconds use _MS such as CURLOPT_TIMEOUT_MS, CURLOPT_CONNECTTIMEOUT_MS, CURLOPT_ACCEPTTIMEOUT_MS. The tool option already uses an -ms suffix, --happy-eyeballs-timeout-ms. Follow-up to 2427d94 which added the lib and tool option yesterday. Ref: https://github.com/curl/curl/pull/2260 Patrick Monnerat (21 Feb 2018) - sasl: prefer PLAIN mechanism over LOGIN SASL PLAIN is a standard, LOGIN only a draft. The LOGIN draft says PLAIN should be used instead if available. Daniel Stenberg (21 Feb 2018) - RELEASE-NOTES: synced with 2427d94c6 Jay Satiro (20 Feb 2018) - [Anders Bakken brought this change] url: Add option CURLOPT_HAPPY_EYEBALLS_TIMEOUT - Add new option CURLOPT_HAPPY_EYEBALLS_TIMEOUT to set libcurl's happy eyeball timeout value. - Add new optval macro CURL_HET_DEFAULT to represent the default happy eyeballs timeout value (currently 200 ms). - Add new tool option --happy-eyeballs-timeout-ms to expose CURLOPT_HAPPY_EYEBALLS_TIMEOUT. The -ms suffix is used because the other -timeout options in the tool expect seconds not milliseconds. Closes https://github.com/curl/curl/pull/2260 - hostip: fix 'potentially uninitialized variable' warning Follow-up to 50d1b33. Caught by AppVeyor. Daniel Stenberg (20 Feb 2018) - TODO: warning if curl version is not in sync with libcurl version Jay Satiro (20 Feb 2018) - [Anders Bakken brought this change] CURLOPT_RESOLVE: Add support for multiple IP addresses per entry This enables users to preresolve but still take advantage of happy eyeballs and trying multiple addresses if some are not connecting. Ref: https://github.com/curl/curl/pull/2260 Daniel Stenberg (20 Feb 2018) - [Sergio Borghese brought this change] examples/sftpuploadresume: resume upload via CURLOPT_APPEND URL: https://curl.haxx.se/mail/lib-2018-02/0072.html - curl --version: show PSL if the run-time lib has it enabled ... not of the #define was set at build-time! - TODO: "Support in-memory certs/ca certs/keys" removed SSLKEYLOGFILE support (fixed) removed "consider SSL patches" (outdated) Closes #2310 - CURLOPT_HEADER.3: clarify problems with different data sizes - test1556: verify >16KB headers to the header callback - header callback: don't chop headers into smaller pieces Reported-by: Guido Berhoerster Fixes #2314 Closes #2316 - test1154: verify that long HTTP headers get rejected - http: fix the max header length detection logic Previously, it would only check for max length if the existing alloc buffer was to small to fit it, which often would make the header still get used. Reported-by: Guido Berhoerster Bug: https://curl.haxx.se/mail/lib-2018-02/0056.html Closes #2315 - CURLOPT_HEADERFUNCTION.3: fix typo from d939226813 Reported-by: Erik Johansson Bug: https://github.com/curl/curl/commit/d9392268131c1b8d18dec3fa30e0bded833a5db7#commitcomment-27607495 - CURLOPT_HEADERFUNCTION.3: mention folded headers - TODO: 1.1 Option to refuse usernames in URLs Also expanded the CURL_REFUSE_CLEARTEXT section with more ideas. - TODO: 1.7 Support HTTP/2 for HTTP(S) proxies - ssh: add two missing state names The list of state names (used in debug builds) was out of sync in relation to the list of states (used in all builds). I now added an assert to make sure the sizes of the two lists match, to aid in detecting this mistake better in the future. Regression since c92d2e14cf, shipped in 7.58.0. Reported-by: Somnath Kundu Fixes #2312 Closes #2313 - Revert "KNOWN_BUGS: 2.5 curl should not offer "ALPN: h2" when using https-proxy" This reverts commit de9fac00c40db321d44fa6fbab6eb62ec4c83998. Reported-by: Jay Satiro Jay Satiro (15 Feb 2018) - non-ascii: fix implicit declaration warning Follow-up to b46cfbc. Caught by Travis CI. Daniel Stenberg (15 Feb 2018) - travis: add build with iconv enabled ... to verify it builds and works fine. Ref: https://curl.haxx.se/mail/lib-2017-09/0031.html Closes #1872 - TODO: 18.18 retry on network is unreachable Closes #1603 - KNOWN_BUGS: 2.5 curl should not offer "ALPN: h2" when using https-proxy Closes #1254 Kamil Dudka (15 Feb 2018) - nss: use PK11_CreateManagedGenericObject() if available ... so that the memory allocated by applications using libcurl does not grow per each TLS connection. Bug: https://bugzilla.redhat.com/1510247 Closes #2297 Daniel Stenberg (15 Feb 2018) - [Björn Stenberg brought this change] TODO fixed: Detect when called from within callbacks Closes #2302 - BINDINGS: fix curb link (and remove ruby-curl-multi) Reported-by: Klaus Stein - curl_gssapi: make sure this file too uses our *printf() - libcurl-security.3: separate file:// section ... just to make it more apparent. Even if it repeats some pieces of information. - libcurl-security.3: the http://192.168.0.1/my_router_config case Mentioned-By: Rich Moore - libcurl-security.3: mention the URL standards problems too - libcurl-security.3: split out from libcurl-tutorial.3 To make more accessible. Merged in some new language from "URLs are dangerous things" as discussed on the mailing list a few days ago: Bug: https://curl.haxx.se/mail/lib-2018-02/0013.html - RELEASE-NOTES: synced with e551910f8 Patrick Monnerat (13 Feb 2018) - tests: new tests for http raw mode Test 319 checks proper raw mode data with non-chunked gzip transfer-encoded server data. Test 326 checks raw mode with chunked server data. Bug: #2303 Closes #2308 Kamil Dudka (12 Feb 2018) - tlsauthtype.d: works only if libcurl is built with TLS-SRP support Bug: https://bugzilla.redhat.com/1542256 Closes #2306 Patrick Monnerat (12 Feb 2018) - smtp: fix processing of initial dot in data RFC 5321 4.1.1.4 specifies the CRLF terminating the DATA command should be taken into account when chasing the . end marker. Thus a leading dot character in data is also subject to escaping. Tests 911 and test server are adapted to this situation. New tests 951 and 952 check proper handling of initial dot in data. Closes #2304 Daniel Stenberg (12 Feb 2018) - sha256: avoid redefine - [Douglas Mencken brought this change] sha256: build with OpenSSL < 0.9.8 too support for SHA-2 was introduced in OpenSSL 0.9.8 Closes #2305 - [Bruno Grasselli brought this change] README: language fix s/off/from Closes #2300 Patrick Monnerat (12 Feb 2018) - http_chunks: don't write chunks twice with CURLOPT_HTTP_TRANSFER_DECODING on Bug: #2303 Reported-By: Henry Roeland Daniel Stenberg (9 Feb 2018) - get_posix_time: only check for overflows if they can happen! Michael Kaufmann (9 Feb 2018) - schannel: fix "no previous prototype" compiler warning Jay Satiro (9 Feb 2018) - [Mohammad AlSaleh brought this change] content_encoding: Add "none" alias to "identity" Some servers return a "content-encoding" header with a non-standard "none" value. Add "none" as an alias to "identity" as a work-around, to avoid unrecognised content encoding type errors. Signed-off-by: Mohammad AlSaleh Closes https://github.com/curl/curl/pull/2298 Steve Holme (8 Feb 2018) - build-openssl.bat: Follow up to 648679ab8e to suppress copy/move output - build-openssl.bat: Fixed incorrect move if destination build folder exists Michael Kaufmann (8 Feb 2018) - schannel: fix compiler warnings Closes #2296 Steve Holme (7 Feb 2018) - curl_addrinfo.c: Allow Unix Domain Sockets to compile under Windows Windows 10.0.17061 SDK introduces support for Unix Domain Sockets. Added the necessary include file to curl_addrinfo.c. Note: The SDK (which is considered beta) has to be installed, VS 2017 project file has to be re-targeted for Windows 10.0.17061 and #define enabled in config-win32.h. Patrick Monnerat (7 Feb 2018) - fnmatch: optimize processing of consecutive *s and ?s pattern characters Reported-By: Daniel Stenberg Fixes #2291 Closes #2293 Steve Holme (6 Feb 2018) - build-openssl.bat/build-wolfssl.bat: Build platform is optional Whilst the compiler parameter is mandatory, platform is optional as it is automatically calculated by the :configure section. This partially reverts commit 6d62d2c55d. Daniel Stenberg (6 Feb 2018) - [Patrick Schlangen brought this change] openssl: Don't add verify locations when verifypeer==0 When peer verification is disabled, calling SSL_CTX_load_verify_locations is not necessary. Only call it when verification is enabled to save resources and increase performance. Closes #2290 Steve Holme (5 Feb 2018) - build-wolfssl.bat: Extend VC15 support to include Enterprise and Professional ...and not just the Community Edition. - build-openssl.bat: Extend VC15 support to include Enterprise and Professional ...and not just the Community Edition. Michael Kaufmann (5 Feb 2018) - time-cond: fix reading the file modification time on Windows On Windows, stat() may adjust the unix file time by a daylight saving time offset. Avoid this by calling GetFileTime() instead. Fixes #2164 Closes #2204 Daniel Stenberg (5 Feb 2018) - formdata: use the mime-content type function Reduce code duplication by making Curl_mime_contenttype available and used by the formdata function. This also makes the formdata function recognize a set of more file extensions by default. PR #2280 brought this to my attention. Closes #2282 - getdate: return -1 for out of range ...as that's how the function is documented to work. Reported-by: Michael Kaufmann Bug found in an autobuild with 32 bit time_t Closes #2278 - [Ben Greear brought this change] build: fix termios issue on android cross-compile Bug: https://curl.haxx.se/mail/lib-2018-01/0122.html Signed-off-by: Ben Greear - time_t-fixes: remove typecasts to 'long' for info.filetime They're now wrong. Reported-by: Michael Kaufmann Closes #2277 - curl_setup: move the precautionary define of SIZEOF_TIME_T ... up to before it may be used for the TIME_T_MAX/MIN logic. Reported-by: Michael Kaufmann - parsedate: s/#if/#ifdef Reported-by: Michael Kaufmann Bug: https://github.com/curl/curl/commit/1c39128d974666107fc6d9ea15f294036851f224#commitcomment-27246479 Patrick Monnerat (31 Jan 2018) - fnmatch: pattern syntax can no longer fail Whenever an expected pattern syntax rule cannot be matched, the character starting the rule loses its special meaning and the parsing is resumed: - backslash at the end of pattern string matches itself. - Error in [:keyword:] results in set containing :\[dekorwy. Unit test 1307 updated for this new situation. Closes #2273 - fnmatch: accept an alphanum to be followed by a non-alphanum in char set Also be more tolerant about set pattern syntax. Update unit test 1307 accordingly. Bug: https://curl.haxx.se/mail/lib-2018-01/0114.html - fnmatch: do not match the empty string with a character set Jay Satiro (30 Jan 2018) - build: fix windows build methods for curl_ctype.c - Fix winbuild and the VS project generator to treat curl_ctype.{c,h} as curlx files since they are required by both src and lib. Follow-up to 4272a0b which added curl_ctype. Daniel Stenberg (30 Jan 2018) - progress-bar.d: update to match implementation ... since commit 993dd5651a6 Reported-by: Martin Dreher Bug: https://github.com/curl/curl/pull/2242#issuecomment-361059228 Closes #2271 - http2: set DEBUG_HTTP2 to enable more HTTP/2 logging ... instead of doing it unconditionally in debug builds. It cluttered up the output a little too much. - [Max Dymond brought this change] file: Check the return code from Curl_range and bail out on error - [Max Dymond brought this change] Curl_range: add check to ensure "from <= to" - [Max Dymond brought this change] Curl_range: commonize FTP and FILE range handling Closes #2205 - RELEASE-NOTES: synced with 811beab9f - curlver: next release will be 7.59.0 - [Michał Janiszewski brought this change] curl/curl.h: fix comment typo for CURLOPT_DNS_LOCAL_IP6 Closes #2275 - time: support > year 2038 time stamps for system with 32bit long ... with the introduction of CURLOPT_TIMEVALUE_LARGE and CURLINFO_FILETIME_T. Fixes #2238 Closes #2264 - curl_easy_reset: clear digest auth state Bug: https://curl.haxx.se/mail/lib-2018-01/0074.html Reported-by: Ruurd Beerstra Fixes #2255 Closes #2272 - [Adam Marcionek brought this change] winbuild: make linker generate proper PDB Link.exe requires /DEBUG to properly generate a full pdb file on release builds. Closes #2274 - curl: add --proxy-pinnedpubkey To verify a proxy's public key. For when using HTTPS proxies. Fixes #2192 Closes #2268 - configure: set PATH_SEPARATOR to colon for PATH w/o separator The logic tries to figure out what the path separator in the $PATH variable is, but if there's only one directory in the $PATH it fails. This change make configure *guess* on colon instead of erroring out, simply because that is probably the more common character. PATH_SEPARATOR can always be set by the user to override the guessing. (tricky bug to reproduce, as in my case for example the configure script requires binaries in more than one directory so passing in a PATH with a single dir fails.) Reported-by: Earnestly on github Fixes #2202 Closes #2265 - curl_ctype: private is*() type macros and functions ... since the libc provided one are locale dependent in a way we don't want. Also, the "native" isalnum() (for example) works differently on different platforms which caused test 1307 failures on macos only. Closes #2269 Marcel Raad (29 Jan 2018) - build: open VC15 projects with VS 2017 Previously, they were opened with Visual Studio 2015 by default, which cannot build them. Daniel Stenberg (29 Jan 2018) - RELEASE-NOTES: synced with 094647fca - TODO: UTF-8 filenames in Content-Disposition Closes #1888 - KNOWN_BUGS: DICT responses show the underlying protocol Closes #1809 Jay Satiro (27 Jan 2018) - [Alessandro Ghedini brought this change] docs: fix typos in man pages Closes https://github.com/curl/curl/pull/2266 Patrick Monnerat (26 Jan 2018) - lib555: drop text conversion and encode data as ascii codes If CURL_DOES_CONVERSION is enabled, uploaded LFs are mapped to CRLFs, giving a result that is different from what is expected. This commit avoids using CURLOPT_TRANSFERTEXT and directly encodes data to upload in ascii. Bug: https://github.com/curl/curl/pull/1872 Daniel Stenberg (26 Jan 2018) - lib517: make variable static to avoid compiler warning ... with clang on macos Patrick Monnerat (26 Jan 2018) - lib544: sync ascii code data with textual data Data mismatch caused test 545 to fail when character encoding conversion is enabled. Bug: https://github.com/curl/curl/pull/1872 Daniel Stenberg (25 Jan 2018) - [Travis Burtrum brought this change] GSKit: restore pinnedpubkey functionality inadvertently removed in 283babfaf8d8f3bab9d3c63cea94eb0b84e79c37 Closes #2263 - [Dair Grant brought this change] darwinssl: Don't import client certificates into Keychain on macOS Closes #2085 - configure: fix the check for unsigned time_t Assign the time_t variable negative value and then check if it is greater than zero, which will evaluate true for unsigned time_t but false for signed time_t. - parsedate: fix date parsing for systems with 32 bit long Make curl_getdate() handle dates before 1970 as well (returning negative values). Make test 517 test dates for 64 bit time_t. This fixes bug (3) mentioned in #2238 Closes #2250 - [McDonough, Tim brought this change] openssl: fix pinned public key build error in FIPS mode Here is a version that should work with all versions of openssl 0.9.7 through 1.1.0. Links to the docs: https://www.openssl.org/docs/man1.0.2/crypto/EVP_DigestInit.html https://www.openssl.org/docs/man1.1.0/crypto/EVP_DigestInit.html At the very bottom of the 1.1.0 documentation there is a history section that states, " stack allocated EVP_MD_CTXs are no longer supported." If EVP_MD_CTX_create and EVP_MD_CTX_destroy are not defined, then a simple mapping can be used as described here: https://wiki.openssl.org/index.php/Talk:OpenSSL_1.1.0_Changes Closes #2258 - [Travis Burtrum brought this change] SChannel/WinSSL: Replace Curl_none_md5sum with Curl_schannel_md5sum - [Travis Burtrum brought this change] SChannel/WinSSL: Implement public key pinning Closes #1429 - bump: towards 7.58.1 - cookies: remove verbose "cookie size:" output It was once used for some debugging/verifying logic but should never have ended up in git! - TODO: hardcode the "localhost" addresses - TODO: CURL_REFUSE_CLEARTEXT An idea that popped up in discussions on twitter. - progress-bar: don't use stderr explicitly, use bar->out Reported-By: Gisle Vanem Bug: https://github.com/curl/curl/commit/993dd5651a6c853bfe3870f6a69c7b329fa4e8ce#commitcomment-27070080 GitHub (24 Jan 2018) - [Gisle Vanem brought this change] Fixes for MSDOS etc. djgpp do have 'mkdir(dir, mode)'. Other DOS-compilers does not But djgpp seems the only choice for MSDOS anyway. PellesC do have a 'F_OK' defined in it's . Update year in Copyright. - [Gisle Vanem brought this change] Fix small typo. Version 7.58.0 (23 Jan 2018) Daniel Stenberg (23 Jan 2018) - RELEASE: 7.58.0 - [Gisle Vanem brought this change] progress-bar: get screen width on windows - test1454: --connect-to with IPv6 address w/o IPv6 support! - CONNECT_TO: fail attempt to set an IPv6 numerical without IPv6 support Bug: https://curl.haxx.se/mail/lib-2018-01/0087.html Reported-by: John Hascall Closes #2257 - docs: fix man page syntax to make test 1140 OK again - http: prevent custom Authorization headers in redirects ... unless CURLOPT_UNRESTRICTED_AUTH is set to allow them. This matches how curl already handles Authorization headers created internally. Note: this changes behavior slightly, for the sake of reducing mistakes. Added test 317 and 318 to verify. Reported-by: Craig de Stigter Bug: https://curl.haxx.se/docs/adv_2018-b3bf.html - curl: progress bar refresh, get width using ioctl() Get screen width from the environment variable COLUMNS first, if set. If not, use ioctl(). If nether works, assume 79. Closes #2242 The "refresh" is for the -# output when no total transfer size is known. It will now only use a single updated line even for this case: The "-=O=-" ship moves when data is transferred. The four flying "hashes" move (on a sine wave) on each refresh, independent of data. - RELEASE-NOTES: synced with bb0ffcc36 - libcurl-env.3: first take - TODO: two possible name resolver improvements - [Kartik Mahajan brought this change] http2: don't close connection when single transfer is stopped Fixes #2237 Closes #2249 - test558: fix for multissl builds vtls.c:multissl_init() might do a curl_free() call so strip that out to make this work with more builds. We just want to verify that memorytracking works so skipping one line is no harm. - examples/url2file.c: add missing curl_global_cleanup() call Reported-by: XhstormR on github Fixes #2245 - [Michael Gmelin brought this change] SSH: Fix state machine for ssh-agent authentication In case an identity didn't match[0], the state machine would fail in state SSH_AUTH_AGENT instead of progressing to the next identity in ssh-agent. As a result, ssh-agent authentication only worked if the identity required happened to be the first added to ssh-agent. This was introduced as part of commit c4eb10e2f06fbd6cc904f1d78e4, which stated that the "else" statement was required to prevent getting stuck in state SSH_AUTH_AGENT. Given the state machine's logic and libssh2's interface I couldn't see how this could happen or reproduce it and I also couldn't find a more detailed description of the problem which would explain a test case to reproduce the problem this was supposed to fix. [0] libssh2_agent_userauth returning LIBSSH2_ERROR_AUTHENTICATION_FAILED Closes #2248 - openssl: fix potential memory leak in SSLKEYLOGFILE logic Coverity CID 1427646. - openssl: fix the libressl build again Follow-up to 84fcaa2e7. libressl does not have the API even if it says it is late OpenSSL version... Fixes #2246 Closes #2247 Reported-by: jungle-boogie on github - unit1307: test many wildcards too - curl_fnmatch: only allow 5 '*' sections in a single pattern ... to avoid excessive recursive calls. The number 5 is totally arbitrary and could be modified if someone has a good motivation. - ftp-wildcard: fix matching an empty string with "*[^a]" .... and avoid advancing the pointer to trigger an out of buffer read. Detected by OSS-fuzz Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=5251 Assisted-by: Max Dymond - SMB: fix numeric constant suffix and variable types 1. don't use "ULL" suffix since unsupported in older MSVC 2. use curl_off_t instead of custom long long ifdefs 3. make get_posix_time() not do unaligned data access Fixes #2211 Closes #2240 Reported-by: Chester Liu - [rouzier brought this change] CURLOPT_TCP_NODELAY.3: fix typo Closes #2239 - smtp/pop3/imap_get_message: decrease the data length too... Follow-up commit to 615edc1f73 which was incomplete. Assisted-by: Max Dymond Detected by OSS-fuzz Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=5206 - openssl: enable SSLKEYLOGFILE support by default Fixes #2210 Closes #2236 Patrick Monnerat (14 Jan 2018) - mime: clone mime tree upon easy handle duplication. A mime tree attached to an easy handle using CURLOPT_MIMEPOST is strongly bound to the handle: there is a pointer to the easy handle in each item of the mime tree and following the parent pointer list of mime items ends in a dummy part stored within the handle. Because of this binding, a mime tree cannot be shared between different easy handles, thus it needs to be cloned upon easy handle duplication. There is no way for the caller to get the duplicated mime tree handle: it is then set to be automatically destroyed upon freeing the new easy handle. New test 654 checks proper mime structure duplication/release. Add a warning note in curl_mime_data_cb() documentation about sharing user data between duplicated handles. Closes #2235 - docs: comment about CURLE_READ_ERROR returned by curl_mime_filedata Daniel Stenberg (13 Jan 2018) - test395: HTTP with overflow Content-Length value - test394: verify abort of rubbish in Content-Length: value - test393: verify --max-filesize with excessive Content-Length - HTTP: bail out on negative Content-Length: values ... and make the max filesize check trigger if the value is too big. Updates test 178. Reported-by: Brad Spencer Fixes #2212 Closes #2223 Marcel Raad (13 Jan 2018) - [Dan Johnson brought this change] configure.ac: append extra linker flags instead of prepending them. Link order should list libraries after the libraries that use them, so when we're guessing that we might also need to add -ldl in order to use -lssl, we should add -ldl after -lssl. Closes https://github.com/curl/curl/pull/2234 Daniel Stenberg (13 Jan 2018) - RELEASE-NOTES: synced with 6fa10c8fa Jay Satiro (13 Jan 2018) - setopt: fix SSLVERSION to allow CURL_SSLVERSION_MAX_ values Broken since f121575 (precedes 7.56.1). Bug: https://github.com/curl/curl/issues/2225 Reported-by: cmfrolick@users.noreply.github.com Closes https://github.com/curl/curl/pull/2227 Patrick Monnerat (13 Jan 2018) - setopt: reintroduce non-static Curl_vsetopt() for OS400 support This also upgrades ILE/RPG bindings with latest setopt options. Reported-By: jonrumsey on github Fixes #2230 Closes #2233 Jay Satiro (11 Jan 2018) - [Zhouyihai Ding brought this change] http2: fix incorrect trailer buffer size Prior to this change the stored byte count of each trailer was miscalculated and 1 less than required. It appears any trailer after the first that was passed to Curl_client_write would be truncated or corrupted as well as the size. Potentially the size of some subsequent trailer could be erroneously extracted from the contents of that trailer, and since that size is used by client write an out-of-bounds read could occur and cause a crash or be otherwise processed by client write. The bug appears to have been born in 0761a51 (precedes 7.49.0). Closes https://github.com/curl/curl/pull/2231 - [Basuke Suzuki brought this change] easy: fix connection ownership in curl_easy_pause Before calling Curl_client_chop_write(), change the owner of connection to the current Curl_easy handle. This will fix the issue #2217. Fixes https://github.com/curl/curl/issues/2217 Closes https://github.com/curl/curl/pull/2221 Daniel Stenberg (9 Jan 2018) - [Dimitrios Apostolou brought this change] system.h: Additionally check __LONG_MAX__ for defining curl_off_t __SIZEOF_LONG__ was introduced in GCC 4.4, __LONG_MAX__ was introduced in GCC 3.3. Closes #2216 - COPYING: it's 2018! - progress: calculate transfer speed on milliseconds if possible to increase accuracy for quick transfers Fixes #2200 Closes #2206 Jay Satiro (7 Jan 2018) - scripts: allow all perl scripts to be run directly - Enable execute permission (chmod +x) - Change interpreter to /usr/bin/env perl Closes https://github.com/curl/curl/pull/2222 - mail-rcpt.d: fix short-text description - build: remove HAVE_LIMITS_H check .. because limits.h presence isn't optional, it's required by C89. Ref: http://port70.net/~nsz/c/c89/c89-draft.html#2.2.4.2 Closes https://github.com/curl/curl/pull/2215 - openssl: fix memory leak of SSLKEYLOGFILE filename - Free the copy of SSLKEYLOGFILE env returned by curl_getenv during ossl initialization. Caught by ASAN. - Revert "curl/system.h: fix compilation with gcc on AIX PPC and IA64 HP-UX" This reverts commit c97648b55080343bb371522bf4233e94a2a13a99. SIZEOF_LONG should not be checked in system.h since that macro is only defined when building libcurl. Ref: https://github.com/curl/curl/pull/2186#issuecomment-354767080 Ref: https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html Michael Kaufmann (30 Dec 2017) - test1554: improve the error handling - test1554: add global initialization and cleanup Daniel Stenberg (29 Dec 2017) - curl_version_info.3: call the argument 'age' Reported-by: Pete Lomax Bug: https://curl.haxx.se/mail/lib-2017-12/0074.html Patrick Monnerat (27 Dec 2017) - [Mikalai Ananenka brought this change] brotli: data at the end of content can be lost Decoding loop implementation did not concern the case when all received data is consumed by Brotli decoder and the size of decoded data internally hold by Brotli decoder is greater than CURL_MAX_WRITE_SIZE. For content with unencoded length greater than CURL_MAX_WRITE_SIZE this can result in the loss of data at the end of content. Closes #2194 Jay Satiro (26 Dec 2017) - examples/cacertinmem: ignore cert-already-exists error - Ignore X509_R_CERT_ALREADY_IN_HASH_TABLE errors in the CTX callback since it's possible the cert may have already been loaded by libcurl. - Remove the EXAMPLE code in the CURLOPT_SSL_CTX_FUNCTION.3 doc. Instead have it direct the reader to this cacertinmem.c example. - Fix the CA certificate to use the right CA for example.com, Digicert. Bug: https://curl.haxx.se/mail/lib-2017-12/0057.html Reported-by: Thomas van Hesteren Closes https://github.com/curl/curl/pull/2182 - [Gisle Vanem brought this change] tool_getparam: Support size modifiers for --max-filesize - Move the size modifier detection code from limit-rate to its own function so that it can also be used with max-filesize. Size modifiers are the suffixes such as G (gigabyte), M (megabyte) etc. For example --max-filesize 1G Ref: https://curl.haxx.se/mail/archive-2017-12/0000.html Closes https://github.com/curl/curl/pull/2179 Steve Holme (22 Dec 2017) - build: Fixed incorrect script termination from commit ad1dc10e61 - Makefile.vc: Added our standard copyright header - winbuild: Added support for VC15 - build: Added Visual Studio 2017 project files - build-wolfssl.bat: Added support for VC15 - build-openssl.bat: Added support for VC15 Jay Satiro (22 Dec 2017) - [Dimitrios Apostolou brought this change] curl/system.h: fix compilation with gcc on AIX PPC and IA64 HP-UX Closes https://github.com/curl/curl/pull/2186 - [Mattias Fornander brought this change] examples/rtsp: fix error handling macros Closes https://github.com/curl/curl/pull/2185 Patrick Monnerat (20 Dec 2017) - curl_easy_reset: release mime-related data. Move curl_mime_initpart() and curl_mime_cleanpart() calls to lower-level functions dealing with UserDefined structure contents. This avoids memory leakages on curl-generated part mime headers. New test 2073 checks this using the cli tool --next option: it triggers a valgrind error if bug is present. Bug: https://curl.haxx.se/mail/lib-2017-12/0060.html Reported-by: Martin Galvan - content_encoding: rework zlib_inflate - When zlib version is < 1.2.0.4, process gzip trailer before considering extra data as an error. - Inflate with Z_BLOCK instead of Z_SYNC_FLUSH to maximize correct data and minimize corrupt data output. - Do not try to restart deflate decompression in raw mode if output has started or if the leading data is not available anymore. - New test 232 checks inflating raw-deflated content. Closes #2068 - brotli: allow compiling with version 0.6.0. Some error codes were not yet defined in brotli 0.6.0: do not issue code for them in this case. Daniel Stenberg (13 Dec 2017) - CURLOPT_READFUNCTION.3: refer to argument with correct name Bug: #2175 [ci skip] - rand: add a clang-analyzer work-around scan-build would warn on a potential access of an uninitialized buffer. I deem it a false positive and had to add this somewhat ugly work-around to silence it. - krb5: fix a potential access of uninitialized memory A scan-build warning. - conncache: fix a return code [regression] This broke in 07cb27c98e. Make sure to return 'result' properly. Pointed out by scan-build! - curl: support >256 bytes warning messsages Bug: #2174 Michael Kaufmann (12 Dec 2017) - libssh: fix a syntax error in configure.ac Follow-up to c92d2e1 Closes #2172 Daniel Stenberg (12 Dec 2017) - examples/smtp-mail.c: use separate defines for options and mail ... to make it clearer that the options want address-only, while the headers in an email can also have the real name. Assisted-by: Sean MacLennan - THANKS: added missing names ... as I reran the contrithanks script after the mailmap name fixups. - mailmap: added/clarified several names - setopt: less *or equal* than INT_MAX/1000 should be fine ... for the CURLOPT_TIMEOUT, CURLOPT_CONNECTTIMEOUT and CURLOPT_SERVER_RESPONSE_TIMEOUT range checks. Reported-by: Dominik Hölzl Bug: https://curl.haxx.se/mail/lib-2017-12/0037.html Closes #2173 - [Dmitry Kostjuchenko brought this change] vtls: replaced getenv() with curl_getenv() Fixed undefined symbol of getenv() which does not exist when compiling for Windows 10 App (CURL_WINDOWS_APP). Replaced getenv() with curl_getenv() which is aware of getenv() absence when CURL_WINDOWS_APP is defined. Closes #2171 - RELEASE-NOTES: synced with 3b9ea70ee - TODO: Expose tried IP addresses that failed Suggested-by: Rainer Canavan Closes #2126 - curl.1: mention http:// and https:// as valid proxy prefixes - curl.1: documented two missing valid exit codes - CURLOPT_DNS_LOCAL_IP4.3: fixed the seel also to not self-reference - Revert "curl: don't set CURLOPT_INTERLEAVEDATA" This reverts commit 9ffad8eb1329bb35c8988115ac7ed85cf91ef955. It was actually added rather recently in 8e8afa82cbb629 due to a crash that would otherwise happen in the RTSP code. As I don't think we've fixed that behavior yet, we better keep this work-around until we have fixed it better. Michael Kaufmann (10 Dec 2017) - tests: mark data files as non-executable in git - tests: update .gitignore for libtests Daniel Stenberg (10 Dec 2017) - multi_done: prune DNS cache Prune the DNS cache immediately after the dns entry is unlocked in multi_done. Timed out entries will then get discarded in a more orderly fashion. Test506 is updated Reported-by: Oleg Pudeyev Fixes #2169 Closes #2170 - mailmap: fixup two old git Author "aliases" Jay Satiro (10 Dec 2017) - openssl: Disable file buffering for Win32 SSLKEYLOGFILE Prior to this change SSLKEYLOGFILE used line buffering on WIN32 just like it does for other platforms. However, the Windows CRT does not actually support line buffering (_IOLBF) and will use full buffering (_IOFBF) instead. We can't use full buffering because multiple processes may be writing to the file and that could lead to corruption, and since full buffering is the only buffering available this commit disables buffering for Windows SSLKEYLOGFILE entirely (_IONBF). Ref: https://github.com/curl/curl/pull/1346#issuecomment-350530901 Daniel Stenberg (10 Dec 2017) - RESOLVE: output verbose text when trying to set a duplicate name ... to help users understand what is or isn't done! - CURLOPT_DNS_CACHE_TIMEOUT.3: see also CURLOPT_RESOLVE - [John DeHelian brought this change] sftp: allow quoted commands to use relative paths Closes #1900 Jay Satiro (8 Dec 2017) - [Richard Alcock brought this change] CURLOPT_PRIVATE.3: fix grammar - Change "never does nothing" double-negative to "never does anything". Closes https://github.com/curl/curl/pull/2168 Daniel Stenberg (8 Dec 2017) - curl: remove __EMX__ #ifdefs These are OS/2-specific things added to the code in the year 2000. They were always ugly. If there's any user left, they still don't need it done this way. Closes #2166 Jay Satiro (8 Dec 2017) - openssl: improve data-pending check for https proxy - Allow proxy_ssl to be checked for pending data even when connssl does not yet have an SSL handle. This change is for posterity. Currently there doesn't seem to be a code path that will cause a pending data check when proxyssl could have pending data and the connssl handle doesn't yet exist [1]. [1]: Recall that an https proxy connection starts out in connssl but if the destination is also https then the proxy SSL backend data is moved from connssl to proxyssl, which means connssl handle is temporarily empty until an SSL handle for the destination can be created. Ref: https://github.com/curl/curl/commit/f4a6238#commitcomment-24396542 Closes https://github.com/curl/curl/pull/1916 Daniel Stenberg (8 Dec 2017) - curl: don't set CURLOPT_INTERLEAVEDATA That data is only ever used by the CURLOPT_INTERLEAVEFUNCTION callback and that option isn't set or used by the curl tool! Updates the 9 tests that verify --libcurl Closes #2167 - curl.h: remove incorrect comment about ERRORBUFFER ... error messages are _not_ sent to stderr if this is not set. - [Michael Felt brought this change] configure: add AX_CODE_COVERAGE only if using gcc Fixes #2076 Closes #2125 - curl: limit -# update frequency for unknown total size Make it use a max 10Hz update frequency for this case as well. Return early if the "point" hasn't moved since last invoke. Reported-by: Elliot Saba Fixes #2158 Closes #2163 - BINDINGS: another PostgreSQL client ...the former link is dead. Reported-by: Frank Gevaerts - [Zachary Seguin brought this change] CONNECT: keep close connection flag in http_connect_state struct Fixes #2088 Closes #2157 - [Per Malmberg brought this change] include: get netinet/in.h before linux/tcp.h ... to allow build on older Linux dists (specifically CentOS 4.8 on gcc 4.8.5) Closes #2160 - openldap: fix checksrc nits - [Stepan Broz brought this change] openldap: add commented out debug possibilities ... to aid debugging openldap library using its built-in debug messages. Closes #2159 - examples: move threaded-shared-conn.c to the "complicated" ones ... due it relying on pthreads to link. - RELEASE-NOTES: synced with b261c44e8 ... and bump next release version to 7.58.0 - [Jan Ehrhardt brought this change] URL: tolerate backslash after drive letter for FILE: ... as in "file://c:\some\path\curl.out" Reviewed-by: Matthew Kerwin Closes #2154 - [Randall S. Becker brought this change] tests: added netinet/in6.h includes in test servers - [Randall S. Becker brought this change] configure: check for netinet/in6.h Needed by HPE NonStop NSE and NSX systems Fixes #2146 Closes #2155 - curl-config: add --ssl-backends Lists all SSL backends that were enabled at build-time. Suggested-by: Oleg Pudeyev Fixes #2128 - conncache: only allow multiplexing within same multi handle Connections that are used for HTTP/1.1 Pipelining or HTTP/2 multiplexing only get additional transfers added to them if the existing connection is held by the same multi or easy handle. libcurl does not support doing HTTP/2 streams in different threads using a shared connection. Closes #2152 - threaded-shared-conn.c: fixed typo in commenta - threaded-shared-conn.c: new example - conncache: fix several lock issues If the lock is released before the dealings with the bundle is over, it may have changed by another thread in the mean time. Fixes #2132 Fixes #2151 Closes #2139 - libssh: remove dead code in sftp_qoute ... by removing a superfluous NULL pointer check that also confuses Coverity. Fixes #2143 Closes #2153 - sasl_getmesssage: make sure we have a long enough string to pass For pop3/imap/smtp, added test 891 to somewhat verify the pop3 case. For this, I enhanced the pingpong test server to be able to send back responses with LF-only instead of always using CRLF. Closes #2150 - libssh2: remove dead code from SSH_SFTP_QUOTE Figured out while reviewing code in the libssh backend. The pointer was checked for NULL after having been dereferenced, so we know it would always equal true or it would've crashed. Pointed-out-by: Nikos Mavrogiannopoulos Bug #2143 Closes #2148 - ssh-libssh.c: please checksrc Nikos Mavrogiannopoulos (4 Dec 2017) - libssh: fixed dereference in statvfs access The behavior is now equivalent to ssh.c when SSH_SFTP_QUOTE_STATVFS handling fails. Fixes #2142 Daniel Stenberg (4 Dec 2017) - [Guitared brought this change] RESOURCES: update spec names Closes #2145 Nikos Mavrogiannopoulos (3 Dec 2017) - libssh: corrected use of sftp_statvfs() in SSH_SFTP_QUOTE_STATVFS The previous code was incorrectly following the libssh2 error detection for libssh2_sftp_statvfs, which is not correct for libssh's sftp_statvfs. Fixes #2142 Signed-off-by: Nikos Mavrogiannopoulos - libssh: no need to call sftp_get_error as ssh_get_error is sufficient Fixes #2141 Signed-off-by: Nikos Mavrogiannopoulos Daniel Stenberg (2 Dec 2017) - libssh: fix minor static code analyzer nits - remove superfluous NULL check which otherwise tricks the static code analyzers to assume NULL pointer dereferences. - fix fallthrough in switch() - indent mistake - openssl: pkcs12 is supported by boringssl Removes another #ifdef for BoringSSL Pointed-out-by: David Benjamin Closes #2134 - [Jay Satiro brought this change] travis: use pip2 instead of pip .. since now mac osx image expects pip2 or pip3, and doesn't know pip: 0.01s$ pip install --user cpp-coveralls /Users/travis/.travis/job_stages: line 57: pip: command not found Ref: https://github.com/travis-ci/travis-ci/issues/8829 Closes https://github.com/curl/curl/pull/2133 - [Nikos Mavrogiannopoulos brought this change] lib582: do not verify host for SFTP This SFTP test fails with libssh back-end due to failure to verify the peer. Disable peer verification in the test as there seems to be the intention of the test. Note that the libssh back-end automatically verifies the peer's host using the default known_hosts file. Signed-off-by: Nikos Mavrogiannopoulos - [Nikos Mavrogiannopoulos brought this change] libssh: added SFTP support The SFTP back-end supports asynchronous reading only, limited to 32-bit file length. Writing is synchronous with no other limitations. This also brings keyboard-interactive authentication. Signed-off-by: Nikos Mavrogiannopoulos - [Nikos Mavrogiannopoulos brought this change] symbols-in-versions: added new symbols with 7.56.3 version Signed-off-by: Nikos Mavrogiannopoulos - [Nikos Mavrogiannopoulos brought this change] .travis.yml: added build --with-libssh Signed-off-by: Nikos Mavrogiannopoulos - [Nikos Mavrogiannopoulos brought this change] libssh2: return CURLE_UPLOAD_FAILED on failure to upload This brings its in sync with the error code returned by the libssh backend. Signed-off-by: Nikos Mavrogiannopoulos - [Nikos Mavrogiannopoulos brought this change] libssh2: send the correct CURLE error code on scp file not found That also updates tests to expect the right error code libssh2 back-end returns CURLE_SSH error if the remote file is not found. Expect instead CURLE_REMOTE_FILE_NOT_FOUND which is sent by the libssh backend. Signed-off-by: Nikos Mavrogiannopoulos - [Nikos Mavrogiannopoulos brought this change] Added support for libssh SSH SCP back-end libssh is an alternative library to libssh2. https://www.libssh.org/ That patch set also introduces support for ECDSA ed25519 keys, as well as gssapi authentication. Signed-off-by: Nikos Mavrogiannopoulos - RELEASE-NOTES: synced with af8cc7a69 - curlver: towards 7.57.1 - [W. Mark Kubacki brought this change] lib: don't export all symbols, just everything curl_* Absent any 'symbol map' or script to limit what gets exported, static linking of libraries previously resulted in a libcurl with curl's and those other symbols being (re-)exported. This did not happen if 'versioned symbols' were enabled (which is not the default) because then a version script is employed. This limits exports to everything starting in 'curl_*'., which is what "libcurl.vers" exports. This avoids strange side-effects such as with mixing methods from system libraries and those erroneously offered by libcurl. Closes #2127 - [Johannes Schindelin brought this change] SSL: Avoid magic allocation of SSL backend specific data Originally, my idea was to allocate the two structures (or more precisely, the connectdata structure and the four SSL backend-specific strucutres required for ssl[0..1] and proxy_ssl[0..1]) in one go, so that they all could be free()d together. However, getting the alignment right is tricky. Too tricky. So let's just bite the bullet and allocate the SSL backend-specific data separately. As a consequence, we now have to be very careful to release the memory allocated for the SSL backend-specific data whenever we release any connectdata. Signed-off-by: Johannes Schindelin Closes #2119 - examples/xmlstream.c: don't switch off CURL_GLOBAL_SSL Reported-by: Dima Tisnek - travis: add boringssl build Uses a separate build without --enable-debug and no valgrind. The debug option causes far too many warnings in boringssl's headers (C++ comments, trailing commas etc). Valgrind triggers some false positive errors in thread-local data used by boringssl. Closes #2118 Version 7.57.0 (29 Nov 2017) Daniel Stenberg (29 Nov 2017) - RELEASE-NOTES: curl 7.57.0 - THANKS: added contributors from 7.57.0 release - openssl: fix boringssl build again commit d3ab7c5a21e broke the boringssl build since it doesn't have RSA_flags(), so we disable that code block for boringssl builds. Reported-by: W. Mark Kubacki Fixes #2117 - curl_ntlm_core.c: use the limits.h's SIZE_T_MAX if provided - libcurl-share.3: the connection cache is shareable now - global_init: ignore CURL_GLOBAL_SSL's absense This bit is no longer used. It is not clear what it meant for users to "init the TLS" in a world with different TLS backends and since the introduction of multissl, libcurl didn't properly work if inited without this bit set. Not a single user responded to the call for users of it: https://curl.haxx.se/mail/lib-2017-11/0072.html Reported-by: Evgeny Grin Assisted-by: Jay Satiro Fixes #2089 Fixes #2083 Closes #2107 - ntlm: avoid integer overflow for malloc size Reported-by: Alex Nichols Assisted-by: Kamil Dudka and Max Dymond CVE-2017-8816 Bug: https://curl.haxx.se/docs/adv_2017-11e7.html - wildcardmatch: fix heap buffer overflow in setcharset The code would previous read beyond the end of the pattern string if the match pattern ends with an open bracket when the default pattern matching function is used. Detected by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4161 CVE-2017-8817 Bug: https://curl.haxx.se/docs/adv_2017-ae72.html - [Jay Satiro brought this change] url: fix alignment of ssl_backend_data struct - Align the array of ssl_backend_data on a max 32 byte boundary. 8 is likely to be ok but I went with 32 for posterity should one of the ssl_backend_data structs change to contain a larger sized variable in the future. Prior to this change (since dev 70f1db3, release 7.56) the connectdata structure was undersized by 4 bytes in 32-bit builds with ssl enabled because long long * was mistakenly used for alignment instead of long long, with the intention being an 8 byte boundary. Also long long may not be an available type. The undersized connectdata could lead to oob read/write past the end in what was expected to be the last 4 bytes of the connection's secondary socket https proxy ssl_backend_data struct (the secondary socket in a connection is used by ftp, others?). Closes https://github.com/curl/curl/issues/2093 CVE-2017-8818 Bug: https://curl.haxx.se/docs/adv_2017-af0a.html - ssh: remove check for a NULL pointer (!) With this check present, scan-build warns that we might dereference this point in other places where it isn't first checked for NULL. Thus, if it *can* be NULL we have a problem on a few places. However, this pointer should not be possible to be NULL here so I remove the check and thus also three different scan-build warnings. Closes #2111 - [Matthew Kerwin brought this change] test: add test for bad UNC/SMB path in file: URL - [Matthew Kerwin brought this change] test: add tests to ensure basic file: URLs - [Matthew Kerwin brought this change] URL: update "file:" URL handling * LOTS of comment updates * explicit error for SMB shares (e.g. "file:////share/path/file") * more strict handling of authority (i.e. "//localhost/") * now accepts dodgy old "C:|" drive letters * more precise handling of drive letters in and out of Windows (especially recognising both "file:c:/" and "file:/c:/") Closes #2110 - metalink: fix memory-leak and NULL pointer dereference Reported by scan-build Closes #2109 - [Alessandro Ghedini brought this change] connect: add support for new TCP Fast Open API on Linux The new API added in Linux 4.11 only requires setting a socket option before connecting, without the whole sento() machinery. Notably, this makes it possible to use TFO with SSL connections on Linux as well, without the need to mess around with OpenSSL (or whatever other SSL library) internals. Closes #2056 - make: fix "make distclean" Fixes #2097 Closes #2108 - RELEASE-NOTES: synced with 31f18d272 Jay Satiro (23 Nov 2017) - connect: improve the bind error message eg consider a non-existent interface eth8, curl --interface eth8 Before: curl: (45) Could not resolve host: eth8 After: curl: (45) Couldn't bind to 'eth8' Bug: https://github.com/curl/curl/issues/2104 Reported-by: Alfonso Martone Daniel Stenberg (23 Nov 2017) - examples/rtsp: clear RANGE again after use Fixes #2106 Reported-by: youngchopin on github - [Michael Kaufmann brought this change] test1264: verify URL with space in host name being rejected - url: reject ASCII control characters and space in host names Host names like "127.0.0.1 moo" would otherwise be accepted by some getaddrinfo() implementations. Updated test 1034 and 1035 accordingly. Fixes #2073 Closes #2092 - Curl_open: fix OOM return error correctly Closes #2098 - http2: fix "Value stored to 'end' is never read" scan-build error - http2: fix "Value stored to 'hdbuf' is never read" scan-build error - openssl: fix "Value stored to 'rc' is never read" scan-build error - mime: fix "Value stored to 'sz' is never read" scan-build error - Curl_llist_remove: fix potential NULL pointer deref Fixes a scan-build warning. - ntlm: remove unnecessary NULL-check to please scan-build - BUGS: spellchecked Jay Satiro (18 Nov 2017) - [fmmedeiros brought this change] examples/curlx: Fix code style - Add braces around multi-line if statement. Closes https://github.com/curl/curl/pull/2096 Daniel Stenberg (17 Nov 2017) - resolve: allow IP address within [] brackets ... so that IPv6 addresses can be passed like they can for connect-to and how they're used in URLs. Added test 1324 to verify Reported-by: Alex Malinovich Fixes #2087 Closes #2091 - [Pavol Markovic brought this change] macOS: Fix missing connectx function with Xcode version older than 9.0 The previous fix https://github.com/curl/curl/pull/1788 worked just for Xcode 9. This commit extends the fix to older Xcode versions effectively by not using connectx function. Fixes https://github.com/curl/curl/issues/1330 Fixes https://github.com/curl/curl/issues/2080 Closes https://github.com/curl/curl/pull/1336 Closes #2082 - [Dirk Feytons brought this change] openssl: fix too broad use of HAVE_OPAQUE_EVP_PKEY Fixes #2079 Closes #2081 - TODO: ignore private IP addresses in PASV response Closes #1455 - RELEASE-NOTES: synced with ae7369b6d Michael Kaufmann (14 Nov 2017) - URL: return error on malformed URLs with junk after IPv6 bracket Follow-up to aadb7c7. Verified by new test 1263. Closes #2072 Daniel Stenberg (14 Nov 2017) - INTERNALS: we may use libidn2 now, not libidn Patrick Monnerat (13 Nov 2017) - zlib/brotli: only include header files in modules needing them There is a conflict on symbol 'free_func' between openssl/crypto.h and zlib.h on AIX. This is an attempt to resolve it. Bug: https://curl.haxx.se/mail/lib-2017-11/0032.html Reported-By: Michael Felt Daniel Stenberg (13 Nov 2017) - SMB: fix uninitialized local variable Reported-by: Brian Carpenter - [Orgad Shaneh brought this change] connect.c: remove executable bit on file Closes #2071 - [hsiao yi brought this change] README.md: fixed layout Closes #2069 - setopt: split out curl_easy_setopt() to its own file ... to make url.c smaller. Closes #1944 Jay Satiro (10 Nov 2017) - [John Starks brought this change] cmake: Add missing setmode check Ensure HAVE_SETMODE is set to 1 on OSes that have setmode. Without this, curl will corrupt binary files when writing them to stdout on Windows. Closes https://github.com/curl/curl/pull/2067 Daniel Stenberg (10 Nov 2017) - curl_share_setopt: va_end was not called if conncache errors CID 984459, detected by Coverity Sergei Nikulov (10 Nov 2017) - [John Starks brought this change] cmake: Correctly include curl.rc in Windows builds (#2064) Update CMakeLists.txt to add curl.rc to the correct list. Daniel Stenberg (9 Nov 2017) - RELEASE-NOTES: synced with 32828cc4f - [Luca Boccassi brought this change] --interface: add support for Linux VRF The --interface command (CURLOPT_INTERFACE option) already uses SO_BINDTODEVICE on Linux, but it tries to parse it as an interface or IP address first, which fails in case the user passes a VRF. Try to use the socket option immediately and parse it as a fallback instead. Update the documentation to mention this feature, and that it requires the binary to be ran by root or with CAP_NET_RAW capabilities for this to work. Closes #2024 - curl_share_setopt.3: document CURL_LOCK_DATA_CONNECT Closes #2043 - examples: add shared-connection-cache - test1554: verify connection cache sharing - share: add support for sharing the connection cache - imap: deal with commands case insensitively As documented in RFC 3501 section 9: https://tools.ietf.org/html/rfc3501#section-9 Closes #2061 - connect: store IPv6 connection status after valid connection ... previously it would store it already in the happy eyeballs stage which could lead to the IPv6 bit being set for an IPv4 connection, leading to curl not wanting to do EPSV=>PASV for FTP transfers. Closes #2053 - curl_multi_fdset.3: emphasize curl_multi_timeout ... even when there's no socket to wait for, the timeout can still be very short. Jay Satiro (9 Nov 2017) - content_encoding: fix inflate_stream for no bytes available - Don't call zlib's inflate() when avail_in stream bytes is 0. This is a follow up to the parent commit 19e66e5. Prior to that change libcurl's inflate_stream could call zlib's inflate even when no bytes were available, causing inflate to return Z_BUF_ERROR, and then inflate_stream would treat that as a hard error and return CURLE_BAD_CONTENT_ENCODING. According to the zlib FAQ, Z_BUF_ERROR is not fatal. This bug would happen randomly since packet sizes are arbitrary. A test of 10,000 transfers had 55 fail (ie 0.55%). Ref: https://zlib.net/zlib_faq.html#faq05 Closes https://github.com/curl/curl/pull/2060 Patrick Monnerat (7 Nov 2017) - content_encoding: do not write 0 length data Daniel Stenberg (6 Nov 2017) - fnmatch: remove dead code There was a duplicate check for backslashes in the setcharset() function. Coverity CID 1420611 - url: remove unncessary NULL-check Since 'conn' won't be NULL in there and we also access the pointer in there without the check. Coverity CID 1420610 Viktor Szakats (6 Nov 2017) - src/Makefile.m32: fix typo in brotli lib customization Ref cc1f4436099decb9d1a7034b2bb773a9f8379d31 - Makefile.m32: allow to customize brotli libs It adds the ability to link against static brotli libs. Also fix brotli include path. Patrick Monnerat (5 Nov 2017) - travis: add a job with brotli enabled - [Viktor Szakats brought this change] Makefile.m32: add brotli support - HTTP: implement Brotli content encoding This uses the brotli external library (https://github.com/google/brotli). Brotli becomes a feature: additional curl_version_info() bit and structure fields are provided for it and CURLVERSION_NOW bumped. Tests 314 and 315 check Brotli content unencoding with correct and erroneous data. Some tests are updated to accomodate with the now configuration dependent parameters of the Accept-Encoding header. - HTTP: support multiple Content-Encodings This is implemented as an output streaming stack of unencoders, the last calling the client write procedure. New test 230 checks this feature. Bug: https://github.com/curl/curl/pull/2002 Reported-By: Daniel Bankhead Jay Satiro (4 Nov 2017) - url: remove arg value check from CURLOPT_SSH_AUTH_TYPES Since CURLSSH_AUTH_ANY (aka CURLSSH_AUTH_DEFAULT) is ~0 an arg value check on this option is incorrect; we have to accept any value. Prior to this change since f121575 (7.56.1+) CURLOPT_SSH_AUTH_TYPES erroneously rejected CURLSSH_AUTH_ANY with CURLE_BAD_FUNCTION_ARGUMENT. Bug: https://github.com/curl/curl/commit/f121575#commitcomment-25347120 Daniel Stenberg (4 Nov 2017) - ntlm: avoid malloc(0) for zero length passwords It triggers an assert() when built with memdebug since malloc(0) may return NULL *or* a valid pointer. Detected by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4054 Assisted-by: Max Dymond Closes #2054 - RELEASE-NOTES: synced with ee8016b3d - curl: speed up handling of many URLs By properly keeping track of the last entry in the list of URLs/uploads to handle, curl now avoids many meaningless traverses of the list which speeds up many-URL handling *MASSIVELY* (several magnitudes on 100K URLs). Added test 1291, to verify that it doesn't take ages - but we don't have any detection of "too slow" command in the test suite. Reported-by: arainchik on github Fixes #1959 Closes #2052 - curl: pass through [] in URLs instead of calling globbing error Assisted-by: Per Lundberg Fixes #2044 Closes #2046 Closes #2048 - CURLOPT_INFILESIZE: accept -1 Regression since f121575 Reported-by: Petr Voytsik Fixes #2047 Jay Satiro (2 Nov 2017) - url: fix CURLOPT_DNS_CACHE_TIMEOUT arg value check to allow -1 Prior to this change since f121575 (7.56.1+) CURLOPT_DNS_CACHE_TIMEOUT erroneously rejected -1 with CURLE_BAD_FUNCTION_ARGUMENT. Dan Fandrich (1 Nov 2017) - http2: Fixed OOM handling in upgrade request This caused the torture tests on test 1800 to fail. - tests: Fixed torture tests on tests 556 and 650 Test cleanup after OOM wasn't being consistently performed. Daniel Stenberg (1 Nov 2017) - CURLOPT_MAXREDIRS: allow -1 as a value ... which is valid according to documentation. Regression since f121575c0b5f. Verified now in test 501. Reported-by: cbartl on github Fixes #2038 Closes #2039 - include: remove conncache.h inclusion from where its not needed Jay Satiro (1 Nov 2017) - url: fix CURLOPT_POSTFIELDSIZE arg value check to allow -1 .. also add same arg value check to CURLOPT_POSTFIELDSIZE_LARGE. Prior to this change since f121575 (7.56.1+) CURLOPT_POSTFIELDSIZE erroneously rejected -1 value with CURLE_BAD_FUNCTION_ARGUMENT. Bug: https://curl.haxx.se/mail/lib-2017-11/0000.html Reported-by: Andrew Lambert Daniel Stenberg (31 Oct 2017) - cookie: avoid NULL dereference ... when expiring old cookies. Reported-by: Pavel Gushchin Fixes #2032 Closes #2035 Marcel Raad (30 Oct 2017) - memdebug: use send/recv signature for curl_dosend/curl_dorecv This avoids build errors and warnings caused by implicit casts. Closes https://github.com/curl/curl/pull/2031 Daniel Stenberg (30 Oct 2017) - [Juro Bystricky brought this change] mkhelp.pl: support reproducible build Do not generate line with the current date, such as: * Generation time: Tue Oct-24 18:01:41 2017 This will improve reproducibility. The generated string is only part of a comment, so there should be no adverse consequences. Signed-off-by: Juro Bystricky closes #2026 Dan Fandrich (30 Oct 2017) - runtests.pl: Fixed typo in message Daniel Stenberg (30 Oct 2017) - curlx: the timeval functions are no longer provided as curlx_* Pointed-out-by: Dmitri Tikhonov Bug: #2034 - select: update comments s/curlx_tvnow/Curl_now - INTERNALS: remove curlx_tv* functions no longer provided - [Dmitri Tikhonov brought this change] timeval: use mach time on MacOS If clock_gettime() is not supported, use mach_absolute_time() on MacOS. closes #2033 monnerat (29 Oct 2017) - [Patrick Monnerat brought this change] cli tool: improve ";type=" handling in -F option arguments - [Patrick Monnerat brought this change] cli tool: in -F option arg, comma is a delimiter for files only Also upgrade test 1133 to cover this case and clarify man page about form data quoting. Bug: https://github.com/curl/curl/issues/2022 Reported-By: omau on github Daniel Stenberg (29 Oct 2017) - timeleft: made two more users of Curl_timeleft use timediff_t Jakub Zakrzewski (28 Oct 2017) - cmake: Export libcurl and curl targets to use by other cmake projects The config files define curl and libcurl targets as imported targets CURL::curl and CURL::libcurl. For backward compatibility with CMake- provided find-module the CURL_INCLUDE_DIRS and CURL_LIBRARIES are also set. Closes #1879 Daniel Stenberg (28 Oct 2017) - RELEASE-NOTES: synced with f20cbac97 - [Florin Petriuc brought this change] auth: Added test cases for RFC7616 Updated docs to include support for RFC7616 Signed-off-by: Florin Closes #1934 - [Florin Petriuc brought this change] auth: add support for RFC7616 - HTTP Digest access authentication Signed-off-by: Florin - [Daniel Bankhead brought this change] TODO: support multiple Content-Encodings Closes #2002 - ROADMAP: cleanup Removed done stuff. Removed entries no longer considered for the near term. - [Magicansk brought this change] ROADMAP.md: spelling fixes Closes #2028 - Curl_timeleft: change return type to timediff_t returning 'time_t' is problematic when that type is unsigned and we return values less than zero to signal "already expired", used in several places in the code. Closes #2021 - appveyor: add a win32 build - setopt: fix CURLOPT_SSH_AUTH_TYPES option read Regression since f121575c0b5f Reported-by: Rob Cotrone Marcel Raad (27 Oct 2017) - resolvers: only include anything if needed This avoids warnings about unused stuff. Closes https://github.com/curl/curl/pull/2023 Daniel Stenberg (27 Oct 2017) - HELP-US: rename the subtitle too since the label is changed "PR-welcome" was the former name. - curl_setup.h: oops, shorten the too long line - [Martin Storsjo brought this change] curl_setup: Improve detection of CURL_WINDOWS_APP If WINAPI_FAMILY is defined, it should be safe to try to include winapifamily.h to check what the define evaluates to. This should fix detection of CURL_WINDOWS_APP if building with _WIN32_WINNT set to 0x0600. Closes #2025 Jay Satiro (26 Oct 2017) - transfer: Fix chunked-encoding upload bug - When uploading via chunked-encoding don't compare file size to bytes sent to determine whether the upload has finished. Chunked-encoding adds its own overhead which why the bytes sent is not equal to the file size. Prior to this change if a file was uploaded in chunked-encoding and its size was known it was possible that the upload could end prematurely without sending the final few chunks. That would result in a server hang waiting for the remaining data, likely followed by a disconnect. The scope of this bug is limited to some arbitrary file sizes which have not been determined. One size that triggers the bug is 475020. Bug: https://github.com/curl/curl/issues/2001 Reported-by: moohoorama@users.noreply.github.com Closes https://github.com/curl/curl/pull/2010 Daniel Stenberg (26 Oct 2017) - timeval: make timediff_t also work on 32bit windows ... by using curl_off_t for the typedef if time_t is larger than 4 bytes. Reported-by: Gisle Vanem Bug: https://github.com/curl/curl/commit/b9d25f9a6b3ca791385b80a6a3c3fa5ae113e1e0#co mmitcomment-25205058 Closes #2019 - curl_fnmatch: return error on illegal wildcard pattern ... instead of doing an infinite loop! Added test 1162 to verify. Reported-by: Max Dymond Fixes #2015 Closes #2017 - [Max Dymond brought this change] wildcards: don't use with non-supported protocols Fixes timeouts in the fuzzing tests for non-FTP protocols. Closes #2016 - [Max Dymond brought this change] multi: allow table handle sizes to be overridden Allow users to specify their own hash define for CURL_CONNECTION_HASH_SIZE so that both values can be overridden. Closes #1982 - time: rename Curl_tvnow to Curl_now ... since the 'tv' stood for timeval and this function does not return a timeval struct anymore. Also, cleaned up the Curl_timediff*() functions to avoid typecasts and clean up the descriptive comments. Closes #2011 - ftplistparser: follow-up cleanup to remove PL_ERROR() - [Max Dymond brought this change] ftplistparser: free off temporary memory always When using the FTP list parser, ensure that the memory that's allocated is always freed. Detected by OSS-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3682 Closes #2013 - timediff: return timediff_t from the time diff functions ... to cater for systems with unsigned time_t variables. - Renamed the functions to curlx_timediff and Curl_timediff_us. - Added overflow protection for both of them in either direction for both 32 bit and 64 bit time_ts - Reprefixed the curlx_time functions to use Curl_* Reported-by: Peter Piekarski Fixes #2004 Closes #2005 - [Paul Howarth brought this change] libtest: Add required test libraries for lib1552 and lib1553 They use $(TESTUTIL) and thus should use $(TESTUTIL_LIBS) too. This fixes build failures on Fedora 13. Closes #2006 - [Alessandro Ghedini brought this change] libcurl-tutorial.3: fix typo closes #2008 Alessandro Ghedini (23 Oct 2017) - curl_mime_filedata.3: fix typos Daniel Stenberg (23 Oct 2017) - RELEASE-NOTES: clean slate towards 7.57.0 - [Max Dymond brought this change] travis: exit if any steps fail We don't expect any steps to fail in travis. Exit the script if they do. Closes #1966 Version 7.56.1 (23 Oct 2017) Daniel Stenberg (23 Oct 2017) - RELEASE-NOTES: 7.56.1 - THANKS: update at 7.56.1 release time - [Jon DeVree brought this change] mk-ca-bundle: Remove URL for aurora Aurora is no longer used by Mozilla https://hacks.mozilla.org/2017/04/simplifying-firefox-release-channels/ - [Jon DeVree brought this change] mk-ca-bundle: Fix URL for NSS The 'tip' is the most recent branch committed to, this should be 'default' like the URLs for the browser are. Closes #1998 - imap: if a FETCH response has no size, don't call write callback CVE-2017-1000257 Reported-by: Brian Carpenter and 0xd34db347 Also detected by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3586 - ftp: reject illegal IP/port in PASV 227 response ... by using range checks. Among other things, this avoids an undefined behavior for a left shift that could happen on negative or very large values. Closes #1997 Detected by OSS-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3694 Patrick Monnerat (20 Oct 2017) - test653: check reuse of easy handle after mime data change See issue #1999 - mime: do not reuse previously computed multipart size The contents might have changed: size must be recomputed. Reported-by: moteus on github Fixes #1999 - test308: disable if MultiSSL feature enabled Even if OpenSSL is enabled, it might not be the default backend when multi-ssl is enabled, causing the test to fail. - runtests: support MultiSSL client feature - vtls: change struct Curl_ssl `close' field name to `close_one'. On OS/400, `close' is an ASCII system macro that corrupts the code if not used in a context not targetting the close() system API. - os400: add missing symbols in config file. Also adjust makefile to renamed files and warn about installation dirs mix-up. - test652: curl_mime_data + base64 encoder with large contents - mime: limit bas64-encoded lines length to 76 characters Daniel Stenberg (16 Oct 2017) - RELEASE-NOTES: synced with f121575c0 - setopt: range check most long options ... filter early instead of risking "funny values" having to be dealt with elsewhere. - setopt: avoid integer overflows when setting millsecond values ... that are multiplied by 1000 when stored. For 32 bit long systems, the max value accepted (2147483 seconds) is > 596 hours which is unlikely to ever be set by a legitimate application - and previously it didn't work either, it just caused undefined behavior. Also updated the man pages for these timeout options to mention the return code. Closes #1938 Viktor Szakats (15 Oct 2017) - makefile.m32: allow to override gcc, ar and ranlib Allow to ovverride certain build tools, making it possible to use LLVM/Clang to build curl. The default behavior is unchanged. To build with clang (as offered by MSYS2), these settings can be used: CURL_CC=clang CURL_AR=llvm-ar CURL_RANLIB=llvm-ranlib Closes https://github.com/curl/curl/pull/1993 - ldap: silence clang warning Use memset() to initialize a structure to avoid LLVM/Clang warning: ldap.c:193:39: warning: missing field 'UserLength' initializer [-Wmissing-field-initializers] Closes https://github.com/curl/curl/pull/1992 Daniel Stenberg (14 Oct 2017) - runtests: use valgrind for torture as well NOTE: it makes them terribly slow. I recommend only using valgrind for specific torture tests or using lots of patience. - memdebug: trace send, recv and socket ... to allow them to be included in torture tests too. closes #1980 - configure: remove the C++ compiler check ... we used it only for the fuzzer, which we now have in a separate git repo. Closes #1990 Patrick Monnerat (13 Oct 2017) - mime: do not call failf() if easy handle is NULL. Daniel Stenberg (13 Oct 2017) - test651: curl_formadd with huge COPYCONTENTS - mime: fix the content reader to handle >16K data properly Reported-by: Jeroen Ooms Closes #1988 Patrick Monnerat (12 Oct 2017) - mime: keep "text/plain" content type if user-specified. Include test cases in 554, 587, 650. Fixes https://github.com/curl/curl/issues/1986 - cli tool: use file2memory() to buffer stdin in -F option. Closes PR https://github.com/curl/curl/pull/1985 - cli tool: reimplement stdin buffering in -F option. If stdin is not a regular file, its content is memory-buffered to enable a possible data "rewind". In all cases, stdin data size is determined before real use to avoid having an unknown part's size. --libcurl generated code is left as an unbuffered stdin fread/fseek callback part with unknown data size. Buffering is not supported in deprecated curl_formadd() API. Daniel Stenberg (12 Oct 2017) - winbuild/BUILD.WINDOWS.txt: mention WITH_NGHTTP2 - HELP-US: the label "PR-welcome" is now renamed to "help wanted" following the new github "standard" - RELEASE-NOTES: synced with 5505df7d2 Jay Satiro (11 Oct 2017) - [Artak Galoyan brought this change] url: Update current connection SSL verify params in setopt Now VERIFYHOST, VERIFYPEER and VERIFYSTATUS options change during active connection updates the current connection's (i.e.'connectdata' structure) appropriate ssl_config (and ssl_proxy_config) structures variables, making these options effective for ongoing connection. This functionality was available before and was broken by the following change: "proxy: Support HTTPS proxy and SOCKS+HTTP(s)" CommitId: cb4e2be7c6d42ca0780f8e0a747cecf9ba45f151. Bug: https://github.com/curl/curl/issues/1941 Closes https://github.com/curl/curl/pull/1951 Daniel Stenberg (11 Oct 2017) - [David Benjamin brought this change] openssl: don't use old BORINGSSL_YYYYMM macros Those were temporary things we'd add and remove for our own convenience long ago. The last few stayed around for too long as an oversight but have since been removed. These days we have a running BORINGSSL_API_VERSION counter which is bumped when we find it convenient, but 2015-11-19 was quite some time ago, so just check OPENSSL_IS_BORINGSSL. Closes #1979 - test950; verify SMTP with custom request - ftpserver: support case insensitive commands - smtp_done: free data before returning (on send failure) ... as otherwise it could leak that memory. Detected by OSS-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3600 Assisted-by: Max Dymond Closes #1977 - FTP: URL decode path for dir listing in nocwd mode Reported-by: Zenju on github Test 244 added to verify Fixes #1974 Closes #1976 - test298: verify --ftp-method nowcwd with URL encoded path Ref: #1974 - CURLOPT_XFERINFODATA.3: fix duplicate see also - CURLOPT_NOPROGRESS.3: also refer to xferinfofunction - FAQ: s/CURLOPT_PROGRESSFUNCTION/CURLOPT_XFERINFOFUNCTION - openssl: enable PKCS12 support for !BoringSSL Enable PKCS12 for all non-boringssl builds without relying on configure or cmake checks. Bug: https://curl.haxx.se/mail/lib-2017-10/0007.html Reported-by: Christian Schmitz Closes #1948 - [Kristiyan Tsaklev brought this change] curl: don't pass semicolons when parsing Content-Disposition Test 1422 updated to verify. Closes #1964 Patrick Monnerat (9 Oct 2017) - mime: properly unbind mime structure in curl_mime_free(). This allows freeing a mime structure bound to the easy handle before curl_easy_cleanup(). Fixes #1970. Daniel Stenberg (9 Oct 2017) - RTSP: avoid integer overflow on funny RTSP response ... like a very large non-existing RTSP version number. Added test 577 to verify. Detected by OSS-fuzz. Closes #1969 Patrick Monnerat (8 Oct 2017) - ftpserver: properly reset $ftptargetdir. - test643: verify curl_mime_subparts() rejects cyclic additions. - mime: refuse to add subparts to one of their own descendants. Reported-by: Alexey Melnichuk Fixes #1962 - mime: avoid resetting a part's encoder when part's contents change. - mime: improve unbinding top multipart from easy handle. Also avoid dangling pointers in referencing parts. Daniel Stenberg (8 Oct 2017) - RELEASE-NOTES: synced with a4c1c75da30af1 - curlver.h: next expected release is 7.57.0 Patrick Monnerat (8 Oct 2017) - mime: be tolerant about setting twice the same header list in a part. - docs: clarify form/mime usage of non-regular data files. Daniel Stenberg (8 Oct 2017) - Revert "multi_done: wait for name resolve to finish if still ongoing" This reverts commit f3e03f6c0ac52a1bf396e03f7d7e9b5b3b7165fe. Caused memory leaks in the fuzzer, needs to be done differently. Disable test 1553 for now too, as it causes memory leaks without this commit! - remove_handle: call multi_done() first, then clear dns cache pointer Closes #1960 - multi_done: wait for name resolve to finish if still ongoing ... as we must clean up memory. - pingpong: return error when trying to send without connection When imap_done() got called before a connection is setup, it would try to "finish up" and dereffed a NULL pointer. Test case 1553 managed to reproduce. I had to actually use a host name to try to resolve to slow it down, as using the normal local server IP will make libcurl get a connection in the first curl_multi_perform() loop and then the bug doesn't trigger. Fixes #1953 Assisted-by: Max Dymond Dan Fandrich (6 Oct 2017) - tests: added flaky keyword to tests 587 and 644 These are around 5% flaky in my Linux x86 autobuilds. Marcel Raad (6 Oct 2017) - vtls: fix warnings with --disable-crypto-auth When CURL_DISABLE_CRYPTO_AUTH is defined, Curl_none_md5sum's parameters are not used. Daniel Stenberg (6 Oct 2017) - multi_cleanup: call DONE on handles that never got that ... fixes a memory leak with at least IMAP when remove_handle is never called and the transfer is abruptly just abandoned early. Test 1552 added to verify Detected by OSS-fuzz Assisted-by: Max Dymond Closes #1954 - [Benbuck Nason brought this change] strtoofft: Remove extraneous null check Fixes #1950: curlx_strtoofft() doesn't fully protect against null 'str' argument. Closes #1952 - openssl: fix build without HAVE_OPAQUE_EVP_PKEY Reported-by: Javier Sixto Fixes #1955 Closes #1956 Viktor Szakats (6 Oct 2017) - lib/config-win32.h: let SMB/SMBS be enabled with OpenSSL/NSS The source code is now prepared to handle the case when both Win32 Crypto and OpenSSL/NSS crypto backends are enabled at the same time, making it now possible to enable `USE_WIN32_CRYPTO` whenever the targeted Windows version supports it. Since this matches the minimum Windows version supported by curl (Windows 2000), enable it unconditionally for the Win32 platform. This in turn enables SMB (and SMBS) protocol support whenever Win32 Crypto is available, regardless of what other crypto backends are enabled. Ref: https://github.com/curl/curl/pull/1840#issuecomment-325682052 Closes https://github.com/curl/curl/pull/1943 Daniel Stenberg (5 Oct 2017) - build: fix --disable-crypto-auth Reported-by: Wyatt O'Day Fixes #1945 Closes #1947 Jay Satiro (5 Oct 2017) - [Nick Zitzmann brought this change] darwinssl: add support for TLSv1.3 Closes https://github.com/curl/curl/pull/1794 Daniel Stenberg (4 Oct 2017) - [Felix Kaiser brought this change] docs: fix typo in curl_mime_data_cb man page Closes #1946 Viktor Szakats (4 Oct 2017) - lib/Makefile.m32: allow customizing dll suffixes - New `CURL_DLL_SUFFIX` envvar will add a suffix to the generated libcurl dll name. Useful to add `-x64` to 64-bit builds so that it can live in the same directory as the 32-bit one. By default this is empty. - New `CURL_DLL_A_SUFFIX` envvar to customize the suffix of the generated import library (implib) for libcurl .dll. It defaults to `dll`, and it's useful to modify that to `.dll` to have the standard naming scheme for mingw-built .dlls, i.e. `libcurl.dll.a`. Closes https://github.com/curl/curl/pull/1942 Daniel Stenberg (4 Oct 2017) - [Max Dymond brought this change] fuzzer: move to using external curl-fuzzer Use the external curl-fuzzer repository for fuzzing. Closes #1923 - failf: skip the sprintf() if there are no consumers Closes #1936 - ftp: UBsan fixup 'pointer index expression overflowed' Closes #1939 - RELEASE-PROCEDURE: update the release schedule Version 7.56.0 (4 Oct 2017) Daniel Stenberg (4 Oct 2017) - RELEASE-NOTES: curl 7.56.0 - THANKS: added new 7.56.0 contributors Jay Satiro (4 Oct 2017) - build-openssl.bat: Warn OpenSSL 1.1.0 not yet supported Ref: https://github.com/curl/curl/issues/1002 Michael Kaufmann (3 Oct 2017) - idn: fix source code comment - vtls: compare and clone ssl configs properly Compare these settings in Curl_ssl_config_matches(): - verifystatus (CURLOPT_SSL_VERIFYSTATUS) - random_file (CURLOPT_RANDOM_FILE) - egdsocket (CURLOPT_EGDSOCKET) Also copy the setting "verifystatus" in Curl_clone_primary_ssl_config(), and copy the setting "sessionid" unconditionally. This means that reusing connections that are secured with a client certificate is now possible, and the statement "TLS session resumption is disabled when a client certificate is used" in the old advisory at https://curl.haxx.se/docs/adv_20170419.html is obsolete. Reviewed-by: Daniel Stenberg Closes #1917 - proxy: read the "no_proxy" variable only if necessary Reviewed-by: Daniel Stenberg Closes #1919 Patrick Monnerat (3 Oct 2017) - libcurl-tutorial: add casts in example to avoid compilation warnings. Daniel Stenberg (3 Oct 2017) - examples: bring back curl_formadd-using examples ... now with a -formadd suffix. While the new mime API is introduced in 7.56.0 we must acknowledge that lots of users can't upgrade their curl versions immediately. - test1153: verify quoted double-qoutes in PWD response - FTP: zero terminate the entry path even on bad input ... a single double quote could leave the entry path buffer without a zero terminating byte. CVE-2017-1000254 Test 1152 added to verify. Reported-by: Max Dymond Bug: https://curl.haxx.se/docs/adv_20171004.html Jay Satiro (2 Oct 2017) - [Sergei Nikulov brought this change] cmake: disable tests and man generation if perl/nroff not found Fixes https://github.com/curl/curl/issues/1500 Reported-by: Jay Satiro Fixes https://github.com/curl/curl/pull/1662 Assisted-by: Tom Seddon Assisted-by: dpull@users.noreply.github.com Assisted-by: elelel@users.noreply.github.com Closes https://github.com/curl/curl/pull/1924 Patrick Monnerat (2 Oct 2017) - libcurl-tutorial: fix two typos. - TODO: remove deprecated form API items. - libcurl-tutorial: describe MIME API and deprecate form API. Include a guide to form/mime API conversion. Daniel Stenberg (30 Sep 2017) - cookie: fix memory leak if path was set twice in header ... this will let the second occurance override the first. Added test 1161 to verify. Reported-by: Max Dymond Fixes #1932 Closes #1933 Dan Fandrich (30 Sep 2017) - test650: Use variable replacement to set the host address and port Otherwise, the test fails when the -b test option is used to set a different test port range. - Set and use more necessary options when some protocols are disabled When curl and libcurl are built with some protocols disabled, they stop setting and receiving some options that don't make sense with those protocols. In particular, when HTTP is disabled many options aren't set that are used only by HTTP. However, some options that appear to be HTTP-only are actually used by other protocols as well (some despite having HTTP in the name) and should be set, but weren't. This change now causes some of these options to be set and used for more (or for all) protocols. In particular, this fixes tests 646 through 649 in an HTTP-disabled build, which use the MIME API in the mail protocols. Daniel Stenberg (29 Sep 2017) - test1160: verifies cookie leak for large cookies The fix done in 20ea22ff735 - cookie: fix memory leak on oversized rejection Regression brought by 2bc230de63b Detected by OSS-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3513 Assisted-by: Max Dymond Closes #1930 - [Anders Bakken brought this change] connect: fix race condition with happy eyeballs timeout The timer should be started after conn->connecttime is set. Otherwise the timer could expire without this condition being true: /* should we try another protocol family? */ if(i == 0 && conn->tempaddr[1] == NULL && curlx_tvdiff(now, conn->connecttime) >= HAPPY_EYEBALLS_TIMEOUT) { Ref: #1928 Michael Kaufmann (28 Sep 2017) - docs: link CURLOPT_CONNECTTIMEOUT and CURLOPT_CONNECTTIMEOUT_MS Closes #1922 - docs: clarify the use of environment variables for proxy Closes #1921 - http: add custom empty headers to repeated requests Closes #1920 - reuse_conn: don't copy flags that are known to be equal A connection can only be reused if the flags "conn_to_host" and "conn_to_port" match. Therefore it is not necessary to copy these flags in reuse_conn(). Closes #1918 Daniel Stenberg (27 Sep 2017) - curl.h: include on cygwin too When building with -std=c++14 on cygwin, this header won't be automatically included as it otherwise is. The include decision should ideally be reversed and be avoided where that header file doesn't exist. Reported-by: Ian Fette Fixes #1925 - RELEASE-NOTES: synced with d8ab5dc50 Michael Kaufmann (24 Sep 2017) - tests: adjust .gitignore for new tests Jay Satiro (23 Sep 2017) - ntlm: move NTLM_NEEDS_NSS_INIT define into core NTLM header .. and include the core NTLM header in all NTLM-related source files. Follow up to 6f86022. Since then http_ntlm checks NTLM_NEEDS_NSS_INIT but did not include vtls.h where it was defined. Closes https://github.com/curl/curl/pull/1911 Daniel Stenberg (23 Sep 2017) - file_range: avoid integer overflow when figuring out byte range When trying to bump the value with one and the value is already at max, it causes an integer overflow. Closes #1908 Detected by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3465 Assisted-by: Max Dymond Michael Kaufmann (23 Sep 2017) - tests: fix a compiler warning in test 643 Jay Satiro (23 Sep 2017) - symbols-in-versions: fix CURLSSLSET_NO_BACKENDS entry - Use spaces instead of tabs as the delimiter. Follow up to 7c52b12 which added the entry. The entry had used tabs but the symbol-scan parser doesn't recognize tabs and would fail the symbol. Viktor Szakats (22 Sep 2017) - metalink: fix NSS issue in MultiSSL builds In MultiSSL mode (i.e. when more than one SSL backend is compiled in), we cannot use the compile time flag `USE_NSS` as indicator that the NSS backend is in use. As far as Metalink is concerned, the SSL backend is only used for MD5, SHA-1 and SHA-256 calculations, therefore one of the available SSL backends is selected at compile time, in a strict order of preference. Let's introduce a new `HAVE_NSS_CONTEXT` constant that can be used to determine whether the SSL backend used for Metalink is the NSS backend, and use that to guard the code that wants to de-initialize the NSS-specific data structure. Ref: https://github.com/curl/curl/pull/1848 - ntlm: use strict order for SSL backend #if branches With the recently introduced MultiSSL support multiple SSL backends can be compiled into cURL That means that now the order of the SSL One option would be to use the same SSL backend as was configured via `curl_global_sslset()`, however, NTLMv2 support would appear to be available only with some SSL backends. For example, when eb88d778e (ntlm: Use Windows Crypt API, 2014-12-02) introduced support for NTLMv1 using Windows' Crypt API, it specifically did *not* introduce NTLMv2 support using Crypt API at the same time. So let's select one specific SSL backend for NTLM support when compiled with multiple SSL backends, using a priority order such that we support NTLMv2 even if only one compiled-in SSL backend can be used for that. Ref: https://github.com/curl/curl/pull/1848 Daniel Stenberg (22 Sep 2017) - symbols-in-versions: add CURLSSLSET_NO_BACKENDS ...fixup from b8e0fe19ec - imap: quote atoms properly when escaping characters Updates test 800 to verify Fixes #1902 Closes #1903 - tests: make the imap server not verify user+password ... as the test cases themselves do that and it makes it easier to add crazy test cases. Test 800 updated to use user name + password that need quoting. Test 856 updated to trigger an auth fail differently. Ref: #1902 - vtls: provide curl_global_sslset() even in non-SSL builds ... it just returns error: Bug: https://github.com/curl/curl/commit/1328f69d53f2f2e937696ea954c480412b018451#commitcomment-24470367 Reported-by: Marcel Raad Closes #1906 Patrick Monnerat (22 Sep 2017) - form/mime: field names are not allowed to contain zero-valued bytes. Also suppress length argument of curl_mime_name() (names are always zero-terminated). Daniel Stenberg (21 Sep 2017) - [Dirk Feytons brought this change] openssl: only verify RSA private key if supported In some cases the RSA key does not support verifying it because it's located on a smart card, an engine wants to hide it, ... Check the flags on the key before trying to verify it. OpenSSL does the same thing internally; see ssl/ssl_rsa.c Closes #1904 Marcel Raad (21 Sep 2017) - examples/post-callback: use long for CURLOPT_POSTFIELDSIZE Otherwise, typecheck-gcc.h warns on MinGW-w64. Patrick Monnerat (20 Sep 2017) - mime: rephrase the multipart output state machine (#1898) ... ... in hope coverity will like it much. - mime: fix an explicit null dereference (#1899) Daniel Stenberg (20 Sep 2017) - curl: check fseek() return code and bail on error Detected by coverity. CID 1418137. - smtp: fix memory leak in OOM Regression since ce0881edee Coverity CID 1418139 and CID 1418136 found it, but it was also seen in torture testing. - RELEASE-NOTES: synced with 5fe85587c - [Pavel Pavlov brought this change] cookies: use lock when using CURLINFO_COOKIELIST Closes #1896 - [Max Dymond brought this change] ossfuzz: changes before merging the generated corpora Before merging in the oss-fuzz corpora from Google, there are some changes to the fuzzer. - Add a read corpus script, to display corpus files nicely. - Change the behaviour of the fuzzer so that TLV parse failures all now go down the same execution paths, which should reduce the size of the corpora. - Make unknown TLVs a failure to parse, which should decrease the size of the corpora as well. Closes #1881 - mime:escape_string minor clarification change ... as it also removes a warning with old gcc versions. Bug: https://curl.haxx.se/mail/lib-2017-09/0049.html Reported-by: Ben Greear - [Max Dymond brought this change] ossfuzz: don't write out to stdout Don't make the fuzzer write out to stdout - instead write some of the contents to a memory block so we exercise the data output code but quietly. Closes #1885 - cookies: reject oversized cookies ... instead of truncating them. There's no fixed limit for acceptable cookie names in RFC 6265, but the entire cookie is said to be less than 4096 bytes (section 6.1). This is also what browsers seem to implement. We now allow max 5000 bytes cookie header. Max 4095 bytes length per cookie name and value. Name + value together may not exceed 4096 bytes. Added test 1151 to verify Bug: https://curl.haxx.se/mail/lib-2017-09/0062.html Reported-by: Kevin Smith Closes #1894 - travis: on mac, don't install openssl or libidn - openssl is already installed and causes warnings when trying to install again - libidn isn't used these days, and homebrew doesn't seem to have a libidn2 package to replace with easily Closes #1895 - curl: make str2udouble not return values on error ... previously it would store a return value even when it returned error, which could make the value get used anyway! Reported-by: Brian Carpenter Closes #1893 Jay Satiro (18 Sep 2017) - socks: fix incorrect port number in SOCKS4 error message Prior to this change it appears the SOCKS5 port parsing was erroneously used for the SOCKS4 error message, and as a result an incorrect port would be shown in the error message. Bug: https://github.com/curl/curl/issues/1892 Reported-by: Jackarain@users.noreply.github.com - [Marc Aldorasi brought this change] schannel: Support partial send for when data is too large Schannel can only encrypt a certain amount of data at once. Instead of failing when too much data is to be sent at once, send as much data as we can and let the caller send the remaining data by calling send again. Bug: https://curl.haxx.se/mail/lib-2014-07/0033.html Closes https://github.com/curl/curl/pull/1890 - [David Benjamin brought this change] openssl: add missing includes lib/vtls/openssl.c uses OpenSSL APIs from BUF_MEM and BIO APIs. Include their headers directly rather than relying on other OpenSSL headers including things. Closes https://github.com/curl/curl/pull/1891 Daniel Stenberg (15 Sep 2017) - conversions: fix several compiler warnings - server/getpart: provide dummy function to build conversion enabled - non-ascii: use iconv() with 'char **' argument Bug: https://curl.haxx.se/mail/lib-2017-09/0031.html - escape.c: error: pointer targets differ in signedness - docs: clarify the CURLOPT_INTERLEAVE* options behavior - [Max Dymond brought this change] rtsp: Segfault in rtsp.c when using WRITEDATA If the INTERLEAVEFUNCTION is defined, then use that plus the INTERLEAVEDATA information when writing RTP. Otherwise, use WRITEFUNCTION and WRITEDATA. Fixes #1880 Closes #1884 Marcel Raad (15 Sep 2017) - [Isaac Boukris brought this change] tests: enable gssapi in travis-ci linux build Closes https://github.com/curl/curl/pull/1687 - [Isaac Boukris brought this change] tests: add initial gssapi test using stub implementation The stub implementation is pre-loaded using LD_PRELOAD and emulates common gssapi uses (only builds if curl is initially built with gssapi support). The initial tests are currently disabled for debug builds as LD_PRELOAD is not used then. Ref: https://github.com/curl/curl/pull/1687 Daniel Stenberg (15 Sep 2017) - test1150: verify same host fetch using different ports over proxy Closes #1889 - URL: on connection re-use, still pick the new remote port ... as when a proxy connection is being re-used, it can still get a different remote port. Fixes #1887 Reported-by: Oli Kingshott - RELEASE-NOTES: synced with 87501e57f - code style: remove wrong uses of multiple spaces Closes #1878 - checksrc: detect and warn for multiple spaces - code style: use space after semicolon - checksrc: verify space after semicolons - code style: use spaces around pluses - checksrc: detect and warn for lack of spaces next to plus signs - code style: use spaces around equals signs - checksrc: verify spaces around equals signs ... as the code style mandates. - Curl_checkheaders: make it available for IMAP and SMTP too ... not only HTTP uses this now. Closes #1875 - travis: add build without HTTP/SMTP/IMAP Jay Satiro (10 Sep 2017) - mbedtls: enable CA path processing CA path processing was implemented when mbedtls.c was added to libcurl in fe7590f, but it was never enabled. Bug: https://github.com/curl/curl/issues/1877 Reported-by: SBKarr@users.noreply.github.com Daniel Stenberg (8 Sep 2017) - rtsp: do not call fwrite() with NULL pointer FILE * If the default write callback is used and no destination has been set, a NULL pointer would be passed to fwrite()'s 4th argument. OSS-fuzz bug https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3327 (not publicly open yet) Detected by OSS-fuzz Closes #1874 - configure: use -Wno-varargs on clang 3.9[.X] debug builds ... to avoid a clang bug - [Max Dymond brought this change] ossfuzz: add some more handled CURL options Add support for HEADER, COOKIE, RANGE, CUSTOMREQUEST, MAIL_RECIPIENT, MAIL_FROM and uploading data. - configure: check for C++ compiler after C, to make it non-fatal The tests for object file/executable file extensions are presumably only done for the first of these macros in the configure file. Bug: https://github.com/curl/curl/pull/1851#issuecomment-327597515 Reported-by: Marcel Raad Closes #1873 Patrick Monnerat (7 Sep 2017) - form API: add new test 650. Now that the form API is deprecated and not used anymore in curl tool, a lot of its features left untested. Test 650 attempts to check all these features not tested elsewhere. Jay Satiro (7 Sep 2017) - configure: fix curl_off_t check's include order - Prepend srcdir include path instead of append. Prior to this change it was possible that during the check for the size of curl_off_t the include path of a user's already installed curl could come before the include path of the to-be-built curl, resulting in the system.h of the former being incorrectly included for that check. Closes https://github.com/curl/curl/pull/1870 Daniel Stenberg (7 Sep 2017) - [Jakub Zakrzewski brought this change] KNOWN_BUGS: Remove CMake symbol hiding issue It has already been fixed in 6140dfc - http-proxy: when not doing CONNECT, that phase is done immediately `conn->connect_state` is NULL when doing a regular non-CONNECT request over the proxy and should therefor be considered complete at once. Fixes #1853 Closes #1862 Reported-by: Lawrence Wagerfield - [Johannes Schindelin brought this change] OpenSSL: fix yet another mistake while encapsulating SSL backend data Another mistake in my manual fixups of the largely mechanical search-and-replace ("connssl->" -> "BACKEND->"), just like the previous commit concerning HTTPS proxies (and hence not caught during my earlier testing). Fixes #1855 Closes #1871 Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] OpenSSL: fix erroneous SSL backend encapsulation In d65e6cc4f (vtls: prepare the SSL backends for encapsulated private data, 2017-06-21), this developer prepared for a separation of the private data of the SSL backends from the general connection data. This conversion was partially automated (search-and-replace) and partially manual (e.g. proxy_ssl's backend data). Sadly, there was a crucial error in the manual part, where the wrong handle was used: rather than connecting ssl[sockindex]' BIO to the proxy_ssl[sockindex]', we reconnected proxy_ssl[sockindex]. The reason was an incorrect location to paste "BACKEND->"... d'oh. Reported by Jay Satiro in https://github.com/curl/curl/issues/1855. Signed-off-by: Johannes Schindelin - [Jay Satiro brought this change] vtls: fix memory corruption Ever since 70f1db321 (vtls: encapsulate SSL backend-specific data, 2017-07-28), the code handling HTTPS proxies was broken because the pointer to the SSL backend data was not swapped between conn->ssl[sockindex] and conn->proxy_ssl[sockindex] as intended, but instead set to NULL (causing segmentation faults). [jes: provided the commit message, tested and verified the patch] Signed-off-by: Johannes Schindelin - vtls: switch to CURL_SHA256_DIGEST_LENGTH define ... instead of the prefix-less version since WolfSSL 3.12 now uses an enum with that name that causes build failures for us. Fixes #1865 Closes #1867 Reported-by: Gisle Vanem - travis: add c-ares enabled builds linux + osx Closes #1868 - HISTORY: added some recent items Jay Satiro (6 Sep 2017) - SSL: fix unused parameter warnings Patrick Monnerat (6 Sep 2017) - mime: drop internal FILE * support. - The part kind MIMEKIND_FILE and associated code are suppressed. - Seek data origin offset not used anymore: suppressed. - MIMEKIND_NAMEDFILE renamed MIMEKIND_FILE; associated fields/functions renamed accordingly. - Curl_getformdata() processes stdin via a callback. Daniel Stenberg (6 Sep 2017) - configure: remove --enable-soname-bump and SONAME_BUMP Back in 2008, (and commit 3f3d6ebe665f3) we changed the logic in how we determine the native type for `curl_off_t`. To really make sure we didn't break ABI without bumping SONAME, we introduced logic that attempted to detect that it would use a different size and thus not be compatible. We also provided a manual switch that allowed users to tell configure to bump SONAME by force. Today, we know of no one who ever got a SONAME bump auto-detected and we don't know of anyone who's using the manual bump feature. The auto- detection is also no longer working since we introduced defining curl_off_t in system.h (7.55.0). Finally, this bumping logic is not present in the cmake build. Closes #1861 Jay Satiro (6 Sep 2017) - [Gisle Vanem brought this change] vtls: select ssl backend case-insensitive (follow-up) - Do a case-insensitive comparison of CURL_SSL_BACKEND env as well. - Change Curl_strcasecompare calls to strcasecompare (maps to the former but shorter). Follow-up to c290b8f. Bug: https://github.com/curl/curl/commit/c290b8f#commitcomment-24094313 Co-authored-by: Jay Satiro - openssl: Integrate Peter Wu's SSLKEYLOGFILE implementation This is an adaptation of 2 of Peter Wu's SSLKEYLOGFILE implementations. The first one, written for old OpenSSL versions: https://git.lekensteyn.nl/peter/wireshark-notes/tree/src/sslkeylog.c The second one, written for BoringSSL and new OpenSSL versions: https://github.com/curl/curl/pull/1346 Note the first one is GPL licensed but the author gave permission to waive that license for libcurl. As of right now this feature is disabled by default, and does not have a configure option to enable it. To enable this feature define ENABLE_SSLKEYLOGFILE when building libcurl and set environment variable SSLKEYLOGFILE to a pathname that will receive the keys. And in Wireshark change your preferences to point to that key file: Edit > Preferences > Protocols > SSL > Master-Secret Co-authored-by: Peter Wu Ref: https://github.com/curl/curl/pull/1030 Ref: https://github.com/curl/curl/pull/1346 Closes https://github.com/curl/curl/pull/1866 Patrick Monnerat (5 Sep 2017) - mime: fix a trivial warning. - mime: replace 'struct Curl_mimepart' by 'curl_mimepart' in encoder code. mime_state is now a typedef. - mime: implement encoders. curl_mime_encoder() is operational and documented. curl tool -F option is extended with ";encoder=". curl tool --libcurl option generates calls to curl_mime_encoder(). New encoder tests 648 & 649. Test 1404 extended with an encoder specification. - runtests.pl: support attribute "nonewline" in part verify/upload. - [Daniel Stenberg brought this change] fixup data/test1135 - [Daniel Stenberg brought this change] mime: unified to use the typedef'd mime structs everywhere ... and slightly edited to follow our code style better. - [Daniel Stenberg brought this change] curl.h: use lower case curl_mime* as for all public symbols - [Daniel Stenberg brought this change] docs/curl_mime_*.3: use correct variable types in examples Kamil Dudka (5 Sep 2017) - openssl: use OpenSSL's default ciphers by default Up2date versions of OpenSSL maintain the default reasonably secure without breaking compatibility, so it is better not to override the default by curl. Suggested at https://bugzilla.redhat.com/1483972 Closes #1846 Viktor Szakats (5 Sep 2017) - examples/mime: minor example code fixes Daniel Stenberg (5 Sep 2017) - docs/curl_mime_*.3: added examples - configure: add MultiSSL to FEATURES when enabled ...for curl-config and its corresponding test 1014 - http-proxy: treat all 2xx as CONNECT success Added test 1904 to verify. Reported-by: Lawrence Wagerfield Fixes #1859 Closes #1860 - MAIL-ETIQUETTE: added "1.9 Your emails are public" - curl.h: fix "unused checksrc ignore", remove dangling reference ... to a README file that doesn't exist anymore Viktor Szakats (4 Sep 2017) - docs: Update to secure URL versions - mime: use CURL_ZERO_TERMINATED in examples and some minor whitespace fixes Daniel Stenberg (4 Sep 2017) - schannel: return CURLE_SSL_CACERT on failed verification ... not *CACERT_BADFILE as it isn't really because of a bad file. Bug: https://curl.haxx.se/mail/lib-2017-09/0002.html Closes #1858 - test1135: fixed after bd8070085f9 - examples/post-callback: stop returning one byte at a time ... since people copy and paste code from this example and thus they get an inefficient POST operation without a good reason and sometimes without understanding why. Instead this now returns as much data as possible. - RELEASE-NOTES: fixed the function counter script - curl.h: make the curl_strequal() protos use the same style ... as the other functions. Makes it easier to machine-parse! - docs: curl_mime_*.3 man page formatting edits - RELEASE-NOTES: synced with 1ab9e9b50 Patrick Monnerat (4 Sep 2017) - lib: bump version info (soname). Adapt and reenable test 1135. Daniel Stenberg (3 Sep 2017) - headers: move the global_sslset() proto from multi.h to curl.h As it was added to multi.h simply to not break test 1135, which now has been disabled due to the mime API addition anyway and su we can now move the sslset stuff to where the other curl_global_* prototypes are. Patrick Monnerat (3 Sep 2017) - mime: fix signed/unsigned conversions. Use and generate CURL_ZERO_TERMINATED in curl tool and tests. Jay Satiro (3 Sep 2017) - tool_formparse: fix some trivial warnings Patrick Monnerat (3 Sep 2017) - mime: use size_t instead of ssize_t in public API interface. To support telling a string is nul-terminated, symbol CURL_ZERO_TERMINATED has been introduced. Documentation updated accordingly. symbols in versions updated. Added form API symbols deprecation info. - mime: remove support "-" stdin pseudo-file name in curl_mime_filedata(). This feature is badly supported in Windows: as a replacement, a caller has to use curl_mime_data_cb() with fread, fseek and possibly fclose callbacks to process opened files. The cli tool and documentation are updated accordingly. The feature is however kept internally for form API compatibility, with the known caveats it always had. As a side effect, stdin size is not determined by the cli tool even if possible and this results in a chunked transfer encoding. Test 173 is updated accordingly. - mime: fix some implicit curl_off_t --> size_t conversion warnings. - mime: tests and examples. Additional mime-specific tests. Existing tests updated to reflect small differences (Expect: 100-continue, data size change due to empty lines, etc). Option -F headers= keyword added to tests. test1135 disabled until the entry point order change is resolved. New example smtp-mime. Examples postit2 and multi-post converted from form API to mime API. - mime: use in curl cli tool instead of form API. Extended -F option syntax to support multipart mail messages. -F keyword headers= added to include custom headers in parts. Documentation upgraded. - mime: new MIME API. Available in HTTP, SMTP and IMAP. Deprecates the FORM API. See CURLOPT_MIMEPOST. Lib code and associated documentation. - test564: Add a warning comment about shell profile output. Shell profile output makes the SSH server failing and this problem reason is not easy to find when no hint is given. - checksrc: disable SPACEBEFOREPAREN for case statement. The case keyword may be followed by a constant expression and thus should allow it to start with an open parenthesis. - runtests.pl: allow tags in client section. This enables tests to create more than one file on the client side. - runtests.pl: Apply strippart to upload too. This will allow substitution of boundaries in mail messages. - Curl_base64_encode: always call with a real data handle. Some calls in different modules were setting the data handle to NULL, causing segmentation faults when using builds that enable character code conversions. - non-ascii: allow conversion functions to be called with a NULL data handle. - http: fix a memory leakage in checkrtspprefix(). Daniel Stenberg (2 Sep 2017) - [Max Dymond brought this change] ossfuzz: Move to C++ for curl_fuzzer. Automake gets confused if you want to use C++ static libraries with C code - basically we need to involve the clang++ linker. The easiest way of achieving this is to rename the C code as C++ code. This gets us a bit further along the path and ought to be compatible with Google's version of clang. - curl_global_sslset: select backend by name case insensitively Closes #1849 - [Max Dymond brought this change] ossfuzz: additional seed corpora Create simple seed corpora for: - FTP - telnet - dict - tftp - imap - pop3 based off the tests of the same number. Closes #1842 - [Max Dymond brought this change] ossfuzz: moving towards the ideal integration - Start with the basic code from the ossfuzz project. - Rewrite fuzz corpora to be binary files full of Type-Length-Value data, and write a glue layer in the fuzzing function to convert corpora into CURL options. - Have supporting functions to generate corpora from existing tests - Integrate with Makefile.am - strcase: corrected comment header for Curl_strcasecompare() - unit1301: fix error message on first test - curl_global_sslset.3: show the struct and enum too ... so that users can actually write code based on the man page alone, not having to read the header file. Jay Satiro (31 Aug 2017) - darwinssl: handle long strings in TLS certs (follow-up) - Fix handling certificate subjects that are already UTF-8 encoded. Follow-up to b3b75d1 from two days ago. Since then a copy would be skipped if the subject was already UTF-8, possibly resulting in a NULL deref later on. Ref: https://github.com/curl/curl/issues/1823 Ref: https://github.com/curl/curl/pull/1831 Closes https://github.com/curl/curl/pull/1836 Daniel Stenberg (31 Aug 2017) - cyassl: call it the "WolfSSL" backend ... instead of cyassl, as this is the current name for it. Closes #1844 - polarssl: fix multissl breakage Reported-by: Dan Fandrich Bug: https://curl.haxx.se/mail/lib-2017-08/0121.html Closes #1843 - configure: remove the leading comma from the backends list ... when darwinssl is used. Reported-by: Viktor Szakats Bug: https://github.com/curl/curl/commit/b0989cd3abaff4f9a0717b4875022fa79e33b481#commitcomment-23943493 Closes #1845 Kamil Dudka (30 Aug 2017) - examples/sslbackend.c: fix failure of 'make checksrc' ./sslbackend.c:58:3: warning: else after closing brace on same line (BRACEELSE) } else if(isdigit(*name)) { ^ ./sslbackend.c:62:3: warning: else after closing brace on same line (BRACEELSE) } else ^ Viktor Szakats (30 Aug 2017) - makefile.m32: add multissl support Closes https://github.com/curl/curl/pull/1840 Daniel Stenberg (30 Aug 2017) - curl.h: CURLSSLBACKEND_WOLFSSL used wrong value The CURLSSLBACKEND_WOLFSSL is supposed to be an alias for CURLSSLBACKEND_CYASSL, but used an erronous value. To reduce the risk for a similar mistake, define the backend aliases to use the enum values instead. Reported-by: Gisle Vanem Bug: https://curl.haxx.se/mail/lib-2017-08/0120.html - curl_global_sslset.3: clarify it is a one time *set*, not necessarily a one time use... it can be called again if the first call failed or just listed the alternatives. clarify that the available backends are the ones this build supports plus add some formatting Reported-by: Rich Gray Bug: https://curl.haxx.se/mail/lib-2017-08/0119.html - curl/multi.h: remove duplicated closing c++ brace Regression since 1328f69d53f2f2e93 Fixes #1841 Reported-by: Andrei Karas - RELEASE-NOTES: synced with 8c33c963a - HELP-US.md: spelling - HELP-US.md: "How to get started helping out in the curl project" Closes #1837 Dan Fandrich (29 Aug 2017) - asyn-thread: Fixed cleanup after OOM destroy_async_data() assumes that if the flag "done" is not set yet, the thread itself will clean up once the request is complete. But if an error (generally OOM) occurs before the thread even has a chance to start, it will never get a chance to clean up and memory will be leaked. By clearing "done" only just before starting the thread, the correct cleanup sequence will happen in all cases. Daniel Stenberg (28 Aug 2017) - curl_global_init.3: mention curl_global_sslset(3) Dan Fandrich (28 Aug 2017) - unit1606: Fixed shadowed variable warning - asyn-thread: Improved cleanup after OOM situations - asyn-thread: Set errno to the proper value ENOMEM in OOM situation This used to be set in some configurations to EAI_MEMORY which is not a valid value for errno and caused Curl_strerror to fail an assertion. Daniel Stenberg (28 Aug 2017) - [Johannes Schindelin brought this change] configure: Handle "MultiSSL" specially When versioning symbols There is a mode in which libcurl is compiled with versioned symbols, depending on the active SSL backend. When multiple SSL backends are active, it does not make sense to favor one over the others, so let's not: introduce a new prefix for the case where multiple SSL backends are compiled into cURL. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] configure: allow setting the default SSL backend Previously, we used as default SSL backend whatever was first in the `available_backends` array. However, some users may want to override that default without patching the source code. Now they can: with the --with-default-ssl-backend= option of the ./configure script. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: use Curl_ssl_multi pseudo backend only when needed When only one SSL backend is configured, it is totally unnecessary to let multissl_init() configure the backend at runtime, we can select the correct backend at build time already. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] version: if built with more than one SSL backend, report all of them To discern the active one from the inactive ones, put the latter into parentheses. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] version: add the CURL_VERSION_MULTI_SSL feature flag This new feature flag reports When cURL was built with multiple SSL backends. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] metalink: allow compiling with multiple SSL backends Previously, the code assumed that at most one of the SSL backends would be compiled in, emulating OpenSSL's functions if the configured backend was not OpenSSL itself. However, now we allow building with multiple SSL backends and choosing one at runtime. Therefore, metalink needs to be adjusted to handle this scenario, too. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] docs/examples: demonstrate how to select SSL backends The newly-introduced curl_global_sslset() function deserves to be show-cased. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] Add a man page for curl_global_sslset() Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: introduce curl_global_sslset() Let's add a compile time safe API to select an SSL backend. This function needs to be called *before* curl_global_init(), and can be called only once. Side note: we do not explicitly test that it is called before curl_global_init(), but we do verify that it is not called multiple times (even implicitly). If SSL is used before the function was called, it will use whatever the CURL_SSL_BACKEND environment variable says (or default to the first available SSL backend), and if a subsequent call to curl_global_sslset() disagrees with the previous choice, it will fail with CURLSSLSET_TOO_LATE. The function also accepts an "avail" parameter to point to a (read-only) NULL-terminated list of available backends. This comes in real handy if an application wants to let the user choose between whatever SSL backends the currently available libcurl has to offer: simply call curl_global_sslset(-1, NULL, &avail); which will return CURLSSLSET_UNKNOWN_BACKEND and populate the avail variable to point to the relevant information to present to the user. Just like with the HTTP/2 push functions, we have to add the function declaration of curl_global_sslset() function to the header file *multi.h* because VMS and OS/400 require a stable order of functions declared in include/curl/*.h (where the header files are sorted alphabetically). This looks a bit funny, but it cannot be helped. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: refactor out essential information about the SSL backends There is information about the compiled-in SSL backends that is really no concern of any code other than the SSL backend itself, such as which function (if any) implements SHA-256 summing. And there is information that is really interesting to the user, such as the name, or the curl_sslbackend value. Let's factor out the latter into a publicly visible struct. This information will be used in the upcoming API to set the SSL backend globally. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: allow selecting which SSL backend to use at runtime When building software for the masses, it is sometimes not possible to decide for all users which SSL backend is appropriate. Git for Windows, for example, uses cURL to perform clones, fetches and pushes via HTTPS, and some users strongly prefer OpenSSL, while other users really need to use Secure Channel because it offers enterprise-ready tools to manage credentials via Windows' Credential Store. The current Git for Windows versions use the ugly work-around of building libcurl once with OpenSSL support and once with Secure Channel support, and switching out the binaries in the installer depending on the user's choice. Needless to say, this is a super ugly workaround that actually only works in some cases: Git for Windows also comes in a portable form, and in a form intended for third-party applications requiring Git functionality, in which cases this "swap out libcurl-4.dll" simply is not an option. Therefore, the Git for Windows project has a vested interest in teaching cURL to make the SSL backend a *runtime* option. This patch makes that possible. By running ./configure with multiple --with- options, cURL will be built with multiple backends. For the moment, the backend can be configured using the environment variable CURL_SSL_BACKEND (valid values are e.g. "openssl" and "schannel"). Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: fold the backend ID into the Curl_ssl structure Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] curl_ntlm_core: don't complain but #include OpenSSL header if needed Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: encapsulate SSL backend-specific data So far, all of the SSL backends' private data has been declared as part of the ssl_connect_data struct, in one big #if .. #elif .. #endif block. This can only work as long as the SSL backend is a compile-time option, something we want to change in the next commits. Therefore, let's encapsulate the exact data needed by each SSL backend into a private struct, and let's avoid bleeding any SSL backend-specific information into urldata.h. This is also necessary to allow multiple SSL backends to be compiled in at the same time, as e.g. OpenSSL's and CyaSSL's headers cannot be included in the same .c file. To avoid too many malloc() calls, we simply append the private structs to the connectdata struct in allocate_conn(). This requires us to take extra care of alignment issues: struct fields often need to be aligned on certain boundaries e.g. 32-bit values need to be stored at addresses that divide evenly by 4 (= 32 bit / 8 bit-per-byte). We do that by assuming that no SSL backend's private data contains any fields that need to be aligned on boundaries larger than `long long` (typically 64-bit) would need. Under this assumption, we simply add a dummy field of type `long long` to the `struct connectdata` struct. This field will never be accessed but acts as a placeholder for the four instances of ssl_backend_data instead. the size of each ssl_backend_data struct is stored in the SSL backend-specific metadata, to allow allocate_conn() to know how much extra space to allocate, and how to initialize the ssl[sockindex]->backend and proxy_ssl[sockindex]->backend pointers. This would appear to be a little complicated at first, but is really necessary to encapsulate the private data of each SSL backend correctly. And we need to encapsulate thusly if we ever want to allow selecting CyaSSL and OpenSSL at runtime, as their headers cannot be included within the same .c file (there are just too many conflicting definitions and declarations for that). Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: prepare the SSL backends for encapsulated private data At the moment, cURL's SSL backend needs to be configured at build time. As such, it is totally okay for them to hard-code their backend-specific data in the ssl_connect_data struct. In preparation for making the SSL backend a runtime option, let's make the access of said private data a bit more abstract so that it can be adjusted later in an easy manner. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] urldata.h: move SSPI-specific #include to correct location In 86b889485 (sasl_gssapi: Added GSS-API based Kerberos V5 variables, 2014-12-03), an SSPI-specific field was added to the kerberos5data struct without moving the #include "curl_sspi.h" later in the same file. This broke the build when SSPI was enabled, unless Secure Channel was used as SSL backend, because it just so happens that Secure Channel also requires "curl_sspi.h" to be #included. In f4739f639 (urldata: include curl_sspi.h when Windows SSPI is enabled, 2017-02-21), this bug was fixed incorrectly: Instead of moving the appropriate conditional #include, the Secure Channel-conditional part was now also SSPI-conditional. Fix this problem by moving the correct #include instead. This is also required for an upcoming patch that moves all the Secure Channel-specific stuff out of urldata.h and encapsulates it properly in vtls/schannel.c instead. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] urldata.h: remove support for obsolete PolarSSL version Since 5017d5ada (polarssl: now require 1.3.0+, 2014-03-17), we require a newer PolarSSL version. No need to keep code trying to support any older version. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] getinfo: access SSL internals via Curl_ssl In the ongoing endeavor to abstract out all SSL backend-specific functionality, this is the next step: Instead of hard-coding how the different SSL backends access their internal data in getinfo.c, let's implement backend-specific functions to do that task. This will also allow for switching SSL backends as a runtime option. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: move SSL backends' private constants out of their header files Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] axtls: use Curl_none_* versions of init() and cleanup() There are convenient no-op versions of the init/cleanup functions now, no need to define private ones for axTLS. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: remove obsolete declarations of SSL backend functionality These functions are all available via the Curl_ssl struct now, no need to declare them separately anymore. As the global declarations are removed, the corresponding function definitions are marked as file-local. The only two exceptions here are Curl_mbedtls_shutdown() and Curl_polarssl_shutdown(): only the declarations were removed, there are no function definitions to mark file-local. Please note that Curl_nss_force_init() is *still* declared globally, as the only SSL backend-specific function, because it was introduced specifically for the use case where cURL was compiled with `--without-ssl --with-nss`. For details, see f3b77e561 (http_ntlm: add support for NSS, 2010-06-27). Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] schannel: reorder functions topologically The _shutdown() function calls the _session_free() function; While this is not a problem now (because schannel.h declares both functions), a patch looming in the immediate future with make all of these functions file-local. So let's just move the _session_free() function's definition before it is called. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] axtls: reorder functions topologically The connect_finish() function (like many other functions after it) calls the Curl_axtls_close() function; While this is not a problem now (because axtls.h declares the latter function), a patch looming in the immediate future with make all of these functions file-local. So let's just move the Curl_axtls_close() function's definition before it is called. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: move the SUPPORT_HTTPS_PROXY flag into the Curl_ssl struct That will allow us to choose the SSL backend at runtime. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: convert the have_curlssl_* constants to runtime flags The entire idea of introducing the Curl_ssl struct to describe SSL backends is to prepare for choosing the SSL backend at runtime. To that end, convert all the #ifdef have_curlssl_* style conditionals to use bit flags instead. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: move sha256sum into the Curl_ssl struct The SHA-256 checksumming is also an SSL backend-specific function. Let's include it in the struct declaring the functionality of SSL backends. In contrast to MD5, there is no fall-back code. To indicate this, the respective entries are NULL for those backends that offer no support for SHA-256 checksumming. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: move md5sum into the Curl_ssl struct The MD5 summing is also an SSL backend-specific function. So let's include it, offering the previous fall-back code as a separate function now: Curl_none_md5sum(). To allow for that, the signature had to be changed so that an error could be returned from the implementation (Curl_none_md5sum() can run out of memory). Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: use the Curl_ssl struct to access all SSL backends' functionality This is the first step to unify the SSL backend handling. Now all the SSL backend-specific functionality is accessed via a global instance of the Curl_ssl struct. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: declare Curl_ssl structs for every SSL backend The idea of introducing the Curl_ssl struct was to unify how the SSL backends are declared and called. To this end, we now provide an instance of the Curl_ssl struct for each and every SSL backend. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: introduce a new struct for SSL backends This new struct is similar in nature to Curl_handler: it will define the functions and capabilities of all the SSL backends (where Curl_handler defines the functions and capabilities of protocol handlers). Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: make sure every _sha256sum()'s first arg is const This patch makes the signature of the _sha256sum() functions consistent among the SSL backends, in preparation for unifying the way all SSL backends are accessed. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: make sure all _data_pending() functions return bool This patch makes the signature of the _data_pending() functions consistent among the SSL backends, in preparation for unifying the way all SSL backends are accessed. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: make sure all _cleanup() functions return void This patch makes the signature of the _cleanup() functions consistent among the SSL backends, in preparation for unifying the way all SSL backends are accessed. Signed-off-by: Johannes Schindelin - [Johannes Schindelin brought this change] vtls: use consistent signature for _random() implementations This will make the upcoming multissl backend much easier to implement. Signed-off-by: Johannes Schindelin - strtooff: fix build for systems with long long but no strtoll option Closes #1829 Reported-by: Dan Fandrich Bug: https://github.com/curl/curl/pull/1758#issuecomment-324861615 - darwinssl: handle long strings in TLS certs ... as the previous fixed length 128 bytes buffer was sometimes too small. Fixes #1823 Closes #1831 Reported-by: Benjamin Sergeant Assisted-by: Bill Pyne, Ray Satiro, Nick Zitzmann - system.h: include sys/poll.h for AIX ... to get the event/revent defines that might be used for the poll struct. Reported-by: Michael Smith Fixes #1828 Closes #1833 Dan Fandrich (26 Aug 2017) - tests: Make sure libtests & unittests call curl_global_cleanup() These were missed in commit c468c27b. Jay Satiro (26 Aug 2017) - [theantigod brought this change] winbuild: fix embedded manifest option Embedded manifest option didn't work due to incorrect path. Fixes https://github.com/curl/curl/issues/1832 Daniel Stenberg (25 Aug 2017) - fuzz/Makefile.am: remove curlbuild.h leftovers - examples/threaded-ssl: mention that this is for openssl before 1.1 - imap: use defined names for response codes When working on this code I found the previous setup a bit weird while using proper defines increases readability. Closes #1824 - CURLOPT_USERPWD.3: see also CURLOPT_PROXYUSERPWD - imap: support PREAUTH It is a defined possible greeting at server startup that means the connection is already authenticated. See https://tools.ietf.org/html/rfc3501#section-7.1.4 Test 846 added to verify. Fixes #1818 Closes #1820 Jay Satiro (23 Aug 2017) - config-tpf: define SIZEOF_LONG Recent changes that replaced CURL_SIZEOF_LONG in the source with SIZEOF_LONG broke builds that use the premade configuration files and don't have SIZEOF_LONG defined. Bug: https://github.com/curl/curl/issues/1816 Dan Fandrich (23 Aug 2017) - test1453: Fixed Daniel Stenberg (22 Aug 2017) - [Gisle Vanem brought this change] config-dos: add missing defines, SIZEOF_* and two others Bug: #1816 - curl: shorten and clean up CA cert verification error message The previous message was just too long for ordinary people and it was encouraging users to use `--insecure` a little too easy. Based-on-work-by: Frank Denis Closes #1810 Closes #1817 - request-target.d: mention added in 7.55.0 Marcel Raad (22 Aug 2017) - tool_main: turn off MinGW CRT's globbing By default, the MinGW CRT globs command-line arguments. This prevents getting a single asterisk into an argument as test 1299 does. Turn off globbing by setting the global variable _CRT_glob to 0 for MinGW. Fixes https://github.com/curl/curl/issues/1751 Closes https://github.com/curl/curl/pull/1813 Viktor Szakats (22 Aug 2017) - makefile.m32: add support for libidn2 libidn was replaced with libidn2 last year in configure. Caveat: libidn2 may depend on a list of further libs. These can be manually specified via CURL_LDFLAG_EXTRAS. Closes https://github.com/curl/curl/pull/1815 Jay Satiro (22 Aug 2017) - [Viktor Szakats brought this change] config-win32: define SIZEOF_LONG Recent changes that replaced CURL_SIZEOF_LONG in the source with SIZEOF_LONG broke builds that use the premade configuration files and don't have SIZEOF_LONG defined. Closes https://github.com/curl/curl/pull/1814 Daniel Stenberg (20 Aug 2017) - cmake: enable picky compiler options with clang and gcc closes #1799 - curl/system.h: fix build for hppa Reported-by: John David Anglin Bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=872502#10 - [Even Rouault brought this change] tftp: fix memory leak on too long filename Fixes $ valgrind --leak-check=full ~/install-curl-git/bin/curl tftp://localhost/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaz ==9752== Memcheck, a memory error detector ==9752== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. ==9752== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==9752== Command: /home/even/install-curl-git/bin/curl tftp://localhost/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaz ==9752== curl: (71) TFTP file name too long ==9752== ==9752== HEAP SUMMARY: ==9752== 505 bytes in 1 blocks are definitely lost in loss record 11 of 11 ==9752== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==9752== by 0x4E61CED: Curl_urldecode (in /home/even/install-curl-git/lib/libcurl.so.4.4.0) ==9752== by 0x4E75868: tftp_state_machine (in /home/even/install-curl-git/lib/libcurl.so.4.4.0) ==9752== by 0x4E761B6: tftp_do (in /home/even/install-curl-git/lib/libcurl.so.4.4.0) ==9752== by 0x4E711B6: multi_runsingle (in /home/even/install-curl-git/lib/libcurl.so.4.4.0) ==9752== by 0x4E71D00: curl_multi_perform (in /home/even/install-curl-git/lib/libcurl.so.4.4.0) ==9752== by 0x4E6950D: curl_easy_perform (in /home/even/install-curl-git/lib/libcurl.so.4.4.0) ==9752== by 0x40E0B7: operate_do (in /home/even/install-curl-git/bin/curl) ==9752== by 0x40E849: operate (in /home/even/install-curl-git/bin/curl) ==9752== by 0x402693: main (in /home/even/install-curl-git/bin/curl) Fixes https://oss-fuzz.com/v2/testcase-detail/5232311106797568 Credit to OSS Fuzz Closes #1808 Dan Fandrich (19 Aug 2017) - runtests: fixed case insensitive matching of keywords Commit 5c2aac71 didn't work in the case of mixed-case keywords given on the command-line. - tests: Make sure libtests call curl_global_cleanup() This ensures that global data allocations are freed so Valgrind stays happy. This was a problem with at least PolarSSL and mbedTLS. Daniel Stenberg (18 Aug 2017) - RELEASE-NOTES: synced with 8baead425 - scripts/contri*sh: use "git log --use-mailmap" - mailmap: de-duplify some git authors - http2_recv: return error better on fatal h2 errors Ref #1012 Figured-out-by: Tatsuhiro Tsujikawa - KNOWN_BUGS: HTTP test server 'connection-monitor' problems Closes #868 - curl/system.h: check for __ppc__ as well ... regression since issue #1774 (commit 10b3df10596a) since obviously some older gcc doesn't know __powerpc__ while some newer doesn't know __ppc__ ... Fixes #1797 Closes #1798 Reported-by: Ryan Schmidt - [Jan Alexander Steffens (heftig) brought this change] http: Don't wait on CONNECT when there is no proxy Since curl 7.55.0, NetworkManager almost always failed its connectivity check by timeout. I bisected this to 5113ad04 (http-proxy: do the HTTP CONNECT process entirely non-blocking). This patch replaces !Curl_connect_complete with Curl_connect_ongoing, which returns false if the CONNECT state was left uninitialized and lets the connection continue. Closes #1803 Fixes #1804 Also-fixed-by: Gergely Nagy - [Johannes Schindelin brought this change] metalink: adjust source code style Signed-off-by: Johannes Schindelin - CURL_SIZEOF_LONG: removed, use only SIZEOF_LONG - lib557: no longer use CURL_SIZEOF_* defines - config-win32: define SIZEOF_CURL_OFF_T - cmake: sizeof curl_off_t, remove unused detections - system.h: remove all CURL_SIZEOF_* defines ... as they're not used externally and internally we check for the sizes already in configure etc. Closes #1767 - ftp: fix CWD when doing multicwd then nocwd on same connection Fixes #1782 Closes #1787 Reported-by: Peter Lamare - CURLOPT_SSH_COMPRESSION.3: enable with 1L (leaves other values reserved for the future) - compressed-ssh.d: "Added: 7.56.0" - curl/system.h: checksrc compliance Jay Satiro (17 Aug 2017) - [Viktor Szakats brought this change] ssh: add the ability to enable compression (for SCP/SFTP) The required low-level logic was already available as part of `libssh2` (via `LIBSSH2_FLAG_COMPRESS` `libssh2_session_flag()`[1] option.) This patch adds the new `libcurl` option `CURLOPT_SSH_COMPRESSION` (boolean) and the new `curl` command-line option `--compressed-ssh` to request this `libssh2` feature. To have compression enabled, it is required that the SSH server supports a (zlib) compatible compression method and that `libssh2` was built with `zlib` support enabled. [1] https://www.libssh2.org/libssh2_session_flag.html Ref: https://github.com/curl/curl/issues/1732 Closes https://github.com/curl/curl/pull/1735 - examples/ftpuploadresume: checksrc compliance - [Maksim Stsepanenka brought this change] http_proxy: fix build error for CURL_DOES_CONVERSIONS Closes https://github.com/curl/curl/pull/1793 GitHub (16 Aug 2017) - [Nick Zitzmann brought this change] configure: check for __builtin_available() availability (#1788) This change does two things: 1. It un-breaks the build in Xcode 9.0. (Xcode 9.0 is currently failing trying to compile connectx() in lib/connect.c.) 2. It finally weak-links the connectx() function, and falls back on connect() when run on older operating systems. Daniel Stenberg (16 Aug 2017) - travis: add metalink to some osx builds Closes #1790 - [Max Dymond brought this change] coverage: Use two coveralls commands to get lib/vtls results closes #1747 - darwinssi: fix error: variable length array used - m4/curl-compilers.m4: use proper quotes around string, not backticks ... when setting clang version to assume 3.7 Caused a lot of "integer expression expected" warnings by configure. - [Benbuck Nason brought this change] cmake: remove dead code for DISABLED_THREADSAFE Closes #1786 Jay Satiro (15 Aug 2017) - [Jakub Zakrzewski brought this change] curl-confopts.m4: fix --disable-threaded-resolver Closes https://github.com/curl/curl/issues/1784 Daniel Stenberg (15 Aug 2017) - [Ryan Winograd brought this change] progress: Track total times following redirects Update the progress timers `t_nslookup`, `t_connect`, `t_appconnect`, `t_pretransfer`, and `t_starttransfer` to track the total times for these activities when a redirect is followed. Previously, only the times for the most recent request would be tracked. Related changes: - Rename `Curl_pgrsResetTimesSizes` to `Curl_pgrsResetTransferSizes` now that the function only resets transfer sizes and no longer modifies any of the progress timers. - Add a bool to the `Progress` struct that is used to prevent double-counting `t_starttransfer` times. Added test case 1399. Fixes #522 and Known Bug 1.8 Closes #1602 Reported-by: joshhe on github - [Benbuck Nason brought this change] cmake: remove dead code for CURL_DISABLE_RTMP Closes #1785 Kamil Dudka (15 Aug 2017) - zsh.pl: produce a working completion script again Commit curl-7_54_0-118-g8b2f22e changed the output format of curl --help to use and instead of FILE and DIR, which caused zsh.pl to produce a broken completion script: % curl -- _curl:10: no such file or directory: seconds Closes #1779 Daniel Stenberg (15 Aug 2017) - curlver: toward 7.56.0? - RELEASE-NOTES: synced with 91c46dc44 - test1449: FTP download range with an too large size - strtoofft: reduce integer overflow risks globally ... make sure we bail out on overflows. Reported-by: Brian Carpenter Closes #1758 - travis: build the examples too to make sure they keep building warning-free Closes #1777 - runtests: match keywords case insensitively - examples/ftpuploadresume.c: use portable code ... converted from the MS specific _snscanf() Version 7.55.1 (13 Aug 2017) Daniel Stenberg (13 Aug 2017) - RELEASE-NOTES/THANKS: curl 7.55.1 release time - gitignore: ignore .xz now instead of .lzma - [Sergei Nikulov brought this change] cmake: Threads detection update. ref: #1702 Closes #1719 - ipv6_scope: support unique local addresses Fixes #1764 Closes #1773 Reported-by: James Slaughter - [Alex Potapenko brought this change] curl/system.h: GCC doesn't define __ppc__ on PowerPC, uses __powerpc__ Closes #1774 - test1448: verify redirect to IDN using URL Closes #1772 - [Salah-Eddin Shaban brought this change] redirect: skip URL encoding for host names This fixes redirects to IDN URLs Fixes #1441 Closes #1762 Reported by: David Lord - test2032: mark as flaky (again) - travis: test cmake build on tarball too Could've prevented #1755 - [Simon Warta brought this change] cmake: allow user to override CMAKE_DEBUG_POSTFIX Closes #1763 - connect-to.d: better language - connect-to.d: clarified - bagder/Curl_tvdiff_us: fix the math Regression since adef394ac5 (released in 7.55.0) Reported-by: Han Qiao Fixes #1769 Closes #1771 - curl/system.h: add Oracle Solaris Studio Fixes #1752 - [Alessandro Ghedini brought this change] docs: fix typo funtion -> function Closes #1770 Alessandro Ghedini (12 Aug 2017) - docs: fix grammar in CURL_SSLVERSION_MAX_DEFAULT description - docs: fix typo stuct -> struct Dan Fandrich (12 Aug 2017) - test1447: require a curl with http support Daniel Stenberg (11 Aug 2017) - [Thomas Petazzoni brought this change] curl/system.h: support more architectures The long list of architectures in include/curl/system.h is annoying to maintain, and needs to be extended for each and every architecture to support. Instead, let's rely on the __SIZEOF_LONG__ define of the gcc compiler (we are in the GNUC condition anyway), which tells us if long is 4 bytes or 8 bytes. This fixes the build of libcurl 7.55.0 on architectures such as OpenRISC or ARC. Closes #1766 Signed-off-by: Thomas Petazzoni - test2033: this went flaky again Suspicion: when we enabled the threaded resolver by default. - test1447: verifies the parse proxy fix in 6e0e152ce5c - [Even Rouault brought this change] parse_proxy(): fix memory leak in case of invalid proxy server name Fixes the below leak: $ valgrind --leak-check=full ~/install-curl-git/bin/curl --proxy "http://a:b@/x" http://127.0.0.1 curl: (5) Couldn't resolve proxy name ==5048== ==5048== HEAP SUMMARY: ==5048== in use at exit: 532 bytes in 12 blocks ==5048== total heap usage: 5,288 allocs, 5,276 frees, 445,271 bytes allocated ==5048== ==5048== 2 bytes in 1 blocks are definitely lost in loss record 1 of 12 ==5048== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==5048== by 0x4E6CB79: parse_login_details (url.c:5614) ==5048== by 0x4E6BA82: parse_proxy (url.c:5091) ==5048== by 0x4E6C46D: create_conn_helper_init_proxy (url.c:5346) ==5048== by 0x4E6EA18: create_conn (url.c:6498) ==5048== by 0x4E6F9B4: Curl_connect (url.c:6967) ==5048== by 0x4E86D05: multi_runsingle (multi.c:1436) ==5048== by 0x4E88432: curl_multi_perform (multi.c:2160) ==5048== by 0x4E7C515: easy_transfer (easy.c:708) ==5048== by 0x4E7C74A: easy_perform (easy.c:794) ==5048== by 0x4E7C7B1: curl_easy_perform (easy.c:813) ==5048== by 0x414025: operate_do (tool_operate.c:1563) ==5048== ==5048== 2 bytes in 1 blocks are definitely lost in loss record 2 of 12 ==5048== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==5048== by 0x4E6CBB6: parse_login_details (url.c:5621) ==5048== by 0x4E6BA82: parse_proxy (url.c:5091) ==5048== by 0x4E6C46D: create_conn_helper_init_proxy (url.c:5346) ==5048== by 0x4E6EA18: create_conn (url.c:6498) ==5048== by 0x4E6F9B4: Curl_connect (url.c:6967) ==5048== by 0x4E86D05: multi_runsingle (multi.c:1436) ==5048== by 0x4E88432: curl_multi_perform (multi.c:2160) ==5048== by 0x4E7C515: easy_transfer (easy.c:708) ==5048== by 0x4E7C74A: easy_perform (easy.c:794) ==5048== by 0x4E7C7B1: curl_easy_perform (easy.c:813) ==5048== by 0x414025: operate_do (tool_operate.c:1563) Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2984 Credit to OSS Fuzz for discovery Closes #1761 - RELEASE-NOTES: synced with 37f2195a9 - curlver: bump to 7.55.1 - openssl: fix "error: this statement may fall through" A gcc7 warning. - [David Benjamin brought this change] openssl: remove CONST_ASN1_BIT_STRING. Just making the pointer as const works for the pre-1.1.0 path too. Closes #1759 - maketgz: remove old *.dist files before making the tarball To avoid "old crap" unintentionally getting shipped. Bug: https://curl.haxx.se/mail/lib-2017-08/0050.html Reported-by: Christian Weisgerber Jay Satiro (10 Aug 2017) - mkhelp.pl: allow executing this script directly - Enable execute permission (chmod +x) - Change interpreter to /usr/bin/env perl Ref: https://github.com/curl/curl/issues/1743 Daniel Stenberg (10 Aug 2017) - configure: use the threaded resolver backend by default if possible Closes #1647 - cmake: move cmake_uninstall.cmake to CMake/ Closes #1756 - metalink: fix error: ‘*’ in boolean context, suggest ‘&&’ instead - dist: fix the cmake build by shipping cmake_uninstall.cmake.in too Fixes #1755 - travis: verify "make install" Help-by: Jay Satiro Closes #1753 Marcel Raad (10 Aug 2017) - build: check out *.sln files with Windows line endings Visual Studio doesn't like LF line endings in solution files and always converts them to CRLF when doing changes to the solution. Notably, this affects the solutions in the release archive. Closes https://github.com/curl/curl/pull/1746 - gitignore: ignore top-level .vs folder This folder is generated when using the CMake build system from within Visual Studio. Closes https://github.com/curl/curl/pull/1746 Jay Satiro (10 Aug 2017) - digest_sspi: Don't reuse context if the user/passwd has changed Bug: https://github.com/curl/curl/issues/1685 Reported-by: paulharris@users.noreply.github.com Assisted-by: Isaac Boukris Closes https://github.com/curl/curl/pull/1742 Daniel Stenberg (9 Aug 2017) - [Adam Sampson brought this change] dist: Add dictserver.py/negtelnetserver.py to EXTRA_DIST These weren't included in the 7.55.0 release, but are required in order to run the full test suite. Closes #1744 - [Adam Sampson brought this change] curl: do bounds check using a double comparison The fix for this in 8661a0aacc01492e0436275ff36a21734f2541bb wasn't complete: if the parsed number in num is larger than will fit in a long, the conversion is undefined behaviour (causing test1427 to fail for me on IA32 with GCC 7.1, although it passes on AMD64 and ARMv7). Getting rid of the cast means the comparison will be done using doubles. It might make more sense for the max argument to also be a double... Fixes #1750 Closes #1749 - make install: add 8 missing man pages to the installation - build: fix 'make install' with configure, install docs/libcurl/* too Broken since d24838d4da9faa Reported-by: Bernard Spil Version 7.55.0 (9 Aug 2017) Daniel Stenberg (9 Aug 2017) - RELEASE-NOTES: curl 7.55.0 - THANKS: 20 new contributors in 7.55.0 - [Viktor Szakats brought this change] docs/comments: Update to secure URL versions Closes #1741 - configure: fix recv/send/select detection on Android ... since they now provide several functions as __attribute__((overloadable)), the argument detection logic need updates. Patched-by: destman at github Fixes #1738 Closes #1739 Marcel Raad (8 Aug 2017) - ax_code_coverage.m4: update to latest version This updates the script to aad5ad5fedb306b39f901a899b7bd305b66c418d from August 01, 2017. Notably, this removes the lconv version whitelist. Closes https://github.com/curl/curl/pull/1716 Daniel Stenberg (7 Aug 2017) - test1427: verify command line parser integer overflow detection - curl: detect and bail out early on parameter integer overflows Make the number parser aware of the maximum limit curl accepts for a value and return an error immediately if larger, instead of running an integer overflow later. Fixes #1730 Closes #1736 - glob: do not continue parsing after a strtoul() overflow range Added test 1289 to verify. CVE-2017-1000101 Bug: https://curl.haxx.se/docs/adv_20170809A.html Reported-by: Brian Carpenter - tftp: reject file name lengths that don't fit ... and thereby avoid telling send() to send off more bytes than the size of the buffer! CVE-2017-1000100 Bug: https://curl.haxx.se/docs/adv_20170809B.html Reported-by: Even Rouault Credit to OSS-Fuzz for the discovery - [Even Rouault brought this change] file: output the correct buffer to the user Regression brought by 7c312f84ea930d8 (April 2017) CVE-2017-1000099 Bug: https://curl.haxx.se/docs/adv_20170809C.html Credit to OSS-Fuzz for the discovery - easy_events: make event data static First: this function is only used in debug-builds and not in release/real builds. It is used to drive tests using the event-based API. A pointer to the local struct is passed to CURLMOPT_TIMERDATA, but the CURLMOPT_TIMERFUNCTION calback can in fact be called even after this funtion returns, namely when curl_multi_remove_handle() is called. Reported-by: Brian Carpenter - getparameter: avoid returning uninitialized 'usedarg' Fixes #1728 Marcel Raad (5 Aug 2017) - [Isaac Boukris brought this change] gssapi: fix memory leak of output token in multi round context When multiple rounds are needed to establish a security context (usually ntlm), we overwrite old token with a new one without free. Found by proposed gss tests using stub a gss implementation (by valgrind error), though I have confirmed the leak with a real gssapi implementation as well. Closes https://github.com/curl/curl/pull/1733 - darwinssl: fix compiler warning clang complains: vtls/darwinssl.c:40:8: error: extra tokens at end of #endif directive [-Werror,-Wextra-tokens] This breaks the darwinssl build on Travis. Fix it by making this token a comment. Closes https://github.com/curl/curl/pull/1734 - CMake: fix CURL_WERROR for MSVC When using CURL_WERROR in MSVC builds, the debug flags were overridden by the release flags and /WX got added twice in debug mode. Closes https://github.com/curl/curl/pull/1715 Daniel Stenberg (4 Aug 2017) - RELEASE-NOTES: synced with 561e9217c - test1010: verify that #1718 is fixed ... by doing two transfers in nocwd mode and check that there's no superfluous CWD command. - FTP: skip unnecessary CWD when in nocwd mode ... when reusing a connection. If it didn't do any CWD previously. Fixes #1718 Marcel Raad (4 Aug 2017) - travis: explicitly specify dist This makes the builds more reproducible as travis is currently rolling out trusty as default dist [1]. Specifically, this avoids coverage check failures when trusty is used as seen in [2] until we figure out what's wrong. [1] https://blog.travis-ci.com/2017-07-11-trusty-as-default-linux-is-coming [2] https://github.com/curl/curl/pull/1692 Closes https://github.com/curl/curl/pull/1725 Daniel Stenberg (4 Aug 2017) - travis: BUILD_TYPE => T (to make the full line appear nicer on travis web UI) - travis: add osx build with darwinssl Closes #1706 - darwin: silence compiler warnings With a clang pragma and three type fixes Fixes #1722 - BUILD.WINDOWS: mention buildconf.bat for builds off git - darwinssl: fix curlssl_sha256sum() compiler warnings on first argument - test130: verify comments in .netrc - [Gisle Vanem brought this change] netrc: skip lines starting with '#' Bug: https://curl.haxx.se/mail/lib-2017-08/0008.html Marcel Raad (3 Aug 2017) - CMake: set MSVC warning level to 4 The MSVC warning level defaults to 3 in CMake. Change it to 4, which is consistent with the Visual Studio and NMake builds. Disable level 4 warning C4127 for the library and additionally C4306 for the test servers to get a clean CURL_WERROR build as that warning is raised in some macros in older Visual Studio versions. Ref: https://github.com/curl/curl/pull/1667#issuecomment-314082794 Closes https://github.com/curl/curl/pull/1711 Daniel Stenberg (2 Aug 2017) - CURLOPT_NETRC.3: fix typo in 7e48aa386156f9c2 Reported-by: Viktor Szakats - CURLOPT_NETRC.3: mention the file name on windows ... and CURLOPT_NETRC_FILE(3). - travis: build osx with libressl too - travis: build osx with openssl too - tests/server/util: fix curltime mistake from 4dee50b9c80f9 Marcel Raad (1 Aug 2017) - curl_threads: fix MSVC compiler warning Use LongToHandle to convert from long to HANDLE in the Win32 implementation. This should fix the following warning when compiling with MSVC 11 (2012) in 64-bit mode: lib\curl_threads.c(113): warning C4306: 'type cast' : conversion from 'long' to 'HANDLE' of greater size Closes https://github.com/curl/curl/pull/1717 Daniel Stenberg (1 Aug 2017) - BUGS: improved phrasing about security bugs Reported-by: Max Dymond - BUGS: clarify how to report security related bugs - [Brad Spencer brought this change] multi: fix request timer management There are some bugs in how timers are managed for a single easy handle that causes the wrong "next timeout" value to be reported to the application when a new minimum needs to be recomputed and that new minimum should be an existing timer that isn't currently set for the easy handle. When the application drives a set of easy handles via the `curl_multi_socket_action()` API (for example), it gets told to wait the wrong amount of time before the next call, which causes requests to linger for a long time (or, it is my guess, possibly forever). Bug: https://curl.haxx.se/mail/lib-2017-07/0033.html Jay Satiro (1 Aug 2017) - curl_setup: Define CURL_NO_OLDIES for building libcurl .. to catch accidental use of deprecated error codes. Ref: https://github.com/curl/curl/issues/1688#issuecomment-316764237 Daniel Stenberg (1 Aug 2017) - [Jeremy Tan brought this change] configure: fix the check for IdnToUnicode Fixes #1669 Closes #1713 - http: fix response code parser to avoid integer overflow test 1429 and 1433 were updated to work with the stricter HTTP status line parser. Closes #1714 Reported-by: Brian Carpenter Jay Satiro (31 Jul 2017) - [Dwarakanath Yadavalli brought this change] libcurl: Stop using error codes defined under CURL_NO_OLDIES Fixes https://github.com/curl/curl/issues/1688 Closes https://github.com/curl/curl/pull/1712 - include.d: clarify --include is only for response headers Follow-up to 171f8de and de6de94. Bug: https://github.com/curl/curl/commit/de6de94#commitcomment-23370851 Reported-by: Daniel Stenberg Daniel Stenberg (30 Jul 2017) - [Jason Juang brought this change] cmake: support make uninstall Closes #1674 - RELEASE-NOTES: synced with 001701c47 Marcel Raad (29 Jul 2017) - AppVeyor: now really use CURL_WERROR It was misspelled as CURL_ERROR in commit 2d86e8d1286e0fbe3d811e2e87fa0b5e53722db4. Closes https://github.com/curl/curl/pull/1686 Jay Satiro (29 Jul 2017) - tool_help: clarify --include is only for response headers Follow-up to 171f8de. Ref: https://github.com/curl/curl/issues/1704 - splay: fix signed/unsigned mismatch warning Follow-up to 4dee50b. Ref: https://github.com/curl/curl/pull/1693 Daniel Stenberg (28 Jul 2017) - include.d: clarify that it concerns the response headers Reported-by: olesteban at github Fixes #1704 - [Johannes Schindelin brought this change] curl_rtmp: fix a compiler warning The headers of librtmp declare the socket as `int`, and on Windows, that disagrees with curl_socket_t. Bug: #1652 Signed-off-by: Johannes Schindelin - test1323: verify curlx_tvdiff - timeval: struct curltime is a struct timeval replacement ... to make all libcurl internals able to use the same data types for the struct members. The timeval struct differs subtly on several platforms so it makes it cumbersome to use everywhere. Ref: #1652 Closes #1693 - darwinssl: fix variable type mistake (regression) ... which made --tlsv1.2 not work because it would blank the max tls version variable. Reported-by: Nick Miyake Bug: #1703 - multi: mention integer overflow risk if using > 500 million sockets Reported-by: ovidiu-benea@users.noreply.github.com Closes #1675 Closes #1683 - checksrc: escape open brace in regex ... to silence warning. Kamil Dudka (20 Jul 2017) - nss: fix a possible use-after-free in SelectClientCert() ... causing a SIGSEGV in showit() in case the handle used to initiate the connection has already been freed. This commit fixes a bug introduced in curl-7_19_5-204-g5f0cae803. Reported-by: Rob Sanders Bug: https://bugzilla.redhat.com/1436158 - nss: unify the coding style of nss_send() and nss_recv() No changes in behavior intended by this commit. Marcel Raad (18 Jul 2017) - tests/server/resolve.c: fix deprecation warning MSVC warns that gethostbyname is deprecated. Always use getaddrinfo instead to fix this when IPv6 is enabled, also for IPv4 resolves. This is also consistent with what libcurl does. Closes https://github.com/curl/curl/pull/1682 Jay Satiro (17 Jul 2017) - darwinssl: fix pinnedpubkey build error - s/SessionHandle/Curl_easy/ Bug: https://github.com/curl/curl/commit/eb16305#commitcomment-23035670 Reported-by: Gisle Vanem Marcel Raad (16 Jul 2017) - rtspd: fix GCC warning after MSVC warning fix Older GCC warns: /tests/server/rtspd.c:1194:10: warning: missing braces around initializer [-Wmissing-braces] Fix this by using memset instead of an initializer. - libtest: fix MSVC warning C4706 With warning level 4, MSVC warns about assignments within conditional expressions. Change the while loop to a do-while loop to fix this. This change is also consistent with CODE_STYLE.md. - sockfilt: suppress conversion warning with explicit cast MSVC warns when implicitly casting -1 to unsigned long. - rtspd: fix MSVC level 4 warning warning C4701: potentially uninitialized local variable 'req' used - winbuild: re-enable warning C4127 for curl tool Disabled in cda19a345f6970e22fe8b7a808aeb8f086a21eac. It only needs to be disabled for libcurl. - winbuild: build with warning level 4 This is consistent with 7bc64561a2e63ca93e4b0b31d350773ba80955c2, which changed the warning level from 3 to 4 for the Visual Studio project files. But disable the level 4 warning C4127 "conditional expression is constant", as that one is issued by older versions of the Windows SDK as well as curl itself under some circumstances. Closes https://github.com/curl/curl/pull/1667 Jay Satiro (12 Jul 2017) - [Max Dymond brought this change] travis: install libidn2 Install libidn2 to increase test coverage (IDN tests) Closes https://github.com/curl/curl/pull/1673 Marcel Raad (12 Jul 2017) - travis: enable warnings also in release mode ... to get warnings also on Linux/GCC and OSX/clang. Closes https://github.com/curl/curl/pull/1666 Daniel Stenberg (12 Jul 2017) - [Max Dymond brought this change] travis: install libssh2 Install libssh2 to increase test coverage (SFTP, SCP) Marcel Raad (12 Jul 2017) - system.h: include winsock2.h before windows.h ... to avoid compiler warnings if the user doesn't want WIN32_LEAN_AND_MEAN. - build: remove WIN32_LEAN_AND_MEAN from individual build systems It's defined for all build systems in curl_setup.h since commit beb08481d01a07a8b10938b1078a5e298b1c2912. This caused macro redefinition warnings in the configure builds. Closes https://github.com/curl/curl/pull/1677 Jay Satiro (11 Jul 2017) - ISSUE_TEMPLATE: Add a comment not to file security issues on github Marcel Raad (11 Jul 2017) - curl_setup: always define WIN32_LEAN_AND_MEAN on Windows Make sure to always define WIN32_LEAN_AND_MEAN before including any Windows headers to avoid pulling in unnecessary headers. This avoids unnecessary macro clashes and compiler warnings. Ref: https://github.com/curl/curl/issues/1562 Closes https://github.com/curl/curl/pull/1672 Jay Satiro (11 Jul 2017) - strerror: Preserve Windows error code in some functions This is a follow-up to af02162 which removed (SET_)ERRNO macros. That commit was an earlier draft that I committed by mistake, which was then remedied by a5834e5 and e909de6, and now this commit. With this commit there is now no difference between the current code and the changes that were approved in the final draft. Thanks-to: Max Dymond, Marcel Raad, Daniel Stenberg, Gisle Vanem Ref: https://github.com/curl/curl/pull/1589 Marcel Raad (10 Jul 2017) - [Max Dymond brought this change] tests: Fix up issues with errno in test files Closes https://github.com/curl/curl/pull/1671 Daniel Stenberg (10 Jul 2017) - errno: fix non-windows builds after af0216251b94e7 - [Ryan Winograd brought this change] make: fix docs build on OpenBSD Ref: #1591 Marcel Raad (10 Jul 2017) - ldap: fix MinGW compiler warning ldap_bind_s is marked as deprecated in w32api's winldap.h shipping with the latest original MinGW, resulting in compiler warnings since commit f0fe66f13c93d3d0af45d9fb1231c9164e0f9dc8. Fix this for the non-SSPI case by using ldap_simple_bind_s again instead of ldap_bind_s with LDAP_AUTH_SIMPLE. Closes https://github.com/curl/curl/pull/1664 - curl-compilers.m4: disable warning spam with Cygwin's clang When building with Cygwin or MinGW, libtool uses a wrapper executable instead of a wrapper script [1], which is written in C and throws missing-variable-declarations warnings. Don't enable these warnings on Cygwin and MinGW in order to avoid warnings for every executable built, which spams the test suite output when using Cygwin's clang. [1] https://www.gnu.org/software/libtool/manual/html_node/Wrapper-executables.html Closes https://github.com/curl/curl/pull/1665 Jay Satiro (10 Jul 2017) - curl_setup_once: Remove ERRNO/SET_ERRNO macros Prior to this change (SET_)ERRNO mapped to GetLastError/SetLastError for Win32 and regular errno otherwise. I reviewed the code and found no justifiable reason for conflating errno on WIN32 with GetLastError/SetLastError. All Win32 CRTs support errno, and any Win32 multithreaded CRT supports thread-local errno. Fixes https://github.com/curl/curl/issues/895 Closes https://github.com/curl/curl/pull/1589 - tool_getparam: fix potentially uninitialized err Marcel Raad (9 Jul 2017) - smb: rename variable to fix shadowing warning GCC 4.6.3 on travis complains: smb.c: In function ‘get_posix_time’: smb.c:725:13: error: declaration of ‘time’ shadows a global declaration [-Werror=shadow] Fix this by renaming the variable. - tool_cb_wrt: fix variable shadowing warning GCC 4.4 complains: tool_cb_wrt.c:81: error: declaration of ‘isatty’ shadows a global declaration /usr/include/unistd.h:782: error: shadowed declaration is here Fix this by renaming the variable. Closes https://github.com/curl/curl/pull/1661 Daniel Stenberg (8 Jul 2017) - RELEASE-NOTES: synced with be2c999b8 - travis: install stunnel - valgrind.supp: supress OpenSSL false positive seen on travis - travis: detect and use valgrind for normal builds Closes #1653 - travis: add SMB, DICT, TELNET torture to coverage test - [Paul Harris brought this change] cmake: offer CMAKE_DEBUG_POSTFIX when building with MSVC Removes BUILD_RELEASE_DEBUG_DIRS since it wasn't used anywhere. Closes #1649 - CURLOPT_POSTFIELDS.3: explain the 100-continue magic better - [Max Dymond brought this change] test1452: add telnet negotiation Add a basic telnet server for negotiating some telnet options before echoing back any data that's sent to it. Closes #1645 - travis: do more tests in the coverage run I added a selection of torture and event tests that run "fast enough" - curl_easy_escape.3: mention the (lack of) encoding Fixes #1612 Reported-by: Jeroen Ooms - [Gisle Vanem brought this change] memdebug: don't setbuf() if the file open failed Bug: https://github.com/curl/curl/issues/828#issuecomment-313475151 - appveyor: enable CURL_WERROR on all builds - cmake: add CURL_WERROR for enabling "warning as errors" - [Hannes Magnusson brought this change] cmake: remove spurious "-l" from linker flags Fixes #1552 - test506: skip if threaded-resolver - runtests: support "threaded-resolver" as a feature ... to let tests require it or skip if present - asyn-thread.c: fix unused variable warnings on macOS - http: s/TINY_INITIAL_POST_SIZE/EXPECT_100_THRESHOLD Make the name reflect its use better, and add a short comment describing what it's for. - cmake: if inet_pton is used, bump _WIN32_WINNT ... and make sure inet_pton is always checked for when *not* using Windows, which is a regression from 4fc6ebe18. Idea-by: Sergei Nikulov - select.h: avoid macro redefinition harder ... by checking the POLLIN define, as the header file checks don't work on Windows. - inet_pton: fix include on windows to get prototype inet_pton() exists on Windows and gets used by our cmake builds. Make sure the correct header file is included to avoid compiler warnings. Closes #1639 - TODO: 1.10 auto-detect proxy Closes #1572 - TODO: HTTP proxy CONNECT is non-blocking now - cmake: fix send/recv argument scanner for windows ... by simply trying the Windows argument types first. Fixes #1640 - RELEASE-NOTES: synced with 596cfb6c0 - [Gisle Vanem brought this change] smb: add support for CURLOPT_FILETIME Bug: https://curl.haxx.se/mail/lib-2017-07/0005.html Closes #1643 - travis: install nghttp2 on linux builds Closes #1642 - [Gisle Vanem brought this change] smb: fix build for djgpp/MSDOS bug: https://curl.haxx.se/mail/lib-2017-07/0005.html - configure: try ldap/lber in reversed order first When scanning for which LDAP libraries to use, try the -lldap -llber combination before the reversed order since it has a greater chance of working when linking with libcurl statically. Fixes #1619 Closes #1634 Reported-by: David E. Narváez - configure: remove checks for 5 functions never used fork, getprotobyname, inet_addr, perror, uname closes #1638 - dist: add SMB python deps into the tarball - [Max Dymond brought this change] test1451: add SMB support to the testbed Add test 1451 which does some very basic SMB testing using the impacket SMB server. Closes #1630 - [Max Dymond brought this change] test: add impacket for SMB testing Import impacket 0.9.15 for use in SMB testing. This was generated by doing "pip2.7 install -t . impacket" Unnecessary files for current testing were deleted. - travis.yml: use --enable-werror on debug builds ... to better detect and fault on compiler warnings/errors Closes #1637 - tool_sleep: typecast to avoid macos compiler warning tool_sleep.c:54:24: error: implicit conversion loses integer precision: 'long' to '__darwin_suseconds_t' (aka 'int') [-Werror,-Wshorten-64-to-32] - [Martin Kepplinger brought this change] timeval.c: Use long long constant type for timeval assignment On a 64 bit host, sparse says: timeval.c:148:15: warning: constant 0x7fffffffffffffff is so big it is long timeval.c:149:12: warning: constant 0x7fffffffffffffff is so big it is long so let's use long long constant types in order to prevent undesired overflow failures. Bug: https://curl.haxx.se/mail/lib-2017-07/0003.html Closes #1636 Signed-off-by: Martin Kepplinger - url: make the original string get used on subsequent transfers ... since CURLOPT_URL should follow the same rules as other options: they remain set until changed or cleared. Added test 1551 to verify. Fixes #1631 Closes #1632 Reported-by: Pavel Rochnyak - [Johannes Schindelin brought this change] gtls: fix build when sizeof(long) < sizeof(void *) - Change gnutls pointer/int macros to pointer/curl_socket_t. Prior to this change they used long type as well. The size of the `long` data type can be shorter than that of pointer types. This is the case most notably on Windows. If C99 were acceptable, we could simply use `intptr_t` here. But we want to retain C89 compatibility. Simply use the trick of performing pointer arithmetic with the NULL pointer: to convert an integer `i` to a pointer, simply take the address of the `i`th element of a hypothetical character array starting at address NULL. To convert back, simply cast the pointer difference. Thanks to Jay Satiro for the initial modification to use curl_socket_t instead of int/long. Closes #1617 Signed-off-by: Johannes Schindelin - [Ryan Winograd brought this change] unit1399: fix integer overflow Bug: #1616 Closes #1633 - [Per Malmberg brought this change] cmake: Added compatibility options for older Windows versions CURL_STATIC_CRT and ENABLE_INET_PTON Closes #1621 - unit1399: add logging to time comparison ... to enable tracking down why autobuilds fail on this Bug: #1616 - make: build the docs subdir only from within src ... and don't build at all in include Prompted-by-work-by: Simon Warta Ref: #1590 Closes #1591 - [Max Dymond brought this change] test1450: fix up DICT server in torture mode As per https://github.com/curl/curl/pull/1615, the DICT server is a little spammy in torture mode due to the sockets being torn down unexpectedly. Fix this by adding some error handling to the handling function. Closes #1629 - [Max Dymond brought this change] test1450: add simple testing for DICT Add a new server which provides a DICT interface. This is intended to begin coverage testing for lib/dict.c Closes #1615 - [Dan Fandrich brought this change] test1521: fix out-of-tree builds, broken with 467da3af The test.h file is no longer in the same directory as the source file, so that directory needs to be added to the include path. Fixes #1627 Closes #1628 - [Max Dymond brought this change] http2: handle PING frames Add a connection check function to HTTP2 based off RTSP. This causes PINGs to be handled the next time the connection is reused. Closes #1521 - [Max Dymond brought this change] handler: refactor connection checking Add a new type of callback to Curl_handler which performs checks on the connection. Alter RTSP so that it uses this callback to do its own check on connection health. - [Dmitry Kostjuchenko brought this change] openssl: improve fallback seed of PRNG with a time based hash Fixes #1620 - [Ryan Winograd brought this change] progress: prevent resetting t_starttransfer Prevent `Curl_pgrsTime` from modifying `t_starttransfer` when invoked with `TIMER_STARTTRANSFER` more than once during a single request. When a redirect occurs, this is considered a new request and `t_starttransfer` can be updated to reflect the `t_starttransfer` time of the redirect request. Closes #1616 Bug: https://github.com/curl/curl/pull/1602#issuecomment-310267370 - curl_strequal.3: fix typo in SYNOPSIS Reported-by: Jesse Chisholm Fixes #1623 - RELEASE-NOTES: synced with ce2c3ebda Kamil Dudka (28 Jun 2017) - curl --socks5-{basic,gssapi}: control socks5 auth Closes https://github.com/curl/curl/pull/1454 - CURLOPT_SOCKS5_AUTH: allowed methods for SOCKS5 proxy auth If libcurl was built with GSS-API support, it unconditionally advertised GSS-API authentication while connecting to a SOCKS5 proxy. This caused problems in environments with improperly configured Kerberos: a stock libcurl failed to connect, despite libcurl built without GSS-API connected fine using username and password. This commit introduces the CURLOPT_SOCKS5_AUTH option to control the allowed methods for SOCKS5 authentication at run time. Note that a new option was preferred over reusing CURLOPT_PROXYAUTH for compatibility reasons because the set of authentication methods allowed by default was different for HTTP and SOCKS5 proxies. Bug: https://curl.haxx.se/mail/lib-2017-01/0005.html Closes https://github.com/curl/curl/pull/1454 - socks: deduplicate the code for auth request - socks: use proxy_user instead of proxy_name ... to make it obvious what the data is used for Daniel Stenberg (27 Jun 2017) - libtest/make: generate lib1521.c ... instead of having the generated code checked in. This saves space in the tarball but primarily automatically adapts to newly added options. Closes #1614 Jay Satiro (26 Jun 2017) - tool_getparam: fix memory leak on test 1147 OOM (torture tests) Bug: https://github.com/curl/curl/pull/1486#issuecomment-310926872 Reported-by: Dan Fandrich Dan Fandrich (25 Jun 2017) - test1537: fixed memory leak on OOM Marcel Raad (25 Jun 2017) - test1521: fix compiler warnings The integer literal 3123123123 doesn't fit into a 32-bit signed integer, so GCC with 32-bit long warns in C90 mode: this decimal constant is unsigned only in ISO C90 [enabled by default] Fix this by using ULONG_MAX, which should fit in any curl_off_t and has the correct suffix to not issue any warnings. Also adds the missing CURLOPT_REQUEST_TARGET from commit 9b167fd090f596eac828817d48c247eeae53407f. Closes https://github.com/curl/curl/pull/1611 Daniel Stenberg (24 Jun 2017) - curl/system.h: add check for XTENSA for 32bit gcc Reported-by: Neil Kolban Fixes: 1598 - [Henrik S. Gaßmann brought this change] winbuild: fix boringssl build Compile with `WIN32_LEAN_AND_MEAN` which prevents `windows.h` from including too much clutter including `wincrypt.h` which in turn contains some preprocessor macros that clash with boringssl symbols. Detect boringssl by checking the existance of `is_boringssl.h` and set the corresponding `HAVE_BORINGSSL` for compilation which is used in `ldap.c` to undefine the evil macros. Closes #1610 - progress: progress.timespent needs to be us follow-up to 64ed44a815e4e to fix test 500 failures Marcel Raad (24 Jun 2017) - curl-compilers.m4: fix unknown-warning-option on Apple clang Since 5598b0bd63f690c151074494ce47ef872f004ab4, clang -v is used to detect the clang version. The version number was expected to come after the word "version". For Apple clang, this doesn't work as it has its own versioning scheme. The version number is now first searched after the string "based on LLVM". This works for Apple clang before version 7, and also for e.g. Ubuntu's clang up to version 3.7. If it's not found and the version string contains "Apple LLVM version", clang version 3.7 is assumed, which is the version that comes with Xcode 7. Otherwise, the version number is still expected after the word "version", which works for very old Apple clang versions. Ref: https://trac.macports.org/wiki/XcodeVersionInfo Fixes https://github.com/curl/curl/issues/1606 Closes https://github.com/curl/curl/pull/1607 Daniel Stenberg (24 Jun 2017) - progress: fix "time spent", broke in adef394ac - CURLINFO_REDIRECT_URL.3: mention the CURLOPT_MAXREDIRS case ... supported since 7.54.1 - maketgz: switch to -6e for xz To reduce the memory requirement for decompress, and still do almost as good compression as with -9e. Pointed-out-by: Dan Fandrich - libtest/Makefile: remove unused lib1541 variables - CONTRIBUTE.md: mention the out-of-tree build test too - maketgz: switch to xz instead of lzma The compressed output size seems to be a tad bit smaller, but generally xz seems more preferred these days and is used directly by for example gentoo instead of bz2. "Users of LZMA Utils should move to XZ Utils" => https://tukaani.org/lzma/ Closes #1604 - --request-target: instead of --strip-path-slash ... and CURLOPT_REQUEST_TARGET instead of CURLOPT_STRIP_PATH_SLASH. This option instead provides the full "alternative" target to use in the request, instead of extracting the path from the URL. Test 1298 and 1299 updated accordingly. Idea-by: Evert Pot Suggestion: https://daniel.haxx.se/blog/2017/06/19/options-with-curl/comment-page-1/#comment-18373 Closes #1593 Marcel Raad (21 Jun 2017) - lib1521: fix missing-variable-declarations clang warnings Declare TU-local variables static. - travis: enable typecheck-gcc warnings - switch debug and release configurations so that we get an optimized build with GCC 4.3+ as required by typecheck-gcc - enable warnings-as-errors for release builds (which have warnings disabled) Closes https://github.com/curl/curl/pull/1595 - typecheck-gcc: add support for CURLINFO_OFF_T typecheck-gcc expected curl_socket_t instead of curl_off_t arguments for CURLINFO_OFF_T. Detected by test1521, unfortunately only when run locally. Closes https://github.com/curl/curl/pull/1592 Daniel Stenberg (21 Jun 2017) - [Simon Warta brought this change] ci: whitelist branches to avoid testing feature branches twice - [Gisle Vanem brought this change] lib: fix the djgpp build Bug: https://github.com/curl/curl/commit/73a2fcea0b4adea6ba342cd7ed1149782c214ae3#commitcomment-22655993 Marcel Raad (20 Jun 2017) - if2ip: fix compiler warning in ISO C90 mode remote_scope_id is only used when both HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID and ENABLE_IPV6 are defined instead of only one of them. Daniel Stenberg (20 Jun 2017) - travis: do the distcheck test build out-of-tree as well - http: add --strip-path-slash and CURLOPT_STRIP_PATH_SLASH ... to enable sending "OPTIONS *" which wasn't possible previously. This option currently only works for HTTP. Added test cases 1298 + 1299 to verify Fixes #1280 Closes #1462 - test1521: test getinfo's OFF_T types too Closes #1588 - lib1521: add curl_easy_getinfo calls to the test set Also added return value checks to make sure no unexpected return codes are used. - [Simon Warta brought this change] automake: use $(MKHELP) variable instead if constant mkhelp.pl this improves symmetry with the rule above - [Simon Warta brought this change] mkhelp.pl: fix script name in usage text - RELEASE-NOTES: synced with 3b80d3ca4 - getinfo: return sizes as curl_off_t This change introduces new alternatives for the existing six curl_easy_getinfo() options that return sizes or speeds as doubles. The new versions are named like the old ones but with an appended '_T': CURLINFO_CONTENT_LENGTH_DOWNLOAD_T CURLINFO_CONTENT_LENGTH_UPLOAD_T CURLINFO_SIZE_DOWNLOAD_T CURLINFO_SIZE_UPLOAD_T CURLINFO_SPEED_DOWNLOAD_T CURLINFO_SPEED_UPLOAD_T Closes #1511 - PIPELINING_SERVER_BL: cleanup the internal list use The list was freed incorrectly since the llist refactor of cbae73e1dd959. Added test 1550 to verify that it works and avoid future regressions. Reported-by: Pascal Terjan Fixes #1584 Closes #1585 - http2: fix OOM crash torture mode with test 1021 found it - CURLOPT_PREQUOTE.3: spellfix man page reference Marcel Raad (18 Jun 2017) - http_proxy: fix build with http and proxy After deff7de0eb0e22d2d142b96b9cc84cd8db5d2a48, the build without CURL_DISABLE_PROXY and CURL_DISABLE_HTTP was failing because of missing includes. - http_proxy: fix compiler warning With CURL_DISABLE_PROXY or CURL_DISABLE_HTTP, GCC complained about a missing prototype for Curl_connect_free. Daniel Stenberg (18 Jun 2017) - TODO: update the TOC too - TODO: implement support for CURLOPT_PREQUOTE with SFTP ... also updated the CURLOPT_PREQUOTE.3 man page to mention the correct protocol support. Closes #1514 - tool_wrte_cb: remove check for config == NULL ... as it really cannot have reached this far with config being NULL, thus this is unnecesary and misleading. Bug: https://news.ycombinator.com/item?id=14577585 and https://daniel.haxx.se/blog/2017/06/17/curl-doesnt-spew-binary-anymore/comment-page-1/#comment-18356 Forwarded-to-us-by: Jakub Wilk - curl: prevent binary output spewed to terminal ... unless "--output -" is used. Binary detection is done by simply checking for a binary zero in early data. Added test 1425 1426 to verify. Closes #1512 Marcel Raad (16 Jun 2017) - Makefile.m32: enable -W for MinGW32 build The configure-based build also has this in addition to -Wall. Closes https://github.com/curl/curl/pull/1578 - curl-compilers.m4: enable comma clang warning It usually warns when using commas instead of semicolons or other operators by accident. Closes https://github.com/curl/curl/pull/1578 - curl-compilers.m4: enable missing-variable-declarations clang warning It usually warns when forgetting to declare TU-local variables static. Closes https://github.com/curl/curl/pull/1578 - curl-compilers.m4: enable double-promotion warning Enable -Wdouble-promotion for both GCC and clang. It warns on implicit promotion from float to double. Closes https://github.com/curl/curl/pull/1578 - curl-compilers.m4: enable vla warning for clang Previously, that warning was only implicitly active in C90 mode. Enable it unconditionally as already done for GCC. Closes https://github.com/curl/curl/pull/1578 Daniel Stenberg (16 Jun 2017) - http-proxy: fix chunked-encoded CONNECT responses Regression since 5113ad0424. ... and remove 'flaky' from test 1061 again Closes #1579 - http-proxy: deal with EAGAIN ... the previous code would reset the header length wrongly (since 5113ad0424). This makes test 1060 reliable again. Also: make sws send even smaller chunks of data to increase the likeliness of this happening. - libtest/libntlmconnect: fix compiler warnings from f94fcdb - [Jay Satiro brought this change] HTTPS-Proxy: don't offer h2 for https proxy connections Bug: https://github.com/curl/curl/issues/1254 Closes #1546 - tests: stabilize test 2032 and 2033 Both these tests run the same underlying test code: libntlmconnect.c - this test code made some assumptions about socket ordering when it used curl_easy_fdset() and when we changed timing or got accidental changes in libcurl the tests would fail. The tests verify that the different transfers keep using the same connections, which I now instead made sure by adding the number of bytes each transfer gets and then verifies that they always get the same amount as when these tests worked. Closes #1576 - test1148: verify the -# progressbar Closes #1569 - test1061: mark as flaky Fails intermittently on travis builds since a few days. Likely due to 5113ad0424. Jay Satiro (16 Jun 2017) - url: refactor the check for Windows drive letter in path - Move the logic to detect a Windows drive letter prefix (eg c: in c:foo) into a function-like macro. Closes https://github.com/curl/curl/pull/1571 - mk-ca-bundle.pl: Check curl's exit code after certdata download - No longer allow partial downloads of certdata. Prior to this change partial downloads were (erroneously?) allowed since only the server code was checked to be 200. Bug: https://github.com/curl/curl/pull/1577 Reported-by: Matteo B. Daniel Stenberg (16 Jun 2017) - dist: add the fuzz dir to the tarball - configure: disable nghttp2 too if HTTP has been disabled - http-proxy: fix build with --disable-proxy or --disable-http Reported-by: Dan Fandrich - fuzz/README: document how to build Fixes #1476 - [Frederik B brought this change] fuzz: corpora file structure, initial commit - [Frederik B brought this change] fuzz: bring oss-fuzz initial code converted to C89 - http-proxy: only attempt FTP over HTTP proxy ... all other non-HTTP protocol schemes are now defaulting to "tunnel trough" mode if a HTTP proxy is specified. In reality there are no HTTP proxies out there that allow those other schemes. Assisted-by: Ray Satiro, Michael Kaufmann Closes #1505 - TODO: the generated include file is gone ... since commit 73a2fcea0b - curl_setup.h: error out on CURL_WANTS_CA_BUNDLE_ENV use ... to make it really apparent if there's any user using this on purpose. Suggested-by: Jay Satiro Closes #1542 - lib/curl_setup.h: remove CURL_WANTS_CA_BUNDLE_ENV When this define was set, libcurl would check the environment variable named CURL_CA_BUNDLE at run-time and use that CA cert bundle. This feature was only defined by the watcom and m32 makefiles and caused inconsistent behaviours among libcurls built on different platforms. The curl tool does already feature its own similar logic and the library does not really need it, and it isn't documented libcurl behavior. So this change removes it. Ref: #1538 - test1147: verify -H on a file - curl: allow --header and --proxy-header read from file So many headers can be provided as @filename. Suggested-by: Timothe Litt Closes #1486 - RELEASE-NOTES: synced with 2ad80eec5 - curl/curlver.h: start working on 7.55.0 - http-proxy: do the HTTP CONNECT process entirely non-blocking Mentioned as a problem since 2007 (8f87c15bdac63) and of course it existed even before that. Closes #1547 - progress: let "current speed" be UL + DL speeds combined Bug #1556 Reported-by: Paul Harris Closes #1559 Marcel Raad (14 Jun 2017) - system.h: fix MinGW build CURLSYS_PULL_WS2TCPIP_H got renamed to CURL_PULL_WS2TCPIP_H in commit 73a2fcea0b4adea6ba342cd7ed1149782c214ae3. Daniel Stenberg (14 Jun 2017) - timers: store internal time stamps as time_t instead of doubles This gives us accurate precision and it allows us to avoid storing "no time" for systems with too low timer resolution as we then bump the time up to 1 microsecond. Should fix test 573 on windows. Remove the now unused curlx_tvdiff_secs() function. Maintains the external getinfo() API with using doubles. Fixes #1531 - dist: make the hugehelp.c not get regenerated unnecessarily The maketgz script now makes sure the generated hugehelp.c file in the tarball is newer than the generated curl.1 man page, so that it doesn't have to get unnecessarily rebuilt first thing in a typical build. It thus also removes the need for perl to build off a plain release tarball. Fixes #1565 - includes: remove curl/curlbuild.h and curl/curlrules.h Rely entirely on curl/system.h now. Introduced in Aug 2008 with commit 14240e9e109f. Now gone. Fixes #1456 Version 7.54.1 (14 Jun 2017) Daniel Stenberg (14 Jun 2017) - release: 7.54.1 Dan Fandrich (13 Jun 2017) - mk-lib1521.pl: updated to match the test changes in 916ec30a Daniel Stenberg (13 Jun 2017) - [Stuart Henderson brought this change] libressl: OCSP and intermediate certs workaround no longer needed lib/vtls/openssl.c has a workaround for a bug with OCSP responses signed by intermediate certs, this was fixed in LibreSSL in https://github.com/libressl-portable/openbsd/commit/912c64f68f7ac4f225b7d1fdc8fbd43168912ba0 Bug: https://curl.haxx.se/mail/lib-2017-06/0038.html - url: fix buffer overwrite with file protocol (CVE-2017-9502) Bug: https://github.com/curl/curl/issues/1540 Advisory: https://curl.haxx.se/docs/adv_20170614.html Assisted-by: Ray Satiro Reported-by: Marcel Raad - urlglob: fix division by zero The multiply() function that is used to avoid integer overflows, was itself reason for a possible division by zero error when passed a specially formatted glob. Reported-by: GwanYeong Kim - configure: update the copyright year in the output - [ygrek brought this change] BINDINGS: update SP-Forth and OCaml urls Michael Kaufmann (11 Jun 2017) - FindWin32CACert: Use a temporary buffer on the stack Don't malloc() the temporary buffer, and use the correct type: SearchPath() works with TCHAR, but SearchPathA() works with char. Set the buffer size to MAX_PATH, because the terminating null byte is already included in MAX_PATH. Reviewed-by: Daniel Stenberg Reviewed-by: Marcel Raad Closes #1548 Dan Fandrich (11 Jun 2017) - test1521: fixed OOM handling Daniel Stenberg (9 Jun 2017) - RELEASE-PROCEDURE: updated future release dates - [Paul Harris brought this change] gitignore: ignore all vim swap files Closes #1561 - lib1521: fix compiler warnings on the use of bad 'long' values Reported-by: Marcel Raad Bug: https://github.com/curl/curl/commit/cccac4fb2b20d6ed87da7978408c3ecacc464fe4#commitcomment-22453387 - setopt: check CURLOPT_ADDRESS_SCOPE option range ... and return error instead of triggering an assert() when being way out of range. Jay Satiro (8 Jun 2017) - [TheAssassin brought this change] cmake: Fix inconsistency regarding mbed TLS include directory Previously, one had to set MBEDTLS_INCLUDE_DIR to make CMake find the headers, but the system complained that mbed TLS wasn't found due to MBEDTLS_INCLUDE_DIRS (note the trailing s) was not set. This commit attempts to fix that. Closes https://github.com/curl/curl/pull/1541 Daniel Stenberg (8 Jun 2017) - [Ryuichi KAWAMATA brought this change] examples/multi-uv.c: fix deprecated symbol Closes #1557 - asyn-ares: s/Curl_expire_latest/Curl_expire - expire: remove Curl_expire_latest() With the introduction of expire IDs and the fact that existing timers can be removed now and thus never expire, the concept with adding a "latest" timer is not working anymore as it risks to not expire at all. So, to be certain the timers actually are in line and will expire, the plain Curl_expire() needs to be used. The _latest() function was added as a sort of shortcut in the past that's quite simply not necessary anymore. Follow-up to 31b39c40cf90 Reported-by: Paul Harris Closes #1555 - [Chris Carlmar brought this change] configure: fix link with librtmp when specifying path Bug: https://curl.haxx.se/mail/lib-2017-06/0017.html - file: make speedcheck use current time for checks ... as it would previously just get the "now" timestamp before the transfer starts and then not update it again. Closes #1550 - metalink: remove unused printf() argument - travis: let some builds *not* use --enable-debug typecheck-gcc and other things require optimized builds Closes #1544 - README.md: show the coverall coverage on github - lib1521: fix compiler warnings - test1521: make the code < 80 columns wide - test1121: use stricter types to work with typcheck-gcc - typecheck-gcc: allow CURLOPT_STDERR to be NULL too - test1521: test *all* curl_easy_setopt options mk-lib1521.pl generates a test program (lib1521.c) that calls curl_easy_setopt() for every known option with a few typical values to make sure they work (ignoring the return codes). Some small changes were necessary to avoid asserts and NULL accesses when doing this. The perl script needs to be manually rerun when we add new options. Closes #1543 Dan Fandrich (5 Jun 2017) - test1538: added "verbose logs" keyword These error messages are not displayed with --disable-verbose Daniel Stenberg (5 Jun 2017) - test1262: verify ftp download with -z for "if older than this" Marcel Raad (5 Jun 2017) - curl_ntlm_core: use Curl_raw_toupper instead of toupper This was the only remaining use of toupper in the entire source code. Suggested-by: Daniel Stenberg Daniel Stenberg (4 Jun 2017) - RELEASE-NOTES: synced with 65ba92650