Mark Wielaard [Tue, 16 Jun 2020 21:22:56 +0000 (23:22 +0200)]
ar, ranlib: Call fchown before fchmod, explicitly check and ignore errors.
In ar and ranlib we don't mind if the fchown call fails (it normally
would, then the file simply gets own by the current user). We used to
call fchown before fchmod, but that might ignore (or reset) some mode
flags, so call fchown first, then try fchmod (like we already do in
elfcompress). Also explicitly test and then ignore any errors for
chown. We used to do some asm trick, but that confuses some static
analyzers (and it is somewhat unreadable). Also split out the giant
if statements to make them a little bit more understandable.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Tue, 16 Jun 2020 16:24:43 +0000 (18:24 +0200)]
libdwfl: Flag an error if CIE return_address_register is invalid.
If the CIE return address register is invalid (unknown) for the
architecture immediately flag an error and return.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Tue, 16 Jun 2020 16:19:45 +0000 (18:19 +0200)]
libdwfl: When we find a compressed image, use that, don't search for others
We try to find a compressed vmlinux image ending with either .gz, bz2 or
xz. Stop searching if we find one. Otherwise we will leak a file descriptor
for an earlier one we opened.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Tue, 16 Jun 2020 16:14:37 +0000 (18:14 +0200)]
elflint: Explicitly check and ignore elf_compress error.
We just want to make sure that any section data is decompressed before
use, if the section was already decompressed that is fine, so just ignore
any errors. The make this more clear, explicitly check for errors, then
don't do anything. This is better than silently ignoring since everywhere
else in the code we do explicitly check for errors.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Tue, 16 Jun 2020 16:05:59 +0000 (18:05 +0200)]
libdw: Add missing FALLTHROUGH in execute_cfi.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Tue, 16 Jun 2020 15:53:08 +0000 (17:53 +0200)]
backends: Make the reloc_nametable zero element an one char array.
We are using the reloc_nametable zero element as an char array.
So make that element an actual array (we are actually after one
of the next string arrays in the table).
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Mon, 15 Jun 2020 22:25:47 +0000 (00:25 +0200)]
tests: Use #/usr/bin/env bash in scripts that use bash
Some systems don't have bash in /bin.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Sergei Trofimovich [Sun, 14 Jun 2020 19:28:07 +0000 (20:28 +0100)]
elfutils/configure.ac: use $READELF, not readelf
Allow user to specify own readelf. Use detected readelf,
not 'readelf'.
Noticed when was building elfutils on tuple-prefixed toolchain:
```
checking whether the compiler generates build-ids...
./configure: line 5197: readelf: command not found
no
```
Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
Mark Wielaard [Thu, 11 Jun 2020 17:07:28 +0000 (19:07 +0200)]
tests: Add GCOV flags when necessary.
The nlist-test has a special compile rule, make sure it gets the
GCOV flags when configuring with --enable-gcov.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Thu, 11 Jun 2020 14:09:03 +0000 (16:09 +0200)]
Prepare for 0.180
Set version to 0.180.
Update NEWS and elfutils.spec.in.
Set copyright year in printversion.
Regenerate po/*.po files.
Update .gitignore.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Wed, 10 Jun 2020 18:55:58 +0000 (20:55 +0200)]
libebl: Remove Ebl struct size check and MODVERSION string.
We used to do several sanity checks when the ebl backend libraries
were loaded to make sure there was no version mismatch. When initializing
the backend we passed the current Ebl struct size so the library could
check it supported the given Ebl struct and we checked that the init
method returned the correct build time module version string. Neither
are necessary now that the backends are builtin.
Remove both the struct size check and the MODVERSION string (which
wasn't actually checked anymore). Make the init function return the
given Ebl handle or NULL on error (no init function currently indicates
any error).
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Sun, 7 Jun 2020 14:13:13 +0000 (16:13 +0200)]
nm: Handle corrupt symbol name table.
We try to sort symbol by name (when neither -n nor -p are given).
This could crash if the symbol name table was corrupt. Use elf_strptr
to get the symbol name and use the empty string in case a name couldn't
be found.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Sat, 6 Jun 2020 23:02:52 +0000 (01:02 +0200)]
nm: Explicitly print weak 'V' or 'T' and common 'C' symbols.
Mimic binutils nm for bsd and posix formats which uses 'V' for weak
symbols, 'C' for common symbols and 'T' for weak functions. Also fix
some formatting issues. Don't print undefined addresses as zeros, but
make sure there is enough padding instead. Just print UNIQUE for
GNU_UNIQUE to make it fit 6 chars, like other binding names in sysv
format.
https://sourceware.org/bugzilla/show_bug.cgi?id=25227
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Thu, 4 Jun 2020 14:38:22 +0000 (16:38 +0200)]
libelf: Sync elf.h from glibc.
There is now finally a PT_GNU_PROPERTY define.
Also EM_ARC_COMPACT2 got renamed to EM_ARCV2.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Thu, 14 May 2020 21:52:54 +0000 (23:52 +0200)]
size: Also obey radix printing for bsd format.
eu-size already prints according to the given radix for sysv format,
but not for the bsd format. Fix that.
Reported-by: Ulrich Drepper <drepper@redhat.com>
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Sat, 9 May 2020 21:31:05 +0000 (23:31 +0200)]
libcpu: Free unused new bitfield on error in i386_parse.y new_bitfield.
GCC10 -fanalyzer detected we didn't free the newly created bitfield
on error. Make sure to free it before returning.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Sat, 9 May 2020 20:09:40 +0000 (22:09 +0200)]
libdwfl: Return failure from dwfl_standard_find_debuginfo for NULL module.
GCC10 -fanalyzer plus -flto notices that some functions called by
dwfl_standard_find_debuginfo check that the given module isn't NULL,
but others expect it to be non-NULL. Just return a failure immediately
when a NULL mod is passed to dwfl_standard_find_debuginfo.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Sat, 9 May 2020 19:05:31 +0000 (21:05 +0200)]
src: Check ebl_openbackend result before using ebl handle.
GCC10 -fanalyzer plus -flto sees that ebl_openbackend can fail and
return NULL. Most of the time we will get a dummy ebl, but in case
of out of memory or corrupt ELF header it might return NULL. Make
sure that we report a (fatal) error in that case in all tools.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Sat, 9 May 2020 03:02:25 +0000 (05:02 +0200)]
libelf: Check for NULL shdr in elf_strptr.
GCC10 -fanalyzer with -flto notices __elf64_getshdr_rdlock can fail
and because the result isn't checked in elf_strptr it can cause a
dereference of NULL.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Sat, 9 May 2020 01:41:56 +0000 (03:41 +0200)]
libelf: Check __gelf_getehdr_rdlock call doesn't fail in elf_getdata.
GCC10 -fanalyzer with -flto notices __gelf_getehdr_rdlock can fail
and that the result of the call in __libelf_set_rawdata_wrlock isn't
checked, which can cause a dereference of NULL.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Fri, 8 May 2020 22:18:20 +0000 (00:18 +0200)]
tests: Make sure to not call memcmp with NULL arguments.
GCC10 -fanalyzer thinks we are too clever:
elfputzdata.c: In function ‘main’:
elfputzdata.c:178:8: warning: use of possibly-NULL ‘orig_buf’ where
non-null expected [CWE-690]
[-Wanalyzer-possible-null-argument]
178 | && memcmp (orig_buf, d->d_buf, orig_size) == 0)
orig_buf can only be NULL when orig_size is zero, but it might still
be undefined behaviour. So don't try to be too smart and just check
whether we actually have an buffer.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Fri, 8 May 2020 21:35:12 +0000 (23:35 +0200)]
libdwfl: Cleanup user_core resources on failure in dwfl_core_file_report.
GCC10 -fanalyzer noticed that we allocate, but don't always cleanup the
dwfl->user_core if it wasn't set yet on error. In theory dwfl_module_end
should take care of it, but it is cleaner and less confusing to just do
it here.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Fri, 8 May 2020 10:18:41 +0000 (12:18 +0200)]
libdw: Skip imported compiler_units in libdw_visit_scopes walking DIE tree
Some gcc -flto versions imported other top-level compile units,
skip those. Otherwise we'll visit various DIE trees multiple times.
Note in the testcase that with newer GCC versions function foo is
fully inlined and does appear only once (as declared, but not as
separate subprogram).
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Fri, 8 May 2020 10:33:11 +0000 (12:33 +0200)]
libdw: Use correct CU to resolve file names in dwarf_decl_file.
dwarf_decl_file uses dwarf_attr_integrate to get the DW_AT_decl_file
attribute. This means the attribute might come from a different DIE
in a different CU. If so, we need to use the CU associated with the
attribute, not the original DIE, to resolve the file name.
Also add a bit more documentation to dwarf_attr_integrate explaining
that the attribute returned might come from a different CU (and even
different Dwarf).
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Thu, 30 Apr 2020 21:57:26 +0000 (23:57 +0200)]
libdwfl: Handle debugaltlink in dwfl_standard_find_debuginfo.
When we fall back to the debuginfod client then we need to do the
same trick we do for local lookups in dwfl_build_id_find_debuginfo.
If the debug file (dw) is already set, then we must be looking for
the altfile. But we cannot use the actual file/path name given as
hint. We'll have to lookup the alt file "build-id". Because the
debuginfod client only handles build-ids.
Previously we would use the build-id of the main file which meant
the debuginfod client would give us another copy of the debug file,
which would then be set as its own altfile. This caused lots of
confusion...
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Sun, 26 Apr 2020 00:10:41 +0000 (02:10 +0200)]
libdwfl: Fix double free on failure path in gzip.c.
GCC10 -fanalyzer found a double free when openstream failed. When
openstream fails __libdw_gunzip will call fail, which frees the
state->buffer. But openstream can call zlib_fail, which will also
call fail. Instead of calling zlib_fail, just return the error
that zlib_fail would have returned.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Sat, 25 Apr 2020 23:41:27 +0000 (01:41 +0200)]
libdw: Call Dwarf oom_handler() when malloc fails in __libdw_alloc_tail.
GCC10 -fanalyzer found a possibly-NULL dereference after a failed
malloc in __libdw_alloc_tail. In this case we should call the Dwarf
oom_handler as is done in other places where an essential malloc
call fails. The oom_handler cannot return and will likely just abort.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Sat, 25 Apr 2020 23:20:57 +0000 (01:20 +0200)]
libasm: Fix double fclose in asm_end.
GCC10 -fanalyzer found a double fclose in asm_end. asm_end can call
text_end, which calls fclose and checks for errors, then asm_end
calls __libasm_finictx which can call fclose again (but doesn't
check for errors). Call fflush in text_end instead. fflush will
generate the same error fclose would if something went wrong writing
out the file.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Fri, 24 Apr 2020 23:21:12 +0000 (01:21 +0200)]
libelf: Fix double free in __libelf_compress on error path.
In commit
2092865a7e589ff805caa47e69ac9630f34d4f2a
"libelf: {de,}compress: ensure zlib resource cleanup" we added a
call to deflate_cleanup to make sure all resources were freed.
As GCC10 -fanalyzer points out that could cause a double free
of out_buf. Fix by removing the free (out_buf) in __libelf_compress.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Fri, 17 Apr 2020 10:31:15 +0000 (12:31 +0200)]
tests: Run run-varlocs-self.sh for object files with --exprlocs.
The varlocs test relies on finding addresses for the CUs, even in
object files. This might not be true (for example when building with
gcc -flto, which only emits partial, type only, debuginfo). Split
running the self tests in running on executables and shared libraries
as normal, but run object files with --exprlocs to test that output
too (and not require addresses).
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Fri, 17 Apr 2020 10:21:46 +0000 (12:21 +0200)]
tests: Build test-nlist with minimal CFLAGS to guarantee symbol order.
test_nlist checks its own symbol table, and expects various symbols
to be in the order as specified in the source file. Explicitly set
minimal CFLAGS.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Frank Ch. Eigler [Tue, 21 Apr 2020 21:11:18 +0000 (17:11 -0400)]
debuginfod sysconfig: add /var/lib/pulp as default search path
Red Hat Satellite 6 (pulp era) stores RPMs under this hierarchy, so if
a user is fortunate enough to have debuginfod on such a machine, we
might as well index that.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Mark Wielaard [Fri, 17 Apr 2020 09:31:40 +0000 (11:31 +0200)]
libdw, readelf: Handle GCC LTO .gnu.debuglto_ prefix.
GCC puts (partial) DWARF debuginfo into sections prefixed with
.gnu.debuglto_. Handle those sections as if they are normal .debug
sections (which they are). This allows showing the DWARF that gcc
puts into ET_REL files compiled with -flto.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Thu, 16 Apr 2020 16:48:58 +0000 (18:48 +0200)]
elflint: Allow SHF_EXCLUDE as generic section flag when --gnu is given.
Strictly speaking SHF_EXCLUDE is a processor specific section flag,
but it is used generically in the GNU toolchain. For example when
adding .gnu.lto_ sections.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Thu, 16 Apr 2020 13:41:57 +0000 (15:41 +0200)]
libcpu: Replace assert with goto invalid_op for bad prefix.
https://sourceware.org/bugzilla/show_bug.cgi?id=25831
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Thu, 16 Apr 2020 15:45:31 +0000 (17:45 +0200)]
libdwfl: Initialize bits to NULL in dwfl_standard_find_debuginfo for LTO.
GCC10 LTO is too smart (and somewhat cryptic):
find-debuginfo.c: In function ‘dwfl_standard_find_debuginfo’:
debuginfod-client.c:85:8: error: ‘bits’ may be used uninitialized
in this function [-Werror=maybe-uninitialized]
find-debuginfo.c:360:24: note: ‘bits’ was declared here
lto1: all warnings being treated as errors
So it inlines __libdwfl_debuginfod_find_debuginfo into
dwfl_standard_find_debuginfo and since it cannot see into the
function pointer (*fp_debuginfod_find_debuginfo), it assumes that
build_id_bit (== bits in dwfl_standard_find_debuginfo) will be used
by the called function and it might not be initialized.
But if you read the code the there is a check for build_id_len > 0
to see whether bits is or isn't initialized before using bits.
But gcc isn't smart enough to figure that out.
Maybe that actually should be an heuristic gcc lto should use.
If the callchain I am inlining is so deep that I cannot figure out
maybe-uninitialized variables anymore I should stop inlining.
For now just help GCC out and initialize bits to NULL.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Mon, 30 Mar 2020 12:39:43 +0000 (14:39 +0200)]
config: chmod uploaded dir and files to make them readable.
Default file permissions on sourceware have changed. Make sure that
the upload-release.sh script sets the permissions right.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Mon, 30 Mar 2020 11:04:00 +0000 (13:04 +0200)]
Prepare for 0.179
Set version to 0.179.
Update NEWS and elfutils.spec.in.
Regenerate po/*.po files.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Yuri Chornoivan [Sat, 28 Mar 2020 13:03:22 +0000 (15:03 +0200)]
Update Ukrainian translation
Signed-off-by: Yuri Chornoivan <yurchor@ukr.net>
Mark Wielaard [Sun, 29 Mar 2020 22:57:30 +0000 (00:57 +0200)]
debuginfod: Document and sanity check debuginfod_add_http_header format.
Document and sanity check the format of the header string form that can
be passed to debuginfod_add_http_header. It should contain precisely
one colon, which cannot be the first or last character. And the function
should only be used to add optional headers, not replace any existing
standard ones. Anything else isn't supported.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Frank Ch. Eigler [Sun, 29 Mar 2020 19:10:37 +0000 (15:10 -0400)]
debuginfod-client default_progressfn: formatting fix
The saga of clean $DEBUGINFOD_PROGRESS=1 output continues. Previous
code would sometimes insert a \n (a blank line) into the output
stream, even if the target file was found in a cache and thus the
progressfn was never called. New code sets a client flag to track
this case more precisely.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Mark Wielaard [Sat, 28 Mar 2020 20:42:04 +0000 (16:42 -0400)]
PR25728: disable debuginfod --port=0
When starting debuginfod with --port=0 it would start using a random port
(possibly different for ipv4 and ipv6). This seems to be an accidental
and not very useful functionality. Just produce an error when started
with --port=0.
https://sourceware.org/bugzilla/show_bug.cgi?id=25728
Signed-off-by: Mark Wielaard <mark@klomp.org>
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Sat, 28 Mar 2020 19:57:09 +0000 (15:57 -0400)]
PR25739: debuginfod correct mtime for fdcache'd files
Files extracted from archives then left in the fdcache need to get get
their mtime set consistently, so that a subsequent cache-hit fetch can
relay the correct mtime to clients.
Reported-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Sat, 28 Mar 2020 00:47:45 +0000 (20:47 -0400)]
PR25722: debuginfod-find: accept /path/names in place of buildid hex
Extend the debuginfod-find command line interface to permit
/file/names instead of only buildid hexadecimal strings.
v2: move code into debuginfod-find; client API remains buildid only.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Wed, 25 Mar 2020 03:46:30 +0000 (23:46 -0400)]
debuginfod: User-Agent and X-Forwarded-For header relay
Extend the debuginfod client API with a function to stuff outgoing
headers into libcurl http transfers. Use this from debuginfod so
federated trees of debuginfod/httpd can trace back to to the
originating client for administrative purposes. Docs & test included.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Thu, 26 Mar 2020 20:44:20 +0000 (16:44 -0400)]
PR25448: debuginfod: add transfer performance metrics
We now export metrics related to the time taken and data sent,
from which prometheus type tools can compute nice time series
with averages.
http_responses_duration_milliseconds_count{code="200"} 63
http_responses_duration_milliseconds_count{code="404"} 2
http_responses_duration_milliseconds_count{code="503"} 1
http_responses_duration_milliseconds_sum{code="200"} 66
http_responses_duration_milliseconds_sum{code="404"} 2
http_responses_duration_milliseconds_sum{code="503"} 0
http_responses_transfer_bytes_count{code="200"} 63
http_responses_transfer_bytes_count{code="404"} 2
http_responses_transfer_bytes_count{code="503"} 1
http_responses_transfer_bytes_sum{code="200"} 425177
http_responses_transfer_bytes_sum{code="404"} 18
http_responses_transfer_bytes_sum{code="503"} 37
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Fri, 27 Mar 2020 01:32:40 +0000 (21:32 -0400)]
PR25583: debuginfod: conditionally prefer bsdtar to dpkg for .deb handling
Prefer /usr/bin/dpkg-deb if installed, as normal on a debian system.
Suggested-by: Mark Wielaard <mark@klomp.org>
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Wed, 25 Mar 2020 14:55:53 +0000 (10:55 -0400)]
PR25583: debuginfod: prefer bsdtar to dpkg for .deb handling
It turns out a bsdtar subshell can do the job of dpkg-deb.
bsdtar comes from/with libarchive so it should be available
everywhere.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Thu, 26 Mar 2020 17:48:20 +0000 (13:48 -0400)]
debuginfod: document and workaround fedora31 zstd compression
Old enough (even RHEL8 era) rpm/libarchive tools cannot grok fedora31
zstd-compressed rpms. Disable those tests if necessary. Document -Z
based workaround for these debuginfod users.
https://fedoraproject.org/wiki/Changes/Switch_RPMs_to_zstd_compression
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Mark Wielaard [Thu, 26 Mar 2020 17:01:29 +0000 (18:01 +0100)]
tests: Add missing debuginfod-rpms/fedora31/hello3-*.rpm to EXTRA_DIST.
The new rpms were used in the new test. Make sure they are in the dist
tar ball.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Frank Ch. Eigler [Thu, 26 Mar 2020 15:12:49 +0000 (11:12 -0400)]
PR25548: CURLOPT_PATH_AS_IS backward compatibility
old libcurl lacks the CURL_AT_LEAST_VERSION macro, so use
LIBCURL_VERSION_NUM testing instead.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Thu, 26 Mar 2020 15:12:49 +0000 (11:12 -0400)]
PR25548: CURLOPT_PATH_AS_IS backward compatibility
libcurl < 7.42 lacks the CURLOPT_PATH_AS_IS flag, but extraneous
client-side canonicalization is mostly harmless.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Reported-by: Mark Wielaard <mark@klomp.org>
Frank Ch. Eigler [Mon, 23 Mar 2020 19:33:56 +0000 (15:33 -0400)]
PR25548: support canonicalized source-path names in debuginfod webapi
Programs are sometimes compiled with source path names containing
noise like /./ or // or /foo/../, and these survive into DWARF. This
code allows either raw or canonicalized pathnames in the webapi, by
letting the client pass things verbatim, and letting the server
store/accept both raw and canonicalized path names for source files.
Tests included & docs updated.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Reported-by: Eli Schwartz <eschwartz@archlinux.org>
Tested-by: Eli Schwartz <eschwartz@archlinux.org>
Frank Ch. Eigler [Wed, 25 Mar 2020 01:57:59 +0000 (21:57 -0400)]
PR25367: improve debuginfod webapi logging
Improve debuginfod logging to show webapi query results including
http status, sizes, and processing times.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Mark Wielaard [Wed, 25 Mar 2020 22:22:55 +0000 (23:22 +0100)]
Update mailinglist subscription info.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Frank Ch. Eigler [Wed, 25 Mar 2020 01:30:02 +0000 (21:30 -0400)]
debuginfod-client thinko: non-default progressfn extra output
A previous commit changed the default_progressfn output format
to \rFOOBAR, to be terminated by an \n when the download finished.
The \n terminator was conditional on the wrong thing (env var
setting, rather than actual progressfn setting), so the \n could
be printed even if an app overrode the default.
Frank Ch. Eigler [Tue, 24 Mar 2020 19:01:58 +0000 (15:01 -0400)]
debuginfod-find: Correct error check for -v source combo
Hardcoding argv[2] is wrong in presence of -v option. Code
immediately following did the correct argv[] indexing already.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Mark Wielaard [Mon, 23 Mar 2020 22:57:51 +0000 (23:57 +0100)]
tests: Fix getphdrnum and run-lfs-symbols.sh testcase.
getphdrnum.c didn't include config.h which is why run-lfs-symbols.sh
flagged it for containing bad (non-lfs) symbols.
run-lfs-symbols.sh was still checking the libebl modules, which we
don't create anymore. But it didn't fail the test for non-existing
tests. Add some extra logging and explicitly check files exist.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Sun, 22 Mar 2020 22:14:24 +0000 (23:14 +0100)]
tests: Add testcase for elf_getphdrnum with more than 65535 entries.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Omar Sandoval [Wed, 18 Mar 2020 20:18:51 +0000 (13:18 -0700)]
libelf: handle PN_XNUM in elf_getphdrnum before shdr 0 is cached
__elf_getphdrnum_rdlock() handles PN_XNUM by getting sh_info from
elf->state.elf{32,64}.scns.data[0].shdr.e{32,64}. However, that is only
a cache that may or may not have been populated by elf_begin() or
elf{32,64}_getshdr(); if it hasn't been cached yet, elf_getphdrnum()
returns 65535 (the value of PN_XNUM) instead. We should explicitly get
the shdr if it isn't cached.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Frank Ch. Eigler [Sat, 21 Mar 2020 01:33:52 +0000 (21:33 -0400)]
debuginfod client API: add get_url function
This function lets a client know, during or after a progressfn
callback, what the url of the winning outgoing download is/was.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Fri, 20 Mar 2020 00:27:11 +0000 (20:27 -0400)]
debuginfod client API: add get/set user_data functions
Add a pair of functions to associate a void* parameter with a client
object. Requested by GDB team as a way to pass file names and such
user-interface data through to a progressfn callback.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Matthias Maennich [Fri, 20 Mar 2020 16:38:55 +0000 (17:38 +0100)]
libelf: {de,}compress: ensure zlib resource cleanup
__libelf_decompress would only cleanup zlib resources via inflateEnd()
in case inflating was successful, but would leak memory if not. Fix this
by calling inflateEnd() unconditionally.
__libelf_decompress did this all the time already, but called
deflateEnd() twice. That is not a (known) issue, but can be cleaned up
by ensuring all error paths use 'return deflate_cleanup' and the success
path calls deflateEnd() only once. Note, the deflate() needs to return
Z_STREAM_END to indicate we are done. Hence change the condition.
Fixes:
272018bba1f2 ("libelf: Add elf_compress and elf_compress_gnu.")
Signed-off-by: Matthias Maennich <maennich@google.com>
Aaron Merey [Tue, 3 Mar 2020 19:11:48 +0000 (14:11 -0500)]
debuginfod-client: Update cache_path when the new default path exists
Update cache_path with the path of the new default directory when this
directory already exists. Previously cache_path was updated only when
creating the new default directory and would otherwise be set to the
old default path.
Signed-off-by: Aaron Merey <amerey@redhat.com>
Mark Wielaard [Wed, 4 Mar 2020 10:35:38 +0000 (11:35 +0100)]
config: Remove rpm, add dpkg as Requires for debuginfod package in spec.
We no longer require rpm2cpio to unpack rpms.
We do now require dpkg-deb to unpack debs.
https://sourceware.org/bugzilla/show_bug.cgi?id=25583
Signed-off-by: Mark Wielaard <mark@klomp.org>
Aaron Merey [Thu, 27 Feb 2020 23:10:01 +0000 (18:10 -0500)]
debuginfod-client: default to XDG cache.
PR25502: debuginfod client should default to XDG cache
Location of client cache now defaults to $XDG_CACHE_HOME/debuginfod_client.
If XDG_CACHE_HOME is not set then fallback to $HOME/.cache/debuginfod_client.
Also maintain backwards compatibility with the previous default path-
if $HOME/.debuginfod_client_cache exists, use that instead of the new
defaults.
Signed-off-by: Aaron Merey <amerey@redhat.com>
Konrad Kleine [Wed, 26 Feb 2020 15:00:43 +0000 (10:00 -0500)]
debuginfod: file:// URLs: handle curl resp. code
When file:// is used for DEBUGINFOD_URLS, then the response code for a
successful server query is 0 and not 200.
Using file:// can be helpful when you want to test your
debuginfod-client integration against a mocked file tree that mimics the
HTTP URLs from the debuginfod server. This way you don't have to run the
debuginfod server at all.
Fixes https://sourceware.org/bugzilla/show_bug.cgi?id=25600
Signed-off-by: Konrad Kleine <kkleine@redhat.com>
Frank Ch. Eigler [Tue, 25 Feb 2020 19:27:33 +0000 (14:27 -0500)]
debuginfod PR25583: map -R to -Z.rpm
It was reported that libarchive (bsdtar) at least as far back as rhel7
(3.1.2) can natively process RPM files, so there's no need to mediate
those accesses through rpm2cpio. There's no noteworthy performance or
testing impact.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Signed-off-by: Mark Wielaard <mark@klomp.org>
Frank Ch. Eigler [Sat, 25 Jan 2020 23:43:07 +0000 (18:43 -0500)]
PR25375: fdcache prefetching to reduce repeated archive decompression
Introduce new option --fdcache-prefetch to accelerate repeated
queries from the same debuginfo archive.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Mark Wielaard [Fri, 21 Feb 2020 12:34:09 +0000 (13:34 +0100)]
tests: Explicitly unset DEBUGINFOD_URLS.
If DEBUGINFOD_URLS is set various tests will try to query the debuginfod
server which can stall the tests a bit. If other evironment variables
like DEBUGINFOD_PROGRESS are set it will make various tests fail because
the expected output doesn't match. Tests should PASS without needing a
debuginfod server, unless they test (and set) one themselves.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Aaron Merey [Tue, 11 Feb 2020 20:54:55 +0000 (15:54 -0500)]
PR25365: debuginfod-client: restrict cleanup to client-pattern files
Avoid deleting general files and directories in case a user
mis-sets $DEBUGINFOD_CACHE_PATH or accidentally moves a file
into the cache.
Signed-off-by: Aaron Merey <amerey@redhat.com>
Mark Wielaard [Sat, 8 Feb 2020 20:40:27 +0000 (21:40 +0100)]
readelf, elflint: Handle PT_GNU_PROPERTY.
binutils 2.32 ld emits a new PT_GNU_PROPERTY segment that overlaps
with the PT_NOTE segment covering the .note.gnu.property section data.
https://sourceware.org/bugzilla/show_bug.cgi?id=25511
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Fri, 7 Feb 2020 17:20:46 +0000 (18:20 +0100)]
elflint: Check sh_info for symtab isn't too big.
The sh_info field of the symtab says how many symbols are in the
section. Make sure at least that many symbols fit in the section.
Reported-by: Ulrich Drepper <drepper@redhat.com>
Signed-off-by: Mark Wielaard <mark@klomp.org>
Konrad Kleine [Wed, 12 Feb 2020 13:03:03 +0000 (14:03 +0100)]
debuginfod/doc: Fix typo debuginfo_progressfn_t -> debuginfod_progressfn_t
I've copied the type name for the progress function from the manual and
noticed that it didn't compile...
Signed-off-by: Konrad Kleine <kkleine@redhat.com>
Frank Ch. Eigler [Mon, 10 Feb 2020 19:33:54 +0000 (14:33 -0500)]
debuginfod testing: SIGUSR2 vs "groom" metric synch
Previous code did not account for a groom job that was already
completed at startup, so the SIGUSR2-triggered one may not have
completed in time for the test. The shell won the race condition on
most buildbot VMs, but a debian builder showed the error of our ways.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Thu, 6 Feb 2020 03:22:13 +0000 (22:22 -0500)]
debuginfod: testsuite for general archive (arch-linux) support
Tested analogously to how RPMs are.
Signed-Off-By: Frank Ch. Eigler <fche@redhat.com>
Eli Schwartz [Wed, 5 Feb 2020 22:09:30 +0000 (17:09 -0500)]
debuginfod arch-linux test binaries
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Mark Wielaard [Fri, 7 Feb 2020 23:17:25 +0000 (18:17 -0500)]
debuginfod: archive processing: handle -Z EXT=cat with direct fopen
Signed-off-by: Mark Wielaard <mark@klomp.org>
Frank Ch. Eigler [Wed, 5 Feb 2020 20:04:18 +0000 (15:04 -0500)]
debuginfod: generalized archive support
Add a '-Z EXT[=CMD]' option to debuginfod, which lets it scan any given
extension and run CMD on it to unwrap distro archives. For example,
for arch-linux pacman files, -Z '.tar.zst=zstdcat' lets debuginfod
grok debug and source content in split-debuginfo files.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Konrad Kleine [Mon, 10 Feb 2020 16:03:02 +0000 (17:03 +0100)]
debuginfod.h: fix typo in comment: legnth -> length
Signed-off-by: Konrad Kleine <kkleine@redhat.com>
Mark Wielaard [Fri, 24 Jan 2020 21:55:48 +0000 (22:55 +0100)]
libdwfl: Fix some GCC10 -Wnull-dereference issues.
GCC10 on some arches will warn about possible NULL derefences.
In the libdwfl linux-kernel-modules.c cases it might be caught already
by earlier calls to get_release (). But it is hard to see that will
really always happen. So do an explicit NULL check just in case.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Frank Ch. Eigler [Mon, 3 Feb 2020 19:21:15 +0000 (14:21 -0500)]
ChangeLog catchup for two recent commits
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Mon, 3 Feb 2020 19:11:53 +0000 (14:11 -0500)]
debuginfod test: assert curl & rpm2cpio during test prologue
On debian & ubuntu hosts, rpm2cpio is not available by default,
and heck, curl might not be either. Check for both these programs
during tests/run-debuginfod-find.sh startup, else exit-77 (skip).
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Mon, 3 Feb 2020 11:29:31 +0000 (06:29 -0500)]
PR24393: debuginfod configury: tolerate CXX= configure invocations
Previous code was fragile with respect to this case and needlessly
tolerant with non-c++11 compilers. New configury makes more muscular
assertion about this, to catch bad $CXX settings or poor language
support.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Wed, 22 Jan 2020 20:27:59 +0000 (15:27 -0500)]
debuginfod: diagnostics verbosity tweak
marxin reported "skipping hat= .... empty comp_dir" messages
were too chatty for the journnal.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Wed, 22 Jan 2020 18:22:31 +0000 (13:22 -0500)]
empty commit for irker test
Frank Ch. Eigler [Mon, 20 Jan 2020 20:37:33 +0000 (15:37 -0500)]
PR25394 cont'd: debuginfod testsuite fix for -USR1 timing
If a SIGUSR1 is sent before the initial traversal, it no longer
results in an extra traversal. That's a sensible effect. The
test case just needs to wait before the kill -USR1.
Frank Ch. Eigler [Mon, 20 Jan 2020 20:37:33 +0000 (15:37 -0500)]
PR25394 cont'd: debuginfod timing fix for fts-traversal thread
The new code neglected to set the last_rescan timestamp, leading
to overly frequent rescanning.
Frank Ch. Eigler [Mon, 20 Jan 2020 19:44:15 +0000 (14:44 -0500)]
PR25394 followup#2: debuginfod casting fixes
Buildbot still reports type warnings in time_t arithmetic.
Explicit (long)er cast pushed as obvious ... or is it? :-)
Frank Ch. Eigler [Mon, 20 Jan 2020 19:44:15 +0000 (14:44 -0500)]
PR25394 followup: debuginfod casting fixes
Buildbot reports type warnings in time_t arithmetic.
Explicit (long) cast pushed as obvious.
Frank Ch. Eigler [Mon, 20 Jan 2020 01:33:32 +0000 (20:33 -0500)]
PR25394: debuginfod mutex between grooming and scanning
Extended the work-queue concept with "idlers" - other threads that
block on the work queue until it becomes empty (rather than normal
consumers that block on it until it becomes non-empty). Use this
facility for the groomer thread to avoid working at the same time as
the scanner threads. Use this for the fts traversal thread for
similar reasons. One user-visible effect: response to SIGUSR1 and
SIGUSR2 will wait until the work queue runs empty, but the man page
was unspecific so does not need changing.
It's not obvious how to test this with a tests/ dataset so small that
scanning takes negligible time, so the former races are very tight.
P.S. We also evaluated using sqlite level transactions to isolate the
scanner thread groups-of-operations from the groomer. These
experiments failed to produce a nominally concurrent debuginfod,
having triggered "database locked" type errors. So we remain
single-threaded (fully serialized) at the sqlite API level.
Frank Ch. Eigler [Sat, 18 Jan 2020 20:32:40 +0000 (15:32 -0500)]
PR25415: fix debuginfod SEGV for empty search path list
Prevent triggering undefined behaviour of fts(3), which causes
a SEGV on fedora glibc 2.29.
Mark Wielaard [Thu, 16 Jan 2020 22:33:52 +0000 (23:33 +0100)]
nm: Fix nm --external sysv format output.
Partial revert of commit
66f4c37d497bdde040a33f299b12163f044b1bf2.
If index zero wasn't a real symbol it has already been filtered out
in show_symbols so don't skip it in show_symbols_sysv.
https://sourceware.org/bugzilla/show_bug.cgi?id=25227
Reported-by: Enzo Matsumiya <ematsumiya@suse.com>
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Fri, 10 Jan 2020 14:46:29 +0000 (15:46 +0100)]
doc: Fix DEBUGINFOD_PROGRESS description to just mention output on stderr.
An earlier variant of the default progress function could write to any
file. Which is still in the documentation. But the actual implementation
just uses stderr. Fix the documentation to match.
Signed-off-by: Mark Wielaard <mark@klomp.org>
Mark Wielaard [Wed, 8 Jan 2020 14:04:50 +0000 (15:04 +0100)]
libasm.h: Don't include libebl.h. Define an opaque Ebl handle.
Using libasm isn't really usable without a way to create an Ebl handle.
But we don't support libebl.h (and libebl itself). Just define the
Ebl handle as an opaque struct. Code that uses it needs to figure out
how to instantiate one itself (they cannot in any supportable way...)
Signed-off-by: Mark Wielaard <mark@klomp.org>
Frank Ch. Eigler [Sun, 12 Jan 2020 16:04:57 +0000 (11:04 -0500)]
elfutils.spec: typo fix
Removed dangling %endif.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Sun, 12 Jan 2020 16:00:20 +0000 (11:00 -0500)]
elfutils.spec: dependency typo fix
Commit
da0959cafda4 introduced a dependency on "debuginfod-client",
whereas it's really "elfutils-debuginfod-client".
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Fri, 27 Dec 2019 00:06:46 +0000 (19:06 -0500)]
debuginfod: extracted-from-archive file cache
Add a facility to service webapi and dwz/altdebug requests that
resolve to archives via a $TMPDIR file cache. This permits
instantaneous dwz resolution during -debuginfo rpm scanning, and also
instantanous duplicate webapi requests. The cache is limited both in
number of entries and in storage space. Heuristics provide
serviceable defaults.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Sat, 11 Jan 2020 21:05:46 +0000 (16:05 -0500)]
debuginfod: print U-A: and X-F-F: request headers
For an incoming webapi request, print two headers that should assist
in the administration of a debuginfod service. At fweimer's
suggestion, added a bit of filtering so the text is more reliably
parseable.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Frank Ch. Eigler [Wed, 1 Jan 2020 01:06:30 +0000 (20:06 -0500)]
debuginfod: rework threading model for file/archive scanning
We switch from a thread per supplied PATH, with a semaphore based
concurrency control, to a fixed number of worker threads collecting
the result of a plain directory traversal being put into a work queue.
This allows maximal continuous concurrency, even if the PATH
directories are dramatically differently sized. There is no more need
to use concurrency-motivated subdirectory wildcards for PATH entries:
just a single top level directory will work fast. doc & tests incl.