Milian Wolff [Tue, 9 Dec 2014 10:18:08 +0000 (11:18 +0100)]
Add compile-check around unw_set_cache_log_size.
Milian Wolff [Tue, 9 Dec 2014 01:11:24 +0000 (02:11 +0100)]
Silence warning when debuggee quit already.
Milian Wolff [Tue, 9 Dec 2014 01:08:55 +0000 (02:08 +0100)]
Restore original function addresses when shutting down injected heaptrack.
This should bring back the original behavior and performance.
And we might also be able to re-attach then if desired.
Milian Wolff [Tue, 9 Dec 2014 00:51:06 +0000 (01:51 +0100)]
Return output stream
Milian Wolff [Tue, 9 Dec 2014 00:49:49 +0000 (01:49 +0100)]
Merge branch 'inject'
Milian Wolff [Tue, 9 Dec 2014 00:45:24 +0000 (01:45 +0100)]
Stop heaptrack when heaptrack_interpret goes away.
Milian Wolff [Tue, 9 Dec 2014 00:13:57 +0000 (01:13 +0100)]
Add -p/--pid option to attach heaptrack to a running process.
We use gdb to attach to the process, then call dlopen there and
finally an initialization hook in the new libheaptrack_inject.so.
To prevent an underflow in the total memory consumption, we
call malloc_info and parse that in heaptrack_interpret to get a
baseline for the current total memory consumption at the point
where we attached to the process.
The code is refactored into a shared libheaptrack.cpp with common
code, and two libraries, libheaptrack_inject.so and the old
libheaptrack_preload.so. The heaptrack bash script is adapted to
support both versions.
Milian Wolff [Mon, 8 Dec 2014 21:13:37 +0000 (22:13 +0100)]
Restructure argument parsing in shell script.
This makes it easier to maintain and the arguments become position-
independent, which is very useful.
Milian Wolff [Mon, 8 Dec 2014 11:19:29 +0000 (12:19 +0100)]
Get rid of temporary string allocations.
Milian Wolff [Mon, 8 Dec 2014 11:09:05 +0000 (12:09 +0100)]
Use lambdas instead of passing printf labels with magic numbers.
This also gets rid of useless string allocations to format the byte
sizes of data members that are never printed.
Milian Wolff [Mon, 8 Dec 2014 10:26:09 +0000 (11:26 +0100)]
Cleanup/fix: We add +1 above already, so no need to do it again.
Milian Wolff [Sun, 7 Dec 2014 14:53:29 +0000 (15:53 +0100)]
Use mprotect on addresses we want to overwrite.
This seems to be more reliable than the other method.
Milian Wolff [Sat, 6 Dec 2014 14:39:36 +0000 (15:39 +0100)]
Also overload posix_memalign.
Milian Wolff [Sat, 6 Dec 2014 14:35:07 +0000 (15:35 +0100)]
Overwrite realloc, calloc and cfree at runtime.
Milian Wolff [Sat, 6 Dec 2014 14:27:25 +0000 (15:27 +0100)]
Overwrite dlopen/dlclose as well at runtime.
Milian Wolff [Sat, 6 Dec 2014 13:51:09 +0000 (14:51 +0100)]
Ensure the hook function is convertible to the original function.
Milian Wolff [Sat, 6 Dec 2014 12:46:36 +0000 (13:46 +0100)]
Further cleanup the code
Milian Wolff [Sat, 6 Dec 2014 12:42:02 +0000 (13:42 +0100)]
Make the hook list easier extensible.
Milian Wolff [Sat, 6 Dec 2014 12:35:34 +0000 (13:35 +0100)]
Cleanup code a bit.
Milian Wolff [Fri, 5 Dec 2014 23:50:02 +0000 (00:50 +0100)]
Also overwrite free.
Milian Wolff [Fri, 5 Dec 2014 23:43:42 +0000 (00:43 +0100)]
Make runtime-patching work!
Wow, awesome. What was missing where the offsets, i.e. the dlpi_addr
must be added to the p_vaddr, as well as the r_offset. Or so it seems.
Now I don't see any crashes anymore, and my custom malloc gest called!
Milian Wolff [Fri, 5 Dec 2014 23:06:19 +0000 (00:06 +0100)]
first work towards runtime injection, not really working yet
Milian Wolff [Fri, 5 Dec 2014 20:35:32 +0000 (21:35 +0100)]
Remove spurious define, only required on some faulty machine of mine.
Milian Wolff [Thu, 4 Dec 2014 18:24:05 +0000 (19:24 +0100)]
Add LGPL v2.1+ text in COPYING file.
Milian Wolff [Wed, 3 Dec 2014 16:11:39 +0000 (17:11 +0100)]
Fix line information written by heaptrack_interpret.
The line numbers where written in decimal format, but heaptrack_print
expects hexadecimal numbers when reading. This lead to completely
wrong line numbers in the end.
Milian Wolff [Wed, 3 Dec 2014 11:07:27 +0000 (12:07 +0100)]
Always show the biggest heap in the total memory consumption.
This still has some issues, but is better than nothing.
Milian Wolff [Tue, 2 Dec 2014 14:11:48 +0000 (15:11 +0100)]
Add some platform checks to give better error messages at cmake time.
Milian Wolff [Tue, 2 Dec 2014 13:51:04 +0000 (14:51 +0100)]
Skip anything below main in massif backtraces.
Milian Wolff [Tue, 2 Dec 2014 13:48:58 +0000 (14:48 +0100)]
Properly handle case where __libc_start_main is encountered before main itself.
We still want to break in main.
Milian Wolff [Tue, 2 Dec 2014 13:34:29 +0000 (14:34 +0100)]
Sort massif output file.
This is expected by e.g. the massif-visualizer and probably other
tools as well.
Milian Wolff [Tue, 2 Dec 2014 11:22:19 +0000 (12:22 +0100)]
Introduce tunable massif threshold and detailed freq parameters.
Milian Wolff [Tue, 2 Dec 2014 03:30:53 +0000 (04:30 +0100)]
Print full backtraces in generated massif files.
These files easily become quite large. We really need our own
heaptrack-visualizer to efficiently read the data and handle it.
Milian Wolff [Tue, 2 Dec 2014 02:36:34 +0000 (03:36 +0100)]
Print first level of backtrace in generated massif files.
Milian Wolff [Tue, 2 Dec 2014 01:56:00 +0000 (02:56 +0100)]
Basic support for massif output file generation.
So far, it only includes the total heap memory usage. Backtraces
will be added in the next step.
Milian Wolff [Tue, 2 Dec 2014 01:54:57 +0000 (02:54 +0100)]
Use thread instead of signal and increase timestamp frequency.
Sadly, this creates a new thread for every timer apparently,
but still, it's better than using signals which randomly influence
other functions and sometimes fubar our output files. Or so I think.
Milian Wolff [Tue, 2 Dec 2014 01:00:23 +0000 (02:00 +0100)]
Remember the debugee command line.
This is sometimes useful and also tracked by massif and other tools.
Milian Wolff [Tue, 2 Dec 2014 00:32:02 +0000 (01:32 +0100)]
Simplify code and write exe path directly to stream.
No need for a temporary string allocation here.
Milian Wolff [Tue, 2 Dec 2014 00:26:40 +0000 (01:26 +0100)]
Simplify code, it's not worth the imagined performance gain.
Milian Wolff [Fri, 28 Nov 2014 15:53:06 +0000 (16:53 +0100)]
Unset automatic locking of stdout and stdin.
This is not required in the heaptrack_interpret process.
Milian Wolff [Fri, 28 Nov 2014 15:50:04 +0000 (16:50 +0100)]
Prefer C I/O API for speed.
It's sad that this gives such a noticeable speedup :(
Milian Wolff [Fri, 28 Nov 2014 15:43:35 +0000 (16:43 +0100)]
Use fputs if we don't need to format anything.
Milian Wolff [Fri, 28 Nov 2014 15:22:44 +0000 (16:22 +0100)]
Cleanup module handling code and change the file format a bit.
We now output a single line per module and delegate the whole
interpretation of the dl_phdr_info data to heaptrack_interpret.
This allows us to reduce the size of Module a bit, leading to hope-
fully faster binary searches amongst other benefits.
Milian Wolff [Mon, 1 Dec 2014 14:50:10 +0000 (15:50 +0100)]
Remove code that depends on local libunwind patches.
Milian Wolff [Thu, 27 Nov 2014 17:23:44 +0000 (18:23 +0100)]
Fixup handling of posix_memaling.
The return value of zero indicates success here. This should fix
some inconsistencies on applications that use this function.
Milian Wolff [Thu, 27 Nov 2014 17:23:38 +0000 (18:23 +0100)]
Don't print 0B leaked lines.
Milian Wolff [Thu, 27 Nov 2014 17:19:36 +0000 (18:19 +0100)]
Properly take cfree into account.
Milian Wolff [Thu, 27 Nov 2014 16:46:11 +0000 (17:46 +0100)]
Ensure we only ever initialize once.
Milian Wolff [Thu, 27 Nov 2014 16:40:10 +0000 (17:40 +0100)]
Use the same error handler for both init error callbacks.
This happens rarely, and the error message is still helpful.
Milian Wolff [Thu, 27 Nov 2014 16:37:39 +0000 (17:37 +0100)]
Fixup callback handlers, the wrong data was passed in.
Milian Wolff [Thu, 27 Nov 2014 16:30:58 +0000 (17:30 +0100)]
Cleanup: get rid of obsolete isExe member in module.
It is only required for the call to backtrace_fileline_initialize.
Milian Wolff [Thu, 27 Nov 2014 14:50:16 +0000 (15:50 +0100)]
Add seconds elapsed to heaptrack log for future evaluation.
Milian Wolff [Fri, 17 Oct 2014 12:59:58 +0000 (14:59 +0200)]
Support building against libunwind from different include directory
Milian Wolff [Tue, 25 Nov 2014 16:12:05 +0000 (17:12 +0100)]
Delete obsolete files.
These are used in VOGL, but not in heaptrack, so get rid of them.
Milian Wolff [Tue, 25 Nov 2014 15:55:22 +0000 (16:55 +0100)]
Support atomic functions if possible.
Milian Wolff [Mon, 24 Nov 2014 16:22:40 +0000 (17:22 +0100)]
Prevent crashes in apps using QProcess.
Milian Wolff [Wed, 19 Nov 2014 12:33:01 +0000 (13:33 +0100)]
Prevent issues and file corruption when tracking a forking process.
Thankfully there is pthread_atfork which can be used to get notified
about fork calls. We stop heaptrack before forking and only continue
it in the parent process. In the child process we stop heaptrack
alltogether.
TODO: make it possible to track child processes.
Milian Wolff [Tue, 18 Nov 2014 19:04:47 +0000 (20:04 +0100)]
Also use LineReader in heaptrack_interpret.
Milian Wolff [Tue, 18 Nov 2014 18:44:20 +0000 (19:44 +0100)]
Be more forgiving when encountering bad data.
Still no clue where the bad data comes from though...
Milian Wolff [Tue, 18 Nov 2014 17:44:58 +0000 (18:44 +0100)]
Optimize findAllocation calls by leveraging monotonicity of indices.
By comparing the incoming trace index to the largest index we so
far encountered, we can already decide whether:
a) if the index is equal to the largest one, the same location is used
repeatedly, i.e. in a loop, and we can directly return the last item.
b) if the index is larger than the last known one, we'll definitely
not find it and can thus insert directly and return
c) only if the index is lower than the last index do we actually need
to do a binary search.
This yields a small but noticeable speedup from ~14.5s to 13.4s for
one of my files.
Milian Wolff [Tue, 18 Nov 2014 17:01:12 +0000 (18:01 +0100)]
Slightly cleanup and optimize LineReader::readHex
Milian Wolff [Tue, 18 Nov 2014 16:42:27 +0000 (17:42 +0100)]
minor cleanup
Milian Wolff [Tue, 18 Nov 2014 16:41:20 +0000 (17:41 +0100)]
Don't fill sizeHistogram if we are not going to print it.
Milian Wolff [Tue, 18 Nov 2014 16:28:13 +0000 (17:28 +0100)]
Introduce optimized reader class for faster reading of hex numbers.
In my benchmark on a big data file this reduces the runtime of
heaptrack_print from 24s to 16s. The whole function is pretty simple
so I'm willing to sacrifice simplicity for this big performance win.
Milian Wolff [Tue, 18 Nov 2014 15:40:35 +0000 (16:40 +0100)]
Improve error checking when reading data file.
Milian Wolff [Tue, 18 Nov 2014 15:03:17 +0000 (16:03 +0100)]
Format byte sizes to KB/MB/GB/TB.
Milian Wolff [Tue, 18 Nov 2014 13:10:18 +0000 (14:10 +0100)]
Fallback to __libc_start_main when main is not found.
Milian Wolff [Tue, 18 Nov 2014 13:10:07 +0000 (14:10 +0100)]
Change the output format slightly for better readability.
Milian Wolff [Tue, 18 Nov 2014 12:44:54 +0000 (13:44 +0100)]
Merge allocations from equivalent locations.
Equivalent means the data is the same except for the instruction pointer
address itself. Useful when we are missing some debug information.
Milian Wolff [Tue, 18 Nov 2014 11:09:59 +0000 (12:09 +0100)]
Cleanup code and also initialize fileline only once.
This fixes a regression when looking at kdepim with heaptrack.
Milian Wolff [Mon, 17 Nov 2014 19:18:09 +0000 (20:18 +0100)]
Reduce looping
Milian Wolff [Mon, 17 Nov 2014 18:51:23 +0000 (19:51 +0100)]
Only call backtrace_create_state once per file.
Also improve the checks for overlapping libraries which was wrong.
Combined this drastically cuts the memory consumption and improve
the runtime of the heaptrack_interpret process again.
Milian Wolff [Mon, 17 Nov 2014 17:59:34 +0000 (18:59 +0100)]
Fix more issues related to improper backtrace symbol lookups.
We now clear the list of known modules also in heaptrack_interpret
and really forget about all modules. This ensures we don't keep
invalid modules around and improperly associate instruction pointer
addresses to these modules.
This fixes the heaptrack usage for me on larger applications such
as KMail or plasma-desktop (KDE4).
Milian Wolff [Mon, 17 Nov 2014 17:32:48 +0000 (18:32 +0100)]
Take each PT_LOAD dlpi_phdr as its own module.
This fixes some issues with incorrect backtrace symbols when shared
libraries are used, as the new example shows. Before e.g. libc.so
overlapped with many other libraries and incorrectly chosen to
find the backtrace symbol. With this patch, this issue is gone.
Milian Wolff [Mon, 17 Nov 2014 16:16:57 +0000 (17:16 +0100)]
Add whitespace before lib name.
Milian Wolff [Mon, 17 Nov 2014 16:16:45 +0000 (17:16 +0100)]
Fix warning about missing debug infos in linux-vdso.so.N
Milian Wolff [Mon, 8 Sep 2014 08:37:23 +0000 (10:37 +0200)]
Check for main index while reading the data in.
Milian Wolff [Mon, 8 Sep 2014 08:23:16 +0000 (10:23 +0200)]
Skip operator new etc. when encountering the trace.
This makes it much simpler and all backtraces are filtered automatically.
Milian Wolff [Sat, 6 Sep 2014 22:42:18 +0000 (00:42 +0200)]
Write histogram to file specified by cli argument
Milian Wolff [Sat, 6 Sep 2014 22:24:05 +0000 (00:24 +0200)]
Improve the output to better understand what the numbers mean.
David Faure [Sat, 6 Sep 2014 21:22:24 +0000 (23:22 +0200)]
Fix compilation on OpenSUSE
David Faure [Sat, 6 Sep 2014 21:05:34 +0000 (23:05 +0200)]
Fix compilation when passing flags on the command line as well
Milian Wolff [Fri, 5 Sep 2014 08:18:25 +0000 (10:18 +0200)]
Skip all operator new locations, even when the IP differs.
This was occurring in a real-world data file for me, where
operator new(unsigned long) was encountered twice at different IPs.
Milian Wolff [Thu, 4 Sep 2014 15:57:41 +0000 (17:57 +0200)]
Don't get confused by operator<, <<, > and >>.
Milian Wolff [Tue, 26 Aug 2014 00:44:08 +0000 (02:44 +0200)]
Share code.
Milian Wolff [Tue, 26 Aug 2014 00:37:58 +0000 (02:37 +0200)]
Make it possible to switch between merged and non-merged backtraces.
Milian Wolff [Tue, 26 Aug 2014 00:30:18 +0000 (02:30 +0200)]
Add some comments, esp. about implementing better trace merging.
Milian Wolff [Tue, 26 Aug 2014 00:26:23 +0000 (02:26 +0200)]
remove obsolete debug output
Milian Wolff [Tue, 26 Aug 2014 00:21:05 +0000 (02:21 +0200)]
Add implicit values to program options.
Milian Wolff [Tue, 26 Aug 2014 00:16:43 +0000 (02:16 +0200)]
Merge code to print merged backtraces :)
C++' pointer-to-member is a really powerful feature - wow!
Milian Wolff [Tue, 26 Aug 2014 00:01:56 +0000 (02:01 +0200)]
Prettify output of top-allocator places.
TODO: do something similar in the other places as well.
Milian Wolff [Mon, 25 Aug 2014 23:45:03 +0000 (01:45 +0200)]
Merge allocation data by instruction pointer.
This is safe to do except for the peak memory consumption where
we cannot ignore the temporal dependency.
Otherwise this greatly improves the output as multiple calls to the
same function from different places get merged and thus show up
as one fat data point, instead of multiple thinner ones.
Milian Wolff [Mon, 25 Aug 2014 23:04:13 +0000 (01:04 +0200)]
Skip operator new(unsigned long) (and new[]) from start of backtraces.
Milian Wolff [Mon, 25 Aug 2014 22:21:52 +0000 (00:21 +0200)]
Make indices typesafe.
Milian Wolff [Mon, 25 Aug 2014 22:01:51 +0000 (00:01 +0200)]
Fix name of variables.
Milian Wolff [Mon, 25 Aug 2014 21:56:32 +0000 (23:56 +0200)]
Add a bit of recursion to the callgraph test debugee
Milian Wolff [Mon, 25 Aug 2014 21:51:58 +0000 (23:51 +0200)]
Fix regression and properly stop backtraces at main function again.
Milian Wolff [Mon, 25 Aug 2014 21:39:17 +0000 (23:39 +0200)]
Cleanup code and reduce duplication.
Milian Wolff [Mon, 25 Aug 2014 21:33:11 +0000 (23:33 +0200)]
Split trace and instruction pointer data.
The debug info for a ip is only required once, whereas a trace
contains the mapping between an IP and the parent IP, which can
be outputted many times in different combinations.
This way we don't call the libbacktrace functions as often and
decrease the file size, at the cost of memory consumption during
the debugee run (in heaptrack_interpret). The memory consumption
significantly decreases in heaptrack_print though.
Milian Wolff [Mon, 25 Aug 2014 20:19:55 +0000 (22:19 +0200)]
Reduce amount of cout flushing.
std::endl also flushes the output which is pretty bad for performance.
Instead, only use endl when it makes sense, i.e. between output blocks.
Milian Wolff [Mon, 25 Aug 2014 20:17:21 +0000 (22:17 +0200)]
Cleanup code: encapsulate data reading and make more stuff private.