sdk/tools/heaptrack.git
9 years agoUse thread instead of signal and increase timestamp frequency.
Milian Wolff [Tue, 2 Dec 2014 01:54:57 +0000 (02:54 +0100)]
Use thread instead of signal and increase timestamp frequency.

Sadly, this creates a new thread for every timer apparently,
but still, it's better than using signals which randomly influence
other functions and sometimes fubar our output files. Or so I think.

9 years agoRemember the debugee command line.
Milian Wolff [Tue, 2 Dec 2014 01:00:23 +0000 (02:00 +0100)]
Remember the debugee command line.

This is sometimes useful and also tracked by massif and other tools.

9 years agoSimplify code and write exe path directly to stream.
Milian Wolff [Tue, 2 Dec 2014 00:32:02 +0000 (01:32 +0100)]
Simplify code and write exe path directly to stream.

No need for a temporary string allocation here.

9 years agoSimplify code, it's not worth the imagined performance gain.
Milian Wolff [Tue, 2 Dec 2014 00:26:40 +0000 (01:26 +0100)]
Simplify code, it's not worth the imagined performance gain.

9 years agoUnset automatic locking of stdout and stdin.
Milian Wolff [Fri, 28 Nov 2014 15:53:06 +0000 (16:53 +0100)]
Unset automatic locking of stdout and stdin.

This is not required in the heaptrack_interpret process.

9 years agoPrefer C I/O API for speed.
Milian Wolff [Fri, 28 Nov 2014 15:50:04 +0000 (16:50 +0100)]
Prefer C I/O API for speed.

It's sad that this gives such a noticeable speedup :(

9 years agoUse fputs if we don't need to format anything.
Milian Wolff [Fri, 28 Nov 2014 15:43:35 +0000 (16:43 +0100)]
Use fputs if we don't need to format anything.

9 years agoCleanup module handling code and change the file format a bit.
Milian Wolff [Fri, 28 Nov 2014 15:22:44 +0000 (16:22 +0100)]
Cleanup module handling code and change the file format a bit.

We now output a single line per module and delegate the whole
interpretation of the dl_phdr_info data to heaptrack_interpret.

This allows us to reduce the size of Module a bit, leading to hope-
fully faster binary searches amongst other benefits.

9 years agoRemove code that depends on local libunwind patches.
Milian Wolff [Mon, 1 Dec 2014 14:50:10 +0000 (15:50 +0100)]
Remove code that depends on local libunwind patches.

9 years agoFixup handling of posix_memaling.
Milian Wolff [Thu, 27 Nov 2014 17:23:44 +0000 (18:23 +0100)]
Fixup handling of posix_memaling.

The return value of zero indicates success here. This should fix
some inconsistencies on applications that use this function.

9 years agoDon't print 0B leaked lines.
Milian Wolff [Thu, 27 Nov 2014 17:23:38 +0000 (18:23 +0100)]
Don't print 0B leaked lines.

9 years agoProperly take cfree into account.
Milian Wolff [Thu, 27 Nov 2014 17:19:36 +0000 (18:19 +0100)]
Properly take cfree into account.

9 years agoEnsure we only ever initialize once.
Milian Wolff [Thu, 27 Nov 2014 16:46:11 +0000 (17:46 +0100)]
Ensure we only ever initialize once.

9 years agoUse the same error handler for both init error callbacks.
Milian Wolff [Thu, 27 Nov 2014 16:40:10 +0000 (17:40 +0100)]
Use the same error handler for both init error callbacks.

This happens rarely, and the error message is still helpful.

9 years agoFixup callback handlers, the wrong data was passed in.
Milian Wolff [Thu, 27 Nov 2014 16:37:39 +0000 (17:37 +0100)]
Fixup callback handlers, the wrong data was passed in.

9 years agoCleanup: get rid of obsolete isExe member in module.
Milian Wolff [Thu, 27 Nov 2014 16:30:58 +0000 (17:30 +0100)]
Cleanup: get rid of obsolete isExe member in module.

It is only required for the call to backtrace_fileline_initialize.

9 years agoAdd seconds elapsed to heaptrack log for future evaluation.
Milian Wolff [Thu, 27 Nov 2014 14:50:16 +0000 (15:50 +0100)]
Add seconds elapsed to heaptrack log for future evaluation.

9 years agoSupport building against libunwind from different include directory
Milian Wolff [Fri, 17 Oct 2014 12:59:58 +0000 (14:59 +0200)]
Support building against libunwind from different include directory

9 years agoDelete obsolete files.
Milian Wolff [Tue, 25 Nov 2014 16:12:05 +0000 (17:12 +0100)]
Delete obsolete files.

These are used in VOGL, but not in heaptrack, so get rid of them.

9 years agoSupport atomic functions if possible.
Milian Wolff [Tue, 25 Nov 2014 15:55:22 +0000 (16:55 +0100)]
Support atomic functions if possible.

9 years agoPrevent crashes in apps using QProcess.
Milian Wolff [Mon, 24 Nov 2014 16:22:40 +0000 (17:22 +0100)]
Prevent crashes in apps using QProcess.

9 years agoPrevent issues and file corruption when tracking a forking process.
Milian Wolff [Wed, 19 Nov 2014 12:33:01 +0000 (13:33 +0100)]
Prevent issues and file corruption when tracking a forking process.

Thankfully there is pthread_atfork which can be used to get notified
about fork calls. We stop heaptrack before forking and only continue
it in the parent process. In the child process we stop heaptrack
alltogether.

TODO: make it possible to track child processes.

9 years agoAlso use LineReader in heaptrack_interpret.
Milian Wolff [Tue, 18 Nov 2014 19:04:47 +0000 (20:04 +0100)]
Also use LineReader in heaptrack_interpret.

9 years agoBe more forgiving when encountering bad data.
Milian Wolff [Tue, 18 Nov 2014 18:44:20 +0000 (19:44 +0100)]
Be more forgiving when encountering bad data.

Still no clue where the bad data comes from though...

9 years agoOptimize findAllocation calls by leveraging monotonicity of indices.
Milian Wolff [Tue, 18 Nov 2014 17:44:58 +0000 (18:44 +0100)]
Optimize findAllocation calls by leveraging monotonicity of indices.

By comparing the incoming trace index to the largest index we so
far encountered, we can already decide whether:

a) if the index is equal to the largest one, the same location is used
repeatedly, i.e. in a loop, and we can directly return the last item.
b) if the index is larger than the last known one, we'll definitely
not find it and can thus insert directly and return
c) only if the index is lower than the last index do we actually need
to do a binary search.

This yields a small but noticeable speedup from ~14.5s to 13.4s for
one of my files.

9 years agoSlightly cleanup and optimize LineReader::readHex
Milian Wolff [Tue, 18 Nov 2014 17:01:12 +0000 (18:01 +0100)]
Slightly cleanup and optimize LineReader::readHex

9 years agominor cleanup
Milian Wolff [Tue, 18 Nov 2014 16:42:27 +0000 (17:42 +0100)]
minor cleanup

9 years agoDon't fill sizeHistogram if we are not going to print it.
Milian Wolff [Tue, 18 Nov 2014 16:41:20 +0000 (17:41 +0100)]
Don't fill sizeHistogram if we are not going to print it.

9 years agoIntroduce optimized reader class for faster reading of hex numbers.
Milian Wolff [Tue, 18 Nov 2014 16:28:13 +0000 (17:28 +0100)]
Introduce optimized reader class for faster reading of hex numbers.

In my benchmark on a big data file this reduces the runtime of
heaptrack_print from 24s to 16s. The whole function is pretty simple
so I'm willing to sacrifice simplicity for this big performance win.

9 years agoImprove error checking when reading data file.
Milian Wolff [Tue, 18 Nov 2014 15:40:35 +0000 (16:40 +0100)]
Improve error checking when reading data file.

9 years agoFormat byte sizes to KB/MB/GB/TB.
Milian Wolff [Tue, 18 Nov 2014 15:03:17 +0000 (16:03 +0100)]
Format byte sizes to KB/MB/GB/TB.

9 years agoFallback to __libc_start_main when main is not found.
Milian Wolff [Tue, 18 Nov 2014 13:10:18 +0000 (14:10 +0100)]
Fallback to __libc_start_main when main is not found.

9 years agoChange the output format slightly for better readability.
Milian Wolff [Tue, 18 Nov 2014 13:10:07 +0000 (14:10 +0100)]
Change the output format slightly for better readability.

9 years agoMerge allocations from equivalent locations.
Milian Wolff [Tue, 18 Nov 2014 12:44:54 +0000 (13:44 +0100)]
Merge allocations from equivalent locations.

Equivalent means the data is the same except for the instruction pointer
address itself. Useful when we are missing some debug information.

9 years agoCleanup code and also initialize fileline only once.
Milian Wolff [Tue, 18 Nov 2014 11:09:59 +0000 (12:09 +0100)]
Cleanup code and also initialize fileline only once.

This fixes a regression when looking at kdepim with heaptrack.

9 years agoReduce looping
Milian Wolff [Mon, 17 Nov 2014 19:18:09 +0000 (20:18 +0100)]
Reduce looping

9 years agoOnly call backtrace_create_state once per file.
Milian Wolff [Mon, 17 Nov 2014 18:51:23 +0000 (19:51 +0100)]
Only call backtrace_create_state once per file.

Also improve the checks for overlapping libraries which was wrong.

Combined this drastically cuts the memory consumption and improve
the runtime of the heaptrack_interpret process again.

9 years agoFix more issues related to improper backtrace symbol lookups.
Milian Wolff [Mon, 17 Nov 2014 17:59:34 +0000 (18:59 +0100)]
Fix more issues related to improper backtrace symbol lookups.

We now clear the list of known modules also in heaptrack_interpret
and really forget about all modules. This ensures we don't keep
invalid modules around and improperly associate instruction pointer
addresses to these modules.

This fixes the heaptrack usage for me on larger applications such
as KMail or plasma-desktop (KDE4).

9 years agoTake each PT_LOAD dlpi_phdr as its own module.
Milian Wolff [Mon, 17 Nov 2014 17:32:48 +0000 (18:32 +0100)]
Take each PT_LOAD dlpi_phdr as its own module.

This fixes some issues with incorrect backtrace symbols when shared
libraries are used, as the new example shows. Before e.g. libc.so
overlapped with many other libraries and incorrectly chosen to
find the backtrace symbol. With this patch, this issue is gone.

9 years agoAdd whitespace before lib name.
Milian Wolff [Mon, 17 Nov 2014 16:16:57 +0000 (17:16 +0100)]
Add whitespace before lib name.

9 years agoFix warning about missing debug infos in linux-vdso.so.N
Milian Wolff [Mon, 17 Nov 2014 16:16:45 +0000 (17:16 +0100)]
Fix warning about missing debug infos in linux-vdso.so.N

9 years agoCheck for main index while reading the data in.
Milian Wolff [Mon, 8 Sep 2014 08:37:23 +0000 (10:37 +0200)]
Check for main index while reading the data in.

9 years agoSkip operator new etc. when encountering the trace.
Milian Wolff [Mon, 8 Sep 2014 08:23:16 +0000 (10:23 +0200)]
Skip operator new etc. when encountering the trace.

This makes it much simpler and all backtraces are filtered automatically.

9 years agoWrite histogram to file specified by cli argument
Milian Wolff [Sat, 6 Sep 2014 22:42:18 +0000 (00:42 +0200)]
Write histogram to file specified by cli argument

9 years agoImprove the output to better understand what the numbers mean.
Milian Wolff [Sat, 6 Sep 2014 22:24:05 +0000 (00:24 +0200)]
Improve the output to better understand what the numbers mean.

9 years agoFix compilation on OpenSUSE
David Faure [Sat, 6 Sep 2014 21:22:24 +0000 (23:22 +0200)]
Fix compilation on OpenSUSE

9 years agoFix compilation when passing flags on the command line as well
David Faure [Sat, 6 Sep 2014 21:05:34 +0000 (23:05 +0200)]
Fix compilation when passing flags on the command line as well

9 years agoSkip all operator new locations, even when the IP differs.
Milian Wolff [Fri, 5 Sep 2014 08:18:25 +0000 (10:18 +0200)]
Skip all operator new locations, even when the IP differs.

This was occurring in a real-world data file for me, where
operator new(unsigned long) was encountered twice at different IPs.

9 years agoDon't get confused by operator<, <<, > and >>.
Milian Wolff [Thu, 4 Sep 2014 15:57:41 +0000 (17:57 +0200)]
Don't get confused by operator<, <<, > and >>.

9 years agoShare code.
Milian Wolff [Tue, 26 Aug 2014 00:44:08 +0000 (02:44 +0200)]
Share code.

9 years agoMake it possible to switch between merged and non-merged backtraces.
Milian Wolff [Tue, 26 Aug 2014 00:37:58 +0000 (02:37 +0200)]
Make it possible to switch between merged and non-merged backtraces.

9 years agoAdd some comments, esp. about implementing better trace merging.
Milian Wolff [Tue, 26 Aug 2014 00:30:18 +0000 (02:30 +0200)]
Add some comments, esp. about implementing better trace merging.

9 years agoremove obsolete debug output
Milian Wolff [Tue, 26 Aug 2014 00:26:23 +0000 (02:26 +0200)]
remove obsolete debug output

9 years agoAdd implicit values to program options.
Milian Wolff [Tue, 26 Aug 2014 00:21:05 +0000 (02:21 +0200)]
Add implicit values to program options.

9 years agoMerge code to print merged backtraces :)
Milian Wolff [Tue, 26 Aug 2014 00:16:43 +0000 (02:16 +0200)]
Merge code to print merged backtraces :)

C++' pointer-to-member is a really powerful feature - wow!

9 years agoPrettify output of top-allocator places.
Milian Wolff [Tue, 26 Aug 2014 00:01:56 +0000 (02:01 +0200)]
Prettify output of top-allocator places.

TODO: do something similar in the other places as well.

9 years agoMerge allocation data by instruction pointer.
Milian Wolff [Mon, 25 Aug 2014 23:45:03 +0000 (01:45 +0200)]
Merge allocation data by instruction pointer.

This is safe to do except for the peak memory consumption where
we cannot ignore the temporal dependency.

Otherwise this greatly improves the output as multiple calls to the
same function from different places get merged and thus show up
as one fat data point, instead of multiple thinner ones.

9 years agoSkip operator new(unsigned long) (and new[]) from start of backtraces.
Milian Wolff [Mon, 25 Aug 2014 23:04:13 +0000 (01:04 +0200)]
Skip operator new(unsigned long) (and new[]) from start of backtraces.

9 years agoMake indices typesafe.
Milian Wolff [Mon, 25 Aug 2014 22:21:52 +0000 (00:21 +0200)]
Make indices typesafe.

9 years agoFix name of variables.
Milian Wolff [Mon, 25 Aug 2014 22:01:51 +0000 (00:01 +0200)]
Fix name of variables.

9 years agoAdd a bit of recursion to the callgraph test debugee
Milian Wolff [Mon, 25 Aug 2014 21:56:32 +0000 (23:56 +0200)]
Add a bit of recursion to the callgraph test debugee

9 years agoFix regression and properly stop backtraces at main function again.
Milian Wolff [Mon, 25 Aug 2014 21:51:58 +0000 (23:51 +0200)]
Fix regression and properly stop backtraces at main function again.

9 years agoCleanup code and reduce duplication.
Milian Wolff [Mon, 25 Aug 2014 21:39:17 +0000 (23:39 +0200)]
Cleanup code and reduce duplication.

9 years agoSplit trace and instruction pointer data.
Milian Wolff [Mon, 25 Aug 2014 21:33:11 +0000 (23:33 +0200)]
Split trace and instruction pointer data.

The debug info for a ip is only required once, whereas a trace
contains the mapping between an IP and the parent IP, which can
be outputted many times in different combinations.

This way we don't call the libbacktrace functions as often and
decrease the file size, at the cost of memory consumption during
the debugee run (in heaptrack_interpret). The memory consumption
significantly decreases in heaptrack_print though.

9 years agoReduce amount of cout flushing.
Milian Wolff [Mon, 25 Aug 2014 20:19:55 +0000 (22:19 +0200)]
Reduce amount of cout flushing.

std::endl also flushes the output which is pretty bad for performance.
Instead, only use endl when it makes sense, i.e. between output blocks.

9 years agoCleanup code: encapsulate data reading and make more stuff private.
Milian Wolff [Mon, 25 Aug 2014 20:17:21 +0000 (22:17 +0200)]
Cleanup code: encapsulate data reading and make more stuff private.

9 years agoAdd test for call graphs, this should be merged better.
Milian Wolff [Sun, 24 Aug 2014 14:45:06 +0000 (16:45 +0200)]
Add test for call graphs, this should be merged better.

9 years agoImprove headings of different output sections.
Milian Wolff [Mon, 25 Aug 2014 17:18:12 +0000 (19:18 +0200)]
Improve headings of different output sections.

9 years agoTake dlclose into account and update module cache accordingly.
Milian Wolff [Mon, 25 Aug 2014 16:52:16 +0000 (18:52 +0200)]
Take dlclose into account and update module cache accordingly.

9 years agoMake it possible to validate the correctness of allocation overloads.
Milian Wolff [Mon, 25 Aug 2014 16:38:04 +0000 (18:38 +0200)]
Make it possible to validate the correctness of allocation overloads.

This is still flaky, apparently some function related to TLS allocation is missing.

9 years agoImprove output of -h
Milian Wolff [Mon, 25 Aug 2014 16:06:13 +0000 (18:06 +0200)]
Improve output of -h

9 years agoDo not clash --print-histogram with --help.
Milian Wolff [Mon, 25 Aug 2014 16:06:04 +0000 (18:06 +0200)]
Do not clash --print-histogram with --help.

9 years agoDo not link directly to libheaptrack.
Milian Wolff [Mon, 25 Aug 2014 15:57:47 +0000 (17:57 +0200)]
Do not link directly to libheaptrack.

The interpreter is important for useful data imo.

9 years agoShorten template identifiers by default.
Milian Wolff [Mon, 25 Aug 2014 15:57:28 +0000 (17:57 +0200)]
Shorten template identifiers by default.

9 years agoUse boost::program_options and make output configurable.
Milian Wolff [Mon, 25 Aug 2014 15:57:00 +0000 (17:57 +0200)]
Use boost::program_options and make output configurable.

9 years agoSimplify code, we use the std namespace.
Milian Wolff [Mon, 25 Aug 2014 13:46:37 +0000 (15:46 +0200)]
Simplify code, we use the std namespace.

9 years agoSimplify code a bit.
Milian Wolff [Mon, 25 Aug 2014 13:42:12 +0000 (15:42 +0200)]
Simplify code a bit.

9 years agoOutput pointer adresses without 0x prefix.
Milian Wolff [Mon, 25 Aug 2014 13:35:07 +0000 (15:35 +0200)]
Output pointer adresses without 0x prefix.

This just needlessly increases the file size.
We now cast to uintptr_t and output as %lx.

9 years agoDo not output function, file and line index for unknown locations.
Milian Wolff [Mon, 25 Aug 2014 13:24:41 +0000 (15:24 +0200)]
Do not output function, file and line index for unknown locations.

The null value is the default anyways and we do not need to
output that information and can thereby decrease the filesize a bit
for debugees with many occasions of missing debug information.

9 years agoAlso show individual peak allocations.
Milian Wolff [Sat, 23 Aug 2014 15:30:48 +0000 (17:30 +0200)]
Also show individual peak allocations.

9 years agoAlso show top-ten allocation places (ignoring deallocations).
Milian Wolff [Sat, 23 Aug 2014 15:23:55 +0000 (17:23 +0200)]
Also show top-ten allocation places (ignoring deallocations).

9 years agoUse %p to print out ptr addresses.
Milian Wolff [Sat, 23 Aug 2014 15:16:48 +0000 (17:16 +0200)]
Use %p to print out ptr addresses.

9 years agoAdd some debug output to show whats currently going on.
Milian Wolff [Mon, 11 Aug 2014 22:12:05 +0000 (00:12 +0200)]
Add some debug output to show whats currently going on.

9 years agoOptimize: don't sync cin/cout/cerr with stdio for our single-threaded c++ apps.
Milian Wolff [Wed, 13 Aug 2014 14:00:18 +0000 (16:00 +0200)]
Optimize: don't sync cin/cout/cerr with stdio for our single-threaded c++ apps.

9 years agoCleanup code a bit.
Milian Wolff [Wed, 13 Aug 2014 12:11:30 +0000 (14:11 +0200)]
Cleanup code a bit.

9 years agoAdd comment on why we use stdio here.
Milian Wolff [Wed, 13 Aug 2014 09:55:25 +0000 (11:55 +0200)]
Add comment on why we use stdio here.

9 years agoFixup check for executable debuggee.
Milian Wolff [Wed, 13 Aug 2014 08:52:17 +0000 (10:52 +0200)]
Fixup check for executable debuggee.

Call which first to also find the exe in PATH.

9 years agoAdd heaptrack --help.
Milian Wolff [Sun, 10 Aug 2014 15:15:12 +0000 (17:15 +0200)]
Add heaptrack --help.

9 years agoCleanup output messages to make it simpler to understand heaptrack.
Milian Wolff [Sun, 10 Aug 2014 14:57:49 +0000 (16:57 +0200)]
Cleanup output messages to make it simpler to understand heaptrack.

9 years agoInstall heaptrack shell script and use the proper paths therein.
Milian Wolff [Sun, 10 Aug 2014 14:50:57 +0000 (16:50 +0200)]
Install heaptrack shell script and use the proper paths therein.

We get the relative paths from the install directory to our lib
and libexec target folders and write that into the shell script.

9 years agoSome more cleanup and restructuring.
Milian Wolff [Wed, 6 Aug 2014 21:19:14 +0000 (23:19 +0200)]
Some more cleanup and restructuring.

9 years agoRename to heaptrack and start cleaning up the code structure.
Milian Wolff [Wed, 6 Aug 2014 20:49:54 +0000 (22:49 +0200)]
Rename to heaptrack and start cleaning up the code structure.

9 years agoRemove mallocinfo code, it's in the old repo.
Milian Wolff [Wed, 6 Aug 2014 20:31:56 +0000 (22:31 +0200)]
Remove mallocinfo code, it's in the old repo.

9 years agoDirectly evaluate the backtrace data.
Milian Wolff [Sun, 3 Aug 2014 19:22:47 +0000 (21:22 +0200)]
Directly evaluate the backtrace data.

This makes the final data interpretable even when the binary or
any library changes in between. It also makes it possible to ship
the data file to an external computer for further evaluation.

10 years agoHide anything above __libc_start_main as its garbage.
Milian Wolff [Sat, 7 Jun 2014 00:45:28 +0000 (02:45 +0200)]
Hide anything above __libc_start_main as its garbage.

10 years agoUse named pipe to compress output data on the fly.
Milian Wolff [Thu, 5 Jun 2014 13:31:48 +0000 (15:31 +0200)]
Use named pipe to compress output data on the fly.

This is plain awesome imo :) Really easy to get implemented and
the evaluation code is also more or less untouched thanks to Boost's
iostreams library with nice support for gzip data.

10 years agoFixup example code, use proper delete[].
Milian Wolff [Thu, 5 Jun 2014 10:22:23 +0000 (12:22 +0200)]
Fixup example code, use proper delete[].

10 years agoDo not output new IP id.
Milian Wolff [Wed, 4 Jun 2014 17:47:02 +0000 (19:47 +0200)]
Do not output new IP id.

The data is outputted in a sequentail way and thus this information is obsolete.

10 years agoDirectly link malloctrace into test executables.
Milian Wolff [Wed, 4 Jun 2014 17:43:08 +0000 (19:43 +0200)]
Directly link malloctrace into test executables.

10 years agoOutput all numbers in hex form.
Milian Wolff [Wed, 4 Jun 2014 16:37:43 +0000 (18:37 +0200)]
Output all numbers in hex form.

This reduces the output data file size slightly and simplifies the
evaluation code as we do not have to switch between different
formats all the time.