Milian Wolff [Fri, 17 Jun 2016 11:56:36 +0000 (13:56 +0200)]
Add some more code comments for the diffing method.
Milian Wolff [Tue, 14 Jun 2016 21:45:36 +0000 (23:45 +0200)]
Convert more data types to signed integers.
We now also calculate the difference between time, peakRSS and
system info when diffing heaptrack data files.
Milian Wolff [Tue, 31 May 2016 22:06:12 +0000 (00:06 +0200)]
Add diff support between files to heaptrack_{print,gui}.
Both executables get a new `--diff <file>` argument. The file
that is passed gets parsed in addition to the primary data file.
The allocation data therein will then be subtracted from the data
found in the primary data file.
The algorithm is somewhat complicated, because we have to ignore
instruction pointer addresses, as these will vary between runs
due to address space randomization. Paired with (partial) lack of
debug symbols, we end up with non-trivial situation to map allocations
from one run to that of another.
Essentially, we need to find the correct allocation that shares the
same backtrace while ignoring instruction pointer addresses.
Milian Wolff [Mon, 30 May 2016 19:27:23 +0000 (21:27 +0200)]
Refactor code to use signed data types.
This is required to handle diffing between files or between
individual timestamps. Since we use 64bit data types, we
can still handle a lot of allocations this way (a bit more than
9 exabytes).
Milian Wolff [Tue, 31 May 2016 22:06:02 +0000 (00:06 +0200)]
Add minus operator to AllocationData
Milian Wolff [Wed, 8 Jun 2016 16:27:34 +0000 (18:27 +0200)]
Make it possible to skip building of heaptrack_gui.
For embedded, we never want to build the gui, and having the
find scripts spit out their errors there is just noisy.
Now, the recommended way to build heaptrack for embedded Linux is:
cmake -DCMAKE_TOOLCHAIN_FILE=... -DHEAPTRACK_BUILD_GUI=OFF
Milian Wolff [Wed, 8 Jun 2016 15:16:27 +0000 (17:16 +0200)]
Refactor code to remove reliance on undefined behavior.
Previously, the code got away with a c-style reinterpret_cast
in a constexpr function, which is not allowed according to the
standard. I think I was just lucky that the compiler didn't saw
through that yet.
Also, the code assumed (and enforced) that all function pointers
are sizeof(void*), which worked, but was a bit clunky.
Now, we simply get rid of the list of addresses, and iterate
manually over all hooks. With C++17's apply, this could be implemented
even nicer with a tuple and std::apply. That doesn't exist yet, so
we manually call the functions.
Milian Wolff [Wed, 8 Jun 2016 13:08:16 +0000 (15:08 +0200)]
Slightly cleanup code of preload hooks.
We instantiate directly with the decltype of a function
pointer now, so thereis no need to construct the pointer
via a secondary type anymore.
Milian Wolff [Wed, 8 Jun 2016 12:48:59 +0000 (14:48 +0200)]
Increase portability to systems that don't have cfree available.
cfree should not be used, according to its man page, but if it's
being used then we need to overload it. On systems where it is not
available, simply skip this step to keep heaptrack compiling.
Milian Wolff [Sat, 4 Jun 2016 17:36:02 +0000 (19:36 +0200)]
Track temporary allocations by their AllocationInfo index.
There is no need to output a boolean '1' for temporary allocations,
as we have the same information when looking at the AllocationInfo
index that corresponds to a unique size and location. I.e. we used
to output
+ 12
- 12 1
+ 42
- 43
But the same information is held by
+ 12
- 12
+ 42
- 43
Milian Wolff [Fri, 3 Jun 2016 19:29:54 +0000 (21:29 +0200)]
Move MergedAllocation struct to heaptrack_print.
It's only being used there.
Milian Wolff [Thu, 2 Jun 2016 10:02:07 +0000 (12:02 +0200)]
Make heaptrack.sh shell script work with dash.
Milian Wolff [Thu, 2 Jun 2016 10:01:29 +0000 (12:01 +0200)]
Improve bash compatibility of heaptrack shell script.
- don't use case fall-through via ;;&
- access /proc/$pid/comm directly
Milian Wolff [Sun, 29 May 2016 18:05:10 +0000 (20:05 +0200)]
Update display text when navigation through history.
Milian Wolff [Sun, 29 May 2016 17:44:32 +0000 (19:44 +0200)]
Make text in labels selectable by mouse.
Milian Wolff [Sun, 29 May 2016 17:39:51 +0000 (19:39 +0200)]
Make it possible to go back/forward in flamegraph via keyboard.
We keep a simple list of items around that got selected and use
that when the user presses the standard back/forward keys to
reset the view state accordingly.
Milian Wolff [Tue, 10 May 2016 08:21:07 +0000 (10:21 +0200)]
Fix compile: include cmath and use std::round.
Milian Wolff [Thu, 28 Apr 2016 17:11:43 +0000 (19:11 +0200)]
Add a dock widget with the important graphs.
This allows the data to be visible when looking at other data
and that makes it easier to correlate then.
Milian Wolff [Thu, 28 Apr 2016 16:13:23 +0000 (18:13 +0200)]
Improve tooltips on total data of charts.
Frederik Schwarzer [Sun, 10 Apr 2016 18:10:23 +0000 (20:10 +0200)]
Remove wrong unit. formatByteSize() returns the right one.
Milian Wolff [Sat, 9 Apr 2016 12:02:02 +0000 (14:02 +0200)]
Add tooltips for more UI elements in the flame graph view.
Milian Wolff [Sat, 9 Apr 2016 11:29:43 +0000 (13:29 +0200)]
Set cursors to show that you can interact with the flame graph.
Milian Wolff [Sat, 9 Apr 2016 11:16:23 +0000 (13:16 +0200)]
Use monospace font for function labels in description.
Note: <tt> uses the wrong font, afaik that got fixed for 5.6.1 or 5.7.
Milian Wolff [Sat, 9 Apr 2016 11:15:06 +0000 (13:15 +0200)]
Enable wordwrap to prevent layout issues on wide function labels.
Stephen Kelly [Wed, 6 Apr 2016 14:41:00 +0000 (16:41 +0200)]
Move the frame description to the bottom of the ui
Otherwise the tooltip fades away too quickly, and often the mouse cursor covers
and hides the number at the start of it.
Stephen Kelly [Wed, 6 Apr 2016 09:48:50 +0000 (11:48 +0200)]
Replace rigid layout with a QSplitter
Milian Wolff [Tue, 5 Apr 2016 21:39:19 +0000 (23:39 +0200)]
HTML-escape strings in tooltips, esp. required for C++ templates.
Milian Wolff [Tue, 5 Apr 2016 21:34:25 +0000 (23:34 +0200)]
Fix wrong conditional that lead to excessive i18n calls.
Milian Wolff [Tue, 5 Apr 2016 21:33:40 +0000 (23:33 +0200)]
Cache QPen and QBrush used by ChartModel.
This removes quite a few allocations that would otherwise happen
when the data is queried by KDiagram.
Milian Wolff [Tue, 5 Apr 2016 20:50:30 +0000 (22:50 +0200)]
Add relative cost numbers to the tree model tooltips.
Milian Wolff [Tue, 5 Apr 2016 20:39:34 +0000 (22:39 +0200)]
Show cost fractions in flamegraph tooltips.
Milian Wolff [Tue, 5 Apr 2016 20:30:23 +0000 (22:30 +0200)]
Introduce cost threshold for flamegraph to minimize memory overhead.
For larger data files, the flamegraph can easily consume hundreds
of megabytes of ram and take multiple seconds to be build up. This
is mostly due to the dozens of small leaf items, all of which only
have tiny cost and will not be shown most of the time.
The new default cut-off of 0.1% drastically reduces the memory
overhead and the flamegraph will be build nearly instantly even for
larger data sets.
In the future, one could even refine the data on-demand.
Milian Wolff [Thu, 24 Mar 2016 23:07:10 +0000 (00:07 +0100)]
Add an experimental perf->heaptrack data converter.
The resulting data file is much smaller than a perf.data file, and
can be visualized with heaptrack_gui. Of course, the labels will be
all off, but this is a good first step towards visualizing perf data
files with the heaptrack code infrastructure.
Milian Wolff [Thu, 24 Mar 2016 22:49:19 +0000 (23:49 +0100)]
Prevent off-by-one data access in chart models.
Milian Wolff [Thu, 24 Mar 2016 21:34:08 +0000 (22:34 +0100)]
Make code more error resilient when parsing data files.
Milian Wolff [Thu, 24 Mar 2016 21:33:36 +0000 (22:33 +0100)]
Check validity of allocation index when parsing data file.
Milian Wolff [Thu, 24 Mar 2016 19:33:10 +0000 (20:33 +0100)]
Never divide by zero.
Milian Wolff [Thu, 24 Mar 2016 19:31:27 +0000 (20:31 +0100)]
Intern caller/callee data by location data.
Milian Wolff [Thu, 24 Mar 2016 18:21:54 +0000 (19:21 +0100)]
Add caller/callee table view of data.
Milian Wolff [Thu, 24 Mar 2016 17:08:07 +0000 (18:08 +0100)]
Reuse common AllocationData class, with operator+=, operator+.
Simplifies the code in some areas.
Milian Wolff [Wed, 23 Mar 2016 15:09:35 +0000 (16:09 +0100)]
Make it possible to select different cost sources for the flame graph.
Also put the bottom-down action into the new control toolbar.
Milian Wolff [Wed, 23 Mar 2016 14:14:57 +0000 (15:14 +0100)]
Don't print unknown file/line location in tooltip.
Milian Wolff [Tue, 15 Mar 2016 19:18:00 +0000 (20:18 +0100)]
Colorize model cost cells background based on relative cost to total.
Maniphest Task: https://phabricator.kde.org/T1709
Milian Wolff [Wed, 9 Mar 2016 17:42:33 +0000 (18:42 +0100)]
Properly skip 32bit versions of operator new.
Milian Wolff [Wed, 9 Mar 2016 17:20:38 +0000 (18:20 +0100)]
Fix compile warnings on 32bit machines
Milian Wolff [Wed, 9 Mar 2016 17:15:47 +0000 (18:15 +0100)]
Update libbacktrace from GCC trunk@234085.
Milian Wolff [Mon, 29 Feb 2016 13:39:46 +0000 (14:39 +0100)]
Don't show functions that triggered no temporary allocations in top list.
Milian Wolff [Fri, 26 Feb 2016 20:47:30 +0000 (21:47 +0100)]
Add executable to measure malloc overhead.
Milian Wolff [Fri, 26 Feb 2016 17:15:44 +0000 (18:15 +0100)]
Filter rows that did not leak anything in TopProxy.
Milian Wolff [Thu, 25 Feb 2016 19:44:39 +0000 (20:44 +0100)]
Include total system memory and peak RSS usage in output.
We now track the RSS by parsing /proc/self/statm on every timestamp.
The total system memory is taken from sysconf.
The RSS may potentially miss its true peaks, which can be handled in
the future via getrusage(). Still, this is a nice and useful addition
I think.
Milian Wolff [Mon, 22 Feb 2016 12:56:37 +0000 (13:56 +0100)]
Show minimal statistics directly after heaptrack finished.
Milian Wolff [Wed, 24 Feb 2016 01:40:53 +0000 (02:40 +0100)]
Pass correct data to dl_iterate_phdr.
I wonder why this didn't crash before...
Milian Wolff [Wed, 24 Feb 2016 01:31:21 +0000 (02:31 +0100)]
Don't access invalid memory for tiny or broken input files.
Milian Wolff [Wed, 24 Feb 2016 00:58:06 +0000 (01:58 +0100)]
Put summary text into three columns to save vertical space.
Milian Wolff [Wed, 24 Feb 2016 00:41:32 +0000 (01:41 +0100)]
Display the time when the memory peak was encountered.
Milian Wolff [Wed, 24 Feb 2016 00:37:03 +0000 (01:37 +0100)]
Remove resize-to-contents call.
Milian Wolff [Wed, 24 Feb 2016 00:24:47 +0000 (01:24 +0100)]
Add summary tab and display all top functions there.
Milian Wolff [Tue, 23 Feb 2016 22:59:56 +0000 (23:59 +0100)]
Show error when we failed to parse an input file.
Milian Wolff [Tue, 23 Feb 2016 22:48:45 +0000 (23:48 +0100)]
Use monospace font family in tooltips.
Bug got fixed upstream in Qt but for older Qt this improves
readability significantly.
Milian Wolff [Tue, 23 Feb 2016 22:48:10 +0000 (23:48 +0100)]
Don't show 'backtrace: ' without a backtrace in the tooltip.
Milian Wolff [Tue, 23 Feb 2016 22:45:45 +0000 (23:45 +0100)]
Add stacks model to display full backtrace for selection.
Milian Wolff [Tue, 23 Feb 2016 22:21:09 +0000 (23:21 +0100)]
Display shorter basename for files in tables by default.
Milian Wolff [Tue, 9 Feb 2016 20:17:48 +0000 (21:17 +0100)]
Don't descend into children of invisible flamegraph items.
Milian Wolff [Fri, 22 Jan 2016 12:26:12 +0000 (13:26 +0100)]
Optimize TreeModel: cache KFormat to prevent repeated allocations.
Aleix Pol [Thu, 28 Jan 2016 13:57:37 +0000 (14:57 +0100)]
Let KDE*CompilerSettings change the policies in the project
As suggested in ECM documentation.
Milian Wolff [Thu, 21 Jan 2016 20:03:54 +0000 (21:03 +0100)]
Decrease indentation which allows us to show deeper backtraces with less space to the left.
There is still work left to be done to have a nice GUI for
deep backtraces, potentially by not increasing the indentation
depth when the merged backtrace has no forks.
Milian Wolff [Fri, 15 Jan 2016 13:06:37 +0000 (14:06 +0100)]
Make chart tooltips better readable.
Milian Wolff [Mon, 4 Jan 2016 21:41:47 +0000 (22:41 +0100)]
Use std::tie instead of std::make_tuple for lexicographical ordering.
std::tie takes the arguments by reference, whereas make_tuple copies
them. We want the former.
See also: http://www.cppsamples.com/common-tasks/lexicographic-ordering.html
Milian Wolff [Wed, 16 Dec 2015 12:37:22 +0000 (13:37 +0100)]
Show data as soon as it is available.
This way one can investigate the data table while waiting for the
charts to be built up, which can take some time for large data
files.
Milian Wolff [Wed, 16 Dec 2015 11:56:54 +0000 (12:56 +0100)]
Sort cost columns initially in descending order.
Milian Wolff [Wed, 16 Dec 2015 11:56:42 +0000 (12:56 +0100)]
Add some indication on the file load process.
Milian Wolff [Wed, 16 Dec 2015 11:30:42 +0000 (12:30 +0100)]
Display pretty-printed byte sizes in tree model.
Milian Wolff [Wed, 16 Dec 2015 10:59:08 +0000 (11:59 +0100)]
Set proper axis type for temporary allocation plot.
Milian Wolff [Wed, 16 Dec 2015 10:34:49 +0000 (11:34 +0100)]
Don't duplicate allocation infos when reparsing.
Milian Wolff [Tue, 15 Dec 2015 17:34:30 +0000 (18:34 +0100)]
Reuse PointerMap for backwards compatibility.
This also makes the size histogram work with older data files.
Milian Wolff [Tue, 15 Dec 2015 16:46:04 +0000 (17:46 +0100)]
Add size histogram to heaptrack_gui.
This tracks the number of times allocations of a certain size are
requested and displays the data in a histogram. Note that the sizes
are binned in the following byte ranges:
0-8
9-16
17-32
33-64
65-128
129-256
257-512
513-1024
>1024
Milian Wolff [Mon, 14 Dec 2015 17:48:03 +0000 (18:48 +0100)]
Reject data files of newer heaptrack versions.
Milian Wolff [Mon, 14 Dec 2015 17:24:34 +0000 (18:24 +0100)]
Optimize tracking of active allocations.
Instead of mapping the pointer to a pair of allocated size and
trace index, we intern the pair and map the pointer to a 32bit index.
This assumes that not more than 4.294.967.295 different pairs occur,
which could theoretically be broken by allocating different sizes
in a loop. Practically, I've never seen this happen. If it really
breaks we can always bump it to a 64bit index later.
Furthermore, this patch introduces a new PointerMap, which drastically
reduces the memory overhead of tracking the allocations. The benchmark
which also gets added here, shows that its overhead is only ~20%
compared to the 100% overhead of a simple hash map. Also note that
even google's sparse_hash_map only gets down to 50% overhead.
Even better, the PointerMap implementation is faster than google's
sparse_hash_map. This is possible by leveraging some information about
memory allocations, which return pointers to pages. Thus the pointers
are clustered and we can shrink a 64bit pointer to an common shared
base pointer and a small 16bit offset.
On my machine, the runtime performance is close to that of the simple
hash map. As such, I decided to move this allocation tracking into
heaptrack_interpret itself. This drastically reduces the file size
of heaptrack data files, sometimes cutting the size into half. Even
better, this speeds some heaptrack benchmarks, as less data is written
to disk. And of course all of this leads to an much improved
performance of heaptrack_gui and heaptrack_print.
Milian Wolff [Fri, 11 Dec 2015 20:34:33 +0000 (21:34 +0100)]
Also make the number of sub traces of peaks configurable.
David Landell [Fri, 11 Dec 2015 20:30:37 +0000 (21:30 +0100)]
Make number of reported peaks configurable
Make it possible to control number of reported peaks in heaptrack_print
REVIEW: 126282
From
98a061bee3879fcb5c01728d63c39b6773705108 Mon Sep 17 00:00:00 2001
From: David Landell <david.landell.dl@gmail.com>
Date: Wed, 2 Dec 2015 19:04:59 +0100
Subject: [PATCH] Make number of reported peaks configurable
Milian Wolff [Fri, 11 Dec 2015 20:27:54 +0000 (21:27 +0100)]
Add .reviewboardrc file for rbt integration.
Milian Wolff [Fri, 11 Dec 2015 20:17:14 +0000 (21:17 +0100)]
Add version to heaptrack data files.
Milian Wolff [Fri, 11 Dec 2015 20:13:23 +0000 (21:13 +0100)]
Don't check module map sanity in non-debug builds of heaptrack.
Milian Wolff [Thu, 10 Dec 2015 14:56:48 +0000 (15:56 +0100)]
Show temporary allocations in heaptrack_gui.
Milian Wolff [Thu, 10 Dec 2015 14:35:43 +0000 (15:35 +0100)]
Find temporary allocations in heaptrack_print.
Milian Wolff [Wed, 9 Dec 2015 20:45:59 +0000 (21:45 +0100)]
Fix regression: the pointer index is still 64bit.
Milian Wolff [Wed, 9 Dec 2015 19:39:58 +0000 (20:39 +0100)]
Fix compiler warning
Milian Wolff [Mon, 7 Dec 2015 22:10:58 +0000 (23:10 +0100)]
Optimize: Reduce memory cost for tracking active allocations.
We split the hash into two, one for small allocations up to
numeric_limits<uint32_t>::max, and one for the bigger rest. This
allows us to reduce the cost by up to a factor of 2 for some apps.
A notable example here is heaptrack_gui itself, as it does tons of
small allocations for its lookup tables, which are long-lived.
In one test case this reduces the memory consumption of heaptrack by
up to 1GB for me.
Milian Wolff [Mon, 7 Dec 2015 21:48:43 +0000 (22:48 +0100)]
Use uint32_t for indices.
This allows us to save some memory and opens up more possibilities
for future optimizations. And 4,294,967,295 should still be more
than enough space for our lists of traces, strings etc. pp.
Milian Wolff [Sun, 6 Dec 2015 16:46:18 +0000 (17:46 +0100)]
Optimize: Reduce memory consumption of RowData for TreeModel by ~20%.
By interning the common LocationData we can save quite some memory
for large data sets. Still, the large tree is still easily hogging
up gigabytes of memory...
Milian Wolff [Sat, 5 Dec 2015 01:27:55 +0000 (02:27 +0100)]
Generate flame graph in background thread.
Also add a temporary text that indicates we are generating the graph.
Milian Wolff [Sat, 5 Dec 2015 01:20:14 +0000 (02:20 +0100)]
Make it possible to switch between bottom-up/top-down flame graph.
The context menu has a new action now to switch the view mode of
the flame graph.
Milian Wolff [Sat, 5 Dec 2015 00:50:39 +0000 (01:50 +0100)]
Build flame graph on demand.
Reduces memory consumption when the flame graph is not being displayed.
Milian Wolff [Sun, 15 Nov 2015 14:01:47 +0000 (15:01 +0100)]
Remove duplicate component
Milian Wolff [Sun, 15 Nov 2015 13:56:42 +0000 (14:56 +0100)]
Default CMAKE_BUILD_TYPE to RelWithDebInfo.
We need compiler optimizations for the heaptrack code, as it
is otherwise really slow to run.
Adrian Băcîrcea [Sun, 25 Oct 2015 15:53:33 +0000 (17:53 +0200)]
Always pass false for exe in elf_add call.
Basically, if you pass the isExe flag to elf_add, libbacktrace will
see that it's an ET_DYN executable which means that the sections
can be relocated on load so it will defer to dl_iterate_phdr to get
the real addresses. It will do that only when you call
backtrace_pcinfo which, behind the scenes, ends up calling
backtrace_initialize and that will do the dl_iterate_phdr for the
executable which is wrong since the executable we want to get the
symbols on is not the current one (which is heaptrack_interpret) and
we already get the base addresses correctly from the main process
through the pipe. So there's no need to ever pass isExe = true.
Milian Wolff [Fri, 23 Oct 2015 09:43:29 +0000 (11:43 +0200)]
Reserve space for chart data.
Milian Wolff [Fri, 23 Oct 2015 09:36:35 +0000 (11:36 +0200)]
Optimize: evaluate top-down data in parallel to chart data.
For a larger data file, this reduces the parse time from ~21s
to 15s on my machine.
Milian Wolff [Fri, 23 Oct 2015 09:11:12 +0000 (11:11 +0200)]
Optimize: only do one hash lookup instead of three per IpIndex.
This is still one of the hotspots when parsing large data files,
but at least it makes it a bit faster.
Milian Wolff [Fri, 23 Oct 2015 08:49:47 +0000 (10:49 +0200)]
Optimize: Replace QHash with std::array.
We now know that chart cost data is monotonously indexed and can
thus use a plain array over a QHash. This is much quicker to index
into in Parser::handleTimeStamp, and removes a hotspot found with
perf when analyzing a big data file.