Add a tool for diffing size remarks
authorJessica Paquette <jpaquette@apple.com>
Fri, 29 Oct 2021 01:51:52 +0000 (18:51 -0700)
committerJessica Paquette <jpaquette@apple.com>
Thu, 3 Feb 2022 00:09:54 +0000 (16:09 -0800)
commitded733bd49ff44cf94bd8a7a821ba37390dc94b1
tree3625e8d716c6bfce9020dded29b95c7f7dac0a6f
parentfa75a62cb5d5b79ed80c6b9372478ef1514f8c8f
Add a tool for diffing size remarks

This is a tool which can handle bitstream and YAML remarks. The idea here is to
provide more insight into which functions changed in a benchmark when testing
compiler changes.

E.g. "foo got 20% bigger, so maybe we should look more closely at that."

To use the tool, you can use...

```
$ llvm-remark-size-diff remarks_file_a remarks_file_b --parser=yaml|bitstream
```

... on two remarks files containing at least instruction count remarks. This
will output some data on instruction count change and also other relevant
information such as stack size change from `remarks_file_a` to `remarks_file_b`.

This is a bit of a WIP so I'm happy to change the format etc. Ultimately I think
it'd be best to have some JSON output which could be consumed by another tool.
But some base-level, greppable output is very handy to have anyway.

The format I'm proposing here is

```
<files> <inc/dec in inst count> <fn name> <inst count change> <stack B change>
```

Where the files and increase/decrease are indicated like below:

- `<files>` is one of `++` (file B), `--` (file A), `==` (both)
- `<inc/dec in inst count>` is one of `>` (increase) or `<` (decrease)

This makes it easy to grep for things like "which functions appeared in A but
did not appear in B?" Or "what are all the instruction count decreases?"

Differential Revision: https://reviews.llvm.org/D112940
18 files changed:
llvm/test/CMakeLists.txt
llvm/test/lit.cfg.py
llvm/test/tools/llvm-remark-size-diff/Inputs/1-func-1-instr-1-stack.yaml [new file with mode: 0644]
llvm/test/tools/llvm-remark-size-diff/Inputs/1-func-2-instr-2-stack.yaml [new file with mode: 0644]
llvm/test/tools/llvm-remark-size-diff/Inputs/2-identical-func-1-instr-1-stack.yaml [new file with mode: 0644]
llvm/test/tools/llvm-remark-size-diff/Inputs/empty-file.yaml [new file with mode: 0644]
llvm/test/tools/llvm-remark-size-diff/Inputs/inconvertible-integer.yaml [new file with mode: 0644]
llvm/test/tools/llvm-remark-size-diff/Inputs/no-instruction-count-remarks.yaml [new file with mode: 0644]
llvm/test/tools/llvm-remark-size-diff/Inputs/unexpected-key.yaml [new file with mode: 0644]
llvm/test/tools/llvm-remark-size-diff/add-remove-func.test [new file with mode: 0644]
llvm/test/tools/llvm-remark-size-diff/empty-file.test [new file with mode: 0644]
llvm/test/tools/llvm-remark-size-diff/inconvertible-integer.test [new file with mode: 0644]
llvm/test/tools/llvm-remark-size-diff/increase-decrease-inst-count.test [new file with mode: 0644]
llvm/test/tools/llvm-remark-size-diff/no-difference.test [new file with mode: 0644]
llvm/test/tools/llvm-remark-size-diff/no-instruction-count-remarks.test [new file with mode: 0644]
llvm/test/tools/llvm-remark-size-diff/unexpected-key.test [new file with mode: 0644]
llvm/tools/llvm-remark-size-diff/CMakeLists.txt [new file with mode: 0644]
llvm/tools/llvm-remark-size-diff/RemarkSizeDiff.cpp [new file with mode: 0644]