Serialize canonical types to avoid testing if types have been emitted
authorDodji Seketeli <dodji@redhat.com>
Fri, 2 Aug 2019 15:06:37 +0000 (17:06 +0200)
committerDodji Seketeli <dodji@redhat.com>
Tue, 17 Sep 2019 15:52:26 +0000 (17:52 +0200)
commitbc2d2ce660d3d28bcedb76f3cde30280c9b667a2
tree0de040ddaf5920103d9b2de3a06521cc9297979d
parent413381ee7f99c0a2bef1c6d76c1c06fb17160a1b
Serialize canonical types to avoid testing if types have been emitted

When emitting abixml, profiling shows that we spend a great deal of
time testing if a given type has been emitted already, to avoid
emitting a given type more than once.  This makes the serialization
phase take more time than the binary analysis phase!

This patch leverages the fact that we already have the set of
canonical types in the system.  While emitting that set entirely, we
don't need to test if a type has been emitted already because we know
by definition that every type is present just once in that set, more
or less.  OK, because there are also types that don't have canonical
types (for instance, declaration-only class/structs), we'll still have
to check of those types have already been emitted, but this is a very
small set to handle.

The patch thus organizes the canonical types per scope, so that when
emitting a scope and the canonical types within it, the type is
emitted in its correct namespace.

Then, when emitting a translation unit and each namespaces in it, the
patch emits the canonical types of those namespaces.

The patch arranges for some ancillary things that are needed to make
the whole picture be coherent enough for things to keep working.

Testing shows that we gained ~ 30% of performance by doing this, while
analysing the whole linux kernel 5.1 version.  We went from ~ 3m30s
minutes to less than 2m30s.

With this patch, the serialization phase now takes less time than the
analysis time.

* include/abg-fwd.h (is_decl_slow)
(peel_pointer_or_reference_type): Declare new functions.
* include/abg-ir.h (struct canonical_type_hash): Define new type.
(type_base_ptr_set_type, type_base_ptrs_type)
(type_base_sptrs_type, canonical_type_sptr_set_type): Define new
typedefs.
(environment::get_canonical_types_map): Declare new member
function.
(scope_decl::{get_canonical_types, get_sorted_canonical_types}):
Declare new member functions.
* src/abg-ir.cc (is_ptr_ref_or_qual_type)
(peel_pointer_or_reference_type, is_decl_slow): Define new
functions.
(environment::{get_canonical_types_map}): Define new member
functions.
(canonical_type_hash::operator()): Likewise.
(scope_decl::{get_canonical_types, get_sorted_canonical_types}):
Likewise.
(struct type_topo_comp): Define new comparison functor type.
(environment::{sorted_canonical_types_}): Define new data member.
(scope_decl::priv::{canonical_types_, sorted_canonical_types_}):
Likewise.
(scope_decl::is_empty): Take the presence of canonical types into
account when determining if a scope is empty or not.
(is_decl): Make this work for cases where the artifact at hand is
a type which has a declaration, as opposed to being a pure
declaration like a variable or a function.
(canonicalize): Add the canonical type the list of canonical types
of its scope.
* src/abg-dwarf-reader.cc (read_context::die_is_in_cplus_plus):
Define new member function.
* src/abg-writer.cc (write_type, write_canonical_types_of_scope):
Define new static functions.
(fn_type_ptr_set_type): Define new typedef.
(write_context::{m_referenced_fn_types_set,
m_referenced_non_canonical_types_set}): Add new data members.
(write_context::m_referenced_types_set): Renamed
m_referenced_types_map into this.
(write_context::get_referenced_types): Adjust.
(write_context::get_referenced_{function_types,
non_canonical_types}):
(write_context::record_type_as_referenced): Adjust to add the
referenced type in the proper set which would be one of the three
following: write_context::{get_referenced_types,
get_referenced_function_types,
get_referenced_non_canonical_types}.
(write_context::{type_is_referenced, clear_referenced}): Adjust.
(write_translation_unit): Use the new
write_canonical_types_of_scope.  Also emit declaration-only
classes that have member types.  Do not test if a given type of a
given scope has been emitted, in general, as this was super slow
given the number of types.  Emit referenced function types (as
these don't belong to any scope).  Rather than using the expensive
"is_function_type" on *all* the referenced types, just walk the
set write_context::get_referenced_function_types.  Likewise,
rather than using type_base::get_naked_canonical_type on
*all* the referenced types, just walk the set
write_context::get_referenced_non_canonical_types
(write_class): Use write_canonical_types_of_scope here.
* tools/abilint.cc (main): Support linting corpus group abixml
files.
* tests/data/test-annotate/libtest23.so.abi: Adjust.
* tests/data/test-annotate/libtest24-drop-fns-2.so.abi: Likewise.
* tests/data/test-annotate/libtest24-drop-fns.so.abi: Likewise.
* tests/data/test-annotate/test-anonymous-members-0.o.abi: Likewise.
* tests/data/test-annotate/test0.abi: Likewise.
* tests/data/test-annotate/test1.abi: Likewise.
* tests/data/test-annotate/test13-pr18894.so.abi: Likewise.
* tests/data/test-annotate/test14-pr18893.so.abi: Likewise.
* tests/data/test-annotate/test15-pr18892.so.abi: Likewise.
* tests/data/test-annotate/test17-pr19027.so.abi: Likewise.
* tests/data/test-annotate/test18-pr19037-libvtkRenderingLIC-6.1.so.abi: Likewise.
* tests/data/test-annotate/test19-pr19023-libtcmalloc_and_profiler.so.abi: Likewise.
* tests/data/test-annotate/test2.so.abi: Likewise.
* tests/data/test-annotate/test20-pr19025-libvtkParallelCore-6.1.so.abi: Likewise.
* tests/data/test-annotate/test21-pr19092.so.abi: Likewise.
* tests/data/test-annotate/test4.so.abi: Likewise.
* tests/data/test-annotate/test6.so.abi: Likewise.
* tests/data/test-annotate/test7.so.abi: Likewise.
* tests/data/test-annotate/test8-qualified-this-pointer.so.abi: Likewise.
* tests/data/test-read-dwarf/PR22015-libboost_iostreams.so.abi: Likewise.
* tests/data/test-read-dwarf/PR22122-libftdc.so.abi: Likewise.
* tests/data/test-read-dwarf/PR24378-fn-is-not-scope.abi: Likewise.
* tests/data/test-read-dwarf/libtest23.so.abi: Likewise.
* tests/data/test-read-dwarf/libtest24-drop-fns-2.so.abi: Likewise.
* tests/data/test-read-dwarf/libtest24-drop-fns.so.abi: Likewise.
* tests/data/test-read-dwarf/test0.abi: Likewise.
* tests/data/test-read-dwarf/test1.abi: Likewise.
* tests/data/test-read-dwarf/test10-pr18818-gcc.so.abi: Likewise.
* tests/data/test-read-dwarf/test11-pr18828.so.abi: Likewise.
* tests/data/test-read-dwarf/test12-pr18844.so.abi: Likewise.
* tests/data/test-read-dwarf/test13-pr18894.so.abi: Likewise.
* tests/data/test-read-dwarf/test14-pr18893.so.abi: Likewise.
* tests/data/test-read-dwarf/test15-pr18892.so.abi: Likewise.
* tests/data/test-read-dwarf/test16-pr18904.so.abi: Likewise.
* tests/data/test-read-dwarf/test17-pr19027.so.abi: Likewise.
* tests/data/test-read-dwarf/test18-pr19037-libvtkRenderingLIC-6.1.so.abi: Likewise.
* tests/data/test-read-dwarf/test19-pr19023-libtcmalloc_and_profiler.so.abi: Likewise.
* tests/data/test-read-dwarf/test2.so.abi: Likewise.
* tests/data/test-read-dwarf/test20-pr19025-libvtkParallelCore-6.1.so.abi: Likewise.
* tests/data/test-read-dwarf/test21-pr19092.so.abi: Likewise.
* tests/data/test-read-dwarf/test22-pr19097-libstdc++.so.6.0.17.so.abi: Likewise.
* tests/data/test-read-dwarf/test4.so.abi: Likewise.
* tests/data/test-read-dwarf/test6.so.abi: Likewise.
* tests/data/test-read-dwarf/test7.so.abi: Likewise.
* tests/data/test-read-dwarf/test8-qualified-this-pointer.so.abi: Likewise.
* tests/data/test-read-dwarf/test9-pr18818-clang.so.abi: Likewise.
* tests/data/test-read-write/test10.xml: Likewise.
* tests/data/test-read-write/test14.xml: Likewise.
* tests/data/test-read-write/test15.xml: Likewise.
* tests/data/test-read-write/test17.xml: Likewise.
* tests/data/test-read-write/test18.xml: Likewise.
* tests/data/test-read-write/test19.xml: Likewise.
* tests/data/test-read-write/test2.xml: Likewise.
* tests/data/test-read-write/test20.xml: Likewise.
* tests/data/test-read-write/test21.xml: Likewise.
* tests/data/test-read-write/test22.xml: Likewise.
* tests/data/test-read-write/test23.xml: Likewise.
* tests/data/test-read-write/test24.xml: Likewise.
* tests/data/test-read-write/test25.xml: Likewise.
* tests/data/test-read-write/test26.xml: Likewise.
* tests/data/test-read-write/test27.xml: Likewise.
* tests/data/test-read-write/test28-without-std-fns-ref.xml: Likewise.
* tests/data/test-read-write/test28-without-std-vars-ref.xml: Likewise.
* tests/data/test-read-write/test3.xml: Likewise.
* tests/data/test-read-write/test6.xml: Likewise.

Signed-off-by: Dodji Seketeli <dodji@redhat.com>
71 files changed:
include/abg-fwd.h
include/abg-ir.h
src/abg-dwarf-reader.cc
src/abg-ir.cc
src/abg-writer.cc
tests/data/test-annotate/libtest23.so.abi
tests/data/test-annotate/libtest24-drop-fns-2.so.abi
tests/data/test-annotate/libtest24-drop-fns.so.abi
tests/data/test-annotate/test-anonymous-members-0.o.abi
tests/data/test-annotate/test0.abi
tests/data/test-annotate/test1.abi
tests/data/test-annotate/test13-pr18894.so.abi
tests/data/test-annotate/test14-pr18893.so.abi
tests/data/test-annotate/test15-pr18892.so.abi
tests/data/test-annotate/test17-pr19027.so.abi
tests/data/test-annotate/test18-pr19037-libvtkRenderingLIC-6.1.so.abi
tests/data/test-annotate/test19-pr19023-libtcmalloc_and_profiler.so.abi
tests/data/test-annotate/test2.so.abi
tests/data/test-annotate/test20-pr19025-libvtkParallelCore-6.1.so.abi
tests/data/test-annotate/test21-pr19092.so.abi
tests/data/test-annotate/test4.so.abi
tests/data/test-annotate/test6.so.abi
tests/data/test-annotate/test7.so.abi
tests/data/test-annotate/test8-qualified-this-pointer.so.abi
tests/data/test-read-dwarf/PR22015-libboost_iostreams.so.abi
tests/data/test-read-dwarf/PR22122-libftdc.so.abi
tests/data/test-read-dwarf/PR24378-fn-is-not-scope.abi
tests/data/test-read-dwarf/libtest23.so.abi
tests/data/test-read-dwarf/libtest24-drop-fns-2.so.abi
tests/data/test-read-dwarf/libtest24-drop-fns.so.abi
tests/data/test-read-dwarf/test0.abi
tests/data/test-read-dwarf/test1.abi
tests/data/test-read-dwarf/test10-pr18818-gcc.so.abi
tests/data/test-read-dwarf/test11-pr18828.so.abi
tests/data/test-read-dwarf/test12-pr18844.so.abi
tests/data/test-read-dwarf/test13-pr18894.so.abi
tests/data/test-read-dwarf/test14-pr18893.so.abi
tests/data/test-read-dwarf/test15-pr18892.so.abi
tests/data/test-read-dwarf/test16-pr18904.so.abi
tests/data/test-read-dwarf/test17-pr19027.so.abi
tests/data/test-read-dwarf/test18-pr19037-libvtkRenderingLIC-6.1.so.abi
tests/data/test-read-dwarf/test19-pr19023-libtcmalloc_and_profiler.so.abi
tests/data/test-read-dwarf/test2.so.abi
tests/data/test-read-dwarf/test20-pr19025-libvtkParallelCore-6.1.so.abi
tests/data/test-read-dwarf/test21-pr19092.so.abi
tests/data/test-read-dwarf/test22-pr19097-libstdc++.so.6.0.17.so.abi
tests/data/test-read-dwarf/test4.so.abi
tests/data/test-read-dwarf/test6.so.abi
tests/data/test-read-dwarf/test7.so.abi
tests/data/test-read-dwarf/test8-qualified-this-pointer.so.abi
tests/data/test-read-dwarf/test9-pr18818-clang.so.abi
tests/data/test-read-write/test10.xml
tests/data/test-read-write/test14.xml
tests/data/test-read-write/test15.xml
tests/data/test-read-write/test17.xml
tests/data/test-read-write/test18.xml
tests/data/test-read-write/test19.xml
tests/data/test-read-write/test2.xml
tests/data/test-read-write/test20.xml
tests/data/test-read-write/test21.xml
tests/data/test-read-write/test22.xml
tests/data/test-read-write/test23.xml
tests/data/test-read-write/test24.xml
tests/data/test-read-write/test25.xml
tests/data/test-read-write/test26.xml
tests/data/test-read-write/test27.xml
tests/data/test-read-write/test28-without-std-fns-ref.xml
tests/data/test-read-write/test28-without-std-vars-ref.xml
tests/data/test-read-write/test3.xml
tests/data/test-read-write/test6.xml
tools/abilint.cc