Remove the per-connection link cache

Remove the per-connection link cache

With fd.o#34393 fixed, retaking the lock to cache unused links
significantly adds to locking overhead (-18% throughput in a synthetic
benchmark on an ARM device). The cache is also unlimited in size, and
probably contributes to memory growth and fragmentation by not being
under the system malloc's control.

Fixing fd.o #34393, but also dropping this cache entirely, turns out to
lead to a 5% increase in throughput on the same synthetic benchmark.

Reviewed-by: Colin Walters <walters@verbum.org>
Bug: https://bugs.freedesktop.org/show_bug.cgi?id=34393