Bring back direct dependencies on System.Runtime.CompilerServices.Unsafe for other...

Bring back direct dependencies on System.Runtime.CompilerServices.Unsafe for other tfms than net7.0 (#67385)

* Add dependency on Unsafe package

With the removal of the Unsafe package, a few projects regressed which
previously had a direct dependency on the latest version of Unsafe
(6.0.0) by now depending on the transitive Unsafe dependency of i.e.
System.Memory which is a very old version (4.5.*).

Fixing those projects to declare the direct dependency again.

* Cleanup