Turn device ports into reference counted objects

Turn device ports into reference counted objects

Since both cards and sinks can hold references to a port, it makes
sense to reference count them. Although no current implementation
actually has sinks with ports but without a card, it felt wrong
to make it harder to make such an implementation in the future.

Signed-off-by: David Henningsson <david.henningsson@canonical.com>