clock: Keep weak reference to underlying clock