make sure priv->adapter exists before unref it when freeing gap agent