Revert "ref/unref display_buffer by using buffer's user_data"