tdm_buffer: clean up tdm_buffer_unref_backend()