core: Refactor code related to transfer flags and timeout handling

core: Refactor code related to transfer flags and timeout handling

Commit a886bb02 sped up the library a bit by removing the serialization
of transfer submission with respect to the flying_transfers list, but
it introduced two separate issues.

1) A deadlock scenario is possible given the following sequence:

   - Thread A submits transfer with very short timeout (say 1ms)
     -> takes transfer->lock
     -> adds transfer to flying_transfers list and arms timerfd
     -> actually calls backend to submit transfer, but it fails
   <context switch>
   - Thread B is doing event handling and sees the timerfd trigger
     -> takes ctx->flying_transfers_lock
     -> finds the transfer above on the list
     -> calls libusb_cancel_transfer() for this transfer
       --> takes transfer->lock
   <context switch>
   - Thread A sees the transfer failed to submit
     -> removes transfer from flying_transfers list
       --> takes ctx->flying_transfers_lock (still holding transfer->lock)
   ** DEADLOCK **

2) The transfer state flags (e.g. submitting, in-flight) were protected
    by transfer->flags_lock, but the timeout-related flags were OR'ed in
    during timeout handling operations outside of the lock. This leads to
    the possibility that transfer state might get overwritten.

This change corrects these issues and simplifies the transfer submission
code a bit by separating the state and timeout flags into their own flag
variables. The state flags are protected by the transfer lock. The timeout
flags are protected by the flying_transfers_lock.

The transfer submission code sheds some weight because it no longer needs
to worry about the timing of events that modify the transfer state flags.
These flags are always viewed and modified under the protection of the
transfer lock. Since libusb_submit_transfer() holds the transfer lock for
the entire duration of the operation, the other code paths that would
possibly touch the transfer (e.g. usbi_handle_disconnect() and
usbi_handle_transfer_completion()) have to wait for transfer submission
to fully complete. This eliminates any possible race conditions.

Signed-off-by: Chris Dickens <christopher.a.dickens@gmail.com>
[hdegoede@redhat.com: Reworked libusb_submit_transfer changes so that in
 case both flying_transfer_lock and itransfer->lock are taken
 flying_transfers_lock is always taken first]
[hdegoede@redhat.com: Removed some unrelated changes (will be submitted
 as separate patches)]
Signed-off-by: Hans de Goede <hdegoede@redhat.com>