baseaudiosink: delay the resyncing of timestamp vs ringbuffertime

A common problem for audio-playback is that the timestamps might not
be completely linear. This is specially common when doing streaming over
a network, where you can have jittery and/or bursty packettransmission,
which again will often be reflected on the buffertimestamps.

Now, the current implementation have a threshold that says how far the
buffertimestamp is allowed o drift from the ideal aligned time in the
ringbuffer. This was an instant reaction, and ment that if one buffer
arrived with a timestamp that would breach the drift-tolerance, a resync
would take place, and the result would be an audible gap for the
listener.

The annoying thing would be that in the case of a "timestamp-outlier",
you would first resync one way, say +100ms, and then, if the next
timestamp was "back on track", you would end up resyncing the other way
(-100ms) So in fact, when you had only one buffer with slightly off
timestamping, you would end up with *two* audible gaps. This is the
problem this patch addresses.

The way to "fix" this problem with the previous implementation, would
have been to increase the "drift-tolerance" to a value that was greater
than the largest timestamp-outlier one would normally expect.  The big
problem with this approach, however, is that it will allow normal
operations with a huge offset timestamp vs running-time, which is
detrimental to lip-sync. If the drift-tolerance is set to 200ms, it
basically means that lip-sync can easily end up being off by that much.

This patch will basically start a timer when the first breach of
drift-tolerance is detected. If any following timestamp for the next n
nanoseconds gets "back on track" within the threshold, it has basically
eliminated the effect of an outlier, and the timer is stopped.  If,
however, all timestamps within this time-limit are breaching the
threshold, we are probably facing a more permanent offset in the
timestamps, and a resync is allowed to happen.

So basically this patch offers something as rare as both higher
accuracy, it terms of allowing smaller drift-tolerances, as well as much
smoother, less glitchy playback!

Commit message and improvments by Havard Graff.

Fixes bug #640859.

diff --git a/gst-libs/gst/audio/gstbaseaudiosink.c b/gst-libs/gst/audio/gstbaseaudiosink.c
index a88f7e65af8a94d2c265fd77daf16c0515f697f8..dd531c919040c18e18d445dc6b604d3732fa71bd 100644 (file)
--- a/gst-libs/gst/audio/gstbaseaudiosink.c
+++ b/gst-libs/gst/audio/gstbaseaudiosink.c
@@ -62,6 +62,9 @@ struct _GstBaseAudioSinkPrivate
    * before resyncing */
   guint64 drift_tolerance;
 
+  /* time of the previous detected discont candidate */
+  GstClockTime discont_time;
+
   /* number of nanoseconds we allow timestamps to drift
    * before resyncing */
   GstClockTime alignment_threshold;
@@ -963,6 +966,7 @@ gst_base_audio_sink_event (GstBaseSink * bsink, GstEvent * event)
       sink->priv->avg_skew = -1;
       sink->next_sample = -1;
       sink->priv->eos_time = -1;
+      sink->priv->discont_time = -1;
       if (sink->ringbuffer)
         gst_ring_buffer_set_flushing (sink->ringbuffer, FALSE);
       break;
@@ -1384,6 +1388,7 @@ gst_base_audio_sink_sync_latency (GstBaseSink * bsink, GstMiniObject * obj)
   sink->priv->avg_skew = -1;
   sink->next_sample = -1;
   sink->priv->eos_time = -1;
+  sink->priv->discont_time = -1;
 
   return GST_FLOW_OK;
 
@@ -1418,6 +1423,7 @@ gst_base_audio_sink_get_alignment (GstBaseAudioSink * sink,
   gint64 samples_done = segdone * ringbuf->samples_per_seg;
   gint64 headroom = sample_offset - samples_done;
   gboolean allow_align = TRUE;
+  gboolean discont = FALSE;
 
   /* now try to align the sample to the previous one, first see how big the
    * difference is. */
@@ -1437,7 +1443,24 @@ gst_base_audio_sink_get_alignment (GstBaseAudioSink * sink,
   if (sample_diff > headroom && align < 0)
     allow_align = FALSE;
 
-  if (G_LIKELY (sample_diff < max_sample_diff && allow_align)) {
+  /* wait before deciding to make a discontinuity */
+  if (G_UNLIKELY (sample_diff >= max_sample_diff)) {
+    GstClockTime time = gst_util_uint64_scale_int (sample_offset,
+        GST_SECOND, ringbuf->spec.rate);
+    if (sink->priv->discont_time == -1) {
+      /* discont candidate */
+      sink->priv->discont_time = time;
+    } else if (time - sink->priv->discont_time >= GST_SECOND) {
+      /* one second passed, discontinuity detected */
+      discont = TRUE;
+      sink->priv->discont_time = -1;
+    }
+  } else if (G_UNLIKELY (sink->priv->discont_time != -1)) {
+    /* we have had a discont, but are now back on track! */
+    sink->priv->discont_time = -1;
+  }
+
+  if (G_LIKELY (!discont && allow_align)) {
     GST_DEBUG_OBJECT (sink,
         "align with prev sample, ABS (%" G_GINT64_FORMAT ") < %"
         G_GINT64_FORMAT, align, max_sample_diff);
@@ -2045,6 +2068,7 @@ gst_base_audio_sink_change_state (GstElement * element,
       sink->next_sample = -1;
       sink->priv->last_align = -1;
       sink->priv->eos_time = -1;
+      sink->priv->discont_time = -1;
       gst_ring_buffer_set_flushing (sink->ringbuffer, FALSE);
       gst_ring_buffer_may_start (sink->ringbuffer, FALSE);