<chapter id="chapter-other-sink" xreflabel="Writing a Sink">
<title>Writing a Sink</title>
<para>
- FIXME: write.
+ Sinks are output elements that, opposite to sources, have no source
+ pads and one or more (usually one) sink pad. They can be sound card
+ outputs, disk writers, etc. This chapter will discuss the basic
+ implementation of sink elements.
</para>
+
+ <sect1 id="other-sink-processing" xreflabel="Data processing, events, synchronization and clocks">
+ <title>Data processing, events, synchronization and clocks</title>
+ <para>
+ Except for corner cases, sink elements will be <function>_chain
+ ()</function>-based elements. The concept of such elements has
+ been discussed before in detail, so that will be skipped here. What
+ is very important in sink elements, specifically in real-time audio
+ and video sources (such as <classname>osssink</classname> or
+ <classname>ximagesink</classname>), is event handling in the
+ <function>_chain ()</function>-function, because most elements rely
+ on EOS-handling of the sink element, and because A/V synchronization
+ can only be perfect if the element takes this into account.
+ </para>
+ <para>
+ How to achieve synchronization between streams depends on whether
+ you're a clock-providing or a clock-receiving element. If you're
+ the clock provider, you can do with time whatever you want. Correct
+ handling would mean that you check whether the end of the previous
+ buffer (if any) and the start of the current buffer are the same.
+ If so, there's no gap between the two and you can continue playing
+ right away. If there is a gap, then you'll need to wait for your
+ clock to reach that time. How to do that depends on the element
+ type. In the case of audio output elements, you would output silence
+ for a while. In the case of video, you would show background color.
+ In case of subtitles, show no subtitles at all.
+ </para>
+ <para>
+ In the case that the provided clock and the received clock are not
+ the same (or in the case where your element provides no clock, which
+ is the same), you simply wait for the clock to reach the timestamp of
+ the current buffer and then you handle the data in it.
+ </para>
+ <para>
+ A simple data handling function would look like this:
+ </para>
+ <programlisting>
+static void
+gst_my_sink_chain (GstPad *pad,
+ GstData *data)
+{
+ GstMySink *sink = GST_MY_SINK (gst_pad_get_parent (pad));
+ GstBuffer *buf;
+ GstClockTime time;
+
+ /* only needed if the element is GST_EVENT_AWARE */
+ if (GST_IS_EVENT (data)) {
+ GstEvent *event = GST_EVENT (data);
+
+ switch (GST_EVENT_TYPE (event)) {
+ case GST_EVENT_EOS:
+ [ if your element provides a clock, disable (inactivate) it here ]
+ /* pass-through */
+
+ default:
+ /* the default handler handles discontinuities, even if your
+ * element provides a clock! */
+ gst_pad_event_default (pad, event);
+ break;
+ }
+
+ return;
+ }
+
+ buf = GST_BUFFER (data);
+ if (GST_BUFFER_TIME_IS_VALID (buf))
+ time = GST_BUFFER_TIMESTAMP (buf);
+ else
+ time = sink->expected_next_time;
+
+ /* Synchronization - the property is only useful in case the
+ * element has the option of not syncing. So it is not useful
+ * for hardware-sync (clock-providing) elements. */
+ if (sink->sync) {
+ /* This check is only needed if you provide a clock. Else,
+ * you can always execute the 'else' clause. */
+ if (sink->provided_clock == sink->received_clock) {
+ /* GST_SECOND / 10 is 0,1 sec, it's an arbitrary value. The
+ * casts are needed because else it'll be unsigned and we
+ * won't detect negative values. */
+ if (llabs ((gint64) sink->expected_next_time - (gint64) time) >
+ (GST_SECOND / 10)) {
+ /* so are we ahead or behind? */
+ if (time > sink->expected_time) {
+ /* we need to wait a while... In case of audio, output
+ * silence. In case of video, output background color.
+ * In case of subtitles, display nothing. */
+ [..]
+ } else {
+ /* Drop data. */
+ [..]
+ }
+ }
+ } else {
+ /* You could do more sophisticated things here, but we'll
+ * keep it simple for the purpose of the example. */
+ gst_element_wait (GST_ELEMENT (sink), time);
+ }
+ }
+
+ /* And now handle the data. */
+[..]
+}
+ </programlisting>
+ </sect1>
+
+ <sect1 id="other-sink-buffers" xreflabel="Special memory">
+ <title>Special memory</title>
+ <para>
+ Like source elements, sink elements can sometimes provide externally
+ allocated (such as X-provided or DMA'able) memory to elements earlier
+ in the pipeline, and thereby prevent the need for
+ <function>memcpy ()</function> for incoming data. We do this by
+ providing a pad-allocate-buffer function.
+ </para>
+ <programlisting>
+static GstBuffer * gst_my_sink_buffer_allocate (GstPad *pad,
+ guint64 offset,
+ guint size);
+
+static void
+gst_my_sink_init (GstMySink *sink)
+{
+[..]
+ gst_pad_set_bufferalloc_function (sink->sinkpad,
+ gst_my_sink_buffer_allocate);
+}
+
+static void
+gst_my_sink_buffer_free (GstBuffer *buf)
+{
+ GstMySink *sink = GST_MY_SINK (GST_BUFFER_PRIVATE (buf));
+
+ /* Do whatever is needed here. */
+[..]
+}
+
+static GstBuffer *
+gst_my_sink_buffer_allocate (GstPad *pad,
+ guint64 offset,
+ guint size)
+{
+ GstBuffer *buf = gst_buffer_new ();
+
+ /* So here it's up to you to wrap your private buffers and
+ * return that. */
+ GST_BUFFER_FREE_DATA_FUNC (buf) = gst_my_sink_buffer_free;
+ GST_BUFFER_PRIVATE (buf) = sink;
+ GST_BUFFER_FLAG_SET (buf, GST_BUFFER_DONTFREE);
+[..]
+
+ return buf;
+}
+ </programlisting>
+ </sect1>
</chapter>
<chapter id="chapter-other-source" xreflabel="Writing a Source">
<title>Writing a Source</title>
<para>
- FIXME: write.
+ Source elements are the start of a data streaming pipeline. Source
+ elements have no sink pads and have one or more source pads. We will
+ focus on single-sourcepad elements here, but the concepts apply equally
+ well to multi-sourcepad elements. This chapter will explain the essentials
+ of source elements, which features it should implement and which it
+ doesn't have to, and how source elements will interact with other
+ elements in a pipeline.
</para>
+
+ <sect1 id="section-source-getfn" xreflabel="The get()-function">
+ <title>The get()-function</title>
+ <para>
+ Source elements have the special option of having a
+ <function>_get ()</function>-function rather than a
+ <function>_loop ()</function>- or <function>_chain
+ ()</function>-function. A <function>_get ()</function>-function is
+ called by the scheduler every time the next elements needs data. Apart
+ from corner cases, every source element will want to be <function>_get
+ ()</function>-based.
+ </para>
+ <programlisting>
+static GstData * gst_my_source_get (GstPad *pad);
+
+static void
+gst_my_source_init (GstMySource *src)
+{
+[..]
+ gst_pad_set_get_function (src->srcpad, gst_my_source_get);
+}
+
+static GstData *
+gst_my_source_get (GstPad *pad)
+{
+ GstBuffer *buffer;
+
+ buffer = gst_buffer_new ();
+ GST_BUFFER_DATA (buf) = g_strdup ("hello pipeline!");
+ GST_BUFFER_SIZE (buf) = strlen (GST_BUFFER_DATA (buf));
+ /* terminating '/0' */
+ GST_BUFFER_MAZSIZE (buf) = GST_BUFFER_SIZE (buf) + 1;
+
+ return GST_DATA (buffer);
+}
+ </programlisting>
+ </sect1>
+
+ <sect1 id="section-source-padfn" xreflabel="Events, querying and converting">
+ <title>Events, querying and converting</title>
+ <para>
+ One of the most important functions of source elements is to
+ implement correct query, convert and event handling functions.
+ Those will continuously describe the current state of the stream.
+ Query functions can be used to get stream properties such as current
+ position and length. This can be used by fellow elements to convert
+ this same value into a different unit, or by appliations to provide
+ information about the length/position of the stream to the user.
+ Conversion functions are used to convert such values from one unit
+ to another. Lastly, events are mostly used to seek to positions
+ inside the stream. Any function is essentially optional, but the
+ element should try to provide as much information as it knows. Note
+ that elements providing an event function should also list their
+ supported events in an <function>_get_event_mask ()</function>
+ function. Elements supporting query operations should list the
+ supported operations in a <function>_get_query_types
+ ()</function> function. Elements supporting either conversion
+ or query operations should also implement a <function>_get_formats
+ ()</function> function.
+ </para>
+ <para>
+ An example source element could, for example, be an element that
+ continuously generates a wave tone at 44,1 kHz, mono, 16-bit. This
+ element will generate 44100 audio samples per second or 88,2 kB/s.
+ This information can be used to implement such functions:
+ </para>
+ <programlisting>
+static GstFormat * gst_my_source_format_list (GstPad *pad);
+static GstQueryType * gst_my_source_query_list (GstPad *pad);
+
+static gboolean gst_my_source_convert (GstPad *pad,
+ GstFormat from_fmt,
+ gint64 from_val,
+ GstFormat *to_fmt,
+ gint64 *to_val);
+static gboolean gst_my_source_query (GstPad *pad,
+ GstQueryType type,
+ GstFormat *to_fmt,
+ gint64 *to_val);
+
+static void
+gst_my_source_init (GstMySource *src)
+{
+[..]
+ gst_pad_set_convert_function (src->srcpad, gst_my_source_convert);
+ gst_pad_set_formats_function (src->srcpad, gst_my_source_format_list);
+ gst_pad_set_query_function (src->srcpad, gst_my_source_query);
+ gst_pad_set_query_type_function (src->srcpad, gst_my_source_query_list);
+}
+
+/*
+ * This function returns an enumeration of supported GstFormat
+ * types in the query() or convert() functions. See gst/gstformat.h
+ * for a full list.
+ */
+
+static GstFormat *
+gst_my_source_format_list (GstPad *pad)
+{
+ static const GstFormat formats[] = {
+ GST_FORMAT_TIME,
+ GST_FORMAT_DEFAULT, /* means "audio samples" */
+ GST_FORMAT_BYTES,
+ 0
+ };
+
+ return formats;
+}
+
+/*
+ * This function returns an enumeration of the supported query()
+ * operations. Since we generate audio internally, we only provide
+ * an indication of how many samples we've played so far. File sources
+ * or such elements could also provide GST_QUERY_TOTAL for the total
+ * stream length, or other things. See gst/gstquery.h for details.
+ */
+
+static GstQueryType *
+gst_my_source_query_list (GstPad *pad)
+{
+ static const GstQueryType query_types[] = {
+ GST_QUERY_POSITION,
+ 0,
+ };
+
+ return query_types;
+}
+
+/*
+ * And below are the logical implementations.
+ */
+
+static gboolean
+gst_my_source_convert (GstPad *pad,
+ GstFormat from_fmt,
+ gint64 from_val,
+ GstFormat *to_fmt,
+ gint64 *to_val)
+{
+ gboolean res = TRUE;
+ GstMySource *src = GST_MY_SOURCE (gst_pad_get_parent (pad));
+
+ switch (from_fmt) {
+ case GST_FORMAT_TIME:
+ switch (*to_fmt) {
+ case GST_FORMAT_TIME:
+ /* nothing */
+ break;
+
+ case GST_FORMAT_BYTES:
+ *to_val = from_val / (GST_SECOND / (44100 * 2));
+ break;
+
+ case GST_FORMAT_DEFAULT:
+ *to_val = from_val / (GST_SECOND / 44100);
+ break;
+
+ default:
+ res = FALSE;
+ break;
+ }
+ break;
+
+ case GST_FORMAT_BYTES:
+ switch (*to_fmt) {
+ case GST_FORMAT_TIME:
+ *to_val = from_val * (GST_SECOND / (44100 * 2));
+ break;
+
+ case GST_FORMAT_BYTES:
+ /* nothing */
+ break;
+
+ case GST_FORMAT_DEFAULT:
+ *to_val = from_val / 2;
+ break;
+
+ default:
+ res = FALSE;
+ break;
+ }
+ break;
+
+ case GST_FORMAT_DEFAULT:
+ switch (*to_fmt) {
+ case GST_FORMAT_TIME:
+ *to_val = from_val * (GST_SECOND / 44100);
+ break;
+
+ case GST_FORMAT_BYTES:
+ *to_val = from_val * 2;
+ break;
+
+ case GST_FORMAT_DEFAULT:
+ /* nothing */
+ break;
+
+ default:
+ res = FALSE;
+ break;
+ }
+ break;
+
+ default:
+ res = FALSE;
+ break;
+ }
+
+ return res;
+}
+
+static gboolean
+gst_my_source_query (GstPad *pad,
+ GstQueryType type,
+ GstFormat *to_fmt,
+ gint64 *to_val)
+{
+ GstMySource *src = GST_MY_SOURCE (gst_pad_get_parent (pad));
+ gboolean res = TRUE;
+
+ switch (type) {
+ case GST_QUERY_POSITION:
+ res = gst_pad_convert (pad, GST_FORMAT_BYTES, src->total_bytes,
+ to_fmt, to_val);
+ break;
+
+ default:
+ res = FALSE;
+ break;
+ }
+
+ return res;
+}
+ </programlisting>
+ <para>
+ Be sure to increase src->total_bytes after each call to your
+ <function>_get ()</function> function.
+ </para>
+ <para>
+ Event handling has already been explained previously in the events
+ chapter.
+ </para>
+ </sect1>
+
+ <sect1 id="section-source-sync" xreflabel="Time, clocking and synchronization">
+ <title>Time, clocking and synchronization</title>
+ <para>
+ The above example does not provide any timing info, but will suffice
+ for elementary data sources such as a file source or network data
+ source element. Things become slightly more complicated, but still
+ very simple, if we create artificial video or audio data sources,
+ such as a video test image source or an artificial audio source (e.g.
+ <classname>sinesrc</classname> or <classname>silence</classname>).
+ It will become more complicated if we want the element to be a
+ realtime capture source, such as a video4linux source (for reading
+ video frames from a TV card) or an ALSA source (for reading data
+ from soundcards supported by an ALSA-driver). Here, we will need to
+ make the element aware of timing and clocking.
+ </para>
+ <para>
+ Timestamps can essentially be generated from all the information
+ given above without any difficulty. We could add a very small amount
+ of code to generate perfectly timestamped buffers from our
+ <function>_get ()</function>-function:
+ </para>
+ <programlisting>
+static void
+gst_my_source_init (GstMySource *src)
+{
+[..]
+ src->total_bytes = 0;
+}
+
+static GstData *
+gst_my_source_get (GstPad *pad)
+{
+ GstMySource *src = GST_MY_SOURCE (gst_pad_get_parent (pad));
+ GstBuffer *buf;
+ GstFormat fmt = GST_FORMAT_TIME;
+[..]
+ GST_BUFFER_DURATION (buf) = GST_BUFFER_SIZE (buf) * (GST_SECOND / (44100 * 2));
+ GST_BUFFER_TIMESTAMP (buf) = src->total_bytes * (GST_SECOND / (44100 * 2));
+ src->total_bytes += GST_BUFFER_SIZE (buf);
+
+ return GST_DATA (buf);
+}
+
+static GstStateReturn
+gst_my_source_change_state (GstElement *element)
+{
+ GstMySource *src = GST_MY_SOURCE (element);
+
+ switch (GST_STATE_PENDING (element)) {
+ case GT_STATE_PAUSED_TO_READY:
+ src->total_bytes = 0;
+ break;
+
+ default:
+ break;
+ }
+
+ if (GST_ELEMENT_CLASS (parent_class)->change_state)
+ return GST_ELEMENT_CLASS (parent_class)->change_state (element);
+
+ return GST_STATE_SUCCESS;
+}
+ </programlisting>
+ <para>
+ That wasn't too hard. Now, let's assume real-time elements. Those
+ can either have hardware-timing, in which case we can rely on backends
+ to provide sync for us (in which case you probably want to provide a
+ clock), or we will have to emulate that internally (e.g. to acquire
+ sync in artificial data elements such as <classname>sinesrc</classname>).
+ Let's first look at the second option (software sync). The first option
+ (hardware sync + providing a clock) does not require any special code
+ with respect to timing, and the clocking section already explained how
+ to provide a clock.
+ </para>
+ <programlisting>
+enum {
+ ARG_0,
+[..]
+ ARG_SYNC,
+[..]
+};
+
+static void
+gst_my_source_class_init (GstMySourceClass *klass)
+{
+ GObjectClass *object_class = G_OBJECT_CLASS (klass);
+[..]
+ g_object_class_install_property (object_class, ARG_SYNC,
+ g_param_spec_boolean ("sync", "Sync", "Synchronize to clock",
+ FALSE, G_PARAM_READWRITE));
+[..]
+}
+
+static void
+gst_my_source_init (GstMySource *src)
+{
+[..]
+ src->sync = FALSE;
+}
+
+static void
+gst_my_source_get (GstPad *pad)
+{
+ GstMySource *src = GST_MY_SOURCE (gst_pad_get_parent (pad));
+ GstBuffer *buf;
+[..]
+ if (src->sync) {
+ /* wait on clock */
+ gst_element_wait (GST_ELEMENT (src), GST_BUFFER_TIMESTAMP (buf));
+ }
+
+ return GST_DATA (buf);
+}
+
+static void
+gst_my_source_get_property (GObject *object,
+ guint prop_id,
+ GParamSpec *pspec,
+ GValue *value)
+{
+ GstMySource *src = GST_MY_SOURCE (gst_pad_get_parent (pad));
+
+ switch (prop_id) {
+[..]
+ case ARG_SYNC:
+ g_value_set_boolean (value, src->sync);
+ break;
+[..]
+ }
+}
+
+static void
+gst_my_source_get_property (GObject *object,
+ guint prop_id,
+ GParamSpec *pspec,
+ const GValue *value)
+{
+ GstMySource *src = GST_MY_SOURCE (gst_pad_get_parent (pad));
+
+ switch (prop_id) {
+[..]
+ case ARG_SYNC:
+ src->sync = g_value_get_boolean (value);
+ break;
+[..]
+ }
+}
+ </programlisting>
+ <para>
+ Most of this is GObject wrapping code. The actual code to do
+ software-sync (in the <function>_get ()</function>-function)
+ is relatively small.
+ </para>
+ </sect1>
+ <sect1 id="section-source-buffers" xreflabel="Using special memory">
+ <title>Using special memory</title>
+ <para>
+ In some cases, it might be useful to use specially allocated memory
+ (e.g. <function>mmap ()</function>'ed DMA'able memory) in
+ your buffers, and those will require special handling when they are
+ being dereferenced. For this, &GStreamer; uses the concept of
+ buffer-free functions. Those are special functions pointers that an
+ element can set on buffers that it created itself. The given function
+ will be called when the buffer has been dereferenced, so that the
+ element can clean up or re-use memory internally rather than using
+ the default implementation (which simply calls
+ <function>g_free ()</function> on the data pointer).
+ </para>
+ <programlisting>
+static void
+gst_my_source_buffer_free (GstBuffer *buf)
+{
+ GstMySource *src = GST_MY_SOURCE (GST_BUFFER_PRIVATE (buf));
+
+ /* do useful things here, like re-queueing the buffer which
+ * makes it available for DMA again. The default handler will
+ * not free this buffer because of the GST_BUFFER_DONTFREE
+ * flag. */
+}
+
+static void
+gst_my_source_get (GstPad *pad)
+{
+ GstMySource *src = GST_MY_SOURCE (gst_pad_get_parent (pad));
+ GstBuffer *buf;
+[..]
+ buf = gst_buffer_new ();
+ GST_BUFFER_FREE_DATA_FUNC (buf) = gst_my_source_buffer_free;
+ GST_BUFFER_PRIVATE (buf) = src;
+ GST_BUFFER_FLAG_SET (buf, GST_BUFFER_READONLY | GST_BUFFER_DONTFREE);
+[..]
+
+ return GST_DATA (buf);
+}
+ </programlisting>
+ <para>
+ Note that this concept should <emphasis>not</emphasis> be used to
+ decrease the number of calls made to functions such as
+ <function>g_malloc ()</function> inside your element. We
+ have better ways of doing that elsewhere (&GStreamer; core, Glib,
+ Glibc, Linux kernel, etc.).
+ </para>
+ </sect1>
</chapter>
projects / platform / upstream / gstreamer.git / commitdiff