-\r
-Debug\r
-Release\r
-ipch\r
-*.user\r
-*.sdf\r
-*.suo\r
-*.opensdf\r
-vs/2010/libs\r
-bin\r
-gen\r
-libs\r
-obj\r
-.classpath\r
-.project\r
-.settings\r
-.libs\r
-.cproject\r
-gst-build\r
-project.properties\r
-gst_sdk\r
-.DS_Store\r
-xcuserdata\r
+/built_doc
+/hotdoc-private*
+Debug
+Release
+ipch
+*.user
+*.sdf
+*.suo
+*.opensdf
+vs/2010/libs
+bin
+gen
+libs
+obj
+.classpath
+.project
+.settings
+.libs
+.cproject
+gst-build
+project.properties
+gst_sdk
+.DS_Store
+xcuserdata
--- /dev/null
+# Introduction
+
+This is a collection of design documents, formerly maintained in various
+different locations and formats, now grouped together and converted
+to commonmark.
+
+# Contributing
+
+## Style
+
+We will follow the commonmark specification.
+
+We *should* try to follow this
+[style guide](http://www.cirosantilli.com/markdown-style-guide/#about),
+but are still [evaluating solutions](https://github.com/jgm/cmark/issues/131)
+for *stable* automatic formatting.
+
+80 columns line width is thus not yet enforced, but strongly suggested.
+
+# Build a web portal from the sources
+
+## Install dependencies
+
+* Follow [hotdoc's installation guide](https://people.collabora.com/~meh/hotdoc_hotdoc/html/installing.html),
+ preferably in a virtualenv.
+
+* We *experimentally* use the hotdoc C extension to include functions by
+ name, follow the steps outlined [here](https://github.com/hotdoc/hotdoc_c_extension)
+
+* Optionally install the `hotdoc_syntax_highlighting_extension`:
+
+```
+pip install hotdoc-syntax-highlighting-extension
+```
+
+* Optionally install the `hotdoc_search_extension`:
+
+```
+pip install hotdoc-search-extension
+```
+
+* Build the portal:
+
+```
+hotdoc run
+```
+
+* And browse it:
+
+```
+xdg-open build/html/index.html
+```
--- /dev/null
+# Todo
+
+This is just a simple TODO list to follow progress of the port from
+gstreamer.com content to hotdoc
+
+Pages to review:
+ - [installing]
+ - sdk-installing-on-linux.md
+ - sdk-installing-on-windows.md
+ - sdk-installing-for-android-development.md
+ - Need to update/review the plugins list
+
+
+Screenshots:
+ - Create new ones with the official GStreamer logo and not saying "0.10.36". Affected:
+ - iOS tutorial 1
+
+Code:
+ - Change logos to the official gstreamer logo
+
+Reviewed pages:
+ - index.md
+ - sdk-basic-tutorials.md
+ - sdk-basic-tutorial-concepts.md
+ - sdk-basic-tutorial-dynamic-pipelines.md
+ - sdk-basic-tutorial-time-management.md
+ - sdk-basic-tutorial-media-formats-and-pad-capabilities.md
+ - sdk-basic-tutorial-multithreading-and-pad-availability.md
+ - sdk-basic-tutorial-short-cutting-the-pipeline.md
+ - sdk-basic-tutorial-media-information-gathering.md
+ - sdk-basic-tutorial-gstreamer-tools.md
+ - sdk-basic-tutorial-debugging-tools.md
+ - sdk-basic-tutorial-streaming.md
+ - sdk-basic-tutorial-playback-speed.md
+ - sdk-basic-tutorial-handy-elements.md
+ - sdk-basic-tutorial-platform-specific-elements.md
+ - sdk-installing.md
+ - sdk-building-from-source-using-cerbero.md
+ - sdk-table-of-concepts.md
+ - sdk-tutorials.md
+ - sdk-android-tutorials.md
+ - sdk-android-tutorial-link-against-gstreamer.md
+ - sdk-android-tutorial-a-running-pipeline.md
+ - sdk-android-tutorial-video.md
+ - sdk-android-tutorial-a-complete-media-player.md
+ - sdk-android-tutorial-media-player.md
+ - sdk-ios-tutorials.md
+ - sdk-ios-tutorial-link-against-gstreamer.md
+ - sdk-ios-tutorial-a-running-pipeline.md
+ - sdk-ios-tutorial-video.md
+ - sdk-ios-tutorial-a-basic-media-player.md
+ - sdk-playback-tutorials.md
+ - sdk-playback-tutorial-playbin-usage.md
+ - sdk-playback-tutorial-subtitle-management.md
+ - sdk-playback-tutorial-short-cutting-the-pipeline.md
+ - sdk-playback-tutorial-progressive-streaming.md
+ - sdk-playback-tutorial-color-balance.md
+ - sdk-playback-tutorial-audio-visualization.md
+ - sdk-playback-tutorial-custom-playbin-sinks.md
+ - sdk-playback-tutorial-hardware-accelerated-video-decoding.md
+ - sdk-playback-tutorial-digital-audio-pass-through.md
+ - sdk-basic-tutorial-hello-world.md
+ - sdk-gst-inspect.md
+ - gst-launch.md
+ - sdk-deploying-your-application.md
+ - sdk-windows-deployment.md
+ - sdk-mac-osx-deployment.md
+ - sdk-multiplatform-deployment-using-cerbero.md
+
+For-later pages:
+ - sdk-qt-tutorials.md [tpm: this should all be rewritten from scratch with qmlglsink; QtGStreamer is outdated and unmaintained, we should not promote it]
+ - sdk-basic-media-player.md
+ - sdk-qt-gstreamer-vs-c-gstreamer.md
+ - sdk-using-appsink-appsrc-in-qt.md
+
+
+Deleted pages:
+ - 2012.11+Brahmaputra.markdown
+ - 2012.5+Amazon.markdown
+ - 2012.7+Amazon+%28Bugfix+Release+1%29.markdown
+ - 2012.9+Amazon+%28Bugfix+Release+2%29.markdown
+ - 2013.6+Congo.markdown
+ - Contact.markdown
+ - Frequently+Asked+Questions.markdown
+ - Releases.markdown
+ - Upcoming+tutorials.markdown
+
+
+old sitemap:
+
+ Installing the SDK
+ Installing on Linux
+ Installing on Mac OS X
+ Installing on Windows
+ Installing for Android development
+ Installing for iOS development
+ Building from source using Cerbero
+ Tutorials
+ Basic tutorials
+ Playback tutorials
+ Android tutorials
+ iOS tutorials
+ Table of Concepts
+ Upcoming tutorials
+ Deploying your application
+ Mac OS X deployment
+ Windows deployment
+ Multiplatform deployment using Cerbero
+ Releases
+ 2013.6 Congo
+ 2012.11 Brahmaputra
+ 2012.9 Amazon (Bugfix Release 2)
+ 2012.7 Amazon (Bugfix Release 1)
+ 2012.5 Amazon
+ GStreamer reference
+ gst-inspect
+ gst-launch
+ Legal information
+ Frequently Asked Questions
+ Contact
--- /dev/null
+hotdoc run
+#cp -R attachments built_doc/html/
+#cp -R images built_doc/html/
--- /dev/null
+#include <gst/gst.h>
+
+int main(int argc, char *argv[]) {
+ GstElement *pipeline;
+ GstBus *bus;
+ GstMessage *msg;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Build the pipeline */
+ pipeline = gst_parse_launch ("playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
+
+ /* Start playing */
+ gst_element_set_state (pipeline, GST_STATE_PLAYING);
+
+ /* Wait until error or EOS */
+ bus = gst_element_get_bus (pipeline);
+ msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
+
+ /* Free resources */
+ if (msg != NULL)
+ gst_message_unref (msg);
+ gst_object_unref (bus);
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+ return 0;
+}
--- /dev/null
+#include <gst/gst.h>
+
+static GMainLoop *loop;
+
+static gboolean
+my_bus_callback (GstBus *bus,
+ GstMessage *message,
+ gpointer data)
+{
+ g_print ("Got %s message\n", GST_MESSAGE_TYPE_NAME (message));
+
+ switch (GST_MESSAGE_TYPE (message)) {
+ case GST_MESSAGE_ERROR: {
+ GError *err;
+ gchar *debug;
+
+ gst_message_parse_error (message, &err, &debug);
+ g_print ("Error: %s\n", err->message);
+ g_error_free (err);
+ g_free (debug);
+
+ g_main_loop_quit (loop);
+ break;
+ }
+ case GST_MESSAGE_EOS:
+ /* end-of-stream */
+ g_main_loop_quit (loop);
+ break;
+ default:
+ /* unhandled message */
+ break;
+ }
+
+ /* we want to be notified again the next time there is a message
+ * on the bus, so returning TRUE (FALSE means we want to stop watching
+ * for messages on the bus and our callback should not be called again)
+ */
+ return TRUE;
+}
+
+gint
+main (gint argc,
+ gchar *argv[])
+{
+ GstElement *pipeline;
+ GstBus *bus;
+ guint bus_watch_id;
+
+ /* init */
+ gst_init (&argc, &argv);
+
+ /* create pipeline, add handler */
+ pipeline = gst_pipeline_new ("my_pipeline");
+
+ /* adds a watch for new message on our pipeline's message bus to
+ * the default GLib main context, which is the main context that our
+ * GLib main loop is attached to below
+ */
+ bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
+ bus_watch_id = gst_bus_add_watch (bus, my_bus_callback, NULL);
+ gst_object_unref (bus);
+
+ /* [...] */
+
+ /* create a mainloop that runs/iterates the default GLib main context
+ * (context NULL), in other words: makes the context check if anything
+ * it watches for has happened. When a message has been posted on the
+ * bus, the default main context will automatically call our
+ * my_bus_callback() function to notify us of that message.
+ * The main loop will be run until someone calls g_main_loop_quit()
+ */
+ loop = g_main_loop_new (NULL, FALSE);
+ g_main_loop_run (loop);
+
+ /* clean up */
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+ g_source_remove (bus_watch_id);
+ g_main_loop_unref (loop);
+
+ return 0;
+}
--- /dev/null
+#include <gst/gst.h>
+
+static void
+link_to_multiplexer (GstPad *tolink_pad,
+ GstElement *mux)
+{
+ GstPad *pad;
+ gchar *srcname, *sinkname;
+
+ srcname = gst_pad_get_name (tolink_pad);
+ pad = gst_element_get_compatible_pad (mux, tolink_pad, NULL);
+ gst_pad_link (tolink_pad, pad);
+ sinkname = gst_pad_get_name (pad);
+ gst_object_unref (GST_OBJECT (pad));
+
+ g_print ("A new pad %s was created and linked to %s\n", sinkname, srcname);
+ g_free (sinkname);
+ g_free (srcname);
+}
+
+static void
+some_function (GstElement *tee)
+{
+ GstPad * pad;
+ gchar *name;
+
+ pad = gst_element_get_request_pad (tee, "src%d");
+ name = gst_pad_get_name (pad);
+ g_print ("A new pad %s was created\n", name);
+ g_free (name);
+
+ /* here, you would link the pad */
+
+ /* [..] */
+
+ /* and, after doing that, free our reference */
+ gst_object_unref (GST_OBJECT (pad));
+}
--- /dev/null
+---
+short-description: The GStreamer Editing Services prototyping tool
+...
+
+# ges-launch-1.0
+
+**ges-launch-1.0** creates a multimedia [timeline](https://phabricator.freedesktop.org/w/gstreamer/gst-editing-services/ges-timeline/) and plays it back, or renders it to the specified format.
+
+It can load a timeline from an existing project, or create one from the specified commands.
+
+Updating an existing project can be done through thanks to the [GstValidate](http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-validate/html/) [scenarios](http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-validate/html/scenarios.html) using the `--set-scenario` argument, if ges-launch-1.0 has been compiled with GstValidate.
+
+You can inspect action types with:
+
+ ges-launch-1.0 --inspect-action-type
+
+By default, ges-launch-1.0 is in "playback-mode".
+
+## Synopsis
+
+ ges-launch-1.0' [-l <path>|--load=<path>] [-s <path>|--save=<path>]
+ [-p <path>|--sample-path=<path>] [-r <path>|--sample-path-recurse=<path>]
+ [-o <uri>|--outputuri=<uri>] [-f <profile>|--format=<profile>]
+ [-e <profile-name>|--encoding-profile=<profile-name>]
+ [-t <track-types>|--track-types=<track-types>]
+ [-v <videosink>|--videosink=<videosink>]
+ [-a <audiosink>---audiosink=<audiosink>]
+ [-m|--mute] [--inspect-action-type[=<action-type>]]
+ [--list-transitions] [--disable-mixing]
+ [-r <times>|--repeat=<times>] [--set-scenario=<scenario-name]
+
+## Define a timeline through the command line
+
+The ges-launch-1.0 tool allows you to simply build a timeline through a dedicated set of commands:
+
+### +clip
+
+Adds a clip to the timeline.
+
+See documentation for the --track-types option to ges-launch-1.0, as it
+will affect the result of this command.
+
+#### Examples:
+
+ ges-launch-1.0 +clip /path/to/media
+
+This will simply play the sample from its beginning to its end.
+
+ ges-launch-1.0 +clip /path/to/media inpoint=4.0
+
+Assuming "media" is a 10 second long media sample, this will play the sample from the 4th second to the 10th, resulting in a 6-seconds long playback.
+
+ ges-launch-1.0 +clip /path/to/media inpoint=4.0 duration=2.0 start=4.0
+
+Assuming "media" is an audio video sample longer than 6 seconds, this will play a black frame and silence for 4 seconds, then the sample from its 4th second to its sixth second, resulting in a 6-seconds long playback.
+
+ ges-launch-1.0 --track-types=audio +clip /path/to/media
+
+Assuming "media" is an audio video sample, this will only play the audio of the sample in its entirety.
+
+ ges-launch-1.0 +clip /path/to/media1 layer=1 set-alpha 0.9 +clip /path/to/media2 layer=0
+
+Assume media1 and media2 both contain audio and video and last for 10 seconds.
+
+This will first add media1 in a new layer of "priority" 1, thus implicitly creating a layer of "priority" 0, the start of the clip will be 0 as no clip had been added in that layer before.
+
+It will then add media2 in the layer of "priority" 0 which was created previously, the start of this new clip will also be 0 as no clip has been added in this layer before.
+
+Both clips will thus overlap on two layers for 10 seconds.
+
+The "alpha" property of the second clip will finally be set to a value of 0.9.
+
+All this will result in a 10 seconds playback, where media2 is barely visible through media1, which is nearly opaque. If alpha was set to 0.5, both clips would be equally visible, and if it was set to 0.0, media1 would be invisible and media2 completely opaque.
+
+#### Mandatory arguments
+
+__path|uri:__
+```
+Specifies the location of the sample to make a clip from.
+```
+
+#### Options
+
+__inpoint[i]=<inpoint>:__
+```
+Sets the inpoint of the clip, that is the
+position in the original sample at which the clip will start outputting
+data.
+
+It is an error to have an inpoint superior to the actual duration of the original sample.
+
+0 by default.
+```
+
+__duration[i]=<duration>:__
+```
+Sets the duration of the clip, that is the
+duration of the media the clip will output.
+
+It is an error to have inpoint + duration be superior to the duration of the
+original sample.
+
+The default is the duration of the original sample - the inpoint of the clip.
+```
+
+__start[s]=<start>:__
+
+```
+Sets the start of the clip, that is its position in
+the timeline.
+
+If not specified, it will be set to the duration of the layer the clip is added on,
+as the expected default behaviour is to queue clips one after another.
+```
+
+__layer[l]=<layer>:__
+```
+Sets the layer of the clip. The video stream in
+overlapping clips on different layers will be blended together according
+to their alpha property, starting with the clip on the last layer. An
+example is shown in the EXAMPLES section.
+
+If not specified, it will be set to the last layer a clip has been added on, or
+a first layer if no clip has been added yet.
+```
+
+#### Properties
+
+##### Video properties
+
+These have no effects if there is no video stream in the sample.
+
+__alpha:__
+```
+This is the amount of transparency of the clip, ranging from 0.0
+to 1.0 Clips overlapping on different layers will be composited
+together, unless --disable-mixing has been specified, in the order of
+the layers.
+```
+
+__posx:__
+```
+This is the x position (offset) of the clip in pixels, relatively
+to the output frame of the timeline.
+```
+
+__posy:__
+```
+This is the y position (offset) of the clip in pixels, relatively
+to the output frame of the timeline.
+```
+
+__width:__
+```
+This is the width in pixels that the clip will occupy in the
+final output frame.
+```
+
+__height:__
+```
+This is the height in pixels that the clip will occupy in the final output frame.
+```
+
+##### Audio properties
+
+__volume:__
+```
+This is the volume that will be set on the audio part of the
+clip, ranging from 0.0 to 10.0, with 1.0 being the default.
+```
+
+__mute:__
+```
+Set to true to mute the audio of the clip. Default is false.
+```
+
+### +effect
+
+#### Mandatory arguments
+
+__bin-description:__
+
+```
+Specifies the description of a GStreamer a bin, in the gst-launch format.
+```
+
+#### Options
+
+Properties can be set on the effect either directly in the bin-description, or separately through the set-<property-name> command, which will lookup any readable property in any of the children of the bin, and set the provided value on it.
+
+#### Examples
+
+ ges-launch-1.0 +clip /path/to/media +effect "agingtv"
+
+This will apply the agingtv effect to "media" and play it back.
+
+### set-<property-name>
+
+Sets the property of an object (for example a clip or an effect). Trying to set a property than can't be looked up is considered an error.
+
+By default, set-<property-name> will lookup the property on the last added object.
+
+#### Examples
+
+ ges-launch-1.0 +clip /path/to/media set-alpha 0.3
+
+This will set the alpha property on "media" then play it back, assuming "media"
+contains a video stream.
+
+ ges-launch-1.0 +clip /path/to/media +effect "agingtv" set-dusts false
+
+This will set the "dusts" property of the agingtv to false and play the
+timeline back.
+
+## Options
+
+### Project-related options
+
+__--load[-l]=<path>:__
+```
+Load project from file. The project be saved again with the --save option.
+```
+
+__-s --save=<path>:__
+```
+Save project to file before rendering. It can then be loaded with the --load option
+```
+
+__-p --sample-path:__
+```
+If some assets are missing when loading a project file, ges-launch-1.0 will try to
+locate them in this path. It is especially useful when sharing a project.
+```
+
+__-r --sample-path-recurse:__
+```
+Identical to --sample-path, but ges-launch-1.0 will also recurse in the subfolders
+to look for assets.
+```
+
+### Rendering options
+
+__-o --outputuri=<uri>:__
+```
+If set, ges-launch-1.0 will render the specified timeline instead
+of playing it back. The default rendering format is ogv, containing
+theora and vorbis.
+```
+
+__-f --format=<profile>:__
+```
+Set an encoding profile on the command line. See ges-launch-1.0 help profile
+for more information.
+This will have no effect if no outputuri has been specified.
+```
+
+__-e --encoding-profile=<profile-name>:__
+```
+Set an encoding profile from a preset file. See ges-launch-1.0 help profile
+for more information.
+This will have no effect if no outputuri has been specified.
+```
+
+__-t --track-types=<track-types>:__
+```
+Specify the track types to be created. When loading a project, only relevant
+tracks will be added to the timeline.
+```
+
+### Playback options
+
+__-v --videosink=<videosink>:__
+```
+Set the videosink used for playback.
+```
+
+__-a --audiosink=<audiosink>:__
+```
+Set the audiosink used for playback.
+```
+
+_ _-m --mute:__
+```
+Mute playback output. This has no effect when rendering.
+```
+
+### Helpful options
+
+__--inspect-action-type=<action-type>:__
+```
+Inspect the available action types that can be defined in a scenario set with --set-scenario.
+Will list all action-types if action-type is empty.
+```
+
+__--list-transitions:__
+```
+List all valid transition types and exit. See ges-launch-1.0 help transition
+for more information.
+```
+
+### Generic options
+
+__--disable-mixing:__
+```
+Do not use mixing elements to mix layers together.
+```
+
+__-r --repeat=<times>:__
+```
+Set the number of times to repeat the timeline.
+```
+
+__--set-scenario:__
+```
+ges-launch-1.0 exposes gst-validate functionalities, such as scenarios.
+Scenarios describe actions to execute, such as seeks or setting of properties.
+GES implements editing-specific actions such as adding or removal of clips.
+```
--- /dev/null
+# gst-inspect-1.0
+
+> ![information] This is the Linux man page for
+> the `gst-inspect-1.0` tool. As such, it is very Linux-centric
+> regarding path specification and plugin names. Please be patient while
+> it is rewritten to be more generic.
+
+## Name
+
+gst-inspect-1.0 - print info about a GStreamer plugin or element
+
+## Synopsis
+
+**gst-inspect-1.0 \[OPTION...\] \[PLUGIN|ELEMENT\]**
+
+## Description
+
+*gst-inspect-1.0* is a tool that prints out information on
+available *GStreamer* plugins, information about a particular plugin, or
+information about a particular element. When executed with no PLUGIN or
+ELEMENT argument, *gst-inspect-1.0* will print a list of all plugins and
+elements together with a sumary. When executed with a PLUGIN or ELEMENT
+argument, *gst-inspect-1.0* will print information about that plug-in or
+element.
+
+## Options
+
+*gst-inspect-1.0* accepts the following arguments and options:
+
+**PLUGIN**
+
+Name of a plugin. This is a file name
+like `%GSTREAMER_ROOT_X86%\lib\gstreamer-1.0\libgstaudiotestsrc.dll`
+for example.
+
+**ELEMENT**
+
+Name of an element. This is the name of an element, like
+`audiotestsrc` for example
+
+**--help**
+
+Print help synopsis and available FLAGS
+
+**--gst-info-mask=FLAGS**
+
+*GStreamer* info flags to set (list with --help)
+
+ **-a, --print-all**
+
+Print all plugins and elements
+
+ **--print-plugin-auto-install-info**
+
+Print a machine-parsable list of features the specified plugin provides.
+Useful in connection with external automatic plugin installation
+mechanisms.
+
+ **--gst-debug-mask=FLAGS**
+
+*GStreamer* debugging flags to set (list with --help)
+
+ **--gst-mask=FLAGS**
+
+*GStreamer* info and debugging flags to set (list with --help)
+
+ **--gst-plugin-spew**
+
+*GStreamer* info flags to set Enable printout of errors while
+loading *GStreamer* plugins
+
+ **--gst-plugin-path=PATH**
+
+Add directories separated with ':' to the plugin search path
+
+## Example
+
+ gst-inspect-1.0 audiotestsrc
+
+should produce:
+
+ Factory Details:
+ Rank none (0)
+ Long-name Audio test source
+ Klass Source/Audio
+ Description Creates audio test signals of given frequency and volume
+ Author Stefan Kost <ensonic@users.sf.net>
+
+ Plugin Details:
+ Name audiotestsrc
+ Description Creates audio test signals of given frequency and volume
+ Filename /usr/lib/gstreamer-1.0/libgstaudiotestsrc.so
+ Version 1.8.1
+ License LGPL
+ Source module gst-plugins-base
+ Source release date 2016-04-20
+ Binary package GStreamer Base Plugins (Arch Linux)
+ Origin URL http://www.archlinux.org/
+
+ GObject
+ +----GInitiallyUnowned
+ +----GstObject
+ +----GstElement
+ +----GstBaseSrc
+ +----GstAudioTestSrc
+
+ Pad Templates:
+ SRC template: 'src'
+ Availability: Always
+ Capabilities:
+ audio/x-raw
+ format: { S16LE, S16BE, U16LE, U16BE, S24_32LE, S24_32BE, U24_32LE, U24_32BE, S32LE, S32BE, U32LE, U32BE, S24LE, S24BE, U24LE, U24BE, S20LE, S20BE, U20LE, U20BE, S18LE, S18BE, U18LE, U18BE, F32LE, F32BE, F64LE, F64BE, S8, U8 }
+ layout: interleaved
+ rate: [ 1, 2147483647 ]
+ channels: [ 1, 2147483647 ]
+
+ Element Flags:
+ no flags set
+
+ Element Implementation:
+ Has change_state() function: gst_base_src_change_state
+
+ Element has no clocking capabilities.
+ Element has no URI handling capabilities.
+
+ Pads:
+ SRC: 'src'
+ Pad Template: 'src'
+
+ Element Properties:
+ name : The name of the object
+ flags: readable, writable
+ String. Default: "audiotestsrc0"
+ parent : The parent of the object
+ flags: readable, writable
+ Object of type "GstObject"
+ blocksize : Size in bytes to read per buffer (-1 = default)
+ flags: readable, writable
+ Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
+ num-buffers : Number of buffers to output before sending EOS (-1 = unlimited)
+ flags: readable, writable
+ Integer. Range: -1 - 2147483647 Default: -1
+ typefind : Run typefind before negotiating
+ flags: readable, writable
+ Boolean. Default: false
+ do-timestamp : Apply current stream time to buffers
+ flags: readable, writable
+ Boolean. Default: false
+ samplesperbuffer : Number of samples in each outgoing buffer
+ flags: readable, writable
+ Integer. Range: 1 - 2147483647 Default: 1024
+ wave : Oscillator waveform
+ flags: readable, writable, controllable
+ Enum "GstAudioTestSrcWave" Default: 0, "sine"
+ (0): sine - Sine
+ (1): square - Square
+ (2): saw - Saw
+ (3): triangle - Triangle
+ (4): silence - Silence
+ (5): white-noise - White uniform noise
+ (6): pink-noise - Pink noise
+ (7): sine-table - Sine table
+ (8): ticks - Periodic Ticks
+ (9): gaussian-noise - White Gaussian noise
+ (10): red-noise - Red (brownian) noise
+ (11): blue-noise - Blue noise
+ (12): violet-noise - Violet noise
+ freq : Frequency of test signal. The sample rate needs to be at least 4 times higher.
+ flags: readable, writable, controllable
+ Double. Range: 0 - 5.368709e+08 Default: 440
+ volume : Volume of test signal
+ flags: readable, writable, controllable
+ Double. Range: 0 - 1 Default: 0.8
+ is-live : Whether to act as a live source
+ flags: readable, writable
+ Boolean. Default: false
+ timestamp-offset : An offset added to timestamps set on buffers (in ns)
+ flags: readable, writable
+ Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0
+ can-activate-push : Can activate in push mode
+ flags: readable, writable
+ Boolean. Default: true
+ can-activate-pull : Can activate in pull mode
+ flags: readable, writable
+ Boolean. Default: false
+
+ [information]: images/icons/emoticons/information.png
--- /dev/null
+# gst-launch-1.0
+
+> ![information] This is the Linux man page for
+> the `gst-inspect-1.0` tool. As such, it is very Linux-centric
+> regarding path specification and plugin names. Please be patient while
+> it is rewritten to be more generic.
+
+## Name
+
+gst-launch-1.0 - build and run a GStreamer pipeline
+
+## Synopsis
+
+**gst-launch-1.0** *\[OPTION...\]* PIPELINE-DESCRIPTION
+
+## Description
+
+*gst-launch-1.0* is a tool that builds and runs
+basic *GStreamer* pipelines.
+
+In simple form, a PIPELINE-DESCRIPTION is a list of elements separated
+by exclamation marks (!). Properties may be appended to elements, in the
+form*property=value*.
+
+For a complete description of possible PIPELINE-DESCRIPTIONS see the
+section*pipeline description* below or consult the GStreamer
+documentation.
+
+Please note that *gst-launch-1.0* is primarily a debugging tool for
+developers and users. You should not build applications on top of it.
+For applications, use the gst\_parse\_launch() function of the GStreamer
+API as an easy way to construct pipelines from pipeline descriptions.
+
+## Options
+
+*gst-launch-1.0* accepts the following options:
+
+**--help**
+
+Print help synopsis and available FLAGS
+
+**-v, --verbose**
+
+Output status information and property notifications
+
+**-q, --quiet**
+
+Do not print any progress information
+
+**-m, --messages**
+
+Output messages posted on the pipeline's bus
+
+**-t, --tags**
+
+Output tags (also known as metadata)
+
+**-o FILE, --output=FILE**
+
+Save XML representation of pipeline to FILE and exit
+
+**-f, --no\_fault**
+
+Do not install a fault handler
+
+**-T, --trace**
+
+Print memory allocation traces. The feature must be enabled at compile
+time to work.
+
+
+
+## Gstreamer Options
+
+*gst-launch-1.0* also accepts the following options that are common to
+all GStreamer applications:
+
+## Pipeline Description
+
+A pipeline consists *elements* and *links*. *Elements* can be put
+into *bins* of different sorts. *Elements*, *links* and *bins* can be
+specified in a pipeline description in any order.
+
+**Elements**
+
+ELEMENTTYPE *\[PROPERTY1 ...\]*
+
+Creates an element of type ELEMENTTYPE and sets the PROPERTIES.
+
+**Properties**
+
+PROPERTY=VALUE ...
+
+Sets the property to the specified value. You can
+use **gst-inspect-1.0**(1) to find out about properties and allowed
+values of different elements. Enumeration properties can be set by name,
+nick or value.
+
+**Bins**
+
+*\[BINTYPE.\]* ( *\[PROPERTY1 ...\]* PIPELINE-DESCRIPTION )
+
+Specifies that a bin of type BINTYPE is created and the given properties
+are set. Every element between the braces is put into the bin. Please
+note the dot that has to be used after the BINTYPE. You will almost
+never need this functionality, it is only really useful for applications
+using the gst\_launch\_parse() API with 'bin' as bintype. That way it is
+possible to build partial pipelines instead of a full-fledged top-level
+pipeline.
+
+**Links**
+
+*\[\[SRCELEMENT\].\[PAD1,...\]\]* ! *\[\[SINKELEMENT\].\[PAD1,...\]\]
+\[\[SRCELEMENT\].\[PAD1,...\]\]* ! CAPS
+! *\[\[SINKELEMENT\].\[PAD1,...\]\]*
+
+Links the element with name SRCELEMENT to the element with name
+SINKELEMENT, using the caps specified in CAPS as a filter. Names can be
+set on elements with the name property. If the name is omitted, the
+element that was specified directly in front of or after the link is
+used. This works across bins. If a padname is given, the link is done
+with these pads. If no pad names are given all possibilities are tried
+and a matching pad is used. If multiple padnames are given, both sides
+must have the same number of pads specified and multiple links are done
+in the given order. So the simplest link is a simple exclamation mark,
+that links the element to the left of it to the element right of it.
+
+**Caps**
+
+MIMETYPE *\[, PROPERTY\[, PROPERTY ...\]\]\] \[; CAPS\[; CAPS ...\]\]*
+
+Creates a capability with the given mimetype and optionally with given
+properties. The mimetype can be escaped using " or '. If you want to
+chain caps, you can add more caps in the same format afterwards.
+
+**Properties**
+
+NAME=*\[(TYPE)\]*VALUE in lists and ranges: *\[(TYPE)\]*VALUE
+
+Sets the requested property in capabilities. The name is an alphanumeric
+value and the type can have the following case-insensitive values:
+- **i** or **int** for integer values or ranges - **f** or **float** for
+float values or ranges - **4** or **fourcc** for FOURCC values
+- **b**, **bool** or **boolean** for boolean values
+- **s**, **str** or **string** for strings - **fraction** for fractions
+(framerate, pixel-aspect-ratio) - **l** or **list** for lists If no type
+was given, the following order is tried: integer, float, boolean,
+string. Integer values must be parsable by **strtol()**, floats
+by **strtod()**. FOURCC values may either be integers or strings.
+Boolean values are (case insensitive) *yes*, *no*, *true* or *false* and
+may like strings be escaped with " or '. Ranges are in this format: \[
+VALUE, VALUE \] Lists use this format: ( VALUE *\[, VALUE ...\]* )
+
+## Pipeline Control
+
+A pipeline can be controlled by signals. SIGUSR2 will stop the pipeline
+(GST\_STATE\_NULL); SIGUSR1 will put it back to play
+(GST\_STATE\_PLAYING). By default, the pipeline will start in the
+playing state. There are currently no signals defined to go into the
+ready or pause (GST\_STATE\_READY and GST\_STATE\_PAUSED) state
+explicitely.
+
+## Pipeline Examples
+
+The examples below assume that you have the correct plug-ins available.
+In general, "osssink" can be substituted with another audio output
+plug-in such as "directsoundsink", "esdsink", "alsasink",
+"osxaudiosink", or "artsdsink". Likewise, "xvimagesink" can be
+substituted with "d3dvideosink", "ximagesink", "sdlvideosink",
+"osxvideosink", or "aasink". Keep in mind though that different sinks
+might accept different formats and even the same sink might accept
+different formats on different machines, so you might need to add
+converter elements like audioconvert and audioresample (for audio) or
+videoconvert (for video) in front of the sink to make things work.
+
+**Audio playback**
+
+`gst-launch-1.0 filesrc location=music.mp3 ! mad ! audioconvert !
+audioresample ! osssink` Play the mp3 music file "music.mp3" using a
+libmad-based plug-in and output to an OSS device
+
+`gst-launch-1.0 filesrc location=music.ogg ! oggdemux ! vorbisdec !
+audioconvert ! audioresample ! osssink` Play an Ogg Vorbis format file
+
+`gst-launch-1.0 gnomevfssrc location=music.mp3 ! mad ! osssink
+gst-launch-1.0 gnomevfssrc location=<http://domain.com/music.mp3> ! mad
+! audioconvert ! audioresample ! osssink` Play an mp3 file or an http
+stream using GNOME-VFS
+
+`gst-launch-1.0 gnomevfssrc location=<smb://computer/music.mp3> ! mad !
+audioconvert ! audioresample ! osssink` Use GNOME-VFS to play an mp3
+file located on an SMB server
+
+**Format conversion**
+
+`gst-launch-1.0 filesrc location=music.mp3 ! mad ! audioconvert !
+vorbisenc ! oggmux ! filesink location=music.ogg` Convert an mp3 music
+file to an Ogg Vorbis file
+
+`gst-launch-1.0 filesrc location=music.mp3 ! mad ! audioconvert !
+flacenc ! filesink location=test.flac` Convert to the FLAC format
+
+**Other**
+
+`gst-launch-1.0 filesrc location=music.wav ! wavparse ! audioconvert !
+audioresample ! osssink` Plays a .WAV file that contains raw audio data
+(PCM).
+
+`gst-launch-1.0 filesrc location=music.wav ! wavparse ! audioconvert !
+vorbisenc ! oggmux ! filesink location=music.ogg gst-launch-1.0 filesrc
+location=music.wav ! wavparse ! audioconvert ! lame ! filesink
+location=music.mp3` Convert a .WAV file containing raw audio data into
+an Ogg Vorbis or mp3 file
+
+`gst-launch-1.0 cdparanoiasrc mode=continuous ! audioconvert ! lame !
+id3v2mux ! filesink location=cd.mp3` rips all tracks from compact disc
+and convert them into a single mp3 file
+
+`gst-launch-1.0 cdparanoiasrc track=5 ! audioconvert ! lame ! id3v2mux
+! filesink location=track5.mp3` rips track 5 from the CD and converts
+it into a single mp3 file
+
+Using **gst-inspect-1.0**(1), it is possible to discover settings like
+the above for cdparanoiasrc that will tell it to rip the entire cd or
+only tracks of it. Alternatively, you can use an URI and gst-launch-1.0
+will find an element (such as cdparanoia) that supports that protocol
+for you, e.g.: `gst-launch-1.0 \[cdda://5\] ! lame vbr=new
+vbr-quality=6 ! filesink location=track5.mp3`
+
+`gst-launch-1.0 osssrc ! audioconvert ! vorbisenc ! oggmux ! filesink
+location=input.ogg` records sound from your audio input and encodes it
+into an ogg file
+
+**Video**
+
+`gst-launch-1.0 filesrc location=JB\_FF9\_TheGravityOfLove.mpg !
+dvddemux ! mpeg2dec ! xvimagesink` Display only the video portion of an
+MPEG-1 video file, outputting to an X display window
+
+`gst-launch-1.0 filesrc location=/flflfj.vob ! dvddemux ! mpeg2dec !
+sdlvideosink` Display the video portion of a .vob file (used on DVDs),
+outputting to an SDL window
+
+`gst-launch-1.0 filesrc location=movie.mpg ! dvddemux name=demuxer
+demuxer. ! queue ! mpeg2dec ! sdlvideosink demuxer. ! queue ! mad !
+audioconvert ! audioresample ! osssink` Play both video and audio
+portions of an MPEG movie
+
+`gst-launch-1.0 filesrc location=movie.mpg ! mpegdemux name=demuxer
+demuxer. ! queue ! mpeg2dec ! videoconvert ! sdlvideosink demuxer. !
+queue ! mad ! audioconvert ! audioresample ! osssink` Play an AVI movie
+with an external text subtitle stream
+
+This example also shows how to refer to specific pads by name if an
+element (here: textoverlay) has multiple sink or source pads.
+
+`gst-launch-1.0 textoverlay name=overlay ! videoconvert !
+videoscale ! autovideosink filesrc location=movie.avi ! decodebin2 !
+videoconvert ! overlay.video\_sink filesrc location=movie.srt !
+subparse ! overlay.text\_sink`
+
+Play an AVI movie with an external text subtitle stream using playbin
+
+`gst-launch-1.0 playbin uri=<file:///path/to/movie.avi>
+suburi=<file:///path/to/movie.srt>`
+
+**Network streaming**
+
+Stream video using RTP and network elements.
+
+`gst-launch-1.0 v4l2src !
+video/x-raw-yuv,width=128,height=96,format='(fourcc)'UYVY !
+videoconvert ! ffenc\_h263 ! video/x-h263 ! rtph263ppay pt=96 !
+udpsink host=192.168.1.1 port=5000 sync=false` Use this command on the
+receiver
+
+`gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,
+clock-rate=90000,payload=96 ! rtph263pdepay queue-delay=0 ! ffdec\_h263
+! xvimagesink` This command would be run on the transmitter
+
+**Diagnostic**
+
+`gst-launch-1.0 -v fakesrc num-buffers=16 ! fakesink` Generate a null
+stream and ignore it (and print out details).
+
+`gst-launch-1.0 audiotestsrc ! audioconvert ! audioresample ! osssink`
+Generate a pure sine tone to test the audio output
+
+`gst-launch-1.0 videotestsrc ! xvimagesink gst-launch-1.0 videotestsrc
+! ximagesink` Generate a familiar test pattern to test the video output
+
+**Automatic linking**
+
+You can use the decodebin element to automatically select the right
+elements to get a working pipeline.
+
+`gst-launch-1.0 filesrc location=musicfile ! decodebin ! audioconvert !
+audioresample ! osssink` Play any supported audio format
+
+`gst-launch-1.0 filesrc location=videofile ! decodebin name=decoder
+decoder. ! queue ! audioconvert ! audioresample ! osssink decoder. !
+videoconvert ! xvimagesink` Play any supported video format with
+video and audio output. Threads are used automatically. To make this
+even easier, you can use the playbin element:
+
+`gst-launch-1.0 playbin uri=<file:///home/joe/foo.avi>`
+
+**Filtered connections**
+
+These examples show you how to use filtered caps.
+
+`gst-launch-1.0 videotestsrc !
+'video/x-raw-yuv,format=(fourcc)YUY2;video/x-raw-yuv,format=(fourcc)YV12'
+! xvimagesink` Show a test image and use the YUY2 or YV12 video format
+for this.
+
+`gst-launch-1.0 osssrc !
+'audio/x-raw-int,rate=\[32000,64000\],width=\[16,32\],depth={16,24,32},signed=(boolean)true'
+! wavenc ! filesink location=recording.wav` record audio and write it
+to a .wav file. Force usage of signed 16 to 32 bit samples and a sample
+rate between 32kHz and 64KHz.
+
+## Environment Variables
+
+`GST\_DEBUG`: Comma-separated list of debug categories and levels,
+e.g. GST\_DEBUG= totem:4,typefind:5
+
+`GST\_DEBUG\_NO\_COLOR`: When this environment variable is set,
+coloured debug output is disabled.
+
+`GST\_DEBUG\_DUMP\_DOT\_DIR`: When set to a filesystem path, store dot
+files of pipeline graphs there.
+
+`GST\_REGISTRY`: Path of the plugin registry file. Default is
+\~/.gstreamer-1.0/registry-CPU.xml where CPU is the machine/cpu type
+GStreamer was compiled for, e.g. 'i486', 'i686', 'x86-64', 'ppc', etc.
+(check the output of "uname -i" and "uname -m" for details).
+
+`GST\_REGISTRY\_UPDATE`: Set to "no" to force GStreamer to assume that
+no plugins have changed, been added or been removed. This will make
+GStreamer skip the initial check whether a rebuild of the registry cache
+is required or not. This may be useful in embedded environments where
+the installed plugins never change. Do not use this option in any other
+setup.
+
+`GST\_PLUGIN\_PATH`: Specifies a list of directories to scan for
+additional plugins. These take precedence over the system plugins.
+
+`GST\_PLUGIN\_SYSTEM\_PATH`: Specifies a list of plugins that are
+always loaded by default. If not set, this defaults to the
+system-installed path, and the plugins installed in the user's home
+directory
+
+`OIL\_CPU\_FLAGS`: Useful liboil environment variable. Set
+OIL\_CPU\_FLAGS=0 when valgrind or other debugging tools trip over
+liboil's CPU detection (quite a few important GStreamer plugins like
+videotestsrc, audioconvert or audioresample use liboil).
+
+`G\_DEBUG`: Useful GLib environment variable. Set
+G\_DEBUG=fatal\_warnings to make GStreamer programs abort when a
+critical warning such as an assertion failure occurs. This is useful if
+you want to find out which part of the code caused that warning to be
+triggered and under what circumstances. Simply set G\_DEBUG as mentioned
+above and run the program in gdb (or let it core dump). Then get a stack
+trace in the usual way
+
+ [information]: images/icons/emoticons/information.png
--- /dev/null
+---
+short-description: Command line tools
+...
+
+# Command line tools
+
+GStreamer comes with several command line tools mostly to help developers get started
+and prototype there application.
--- /dev/null
+{
+ "add_anchors": true,
+ "extra_assets": [
+ "attachments",
+ "images"
+ ],
+ "index": "index.md",
+ "output": "built_doc",
+ "pkg_config_packages": [
+ "gstreamer-1.0"
+ ],
+ "project_name": "gstdotcom",
+ "sitemap": "sitemap.txt",
+ "syntax_highlighting_activate": true
+}
\ No newline at end of file
--- /dev/null
+# GStreamer documentation
+
+Feel free to jump straight to the download section, start practicing
+with the tutorials, or read the F.A.Q. if you don’t know what this is
+all about.
+
+
+
+## General
+
+| - | - |
+| Application Development Manual (Read this first) | [HTML](manual-index.md) |
+| Frequently Asked Questions | [HTML](/data/doc/gstreamer/head/faq/html/index.html) \| [PDF](/data/doc/gstreamer/head/faq/faq.pdf) |
+| Plugin Writer's Guide | [HTML](pwg-index.md) |
+| Core Reference | <a href="/data/doc/gstreamer/head/gstreamer/html/">HTML</a> |
+| Core Libraries Reference | <a href="/data/doc/gstreamer/head/gstreamer-libs/html/">HTML</a> |
+| Core Design Documentation | <a href="http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/">HTML</a> |
+| GStreamer 0.10 to 1.0 porting guide | <a href="http://cgit.freedesktop.org/gstreamer/gstreamer/plain/docs/random/porting-to-1.0.txt">HTML</a> |
+
+<!-- FIXME: save useful bits from wiki
+| GStreamer Wiki (see esp. <a href="&site;/wiki/ReleasePlanning">ReleasePlanning</a> and <a href="&site;/wiki/SubmittingPatches">SubmittingPatches</a>)
+| <a href="&site;/wiki/">HTML</a> |
+-->
+
+
+## GStreamer Plugins-Base Module Libraries Reference
+
+|-|-|
+| GStreamer Base Plugins Libraries Reference | <a href="/data/doc/gstreamer/head/gst-plugins-base-libs/html/">HTML</a> |
+
+## GStreamer Plugins-Bad Module Libraries Reference
+
+Note: The API from these gst-plugins-bad libraries is unstable and can
+change between releases.
+
+Only the API in libraries from GStreamer core and
+gst-plugins-base are guaranteed to be API and ABI stable
+
+| GStreamer Bad Plugins Libraries Reference | <a href="/data/doc/gstreamer/head/gst-plugins-bad-libs/html/">HTML</a> |
+
+## Plugin Modules
+
+|-|-|
+| Overview of all Plug-ins | <a href="plugins.html">HTML</a> |
+| GStreamer Core Plugins Reference | <a href="/data/doc/gstreamer/head/gstreamer-plugins/html/">HTML</a> |
+
+| GStreamer Base Plugins Reference | <a href="/data/doc/gstreamer/head/gst-plugins-base-plugins/html/">HTML</a> |
+| GStreamer Good Plugins Reference | <a href="/data/doc/gstreamer/head/gst-plugins-good-plugins/html/">HTML</a> |
+| GStreamer Ugly Plugins Reference | <a href="/data/doc/gstreamer/head/gst-plugins-ugly-plugins/html/">HTML</a> |
+| GStreamer Bad Plugins Reference | <a href="/data/doc/gstreamer/head/gst-plugins-bad-plugins/html/">HTML</a> |
+
+## Other modules
+
+| GStreamer Editing Services Reference | <a href="/data/doc/gstreamer/head/gstreamer-editing-services/html/">HTML</a> |
+| GStreamer RTSP Server Reference | <a href="/data/doc/gstreamer/head/gst-rtsp-server/html/">HTML</a> |
+| GStreamer VAAPI Reference | <a href="/data/doc/gstreamer/head/gstreamer-vaapi-plugins/html/">HTML</a> |
+| QtGStreamer Reference | <a href="/data/doc/gstreamer/head/qt-gstreamer/html/">HTML</a> |
+| GstValidate Reference | <a href="/data/doc/gstreamer/head/gst-validate/html/">HTML</a> |
+| Orc - Optimized inner loop Runtime Compiler | <a href="/data/doc/orc/">HTML</a> |
+| GStreamer Non-Linear Multimedia Editing Plugins Reference (deprecated) | <a href="/data/doc/gstreamer/head/gnonlin/html/">HTML</a> |
+
+## GStreamer Conference Videos and Slides
+ * <a href="http://gstconf.ubicast.tv/channels/#gstreamer-conference-2015">GStreamer Conference 2015: Videos and Slides</a> <a href="/data/events/gstreamer-conference/2015/">(PDF slides)</a>
+ * <a href="http://gstconf.ubicast.tv/channels/#gstreamer-conference-2014">GStreamer Conference 2014: Videos and Slides</a> <a href="/data/events/gstreamer-conference/2014/">(PDF slides)</a>
+ * <a href="http://gstconf.ubicast.tv/channels/#gstreamer-conference-2013">GStreamer Conference 2013: Videos and Slides</a> <a href="/data/events/gstreamer-conference/2013/">(PDF slides)</a>
+ * <a href="http://gstconf.ubicast.tv/channels/#gstreamer-conference-2012">GStreamer Conference 2012: Videos and Slides</a> <a href="/data/events/gstreamer-conference/2012/">(PDF slides)</a>
+ * <a href="http://gstconf.ubicast.tv/channels/#conferences-2011">GStreamer Conference 2011: Videos and Slides</a> <a href="/data/events/gstreamer-conference/2011/">(PDF slides)</a>
+* <a href="http://gstconf.ubicast.tv/channels/#conferences-2010">GStreamer Conference 2010: Videos and Slides</a> <a href="/data/events/gstreamer-conference/2010/">(PDF slides)</a>
+
+
+## GStreamer licensing advisory
+The GStreamer community provides the following licensing advisory for
+developers planing on or already using GStreamer for their applications:
+
+<a href="/documentation/licensing.html">GStreamer Licensing Advisory</a>
+
+
+## GStreamer plugin module split-up
+<a href="/documentation/splitup.html">
+An explanation of the various plugin modules and how they were split up</a>
+
+## RTP and RTSP support in GStreamer
+<a href="/documentation/rtp.html">RTSP and RTP component overview</a>
--- /dev/null
+---
+title: Advanced GStreamer concepts
+...
+
+# Advanced GStreamer concepts
+
+In this part we will cover the more advanced features of GStreamer. With
+the basics you learned in the previous part you should be able to create
+a *simple* application. However, GStreamer provides much more candy than
+just the basics of playing back audio files. In this chapter, you will
+learn more of the low-level features and internals of GStreamer.
+
+Some parts of this part will serve mostly as an explanation of how
+GStreamer works internally; they are not actually needed for actual
+application development. This includes chapters such as the ones
+covering scheduling, autoplugging and synchronization. Other chapters,
+however, discuss more advanced ways of pipeline-application interaction,
+and can turn out to be very useful for certain applications. This
+includes the chapters on metadata, querying and events, interfaces,
+dynamic parameters and pipeline data manipulation.
+
--- /dev/null
+---
+title: Appendices
+...
+
+# Appendices
+
+By now, you've learned all about the internals of GStreamer and
+application programming using the GStreamer framework. This part will go
+into some random bits that are useful to know if you're going to use
+GStreamer for serious application programming. It will touch upon things
+related to integration with popular desktop environments that we run on
+(GNOME, KDE, OS X, Windows), it will shortly explain how applications
+included with GStreamer can help making your life easier, and some
+information on debugging.
+
+In addition, we also provide a porting guide which will explain easily
+how to port GStreamer-0.10 applications to GStreamer-1.0.
+
--- /dev/null
+---
+title: Autoplugging
+...
+
+# Autoplugging
+
+In [Your first application](manual-helloworld.md), you've learned to
+build a simple media player for Ogg/Vorbis files. By using alternative
+elements, you are able to build media players for other media types,
+such as Ogg/Speex, MP3 or even video formats. However, you would rather
+want to build an application that can automatically detect the media
+type of a stream and automatically generate the best possible pipeline
+by looking at all available elements in a system. This process is called
+autoplugging, and GStreamer contains high-quality autopluggers. If
+you're looking for an autoplugger, don't read any further and go to
+[Playback Components](manual-playback-components.md). This chapter will
+explain the *concept* of autoplugging and typefinding. It will explain
+what systems GStreamer includes to dynamically detect the type of a
+media stream, and how to generate a pipeline of decoder elements to
+playback this media. The same principles can also be used for
+transcoding. Because of the full dynamicity of this concept, GStreamer
+can be automatically extended to support new media types without needing
+any adaptations to its autopluggers.
+
+We will first introduce the concept of Media types as a dynamic and
+extendible way of identifying media streams. After that, we will
+introduce the concept of typefinding to find the type of a media stream.
+Lastly, we will explain how autoplugging and the GStreamer registry can
+be used to setup a pipeline that will convert media from one mediatype
+to another, for example for media decoding.
+
+## Media types as a way to identify streams
+
+We have previously introduced the concept of capabilities as a way for
+elements (or, rather, pads) to agree on a media type when streaming data
+from one element to the next (see [Capabilities of a
+pad](manual-pads.md#capabilities-of-a-pad)). We have explained that a
+capability is a combination of a media type and a set of properties. For
+most container formats (those are the files that you will find on your
+hard disk; Ogg, for example, is a container format), no properties are
+needed to describe the stream. Only a media type is needed. A full list
+of media types and accompanying properties can be found in [the Plugin
+Writer's
+Guide](http://gstreamer.freedesktop.org/data/doc/gstreamer/head/pwg/html/section-types-definitions.html).
+
+An element must associate a media type to its source and sink pads when
+it is loaded into the system. GStreamer knows about the different
+elements and what type of data they expect and emit through the
+GStreamer registry. This allows for very dynamic and extensible element
+creation as we will see.
+
+In [Your first application](manual-helloworld.md), we've learned to
+build a music player for Ogg/Vorbis files. Let's look at the media types
+associated with each pad in this pipeline. [The Hello world pipeline
+with media types](#the-hello-world-pipeline-with-media-types) shows what
+media type belongs to each pad in this pipeline.
+
+![The Hello world pipeline with media types](images/mime-world.png
+"fig:")
+
+Now that we have an idea how GStreamer identifies known media streams,
+we can look at methods GStreamer uses to setup pipelines for media
+handling and for media type detection.
+
+## Media stream type detection
+
+Usually, when loading a media stream, the type of the stream is not
+known. This means that before we can choose a pipeline to decode the
+stream, we first need to detect the stream type. GStreamer uses the
+concept of typefinding for this. Typefinding is a normal part of a
+pipeline, it will read data for as long as the type of a stream is
+unknown. During this period, it will provide data to all plugins that
+implement a typefinder. When one of the typefinders recognizes the
+stream, the typefind element will emit a signal and act as a passthrough
+module from that point on. If no type was found, it will emit an error
+and further media processing will stop.
+
+Once the typefind element has found a type, the application can use this
+to plug together a pipeline to decode the media stream. This will be
+discussed in the next section.
+
+Plugins in GStreamer can, as mentioned before, implement typefinder
+functionality. A plugin implementing this functionality will submit a
+media type, optionally a set of file extensions commonly used for this
+media type, and a typefind function. Once this typefind function inside
+the plugin is called, the plugin will see if the data in this media
+stream matches a specific pattern that marks the media type identified
+by that media type. If it does, it will notify the typefind element of
+this fact, telling which mediatype was recognized and how certain we are
+that this stream is indeed that mediatype. Once this run has been
+completed for all plugins implementing a typefind functionality, the
+typefind element will tell the application what kind of media stream it
+thinks to have recognized.
+
+The following code should explain how to use the typefind element. It
+will print the detected media type, or tell that the media type was not
+found. The next section will introduce more useful behaviours, such as
+plugging together a decoding pipeline.
+
+``` c
+#include <gst/gst.h>
+
+[.. my_bus_callback goes here ..]
+
+static gboolean
+idle_exit_loop (gpointer data)
+{
+ g_main_loop_quit ((GMainLoop *) data);
+
+ /* once */
+ return FALSE;
+}
+
+static void
+cb_typefound (GstElement *typefind,
+ guint probability,
+ GstCaps *caps,
+ gpointer data)
+{
+ GMainLoop *loop = data;
+ gchar *type;
+
+ type = gst_caps_to_string (caps);
+ g_print ("Media type %s found, probability %d%%\n", type, probability);
+ g_free (type);
+
+ /* since we connect to a signal in the pipeline thread context, we need
+ * to set an idle handler to exit the main loop in the mainloop context.
+ * Normally, your app should not need to worry about such things. */
+ g_idle_add (idle_exit_loop, loop);
+}
+
+gint
+main (gint argc,
+ gchar *argv[])
+{
+ GMainLoop *loop;
+ GstElement *pipeline, *filesrc, *typefind, *fakesink;
+ GstBus *bus;
+
+ /* init GStreamer */
+ gst_init (&argc, &argv);
+ loop = g_main_loop_new (NULL, FALSE);
+
+ /* check args */
+ if (argc != 2) {
+ g_print ("Usage: %s <filename>\n", argv[0]);
+ return -1;
+ }
+
+ /* create a new pipeline to hold the elements */
+ pipeline = gst_pipeline_new ("pipe");
+
+ bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
+ gst_bus_add_watch (bus, my_bus_callback, NULL);
+ gst_object_unref (bus);
+
+ /* create file source and typefind element */
+ filesrc = gst_element_factory_make ("filesrc", "source");
+ g_object_set (G_OBJECT (filesrc), "location", argv[1], NULL);
+ typefind = gst_element_factory_make ("typefind", "typefinder");
+ g_signal_connect (typefind, "have-type", G_CALLBACK (cb_typefound), loop);
+ fakesink = gst_element_factory_make ("fakesink", "sink");
+
+ /* setup */
+ gst_bin_add_many (GST_BIN (pipeline), filesrc, typefind, fakesink, NULL);
+ gst_element_link_many (filesrc, typefind, fakesink, NULL);
+ gst_element_set_state (GST_ELEMENT (pipeline), GST_STATE_PLAYING);
+ g_main_loop_run (loop);
+
+ /* unset */
+ gst_element_set_state (GST_ELEMENT (pipeline), GST_STATE_NULL);
+ gst_object_unref (GST_OBJECT (pipeline));
+
+ return 0;
+}
+
+```
+
+Once a media type has been detected, you can plug an element (e.g. a
+demuxer or decoder) to the source pad of the typefind element, and
+decoding of the media stream will start right after.
+
+## Dynamically autoplugging a pipeline
+
+See [Playback Components](manual-playback-components.md) for using the
+high level object that you can use to dynamically construct pipelines.
+
--- /dev/null
+---
+title: Bins
+...
+
+# Bins
+
+A bin is a container element. You can add elements to a bin. Since a bin
+is an element itself, a bin can be handled in the same way as any other
+element. Therefore, the whole previous chapter
+([Elements](manual-elements.md)) applies to bins as well.
+
+## What are bins
+
+Bins allow you to combine a group of linked elements into one logical
+element. You do not deal with the individual elements anymore but with
+just one element, the bin. We will see that this is extremely powerful
+when you are going to construct complex pipelines since it allows you to
+break up the pipeline in smaller chunks.
+
+The bin will also manage the elements contained in it. It will perform
+state changes on the elements as well as collect and forward bus
+messages.
+
+
+
+There is one specialized type of bin available to the GStreamer
+programmer:
+
+ - A pipeline: a generic container that manages the synchronization and
+ bus messages of the contained elements. The toplevel bin has to be a
+ pipeline, every application thus needs at least one of these.
+
+## Creating a bin
+
+Bins are created in the same way that other elements are created, i.e.
+using an element factory. There are also convenience functions available
+(`gst_bin_new ()` and `gst_pipeline_new ()`). To add elements to a bin
+or remove elements from a bin, you can use `gst_bin_add ()` and
+`gst_bin_remove ()`. Note that the bin that you add an element to will
+take ownership of that element. If you destroy the bin, the element will
+be dereferenced with it. If you remove an element from a bin, it will be
+dereferenced automatically.
+
+```
+#include <gst/gst.h>
+
+int
+main (int argc,
+ char *argv[])
+{
+ GstElement *bin, *pipeline, *source, *sink;
+
+ /* init */
+ gst_init (&argc, &argv);
+
+ /* create */
+ pipeline = gst_pipeline_new ("my_pipeline");
+ bin = gst_bin_new ("my_bin");
+ source = gst_element_factory_make ("fakesrc", "source");
+ sink = gst_element_factory_make ("fakesink", "sink");
+
+ /* First add the elements to the bin */
+ gst_bin_add_many (GST_BIN (bin), source, sink, NULL);
+ /* add the bin to the pipeline */
+ gst_bin_add (GST_BIN (pipeline), bin);
+
+ /* link the elements */
+ gst_element_link (source, sink);
+
+[..]
+
+}
+
+```
+
+There are various functions to lookup elements in a bin. The most
+commonly used are `gst_bin_get_by_name ()` and `gst_bin_get_by_interface
+()`. You can also iterate over all elements that a bin contains using
+the function `gst_bin_iterate_elements ()`. See the API references of
+[`GstBin`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstBin.html)
+for details.
+
+## Custom bins
+
+The application programmer can create custom bins packed with elements
+to perform a specific task. This allows you, for example, to write an
+Ogg/Vorbis decoder with just the following lines of code:
+
+```
+int
+main (int argc,
+ char *argv[])
+{
+ GstElement *player;
+
+ /* init */
+ gst_init (&argc, &argv);
+
+ /* create player */
+ player = gst_element_factory_make ("oggvorbisplayer", "player");
+
+ /* set the source audio file */
+ g_object_set (player, "location", "helloworld.ogg", NULL);
+
+ /* start playback */
+ gst_element_set_state (GST_ELEMENT (player), GST_STATE_PLAYING);
+[..]
+}
+
+```
+
+(This is a silly example of course, there already exists a much more
+powerful and versatile custom bin like this: the playbin element.)
+
+Custom bins can be created with a plugin or from the application. You
+will find more information about creating custom bin in the [Plugin
+Writers
+Guide](http://gstreamer.freedesktop.org/data/doc/gstreamer/head/pwg/html/index.html).
+
+Examples of such custom bins are the playbin and uridecodebin elements
+from[gst-plugins-base](http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-plugins/html/index.html).
+
+## Bins manage states of their children
+
+Bins manage the state of all elements contained in them. If you set a
+bin (or a pipeline, which is a special top-level type of bin) to a
+certain target state using `gst_element_set_state ()`, it will make sure
+all elements contained within it will also be set to this state. This
+means it's usually only necessary to set the state of the top-level
+pipeline to start up the pipeline or shut it down.
+
+The bin will perform the state changes on all its children from the sink
+element to the source element. This ensures that the downstream element
+is ready to receive data when the upstream element is brought to PAUSED
+or PLAYING. Similarly when shutting down, the sink elements will be set
+to READY or NULL first, which will cause the upstream elements to
+receive a FLUSHING error and stop the streaming threads before the
+elements are set to the READY or NULL state.
+
+Note, however, that if elements are added to a bin or pipeline that's
+already running, , e.g. from within a "pad-added" signal callback, its
+state will not automatically be brought in line with the current state
+or target state of the bin or pipeline it was added to. Instead, you
+have to need to set it to the desired target state yourself using
+`gst_element_set_state ()` or `gst_element_sync_state_with_parent ()`
+when adding elements to an already-running pipeline.
+
--- /dev/null
+---
+title: Buffering
+...
+
+# Buffering
+
+The purpose of buffering is to accumulate enough data in a pipeline so
+that playback can occur smoothly and without interruptions. It is
+typically done when reading from a (slow) and non-live network source
+but can also be used for live sources.
+
+GStreamer provides support for the following use cases:
+
+ - Buffering up to a specific amount of data, in memory, before
+ starting playback so that network fluctuations are minimized. See
+ [Stream buffering](#stream-buffering).
+
+ - Download of the network file to a local disk with fast seeking in
+ the downloaded data. This is similar to the quicktime/youtube
+ players. See [Download buffering](#download-buffering).
+
+ - Caching of (semi)-live streams to a local, on disk, ringbuffer with
+ seeking in the cached area. This is similar to tivo-like
+ timeshifting. See [Timeshift buffering](#timeshift-buffering).
+
+GStreamer can provide the application with progress reports about the
+current buffering state as well as let the application decide on how to
+buffer and when the buffering stops.
+
+In the most simple case, the application has to listen for BUFFERING
+messages on the bus. If the percent indicator inside the BUFFERING
+message is smaller than 100, the pipeline is buffering. When a message
+is received with 100 percent, buffering is complete. In the buffering
+state, the application should keep the pipeline in the PAUSED state.
+When buffering completes, it can put the pipeline (back) in the PLAYING
+state.
+
+What follows is an example of how the message handler could deal with
+the BUFFERING messages. We will see more advanced methods in [Buffering
+strategies](#buffering-strategies).
+
+``` c
+
+ [...]
+
+ switch (GST_MESSAGE_TYPE (message)) {
+ case GST_MESSAGE_BUFFERING:{
+ gint percent;
+
+ /* no state management needed for live pipelines */
+ if (is_live)
+ break;
+
+ gst_message_parse_buffering (message, &percent);
+
+ if (percent == 100) {
+ /* a 100% message means buffering is done */
+ buffering = FALSE;
+ /* if the desired state is playing, go back */
+ if (target_state == GST_STATE_PLAYING) {
+ gst_element_set_state (pipeline, GST_STATE_PLAYING);
+ }
+ } else {
+ /* buffering busy */
+ if (!buffering && target_state == GST_STATE_PLAYING) {
+ /* we were not buffering but PLAYING, PAUSE the pipeline. */
+ gst_element_set_state (pipeline, GST_STATE_PAUSED);
+ }
+ buffering = TRUE;
+ }
+ break;
+ case ...
+
+ [...]
+
+
+```
+
+## Stream buffering
+
+```
+ +---------+ +---------+ +-------+
+ | httpsrc | | buffer | | demux |
+ | src - sink src - sink ....
+ +---------+ +---------+ +-------+
+
+```
+
+In this case we are reading from a slow network source into a buffer
+element (such as queue2).
+
+The buffer element has a low and high watermark expressed in bytes. The
+buffer uses the watermarks as follows:
+
+ - The buffer element will post BUFFERING messages until the high
+ watermark is hit. This instructs the application to keep the
+ pipeline PAUSED, which will eventually block the srcpad from pushing
+ while data is prerolled in the sinks.
+
+ - When the high watermark is hit, a BUFFERING message with 100% will
+ be posted, which instructs the application to continue playback.
+
+ - When during playback, the low watermark is hit, the queue will start
+ posting BUFFERING messages again, making the application PAUSE the
+ pipeline again until the high watermark is hit again. This is called
+ the rebuffering stage.
+
+ - During playback, the queue level will fluctuate between the high and
+ the low watermark as a way to compensate for network irregularities.
+
+This buffering method is usable when the demuxer operates in push mode.
+Seeking in the stream requires the seek to happen in the network source.
+It is mostly desirable when the total duration of the file is not known,
+such as in live streaming or when efficient seeking is not
+possible/required.
+
+The problem is configuring a good low and high watermark. Here are some
+ideas:
+
+ - It is possible to measure the network bandwidth and configure the
+ low/high watermarks in such a way that buffering takes a fixed
+ amount of time.
+
+ The queue2 element in GStreamer core has the max-size-time property
+ that, together with the use-rate-estimate property, does exactly
+ that. Also the playbin buffer-duration property uses the rate
+ estimate to scale the amount of data that is buffered.
+
+ - Based on the codec bitrate, it is also possible to set the
+ watermarks in such a way that a fixed amount of data is buffered
+ before playback starts. Normally, the buffering element doesn't know
+ about the bitrate of the stream but it can get this with a query.
+
+ - Start with a fixed amount of bytes, measure the time between
+ rebuffering and increase the queue size until the time between
+ rebuffering is within the application's chosen limits.
+
+The buffering element can be inserted anywhere in the pipeline. You
+could, for example, insert the buffering element before a decoder. This
+would make it possible to set the low/high watermarks based on time.
+
+The buffering flag on playbin, performs buffering on the parsed data.
+Another advantage of doing the buffering at a later stage is that you
+can let the demuxer operate in pull mode. When reading data from a slow
+network drive (with filesrc) this can be an interesting way to buffer.
+
+## Download buffering
+
+```
+ +---------+ +---------+ +-------+
+ | httpsrc | | buffer | | demux |
+ | src - sink src - sink ....
+ +---------+ +----|----+ +-------+
+ V
+ file
+
+```
+
+If we know the server is streaming a fixed length file to the client,
+the application can choose to download the entire file on disk. The
+buffer element will provide a push or pull based srcpad to the demuxer
+to navigate in the downloaded file.
+
+This mode is only suitable when the client can determine the length of
+the file on the server.
+
+In this case, buffering messages will be emitted as usual when the
+requested range is not within the downloaded area + buffersize. The
+buffering message will also contain an indication that incremental
+download is being performed. This flag can be used to let the
+application control the buffering in a more intelligent way, using the
+BUFFERING query, for example. See [Buffering
+strategies](#buffering-strategies).
+
+## Timeshift buffering
+
+```
+ +---------+ +---------+ +-------+
+ | httpsrc | | buffer | | demux |
+ | src - sink src - sink ....
+ +---------+ +----|----+ +-------+
+ V
+ file-ringbuffer
+
+```
+
+In this mode, a fixed size ringbuffer is kept to download the server
+content. This allows for seeking in the buffered data. Depending on the
+size of the ringbuffer one can seek further back in time.
+
+This mode is suitable for all live streams. As with the incremental
+download mode, buffering messages are emitted along with an indication
+that timeshifting download is in progress.
+
+## Live buffering
+
+In live pipelines we usually introduce some fixed latency between the
+capture and the playback elements. This latency can be introduced by a
+queue (such as a jitterbuffer) or by other means (in the audiosink).
+
+Buffering messages can be emitted in those live pipelines as well and
+serve as an indication to the user of the latency buffering. The
+application usually does not react to these buffering messages with a
+state change.
+
+## Buffering strategies
+
+What follows are some ideas for implementing different buffering
+strategies based on the buffering messages and buffering query.
+
+### No-rebuffer strategy
+
+We would like to buffer enough data in the pipeline so that playback
+continues without interruptions. What we need to know to implement this
+is know the total remaining playback time in the file and the total
+remaining download time. If the buffering time is less than the playback
+time, we can start playback without interruptions.
+
+We have all this information available with the DURATION, POSITION and
+BUFFERING queries. We need to periodically execute the buffering query
+to get the current buffering status. We also need to have a large enough
+buffer to hold the complete file, worst case. It is best to use this
+buffering strategy with download buffering (see [Download
+buffering](#download-buffering)).
+
+This is what the code would look like:
+
+``` c
+
+
+#include <gst/gst.h>
+
+GstState target_state;
+static gboolean is_live;
+static gboolean is_buffering;
+
+static gboolean
+buffer_timeout (gpointer data)
+{
+ GstElement *pipeline = data;
+ GstQuery *query;
+ gboolean busy;
+ gint percent;
+ gint64 estimated_total;
+ gint64 position, duration;
+ guint64 play_left;
+
+ query = gst_query_new_buffering (GST_FORMAT_TIME);
+
+ if (!gst_element_query (pipeline, query))
+ return TRUE;
+
+ gst_query_parse_buffering_percent (query, &busy, &percent);
+ gst_query_parse_buffering_range (query, NULL, NULL, NULL, &estimated_total);
+
+ if (estimated_total == -1)
+ estimated_total = 0;
+
+ /* calculate the remaining playback time */
+ if (!gst_element_query_position (pipeline, GST_FORMAT_TIME, &position))
+ position = -1;
+ if (!gst_element_query_duration (pipeline, GST_FORMAT_TIME, &duration))
+ duration = -1;
+
+ if (duration != -1 && position != -1)
+ play_left = GST_TIME_AS_MSECONDS (duration - position);
+ else
+ play_left = 0;
+
+ g_message ("play_left %" G_GUINT64_FORMAT", estimated_total %" G_GUINT64_FORMAT
+ ", percent %d", play_left, estimated_total, percent);
+
+ /* we are buffering or the estimated download time is bigger than the
+ * remaining playback time. We keep buffering. */
+ is_buffering = (busy || estimated_total * 1.1 > play_left);
+
+ if (!is_buffering)
+ gst_element_set_state (pipeline, target_state);
+
+ return is_buffering;
+}
+
+static void
+on_message_buffering (GstBus *bus, GstMessage *message, gpointer user_data)
+{
+ GstElement *pipeline = user_data;
+ gint percent;
+
+ /* no state management needed for live pipelines */
+ if (is_live)
+ return;
+
+ gst_message_parse_buffering (message, &percent);
+
+ if (percent < 100) {
+ /* buffering busy */
+ if (!is_buffering) {
+ is_buffering = TRUE;
+ if (target_state == GST_STATE_PLAYING) {
+ /* we were not buffering but PLAYING, PAUSE the pipeline. */
+ gst_element_set_state (pipeline, GST_STATE_PAUSED);
+ }
+ }
+ }
+}
+
+static void
+on_message_async_done (GstBus *bus, GstMessage *message, gpointer user_data)
+{
+ GstElement *pipeline = user_data;
+
+ if (!is_buffering)
+ gst_element_set_state (pipeline, target_state);
+ else
+ g_timeout_add (500, buffer_timeout, pipeline);
+}
+
+gint
+main (gint argc,
+ gchar *argv[])
+{
+ GstElement *pipeline;
+ GMainLoop *loop;
+ GstBus *bus;
+ GstStateChangeReturn ret;
+
+ /* init GStreamer */
+ gst_init (&argc, &argv);
+ loop = g_main_loop_new (NULL, FALSE);
+
+ /* make sure we have a URI */
+ if (argc != 2) {
+ g_print ("Usage: %s <URI>\n", argv[0]);
+ return -1;
+ }
+
+ /* set up */
+ pipeline = gst_element_factory_make ("playbin", "pipeline");
+ g_object_set (G_OBJECT (pipeline), "uri", argv[1], NULL);
+ g_object_set (G_OBJECT (pipeline), "flags", 0x697 , NULL);
+
+ bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
+ gst_bus_add_signal_watch (bus);
+
+ g_signal_connect (bus, "message::buffering",
+ (GCallback) on_message_buffering, pipeline);
+ g_signal_connect (bus, "message::async-done",
+ (GCallback) on_message_async_done, pipeline);
+ gst_object_unref (bus);
+
+ is_buffering = FALSE;
+ target_state = GST_STATE_PLAYING;
+ ret = gst_element_set_state (pipeline, GST_STATE_PAUSED);
+
+ switch (ret) {
+ case GST_STATE_CHANGE_SUCCESS:
+ is_live = FALSE;
+ break;
+
+ case GST_STATE_CHANGE_FAILURE:
+ g_warning ("failed to PAUSE");
+ return -1;
+
+ case GST_STATE_CHANGE_NO_PREROLL:
+ is_live = TRUE;
+ break;
+
+ default:
+ break;
+ }
+
+ /* now run */
+ g_main_loop_run (loop);
+
+ /* also clean up */
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (GST_OBJECT (pipeline));
+ g_main_loop_unref (loop);
+
+ return 0;
+}
+
+
+
+```
+
+See how we set the pipeline to the PAUSED state first. We will receive
+buffering messages during the preroll state when buffering is needed.
+When we are prerolled (on\_message\_async\_done) we see if buffering is
+going on, if not, we start playback. If buffering was going on, we start
+a timeout to poll the buffering state. If the estimated time to download
+is less than the remaining playback time, we start playback.
+
--- /dev/null
+---
+title: Building an Application
+...
+
+# Building an Application
+
+In these chapters, we will discuss the basic concepts of GStreamer and
+the most-used objects, such as elements, pads and buffers. We will use a
+visual representation of these objects so that we can visualize the more
+complex pipelines you will learn to build later on. You will get a first
+glance at the GStreamer API, which should be enough for building
+elementary applications. Later on in this part, you will also learn to
+build a basic command-line application.
+
+Note that this part will give a look into the low-level API and concepts
+of GStreamer. Once you're going to build applications, you might want to
+use higher-level APIs. Those will be discussed later on in this manual.
+
--- /dev/null
+---
+title: Bus
+...
+
+# Bus
+
+A bus is a simple system that takes care of forwarding messages from the
+streaming threads to an application in its own thread context. The
+advantage of a bus is that an application does not need to be
+thread-aware in order to use GStreamer, even though GStreamer itself is
+heavily threaded.
+
+Every pipeline contains a bus by default, so applications do not need to
+create a bus or anything. The only thing applications should do is set a
+message handler on a bus, which is similar to a signal handler to an
+object. When the mainloop is running, the bus will periodically be
+checked for new messages, and the callback will be called when any
+message is available.
+
+## How to use a bus
+
+There are two different ways to use a bus:
+
+ - Run a GLib/Gtk+ main loop (or iterate the default GLib main context
+ yourself regularly) and attach some kind of watch to the bus. This
+ way the GLib main loop will check the bus for new messages and
+ notify you whenever there are messages.
+
+ Typically you would use `gst_bus_add_watch ()` or
+ `gst_bus_add_signal_watch ()` in this case.
+
+ To use a bus, attach a message handler to the bus of a pipeline
+ using `gst_bus_add_watch ()`. This handler will be called whenever
+ the pipeline emits a message to the bus. In this handler, check the
+ signal type (see next section) and do something accordingly. The
+ return value of the handler should be TRUE to keep the handler
+ attached to the bus, return FALSE to remove it.
+
+ - Check for messages on the bus yourself. This can be done using
+ `gst_bus_peek ()` and/or `gst_bus_poll ()`.
+
+
+{{ examples/bus_example.c }}
+
+It is important to know that the handler will be called in the thread
+context of the mainloop. This means that the interaction between the
+pipeline and application over the bus is *asynchronous*, and thus not
+suited for some real-time purposes, such as cross-fading between audio
+tracks, doing (theoretically) gapless playback or video effects. All
+such things should be done in the pipeline context, which is easiest by
+writing a GStreamer plug-in. It is very useful for its primary purpose,
+though: passing messages from pipeline to application. The advantage of
+this approach is that all the threading that GStreamer does internally
+is hidden from the application and the application developer does not
+have to worry about thread issues at all.
+
+Note that if you're using the default GLib mainloop integration, you
+can, instead of attaching a watch, connect to the “message” signal on
+the bus. This way you don't have to `switch()` on all possible message
+types; just connect to the interesting signals in form of
+“message::\<type\>”, where \<type\> is a specific message type (see
+the next section for an explanation of message types).
+
+The above snippet could then also be written as:
+
+``` c
+GstBus *bus;
+
+[..]
+
+bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline);
+gst_bus_add_signal_watch (bus);
+g_signal_connect (bus, "message::error", G_CALLBACK (cb_message_error), NULL);
+g_signal_connect (bus, "message::eos", G_CALLBACK (cb_message_eos), NULL);
+
+[..]
+
+```
+
+If you aren't using GLib mainloop, the asynchronous message signals
+won't be available by default. You can however install a custom sync
+handler that wakes up the custom mainloop and that uses
+`gst_bus_async_signal_func ()` to emit the signals. (see also
+[documentation](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstBus.html)
+for details)
+
+## Message types
+
+GStreamer has a few pre-defined message types that can be passed over
+the bus. The messages are extensible, however. Plug-ins can define
+additional messages, and applications can decide to either have specific
+code for those or ignore them. All applications are strongly recommended
+to at least handle error messages by providing visual feedback to the
+user.
+
+All messages have a message source, type and timestamp. The message
+source can be used to see which element emitted the message. For some
+messages, for example, only the ones emitted by the top-level pipeline
+will be interesting to most applications (e.g. for state-change
+notifications). Below is a list of all messages and a short explanation
+of what they do and how to parse message-specific content.
+
+ - Error, warning and information notifications: those are used by
+ elements if a message should be shown to the user about the state of
+ the pipeline. Error messages are fatal and terminate the
+ data-passing. The error should be repaired to resume pipeline
+ activity. Warnings are not fatal, but imply a problem nevertheless.
+ Information messages are for non-problem notifications. All those
+ messages contain a `GError` with the main error type and message,
+ and optionally a debug string. Both can be extracted using
+ `gst_message_parse_error
+ ()`, `_parse_warning ()` and `_parse_info ()`. Both error and debug
+ strings should be freed after use.
+
+ - End-of-stream notification: this is emitted when the stream has
+ ended. The state of the pipeline will not change, but further media
+ handling will stall. Applications can use this to skip to the next
+ song in their playlist. After end-of-stream, it is also possible to
+ seek back in the stream. Playback will then continue automatically.
+ This message has no specific arguments.
+
+ - Tags: emitted when metadata was found in the stream. This can be
+ emitted multiple times for a pipeline (e.g. once for descriptive
+ metadata such as artist name or song title, and another one for
+ stream-information, such as samplerate and bitrate). Applications
+ should cache metadata internally. `gst_message_parse_tag
+ ()` should be used to parse the taglist, which should be
+ `gst_tag_list_unref ()`'ed when no longer needed.
+
+ - State-changes: emitted after a successful state change.
+ `gst_message_parse_state_changed ()` can be used to parse the old
+ and new state of this transition.
+
+ - Buffering: emitted during caching of network-streams. One can
+ manually extract the progress (in percent) from the message by
+ extracting the “buffer-percent” property from the structure returned
+ by `gst_message_get_structure
+ ()`. See also [Buffering](manual-buffering.md).
+
+ - Element messages: these are special messages that are unique to
+ certain elements and usually represent additional features. The
+ element's documentation should mention in detail which element
+ messages a particular element may send. As an example, the 'qtdemux'
+ QuickTime demuxer element may send a 'redirect' element message on
+ certain occasions if the stream contains a redirect instruction.
+
+ - Application-specific messages: any information on those can be
+ extracted by getting the message structure (see above) and reading
+ its fields. Usually these messages can safely be ignored.
+
+ Application messages are primarily meant for internal use in
+ applications in case the application needs to marshal information
+ from some thread into the main thread. This is particularly useful
+ when the application is making use of element signals (as those
+ signals will be emitted in the context of the streaming thread).
+
--- /dev/null
+---
+title: Things to check when writing an application
+...
+
+# Things to check when writing an application
+
+This chapter contains a fairly random selection of things that can be
+useful to keep in mind when writing GStreamer-based applications. It's
+up to you how much you're going to use the information provided here. We
+will shortly discuss how to debug pipeline problems using GStreamer
+applications. Also, we will touch upon how to acquire knowledge about
+plugins and elements and how to test simple pipelines before building
+applications around them.
+
+## Good programming habits
+
+ - Always add a `GstBus` handler to your pipeline. Always report errors
+ in your application, and try to do something with warnings and
+ information messages, too.
+
+ - Always check return values of GStreamer functions. Especially, check
+ return values of `gst_element_link ()` and `gst_element_set_state
+ ()`.
+
+ - Dereference return values of all functions returning a non-base
+ type, such as `gst_element_get_pad ()`. Also, always free non-const
+ string returns, such as `gst_object_get_name ()`.
+
+ - Always use your pipeline object to keep track of the current state
+ of your pipeline. Don't keep private variables in your application.
+ Also, don't update your user interface if a user presses the “play”
+ button. Instead, listen for the “state-changed” message on the
+ `GstBus` and only update the user interface whenever this message is
+ received.
+
+ - Report all bugs that you find in GStreamer bugzilla at
+ [http://bugzilla.gnome.org/](http://bugzilla.gnome.org).
+
+## Debugging
+
+Applications can make use of the extensive GStreamer debugging system to
+debug pipeline problems. Elements will write output to this system to
+log what they're doing. It's not used for error reporting, but it is
+very useful for tracking what an element is doing exactly, which can
+come in handy when debugging application issues (such as failing seeks,
+out-of-sync media, etc.).
+
+Most GStreamer-based applications accept the commandline option
+`--gst-debug=LIST` and related family members. The list consists of a
+comma-separated list of category/level pairs, which can set the
+debugging level for a specific debugging category. For example,
+`--gst-debug=oggdemux:5` would turn on debugging for the Ogg demuxer
+element. You can use wildcards as well. A debugging level of 0 will turn
+off all debugging, and a level of 9 will turn on all debugging.
+Intermediate values only turn on some debugging (based on message
+severity; 2, for example, will only display errors and warnings). Here's
+a list of all available options:
+
+ - `--gst-debug-help` will print available debug categories and exit.
+
+ - `--gst-debug-level=LEVEL` will set the default debug level (which
+ can range from 0 (no output) to 9 (everything)).
+
+ - `--gst-debug=LIST` takes a comma-separated list of
+ category\_name:level pairs to set specific levels for the individual
+ categories. Example: `GST_AUTOPLUG:5,avidemux:3`. Alternatively, you
+ can also set the `GST_DEBUG` environment variable, which has the
+ same effect.
+
+ - `--gst-debug-no-color` will disable color debugging. You can also
+ set the GST\_DEBUG\_NO\_COLOR environment variable to 1 if you want
+ to disable colored debug output permanently. Note that if you are
+ disabling color purely to avoid messing up your pager output, try
+ using `less -R`.
+
+ - `--gst-debug-color-mode=MODE` will change debug log coloring mode.
+ MODE can be one of the following: `on`, `off`, `auto`, `disable`,
+ `unix`. You can also set the GST\_DEBUG\_COLOR\_MODE environment
+ variable if you want to change colored debug output permanently.
+ Note that if you are disabling color purely to avoid messing up your
+ pager output, try using `less -R`.
+
+ - `--gst-debug-disable` disables debugging altogether.
+
+ - `--gst-plugin-spew` enables printout of errors while loading
+ GStreamer plugins.
+
+## Conversion plugins
+
+GStreamer contains a bunch of conversion plugins that most applications
+will find useful. Specifically, those are videoscalers (videoscale),
+colorspace convertors (videoconvert), audio format convertors and
+channel resamplers (audioconvert) and audio samplerate convertors
+(audioresample). Those convertors don't do anything when not required,
+they will act in passthrough mode. They will activate when the hardware
+doesn't support a specific request, though. All applications are
+recommended to use those elements.
+
+## Utility applications provided with GStreamer
+
+GStreamer comes with a default set of command-line utilities that can
+help in application development. We will discuss only `gst-launch` and
+`gst-inspect` here.
+
+### `gst-launch`
+
+`gst-launch` is a simple script-like commandline application that can be
+used to test pipelines. For example, the command `gst-launch
+audiotestsrc ! audioconvert !
+audio/x-raw,channels=2 ! alsasink` will run a pipeline which generates a
+sine-wave audio stream and plays it to your ALSA audio card.
+`gst-launch` also allows the use of threads (will be used automatically
+as required or as queue elements are inserted in the pipeline) and bins
+(using brackets, so “(” and “)”). You can use dots to imply padnames on
+elements, or even omit the padname to automatically select a pad. Using
+all this, the pipeline `gst-launch filesrc location=file.ogg ! oggdemux
+name=d
+d. ! queue ! theoradec ! videoconvert ! xvimagesink
+d. ! queue ! vorbisdec ! audioconvert ! audioresample ! alsasink
+` will play an Ogg file containing a Theora video-stream and a Vorbis
+audio-stream. You can also use autopluggers such as decodebin on the
+commandline. See the manual page of `gst-launch` for more information.
+
+### `gst-inspect`
+
+`gst-inspect` can be used to inspect all properties, signals, dynamic
+parameters and the object hierarchy of an element. This can be very
+useful to see which `GObject` properties or which signals (and using
+what arguments) an element supports. Run `gst-inspect fakesrc` to get an
+idea of what it does. See the manual page of `gst-inspect` for more
+information.
+
--- /dev/null
+---
+title: Clocks and synchronization in GStreamer
+...
+
+# Clocks and synchronization in GStreamer
+
+When playing complex media, each sound and video sample must be played
+in a specific order at a specific time. For this purpose, GStreamer
+provides a synchronization mechanism.
+
+GStreamer provides support for the following use cases:
+
+ - Non-live sources with access faster than playback rate. This is the
+ case where one is reading media from a file and playing it back in a
+ synchronized fashion. In this case, multiple streams need to be
+ synchronized, like audio, video and subtitles.
+
+ - Capture and synchronized muxing/mixing of media from multiple live
+ sources. This is a typical use case where you record audio and video
+ from a microphone/camera and mux it into a file for storage.
+
+ - Streaming from (slow) network streams with buffering. This is the
+ typical web streaming case where you access content from a streaming
+ server with http.
+
+ - Capture from live source and and playback to live source with
+ configurable latency. This is used when, for example, capture from a
+ camera, apply an effect and display the result. It is also used when
+ streaming low latency content over a network with UDP.
+
+ - Simultaneous live capture and playback from prerecorded content.
+ This is used in audio recording cases where you play a previously
+ recorded audio and record new samples, the purpose is to have the
+ new audio perfectly in sync with the previously recorded data.
+
+GStreamer uses a `GstClock` object, buffer timestamps and a SEGMENT
+event to synchronize streams in a pipeline as we will see in the next
+sections.
+
+## Clock running-time
+
+In a typical computer, there are many sources that can be used as a time
+source, e.g., the system time, soundcards, CPU performance counters, ...
+For this reason, there are many `GstClock` implementations available in
+GStreamer. The clock time doesn't always start from 0 or from some known
+value. Some clocks start counting from some known start date, other
+clocks start counting since last reboot, etc...
+
+A `GstClock` returns the **absolute-time** according to that clock with
+`gst_clock_get_time ()`. The absolute-time (or clock time) of a clock is
+monotonically increasing. From the absolute-time is a **running-time**
+calculated, which is simply the difference between a previous snapshot
+of the absolute-time called the **base-time**. So:
+
+running-time = absolute-time - base-time
+
+A GStreamer `GstPipeline` object maintains a `GstClock` object and a
+base-time when it goes to the PLAYING state. The pipeline gives a handle
+to the selected `GstClock` to each element in the pipeline along with
+selected base-time. The pipeline will select a base-time in such a way
+that the running-time reflects the total time spent in the PLAYING
+state. As a result, when the pipeline is PAUSED, the running-time stands
+still.
+
+Because all objects in the pipeline have the same clock and base-time,
+they can thus all calculate the running-time according to the pipeline
+clock.
+
+## Buffer running-time
+
+To calculate a buffer running-time, we need a buffer timestamp and the
+SEGMENT event that preceeded the buffer. First we can convert the
+SEGMENT event into a `GstSegment` object and then we can use the
+`gst_segment_to_running_time ()` function to perform the calculation of
+the buffer running-time.
+
+Synchronization is now a matter of making sure that a buffer with a
+certain running-time is played when the clock reaches the same
+running-time. Usually this task is done by sink elements. Sink also have
+to take into account the latency configured in the pipeline and add this
+to the buffer running-time before synchronizing to the pipeline clock.
+
+Non-live sources timestamp buffers with a running-time starting from 0.
+After a flushing seek, they will produce buffers again from a
+running-time of 0.
+
+Live sources need to timestamp buffers with a running-time matching the
+pipeline running-time when the first byte of the buffer was captured.
+
+## Buffer stream-time
+
+The buffer stream-time, also known as the position in the stream, is
+calculated from the buffer timestamps and the preceding SEGMENT event.
+It represents the time inside the media as a value between 0 and the
+total duration of the media.
+
+The stream-time is used in:
+
+ - Report the current position in the stream with the POSITION query.
+
+ - The position used in the seek events and queries.
+
+ - The position used to synchronize controlled values.
+
+The stream-time is never used to synchronize streams, this is only done
+with the running-time.
+
+## Time overview
+
+Here is an overview of the various timelines used in GStreamer.
+
+The image below represents the different times in the pipeline when
+playing a 100ms sample and repeating the part between 50ms and 100ms.
+
+
+
+You can see how the running-time of a buffer always increments
+monotonically along with the clock-time. Buffers are played when their
+running-time is equal to the clock-time - base-time. The stream-time
+represents the position in the stream and jumps backwards when
+repeating.
+
+## Clock providers
+
+A clock provider is an element in the pipeline that can provide a
+`GstClock` object. The clock object needs to report an absolute-time
+that is monotonically increasing when the element is in the PLAYING
+state. It is allowed to pause the clock while the element is PAUSED.
+
+Clock providers exist because they play back media at some rate, and
+this rate is not necessarily the same as the system clock rate. For
+example, a soundcard may playback at 44,1 kHz, but that doesn't mean
+that after *exactly* 1 second *according to the system clock*, the
+soundcard has played back 44.100 samples. This is only true by
+approximation. In fact, the audio device has an internal clock based on
+the number of samples played that we can expose.
+
+If an element with an internal clock needs to synchronize, it needs to
+estimate when a time according to the pipeline clock will take place
+according to the internal clock. To estimate this, it needs to slave its
+clock to the pipeline clock.
+
+If the pipeline clock is exactly the internal clock of an element, the
+element can skip the slaving step and directly use the pipeline clock to
+schedule playback. This can be both faster and more accurate. Therefore,
+generally, elements with an internal clock like audio input or output
+devices will be a clock provider for the pipeline.
+
+When the pipeline goes to the PLAYING state, it will go over all
+elements in the pipeline from sink to source and ask each element if
+they can provide a clock. The last element that can provide a clock will
+be used as the clock provider in the pipeline. This algorithm prefers a
+clock from an audio sink in a typical playback pipeline and a clock from
+source elements in a typical capture pipeline.
+
+There exist some bus messages to let you know about the clock and clock
+providers in the pipeline. You can see what clock is selected in the
+pipeline by looking at the NEW\_CLOCK message on the bus. When a clock
+provider is removed from the pipeline, a CLOCK\_LOST message is posted
+and the application should go to PAUSED and back to PLAYING to select a
+new clock.
+
+## Latency
+
+The latency is the time it takes for a sample captured at timestamp X to
+reach the sink. This time is measured against the clock in the pipeline.
+For pipelines where the only elements that synchronize against the clock
+are the sinks, the latency is always 0 since no other element is
+delaying the buffer.
+
+For pipelines with live sources, a latency is introduced, mostly because
+of the way a live source works. Consider an audio source, it will start
+capturing the first sample at time 0. If the source pushes buffers with
+44100 samples at a time at 44100Hz it will have collected the buffer at
+second 1. Since the timestamp of the buffer is 0 and the time of the
+clock is now \>= 1 second, the sink will drop this buffer because it is
+too late. Without any latency compensation in the sink, all buffers will
+be dropped.
+
+### Latency compensation
+
+Before the pipeline goes to the PLAYING state, it will, in addition to
+selecting a clock and calculating a base-time, calculate the latency in
+the pipeline. It does this by doing a LATENCY query on all the sinks in
+the pipeline. The pipeline then selects the maximum latency in the
+pipeline and configures this with a LATENCY event.
+
+All sink elements will delay playback by the value in the LATENCY event.
+Since all sinks delay with the same amount of time, they will be
+relative in sync.
+
+### Dynamic Latency
+
+Adding/removing elements to/from a pipeline or changing element
+properties can change the latency in a pipeline. An element can request
+a latency change in the pipeline by posting a LATENCY message on the
+bus. The application can then decide to query and redistribute a new
+latency or not. Changing the latency in a pipeline might cause visual or
+audible glitches and should therefore only be done by the application
+when it is allowed.
+
--- /dev/null
+---
+title: Compiling
+...
+
+# Compiling
+
+This section talks about the different things you can do when building
+and shipping your applications and plugins.
+
+## Embedding static elements in your application
+
+The [Plugin Writer's
+Guide](http://gstreamer.freedesktop.org/data/doc/gstreamer/head/pwg/html/index.html)
+describes in great detail how to write elements for the GStreamer
+framework. In this section, we will solely discuss how to embed such
+elements statically in your application. This can be useful for
+application-specific elements that have no use elsewhere in GStreamer.
+
+Dynamically loaded plugins contain a structure that's defined using
+`GST_PLUGIN_DEFINE ()`. This structure is loaded when the plugin is
+loaded by the GStreamer core. The structure contains an initialization
+function (usually called `plugin_init`) that will be called right after
+that. It's purpose is to register the elements provided by the plugin
+with the GStreamer framework. If you want to embed elements directly in
+your application, the only thing you need to do is to replace
+`GST_PLUGIN_DEFINE ()` with a call to `gst_plugin_register_static ()`.
+As soon as you call `gst_plugin_register_static ()`, the elements will
+from then on be available like any other element, without them having to
+be dynamically loadable libraries. In the example below, you would be
+able to call `gst_element_factory_make
+("my-element-name", "some-name")` to create an instance of the element.
+
+``` c
+
+/*
+ * Here, you would write the actual plugin code.
+ */
+
+[..]
+
+static gboolean
+register_elements (GstPlugin *plugin)
+{
+ return gst_element_register (plugin, "my-element-name",
+ GST_RANK_NONE, MY_PLUGIN_TYPE);
+}
+
+static
+my_code_init (void)
+{
+ ...
+
+ gst_plugin_register_static (
+ GST_VERSION_MAJOR,
+ GST_VERSION_MINOR,
+ "my-private-plugins",
+ "Private elements of my application",
+ register_elements,
+ VERSION,
+ "LGPL",
+ "my-application-source",
+ "my-application",
+ "http://www.my-application.net/")
+
+ ...
+}
+
+
+```
+
--- /dev/null
+---
+title: Buffers and Events
+...
+
+# Buffers and Events
+
+The data flowing through a pipeline consists of a combination of buffers
+and events. Buffers contain the actual media data. Events contain
+control information, such as seeking information and end-of-stream
+notifiers. All this will flow through the pipeline automatically when
+it's running. This chapter is mostly meant to explain the concept to
+you; you don't need to do anything for this.
+
+## Buffers
+
+Buffers contain the data that will flow through the pipeline you have
+created. A source element will typically create a new buffer and pass it
+through a pad to the next element in the chain. When using the GStreamer
+infrastructure to create a media pipeline you will not have to deal with
+buffers yourself; the elements will do that for you.
+
+A buffer consists, amongst others, of:
+
+ - Pointers to memory objects. Memory objects encapsulate a region in
+ the memory.
+
+ - A timestamp for the buffer.
+
+ - A refcount that indicates how many elements are using this buffer.
+ This refcount will be used to destroy the buffer when no element has
+ a reference to it.
+
+ - Buffer flags.
+
+The simple case is that a buffer is created, memory allocated, data put
+in it, and passed to the next element. That element reads the data, does
+something (like creating a new buffer and decoding into it), and
+unreferences the buffer. This causes the data to be free'ed and the
+buffer to be destroyed. A typical video or audio decoder works like
+this.
+
+There are more complex scenarios, though. Elements can modify buffers
+in-place, i.e. without allocating a new one. Elements can also write to
+hardware memory (such as from video-capture sources) or memory allocated
+from the X-server (using XShm). Buffers can be read-only, and so on.
+
+## Events
+
+Events are control particles that are sent both up- and downstream in a
+pipeline along with buffers. Downstream events notify fellow elements of
+stream states. Possible events include seeking, flushes, end-of-stream
+notifications and so on. Upstream events are used both in
+application-element interaction as well as element-element interaction
+to request changes in stream state, such as seeks. For applications,
+only upstream events are important. Downstream events are just explained
+to get a more complete picture of the data concept.
+
+Since most applications seek in time units, our example below does so
+too:
+
+``` c
+static void
+seek_to_time (GstElement *element,
+ guint64 time_ns)
+{
+ GstEvent *event;
+
+ event = gst_event_new_seek (1.0, GST_FORMAT_TIME,
+ GST_SEEK_FLAG_NONE,
+ GST_SEEK_METHOD_SET, time_ns,
+ GST_SEEK_TYPE_NONE, G_GUINT64_CONSTANT (0));
+ gst_element_send_event (element, event);
+}
+
+```
+
+The function `gst_element_seek ()` is a shortcut for this. This is
+mostly just to show how it all works.
+
--- /dev/null
+---
+title: Pipeline manipulation
+...
+
+# Pipeline manipulation
+
+This chapter will discuss how you can manipulate your pipeline in
+several ways from your application on. Parts of this chapter are very
+lowlevel, so be assured that you'll need some programming knowledge and
+a good understanding of GStreamer before you start reading this.
+
+Topics that will be discussed here include how you can insert data into
+a pipeline from your application, how to read data from a pipeline, how
+to manipulate the pipeline's speed, length, starting point and how to
+listen to a pipeline's data processing.
+
+## Using probes
+
+Probing is best envisioned as a pad listener. Technically, a probe is
+nothing more than a callback that can be attached to a pad. You can
+attach a probe using `gst_pad_add_probe ()`. Similarly, one can use the
+`gst_pad_remove_probe ()` to remove the callback again. The probe
+notifies you of any activity that happens on the pad, like buffers,
+events and queries. You can define what kind of notifications you are
+interested in when you add the probe.
+
+The probe can notify you of the following activity on pads:
+
+ - A buffer is pushed or pulled. You want to specify the
+ GST\_PAD\_PROBE\_TYPE\_BUFFER when registering the probe. Because
+ the pad can be scheduled in different ways, it is possible to also
+ specify in what scheduling mode you are interested with the optional
+ GST\_PAD\_PROBE\_TYPE\_PUSH and GST\_PAD\_PROBE\_TYPE\_PULL flags.
+
+ You can use this probe to inspect, modify or drop the buffer. See
+ [Data probes](#data-probes).
+
+ - A bufferlist is pushed. Use the GST\_PAD\_PROBE\_TYPE\_BUFFER\_LIST
+ when registering the probe.
+
+ - An event travels over a pad. Use the
+ GST\_PAD\_PROBE\_TYPE\_EVENT\_DOWNSTREAM and
+ GST\_PAD\_PROBE\_TYPE\_EVENT\_UPSTREAM flags to select downstream
+ and upstream events. There is also a convenience
+ GST\_PAD\_PROBE\_TYPE\_EVENT\_BOTH to be notified of events going
+ both upstream and downstream. By default, flush events do not cause
+ a notification. You need to explicitly enable
+ GST\_PAD\_PROBE\_TYPE\_EVENT\_FLUSH to receive callbacks from
+ flushing events. Events are always only notified in push mode.
+
+ You can use this probe to inspect, modify or drop the event.
+
+ - A query travels over a pad. Use the
+ GST\_PAD\_PROBE\_TYPE\_QUERY\_DOWNSTREAM and
+ GST\_PAD\_PROBE\_TYPE\_QUERY\_UPSTREAM flags to select downstream
+ and upstream queries. The convenience
+ GST\_PAD\_PROBE\_TYPE\_QUERY\_BOTH can also be used to select both
+ directions. Query probes will be notified twice, once when the query
+ travels upstream/downstream and once when the query result is
+ returned. You can select in what stage the callback will be called
+ with the GST\_PAD\_PROBE\_TYPE\_PUSH and
+ GST\_PAD\_PROBE\_TYPE\_PULL, respectively when the query is
+ performed and when the query result is returned.
+
+ You can use this probe to inspect or modify the query. You can also
+ answer the query in the probe callback by placing the result value
+ in the query and by returning GST\_PAD\_PROBE\_DROP from the
+ callback.
+
+ - In addition to notifying you of dataflow, you can also ask the probe
+ to block the dataflow when the callback returns. This is called a
+ blocking probe and is activated by specifying the
+ GST\_PAD\_PROBE\_TYPE\_BLOCK flag. You can use this flag with the
+ other flags to only block dataflow on selected activity. A pad
+ becomes unblocked again if you remove the probe or when you return
+ GST\_PAD\_PROBE\_REMOVE from the callback. You can let only the
+ currently blocked item pass by returning GST\_PAD\_PROBE\_PASS from
+ the callback, it will block again on the next item.
+
+ Blocking probes are used to temporarily block pads because they are
+ unlinked or because you are going to unlink them. If the dataflow is
+ not blocked, the pipeline would go into an error state if data is
+ pushed on an unlinked pad. We will se how to use blocking probes to
+ partially preroll a pipeline. See also [Play a region of a media
+ file](#play-a-region-of-a-media-file).
+
+ - Be notified when no activity is happening on a pad. You install this
+ probe with the GST\_PAD\_PROBE\_TYPE\_IDLE flag. You can specify
+ GST\_PAD\_PROBE\_TYPE\_PUSH and/or GST\_PAD\_PROBE\_TYPE\_PULL to
+ only be notified depending on the pad scheduling mode. The IDLE
+ probe is also a blocking probe in that it will not let any data pass
+ on the pad for as long as the IDLE probe is installed.
+
+ You can use idle probes to dynamically relink a pad. We will see how
+ to use idle probes to replace an element in the pipeline. See also
+ [Dynamically changing the
+ pipeline](#dynamically-changing-the-pipeline).
+
+### Data probes
+
+Data probes allow you to be notified when there is data passing on a
+pad. When adding the probe, specify the GST\_PAD\_PROBE\_TYPE\_BUFFER
+and/or GST\_PAD\_PROBE\_TYPE\_BUFFER\_LIST.
+
+Data probes run in pipeline streaming thread context, so callbacks
+should try to not block and generally not do any weird stuff, since this
+could have a negative impact on pipeline performance or, in case of
+bugs, cause deadlocks or crashes. More precisely, one should usually not
+call any GUI-related functions from within a probe callback, nor try to
+change the state of the pipeline. An application may post custom
+messages on the pipeline's bus though to communicate with the main
+application thread and have it do things like stop the pipeline.
+
+In any case, most common buffer operations that elements can do in
+`_chain ()` functions, can be done in probe callbacks as well. The
+example below gives a short impression on how to use them.
+
+``` c
+
+
+#include <gst/gst.h>
+
+static GstPadProbeReturn
+cb_have_data (GstPad *pad,
+ GstPadProbeInfo *info,
+ gpointer user_data)
+{
+ gint x, y;
+ GstMapInfo map;
+ guint16 *ptr, t;
+ GstBuffer *buffer;
+
+ buffer = GST_PAD_PROBE_INFO_BUFFER (info);
+
+ buffer = gst_buffer_make_writable (buffer);
+
+ /* Making a buffer writable can fail (for example if it
+ * cannot be copied and is used more than once)
+ */
+ if (buffer == NULL)
+ return GST_PAD_PROBE_OK;
+
+ /* Mapping a buffer can fail (non-writable) */
+ if (gst_buffer_map (buffer, &map, GST_MAP_WRITE)) {
+ ptr = (guint16 *) map.data;
+ /* invert data */
+ for (y = 0; y < 288; y++) {
+ for (x = 0; x < 384 / 2; x++) {
+ t = ptr[384 - 1 - x];
+ ptr[384 - 1 - x] = ptr[x];
+ ptr[x] = t;
+ }
+ ptr += 384;
+ }
+ gst_buffer_unmap (buffer, &map);
+ }
+
+ GST_PAD_PROBE_INFO_DATA (info) = buffer;
+
+ return GST_PAD_PROBE_OK;
+}
+
+gint
+main (gint argc,
+ gchar *argv[])
+{
+ GMainLoop *loop;
+ GstElement *pipeline, *src, *sink, *filter, *csp;
+ GstCaps *filtercaps;
+ GstPad *pad;
+
+ /* init GStreamer */
+ gst_init (&argc, &argv);
+ loop = g_main_loop_new (NULL, FALSE);
+
+ /* build */
+ pipeline = gst_pipeline_new ("my-pipeline");
+ src = gst_element_factory_make ("videotestsrc", "src");
+ if (src == NULL)
+ g_error ("Could not create 'videotestsrc' element");
+
+ filter = gst_element_factory_make ("capsfilter", "filter");
+ g_assert (filter != NULL); /* should always exist */
+
+ csp = gst_element_factory_make ("videoconvert", "csp");
+ if (csp == NULL)
+ g_error ("Could not create 'videoconvert' element");
+
+ sink = gst_element_factory_make ("xvimagesink", "sink");
+ if (sink == NULL) {
+ sink = gst_element_factory_make ("ximagesink", "sink");
+ if (sink == NULL)
+ g_error ("Could not create neither 'xvimagesink' nor 'ximagesink' element");
+ }
+
+ gst_bin_add_many (GST_BIN (pipeline), src, filter, csp, sink, NULL);
+ gst_element_link_many (src, filter, csp, sink, NULL);
+ filtercaps = gst_caps_new_simple ("video/x-raw",
+ "format", G_TYPE_STRING, "RGB16",
+ "width", G_TYPE_INT, 384,
+ "height", G_TYPE_INT, 288,
+ "framerate", GST_TYPE_FRACTION, 25, 1,
+ NULL);
+ g_object_set (G_OBJECT (filter), "caps", filtercaps, NULL);
+ gst_caps_unref (filtercaps);
+
+ pad = gst_element_get_static_pad (src, "src");
+ gst_pad_add_probe (pad, GST_PAD_PROBE_TYPE_BUFFER,
+ (GstPadProbeCallback) cb_have_data, NULL, NULL);
+ gst_object_unref (pad);
+
+ /* run */
+ gst_element_set_state (pipeline, GST_STATE_PLAYING);
+
+ /* wait until it's up and running or failed */
+ if (gst_element_get_state (pipeline, NULL, NULL, -1) == GST_STATE_CHANGE_FAILURE) {
+ g_error ("Failed to go into PLAYING state");
+ }
+
+ g_print ("Running ...\n");
+ g_main_loop_run (loop);
+
+ /* exit */
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+
+ return 0;
+}
+
+
+
+```
+
+Compare that output with the output of “gst-launch-1.0 videotestsrc \!
+xvimagesink”, just so you know what you're looking for.
+
+Strictly speaking, a pad probe callback is only allowed to modify the
+buffer content if the buffer is writable. Whether this is the case or
+not depends a lot on the pipeline and the elements involved. Often
+enough, this is the case, but sometimes it is not, and if it is not then
+unexpected modification of the data or metadata can introduce bugs that
+are very hard to debug and track down. You can check if a buffer is
+writable with `gst_buffer_is_writable ()`. Since you can pass back a
+different buffer than the one passed in, it is a good idea to make the
+buffer writable in the callback function with `gst_buffer_make_writable
+()`.
+
+Pad probes are suited best for looking at data as it passes through the
+pipeline. If you need to modify data, you should better write your own
+GStreamer element. Base classes like GstAudioFilter, GstVideoFilter or
+GstBaseTransform make this fairly easy.
+
+If you just want to inspect buffers as they pass through the pipeline,
+you don't even need to set up pad probes. You could also just insert an
+identity element into the pipeline and connect to its "handoff" signal.
+The identity element also provides a few useful debugging tools like the
+"dump" property or the "last-message" property (the latter is enabled by
+passing the '-v' switch to gst-launch and by setting the silent property
+on the identity to FALSE).
+
+### Play a region of a media file
+
+In this example we will show you how to play back a region of a media
+file. The goal is to only play the part of a file from 2 seconds to 5
+seconds and then EOS.
+
+In a first step we will set a uridecodebin element to the PAUSED state
+and make sure that we block all the source pads that are created. When
+all the source pads are blocked, we have data on all source pads and we
+say that the uridecodebin is prerolled.
+
+In a prerolled pipeline we can ask for the duration of the media and we
+can also perform seeks. We are interested in performing a seek operation
+on the pipeline to select the range of media that we are interested in.
+
+After we configure the region we are interested in, we can link the sink
+element, unblock the source pads and set the pipeline to the playing
+state. You will see that exactly the requested region is played by the
+sink before it goes to EOS.
+
+What follows is an example application that loosly follows this
+algorithm.
+
+``` c
+
+
+#include <gst/gst.h>
+
+static GMainLoop *loop;
+static volatile gint counter;
+static GstBus *bus;
+static gboolean prerolled = FALSE;
+static GstPad *sinkpad;
+
+static void
+dec_counter (GstElement * pipeline)
+{
+ if (prerolled)
+ return;
+
+ if (g_atomic_int_dec_and_test (&counter)) {
+ /* all probes blocked and no-more-pads signaled, post
+ * message on the bus. */
+ prerolled = TRUE;
+
+ gst_bus_post (bus, gst_message_new_application (
+ GST_OBJECT_CAST (pipeline),
+ gst_structure_new_empty ("ExPrerolled")));
+ }
+}
+
+/* called when a source pad of uridecodebin is blocked */
+static GstPadProbeReturn
+cb_blocked (GstPad *pad,
+ GstPadProbeInfo *info,
+ gpointer user_data)
+{
+ GstElement *pipeline = GST_ELEMENT (user_data);
+
+ if (prerolled)
+ return GST_PAD_PROBE_REMOVE;
+
+ dec_counter (pipeline);
+
+ return GST_PAD_PROBE_OK;
+}
+
+/* called when uridecodebin has a new pad */
+static void
+cb_pad_added (GstElement *element,
+ GstPad *pad,
+ gpointer user_data)
+{
+ GstElement *pipeline = GST_ELEMENT (user_data);
+
+ if (prerolled)
+ return;
+
+ g_atomic_int_inc (&counter);
+
+ gst_pad_add_probe (pad, GST_PAD_PROBE_TYPE_BLOCK_DOWNSTREAM,
+ (GstPadProbeCallback) cb_blocked, pipeline, NULL);
+
+ /* try to link to the video pad */
+ gst_pad_link (pad, sinkpad);
+}
+
+/* called when uridecodebin has created all pads */
+static void
+cb_no_more_pads (GstElement *element,
+ gpointer user_data)
+{
+ GstElement *pipeline = GST_ELEMENT (user_data);
+
+ if (prerolled)
+ return;
+
+ dec_counter (pipeline);
+}
+
+/* called when a new message is posted on the bus */
+static void
+cb_message (GstBus *bus,
+ GstMessage *message,
+ gpointer user_data)
+{
+ GstElement *pipeline = GST_ELEMENT (user_data);
+
+ switch (GST_MESSAGE_TYPE (message)) {
+ case GST_MESSAGE_ERROR:
+ g_print ("we received an error!\n");
+ g_main_loop_quit (loop);
+ break;
+ case GST_MESSAGE_EOS:
+ g_print ("we reached EOS\n");
+ g_main_loop_quit (loop);
+ break;
+ case GST_MESSAGE_APPLICATION:
+ {
+ if (gst_message_has_name (message, "ExPrerolled")) {
+ /* it's our message */
+ g_print ("we are all prerolled, do seek\n");
+ gst_element_seek (pipeline,
+ 1.0, GST_FORMAT_TIME,
+ GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE,
+ GST_SEEK_TYPE_SET, 2 * GST_SECOND,
+ GST_SEEK_TYPE_SET, 5 * GST_SECOND);
+
+ gst_element_set_state (pipeline, GST_STATE_PLAYING);
+ }
+ break;
+ }
+ default:
+ break;
+ }
+}
+
+gint
+main (gint argc,
+ gchar *argv[])
+{
+ GstElement *pipeline, *src, *csp, *vs, *sink;
+
+ /* init GStreamer */
+ gst_init (&argc, &argv);
+ loop = g_main_loop_new (NULL, FALSE);
+
+ if (argc < 2) {
+ g_print ("usage: %s <uri>", argv[0]);
+ return -1;
+ }
+
+ /* build */
+ pipeline = gst_pipeline_new ("my-pipeline");
+
+ bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
+ gst_bus_add_signal_watch (bus);
+ g_signal_connect (bus, "message", (GCallback) cb_message,
+ pipeline);
+
+ src = gst_element_factory_make ("uridecodebin", "src");
+ if (src == NULL)
+ g_error ("Could not create 'uridecodebin' element");
+
+ g_object_set (src, "uri", argv[1], NULL);
+
+ csp = gst_element_factory_make ("videoconvert", "csp");
+ if (csp == NULL)
+ g_error ("Could not create 'videoconvert' element");
+
+ vs = gst_element_factory_make ("videoscale", "vs");
+ if (csp == NULL)
+ g_error ("Could not create 'videoscale' element");
+
+ sink = gst_element_factory_make ("autovideosink", "sink");
+ if (sink == NULL)
+ g_error ("Could not create 'autovideosink' element");
+
+ gst_bin_add_many (GST_BIN (pipeline), src, csp, vs, sink, NULL);
+
+ /* can't link src yet, it has no pads */
+ gst_element_link_many (csp, vs, sink, NULL);
+
+ sinkpad = gst_element_get_static_pad (csp, "sink");
+
+ /* for each pad block that is installed, we will increment
+ * the counter. for each pad block that is signaled, we
+ * decrement the counter. When the counter is 0 we post
+ * an app message to tell the app that all pads are
+ * blocked. Start with 1 that is decremented when no-more-pads
+ * is signaled to make sure that we only post the message
+ * after no-more-pads */
+ g_atomic_int_set (&counter, 1);
+
+ g_signal_connect (src, "pad-added",
+ (GCallback) cb_pad_added, pipeline);
+ g_signal_connect (src, "no-more-pads",
+ (GCallback) cb_no_more_pads, pipeline);
+
+ gst_element_set_state (pipeline, GST_STATE_PAUSED);
+
+ g_main_loop_run (loop);
+
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+
+ gst_object_unref (sinkpad);
+ gst_object_unref (bus);
+ gst_object_unref (pipeline);
+ g_main_loop_unref (loop);
+
+ return 0;
+}
+
+```
+
+Note that we use a custom application message to signal the main thread
+that the uridecidebin is prerolled. The main thread will then issue a
+flushing seek to the requested region. The flush will temporarily
+unblock the pad and reblock them when new data arrives again. We detect
+this second block to remove the probes. Then we set the pipeline to
+PLAYING and it should play from 2 to 5 seconds, then EOS and exit the
+application.
+
+## Manually adding or removing data from/to a pipeline
+
+Many people have expressed the wish to use their own sources to inject
+data into a pipeline. Some people have also expressed the wish to grab
+the output in a pipeline and take care of the actual output inside their
+application. While either of these methods are strongly discouraged,
+GStreamer offers support for this. *Beware\! You need to know what you
+are doing.* Since you don't have any support from a base class you need
+to thoroughly understand state changes and synchronization. If it
+doesn't work, there are a million ways to shoot yourself in the foot.
+It's always better to simply write a plugin and have the base class
+manage it. See the Plugin Writer's Guide for more information on this
+topic. Also see the next section, which will explain how to embed
+plugins statically in your application.
+
+There's two possible elements that you can use for the above-mentioned
+purposes. Those are called “appsrc” (an imaginary source) and “appsink”
+(an imaginary sink). The same method applies to each of those elements.
+Here, we will discuss how to use those elements to insert (using appsrc)
+or grab (using appsink) data from a pipeline, and how to set
+negotiation.
+
+Both appsrc and appsink provide 2 sets of API. One API uses standard
+GObject (action) signals and properties. The same API is also available
+as a regular C api. The C api is more performant but requires you to
+link to the app library in order to use the elements.
+
+### Inserting data with appsrc
+
+First we look at some examples for appsrc, which lets you insert data
+into the pipeline from the application. Appsrc has some configuration
+options that define how it will operate. You should decide about the
+following configurations:
+
+ - Will the appsrc operate in push or pull mode. The stream-type
+ property can be used to control this. stream-type of “random-access”
+ will activate pull mode scheduling while the other stream-types
+ activate push mode.
+
+ - The caps of the buffers that appsrc will push out. This needs to be
+ configured with the caps property. The caps must be set to a fixed
+ caps and will be used to negotiate a format downstream.
+
+ - If the appsrc operates in live mode or not. This can be configured
+ with the is-live property. When operating in live-mode it is
+ important to configure the min-latency and max-latency in appsrc.
+ The min-latency should be set to the amount of time it takes between
+ capturing a buffer and when it is pushed inside appsrc. In live
+ mode, you should timestamp the buffers with the pipeline
+ running-time when the first byte of the buffer was captured before
+ feeding them to appsrc. You can let appsrc do the timestaping with
+ the do-timestamp property (but then the min-latency must be set to 0
+ because it timestamps based on the running-time when the buffer
+ entered appsrc).
+
+ - The format of the SEGMENT event that appsrc will push. The format
+ has implications for how the running-time of the buffers will be
+ calculated so you must be sure you understand this. For live sources
+ you probably want to set the format property to GST\_FORMAT\_TIME.
+ For non-live source it depends on the media type that you are
+ handling. If you plan to timestamp the buffers, you should probably
+ put a GST\_FORMAT\_TIME format, otherwise GST\_FORMAT\_BYTES might
+ be appropriate.
+
+ - If appsrc operates in random-access mode, it is important to
+ configure the size property of appsrc with the number of bytes in
+ the stream. This will allow downstream elements to know the size of
+ the media and alows them to seek to the end of the stream when
+ needed.
+
+The main way of handling data to appsrc is by using the function
+`gst_app_src_push_buffer ()` or by emiting the push-buffer action
+signal. This will put the buffer onto a queue from which appsrc will
+read from in its streaming thread. It is important to note that data
+transport will not happen from the thread that performed the push-buffer
+call.
+
+The “max-bytes” property controls how much data can be queued in appsrc
+before appsrc considers the queue full. A filled internal queue will
+always signal the “enough-data” signal, which signals the application
+that it should stop pushing data into appsrc. The “block” property will
+cause appsrc to block the push-buffer method until free data becomes
+available again.
+
+When the internal queue is running out of data, the “need-data” signal
+is emitted, which signals the application that it should start pushing
+more data into appsrc.
+
+In addition to the “need-data” and “enough-data” signals, appsrc can
+emit the “seek-data” signal when the “stream-mode” property is set to
+“seekable” or “random-access”. The signal argument will contain the
+new desired position in the stream expressed in the unit set with the
+“format” property. After receiving the seek-data signal, the
+application should push-buffers from the new position.
+
+When the last byte is pushed into appsrc, you must call
+`gst_app_src_end_of_stream ()` to make it send an EOS downstream.
+
+These signals allow the application to operate appsrc in push and pull
+mode as will be explained next.
+
+#### Using appsrc in push mode
+
+When appsrc is configured in push mode (stream-type is stream or
+seekable), the application repeatedly calls the push-buffer method with
+a new buffer. Optionally, the queue size in the appsrc can be controlled
+with the enough-data and need-data signals by respectively
+stopping/starting the push-buffer calls. The value of the min-percent
+property defines how empty the internal appsrc queue needs to be before
+the need-data signal will be fired. You can set this to some value \>0
+to avoid completely draining the queue.
+
+When the stream-type is set to seekable, don't forget to implement a
+seek-data callback.
+
+Use this model when implementing various network protocols or hardware
+devices.
+
+#### Using appsrc in pull mode
+
+In the pull model, data is fed to appsrc from the need-data signal
+handler. You should push exactly the amount of bytes requested in the
+need-data signal. You are only allowed to push less bytes when you are
+at the end of the stream.
+
+Use this model for file access or other randomly accessable sources.
+
+#### Appsrc example
+
+This example application will generate black/white (it switches every
+second) video to an Xv-window output by using appsrc as a source with
+caps to force a format. We use a colorspace conversion element to make
+sure that we feed the right format to your X server. We configure a
+video stream with a variable framerate (0/1) and we set the timestamps
+on the outgoing buffers in such a way that we play 2 frames per second.
+
+Note how we use the pull mode method of pushing new buffers into appsrc
+although appsrc is running in push mode.
+
+``` c
+
+
+#include <gst/gst.h>
+
+static GMainLoop *loop;
+
+static void
+cb_need_data (GstElement *appsrc,
+ guint unused_size,
+ gpointer user_data)
+{
+ static gboolean white = FALSE;
+ static GstClockTime timestamp = 0;
+ GstBuffer *buffer;
+ guint size;
+ GstFlowReturn ret;
+
+ size = 385 * 288 * 2;
+
+ buffer = gst_buffer_new_allocate (NULL, size, NULL);
+
+ /* this makes the image black/white */
+ gst_buffer_memset (buffer, 0, white ? 0xff : 0x0, size);
+
+ white = !white;
+
+ GST_BUFFER_PTS (buffer) = timestamp;
+ GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1, GST_SECOND, 2);
+
+ timestamp += GST_BUFFER_DURATION (buffer);
+
+ g_signal_emit_by_name (appsrc, "push-buffer", buffer, &ret);
+ gst_buffer_unref (buffer);
+
+ if (ret != GST_FLOW_OK) {
+ /* something wrong, stop pushing */
+ g_main_loop_quit (loop);
+ }
+}
+
+gint
+main (gint argc,
+ gchar *argv[])
+{
+ GstElement *pipeline, *appsrc, *conv, *videosink;
+
+ /* init GStreamer */
+ gst_init (&argc, &argv);
+ loop = g_main_loop_new (NULL, FALSE);
+
+ /* setup pipeline */
+ pipeline = gst_pipeline_new ("pipeline");
+ appsrc = gst_element_factory_make ("appsrc", "source");
+ conv = gst_element_factory_make ("videoconvert", "conv");
+ videosink = gst_element_factory_make ("xvimagesink", "videosink");
+
+ /* setup */
+ g_object_set (G_OBJECT (appsrc), "caps",
+ gst_caps_new_simple ("video/x-raw",
+ "format", G_TYPE_STRING, "RGB16",
+ "width", G_TYPE_INT, 384,
+ "height", G_TYPE_INT, 288,
+ "framerate", GST_TYPE_FRACTION, 0, 1,
+ NULL), NULL);
+ gst_bin_add_many (GST_BIN (pipeline), appsrc, conv, videosink, NULL);
+ gst_element_link_many (appsrc, conv, videosink, NULL);
+
+ /* setup appsrc */
+ g_object_set (G_OBJECT (appsrc),
+ "stream-type", 0,
+ "format", GST_FORMAT_TIME, NULL);
+ g_signal_connect (appsrc, "need-data", G_CALLBACK (cb_need_data), NULL);
+
+ /* play */
+ gst_element_set_state (pipeline, GST_STATE_PLAYING);
+ g_main_loop_run (loop);
+
+ /* clean up */
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (GST_OBJECT (pipeline));
+ g_main_loop_unref (loop);
+
+ return 0;
+ }
+
+
+
+```
+
+### Grabbing data with appsink
+
+Unlike appsrc, appsink is a little easier to use. It also supports a
+pull and push based model of getting data from the pipeline.
+
+The normal way of retrieving samples from appsink is by using the
+`gst_app_sink_pull_sample()` and `gst_app_sink_pull_preroll()` methods
+or by using the “pull-sample” and “pull-preroll” signals. These methods
+block until a sample becomes available in the sink or when the sink is
+shut down or reaches EOS.
+
+Appsink will internally use a queue to collect buffers from the
+streaming thread. If the application is not pulling samples fast enough,
+this queue will consume a lot of memory over time. The “max-buffers”
+property can be used to limit the queue size. The “drop” property
+controls whether the streaming thread blocks or if older buffers are
+dropped when the maximum queue size is reached. Note that blocking the
+streaming thread can negatively affect real-time performance and should
+be avoided.
+
+If a blocking behaviour is not desirable, setting the “emit-signals”
+property to TRUE will make appsink emit the “new-sample” and
+“new-preroll” signals when a sample can be pulled without blocking.
+
+The “caps” property on appsink can be used to control the formats that
+appsink can receive. This property can contain non-fixed caps, the
+format of the pulled samples can be obtained by getting the sample caps.
+
+If one of the pull-preroll or pull-sample methods return NULL, the
+appsink is stopped or in the EOS state. You can check for the EOS state
+with the “eos” property or with the `gst_app_sink_is_eos()` method.
+
+The eos signal can also be used to be informed when the EOS state is
+reached to avoid polling.
+
+Consider configuring the following properties in the appsink:
+
+ - The “sync” property if you want to have the sink base class
+ synchronize the buffer against the pipeline clock before handing you
+ the sample.
+
+ - Enable Quality-of-Service with the “qos” property. If you are
+ dealing with raw video frames and let the base class sycnhronize on
+ the clock, it might be a good idea to also let the base class send
+ QOS events upstream.
+
+ - The caps property that contains the accepted caps. Upstream elements
+ will try to convert the format so that it matches the configured
+ caps on appsink. You must still check the `GstSample` to get the
+ actual caps of the buffer.
+
+#### Appsink example
+
+What follows is an example on how to capture a snapshot of a video
+stream using appsink.
+
+``` c
+
+
+#include <gst/gst.h>
+#ifdef HAVE_GTK
+#include <gtk/gtk.h>
+#endif
+
+#include <stdlib.h>
+
+#define CAPS "video/x-raw,format=RGB,width=160,pixel-aspect-ratio=1/1"
+
+int
+main (int argc, char *argv[])
+{
+ GstElement *pipeline, *sink;
+ gint width, height;
+ GstSample *sample;
+ gchar *descr;
+ GError *error = NULL;
+ gint64 duration, position;
+ GstStateChangeReturn ret;
+ gboolean res;
+ GstMapInfo map;
+
+ gst_init (&argc, &argv);
+
+ if (argc != 2) {
+ g_print ("usage: %s <uri>\n Writes snapshot.png in the current directory\n",
+ argv[0]);
+ exit (-1);
+ }
+
+ /* create a new pipeline */
+ descr =
+ g_strdup_printf ("uridecodebin uri=%s ! videoconvert ! videoscale ! "
+ " appsink name=sink caps=\"" CAPS "\"", argv[1]);
+ pipeline = gst_parse_launch (descr, &error);
+
+ if (error != NULL) {
+ g_print ("could not construct pipeline: %s\n", error->message);
+ g_clear_error (&error);
+ exit (-1);
+ }
+
+ /* get sink */
+ sink = gst_bin_get_by_name (GST_BIN (pipeline), "sink");
+
+ /* set to PAUSED to make the first frame arrive in the sink */
+ ret = gst_element_set_state (pipeline, GST_STATE_PAUSED);
+ switch (ret) {
+ case GST_STATE_CHANGE_FAILURE:
+ g_print ("failed to play the file\n");
+ exit (-1);
+ case GST_STATE_CHANGE_NO_PREROLL:
+ /* for live sources, we need to set the pipeline to PLAYING before we can
+ * receive a buffer. We don't do that yet */
+ g_print ("live sources not supported yet\n");
+ exit (-1);
+ default:
+ break;
+ }
+ /* This can block for up to 5 seconds. If your machine is really overloaded,
+ * it might time out before the pipeline prerolled and we generate an error. A
+ * better way is to run a mainloop and catch errors there. */
+ ret = gst_element_get_state (pipeline, NULL, NULL, 5 * GST_SECOND);
+ if (ret == GST_STATE_CHANGE_FAILURE) {
+ g_print ("failed to play the file\n");
+ exit (-1);
+ }
+
+ /* get the duration */
+ gst_element_query_duration (pipeline, GST_FORMAT_TIME, &duration);
+
+ if (duration != -1)
+ /* we have a duration, seek to 5% */
+ position = duration * 5 / 100;
+ else
+ /* no duration, seek to 1 second, this could EOS */
+ position = 1 * GST_SECOND;
+
+ /* seek to the a position in the file. Most files have a black first frame so
+ * by seeking to somewhere else we have a bigger chance of getting something
+ * more interesting. An optimisation would be to detect black images and then
+ * seek a little more */
+ gst_element_seek_simple (pipeline, GST_FORMAT_TIME,
+ GST_SEEK_FLAG_KEY_UNIT | GST_SEEK_FLAG_FLUSH, position);
+
+ /* get the preroll buffer from appsink, this block untils appsink really
+ * prerolls */
+ g_signal_emit_by_name (sink, "pull-preroll", &sample, NULL);
+
+ /* if we have a buffer now, convert it to a pixbuf. It's possible that we
+ * don't have a buffer because we went EOS right away or had an error. */
+ if (sample) {
+ GstBuffer *buffer;
+ GstCaps *caps;
+ GstStructure *s;
+
+ /* get the snapshot buffer format now. We set the caps on the appsink so
+ * that it can only be an rgb buffer. The only thing we have not specified
+ * on the caps is the height, which is dependant on the pixel-aspect-ratio
+ * of the source material */
+ caps = gst_sample_get_caps (sample);
+ if (!caps) {
+ g_print ("could not get snapshot format\n");
+ exit (-1);
+ }
+ s = gst_caps_get_structure (caps, 0);
+
+ /* we need to get the final caps on the buffer to get the size */
+ res = gst_structure_get_int (s, "width", &width);
+ res |= gst_structure_get_int (s, "height", &height);
+ if (!res) {
+ g_print ("could not get snapshot dimension\n");
+ exit (-1);
+ }
+
+ /* create pixmap from buffer and save, gstreamer video buffers have a stride
+ * that is rounded up to the nearest multiple of 4 */
+ buffer = gst_sample_get_buffer (sample);
+ /* Mapping a buffer can fail (non-readable) */
+ if (gst_buffer_map (buffer, &map, GST_MAP_READ)) {
+#ifdef HAVE_GTK
+ pixbuf = gdk_pixbuf_new_from_data (map.data,
+ GDK_COLORSPACE_RGB, FALSE, 8, width, height,
+ GST_ROUND_UP_4 (width * 3), NULL, NULL);
+
+ /* save the pixbuf */
+ gdk_pixbuf_save (pixbuf, "snapshot.png", "png", &error, NULL);
+#endif
+ gst_buffer_unmap (buffer, &map);
+ }
+ gst_sample_unref (sample);
+ } else {
+ g_print ("could not make snapshot\n");
+ }
+
+ /* cleanup and exit */
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+
+ exit (0);
+}
+
+```
+
+## Forcing a format
+
+Sometimes you'll want to set a specific format, for example a video size
+and format or an audio bitsize and number of channels. You can do this
+by forcing a specific `GstCaps` on the pipeline, which is possible by
+using *filtered caps*. You can set a filtered caps on a link by using
+the “capsfilter” element in between the two elements, and specifying a
+`GstCaps` as “caps” property on this element. It will then only allow
+types matching that specified capability set for negotiation. See also
+[Creating capabilities for
+filtering](manual-pads.md#creating-capabilities-for-filtering).
+
+### Changing format in a PLAYING pipeline
+
+It is also possible to dynamically change the format in a pipeline while
+PLAYING. This can simply be done by changing the caps property on a
+capsfilter. The capsfilter will send a RECONFIGURE event upstream that
+will make the upstream element attempt to renegotiate a new format and
+allocator. This only works if the upstream element is not using fixed
+caps on the source pad.
+
+Below is an example of how you can change the caps of a pipeline while
+in the PLAYING state:
+
+``` c
+
+
+#include <stdlib.h>
+
+#include <gst/gst.h>
+
+#define MAX_ROUND 100
+
+int
+main (int argc, char **argv)
+{
+ GstElement *pipe, *filter;
+ GstCaps *caps;
+ gint width, height;
+ gint xdir, ydir;
+ gint round;
+ GstMessage *message;
+
+ gst_init (&argc, &argv);
+
+ pipe = gst_parse_launch_full ("videotestsrc ! capsfilter name=filter ! "
+ "ximagesink", NULL, GST_PARSE_FLAG_NONE, NULL);
+ g_assert (pipe != NULL);
+
+ filter = gst_bin_get_by_name (GST_BIN (pipe), "filter");
+ g_assert (filter);
+
+ width = 320;
+ height = 240;
+ xdir = ydir = -10;
+
+ for (round = 0; round < MAX_ROUND; round++) {
+ gchar *capsstr;
+ g_print ("resize to %dx%d (%d/%d) \r", width, height, round, MAX_ROUND);
+
+ /* we prefer our fixed width and height but allow other dimensions to pass
+ * as well */
+ capsstr = g_strdup_printf ("video/x-raw, width=(int)%d, height=(int)%d",
+ width, height);
+
+ caps = gst_caps_from_string (capsstr);
+ g_free (capsstr);
+ g_object_set (filter, "caps", caps, NULL);
+ gst_caps_unref (caps);
+
+ if (round == 0)
+ gst_element_set_state (pipe, GST_STATE_PLAYING);
+
+ width += xdir;
+ if (width >= 320)
+ xdir = -10;
+ else if (width < 200)
+ xdir = 10;
+
+ height += ydir;
+ if (height >= 240)
+ ydir = -10;
+ else if (height < 150)
+ ydir = 10;
+
+ message =
+ gst_bus_poll (GST_ELEMENT_BUS (pipe), GST_MESSAGE_ERROR,
+ 50 * GST_MSECOND);
+ if (message) {
+ g_print ("got error \n");
+
+ gst_message_unref (message);
+ }
+ }
+ g_print ("done \n");
+
+ gst_object_unref (filter);
+ gst_element_set_state (pipe, GST_STATE_NULL);
+ gst_object_unref (pipe);
+
+ return 0;
+}
+
+
+
+```
+
+Note how we use `gst_bus_poll()` with a small timeout to get messages
+and also introduce a short sleep.
+
+It is possible to set multiple caps for the capsfilter separated with a
+;. The capsfilter will try to renegotiate to the first possible format
+from the list.
+
+## Dynamically changing the pipeline
+
+In this section we talk about some techniques for dynamically modifying
+the pipeline. We are talking specifically about changing the pipeline
+while it is in the PLAYING state without interrupting the flow.
+
+There are some important things to consider when building dynamic
+pipelines:
+
+ - When removing elements from the pipeline, make sure that there is no
+ dataflow on unlinked pads because that will cause a fatal pipeline
+ error. Always block source pads (in push mode) or sink pads (in pull
+ mode) before unlinking pads. See also [Changing elements in a
+ pipeline](#changing-elements-in-a-pipeline).
+
+ - When adding elements to a pipeline, make sure to put the element
+ into the right state, usually the same state as the parent, before
+ allowing dataflow the element. When an element is newly created, it
+ is in the NULL state and will return an error when it receives data.
+ See also [Changing elements in a
+ pipeline](#changing-elements-in-a-pipeline).
+
+ - When adding elements to a pipeline, GStreamer will by default set
+ the clock and base-time on the element to the current values of the
+ pipeline. This means that the element will be able to construct the
+ same pipeline running-time as the other elements in the pipeline.
+ This means that sinks will synchronize buffers like the other sinks
+ in the pipeline and that sources produce buffers with a running-time
+ that matches the other sources.
+
+ - When unlinking elements from an upstream chain, always make sure to
+ flush any queued data in the element by sending an EOS event down
+ the element sink pad(s) and by waiting that the EOS leaves the
+ elements (with an event probe).
+
+ If you do not do this, you will lose the data which is buffered by
+ the unlinked element. This can result in a simple frame loss (one or
+ more video frames, several milliseconds of audio). However if you
+ remove a muxer (and in some cases an encoder or similar elements)
+ from the pipeline, you risk getting a corrupted file which could not
+ be played properly, as some relevant metadata (header, seek/index
+ tables, internal sync tags) will not be stored or updated properly.
+
+ See also [Changing elements in a
+ pipeline](#changing-elements-in-a-pipeline).
+
+ - A live source will produce buffers with a running-time of the
+ current running-time in the pipeline.
+
+ A pipeline without a live source produces buffers with a
+ running-time starting from 0. Likewise, after a flushing seek, those
+ pipelines reset the running-time back to 0.
+
+ The running-time can be changed with `gst_pad_set_offset ()`. It is
+ important to know the running-time of the elements in the pipeline
+ in order to maintain synchronization.
+
+ - Adding elements might change the state of the pipeline. Adding a
+ non-prerolled sink, for example, brings the pipeline back to the
+ prerolling state. Removing a non-prerolled sink, for example, might
+ change the pipeline to PAUSED and PLAYING state.
+
+ Adding a live source cancels the preroll stage and put the pipeline
+ to the playing state. Adding a live source or other live elements
+ might also change the latency of a pipeline.
+
+ Adding or removing elements to the pipeline might change the clock
+ selection of the pipeline. If the newly added element provides a
+ clock, it might be worth changing the clock in the pipeline to the
+ new clock. If, on the other hand, the element that provides the
+ clock for the pipeline is removed, a new clock has to be selected.
+
+ - Adding and removing elements might cause upstream or downstream
+ elements to renegotiate caps and or allocators. You don't really
+ need to do anything from the application, plugins largely adapt
+ themself to the new pipeline topology in order to optimize their
+ formats and allocation strategy.
+
+ What is important is that when you add, remove or change elements in
+ the pipeline, it is possible that the pipeline needs to negotiate a
+ new format and this can fail. Usually you can fix this by inserting
+ the right converter elements where needed. See also [Changing
+ elements in a pipeline](#changing-elements-in-a-pipeline).
+
+GStreamer offers support for doing about any dynamic pipeline
+modification but it requires you to know a bit of details before you can
+do this without causing pipeline errors. In the following sections we
+will demonstrate a couple of typical use-cases.
+
+### Changing elements in a pipeline
+
+In the next example we look at the following chain of elements:
+
+```
+ - ----. .----------. .---- -
+ element1 | | element2 | | element3
+ src -> sink src -> sink
+ - ----' '----------' '---- -
+
+```
+
+We want to change element2 by element4 while the pipeline is in the
+PLAYING state. Let's say that element2 is a visualization and that you
+want to switch the visualization in the pipeline.
+
+We can't just unlink element2's sinkpad from element1's source pad
+because that would leave element1's source pad unlinked and would cause
+a streaming error in the pipeline when data is pushed on the source pad.
+The technique is to block the dataflow from element1's source pad before
+we change element2 by element4 and then resume dataflow as shown in the
+following steps:
+
+ - Block element1's source pad with a blocking pad probe. When the pad
+ is blocked, the probe callback will be called.
+
+ - Inside the block callback nothing is flowing between element1 and
+ element2 and nothing will flow until unblocked.
+
+ - Unlink element1 and element2.
+
+ - Make sure data is flushed out of element2. Some elements might
+ internally keep some data, you need to make sure not to lose data by
+ forcing it out of element2. You can do this by pushing EOS into
+ element2, like this:
+
+ - Put an event probe on element2's source pad.
+
+ - Send EOS to element2's sinkpad. This makes sure the all the data
+ inside element2 is forced out.
+
+ - Wait for the EOS event to appear on element2's source pad. When
+ the EOS is received, drop it and remove the event probe.
+
+ - Unlink element2 and element3. You can now also remove element2 from
+ the pipeline and set the state to NULL.
+
+ - Add element4 to the pipeline, if not already added. Link element4
+ and element3. Link element1 and element4.
+
+ - Make sure element4 is in the same state as the rest of the elements
+ in the pipeline. It should be at least in the PAUSED state before it
+ can receive buffers and events.
+
+ - Unblock element1's source pad probe. This will let new data into
+ element4 and continue streaming.
+
+The above algorithm works when the source pad is blocked, i.e. when
+there is dataflow in the pipeline. If there is no dataflow, there is
+also no point in changing the element (just yet) so this algorithm can
+be used in the PAUSED state as well.
+
+Let show you how this works with an example. This example changes the
+video effect on a simple pipeline every second.
+
+``` c
+
+
+#include <gst/gst.h>
+
+static gchar *opt_effects = NULL;
+
+#define DEFAULT_EFFECTS "identity,exclusion,navigationtest," \
+ "agingtv,videoflip,vertigotv,gaussianblur,shagadelictv,edgetv"
+
+static GstPad *blockpad;
+static GstElement *conv_before;
+static GstElement *conv_after;
+static GstElement *cur_effect;
+static GstElement *pipeline;
+
+static GQueue effects = G_QUEUE_INIT;
+
+static GstPadProbeReturn
+event_probe_cb (GstPad * pad, GstPadProbeInfo * info, gpointer user_data)
+{
+ GMainLoop *loop = user_data;
+ GstElement *next;
+
+ if (GST_EVENT_TYPE (GST_PAD_PROBE_INFO_DATA (info)) != GST_EVENT_EOS)
+ return GST_PAD_PROBE_PASS;
+
+ gst_pad_remove_probe (pad, GST_PAD_PROBE_INFO_ID (info));
+
+ /* push current effect back into the queue */
+ g_queue_push_tail (&effects, gst_object_ref (cur_effect));
+ /* take next effect from the queue */
+ next = g_queue_pop_head (&effects);
+ if (next == NULL) {
+ GST_DEBUG_OBJECT (pad, "no more effects");
+ g_main_loop_quit (loop);
+ return GST_PAD_PROBE_DROP;
+ }
+
+ g_print ("Switching from '%s' to '%s'..\n", GST_OBJECT_NAME (cur_effect),
+ GST_OBJECT_NAME (next));
+
+ gst_element_set_state (cur_effect, GST_STATE_NULL);
+
+ /* remove unlinks automatically */
+ GST_DEBUG_OBJECT (pipeline, "removing %" GST_PTR_FORMAT, cur_effect);
+ gst_bin_remove (GST_BIN (pipeline), cur_effect);
+
+ GST_DEBUG_OBJECT (pipeline, "adding %" GST_PTR_FORMAT, next);
+ gst_bin_add (GST_BIN (pipeline), next);
+
+ GST_DEBUG_OBJECT (pipeline, "linking..");
+ gst_element_link_many (conv_before, next, conv_after, NULL);
+
+ gst_element_set_state (next, GST_STATE_PLAYING);
+
+ cur_effect = next;
+ GST_DEBUG_OBJECT (pipeline, "done");
+
+ return GST_PAD_PROBE_DROP;
+}
+
+static GstPadProbeReturn
+pad_probe_cb (GstPad * pad, GstPadProbeInfo * info, gpointer user_data)
+{
+ GstPad *srcpad, *sinkpad;
+
+ GST_DEBUG_OBJECT (pad, "pad is blocked now");
+
+ /* remove the probe first */
+ gst_pad_remove_probe (pad, GST_PAD_PROBE_INFO_ID (info));
+
+ /* install new probe for EOS */
+ srcpad = gst_element_get_static_pad (cur_effect, "src");
+ gst_pad_add_probe (srcpad, GST_PAD_PROBE_TYPE_BLOCK |
+ GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM, event_probe_cb, user_data, NULL);
+ gst_object_unref (srcpad);
+
+ /* push EOS into the element, the probe will be fired when the
+ * EOS leaves the effect and it has thus drained all of its data */
+ sinkpad = gst_element_get_static_pad (cur_effect, "sink");
+ gst_pad_send_event (sinkpad, gst_event_new_eos ());
+ gst_object_unref (sinkpad);
+
+ return GST_PAD_PROBE_OK;
+}
+
+static gboolean
+timeout_cb (gpointer user_data)
+{
+ gst_pad_add_probe (blockpad, GST_PAD_PROBE_TYPE_BLOCK_DOWNSTREAM,
+ pad_probe_cb, user_data, NULL);
+
+ return TRUE;
+}
+
+static gboolean
+bus_cb (GstBus * bus, GstMessage * msg, gpointer user_data)
+{
+ GMainLoop *loop = user_data;
+
+ switch (GST_MESSAGE_TYPE (msg)) {
+ case GST_MESSAGE_ERROR:{
+ GError *err = NULL;
+ gchar *dbg;
+
+ gst_message_parse_error (msg, &err, &dbg);
+ gst_object_default_error (msg->src, err, dbg);
+ g_clear_error (&err);
+ g_free (dbg);
+ g_main_loop_quit (loop);
+ break;
+ }
+ default:
+ break;
+ }
+ return TRUE;
+}
+
+int
+main (int argc, char **argv)
+{
+ GOptionEntry options[] = {
+ {"effects", 'e', 0, G_OPTION_ARG_STRING, &opt_effects,
+ "Effects to use (comma-separated list of element names)", NULL},
+ {NULL}
+ };
+ GOptionContext *ctx;
+ GError *err = NULL;
+ GMainLoop *loop;
+ GstElement *src, *q1, *q2, *effect, *filter1, *filter2, *sink;
+ gchar **effect_names, **e;
+
+ ctx = g_option_context_new ("");
+ g_option_context_add_main_entries (ctx, options, NULL);
+ g_option_context_add_group (ctx, gst_init_get_option_group ());
+ if (!g_option_context_parse (ctx, &argc, &argv, &err)) {
+ g_print ("Error initializing: %s\n", err->message);
+ g_clear_error (&err);
+ g_option_context_free (ctx);
+ return 1;
+ }
+ g_option_context_free (ctx);
+
+ if (opt_effects != NULL)
+ effect_names = g_strsplit (opt_effects, ",", -1);
+ else
+ effect_names = g_strsplit (DEFAULT_EFFECTS, ",", -1);
+
+ for (e = effect_names; e != NULL && *e != NULL; ++e) {
+ GstElement *el;
+
+ el = gst_element_factory_make (*e, NULL);
+ if (el) {
+ g_print ("Adding effect '%s'\n", *e);
+ g_queue_push_tail (&effects, el);
+ }
+ }
+
+ pipeline = gst_pipeline_new ("pipeline");
+
+ src = gst_element_factory_make ("videotestsrc", NULL);
+ g_object_set (src, "is-live", TRUE, NULL);
+
+ filter1 = gst_element_factory_make ("capsfilter", NULL);
+ gst_util_set_object_arg (G_OBJECT (filter1), "caps",
+ "video/x-raw, width=320, height=240, "
+ "format={ I420, YV12, YUY2, UYVY, AYUV, Y41B, Y42B, "
+ "YVYU, Y444, v210, v216, NV12, NV21, UYVP, A420, YUV9, YVU9, IYU1 }");
+
+ q1 = gst_element_factory_make ("queue", NULL);
+
+ blockpad = gst_element_get_static_pad (q1, "src");
+
+ conv_before = gst_element_factory_make ("videoconvert", NULL);
+
+ effect = g_queue_pop_head (&effects);
+ cur_effect = effect;
+
+ conv_after = gst_element_factory_make ("videoconvert", NULL);
+
+ q2 = gst_element_factory_make ("queue", NULL);
+
+ filter2 = gst_element_factory_make ("capsfilter", NULL);
+ gst_util_set_object_arg (G_OBJECT (filter2), "caps",
+ "video/x-raw, width=320, height=240, "
+ "format={ RGBx, BGRx, xRGB, xBGR, RGBA, BGRA, ARGB, ABGR, RGB, BGR }");
+
+ sink = gst_element_factory_make ("ximagesink", NULL);
+
+ gst_bin_add_many (GST_BIN (pipeline), src, filter1, q1, conv_before, effect,
+ conv_after, q2, sink, NULL);
+
+ gst_element_link_many (src, filter1, q1, conv_before, effect, conv_after,
+ q2, sink, NULL);
+
+ gst_element_set_state (pipeline, GST_STATE_PLAYING);
+
+ loop = g_main_loop_new (NULL, FALSE);
+
+ gst_bus_add_watch (GST_ELEMENT_BUS (pipeline), bus_cb, loop);
+
+ g_timeout_add_seconds (1, timeout_cb, loop);
+
+ g_main_loop_run (loop);
+
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+
+ return 0;
+}
+
+
+
+```
+
+Note how we added videoconvert elements before and after the effect.
+This is needed because some elements might operate in different
+colorspaces than other elements. By inserting the conversion elements
+you ensure that the right format can be negotiated at any time.
+
--- /dev/null
+---
+title: Dynamic Controllable Parameters
+...
+
+# Dynamic Controllable Parameters
+
+## Getting Started
+
+The controller subsystem offers a lightweight way to adjust gobject
+properties over stream-time. Normally these properties are changed using
+`g_object_set()`. Timing those calls reliably so that the changes affect
+certain stream times is close to impossible. The controller takes time
+into account. It works by attaching control-sources to properties using
+control-bindings. Control-sources provide values for a given time-stamp
+that are usually in the range of 0.0 to 1.0. Control-bindings map the
+control-value to a gobject property they are bound to - converting the
+type and scaling to the target property value range. At run-time the
+elements continuously pull values changes for the current stream-time to
+update the gobject properties. GStreamer includes a few different
+control-sources and control-bindings already, but applications can
+define their own by sub-classing from the respective base classes.
+
+Most parts of the controller mechanism is implemented in GstObject. Also
+the base classes for control-sources and control-bindings are included
+in the core library. The existing implementations are contained within
+the `gstcontroller` library. You need to include the header in your
+application's source file:
+
+``` c
+...
+#include <gst/gst.h>
+#include <gst/controller/gstinterpolationcontrolsource.h>
+#include <gst/controller/gstdirectcontrolbinding.h>
+...
+
+```
+
+Your application should link to the shared library
+`gstreamer-controller`. One can get the required flag for compiler and
+linker by using pkg-config for gstreamer-controller-1.0.
+
+## Setting up parameter control
+
+If we have our pipeline set up and want to control some parameters, we
+first need to create a control-source. Lets use an interpolation
+control-source:
+
+``` c
+ csource = gst_interpolation_control_source_new ();
+ g_object_set (csource, "mode", GST_INTERPOLATION_MODE_LINEAR, NULL);
+
+```
+
+Now we need to attach the control-source to the gobject property. This
+is done with a control-binding. One control source can be attached to
+several object properties (even in different objects) using separate
+control-bindings.
+
+``` c
+ gst_object_add_control_binding (object, gst_direct_control_binding_new (object, "prop1", csource));
+
+```
+
+This type control-source takes new property values from a list of
+time-stamped parameter changes. The source can e.g. fill gaps by
+smoothing parameter changes This behavior can be configured by setting
+the mode property of the control-source. Other control sources e.g.
+produce a stream of values by calling `sin()` function. They have
+parameters to control e.g. the frequency. As control-sources are
+GstObjects too, one can attach control-sources to these properties too.
+
+Now we can set some control points. These are time-stamped gdouble
+values and are usually in the range of 0.0 to 1.0. A value of 1.0 is
+later mapped to the maximum value in the target properties value range.
+The values become active when the timestamp is reached. They still stay
+in the list. If e.g. the pipeline runs a loop (using a segmented seek),
+the control-curve gets repeated as
+well.
+
+``` c
+ GstTimedValueControlSource *tv_csource = (GstTimedValueControlSource *)csource;
+ gst_timed_value_control_source_set (tv_csource, 0 * GST_SECOND, 0.0);
+ gst_timed_value_control_source_set (tv_csource, 1 * GST_SECOND, 1.0);
+
+```
+
+Now everything is ready to play. If the control-source is e.g. bound to
+a volume property, we will head a fade-in over 1 second. One word of
+caution, the volume element that comes with gstreamer has a value range
+of 0.0 to 4.0 on its volume property. If the above control-source is
+attached to the property the volume will ramp up to 400%\!
+
+One final note - the controller subsystem has a built-in live-mode. Even
+though a property has a control-source assigned one can change the
+GObject property through the `g_object_set()`. This is highly useful
+when binding the GObject properties to GUI widgets. When the user
+adjusts the value with the widget, one can set the GObject property and
+this remains active until the next programmed control-source value
+overrides it. This also works with smoothed parameters. It does not work
+for control-sources that constantly update the property (e.g. the
+lfo\_control\_source).
+
--- /dev/null
+---
+title: Elements
+...
+
+# Elements
+
+The most important object in GStreamer for the application programmer is
+the
+[`GstElement`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstElement.html)
+object. An element is the basic building block for a media pipeline. All
+the different high-level components you will use are derived from
+`GstElement`. Every decoder, encoder, demuxer, video or audio output is
+in fact a `GstElement`
+
+## What are elements?
+
+For the application programmer, elements are best visualized as black
+boxes. On the one end, you might put something in, the element does
+something with it and something else comes out at the other side. For a
+decoder element, for example, you'd put in encoded data, and the element
+would output decoded data. In the next chapter (see [Pads and
+capabilities](manual-pads.md)), you will learn more about data input
+and output in elements, and how you can set that up in your application.
+
+### Source elements
+
+Source elements generate data for use by a pipeline, for example reading
+from disk or from a sound card. [Visualisation of a source
+element](#visualisation-of-a-source-element) shows how we will visualise
+a source element. We always draw a source pad to the right of the
+element.
+
+
+
+Source elements do not accept data, they only generate data. You can see
+this in the figure because it only has a source pad (on the right). A
+source pad can only generate data.
+
+### Filters, convertors, demuxers, muxers and codecs
+
+Filters and filter-like elements have both input and outputs pads. They
+operate on data that they receive on their input (sink) pads, and will
+provide data on their output (source) pads. Examples of such elements
+are a volume element (filter), a video scaler (convertor), an Ogg
+demuxer or a Vorbis decoder.
+
+Filter-like elements can have any number of source or sink pads. A video
+demuxer, for example, would have one sink pad and several (1-N) source
+pads, one for each elementary stream contained in the container format.
+Decoders, on the other hand, will only have one source and sink pads.
+
+
+
+[Visualisation of a filter element](#visualisation-of-a-filter-element)
+shows how we will visualise a filter-like element. This specific element
+has one source and one sink element. Sink pads, receiving input data,
+are depicted at the left of the element; source pads are still on the
+right.
+
+
+
+[Visualisation of a filter element with more than one output
+pad](#visualisation-of-a-filter-element-with----more-than-one-output-pad)
+shows another filter-like element, this one having more than one output
+(source) pad. An example of one such element could, for example, be an
+Ogg demuxer for an Ogg stream containing both audio and video. One
+source pad will contain the elementary video stream, another will
+contain the elementary audio stream. Demuxers will generally fire
+signals when a new pad is created. The application programmer can then
+handle the new elementary stream in the signal handler.
+
+### Sink elements
+
+Sink elements are end points in a media pipeline. They accept data but
+do not produce anything. Disk writing, soundcard playback, and video
+output would all be implemented by sink elements. [Visualisation of a
+sink element](#visualisation-of-a-sink-element) shows a sink element.
+
+
+
+## Creating a `GstElement`
+
+The simplest way to create an element is to use
+[`gst_element_factory_make
+()`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstElementFactory.html#gst-element-factory-make).
+This function takes a factory name and an element name for the newly
+created element. The name of the element is something you can use later
+on to look up the element in a bin, for example. The name will also be
+used in debug output. You can pass `NULL` as the name argument to get a
+unique, default name.
+
+When you don't need the element anymore, you need to unref it using
+[`gst_object_unref
+()`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstObject.html#gst-object-unref).
+This decreases the reference count for the element by 1. An element has
+a refcount of 1 when it gets created. An element gets destroyed
+completely when the refcount is decreased to 0.
+
+The following example \[1\] shows how to create an element named
+*source* from the element factory named *fakesrc*. It checks if the
+creation succeeded. After checking, it unrefs the element.
+
+``` c
+#include <gst/gst.h>
+
+int
+main (int argc,
+ char *argv[])
+{
+ GstElement *element;
+
+ /* init GStreamer */
+ gst_init (&argc, &argv);
+
+ /* create element */
+ element = gst_element_factory_make ("fakesrc", "source");
+ if (!element) {
+ g_print ("Failed to create element of type 'fakesrc'\n");
+ return -1;
+ }
+
+ gst_object_unref (GST_OBJECT (element));
+
+ return 0;
+}
+
+```
+
+`gst_element_factory_make` is actually a shorthand for a combination of
+two functions. A
+[`GstElement`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstElement.html)
+object is created from a factory. To create the element, you have to get
+access to a
+[`GstElementFactory`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstElementFactory.html)
+object using a unique factory name. This is done with
+[`gst_element_factory_find
+()`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstElementFactory.html#gst-element-factory-find).
+
+The following code fragment is used to get a factory that can be used to
+create the *fakesrc* element, a fake data source. The function
+[`gst_element_factory_create
+()`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstElementFactory.html#gst-element-factory-create)
+will use the element factory to create an element with the given name.
+
+``` c
+#include <gst/gst.h>
+
+int
+main (int argc,
+ char *argv[])
+{
+ GstElementFactory *factory;
+ GstElement * element;
+
+ /* init GStreamer */
+ gst_init (&argc, &argv);
+
+ /* create element, method #2 */
+ factory = gst_element_factory_find ("fakesrc");
+ if (!factory) {
+ g_print ("Failed to find factory of type 'fakesrc'\n");
+ return -1;
+ }
+ element = gst_element_factory_create (factory, "source");
+ if (!element) {
+ g_print ("Failed to create element, even though its factory exists!\n");
+ return -1;
+ }
+
+ gst_object_unref (GST_OBJECT (element));
+
+ return 0;
+}
+
+```
+
+## Using an element as a `GObject`
+
+A
+[`GstElement`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstElement.html)
+can have several properties which are implemented using standard
+`GObject` properties. The usual `GObject` methods to query, set and get
+property values and `GParamSpecs` are therefore supported.
+
+Every `GstElement` inherits at least one property from its parent
+`GstObject`: the "name" property. This is the name you provide to the
+functions `gst_element_factory_make ()` or `gst_element_factory_create
+()`. You can get and set this property using the functions
+`gst_object_set_name` and `gst_object_get_name` or use the `GObject`
+property mechanism as shown below.
+
+``` c
+#include <gst/gst.h>
+
+int
+main (int argc,
+ char *argv[])
+{
+ GstElement *element;
+ gchar *name;
+
+ /* init GStreamer */
+ gst_init (&argc, &argv);
+
+ /* create element */
+ element = gst_element_factory_make ("fakesrc", "source");
+
+ /* get name */
+ g_object_get (G_OBJECT (element), "name", &name, NULL);
+ g_print ("The name of the element is '%s'.\n", name);
+ g_free (name);
+
+ gst_object_unref (GST_OBJECT (element));
+
+ return 0;
+}
+
+```
+
+Most plugins provide additional properties to provide more information
+about their configuration or to configure the element. `gst-inspect` is
+a useful tool to query the properties of a particular element, it will
+also use property introspection to give a short explanation about the
+function of the property and about the parameter types and ranges it
+supports. See [gst-inspect](manual-checklist-element.md#gst-inspect) in
+the appendix for details about `gst-inspect`.
+
+For more information about `GObject` properties we recommend you read
+the [GObject
+manual](http://developer.gnome.org/gobject/stable/rn01.html) and an
+introduction to [The Glib Object
+system](http://developer.gnome.org/gobject/stable/pt01.html).
+
+A
+[`GstElement`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstElement.html)
+also provides various `GObject` signals that can be used as a flexible
+callback mechanism. Here, too, you can use `gst-inspect` to see which
+signals a specific element supports. Together, signals and properties
+are the most basic way in which elements and applications interact.
+
+## More about element factories
+
+In the previous section, we briefly introduced the
+[`GstElementFactory`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstElementFactory.html)
+object already as a way to create instances of an element. Element
+factories, however, are much more than just that. Element factories are
+the basic types retrieved from the GStreamer registry, they describe all
+plugins and elements that GStreamer can create. This means that element
+factories are useful for automated element instancing, such as what
+autopluggers do, and for creating lists of available elements.
+
+### Getting information about an element using a factory
+
+Tools like `gst-inspect` will provide some generic information about an
+element, such as the person that wrote the plugin, a descriptive name
+(and a shortname), a rank and a category. The category can be used to
+get the type of the element that can be created using this element
+factory. Examples of categories include `Codec/Decoder/Video` (video
+decoder), `Codec/Encoder/Video` (video encoder), `Source/Video` (a video
+generator), `Sink/Video` (a video output), and all these exist for audio
+as well, of course. Then, there's also `Codec/Demuxer` and `Codec/Muxer`
+and a whole lot more. `gst-inspect` will give a list of all factories,
+and `gst-inspect <factory-name>` will list all of the above information,
+and a lot more.
+
+``` c
+#include <gst/gst.h>
+
+int
+main (int argc,
+ char *argv[])
+{
+ GstElementFactory *factory;
+
+ /* init GStreamer */
+ gst_init (&argc, &argv);
+
+ /* get factory */
+ factory = gst_element_factory_find ("fakesrc");
+ if (!factory) {
+ g_print ("You don't have the 'fakesrc' element installed!\n");
+ return -1;
+ }
+
+ /* display information */
+ g_print ("The '%s' element is a member of the category %s.\n"
+ "Description: %s\n",
+ gst_plugin_feature_get_name (GST_PLUGIN_FEATURE (factory)),
+ gst_element_factory_get_metadata (factory, GST_ELEMENT_METADATA_KLASS),
+ gst_element_factory_get_metadata (factory, GST_ELEMENT_METADATA_DESCRIPTION));
+
+ return 0;
+}
+
+```
+
+You can use `gst_registry_pool_feature_list (GST_TYPE_ELEMENT_FACTORY)`
+to get a list of all the element factories that GStreamer knows about.
+
+### Finding out what pads an element can contain
+
+Perhaps the most powerful feature of element factories is that they
+contain a full description of the pads that the element can generate,
+and the capabilities of those pads (in layman words: what types of media
+can stream over those pads), without actually having to load those
+plugins into memory. This can be used to provide a codec selection list
+for encoders, or it can be used for autoplugging purposes for media
+players. All current GStreamer-based media players and autopluggers work
+this way. We'll look closer at these features as we learn about `GstPad`
+and `GstCaps` in the next chapter: [Pads and
+capabilities](manual-pads.md)
+
+## Linking elements
+
+By linking a source element with zero or more filter-like elements and
+finally a sink element, you set up a media pipeline. Data will flow
+through the elements. This is the basic concept of media handling in
+GStreamer.
+
+![Visualisation of three linked elements](images/linked-elements.png
+"fig:")
+
+By linking these three elements, we have created a very simple chain of
+elements. The effect of this will be that the output of the source
+element (“element1”) will be used as input for the filter-like element
+(“element2”). The filter-like element will do something with the data
+and send the result to the final sink element (“element3”).
+
+Imagine the above graph as a simple Ogg/Vorbis audio decoder. The source
+is a disk source which reads the file from disc. The second element is a
+Ogg/Vorbis audio decoder. The sink element is your soundcard, playing
+back the decoded audio data. We will use this simple graph to construct
+an Ogg/Vorbis player later in this manual.
+
+In code, the above graph is written like this:
+
+``` c
+#include <gst/gst.h>
+
+int
+main (int argc,
+ char *argv[])
+{
+ GstElement *pipeline;
+ GstElement *source, *filter, *sink;
+
+ /* init */
+ gst_init (&argc, &argv);
+
+ /* create pipeline */
+ pipeline = gst_pipeline_new ("my-pipeline");
+
+ /* create elements */
+ source = gst_element_factory_make ("fakesrc", "source");
+ filter = gst_element_factory_make ("identity", "filter");
+ sink = gst_element_factory_make ("fakesink", "sink");
+
+ /* must add elements to pipeline before linking them */
+ gst_bin_add_many (GST_BIN (pipeline), source, filter, sink, NULL);
+
+ /* link */
+ if (!gst_element_link_many (source, filter, sink, NULL)) {
+ g_warning ("Failed to link elements!");
+ }
+
+[..]
+
+}
+
+```
+
+For more specific behaviour, there are also the functions
+`gst_element_link ()` and `gst_element_link_pads ()`. You can also
+obtain references to individual pads and link those using various
+`gst_pad_link_* ()` functions. See the API references for more details.
+
+Important: you must add elements to a bin or pipeline *before* linking
+them, since adding an element to a bin will disconnect any already
+existing links. Also, you cannot directly link elements that are not in
+the same bin or pipeline; if you want to link elements or pads at
+different hierarchy levels, you will need to use ghost pads (more about
+ghost pads later, see [Ghost pads](manual-pads.md#ghost-pads)).
+
+## Element States
+
+After being created, an element will not actually perform any actions
+yet. You need to change elements state to make it do something.
+GStreamer knows four element states, each with a very specific meaning.
+Those four states are:
+
+ - `GST_STATE_NULL`: this is the default state. No resources are
+ allocated in this state, so, transitioning to it will free all
+ resources. The element must be in this state when its refcount
+ reaches 0 and it is freed.
+
+ - `GST_STATE_READY`: in the ready state, an element has allocated all
+ of its global resources, that is, resources that can be kept within
+ streams. You can think about opening devices, allocating buffers and
+ so on. However, the stream is not opened in this state, so the
+ stream positions is automatically zero. If a stream was previously
+ opened, it should be closed in this state, and position, properties
+ and such should be reset.
+
+ - `GST_STATE_PAUSED`: in this state, an element has opened the stream,
+ but is not actively processing it. An element is allowed to modify a
+ stream's position, read and process data and such to prepare for
+ playback as soon as state is changed to PLAYING, but it is *not*
+ allowed to play the data which would make the clock run. In summary,
+ PAUSED is the same as PLAYING but without a running clock.
+
+ Elements going into the PAUSED state should prepare themselves for
+ moving over to the PLAYING state as soon as possible. Video or audio
+ outputs would, for example, wait for data to arrive and queue it so
+ they can play it right after the state change. Also, video sinks can
+ already play the first frame (since this does not affect the clock
+ yet). Autopluggers could use this same state transition to already
+ plug together a pipeline. Most other elements, such as codecs or
+ filters, do not need to explicitly do anything in this state,
+ however.
+
+ - `GST_STATE_PLAYING`: in the PLAYING state, an element does exactly
+ the same as in the PAUSED state, except that the clock now runs.
+
+You can change the state of an element using the function
+`gst_element_set_state ()`. If you set an element to another state,
+GStreamer will internally traverse all intermediate states. So if you
+set an element from NULL to PLAYING, GStreamer will internally set the
+element to READY and PAUSED in between.
+
+When moved to `GST_STATE_PLAYING`, pipelines will process data
+automatically. They do not need to be iterated in any form. Internally,
+GStreamer will start threads that take this task on to them. GStreamer
+will also take care of switching messages from the pipeline's thread
+into the application's own thread, by using a
+[`GstBus`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstBus.html).
+See [Bus](manual-bus.md) for details.
+
+When you set a bin or pipeline to a certain target state, it will
+usually propagate the state change to all elements within the bin or
+pipeline automatically, so it's usually only necessary to set the state
+of the top-level pipeline to start up the pipeline or shut it down.
+However, when adding elements dynamically to an already-running
+pipeline, e.g. from within a "pad-added" signal callback, you need to
+set it to the desired target state yourself using `gst_element_set_state
+()` or `gst_element_sync_state_with_parent ()`.
+
+1. The code for this example is automatically extracted from the
+ documentation and built under `tests/examples/manual` in the
+ GStreamer tarball.
+
--- /dev/null
+---
+title: What is GStreamer?
+...
+
+# What is GStreamer?
+
+GStreamer is a framework for creating streaming media applications. The
+fundamental design comes from the video pipeline at Oregon Graduate
+Institute, as well as some ideas from DirectShow.
+
+GStreamer's development framework makes it possible to write any type of
+streaming multimedia application. The GStreamer framework is designed to
+make it easy to write applications that handle audio or video or both.
+It isn't restricted to audio and video, and can process any kind of data
+flow. The pipeline design is made to have little overhead above what the
+applied filters induce. This makes GStreamer a good framework for
+designing even high-end audio applications which put high demands on
+latency.
+
+One of the most obvious uses of GStreamer is using it to build a media
+player. GStreamer already includes components for building a media
+player that can support a very wide variety of formats, including MP3,
+Ogg/Vorbis, MPEG-1/2, AVI, Quicktime, mod, and more. GStreamer, however,
+is much more than just another media player. Its main advantages are
+that the pluggable components can be mixed and matched into arbitrary
+pipelines so that it's possible to write a full-fledged video or audio
+editing application.
+
+The framework is based on plugins that will provide the various codec
+and other functionality. The plugins can be linked and arranged in a
+pipeline. This pipeline defines the flow of the data. Pipelines can also
+be edited with a GUI editor and saved as XML so that pipeline libraries
+can be made with a minimum of effort.
+
+The GStreamer core function is to provide a framework for plugins, data
+flow and media type handling/negotiation. It also provides an API to
+write applications using the various plugins.
+
+Specifically, GStreamer provides
+
+ - an API for multimedia applications
+
+ - a plugin architecture
+
+ - a pipeline architecture
+
+ - a mechanism for media type handling/negotiation
+
+ - a mechanism for synchronization
+
+ - over 250 plug-ins providing more than 1000 elements
+
+ - a set of tools
+
+GStreamer plug-ins could be classified into
+
+ - protocols handling
+
+ - sources: for audio and video (involves protocol plugins)
+
+ - formats: parsers, formaters, muxers, demuxers, metadata, subtitles
+
+ - codecs: coders and decoders
+
+ - filters: converters, mixers, effects, ...
+
+ - sinks: for audio and video (involves protocol plugins)
+
+
+
+GStreamer is packaged into
+
+ - gstreamer: the core package
+
+ - gst-plugins-base: an essential exemplary set of elements
+
+ - gst-plugins-good: a set of good-quality plug-ins under LGPL
+
+ - gst-plugins-ugly: a set of good-quality plug-ins that might pose
+ distribution problems
+
+ - gst-plugins-bad: a set of plug-ins that need more quality
+
+ - gst-libav: a set of plug-ins that wrap libav for decoding and
+ encoding
+
+ - a few others packages
+
--- /dev/null
+---
+title: Your first application
+...
+
+# Your first application
+
+This chapter will summarize everything you've learned in the previous
+chapters. It describes all aspects of a simple GStreamer application,
+including initializing libraries, creating elements, packing elements
+together in a pipeline and playing this pipeline. By doing all this, you
+will be able to build a simple Ogg/Vorbis audio player.
+
+## Hello world
+
+We're going to create a simple first application, a simple Ogg/Vorbis
+command-line audio player. For this, we will use only standard GStreamer
+components. The player will read a file specified on the command-line.
+Let's get started\!
+
+We've learned, in [Initializing GStreamer](manual-init.md), that the
+first thing to do in your application is to initialize GStreamer by
+calling `gst_init ()`. Also, make sure that the application includes
+`gst/gst.h` so all function names and objects are properly defined. Use
+`#include
+<gst/gst.h>` to do that.
+
+Next, you'll want to create the different elements using
+`gst_element_factory_make ()`. For an Ogg/Vorbis audio player, we'll
+need a source element that reads files from a disk. GStreamer includes
+this element under the name “filesrc”. Next, we'll need something to
+parse the file and decode it into raw audio. GStreamer has two elements
+for this: the first parses Ogg streams into elementary streams (video,
+audio) and is called “oggdemux”. The second is a Vorbis audio decoder,
+it's conveniently called “vorbisdec”. Since “oggdemux” creates dynamic
+pads for each elementary stream, you'll need to set a “pad-added” event
+handler on the “oggdemux” element, like you've learned in [Dynamic (or
+sometimes) pads](manual-pads.md#dynamic-or-sometimes-pads), to link the
+Ogg demuxer and the Vorbis decoder elements together. At last, we'll
+also need an audio output element, we will use “autoaudiosink”, which
+automatically detects your audio device.
+
+The last thing left to do is to add all elements into a container
+element, a `GstPipeline`, and wait until we've played the whole song.
+We've previously learned how to add elements to a container bin in
+[Bins](manual-bins.md), and we've learned about element states in
+[Element States](manual-elements.md#element-states). We will also
+attach a message handler to the pipeline bus so we can retrieve errors
+and detect the end-of-stream.
+
+Let's now add all the code together to get our very first audio player:
+
+``` c
+
+#include <gst/gst.h>
+#include <glib.h>
+
+
+static gboolean
+bus_call (GstBus *bus,
+ GstMessage *msg,
+ gpointer data)
+{
+ GMainLoop *loop = (GMainLoop *) data;
+
+ switch (GST_MESSAGE_TYPE (msg)) {
+
+ case GST_MESSAGE_EOS:
+ g_print ("End of stream\n");
+ g_main_loop_quit (loop);
+ break;
+
+ case GST_MESSAGE_ERROR: {
+ gchar *debug;
+ GError *error;
+
+ gst_message_parse_error (msg, &error, &debug);
+ g_free (debug);
+
+ g_printerr ("Error: %s\n", error->message);
+ g_error_free (error);
+
+ g_main_loop_quit (loop);
+ break;
+ }
+ default:
+ break;
+ }
+
+ return TRUE;
+}
+
+
+static void
+on_pad_added (GstElement *element,
+ GstPad *pad,
+ gpointer data)
+{
+ GstPad *sinkpad;
+ GstElement *decoder = (GstElement *) data;
+
+ /* We can now link this pad with the vorbis-decoder sink pad */
+ g_print ("Dynamic pad created, linking demuxer/decoder\n");
+
+ sinkpad = gst_element_get_static_pad (decoder, "sink");
+
+ gst_pad_link (pad, sinkpad);
+
+ gst_object_unref (sinkpad);
+}
+
+
+
+int
+main (int argc,
+ char *argv[])
+{
+ GMainLoop *loop;
+
+ GstElement *pipeline, *source, *demuxer, *decoder, *conv, *sink;
+ GstBus *bus;
+ guint bus_watch_id;
+
+ /* Initialisation */
+ gst_init (&argc, &argv);
+
+ loop = g_main_loop_new (NULL, FALSE);
+
+
+ /* Check input arguments */
+ if (argc != 2) {
+ g_printerr ("Usage: %s <Ogg/Vorbis filename>\n", argv[0]);
+ return -1;
+ }
+
+
+ /* Create gstreamer elements */
+ pipeline = gst_pipeline_new ("audio-player");
+ source = gst_element_factory_make ("filesrc", "file-source");
+ demuxer = gst_element_factory_make ("oggdemux", "ogg-demuxer");
+ decoder = gst_element_factory_make ("vorbisdec", "vorbis-decoder");
+ conv = gst_element_factory_make ("audioconvert", "converter");
+ sink = gst_element_factory_make ("autoaudiosink", "audio-output");
+
+ if (!pipeline || !source || !demuxer || !decoder || !conv || !sink) {
+ g_printerr ("One element could not be created. Exiting.\n");
+ return -1;
+ }
+
+ /* Set up the pipeline */
+
+ /* we set the input filename to the source element */
+ g_object_set (G_OBJECT (source), "location", argv[1], NULL);
+
+ /* we add a message handler */
+ bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
+ bus_watch_id = gst_bus_add_watch (bus, bus_call, loop);
+ gst_object_unref (bus);
+
+ /* we add all elements into the pipeline */
+ /* file-source | ogg-demuxer | vorbis-decoder | converter | alsa-output */
+ gst_bin_add_many (GST_BIN (pipeline),
+ source, demuxer, decoder, conv, sink, NULL);
+
+ /* we link the elements together */
+ /* file-source -> ogg-demuxer ~> vorbis-decoder -> converter -> alsa-output */
+ gst_element_link (source, demuxer);
+ gst_element_link_many (decoder, conv, sink, NULL);
+ g_signal_connect (demuxer, "pad-added", G_CALLBACK (on_pad_added), decoder);
+
+ /* note that the demuxer will be linked to the decoder dynamically.
+ The reason is that Ogg may contain various streams (for example
+ audio and video). The source pad(s) will be created at run time,
+ by the demuxer when it detects the amount and nature of streams.
+ Therefore we connect a callback function which will be executed
+ when the "pad-added" is emitted.*/
+
+
+ /* Set the pipeline to "playing" state*/
+ g_print ("Now playing: %s\n", argv[1]);
+ gst_element_set_state (pipeline, GST_STATE_PLAYING);
+
+
+ /* Iterate */
+ g_print ("Running...\n");
+ g_main_loop_run (loop);
+
+
+ /* Out of the main loop, clean up nicely */
+ g_print ("Returned, stopping playback\n");
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+
+ g_print ("Deleting pipeline\n");
+ gst_object_unref (GST_OBJECT (pipeline));
+ g_source_remove (bus_watch_id);
+ g_main_loop_unref (loop);
+
+ return 0;
+}
+
+
+```
+
+We now have created a complete pipeline. We can visualise the pipeline
+as follows:
+
+
+
+## Compiling and Running helloworld.c
+
+To compile the helloworld example, use: `gcc -Wall
+helloworld.c -o helloworld
+$(pkg-config --cflags --libs gstreamer-1.0)`. GStreamer makes use of
+`pkg-config` to get compiler and linker flags needed to compile this
+application.
+
+If you're running a non-standard installation (ie. you've installed
+GStreamer from source yourself instead of using pre-built packages),
+make sure the `PKG_CONFIG_PATH` environment variable is set to the
+correct location (`$libdir/pkgconfig`).
+
+In the unlikely case that you are using an uninstalled GStreamer setup
+(ie. gst-uninstalled), you will need to use libtool to build the hello
+world program, like this: `libtool --mode=link gcc -Wall
+helloworld.c -o helloworld
+$(pkg-config --cflags --libs gstreamer-1.0)`.
+
+You can run this example application with `./helloworld
+file.ogg`. Substitute `file.ogg` with your favourite Ogg/Vorbis file.
+
+## Conclusion
+
+This concludes our first example. As you see, setting up a pipeline is
+very low-level but powerful. You will see later in this manual how you
+can create a more powerful media player with even less effort using
+higher-level interfaces. We will discuss all that in [Higher-level
+interfaces for GStreamer applications](manual-highlevel.md). We will
+first, however, go more in-depth into more advanced GStreamer internals.
+
+It should be clear from the example that we can very easily replace the
+“filesrc” element with some other element that reads data from a
+network, or some other data source element that is better integrated
+with your desktop environment. Also, you can use other decoders and
+parsers/demuxers to support other media types. You can use another audio
+sink if you're not running Linux, but Mac OS X, Windows or FreeBSD, or
+you can instead use a filesink to write audio files to disk instead of
+playing them back. By using an audio card source, you can even do audio
+capture instead of playback. All this shows the reusability of GStreamer
+elements, which is its greatest advantage.
+
--- /dev/null
+---
+title: Higher-level interfaces for GStreamer applications
+...
+
+# Higher-level interfaces for GStreamer applications
+
+In the previous two parts, you have learned many of the internals and
+their corresponding low-level interfaces into GStreamer application
+programming. Many people will, however, not need so much control (and as
+much code), but will prefer to use a standard playback interface that
+does most of the difficult internals for them. In this chapter, we will
+introduce you into the concept of autopluggers, playback managing
+elements and other such things. Those higher-level interfaces are
+intended to simplify GStreamer-based application programming. They do,
+however, also reduce the flexibility. It is up to the application
+developer to choose which interface he will want to use.
+
--- /dev/null
+---
+title: Application Development Manual
+short-description: Complete walkthrough for building an application using GStreamer
+...
+
+# Application Development Manual
+
+## Foreword
+
+GStreamer is an extremely powerful and versatile framework for creating
+streaming media applications. Many of the virtues of the GStreamer
+framework come from its modularity: GStreamer can seamlessly incorporate
+new plugin modules. But because modularity and power often come at a
+cost of greater complexity, writing new applications is not always easy.
+
+This guide is intended to help you understand the GStreamer framework
+so you can develop applications based on it. The first
+chapters will focus on development of a simple audio player, with much
+effort going into helping you understand GStreamer concepts. Later
+chapters will go into more advanced topics related to media playback,
+but also at other forms of media processing (capture, editing, etc.).
+
+## Introduction
+
+### Who should read this manual?
+
+This book is about GStreamer from an application developer's point of
+view; it describes how to write a GStreamer application using the
+GStreamer libraries and tools. For an explanation about writing plugins,
+we suggest the [Plugin Writers Guide](pwg-index.md).
+
+Also check out the other documentation available on the [GStreamer web
+site](http://gstreamer.freedesktop.org/documentation/).
+
+### Preliminary reading
+
+In order to understand this manual, you need to have a basic
+understanding of the *C language*.
+
+Since GStreamer adheres to the GObject programming model, this guide
+also assumes that you understand the basics of
+[GObject](http://library.gnome.org/devel/gobject/stable/) and
+[glib](http://library.gnome.org/devel/glib/stable/) programming.
+Especially,
+
+ - GObject instantiation
+
+ - GObject properties (set/get)
+
+ - GObject casting
+
+ - GObject referecing/dereferencing
+
+ - glib memory management
+
+ - glib signals and callbacks
+
+ - glib main loop
+
+### Structure of this manual
+
+To help you navigate through this guide, it is divided into several
+large parts. Each part addresses a particular broad topic concerning
+GStreamer appliction development. The parts of this guide are laid out
+in the following order:
+
+[About GStreamer](manual-introduction.md) gives you an overview of
+GStreamer, it's design principles and foundations.
+
+[Building an Application](manual-building.md) covers the basics of
+GStreamer application programming. At the end of this part, you should
+be able to build your own audio player using GStreamer
+
+In [Advanced GStreamer concepts](manual-advanced.md), we will move on to
+advanced subjects which make GStreamer stand out of its competitors. We
+will discuss application-pipeline interaction using dynamic parameters
+and interfaces, we will discuss threading and threaded pipelines,
+scheduling and clocks (and synchronization). Most of those topics are
+not just there to introduce you to their API, but primarily to give a
+deeper insight in solving application programming problems with
+GStreamer and understanding their concepts.
+
+Next, in [Higher-level interfaces for GStreamer
+applications](manual-highlevel.md), we will go into higher-level
+programming APIs for GStreamer. You don't exactly need to know all the
+details from the previous parts to understand this, but you will need to
+understand basic GStreamer concepts nevertheless. We will, amongst
+others, discuss XML, playbin and autopluggers.
+
+Finally in [Appendices](manual-appendices.md), you will find some random
+information on integrating with GNOME, KDE, OS X or Windows, some
+debugging help and general tips to improve and simplify GStreamer
+programming.
+
--- /dev/null
+---
+title: Initializing GStreamer
+...
+
+# Initializing GStreamer
+
+When writing a GStreamer application, you can simply include `gst/gst.h`
+to get access to the library functions. Besides that, you will also need
+to initialize the GStreamer library.
+
+## Simple initialization
+
+Before the GStreamer libraries can be used, `gst_init` has to be called
+from the main application. This call will perform the necessary
+initialization of the library as well as parse the GStreamer-specific
+command line options.
+
+A typical program \[1\] would have code to initialize GStreamer that
+looks like this:
+
+``` c
+
+#include <stdio.h>
+#include <gst/gst.h>
+
+int
+main (int argc,
+ char *argv[])
+{
+ const gchar *nano_str;
+ guint major, minor, micro, nano;
+
+ gst_init (&argc, &argv);
+
+ gst_version (&major, &minor, µ, &nano);
+
+ if (nano == 1)
+ nano_str = "(CVS)";
+ else if (nano == 2)
+ nano_str = "(Prerelease)";
+ else
+ nano_str = "";
+
+ printf ("This program is linked against GStreamer %d.%d.%d %s\n",
+ major, minor, micro, nano_str);
+
+ return 0;
+}
+
+
+```
+
+Use the `GST_VERSION_MAJOR`, `GST_VERSION_MINOR` and `GST_VERSION_MICRO`
+macros to get the GStreamer version you are building against, or use the
+function `gst_version` to get the version your application is linked
+against. GStreamer currently uses a scheme where versions with the same
+major and minor versions are API-/ and ABI-compatible.
+
+It is also possible to call the `gst_init` function with two `NULL`
+arguments, in which case no command line options will be parsed by
+GStreamer.
+
+## The GOption interface
+
+You can also use a GOption table to initialize your own parameters as
+shown in the next example:
+
+``` c
+
+#include <gst/gst.h>
+
+int
+main (int argc,
+ char *argv[])
+{
+ gboolean silent = FALSE;
+ gchar *savefile = NULL;
+ GOptionContext *ctx;
+ GError *err = NULL;
+ GOptionEntry entries[] = {
+ { "silent", 's', 0, G_OPTION_ARG_NONE, &silent,
+ "do not output status information", NULL },
+ { "output", 'o', 0, G_OPTION_ARG_STRING, &savefile,
+ "save xml representation of pipeline to FILE and exit", "FILE" },
+ { NULL }
+ };
+
+ ctx = g_option_context_new ("- Your application");
+ g_option_context_add_main_entries (ctx, entries, NULL);
+ g_option_context_add_group (ctx, gst_init_get_option_group ());
+ if (!g_option_context_parse (ctx, &argc, &argv, &err)) {
+ g_print ("Failed to initialize: %s\n", err->message);
+ g_clear_error (&err);
+ g_option_context_free (ctx);
+ return 1;
+ }
+ g_option_context_free (ctx);
+
+ printf ("Run me with --help to see the Application options appended.\n");
+
+ return 0;
+}
+
+
+```
+
+As shown in this fragment, you can use a
+[GOption](http://developer.gnome.org/glib/stable/glib-Commandline-option-parser.html)
+table to define your application-specific command line options, and pass
+this table to the GLib initialization function along with the option
+group returned from the function `gst_init_get_option_group`. Your
+application options will be parsed in addition to the standard GStreamer
+options.
+
+1. The code for this example is automatically extracted from the
+ documentation and built under `tests/examples/manual` in the
+ GStreamer tarball.
+
--- /dev/null
+---
+title: Integration
+...
+
+# Integration
+
+GStreamer tries to integrate closely with operating systems (such as
+Linux and UNIX-like operating systems, OS X or Windows) and desktop
+environments (such as GNOME or KDE). In this chapter, we'll mention some
+specific techniques to integrate your application with your operating
+system or desktop environment of choice.
+
+## Linux and UNIX-like operating systems
+
+GStreamer provides a basic set of elements that are useful when
+integrating with Linux or a UNIX-like operating system.
+
+ - For audio input and output, GStreamer provides input and output
+ elements for several audio subsystems. Amongst others, GStreamer
+ includes elements for ALSA (alsasrc, alsasink), OSS (osssrc,
+ osssink) Pulesaudio (pulsesrc, pulsesink) and Sun audio
+ (sunaudiosrc, sunaudiomixer, sunaudiosink).
+
+ - For video input, GStreamer contains source elements for Video4linux2
+ (v4l2src, v4l2element, v4l2sink).
+
+ - For video output, GStreamer provides elements for output to
+ X-windows (ximagesink), Xv-windows (xvimagesink; for
+ hardware-accelerated video), direct-framebuffer (dfbimagesink) and
+ openGL image contexts (glsink).
+
+## GNOME desktop
+
+GStreamer has been the media backend of the
+[GNOME](http://www.gnome.org/) desktop since GNOME-2.2 onwards.
+Nowadays, a whole bunch of GNOME applications make use of GStreamer for
+media-processing, including (but not limited to)
+[Rhythmbox](http://www.rhythmbox.org/),
+[Videos](https://wiki.gnome.org/Apps/Videos) and [Sound
+Juicer](https://wiki.gnome.org/Apps/SoundJuicer).
+
+Most of these GNOME applications make use of some specific techniques to
+integrate as closely as possible with the GNOME desktop:
+
+ - GNOME applications usually call `gtk_init ()` to parse command-line
+ options and initialize GTK. GStreamer applications would normally
+ call `gst_init ()` to do the same for GStreamer. This would mean
+ that only one of the two can parse command-line options. To work
+ around this issue, GStreamer can provide a GLib `GOptionGroup` which
+ can be passed to `gnome_program_init ()`. The following example
+ requires GTK 2.6 or newer (previous GTK versions do not support
+ command line parsing via GOption yet)
+
+ ``` c
+ #include <gtk/gtk.h>
+ #include <gst/gst.h>
+
+ static gchar **cmd_filenames = NULL;
+
+ static GOptionEntries cmd_options[] = {
+ /* here you can add command line options for your application. Check
+ * the GOption section in the GLib API reference for a more elaborate
+ * example of how to add your own command line options here */
+
+ /* at the end we have a special option that collects all remaining
+ * command line arguments (like filenames) for us. If you don't
+ * need this, you can safely remove it */
+ { G_OPTION_REMAINING, 0, 0, G_OPTION_ARG_FILENAME_ARRAY, &cmd_filenames,
+ "Special option that collects any remaining arguments for us" },
+
+ /* mark the end of the options array with a NULL option */
+ { NULL, }
+ };
+
+ /* this should usually be defined in your config.h */
+ #define VERSION "0.0.1"
+
+ gint
+ main (gint argc, gchar **argv)
+ {
+ GOptionContext *context;
+ GOptionGroup *gstreamer_group, *gtk_group;
+ GError *err = NULL;
+
+ context = g_option_context_new ("gtk-demo-app");
+
+ /* get command line options from GStreamer and add them to the group */
+ gstreamer_group = gst_init_get_option_group ();
+ g_option_context_add_group (context, gstreamer_group);
+ gtk_group = gtk_get_option_group (TRUE);
+ g_option_context_add_group (context, gtk_group);
+
+ /* add our own options. If you are using gettext for translation of your
+ * strings, use GETTEXT_PACKAGE here instead of NULL */
+ g_option_context_add_main_entries (context, cmd_options, NULL);
+
+ /* now parse the commandline options, note that this already
+ * calls gtk_init() and gst_init() */
+ if (!g_option_context_parse (ctx, &argc, &argv, &err)) {
+ g_print ("Error initializing: %s\n", err->message);
+ g_clear_error (&err);
+ g_option_context_free (ctx);
+ exit (1);
+ }
+ g_option_context_free (ctx);
+
+ /* any filenames we got passed on the command line? parse them! */
+ if (cmd_filenames != NULL) {
+ guint i, num;
+
+ num = g_strv_length (cmd_filenames);
+ for (i = 0; i < num; ++i) {
+ /* do something with the filename ... */
+ g_print ("Adding to play queue: %s\n", cmd_filenames[i]);
+ }
+
+ g_strfreev (cmd_filenames);
+ cmd_filenames = NULL;
+ }
+
+ [..]
+
+ }
+ ```
+
+ - GNOME uses Pulseaudio for audio, use the pulsesrc and pulsesink
+ elements to have access to all the features.
+
+ - GStreamer provides data input/output elements for use with the GIO
+ VFS system. These elements are called “giosrc” and “giosink”. The
+ deprecated GNOME-VFS system is supported too but shouldn't be used
+ for any new applications.
+
+## KDE desktop
+
+GStreamer has been proposed for inclusion in KDE-4.0. Currently,
+GStreamer is included as an optional component, and it's used by several
+KDE applications, including [AmaroK](http://amarok.kde.org/),
+[KMPlayer](http://www.xs4all.nl/~jjvrieze/kmplayer.html) and
+[Kaffeine](http://kaffeine.sourceforge.net/).
+
+Although not yet as complete as the GNOME integration bits, there are
+already some KDE integration specifics available. This list will
+probably grow as GStreamer starts to be used in KDE-4.0:
+
+ - AmaroK contains a kiosrc element, which is a source element that
+ integrates with the KDE VFS subsystem KIO.
+
+## OS X
+
+GStreamer provides native video and audio output elements for OS X. It
+builds using the standard development tools for OS X.
+
+## Windows
+
+> **Warning**
+>
+> Note: this section is out of date. GStreamer-1.0 has much better
+> support for win32 than previous versions though and should usually
+> compile and work out-of-the-box both using MSYS/MinGW or Microsoft
+> compilers. The [GStreamer web site](http://gstreamer.freedesktop.org)
+> and the [mailing list
+> archives](http://news.gmane.org/gmane.comp.video.gstreamer.devel) are
+> a good place to check the latest win32-related news.
+
+GStreamer builds using Microsoft Visual C .NET 2003 and using Cygwin.
+
+### Building GStreamer under Win32
+
+There are different makefiles that can be used to build GStreamer with
+the usual Microsoft compiling tools.
+
+The Makefile is meant to be used with the GNU make program and the free
+version of the Microsoft compiler
+(<http://msdn.microsoft.com/visualc/vctoolkit2003/>). You also have to
+modify your system environment variables to use it from the
+command-line. You will also need a working Platform SDK for Windows that
+is available for free from Microsoft.
+
+The projects/makefiles will generate automatically some source files
+needed to compile GStreamer. That requires that you have installed on
+your system some GNU tools and that they are available in your system
+PATH.
+
+The GStreamer project depends on other libraries, namely :
+
+ - GLib
+
+ - libxml2
+
+ - libintl
+
+ - libiconv
+
+Work is being done to provide pre-compiled GStreamer-1.0 libraries as a
+packages for win32. Check the [GStreamer web
+site](http://gstreamer.freedesktop.org) and check our [mailing
+list](http://news.gmane.org/gmane.comp.video.gstreamer.devel) for the
+latest developments in this respect.
+
+> **Note**
+>
+> GNU tools needed that you can find on
+> <http://gnuwin32.sourceforge.net/>
+>
+> - GNU flex (tested with 2.5.4)
+>
+> - GNU bison (tested with 1.35)
+>
+> and <http://www.mingw.org/>
+>
+> - GNU make (tested with 3.80)
+>
+> the generated files from the -auto makefiles will be available soon
+> separately on the net for convenience (people who don't want to
+> install GNU tools).
+
+### Installation on the system
+
+FIXME: This section needs be updated for GStreamer-1.0.
+
--- /dev/null
+---
+title: Interfaces
+...
+
+# Interfaces
+
+In [Using an element as a
+GObject](manual-elements.md#using-an-element-as-a-gobject), you have
+learned how to use `GObject` properties as a simple way to do
+interaction between applications and elements. This method suffices for
+the simple'n'straight settings, but fails for anything more complicated
+than a getter and setter. For the more complicated use cases, GStreamer
+uses interfaces based on the GObject
+[`GTypeInterface`](http://library.gnome.org/devel/gobject/stable/gtype-non-instantiable-classed.html)
+type.
+
+Most of the interfaces handled here will not contain any example code.
+See the API references for details. Here, we will just describe the
+scope and purpose of each interface.
+
+## The URI interface
+
+In all examples so far, we have only supported local files through the
+“filesrc” element. GStreamer, obviously, supports many more location
+sources. However, we don't want applications to need to know any
+particular element implementation details, such as element names for
+particular network source types and so on. Therefore, there is a URI
+interface, which can be used to get the source element that supports a
+particular URI type. There is no strict rule for URI naming, but in
+general we follow naming conventions that others use, too. For example,
+assuming you have the correct plugins installed, GStreamer supports
+“file:///\<path\>/\<file\>”, “http://\<host\>/\<path\>/\<file\>”,
+“mms://\<host\>/\<path\>/\<file\>”, and so on.
+
+In order to get the source or sink element supporting a particular URI,
+use `gst_element_make_from_uri ()`, with the URI type being either
+`GST_URI_SRC` for a source element, or `GST_URI_SINK` for a sink
+element.
+
+You can convert filenames to and from URIs using GLib's
+`g_filename_to_uri ()` and `g_uri_to_filename ()`.
+
+## The Color Balance interface
+
+The colorbalance interface is a way to control video-related properties
+on an element, such as brightness, contrast and so on. It's sole reason
+for existence is that, as far as its authors know, there's no way to
+dynamically register properties using `GObject`.
+
+The colorbalance interface is implemented by several plugins, including
+xvimagesink and the Video4linux2 elements.
+
+## The Video Overlay interface
+
+The Video Overlay interface was created to solve the problem of
+embedding video streams in an application window. The application
+provides an window handle to the element implementing this interface to
+draw on, and the element will then use this window handle to draw on
+rather than creating a new toplevel window. This is useful to embed
+video in video players.
+
+This interface is implemented by, amongst others, the Video4linux2
+elements and by ximagesink, xvimagesink and sdlvideosink.
+
--- /dev/null
+---
+title: Foundations
+...
+
+# Foundations
+
+This chapter of the guide introduces the basic concepts of GStreamer.
+Understanding these concepts will be important in reading any of the
+rest of this guide, all of them assume understanding of these basic
+concepts.
+
+## Elements
+
+An *element* is the most important class of objects in GStreamer. You
+will usually create a chain of elements linked together and let data
+flow through this chain of elements. An element has one specific
+function, which can be the reading of data from a file, decoding of this
+data or outputting this data to your sound card (or anything else). By
+chaining together several such elements, you create a *pipeline* that
+can do a specific task, for example media playback or capture. GStreamer
+ships with a large collection of elements by default, making the
+development of a large variety of media applications possible. If
+needed, you can also write new elements. That topic is explained in
+great deal in the *GStreamer Plugin Writer's Guide*.
+
+## Pads
+
+*Pads* are element's input and output, where you can connect other
+elements. They are used to negotiate links and data flow between
+elements in GStreamer. A pad can be viewed as a “plug” or “port” on an
+element where links may be made with other elements, and through which
+data can flow to or from those elements. Pads have specific data
+handling capabilities: a pad can restrict the type of data that flows
+through it. Links are only allowed between two pads when the allowed
+data types of the two pads are compatible. Data types are negotiated
+between pads using a process called *caps negotiation*. Data types are
+described as a `GstCaps`.
+
+An analogy may be helpful here. A pad is similar to a plug or jack on a
+physical device. Consider, for example, a home theater system consisting
+of an amplifier, a DVD player, and a (silent) video projector. Linking
+the DVD player to the amplifier is allowed because both devices have
+audio jacks, and linking the projector to the DVD player is allowed
+because both devices have compatible video jacks. Links between the
+projector and the amplifier may not be made because the projector and
+amplifier have different types of jacks. Pads in GStreamer serve the
+same purpose as the jacks in the home theater system.
+
+For the most part, all data in GStreamer flows one way through a link
+between elements. Data flows out of one element through one or more
+*source pads*, and elements accept incoming data through one or more
+*sink pads*. Source and sink elements have only source and sink pads,
+respectively. Data usually means buffers (described by the
+[`GstBuffer`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/gstreamer-GstBuffer.html)
+object) and events (described by the
+[`GstEvent`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/gstreamer-GstEvent.html)
+object).
+
+## Bins and pipelines
+
+A *bin* is a container for a collection of elements. Since bins are
+subclasses of elements themselves, you can mostly control a bin as if it
+were an element, thereby abstracting away a lot of complexity for your
+application. You can, for example change state on all elements in a bin
+by changing the state of that bin itself. Bins also forward bus messages
+from their contained children (such as error messages, tag messages or
+EOS messages).
+
+A *pipeline* is a top-level bin. It provides a bus for the application
+and manages the synchronization for its children. As you set it to
+PAUSED or PLAYING state, data flow will start and media processing will
+take place. Once started, pipelines will run in a separate thread until
+you stop them or the end of the data stream is reached.
+
+![GStreamer pipeline for a simple ogg player](images/simple-player.png
+"fig:")
+
+## Communication
+
+GStreamer provides several mechanisms for communication and data
+exchange between the *application* and the *pipeline*.
+
+ - *buffers* are objects for passing streaming data between elements in
+ the pipeline. Buffers always travel from sources to sinks
+ (downstream).
+
+ - *events* are objects sent between elements or from the application
+ to elements. Events can travel upstream and downstream. Downstream
+ events can be synchronised to the data flow.
+
+ - *messages* are objects posted by elements on the pipeline's message
+ bus, where they will be held for collection by the application.
+ Messages can be intercepted synchronously from the streaming thread
+ context of the element posting the message, but are usually handled
+ asynchronously by the application from the application's main
+ thread. Messages are used to transmit information such as errors,
+ tags, state changes, buffering state, redirects etc. from elements
+ to the application in a thread-safe way.
+
+ - *queries* allow applications to request information such as duration
+ or current playback position from the pipeline. Queries are always
+ answered synchronously. Elements can also use queries to request
+ information from their peer elements (such as the file size or
+ duration). They can be used both ways within a pipeline, but
+ upstream queries are more common.
+
+
+
--- /dev/null
+---
+title: About GStreamer
+...
+
+# About GStreamer
+
+This part gives you an overview of the technologies described in this
+book.
+
--- /dev/null
+---
+title: Licensing advisory
+...
+
+# Licensing advisory
+
+## How to license the applications you build with GStreamer
+
+The licensing of GStreamer is no different from a lot of other libraries
+out there like GTK+ or glibc: we use the LGPL. What complicates things
+with regards to GStreamer is its plugin-based design and the heavily
+patented and proprietary nature of many multimedia codecs. While patents
+on software are currently only allowed in a small minority of world
+countries (the US and Australia being the most important of those), the
+problem is that due to the central place the US hold in the world
+economy and the computing industry, software patents are hard to ignore
+wherever you are. Due to this situation, many companies, including major
+GNU/Linux distributions, get trapped in a situation where they either
+get bad reviews due to lacking out-of-the-box media playback
+capabilities (and attempts to educate the reviewers have met with little
+success so far), or go against their own - and the free software
+movement's - wish to avoid proprietary software. Due to competitive
+pressure, most choose to add some support. Doing that through pure free
+software solutions would have them risk heavy litigation and punishment
+from patent owners. So when the decision is made to include support for
+patented codecs, it leaves them the choice of either using special
+proprietary applications, or try to integrate the support for these
+codecs through proprietary plugins into the multimedia infrastructure
+provided by GStreamer. Faced with one of these two evils the GStreamer
+community of course prefer the second option.
+
+The problem which arises is that most free software and open source
+applications developed use the GPL as their license. While this is
+generally a good thing, it creates a dilemma for people who want to put
+together a distribution. The dilemma they face is that if they include
+proprietary plugins in GStreamer to support patented formats in a way
+that is legal for them, they do risk running afoul of the GPL license of
+the applications. We have gotten some conflicting reports from lawyers
+on whether this is actually a problem, but the official stance of the
+FSF is that it is a problem. We view the FSF as an authority on this
+matter, so we are inclined to follow their interpretation of the GPL
+license.
+
+So what does this mean for you as an application developer? Well, it
+means you have to make an active decision on whether you want your
+application to be used together with proprietary plugins or not. What
+you decide here will also influence the chances of commercial
+distributions and Unix vendors shipping your application. The GStreamer
+community suggest you license your software using a license that will
+allow proprietary plugins to be bundled with GStreamer and your
+applications, in order to make sure that as many vendors as possible go
+with GStreamer instead of less free solutions. This in turn we hope and
+think will let GStreamer be a vehicle for wider use of free formats like
+the Xiph.org formats.
+
+If you do decide that you want to allow for non-free plugins to be used
+with your application you have a variety of choices. One of the simplest
+is using licenses like LGPL, MPL or BSD for your application instead of
+the GPL. Or you can add an exception clause to your GPL license stating
+that you except GStreamer plugins from the obligations of the GPL.
+
+A good example of such a GPL exception clause would be, using the Totem
+video player project as an example: The authors of the Totem video
+player project hereby grants permission for non-GPL-compatible GStreamer
+plugins to be used and distributed together with GStreamer and Totem.
+This permission goes above and beyond the permissions granted by the GPL
+license Totem is covered by.
+
+Our suggestion among these choices is to use the LGPL license, as it is
+what resembles the GPL most and it makes it a good licensing fit with
+the major GNU/Linux desktop projects like GNOME and KDE. It also allows
+you to share code more openly with projects that have compatible
+licenses. Obviously, pure GPL code without the above-mentioned clause is
+not usable in your application as such. By choosing the LGPL, there is
+no need for an exception clause and thus code can be shared more freely.
+
+I have above outlined the practical reasons for why the GStreamer
+community suggests you allow non-free plugins to be used with your
+applications. We feel that in the multimedia arena, the free software
+community is still not strong enough to set the agenda and that blocking
+non-free plugins to be used in our infrastructure hurts us more than it
+hurts the patent owners and their ilk.
+
+This view is not shared by everyone. The Free Software Foundation urges
+you to use an unmodified GPL for your applications, so as to push back
+against the temptation to use non-free plug-ins. They say that since not
+everyone else has the strength to reject them because they are
+unethical, they ask your help to give them a legal reason to do so.
+
+This advisory is part of a bigger advisory with a FAQ which you can find
+on the [GStreamer
+website](http://gstreamer.freedesktop.org/documentation/licensing.html)
+
--- /dev/null
+---
+title: Metadata
+...
+
+# Metadata
+
+GStreamer makes a clear distinction between two types of metadata, and
+has support for both types. The first is stream tags, which describe the
+content of a stream in a non-technical way. Examples include the author
+of a song, the title of that very same song or the album it is a part
+of. The other type of metadata is stream-info, which is a somewhat
+technical description of the properties of a stream. This can include
+video size, audio samplerate, codecs used and so on. Tags are handled
+using the GStreamer tagging system. Stream-info can be retrieved from a
+`GstPad` by getting the current (negotiated) `GstCaps` for that pad.
+
+## Metadata reading
+
+Stream information can most easily be read by reading it from a
+`GstPad`. This has already been discussed before in [Using capabilities
+for metadata](manual-pads.md#using-capabilities-for-metadata).
+Therefore, we will skip it here. Note that this requires access to all
+pads of which you want stream information.
+
+Tag reading is done through a bus in GStreamer, which has been discussed
+previously in [Bus](manual-bus.md). You can listen for
+`GST_MESSAGE_TAG` messages and handle them as you wish.
+
+Note, however, that the `GST_MESSAGE_TAG` message may be fired multiple
+times in the pipeline. It is the application's responsibility to put all
+those tags together and display them to the user in a nice, coherent
+way. Usually, using `gst_tag_list_merge ()` is a good enough way of
+doing this; make sure to empty the cache when loading a new song, or
+after every few minutes when listening to internet radio. Also, make
+sure you use `GST_TAG_MERGE_PREPEND` as merging mode, so that a new
+title (which came in later) has a preference over the old one for
+display.
+
+The following example will extract tags from a file and print them:
+
+``` c
+/* compile with:
+ * gcc -o tags tags.c `pkg-config --cflags --libs gstreamer-1.0` */
+#include <gst/gst.h>
+
+static void
+print_one_tag (const GstTagList * list, const gchar * tag, gpointer user_data)
+{
+ int i, num;
+
+ num = gst_tag_list_get_tag_size (list, tag);
+ for (i = 0; i < num; ++i) {
+ const GValue *val;
+
+ /* Note: when looking for specific tags, use the gst_tag_list_get_xyz() API,
+ * we only use the GValue approach here because it is more generic */
+ val = gst_tag_list_get_value_index (list, tag, i);
+ if (G_VALUE_HOLDS_STRING (val)) {
+ g_print ("\t%20s : %s\n", tag, g_value_get_string (val));
+ } else if (G_VALUE_HOLDS_UINT (val)) {
+ g_print ("\t%20s : %u\n", tag, g_value_get_uint (val));
+ } else if (G_VALUE_HOLDS_DOUBLE (val)) {
+ g_print ("\t%20s : %g\n", tag, g_value_get_double (val));
+ } else if (G_VALUE_HOLDS_BOOLEAN (val)) {
+ g_print ("\t%20s : %s\n", tag,
+ (g_value_get_boolean (val)) ? "true" : "false");
+ } else if (GST_VALUE_HOLDS_BUFFER (val)) {
+ GstBuffer *buf = gst_value_get_buffer (val);
+ guint buffer_size = gst_buffer_get_size (buf);
+
+ g_print ("\t%20s : buffer of size %u\n", tag, buffer_size);
+ } else if (GST_VALUE_HOLDS_DATE_TIME (val)) {
+ GstDateTime *dt = g_value_get_boxed (val);
+ gchar *dt_str = gst_date_time_to_iso8601_string (dt);
+
+ g_print ("\t%20s : %s\n", tag, dt_str);
+ g_free (dt_str);
+ } else {
+ g_print ("\t%20s : tag of type '%s'\n", tag, G_VALUE_TYPE_NAME (val));
+ }
+ }
+}
+
+static void
+on_new_pad (GstElement * dec, GstPad * pad, GstElement * fakesink)
+{
+ GstPad *sinkpad;
+
+ sinkpad = gst_element_get_static_pad (fakesink, "sink");
+ if (!gst_pad_is_linked (sinkpad)) {
+ if (gst_pad_link (pad, sinkpad) != GST_PAD_LINK_OK)
+ g_error ("Failed to link pads!");
+ }
+ gst_object_unref (sinkpad);
+}
+
+int
+main (int argc, char ** argv)
+{
+ GstElement *pipe, *dec, *sink;
+ GstMessage *msg;
+ gchar *uri;
+
+ gst_init (&argc, &argv);
+
+ if (argc < 2)
+ g_error ("Usage: %s FILE or URI", argv[0]);
+
+ if (gst_uri_is_valid (argv[1])) {
+ uri = g_strdup (argv[1]);
+ } else {
+ uri = gst_filename_to_uri (argv[1], NULL);
+ }
+
+ pipe = gst_pipeline_new ("pipeline");
+
+ dec = gst_element_factory_make ("uridecodebin", NULL);
+ g_object_set (dec, "uri", uri, NULL);
+ gst_bin_add (GST_BIN (pipe), dec);
+
+ sink = gst_element_factory_make ("fakesink", NULL);
+ gst_bin_add (GST_BIN (pipe), sink);
+
+ g_signal_connect (dec, "pad-added", G_CALLBACK (on_new_pad), sink);
+
+ gst_element_set_state (pipe, GST_STATE_PAUSED);
+
+ while (TRUE) {
+ GstTagList *tags = NULL;
+
+ msg = gst_bus_timed_pop_filtered (GST_ELEMENT_BUS (pipe),
+ GST_CLOCK_TIME_NONE,
+ GST_MESSAGE_ASYNC_DONE | GST_MESSAGE_TAG | GST_MESSAGE_ERROR);
+
+ if (GST_MESSAGE_TYPE (msg) != GST_MESSAGE_TAG) /* error or async_done */
+ break;
+
+ gst_message_parse_tag (msg, &tags);
+
+ g_print ("Got tags from element %s:\n", GST_OBJECT_NAME (msg->src));
+ gst_tag_list_foreach (tags, print_one_tag, NULL);
+ g_print ("\n");
+ gst_tag_list_unref (tags);
+
+ gst_message_unref (msg);
+ }
+
+ if (GST_MESSAGE_TYPE (msg) == GST_MESSAGE_ERROR) {
+ GError *err = NULL;
+
+ gst_message_parse_error (msg, &err, NULL);
+ g_printerr ("Got error: %s\n", err->message);
+ g_error_free (err);
+ }
+
+ gst_message_unref (msg);
+ gst_element_set_state (pipe, GST_STATE_NULL);
+ gst_object_unref (pipe);
+ g_free (uri);
+ return 0;
+}
+
+```
+
+## Tag writing
+
+Tag writing is done using the
+[`GstTagSetter`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstTagSetter.html)
+interface. All that's required is a tag-set-supporting element in your
+pipeline. In order to see if any of the elements in your pipeline
+supports tag writing, you can use the function
+`gst_bin_iterate_all_by_interface (pipeline,
+GST_TYPE_TAG_SETTER)`. On the resulting element, usually an encoder or
+muxer, you can use `gst_tag_setter_merge
+()` (with a taglist) or `gst_tag_setter_add
+()` (with individual tags) to set tags on it.
+
+A nice extra feature in GStreamer tag support is that tags are preserved
+in pipelines. This means that if you transcode one file containing tags
+into another media type, and that new media type supports tags too, then
+the tags will be handled as part of the data stream and be merged into
+the newly written media file, too.
+
--- /dev/null
+---
+title: Design principles
+...
+
+# Design principles
+
+## Clean and powerful
+
+GStreamer provides a clean interface to:
+
+ - The application programmer who wants to build a media pipeline. The
+ programmer can use an extensive set of powerful tools to create
+ media pipelines without writing a single line of code. Performing
+ complex media manipulations becomes very easy.
+
+ - The plugin programmer. Plugin programmers are provided a clean and
+ simple API to create self-contained plugins. An extensive debugging
+ and tracing mechanism has been integrated. GStreamer also comes with
+ an extensive set of real-life plugins that serve as examples too.
+
+## Object oriented
+
+GStreamer adheres to GObject, the GLib 2.0 object model. A programmer
+familiar with GLib 2.0 or GTK+ will be comfortable with GStreamer.
+
+GStreamer uses the mechanism of signals and object properties.
+
+All objects can be queried at runtime for their various properties and
+capabilities.
+
+GStreamer intends to be similar in programming methodology to GTK+. This
+applies to the object model, ownership of objects, reference counting,
+etc.
+
+## Extensible
+
+All GStreamer Objects can be extended using the GObject inheritance
+methods.
+
+All plugins are loaded dynamically and can be extended and upgraded
+independently.
+
+## Allow binary-only plugins
+
+Plugins are shared libraries that are loaded at runtime. Since all the
+properties of the plugin can be set using the GObject properties, there
+is no need (and in fact no way) to have any header files installed for
+the plugins.
+
+Special care has been taken to make plugins completely self-contained.
+All relevant aspects of plugins can be queried at run-time.
+
+## High performance
+
+High performance is obtained by:
+
+ - using GLib's `GSlice` allocator
+
+ - extremely light-weight links between plugins. Data can travel the
+ pipeline with minimal overhead. Data passing between plugins only
+ involves a pointer dereference in a typical pipeline.
+
+ - providing a mechanism to directly work on the target memory. A
+ plugin can for example directly write to the X server's shared
+ memory space. Buffers can also point to arbitrary memory, such as a
+ sound card's internal hardware buffer.
+
+ - refcounting and copy on write minimize usage of memcpy. Sub-buffers
+ efficiently split buffers into manageable pieces.
+
+ - dedicated streaming threads, with scheduling handled by the kernel.
+
+ - allowing hardware acceleration by using specialized plugins.
+
+ - using a plugin registry with the specifications of the plugins so
+ that the plugin loading can be delayed until the plugin is actually
+ used.
+
+## Clean core/plugins separation
+
+The core of GStreamer is essentially media-agnostic. It only knows about
+bytes and blocks, and only contains basic elements. The core of
+GStreamer is functional enough to even implement low-level system tools,
+like cp.
+
+All of the media handling functionality is provided by plugins external
+to the core. These tell the core how to handle specific types of media.
+
+## Provide a framework for codec experimentation
+
+GStreamer also wants to be an easy framework where codec developers can
+experiment with different algorithms, speeding up the development of
+open and free multimedia codecs like those developed by the [Xiph.Org
+Foundation](http://www.xiph.org) (such as Theora and Vorbis).
+
--- /dev/null
+---
+title: Pads and capabilities
+...
+
+# Pads and capabilities
+
+As we have seen in [Elements](manual-elements.md), the pads are the
+element's interface to the outside world. Data streams from one
+element's source pad to another element's sink pad. The specific type of
+media that the element can handle will be exposed by the pad's
+capabilities. We will talk more on capabilities later in this chapter
+(see [Capabilities of a pad](#capabilities-of-a-pad)).
+
+## Pads
+
+A pad type is defined by two properties: its direction and its
+availability. As we've mentioned before, GStreamer defines two pad
+directions: source pads and sink pads. This terminology is defined from
+the view of within the element: elements receive data on their sink pads
+and generate data on their source pads. Schematically, sink pads are
+drawn on the left side of an element, whereas source pads are drawn on
+the right side of an element. In such graphs, data flows from left to
+right. \[1\]
+
+Pad directions are very simple compared to pad availability. A pad can
+have any of three availabilities: always, sometimes and on request. The
+meaning of those three types is exactly as it says: always pads always
+exist, sometimes pad exist only in certain cases (and can disappear
+randomly), and on-request pads appear only if explicitly requested by
+applications.
+
+### Dynamic (or sometimes) pads
+
+Some elements might not have all of their pads when the element is
+created. This can happen, for example, with an Ogg demuxer element. The
+element will read the Ogg stream and create dynamic pads for each
+contained elementary stream (vorbis, theora) when it detects such a
+stream in the Ogg stream. Likewise, it will delete the pad when the
+stream ends. This principle is very useful for demuxer elements, for
+example.
+
+Running gst-inspect oggdemux will show that the element has only one
+pad: a sink pad called 'sink'. The other pads are “dormant”. You can see
+this in the pad template because there is an “Exists: Sometimes”
+property. Depending on the type of Ogg file you play, the pads will be
+created. We will see that this is very important when you are going to
+create dynamic pipelines. You can attach a signal handler to an element
+to inform you when the element has created a new pad from one of its
+“sometimes” pad templates. The following piece of code is an example
+of how to do this:
+
+``` c
+#include <gst/gst.h>
+
+static void
+cb_new_pad (GstElement *element,
+ GstPad *pad,
+ gpointer data)
+{
+ gchar *name;
+
+ name = gst_pad_get_name (pad);
+ g_print ("A new pad %s was created\n", name);
+ g_free (name);
+
+ /* here, you would setup a new pad link for the newly created pad */
+[..]
+
+}
+
+int
+main (int argc,
+ char *argv[])
+{
+ GstElement *pipeline, *source, *demux;
+ GMainLoop *loop;
+
+ /* init */
+ gst_init (&argc, &argv);
+
+ /* create elements */
+ pipeline = gst_pipeline_new ("my_pipeline");
+ source = gst_element_factory_make ("filesrc", "source");
+ g_object_set (source, "location", argv[1], NULL);
+ demux = gst_element_factory_make ("oggdemux", "demuxer");
+
+ /* you would normally check that the elements were created properly */
+
+ /* put together a pipeline */
+ gst_bin_add_many (GST_BIN (pipeline), source, demux, NULL);
+ gst_element_link_pads (source, "src", demux, "sink");
+
+ /* listen for newly created pads */
+ g_signal_connect (demux, "pad-added", G_CALLBACK (cb_new_pad), NULL);
+
+ /* start the pipeline */
+ gst_element_set_state (GST_ELEMENT (pipeline), GST_STATE_PLAYING);
+ loop = g_main_loop_new (NULL, FALSE);
+ g_main_loop_run (loop);
+
+[..]
+
+}
+
+```
+
+It is not uncommon to add elements to the pipeline only from within the
+"pad-added" callback. If you do this, don't forget to set the state of
+the newly-added elements to the target state of the pipeline using
+`gst_element_set_state ()` or `gst_element_sync_state_with_parent ()`.
+
+### Request pads
+
+An element can also have request pads. These pads are not created
+automatically but are only created on demand. This is very useful for
+multiplexers, aggregators and tee elements. Aggregators are elements
+that merge the content of several input streams together into one output
+stream. Tee elements are the reverse: they are elements that have one
+input stream and copy this stream to each of their output pads, which
+are created on request. Whenever an application needs another copy of
+the stream, it can simply request a new output pad from the tee element.
+
+The following piece of code shows how you can request a new output pad
+from a “tee” element:
+
+{{ examples/snippets.c#some_function }}
+
+The `gst_element_get_request_pad ()` method can be used to get a pad
+from the element based on the name of the pad template. It is also
+possible to request a pad that is compatible with another pad template.
+This is very useful if you want to link an element to a multiplexer
+element and you need to request a pad that is compatible. The method
+`gst_element_get_compatible_pad ()` can be used to request a compatible
+pad, as shown in the next example. It will request a compatible pad from
+an Ogg multiplexer from any input.
+
+{{ examples/snippets.c#link_to_multiplexer }}
+
+## Capabilities of a pad
+
+Since the pads play a very important role in how the element is viewed
+by the outside world, a mechanism is implemented to describe the data
+that can flow or currently flows through the pad by using capabilities.
+Here, we will briefly describe what capabilities are and how to use
+them, enough to get an understanding of the concept. For an in-depth
+look into capabilities and a list of all capabilities defined in
+GStreamer, see the [Plugin Writers
+Guide](http://gstreamer.freedesktop.org/data/doc/gstreamer/head/pwg/html/index.html).
+
+Capabilities are attached to pad templates and to pads. For pad
+templates, it will describe the types of media that may stream over a
+pad created from this template. For pads, it can either be a list of
+possible caps (usually a copy of the pad template's capabilities), in
+which case the pad is not yet negotiated, or it is the type of media
+that currently streams over this pad, in which case the pad has been
+negotiated already.
+
+### Dissecting capabilities
+
+A pad's capabilities are described in a `GstCaps` object. Internally, a
+[`GstCaps`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/gstreamer-GstCaps.html)
+will contain one or more
+[`GstStructure`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/gstreamer-GstStructure.html)
+that will describe one media type. A negotiated pad will have
+capabilities set that contain exactly *one* structure. Also, this
+structure will contain only *fixed* values. These constraints are not
+true for unnegotiated pads or pad templates.
+
+As an example, below is a dump of the capabilities of the “vorbisdec”
+element, which you will get by running `gst-inspect vorbisdec`. You will
+see two pads: a source and a sink pad. Both of these pads are always
+available, and both have capabilities attached to them. The sink pad
+will accept vorbis-encoded audio data, with the media type
+“audio/x-vorbis”. The source pad will be used to send raw (decoded)
+audio samples to the next element, with a raw audio media type (in this
+case, “audio/x-raw”). The source pad will also contain properties for
+the audio samplerate and the amount of channels, plus some more that you
+don't need to worry about for now.
+
+```
+
+Pad Templates:
+ SRC template: 'src'
+ Availability: Always
+ Capabilities:
+ audio/x-raw
+ format: F32LE
+ rate: [ 1, 2147483647 ]
+ channels: [ 1, 256 ]
+
+ SINK template: 'sink'
+ Availability: Always
+ Capabilities:
+ audio/x-vorbis
+
+```
+
+### Properties and values
+
+Properties are used to describe extra information for capabilities. A
+property consists of a key (a string) and a value. There are different
+possible value types that can be used:
+
+ - Basic types, this can be pretty much any `GType` registered with
+ Glib. Those properties indicate a specific, non-dynamic value for
+ this property. Examples include:
+
+ - An integer value (`G_TYPE_INT`): the property has this exact
+ value.
+
+ - A boolean value (`G_TYPE_BOOLEAN`): the property is either TRUE
+ or FALSE.
+
+ - A float value (`G_TYPE_FLOAT`): the property has this exact
+ floating point value.
+
+ - A string value (`G_TYPE_STRING`): the property contains a UTF-8
+ string.
+
+ - A fraction value (`GST_TYPE_FRACTION`): contains a fraction
+ expressed by an integer numerator and denominator.
+
+ - Range types are `GType`s registered by GStreamer to indicate a range
+ of possible values. They are used for indicating allowed audio
+ samplerate values or supported video sizes. The two types defined in
+ GStreamer are:
+
+ - An integer range value (`GST_TYPE_INT_RANGE`): the property
+ denotes a range of possible integers, with a lower and an upper
+ boundary. The “vorbisdec” element, for example, has a rate
+ property that can be between 8000 and 50000.
+
+ - A float range value (`GST_TYPE_FLOAT_RANGE`): the property
+ denotes a range of possible floating point values, with a lower
+ and an upper boundary.
+
+ - A fraction range value (`GST_TYPE_FRACTION_RANGE`): the property
+ denotes a range of possible fraction values, with a lower and an
+ upper boundary.
+
+ - A list value (`GST_TYPE_LIST`): the property can take any value from
+ a list of basic values given in this list.
+
+ Example: caps that express that either a sample rate of 44100 Hz and
+ a sample rate of 48000 Hz is supported would use a list of integer
+ values, with one value being 44100 and one value being 48000.
+
+ - An array value (`GST_TYPE_ARRAY`): the property is an array of
+ values. Each value in the array is a full value on its own, too. All
+ values in the array should be of the same elementary type. This
+ means that an array can contain any combination of integers, lists
+ of integers, integer ranges together, and the same for floats or
+ strings, but it can not contain both floats and ints at the same
+ time.
+
+ Example: for audio where there are more than two channels involved
+ the channel layout needs to be specified (for one and two channel
+ audio the channel layout is implicit unless stated otherwise in the
+ caps). So the channel layout would be an array of integer enum
+ values where each enum value represents a loudspeaker position.
+ Unlike a `GST_TYPE_LIST`, the values in an array will be interpreted
+ as a whole.
+
+## What capabilities are used for
+
+Capabilities (short: caps) describe the type of data that is streamed
+between two pads, or that one pad (template) supports. This makes them
+very useful for various purposes:
+
+ - Autoplugging: automatically finding elements to link to a pad based
+ on its capabilities. All autopluggers use this method.
+
+ - Compatibility detection: when two pads are linked, GStreamer can
+ verify if the two pads are talking about the same media type. The
+ process of linking two pads and checking if they are compatible is
+ called “caps negotiation”.
+
+ - Metadata: by reading the capabilities from a pad, applications can
+ provide information about the type of media that is being streamed
+ over the pad, which is information about the stream that is
+ currently being played back.
+
+ - Filtering: an application can use capabilities to limit the possible
+ media types that can stream between two pads to a specific subset of
+ their supported stream types. An application can, for example, use
+ “filtered caps” to set a specific (fixed or non-fixed) video size
+ that should stream between two pads. You will see an example of
+ filtered caps later in this manual, in [Manually adding or removing
+ data from/to a
+ pipeline](manual-dataaccess.md#manually-adding-or-removing-data-fromto-a-pipeline).
+ You can do caps filtering by inserting a capsfilter element into
+ your pipeline and setting its “caps” property. Caps filters are
+ often placed after converter elements like audioconvert,
+ audioresample, videoconvert or videoscale to force those converters
+ to convert data to a specific output format at a certain point in a
+ stream.
+
+### Using capabilities for metadata
+
+A pad can have a set (i.e. one or more) of capabilities attached to it.
+Capabilities (`GstCaps`) are represented as an array of one or more
+`GstStructure`s, and each `GstStructure` is an array of fields where
+each field consists of a field name string (e.g. "width") and a typed
+value (e.g. `G_TYPE_INT` or `GST_TYPE_INT_RANGE`).
+
+Note that there is a distinct difference between the *possible*
+capabilities of a pad (ie. usually what you find as caps of pad
+templates as they are shown in gst-inspect), the *allowed* caps of a pad
+(can be the same as the pad's template caps or a subset of them,
+depending on the possible caps of the peer pad) and lastly *negotiated*
+caps (these describe the exact format of a stream or buffer and contain
+exactly one structure and have no variable bits like ranges or lists,
+ie. they are fixed caps).
+
+You can get values of properties in a set of capabilities by querying
+individual properties of one structure. You can get a structure from a
+caps using `gst_caps_get_structure ()` and the number of structures in a
+`GstCaps` using `gst_caps_get_size ()`.
+
+Caps are called *simple caps* when they contain only one structure, and
+*fixed caps* when they contain only one structure and have no variable
+field types (like ranges or lists of possible values). Two other special
+types of caps are *ANY caps* and *empty caps*.
+
+Here is an example of how to extract the width and height from a set of
+fixed video caps:
+
+``` c
+static void
+read_video_props (GstCaps *caps)
+{
+ gint width, height;
+ const GstStructure *str;
+
+ g_return_if_fail (gst_caps_is_fixed (caps));
+
+ str = gst_caps_get_structure (caps, 0);
+ if (!gst_structure_get_int (str, "width", &width) ||
+ !gst_structure_get_int (str, "height", &height)) {
+ g_print ("No width/height available\n");
+ return;
+ }
+
+ g_print ("The video size of this set of capabilities is %dx%d\n",
+ width, height);
+}
+
+```
+
+### Creating capabilities for filtering
+
+While capabilities are mainly used inside a plugin to describe the media
+type of the pads, the application programmer often also has to have
+basic understanding of capabilities in order to interface with the
+plugins, especially when using filtered caps. When you're using filtered
+caps or fixation, you're limiting the allowed types of media that can
+stream between two pads to a subset of their supported media types. You
+do this using a `capsfilter` element in your pipeline. In order to do
+this, you also need to create your own `GstCaps`. The easiest way to do
+this is by using the convenience function `gst_caps_new_simple ()`:
+
+``` c
+static gboolean
+link_elements_with_filter (GstElement *element1, GstElement *element2)
+{
+ gboolean link_ok;
+ GstCaps *caps;
+
+ caps = gst_caps_new_simple ("video/x-raw",
+ "format", G_TYPE_STRING, "I420",
+ "width", G_TYPE_INT, 384,
+ "height", G_TYPE_INT, 288,
+ "framerate", GST_TYPE_FRACTION, 25, 1,
+ NULL);
+
+ link_ok = gst_element_link_filtered (element1, element2, caps);
+ gst_caps_unref (caps);
+
+ if (!link_ok) {
+ g_warning ("Failed to link element1 and element2!");
+ }
+
+ return link_ok;
+}
+
+```
+
+This will force the data flow between those two elements to a certain
+video format, width, height and framerate (or the linking will fail if
+that cannot be achieved in the context of the elements involved). Keep
+in mind that when you use `
+gst_element_link_filtered ()` it will automatically create a
+`capsfilter` element for you and insert it into your bin or pipeline
+between the two elements you want to connect (this is important if you
+ever want to disconnect those elements because then you will have to
+disconnect both elements from the capsfilter instead).
+
+In some cases, you will want to create a more elaborate set of
+capabilities to filter a link between two pads. Then, this function is
+too simplistic and you'll want to use the method `gst_caps_new_full ()`:
+
+``` c
+static gboolean
+link_elements_with_filter (GstElement *element1, GstElement *element2)
+{
+ gboolean link_ok;
+ GstCaps *caps;
+
+ caps = gst_caps_new_full (
+ gst_structure_new ("video/x-raw",
+ "width", G_TYPE_INT, 384,
+ "height", G_TYPE_INT, 288,
+ "framerate", GST_TYPE_FRACTION, 25, 1,
+ NULL),
+ gst_structure_new ("video/x-bayer",
+ "width", G_TYPE_INT, 384,
+ "height", G_TYPE_INT, 288,
+ "framerate", GST_TYPE_FRACTION, 25, 1,
+ NULL),
+ NULL);
+
+ link_ok = gst_element_link_filtered (element1, element2, caps);
+ gst_caps_unref (caps);
+
+ if (!link_ok) {
+ g_warning ("Failed to link element1 and element2!");
+ }
+
+ return link_ok;
+}
+
+```
+
+See the API references for the full API of
+[`GstStructure`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/gstreamer-GstStructure.html)
+and
+[`GstCaps`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/gstreamer-GstCaps.html).
+
+## Ghost pads
+
+You can see from [Visualisation of a GstBin element without ghost
+pads](#visualisation-of-a-gstbin-------element-without-ghost-pads) how a
+bin has no pads of its own. This is where "ghost pads" come into play.
+
+
+element without ghost pads](images/bin-element-noghost.png "fig:")
+
+A ghost pad is a pad from some element in the bin that can be accessed
+directly from the bin as well. Compare it to a symbolic link in UNIX
+filesystems. Using ghost pads on bins, the bin also has a pad and can
+transparently be used as an element in other parts of your code.
+
+
+element with a ghost pad](images/bin-element-ghost.png "fig:")
+
+[Visualisation of a GstBin element with a ghost
+pad](#visualisation-of-a-gstbin-------element-with-a-ghost-pad) is a
+representation of a ghost pad. The sink pad of element one is now also a
+pad of the bin. Because ghost pads look and work like any other pads,
+they can be added to any type of elements, not just to a `GstBin`, just
+like ordinary pads.
+
+A ghostpad is created using the function `gst_ghost_pad_new ()`:
+
+``` c
+#include <gst/gst.h>
+
+int
+main (int argc,
+ char *argv[])
+{
+ GstElement *bin, *sink;
+ GstPad *pad;
+
+ /* init */
+ gst_init (&argc, &argv);
+
+ /* create element, add to bin */
+ sink = gst_element_factory_make ("fakesink", "sink");
+ bin = gst_bin_new ("mybin");
+ gst_bin_add (GST_BIN (bin), sink);
+
+ /* add ghostpad */
+ pad = gst_element_get_static_pad (sink, "sink");
+ gst_element_add_pad (bin, gst_ghost_pad_new ("sink", pad));
+ gst_object_unref (GST_OBJECT (pad));
+
+[..]
+
+}
+
+```
+
+In the above example, the bin now also has a pad: the pad called “sink”
+of the given element. The bin can, from here on, be used as a substitute
+for the sink element. You could, for example, link another element to
+the bin.
+
+1. In reality, there is no objection to data flowing from a source pad
+ to the sink pad of an element upstream (to the left of this element
+ in drawings). Data will, however, always flow from a source pad of
+ one element to the sink pad of another.
+
--- /dev/null
+---
+title: Playback Components
+...
+
+# Playback Components
+
+GStreamer includes several higher-level components to simplify an
+application developer's life. All of the components discussed here (for
+now) are targetted at media playback. The idea of each of these
+components is to integrate as closely as possible with a GStreamer
+pipeline, but to hide the complexity of media type detection and several
+other rather complex topics that have been discussed in [Advanced
+GStreamer concepts](manual-advanced.md).
+
+We currently recommend people to use either playbin (see
+[Playbin](#playbin)) or decodebin (see [Decodebin](#decodebin)),
+depending on their needs. Playbin is the recommended solution for
+everything related to simple playback of media that should just work.
+Decodebin is a more flexible autoplugger that could be used to add more
+advanced features, such as playlist support, crossfading of audio tracks
+and so on. Its programming interface is more low-level than that of
+playbin, though.
+
+## Playbin
+
+Playbin is an element that can be created using the standard GStreamer
+API (e.g. `gst_element_factory_make ()`). The factory is conveniently
+called “playbin”. By being a `GstPipeline` (and thus a `GstElement`),
+playbin automatically supports all of the features of this class,
+including error handling, tag support, state handling, getting stream
+positions, seeking, and so on.
+
+Setting up a playbin pipeline is as simple as creating an instance of
+the playbin element, setting a file location using the “uri” property on
+playbin, and then setting the element to the `GST_STATE_PLAYING` state
+(the location has to be a valid URI, so “\<protocol\>://\<location\>”,
+e.g. file:///tmp/my.ogg or http://www.example.org/stream.ogg).
+Internally, playbin will set up a pipeline to playback the media
+location.
+
+``` c
+#include <gst/gst.h>
+
+[.. my_bus_callback goes here ..]
+
+gint
+main (gint argc,
+ gchar *argv[])
+{
+ GMainLoop *loop;
+ GstElement *play;
+ GstBus *bus;
+
+ /* init GStreamer */
+ gst_init (&argc, &argv);
+ loop = g_main_loop_new (NULL, FALSE);
+
+ /* make sure we have a URI */
+ if (argc != 2) {
+ g_print ("Usage: %s <URI>\n", argv[0]);
+ return -1;
+ }
+
+ /* set up */
+ play = gst_element_factory_make ("playbin", "play");
+ g_object_set (G_OBJECT (play), "uri", argv[1], NULL);
+
+ bus = gst_pipeline_get_bus (GST_PIPELINE (play));
+ gst_bus_add_watch (bus, my_bus_callback, loop);
+ gst_object_unref (bus);
+
+ gst_element_set_state (play, GST_STATE_PLAYING);
+
+ /* now run */
+ g_main_loop_run (loop);
+
+ /* also clean up */
+ gst_element_set_state (play, GST_STATE_NULL);
+ gst_object_unref (GST_OBJECT (play));
+
+ return 0;
+}
+
+```
+
+Playbin has several features that have been discussed previously:
+
+ - Settable video and audio output (using the “video-sink” and
+ “audio-sink” properties).
+
+ - Mostly controllable and trackable as a `GstElement`, including error
+ handling, eos handling, tag handling, state handling (through the
+ `GstBus`), media position handling and seeking.
+
+ - Buffers network-sources, with buffer fullness notifications being
+ passed through the `GstBus`.
+
+ - Supports visualizations for audio-only media.
+
+ - Supports subtitles, both in the media as well as from separate
+ files. For separate subtitle files, use the “suburi” property.
+
+ - Supports stream selection and disabling. If your media has multiple
+ audio or subtitle tracks, you can dynamically choose which one to
+ play back, or decide to turn it off altogether (which is especially
+ useful to turn off subtitles). For each of those, use the
+ “current-text” and other related properties.
+
+For convenience, it is possible to test “playbin” on the commandline,
+using the command “gst-launch-1.0 playbin uri=file:///path/to/file”.
+
+## Decodebin
+
+Decodebin is the actual autoplugger backend of playbin, which was
+discussed in the previous section. Decodebin will, in short, accept
+input from a source that is linked to its sinkpad and will try to detect
+the media type contained in the stream, and set up decoder routines for
+each of those. It will automatically select decoders. For each decoded
+stream, it will emit the “pad-added” signal, to let the client know
+about the newly found decoded stream. For unknown streams (which might
+be the whole stream), it will emit the “unknown-type” signal. The
+application is then responsible for reporting the error to the user.
+
+``` c
+
+#include <gst/gst.h>
+
+
+[.. my_bus_callback goes here ..]
+
+
+
+GstElement *pipeline, *audio;
+
+static void
+cb_newpad (GstElement *decodebin,
+ GstPad *pad,
+ gpointer data)
+{
+ GstCaps *caps;
+ GstStructure *str;
+ GstPad *audiopad;
+
+ /* only link once */
+ audiopad = gst_element_get_static_pad (audio, "sink");
+ if (GST_PAD_IS_LINKED (audiopad)) {
+ g_object_unref (audiopad);
+ return;
+ }
+
+ /* check media type */
+ caps = gst_pad_query_caps (pad, NULL);
+ str = gst_caps_get_structure (caps, 0);
+ if (!g_strrstr (gst_structure_get_name (str), "audio")) {
+ gst_caps_unref (caps);
+ gst_object_unref (audiopad);
+ return;
+ }
+ gst_caps_unref (caps);
+
+ /* link'n'play */
+ gst_pad_link (pad, audiopad);
+
+ g_object_unref (audiopad);
+}
+
+gint
+main (gint argc,
+ gchar *argv[])
+{
+ GMainLoop *loop;
+ GstElement *src, *dec, *conv, *sink;
+ GstPad *audiopad;
+ GstBus *bus;
+
+ /* init GStreamer */
+ gst_init (&argc, &argv);
+ loop = g_main_loop_new (NULL, FALSE);
+
+ /* make sure we have input */
+ if (argc != 2) {
+ g_print ("Usage: %s <filename>\n", argv[0]);
+ return -1;
+ }
+
+ /* setup */
+ pipeline = gst_pipeline_new ("pipeline");
+
+ bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
+ gst_bus_add_watch (bus, my_bus_callback, loop);
+ gst_object_unref (bus);
+
+ src = gst_element_factory_make ("filesrc", "source");
+ g_object_set (G_OBJECT (src), "location", argv[1], NULL);
+ dec = gst_element_factory_make ("decodebin", "decoder");
+ g_signal_connect (dec, "pad-added", G_CALLBACK (cb_newpad), NULL);
+ gst_bin_add_many (GST_BIN (pipeline), src, dec, NULL);
+ gst_element_link (src, dec);
+
+ /* create audio output */
+ audio = gst_bin_new ("audiobin");
+ conv = gst_element_factory_make ("audioconvert", "aconv");
+ audiopad = gst_element_get_static_pad (conv, "sink");
+ sink = gst_element_factory_make ("alsasink", "sink");
+ gst_bin_add_many (GST_BIN (audio), conv, sink, NULL);
+ gst_element_link (conv, sink);
+ gst_element_add_pad (audio,
+ gst_ghost_pad_new ("sink", audiopad));
+ gst_object_unref (audiopad);
+ gst_bin_add (GST_BIN (pipeline), audio);
+
+ /* run */
+ gst_element_set_state (pipeline, GST_STATE_PLAYING);
+ g_main_loop_run (loop);
+
+ /* cleanup */
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (GST_OBJECT (pipeline));
+
+ return 0;
+}
+
+
+```
+
+Decodebin, similar to playbin, supports the following features:
+
+ - Can decode an unlimited number of contained streams to decoded
+ output pads.
+
+ - Is handled as a `GstElement` in all ways, including tag or error
+ forwarding and state handling.
+
+Although decodebin is a good autoplugger, there's a whole lot of things
+that it does not do and is not intended to do:
+
+ - Taking care of input streams with a known media type (e.g. a DVD, an
+ audio-CD or such).
+
+ - Selection of streams (e.g. which audio track to play in case of
+ multi-language media streams).
+
+ - Overlaying subtitles over a decoded video stream.
+
+Decodebin can be easily tested on the commandline, e.g. by using the
+command `gst-launch-1.0 filesrc location=file.ogg ! decodebin
+! audioconvert ! audioresample ! autoaudiosink`.
+
+## URIDecodebin
+
+The uridecodebin element is very similar to decodebin, only that it
+automatically plugs a source plugin based on the protocol of the URI
+given.
+
+Uridecodebin will also automatically insert buffering elements when the
+uri is a slow network source. The buffering element will post BUFFERING
+messages that the application needs to handle as explained in
+[Buffering](manual-buffering.md). The following properties can be used
+to configure the buffering method:
+
+ - The buffer-size property allows you to configure a maximum size in
+ bytes for the buffer element.
+
+ - The buffer-duration property allows you to configure a maximum size
+ in time for the buffer element. The time will be estimated based on
+ the bitrate of the network.
+
+ - With the download property you can enable the download buffering
+ method as described in [Download
+ buffering](manual-buffering.md#download-buffering). Setting this
+ option to TRUE will only enable download buffering for selected
+ formats such as quicktime, flash video, avi and webm.
+
+ - You can also enable buffering on the parsed/demuxed data with the
+ use-buffering property. This is interesting to enable buffering on
+ slower random access media such as a network file server.
+
+URIDecodebin can be easily tested on the commandline, e.g. by using the
+command `gst-launch-1.0 uridecodebin uri=file:///file.ogg !
+! audioconvert ! audioresample ! autoaudiosink`.
+
+## Playsink
+
+The playsink element is a powerful sink element. It has request pads for
+raw decoded audio, video and text and it will configure itself to play
+the media streams. It has the following features:
+
+ - It exposes GstStreamVolume, GstVideoOverlay, GstNavigation and
+ GstColorBalance interfaces and automatically plugs software elements
+ to implement the interfaces when needed.
+
+ - It will automatically plug conversion elements.
+
+ - Can optionally render visualizations when there is no video input.
+
+ - Configurable sink elements.
+
+ - Configurable audio/video sync offset to fine-tune synchronization in
+ badly muxed files.
+
+ - Support for taking a snapshot of the last video frame.
+
+Below is an example of how you can use playsink. We use a uridecodebin
+element to decode into raw audio and video streams which we then link to
+the playsink request pads. We only link the first audio and video pads,
+you could use an input-selector to link all pads.
+
+``` c
+
+
+#include <gst/gst.h>
+
+
+[.. my_bus_callback goes here ..]
+
+
+
+
+
+GstElement *pipeline, *sink;
+
+static void
+cb_pad_added (GstElement *dec,
+ GstPad *pad,
+ gpointer data)
+{
+ GstCaps *caps;
+ GstStructure *str;
+ const gchar *name;
+ GstPadTemplate *templ;
+ GstElementClass *klass;
+
+ /* check media type */
+ caps = gst_pad_query_caps (pad, NULL);
+ str = gst_caps_get_structure (caps, 0);
+ name = gst_structure_get_name (str);
+
+ klass = GST_ELEMENT_GET_CLASS (sink);
+
+ if (g_str_has_prefix (name, "audio")) {
+ templ = gst_element_class_get_pad_template (klass, "audio_sink");
+ } else if (g_str_has_prefix (name, "video")) {
+ templ = gst_element_class_get_pad_template (klass, "video_sink");
+ } else if (g_str_has_prefix (name, "text")) {
+ templ = gst_element_class_get_pad_template (klass, "text_sink");
+ } else {
+ templ = NULL;
+ }
+
+ if (templ) {
+ GstPad *sinkpad;
+
+ sinkpad = gst_element_request_pad (sink, templ, NULL, NULL);
+
+ if (!gst_pad_is_linked (sinkpad))
+ gst_pad_link (pad, sinkpad);
+
+ gst_object_unref (sinkpad);
+ }
+}
+
+gint
+main (gint argc,
+ gchar *argv[])
+{
+ GMainLoop *loop;
+ GstElement *dec;
+ GstBus *bus;
+
+ /* init GStreamer */
+ gst_init (&argc, &argv);
+ loop = g_main_loop_new (NULL, FALSE);
+
+ /* make sure we have input */
+ if (argc != 2) {
+ g_print ("Usage: %s <uri>\n", argv[0]);
+ return -1;
+ }
+
+ /* setup */
+ pipeline = gst_pipeline_new ("pipeline");
+
+ bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
+ gst_bus_add_watch (bus, my_bus_callback, loop);
+ gst_object_unref (bus);
+
+ dec = gst_element_factory_make ("uridecodebin", "source");
+ g_object_set (G_OBJECT (dec), "uri", argv[1], NULL);
+ g_signal_connect (dec, "pad-added", G_CALLBACK (cb_pad_added), NULL);
+
+ /* create audio output */
+ sink = gst_element_factory_make ("playsink", "sink");
+ gst_util_set_object_arg (G_OBJECT (sink), "flags",
+ "soft-colorbalance+soft-volume+vis+text+audio+video");
+ gst_bin_add_many (GST_BIN (pipeline), dec, sink, NULL);
+
+ /* run */
+ gst_element_set_state (pipeline, GST_STATE_PLAYING);
+ g_main_loop_run (loop);
+
+ /* cleanup */
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (GST_OBJECT (pipeline));
+
+ return 0;
+}
+
+
+
+```
+
+This example will show audio and video depending on what you give it.
+Try this example on an audio file and you will see that it shows
+visualizations. You can change the visualization at runtime by changing
+the vis-plugin property.
+
--- /dev/null
+---
+title: Porting 0.10 applications to 1.0
+...
+
+# Porting 0.10 applications to 1.0
+
+This section outlines some of the changes necessary to port applications
+from GStreamer-0.10 to GStreamer-1.0. For a comprehensive and up-to-date
+list, see the separate [Porting
+to 1.0](http://cgit.freedesktop.org/gstreamer/gstreamer/plain/docs/random/porting-to-1.0.txt)
+document.
+
+It should be possible to port simple applications to GStreamer-1.0 in
+less than a day.
+
+## List of changes
+
+ - All deprecated methods were removed. Recompile against 0.10 with
+ GST\_DISABLE\_DEPRECATED defined (such as by adding
+ -DGST\_DISABLE\_DEPRECATED to the compiler flags) and fix issues
+ before attempting to port to 1.0.
+
+ - "playbin2" has been renamed to "playbin", with similar API
+
+ - "decodebin2" has been renamed to "decodebin", with similar API. Note
+ that there is no longer a "new-decoded-pad" signal, just use
+ GstElement's "pad-added" signal instead (but don't forget to remove
+ the 'gboolean last' argument from your old signal callback functino
+ signature).
+
+ - the names of some "formatted" pad templates has been changed from
+ e.g. "src%d" to "src%u" or "src\_%u" or similar, since we don't want
+ to see negative numbers in pad names. This mostly affects
+ applications that create request pads from elements.
+
+ - some elements that used to have a single dynamic source pad have a
+ source pad now. Example: wavparse, id3demux, iceydemux, apedemux.
+ (This does not affect applications using decodebin or playbin).
+
+ - playbin now proxies the GstVideoOverlay (former GstXOverlay)
+ interface, so most applications can just remove the sync bus handler
+ where they would set the window ID, and instead just set the window
+ ID on playbin from the application thread before starting playback.
+
+ playbin also proxies the GstColorBalance and GstNavigation
+ interfaces, so applications that use this don't need to go fishing
+ for elements that may implement those any more, but can just use on
+ playbin unconditionally.
+
+ - multifdsink, tcpclientsink, tcpclientsrc, tcpserversrc the protocol
+ property is removed, use gdppay and gdpdepay.
+
+ - XML serialization was removed.
+
+ - Probes and pad blocking was merged into new pad probes.
+
+ - Position, duration and convert functions no longer use an inout
+ parameter for the destination format.
+
+ - Video and audio caps were simplified. audio/x-raw-int and
+ audio/x-raw-float are now all under the audio/x-raw media type.
+ Similarly, video/x-raw-rgb and video/x-raw-yuv are now video/x-raw.
+
+ - ffmpegcolorspace was removed and replaced with videoconvert.
+
+ - GstMixerInterface / GstTunerInterface were removed without
+ replacement.
+
+ - The GstXOverlay interface was renamed to GstVideoOverlay, and now
+ part of the video library in gst-plugins-base, as the interfaces
+ library no longer exists.
+
+ The name of the GstXOverlay "prepare-xwindow-id" message has changed
+ to "prepare-window-handle" (and GstXOverlay has been renamed to
+ GstVideoOverlay). Code that checks for the string directly should be
+ changed to use
+ gst\_is\_video\_overlay\_prepare\_window\_handle\_message(message)
+ instead.
+
+ - The GstPropertyProbe interface was removed. There is no replacement
+ for it in GStreamer 1.0.x and 1.2.x, but since version 1.4 there is
+ a more featureful replacement for device discovery and feature
+ querying provided by GstDeviceMonitor, GstDevice, and friends. See
+ the ["GStreamer Device Discovery and Device Probing"
+ documentation](http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-device-probing.html).
+
+ - gst\_uri\_handler\_get\_uri() and the get\_uri vfunc now return a
+ copy of the URI string
+
+ gst\_uri\_handler\_set\_uri() and the set\_uri vfunc now take an
+ additional GError argument so the handler can notify the caller why
+ it didn't accept a particular URI.
+
+ gst\_uri\_handler\_set\_uri() now checks if the protocol of the URI
+ passed is one of the protocols advertised by the uri handler, so
+ set\_uri vfunc implementations no longer need to check that as well.
+
+ - GstTagList is now an opaque mini object instead of being typedefed
+ to a GstStructure. While it was previously okay (and in some cases
+ required because of missing taglist API) to cast a GstTagList to a
+ GstStructure or use gst\_structure\_\* API on taglists, you can no
+ longer do that. Doing so will cause crashes.
+
+ Also, tag lists are refcounted now, and can therefore not be freely
+ modified any longer. Make sure to call
+ gst\_tag\_list\_make\_writable (taglist) before adding, removing or
+ changing tags in the taglist.
+
+ GST\_TAG\_IMAGE, GST\_TAG\_PREVIEW\_IMAGE, GST\_TAG\_ATTACHMENT:
+ many tags that used to be of type GstBuffer are now of type
+ GstSample (which is basically a struct containing a buffer alongside
+ caps and some other info).
+
+ - GstController has now been merged into GstObject. It does not exists
+ as an individual object anymore. In addition core contains a
+ GstControlSource base class and the GstControlBinding. The actual
+ control sources are in the controller library as before. The 2nd big
+ change is that control sources generate a sequence of gdouble values
+ and those are mapped to the property type and value range by
+ GstControlBindings.
+
+ The whole gst\_controller\_\* API is gone and now available in
+ simplified form under gst\_object\_\*. ControlSources are now
+ attached via GstControlBinding to properties. There are no GValue
+ arguments used anymore when programming control sources.
+
--- /dev/null
+---
+title: Porting 0.8 applications to 0.10
+...
+
+# Porting 0.8 applications to 0.10
+
+This section of the appendix will discuss shortly what changes to
+applications will be needed to quickly and conveniently port most
+applications from GStreamer-0.8 to GStreamer-0.10, with references to
+the relevant sections in this Application Development Manual where
+needed. With this list, it should be possible to port simple
+applications to GStreamer-0.10 in less than a day.
+
+## List of changes
+
+ - Most functions returning an object or an object property have been
+ changed to return its own reference rather than a constant reference
+ of the one owned by the object itself. The reason for this change is
+ primarily thread safety. This means, effectively, that return values
+ of functions such as `gst_element_get_pad ()`, `gst_pad_get_name ()`
+ and many more like these have to be free'ed or unreferenced after
+ use. Check the API references of each function to know for sure
+ whether return values should be free'ed or not. It is important that
+ all objects derived from GstObject are ref'ed/unref'ed using
+ gst\_object\_ref() and gst\_object\_unref() respectively (instead of
+ g\_object\_ref/unref).
+
+ - Applications should no longer use signal handlers to be notified of
+ errors, end-of-stream and other similar pipeline events. Instead,
+ they should use the `GstBus`, which has been discussed in
+ [Bus](manual-bus.md). The bus will take care that the messages will
+ be delivered in the context of a main loop, which is almost
+ certainly the application's main thread. The big advantage of this
+ is that applications no longer need to be thread-aware; they don't
+ need to use `g_idle_add
+ ()` in the signal handler and do the actual real work in the
+ idle-callback. GStreamer now does all that internally.
+
+ - Related to this, `gst_bin_iterate ()` has been removed. Pipelines
+ will iterate in their own thread, and applications can simply run a
+ `GMainLoop` (or call the mainloop of their UI toolkit, such as
+ `gtk_main
+ ()`).
+
+ - State changes can be delayed (ASYNC). Due to the new fully threaded
+ nature of GStreamer-0.10, state changes are not always immediate, in
+ particular changes including the transition from READY to PAUSED
+ state. This means two things in the context of porting applications:
+ first of all, it is no longer always possible to do
+ `gst_element_set_state ()` and check for a return value of
+ GST\_STATE\_CHANGE\_SUCCESS, as the state change might be delayed
+ (ASYNC) and the result will not be known until later. You should
+ still check for GST\_STATE\_CHANGE\_FAILURE right away, it is just
+ no longer possible to assume that everything that is not SUCCESS
+ means failure. Secondly, state changes might not be immediate, so
+ your code needs to take that into account. You can wait for a state
+ change to complete if you use GST\_CLOCK\_TIME\_NONE as timeout
+ interval with `gst_element_get_state ()`.
+
+ - In 0.8, events and queries had to manually be sent to sinks in
+ pipelines (unless you were using playbin). This is no longer the
+ case in 0.10. In 0.10, queries and events can be sent to toplevel
+ pipelines, and the pipeline will do the dispatching internally for
+ you. This means less bookkeeping in your application. For a short
+ code example, see [Position tracking and
+ seeking](manual-queryevents.md). Related, seeking is now
+ threadsafe, and your video output will show the new video position's
+ frame while seeking, providing a better user experience.
+
+ - The `GstThread` object has been removed. Applications can now simply
+ put elements in a pipeline with optionally some “queue” elements in
+ between for buffering, and GStreamer will take care of creating
+ threads internally. It is still possible to have parts of a pipeline
+ run in different threads than others, by using the “queue” element.
+ See [Threads](manual-threads.md) for details.
+
+ - Filtered caps -\> capsfilter element (the pipeline syntax for
+ gst-launch has not changed though).
+
+ - libgstgconf-0.10.la does not exist. Use the “gconfvideosink” and
+ “gconfaudiosink” elements instead, which will do live-updates and
+ require no library linking.
+
+ - The “new-pad” and “state-change” signals on `GstElement` were
+ renamed to “pad-added” and “state-changed”.
+
+ - `gst_init_get_popt_table ()` has been removed in favour of the new
+ GOption command line option API that was added to GLib 2.6.
+ `gst_init_get_option_group ()` is the new GOption-based equivalent
+ to `gst_init_get_ptop_table ()`.
+
--- /dev/null
+---
+title: Programs
+...
+
+# Programs
+
+## `gst-launch`
+
+This is a tool that will construct pipelines based on a command-line
+syntax.
+
+A simple commandline looks like:
+
+```
+gst-launch filesrc location=hello.mp3 ! mad ! audioresample ! osssink
+
+```
+
+A more complex pipeline looks like:
+
+```
+gst-launch filesrc location=redpill.vob ! dvddemux name=demux \
+ demux.audio_00 ! queue ! a52dec ! audioconvert ! audioresample ! osssink \
+ demux.video_00 ! queue ! mpeg2dec ! videoconvert ! xvimagesink
+
+```
+
+You can also use the parser in you own code. GStreamer provides a
+function gst\_parse\_launch () that you can use to construct a pipeline.
+The following program lets you create an MP3 pipeline using the
+gst\_parse\_launch () function:
+
+``` c
+#include <gst/gst.h>
+
+int
+main (int argc, char *argv[])
+{
+ GstElement *pipeline;
+ GstElement *filesrc;
+ GstMessage *msg;
+ GstBus *bus;
+ GError *error = NULL;
+
+ gst_init (&argc, &argv);
+
+ if (argc != 2) {
+ g_print ("usage: %s <filename>\n", argv[0]);
+ return -1;
+ }
+
+ pipeline = gst_parse_launch ("filesrc name=my_filesrc ! mad ! osssink", &error);
+ if (!pipeline) {
+ g_print ("Parse error: %s\n", error->message);
+ exit (1);
+ }
+
+ filesrc = gst_bin_get_by_name (GST_BIN (pipeline), "my_filesrc");
+ g_object_set (filesrc, "location", argv[1], NULL);
+ g_object_unref (filesrc);
+
+ gst_element_set_state (pipeline, GST_STATE_PLAYING);
+
+ bus = gst_element_get_bus (pipeline);
+
+ /* wait until we either get an EOS or an ERROR message. Note that in a real
+ * program you would probably not use gst_bus_poll(), but rather set up an
+ * async signal watch on the bus and run a main loop and connect to the
+ * bus's signals to catch certain messages or all messages */
+ msg = gst_bus_poll (bus, GST_MESSAGE_EOS | GST_MESSAGE_ERROR, -1);
+
+ switch (GST_MESSAGE_TYPE (msg)) {
+ case GST_MESSAGE_EOS: {
+ g_print ("EOS\n");
+ break;
+ }
+ case GST_MESSAGE_ERROR: {
+ GError *err = NULL; /* error to show to users */
+ gchar *dbg = NULL; /* additional debug string for developers */
+
+ gst_message_parse_error (msg, &err, &dbg);
+ if (err) {
+ g_printerr ("ERROR: %s\n", err->message);
+ g_error_free (err);
+ }
+ if (dbg) {
+ g_printerr ("[Debug details: %s]\n", dbg);
+ g_free (dbg);
+ }
+ }
+ default:
+ g_printerr ("Unexpected message of type %d", GST_MESSAGE_TYPE (msg));
+ break;
+ }
+ gst_message_unref (msg);
+
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+ gst_object_unref (bus);
+
+ return 0;
+}
+
+```
+
+Note how we can retrieve the filesrc element from the constructed bin
+using the element name.
+
+### Grammar Reference
+
+The `gst-launch` syntax is processed by a flex/bison parser. This
+section is intended to provide a full specification of the grammar; any
+deviations from this specification is considered a bug.
+
+#### Elements
+
+```
+ ... mad ...
+
+```
+
+A bare identifier (a string beginning with a letter and containing only
+letters, numbers, dashes, underscores, percent signs, or colons) will
+create an element from a given element factory. In this example, an
+instance of the "mad" MP3 decoding plugin will be created.
+
+#### Links
+
+```
+ ... !sink ...
+
+```
+
+An exclamation point, optionally having a qualified pad name (an the
+name of the pad, optionally preceded by the name of the element) on both
+sides, will link two pads. If the source pad is not specified, a source
+pad from the immediately preceding element will be automatically chosen.
+If the sink pad is not specified, a sink pad from the next element to be
+constructed will be chosen. An attempt will be made to find compatible
+pads. Pad names may be preceded by an element name, as in
+`my_element_name.sink_pad`.
+
+#### Properties
+
+```
+ ... location="http://gstreamer.net" ...
+
+```
+
+The name of a property, optionally qualified with an element name, and a
+value, separated by an equals sign, will set a property on an element.
+If the element is not specified, the previous element is assumed.
+Strings can optionally be enclosed in quotation marks. Characters in
+strings may be escaped with the backtick (`\`). If the right-hand side
+is all digits, it is considered to be an integer. If it is all digits
+and a decimal point, it is a double. If it is "true", "false", "TRUE",
+or "FALSE" it is considered to be boolean. Otherwise, it is parsed as a
+string. The type of the property is determined later on in the parsing,
+and the value is converted to the target type. This conversion is not
+guaranteed to work, it relies on the g\_value\_convert routines. No
+error message will be displayed on an invalid conversion, due to
+limitations in the value convert API.
+
+#### Bins, Threads, and Pipelines
+
+```
+ ( ... )
+
+```
+
+A pipeline description between parentheses is placed into a bin. The
+open paren may be preceded by a type name, as in `jackbin.( ... )` to
+make a bin of a specified type. Square brackets make pipelines, and
+curly braces make threads. The default toplevel bin type is a pipeline,
+although putting the whole description within parentheses or braces can
+override this default.
+
+## `gst-inspect`
+
+This is a tool to query a plugin or an element about its properties.
+
+To query the information about the element mad, you would specify:
+
+```
+gst-inspect mad
+
+```
+
+Below is the output of a query for the osssink element:
+
+```
+
+Factory Details:
+ Rank: secondary (128)
+ Long-name: Audio Sink (OSS)
+ Klass: Sink/Audio
+ Description: Output to a sound card via OSS
+ Author: Erik Walthinsen <omega@cse.ogi.edu>, Wim Taymans <wim.taymans@chello.be>
+
+Plugin Details:
+ Name: ossaudio
+ Description: OSS (Open Sound System) support for GStreamer
+ Filename: /home/wim/gst/head/gst-plugins-good/sys/oss/.libs/libgstossaudio.so
+ Version: 1.0.0.1
+ License: LGPL
+ Source module: gst-plugins-good
+ Source release date: 2012-09-25 12:52 (UTC)
+ Binary package: GStreamer Good Plug-ins git
+ Origin URL: Unknown package origin
+
+GObject
+ +----GInitiallyUnowned
+ +----GstObject
+ +----GstElement
+ +----GstBaseSink
+ +----GstAudioBaseSink
+ +----GstAudioSink
+ +----GstOssSink
+
+Pad Templates:
+ SINK template: 'sink'
+ Availability: Always
+ Capabilities:
+ audio/x-raw
+ format: { S16LE, U16LE, S8, U8 }
+ layout: interleaved
+ rate: [ 1, 2147483647 ]
+ channels: 1
+ audio/x-raw
+ format: { S16LE, U16LE, S8, U8 }
+ layout: interleaved
+ rate: [ 1, 2147483647 ]
+ channels: 2
+ channel-mask: 0x0000000000000003
+
+
+Element Flags:
+ no flags set
+
+Element Implementation:
+ Has change_state() function: gst_audio_base_sink_change_state
+
+Clocking Interaction:
+ element is supposed to provide a clock but returned NULL
+
+Element has no indexing capabilities.
+Element has no URI handling capabilities.
+
+Pads:
+ SINK: 'sink'
+ Implementation:
+ Has chainfunc(): gst_base_sink_chain
+ Has custom eventfunc(): gst_base_sink_event
+ Has custom queryfunc(): gst_base_sink_sink_query
+ Has custom iterintlinkfunc(): gst_pad_iterate_internal_links_default
+ Pad Template: 'sink'
+
+Element Properties:
+ name : The name of the object
+ flags: readable, writable
+ String. Default: "osssink0"
+ parent : The parent of the object
+ flags: readable, writable
+ Object of type "GstObject"
+ sync : Sync on the clock
+ flags: readable, writable
+ Boolean. Default: true
+ max-lateness : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
+ flags: readable, writable
+ Integer64. Range: -1 - 9223372036854775807 Default: -1
+ qos : Generate Quality-of-Service events upstream
+ flags: readable, writable
+ Boolean. Default: false
+ async : Go asynchronously to PAUSED
+ flags: readable, writable
+ Boolean. Default: true
+ ts-offset : Timestamp offset in nanoseconds
+ flags: readable, writable
+ Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0
+ enable-last-sample : Enable the last-sample property
+ flags: readable, writable
+ Boolean. Default: false
+ last-sample : The last sample received in the sink
+ flags: readable
+ Boxed pointer of type "GstSample"
+ blocksize : Size in bytes to pull per buffer (0 = default)
+ flags: readable, writable
+ Unsigned Integer. Range: 0 - 4294967295 Default: 4096
+ render-delay : Additional render delay of the sink in nanoseconds
+ flags: readable, writable
+ Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
+ throttle-time : The time to keep between rendered buffers
+ flags: readable, writable
+ Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
+ buffer-time : Size of audio buffer in microseconds, this is the minimum latency that the sink reports
+ flags: readable, writable
+ Integer64. Range: 1 - 9223372036854775807 Default: 200000
+ latency-time : The minimum amount of data to write in each iteration in microseconds
+ flags: readable, writable
+ Integer64. Range: 1 - 9223372036854775807 Default: 10000
+ provide-clock : Provide a clock to be used as the global pipeline clock
+ flags: readable, writable
+ Boolean. Default: true
+ slave-method : Algorithm to use to match the rate of the masterclock
+ flags: readable, writable
+ Enum "GstAudioBaseSinkSlaveMethod" Default: 1, "skew"
+ (0): resample - GST_AUDIO_BASE_SINK_SLAVE_RESAMPLE
+ (1): skew - GST_AUDIO_BASE_SINK_SLAVE_SKEW
+ (2): none - GST_AUDIO_BASE_SINK_SLAVE_NONE
+ can-activate-pull : Allow pull-based scheduling
+ flags: readable, writable
+ Boolean. Default: false
+ alignment-threshold : Timestamp alignment threshold in nanoseconds
+ flags: readable, writable
+ Unsigned Integer64. Range: 1 - 18446744073709551614 Default: 40000000
+ drift-tolerance : Tolerance for clock drift in microseconds
+ flags: readable, writable
+ Integer64. Range: 1 - 9223372036854775807 Default: 40000
+ discont-wait : Window of time in nanoseconds to wait before creating a discontinuity
+ flags: readable, writable
+ Unsigned Integer64. Range: 0 - 18446744073709551614 Default: 1000000000
+ device : OSS device (usually /dev/dspN)
+ flags: readable, writable
+ String. Default: "/dev/dsp"
+
+
+```
+
+To query the information about a plugin, you would do:
+
+```
+gst-inspect gstelements
+
+```
+
--- /dev/null
+---
+title: Position tracking and seeking
+...
+
+# Position tracking and seeking
+
+So far, we've looked at how to create a pipeline to do media processing
+and how to make it run. Most application developers will be interested
+in providing feedback to the user on media progress. Media players, for
+example, will want to show a slider showing the progress in the song,
+and usually also a label indicating stream length. Transcoding
+applications will want to show a progress bar on how much percent of the
+task is done. GStreamer has built-in support for doing all this using a
+concept known as *querying*. Since seeking is very similar, it will be
+discussed here as well. Seeking is done using the concept of *events*.
+
+## Querying: getting the position or length of a stream
+
+Querying is defined as requesting a specific stream property related to
+progress tracking. This includes getting the length of a stream (if
+available) or getting the current position. Those stream properties can
+be retrieved in various formats such as time, audio samples, video
+frames or bytes. The function most commonly used for this is
+`gst_element_query ()`, although some convenience wrappers are provided
+as well (such as `gst_element_query_position ()` and
+`gst_element_query_duration ()`). You can generally query the pipeline
+directly, and it'll figure out the internal details for you, like which
+element to query.
+
+Internally, queries will be sent to the sinks, and “dispatched”
+backwards until one element can handle it; that result will be sent back
+to the function caller. Usually, that is the demuxer, although with live
+sources (from a webcam), it is the source itself.
+
+``` c
+
+#include <gst/gst.h>
+
+
+
+
+static gboolean
+cb_print_position (GstElement *pipeline)
+{
+ gint64 pos, len;
+
+ if (gst_element_query_position (pipeline, GST_FORMAT_TIME, &pos)
+ && gst_element_query_duration (pipeline, GST_FORMAT_TIME, &len)) {
+ g_print ("Time: %" GST_TIME_FORMAT " / %" GST_TIME_FORMAT "\r",
+ GST_TIME_ARGS (pos), GST_TIME_ARGS (len));
+ }
+
+ /* call me again */
+ return TRUE;
+}
+
+gint
+main (gint argc,
+ gchar *argv[])
+{
+ GstElement *pipeline;
+
+[..]
+
+ /* run pipeline */
+ g_timeout_add (200, (GSourceFunc) cb_print_position, pipeline);
+ g_main_loop_run (loop);
+
+[..]
+
+}
+
+```
+
+## Events: seeking (and more)
+
+Events work in a very similar way as queries. Dispatching, for example,
+works exactly the same for events (and also has the same limitations),
+and they can similarly be sent to the toplevel pipeline and it will
+figure out everything for you. Although there are more ways in which
+applications and elements can interact using events, we will only focus
+on seeking here. This is done using the seek-event. A seek-event
+contains a playback rate, a seek offset format (which is the unit of the
+offsets to follow, e.g. time, audio samples, video frames or bytes),
+optionally a set of seeking-related flags (e.g. whether internal buffers
+should be flushed), a seek method (which indicates relative to what the
+offset was given), and seek offsets. The first offset (cur) is the new
+position to seek to, while the second offset (stop) is optional and
+specifies a position where streaming is supposed to stop. Usually it is
+fine to just specify GST\_SEEK\_TYPE\_NONE and -1 as end\_method and end
+offset. The behaviour of a seek is also wrapped in the `gst_element_seek
+()`.
+
+``` c
+static void
+seek_to_time (GstElement *pipeline,
+ gint64 time_nanoseconds)
+{
+ if (!gst_element_seek (pipeline, 1.0, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH,
+ GST_SEEK_TYPE_SET, time_nanoseconds,
+ GST_SEEK_TYPE_NONE, GST_CLOCK_TIME_NONE)) {
+ g_print ("Seek failed!\n");
+ }
+}
+
+```
+
+Seeks with the GST\_SEEK\_FLAG\_FLUSH should be done when the pipeline
+is in PAUSED or PLAYING state. The pipeline will automatically go to
+preroll state until the new data after the seek will cause the pipeline
+to preroll again. After the pipeline is prerolled, it will go back to
+the state (PAUSED or PLAYING) it was in when the seek was executed. You
+can wait (blocking) for the seek to complete with
+`gst_element_get_state()` or by waiting for the ASYNC\_DONE message to
+appear on the bus.
+
+Seeks without the GST\_SEEK\_FLAG\_FLUSH should only be done when the
+pipeline is in the PLAYING state. Executing a non-flushing seek in the
+PAUSED state might deadlock because the pipeline streaming threads might
+be blocked in the sinks.
+
+It is important to realise that seeks will not happen instantly in the
+sense that they are finished when the function `gst_element_seek ()`
+returns. Depending on the specific elements involved, the actual seeking
+might be done later in another thread (the streaming thread), and it
+might take a short time until buffers from the new seek position will
+reach downstream elements such as sinks (if the seek was non-flushing
+then it might take a bit longer).
+
+It is possible to do multiple seeks in short time-intervals, such as a
+direct response to slider movement. After a seek, internally, the
+pipeline will be paused (if it was playing), the position will be re-set
+internally, the demuxers and decoders will decode from the new position
+onwards and this will continue until all sinks have data again. If it
+was playing originally, it will be set to playing again, too. Since the
+new position is immediately available in a video output, you will see
+the new frame, even if your pipeline is not in the playing state.
+
--- /dev/null
+---
+title: Quotes from the Developers
+...
+
+# Quotes from the Developers
+
+As well as being a cool piece of software, GStreamer is a lively
+project, with developers from around the globe very actively
+contributing. We often hang out on the \#gstreamer IRC channel on
+irc.freenode.net: the following are a selection of amusing\[1\] quotes
+from our conversations.
+
+ - 6 Mar 2006
+ When I opened my eyes I was in a court room. There were masters
+ McIlroy and Thompson sitting in the jury and master Kernighan too.
+ There were the GStreamer developers standing in the defendant's
+ place, accused of violating several laws of Unix philosophy and
+ customer lock-down via running on a proprietary pipeline, different
+ from that of the Unix systems. I heard Eric Raymond whispering "got
+ to add this case to my book.
+
+ *behdad's blog*
+
+ - 22 May 2007
+ *\<\_\_tim\>* Uraeus: amusing, isn't it?
+
+ *\<Uraeus\>* \_\_tim: I wrote that :)
+
+ *\<\_\_tim\>* Uraeus: of course you did; your refusal to surrender
+ to the oppressive regime of the third-person-singular-rule is so
+ unique in its persistence that it's hard to miss :)
+
+ - 12 Sep 2005
+ *\<wingo\>* we just need to get rid of that mmap stuff
+
+ *\<wingo\>* i think gnomevfssrc is faster for files even
+
+ *\<BBB\>* wingo, no
+
+ *\<BBB\>* and no
+
+ *\<wingo\>* good points ronald
+
+ - 23 Jun 2005
+ *\* wingo* back
+
+ *\* thomasvs* back
+
+ \--- You are now known as everybody
+
+ *\* everybody* back back
+
+ *\<everybody\>* now break it down
+
+ \--- You are now known as thomasvs
+
+ *\* bilboed* back
+
+ \--- bilboed is now known as john-sebastian
+
+ *\* john-sebastian* bach
+
+ \--- john-sebastian is now known as bilboed
+
+ \--- You are now known as scratch\_my
+
+ *\* scratch\_my* back
+
+ \--- bilboed is now known as Illbe
+
+ \--- You are now known as thomasvs
+
+ *\* Illbe* back
+
+ \--- Illbe is now known as bilboed
+
+ - 20 Apr 2005
+ *thomas*: jrb, somehow his screenshotsrc grabs whatever X is showing
+ and makes it available as a stream of frames
+
+ *jrb*: thomas: so, is the point that the screenshooter takes a
+ video? but won't the dialog be in the video? oh, nevermind. I'll
+ just send mail...
+
+ *thomas*: jrb, well, it would shoot first and ask questions later
+
+ - 2 Nov 2004
+ *zaheerm*: wtay: unfair u fixed the bug i was using as a feature\!
+
+ - 14 Oct 2004
+ *\* zaheerm* wonders how he can break gstreamer today :)
+
+ *ensonic*: zaheerm, spider is always a good starting point
+
+ - 14 Jun 2004
+ *teuf*: ok, things work much better when I don't write incredibly
+ stupid and buggy code
+
+ *thaytan*: I find that too
+
+ - 23 Nov 2003
+ *Uraeus*: ah yes, the sleeping part, my mind is not multitasking so
+ I was still thinking about exercise
+
+ *dolphy*: Uraeus: your mind is multitasking
+
+ *dolphy*: Uraeus: you just miss low latency patches
+
+ - 14 Sep 2002
+ \--- *wingo-party* is now known as *wingo*
+
+ \* *wingo* holds head
+
+ - 4 Jun 2001
+ *taaz:* you witchdoctors and your voodoo mpeg2 black magic...
+
+ *omega\_:* um. I count three, no four different cults there \<g\>
+
+ *ajmitch:* hehe
+
+ *omega\_:* witchdoctors, voodoo, black magic,
+
+ *omega\_:* and mpeg
+
+ - 16 Feb 2001
+ *wtay:* I shipped a few commerical products to \>40000 people now
+ but GStreamer is way more exciting...
+
+ - 16 Feb 2001
+ \* *tool-man* is a gstreamer groupie
+
+ - 14 Jan 2001
+ *Omega:* did you run ldconfig? maybe it talks to init?
+
+ *wtay:* not sure, don't think so... I did run gstreamer-register
+ though :-)
+
+ *Omega:* ah, that did it then ;-)
+
+ *wtay:* right
+
+ *Omega:* probably not, but in case GStreamer starts turning into an
+ OS, someone please let me know?
+
+ - 9 Jan 2001
+ *wtay:* me tar, you rpm?
+
+ *wtay:* hehe, forgot "zan"
+
+ *Omega:* ?
+
+ *wtay:* me tar"zan", you ...
+
+ - 7 Jan 2001
+ *Omega:* that means probably building an agreggating,
+ cache-massaging queue to shove N buffers across all at once, forcing
+ cache transfer.
+
+ *wtay:* never done that before...
+
+ *Omega:* nope, but it's easy to do in gstreamer \<g\>
+
+ *wtay:* sure, I need to rewrite cp with gstreamer too, someday :-)
+
+ - 7 Jan 2001
+ *wtay:* GStreamer; always at least one developer is awake...
+
+ - 5/6 Jan 2001
+ *wtay:* we need to cut down the time to create an mp3 player down to
+ seconds...
+
+ *richardb:* :)
+
+ *Omega:* I'm wanting to something more interesting soon, I did the
+ "draw an mp3 player in 15sec" back in October '99.
+
+ *wtay:* by the time Omega gets his hands on the editor, you'll see a
+ complete audio mixer in the editor :-)
+
+ *richardb:* Well, it clearly has the potential...
+
+ *Omega:* Working on it... ;-)
+
+ - 28 Dec 2000
+ *MPAA:* We will sue you now, you have violated our IP rights\!
+
+ *wtay:* hehehe
+
+ *MPAA:* How dare you laugh at us? We have lawyers\! We have
+ Congressmen\! We have *LARS*\!
+
+ *wtay:* I'm so sorry your honor
+
+ *MPAA:* Hrumph.
+
+ \* *wtay* bows before thy
+
+<!-- end list -->
+
+1. No guarantee of sense of humour compatibility is given.
+
--- /dev/null
+---
+title: Threads
+...
+
+# Threads
+
+GStreamer is inherently multi-threaded, and is fully thread-safe. Most
+threading internals are hidden from the application, which should make
+application development easier. However, in some cases, applications may
+want to have influence on some parts of those. GStreamer allows
+applications to force the use of multiple threads over some parts of a
+pipeline. See [When would you want to force a
+thread?](#when-would-you-want-to-force-a-thread).
+
+GStreamer can also notify you when threads are created so that you can
+configure things such as the thread priority or the threadpool to use.
+See [Configuring Threads in
+GStreamer](#configuring-threads-in-gstreamer).
+
+## Scheduling in GStreamer
+
+Each element in the GStreamer pipeline decides how it is going to be
+scheduled. Elements can choose if their pads are to be scheduled
+push-based or pull-based. An element can, for example, choose to start a
+thread to start pulling from the sink pad or/and start pushing on the
+source pad. An element can also choose to use the upstream or downstream
+thread for its data processing in push and pull mode respectively.
+GStreamer does not pose any restrictions on how the element chooses to
+be scheduled. See the Plugin Writer Guide for more details.
+
+What will happen in any case is that some elements will start a thread
+for their data processing, called the “streaming threads”. The streaming
+threads, or `GstTask` objects, are created from a `GstTaskPool` when the
+element needs to make a streaming thread. In the next section we see how
+we can receive notifications of the tasks and pools.
+
+## Configuring Threads in GStreamer
+
+A STREAM\_STATUS message is posted on the bus to inform you about the
+status of the streaming threads. You will get the following information
+from the message:
+
+ - When a new thread is about to be created, you will be notified of
+ this with a GST\_STREAM\_STATUS\_TYPE\_CREATE type. It is then
+ possible to configure a `GstTaskPool` in the `GstTask`. The custom
+ taskpool will provide custom threads for the task to implement the
+ streaming threads.
+
+ This message needs to be handled synchronously if you want to
+ configure a custom taskpool. If you don't configure the taskpool on
+ the task when this message returns, the task will use its default
+ pool.
+
+ - When a thread is entered or left. This is the moment where you could
+ configure thread priorities. You also get a notification when a
+ thread is destroyed.
+
+ - You get messages when the thread starts, pauses and stops. This
+ could be used to visualize the status of streaming threads in a gui
+ application.
+
+We will now look at some examples in the next sections.
+
+### Boost priority of a thread
+
+```
+ .----------. .----------.
+ | faksesrc | | fakesink |
+ | src->sink |
+ '----------' '----------'
+
+```
+
+Let's look at the simple pipeline above. We would like to boost the
+priority of the streaming thread. It will be the fakesrc element that
+starts the streaming thread for generating the fake data pushing them to
+the peer fakesink. The flow for changing the priority would go like
+this:
+
+ - When going from READY to PAUSED state, fakesrc will require a
+ streaming thread for pushing data into the fakesink. It will post a
+ STREAM\_STATUS message indicating its requirement for a streaming
+ thread.
+
+ - The application will react to the STREAM\_STATUS messages with a
+ sync bus handler. It will then configure a custom `GstTaskPool` on
+ the `GstTask` inside the message. The custom taskpool is responsible
+ for creating the threads. In this example we will make a thread with
+ a higher priority.
+
+ - Alternatively, since the sync message is called in the thread
+ context, you can use thread ENTER/LEAVE notifications to change the
+ priority or scheduling pollicy of the current thread.
+
+In a first step we need to implement a custom `GstTaskPool` that we can
+configure on the task. Below is the implementation of a `GstTaskPool`
+subclass that uses pthreads to create a SCHED\_RR real-time thread. Note
+that creating real-time threads might require extra priveleges.
+
+``` c
+#include <pthread.h>
+
+typedef struct
+{
+ pthread_t thread;
+} TestRTId;
+
+G_DEFINE_TYPE (TestRTPool, test_rt_pool, GST_TYPE_TASK_POOL);
+
+static void
+default_prepare (GstTaskPool * pool, GError ** error)
+{
+ /* we don't do anything here. We could construct a pool of threads here that
+ * we could reuse later but we don't */
+}
+
+static void
+default_cleanup (GstTaskPool * pool)
+{
+}
+
+static gpointer
+default_push (GstTaskPool * pool, GstTaskPoolFunction func, gpointer data,
+ GError ** error)
+{
+ TestRTId *tid;
+ gint res;
+ pthread_attr_t attr;
+ struct sched_param param;
+
+ tid = g_slice_new0 (TestRTId);
+
+ pthread_attr_init (&attr);
+ if ((res = pthread_attr_setschedpolicy (&attr, SCHED_RR)) != 0)
+ g_warning ("setschedpolicy: failure: %p", g_strerror (res));
+
+ param.sched_priority = 50;
+ if ((res = pthread_attr_setschedparam (&attr, ¶m)) != 0)
+ g_warning ("setschedparam: failure: %p", g_strerror (res));
+
+ if ((res = pthread_attr_setinheritsched (&attr, PTHREAD_EXPLICIT_SCHED)) != 0)
+ g_warning ("setinheritsched: failure: %p", g_strerror (res));
+
+ res = pthread_create (&tid->thread, &attr, (void *(*)(void *)) func, data);
+
+ if (res != 0) {
+ g_set_error (error, G_THREAD_ERROR, G_THREAD_ERROR_AGAIN,
+ "Error creating thread: %s", g_strerror (res));
+ g_slice_free (TestRTId, tid);
+ tid = NULL;
+ }
+
+ return tid;
+}
+
+static void
+default_join (GstTaskPool * pool, gpointer id)
+{
+ TestRTId *tid = (TestRTId *) id;
+
+ pthread_join (tid->thread, NULL);
+
+ g_slice_free (TestRTId, tid);
+}
+
+static void
+test_rt_pool_class_init (TestRTPoolClass * klass)
+{
+ GstTaskPoolClass *gsttaskpool_class;
+
+ gsttaskpool_class = (GstTaskPoolClass *) klass;
+
+ gsttaskpool_class->prepare = default_prepare;
+ gsttaskpool_class->cleanup = default_cleanup;
+ gsttaskpool_class->push = default_push;
+ gsttaskpool_class->join = default_join;
+}
+
+static void
+test_rt_pool_init (TestRTPool * pool)
+{
+}
+
+GstTaskPool *
+test_rt_pool_new (void)
+{
+ GstTaskPool *pool;
+
+ pool = g_object_new (TEST_TYPE_RT_POOL, NULL);
+
+ return pool;
+}
+
+
+
+```
+
+The important function to implement when writing an taskpool is the
+“push” function. The implementation should start a thread that calls
+the given function. More involved implementations might want to keep
+some threads around in a pool because creating and destroying threads is
+not always the fastest operation.
+
+In a next step we need to actually configure the custom taskpool when
+the fakesrc needs it. For this we intercept the STREAM\_STATUS messages
+with a sync handler.
+
+``` c
+
+
+static GMainLoop* loop;
+
+static void
+on_stream_status (GstBus *bus,
+ GstMessage *message,
+ gpointer user_data)
+{
+ GstStreamStatusType type;
+ GstElement *owner;
+ const GValue *val;
+ GstTask *task = NULL;
+
+ gst_message_parse_stream_status (message, &type, &owner);
+
+ val = gst_message_get_stream_status_object (message);
+
+ /* see if we know how to deal with this object */
+ if (G_VALUE_TYPE (val) == GST_TYPE_TASK) {
+ task = g_value_get_object (val);
+ }
+
+ switch (type) {
+ case GST_STREAM_STATUS_TYPE_CREATE:
+ if (task) {
+ GstTaskPool *pool;
+
+ pool = test_rt_pool_new();
+
+ gst_task_set_pool (task, pool);
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+static void
+on_error (GstBus *bus,
+ GstMessage *message,
+ gpointer user_data)
+{
+ g_message ("received ERROR");
+ g_main_loop_quit (loop);
+}
+
+static void
+on_eos (GstBus *bus,
+ GstMessage *message,
+ gpointer user_data)
+{
+ g_main_loop_quit (loop);
+}
+
+int
+main (int argc, char *argv[])
+{
+ GstElement *bin, *fakesrc, *fakesink;
+ GstBus *bus;
+ GstStateChangeReturn ret;
+
+ gst_init (&argc, &argv);
+
+ /* create a new bin to hold the elements */
+ bin = gst_pipeline_new ("pipeline");
+ g_assert (bin);
+
+ /* create a source */
+ fakesrc = gst_element_factory_make ("fakesrc", "fakesrc");
+ g_assert (fakesrc);
+ g_object_set (fakesrc, "num-buffers", 50, NULL);
+
+ /* and a sink */
+ fakesink = gst_element_factory_make ("fakesink", "fakesink");
+ g_assert (fakesink);
+
+ /* add objects to the main pipeline */
+ gst_bin_add_many (GST_BIN (bin), fakesrc, fakesink, NULL);
+
+ /* link the elements */
+ gst_element_link (fakesrc, fakesink);
+
+ loop = g_main_loop_new (NULL, FALSE);
+
+ /* get the bus, we need to install a sync handler */
+ bus = gst_pipeline_get_bus (GST_PIPELINE (bin));
+ gst_bus_enable_sync_message_emission (bus);
+ gst_bus_add_signal_watch (bus);
+
+ g_signal_connect (bus, "sync-message::stream-status",
+ (GCallback) on_stream_status, NULL);
+ g_signal_connect (bus, "message::error",
+ (GCallback) on_error, NULL);
+ g_signal_connect (bus, "message::eos",
+ (GCallback) on_eos, NULL);
+
+ /* start playing */
+ ret = gst_element_set_state (bin, GST_STATE_PLAYING);
+ if (ret != GST_STATE_CHANGE_SUCCESS) {
+ g_message ("failed to change state");
+ return -1;
+ }
+
+ /* Run event loop listening for bus messages until EOS or ERROR */
+ g_main_loop_run (loop);
+
+ /* stop the bin */
+ gst_element_set_state (bin, GST_STATE_NULL);
+ gst_object_unref (bus);
+ g_main_loop_unref (loop);
+
+ return 0;
+}
+
+
+
+``` c
+
+Note that this program likely needs root permissions in order to create
+real-time threads. When the thread can't be created, the state change
+function will fail, which we catch in the application above.
+
+When there are multiple threads in the pipeline, you will receive
+multiple STREAM\_STATUS messages. You should use the owner of the
+message, which is likely the pad or the element that starts the thread,
+to figure out what the function of this thread is in the context of the
+application.
+
+## When would you want to force a thread?
+
+We have seen that threads are created by elements but it is also
+possible to insert elements in the pipeline for the sole purpose of
+forcing a new thread in the pipeline.
+
+There are several reasons to force the use of threads. However, for
+performance reasons, you never want to use one thread for every element
+out there, since that will create some overhead. Let's now list some
+situations where threads can be particularly useful:
+
+ - Data buffering, for example when dealing with network streams or
+ when recording data from a live stream such as a video or audio
+ card. Short hickups elsewhere in the pipeline will not cause data
+ loss. See also [Stream
+ buffering](manual-buffering.md#stream-buffering) about network
+ buffering with queue2.
+
+ 
+
+ - Synchronizing output devices, e.g. when playing a stream containing
+ both video and audio data. By using threads for both outputs, they
+ will run independently and their synchronization will be better.
+
+ 
+
+Above, we've mentioned the “queue” element several times now. A queue is
+the thread boundary element through which you can force the use of
+threads. It does so by using a classic provider/consumer model as
+learned in threading classes at universities all around the world. By
+doing this, it acts both as a means to make data throughput between
+threads threadsafe, and it can also act as a buffer. Queues have several
+`GObject` properties to be configured for specific uses. For example,
+you can set lower and upper thresholds for the element. If there's less
+data than the lower threshold (default: disabled), it will block output.
+If there's more data than the upper threshold, it will block input or
+(if configured to do so) drop data.
+
+To use a queue (and therefore force the use of two distinct threads in
+the pipeline), one can simply create a “queue” element and put this in
+as part of the pipeline. GStreamer will take care of all threading
+details internally.
+
--- /dev/null
+---
+title: Clocking
+...
+
+# Clocking
+
+When playing complex media, each sound and video sample must be played
+in a specific order at a specific time. For this purpose, GStreamer
+provides a synchronization mechanism.
+
+## Clocks
+
+Time in GStreamer is defined as the value returned from a particular
+`GstClock` object from the method `gst_clock_get_time ()`.
+
+In a typical computer, there are many sources that can be used as a time
+source, e.g., the system time, soundcards, CPU performance counters, ...
+For this reason, there are many `GstClock` implementations available in
+GStreamer. The clock time doesn't always start from 0 or from some known
+value. Some clocks start counting from some known start date, other
+clocks start counting since last reboot, etc...
+
+As clocks return an absolute measure of time, they are not usually used
+directly. Instead, differences between two clock times are used to
+measure elapsed time according to a clock.
+
+## Clock running-time
+
+A clock returns the **absolute-time** according to that clock with
+`gst_clock_get_time ()`. From the absolute-time is a **running-time**
+calculated, which is simply the difference between a previous snapshot
+of the absolute-time called the **base-time**. So:
+
+running-time = absolute-time - base-time
+
+A GStreamer `GstPipeline` object maintains a `GstClock` object and a
+base-time when it goes to the PLAYING state. The pipeline gives a handle
+to the selected `GstClock` to each element in the pipeline along with
+selected base-time. The pipeline will select a base-time in such a way
+that the running-time reflects the total time spent in the PLAYING
+state. As a result, when the pipeline is PAUSED, the running-time stands
+still.
+
+Because all objects in the pipeline have the same clock and base-time,
+they can thus all calculate the running-time according to the pipeline
+clock.
+
+## Buffer running-time
+
+To calculate a buffer running-time, we need a buffer timestamp and the
+SEGMENT event that preceded the buffer. First we can convert the SEGMENT
+event into a `GstSegment` object and then we can use the
+`gst_segment_to_running_time ()` function to perform the calculation of
+the buffer running-time.
+
+Synchronization is now a matter of making sure that a buffer with a
+certain running-time is played when the clock reaches the same
+running-time. Usually this task is done by sink elements. Sink also have
+to take into account the latency configured in the pipeline and add this
+to the buffer running-time before synchronizing to the pipeline clock.
+
+## Obligations of each element.
+
+Let us clarify the contract between GStreamer and each element in the
+pipeline.
+
+### Non-live source elements
+
+Non-live source elements must place a timestamp in each buffer that they
+deliver when this is possible. They must choose the timestamps and the
+values of the SEGMENT event in such a way that the running-time of the
+buffer starts from 0.
+
+Some sources, such as filesrc, is not able to generate timestamps on all
+buffers. It can and must however create a timestamp on the first buffer
+(with a running-time of 0).
+
+The source then pushes out the SEGMENT event followed by the timestamped
+buffers.
+
+### Live source elements
+
+Live source elements must place a timestamp in each buffer that they
+deliver. They must choose the timestamps and the values of the SEGMENT
+event in such a way that the running-time of the buffer matches exactly
+the running-time of the pipeline clock when the first byte in the buffer
+was captured.
+
+### Parser/Decoder/Encoder elements
+
+Parser/Decoder elements must use the incoming timestamps and transfer
+those to the resulting output buffers. They are allowed to interpolate
+or reconstruct timestamps on missing input buffers when they can.
+
+### Demuxer elements
+
+Demuxer elements can usually set the timestamps stored inside the media
+file onto the outgoing buffers. They need to make sure that outgoing
+buffers that are to be played at the same time have the same
+running-time. Demuxers also need to take into account the incoming
+timestamps on buffers and use that to calculate an offset on the
+outgoing buffer timestamps.
+
+### Muxer elements
+
+Muxer elements should use the incoming buffer running-time to mux the
+different streams together. They should copy the incoming running-time
+to the outgoing buffers.
+
+### Sink elements
+
+If the element is intended to emit samples at a specific time (real time
+playing), the element should require a clock, and thus implement the
+method `set_clock`.
+
+The sink should then make sure that the sample with running-time is
+played exactly when the pipeline clock reaches that running-time +
+latency. Some elements might use the clock API such as
+`gst_clock_id_wait()` to perform this action. Other sinks might need to
+use other means of scheduling timely playback of the data.
+
--- /dev/null
+---
+title: 'Events: Seeking, Navigation and More'
+...
+
+# Events: Seeking, Navigation and More
+
+There are many different event types but only two ways they can travel
+in the pipeline: downstream or upstream. It is very important to
+understand how both of these methods work because if one element in the
+pipeline is not handling them correctly the whole event system of the
+pipeline is broken. We will try to explain here how these methods work
+and how elements are supposed to implement them.
+
+## Downstream events
+
+Downstream events are received through the sink pad's event handler, as
+set using `gst_pad_set_event_function ()` when the pad was created.
+
+Downstream events can travel in two ways: they can be in-band
+(serialised with the buffer flow) or out-of-band (travelling through the
+pipeline instantly, possibly not in the same thread as the streaming
+thread that is processing the buffers, skipping ahead of buffers being
+processed or queued in the pipeline). The most common downstream events
+(SEGMENT, CAPS, TAG, EOS) are all serialised with the buffer flow.
+
+Here is a typical event function:
+
+``` c
+static gboolean
+gst_my_filter_sink_event (GstPad *pad, GstObject * parent, GstEvent * event)
+{
+ GstMyFilter *filter;
+ gboolean ret;
+
+ filter = GST_MY_FILTER (parent);
+ ...
+
+ switch (GST_EVENT_TYPE (event)) {
+ case GST_EVENT_SEGMENT:
+ /* maybe save and/or update the current segment (e.g. for output
+ * clipping) or convert the event into one in a different format
+ * (e.g. BYTES to TIME) or drop it and set a flag to send a segment
+ * event in a different format later */
+ ret = gst_pad_push_event (filter->src_pad, event);
+ break;
+ case GST_EVENT_EOS:
+ /* end-of-stream, we should close down all stream leftovers here */
+ gst_my_filter_stop_processing (filter);
+ ret = gst_pad_push_event (filter->src_pad, event);
+ break;
+ case GST_EVENT_FLUSH_STOP:
+ gst_my_filter_clear_temporary_buffers (filter);
+ ret = gst_pad_push_event (filter->src_pad, event);
+ break;
+ default:
+ ret = gst_pad_event_default (pad, parent, event);
+ break;
+ }
+
+ ...
+ return ret;
+}
+
+```
+
+If your element is chain-based, you will almost always have to implement
+a sink event function, since that is how you are notified about
+segments, caps and the end of the stream.
+
+If your element is exclusively loop-based, you may or may not want a
+sink event function (since the element is driving the pipeline it will
+know the length of the stream in advance or be notified by the flow
+return value of `gst_pad_pull_range()`. In some cases even loop-based
+element may receive events from upstream though (for example audio
+decoders with an id3demux or apedemux element in front of them, or
+demuxers that are being fed input from sources that send additional
+information about the stream in custom events, as DVD sources do).
+
+## Upstream events
+
+Upstream events are generated by an element somewhere downstream in the
+pipeline (example: a video sink may generate navigation events that
+informs upstream elements about the current position of the mouse
+pointer). This may also happen indirectly on request of the application,
+for example when the application executes a seek on a pipeline this seek
+request will be passed on to a sink element which will then in turn
+generate an upstream seek event.
+
+The most common upstream events are seek events, Quality-of-Service
+(QoS) and reconfigure events.
+
+An upstream event can be sent using the `gst_pad_send_event` function.
+This function simply call the default event handler of that pad. The
+default event handler of pads is `gst_pad_event_default`, and it
+basically sends the event to the peer of the internally linked pad. So
+upstream events always arrive on the src pad of your element and are
+handled by the default event handler except if you override that handler
+to handle it yourself. There are some specific cases where you have to
+do that :
+
+ - If you have multiple sink pads in your element. In that case you
+ will have to decide which one of the sink pads you will send the
+ event to (if not all of them).
+
+ - If you need to handle that event locally. For example a navigation
+ event that you will want to convert before sending it upstream, or a
+ QoS event that you want to handle.
+
+The processing you will do in that event handler does not really matter
+but there are important rules you have to absolutely respect because one
+broken element event handler is breaking the whole pipeline event
+handling. Here they are :
+
+ - Always handle events you won't handle using the default
+ `gst_pad_event_default` method. This method will depending on the
+ event, forward the event or drop it.
+
+ - If you are generating some new event based on the one you received
+ don't forget to gst\_event\_unref the event you received.
+
+ - Event handler function are supposed to return TRUE or FALSE
+ indicating if the event has been handled or not. Never simply return
+ TRUE/FALSE in that handler except if you really know that you have
+ handled that event.
+
+ - Remember that the event handler might be called from a different
+ thread than the streaming thread, so make sure you use appropriate
+ locking everywhere.
+
+## All Events Together
+
+In this chapter follows a list of all defined events that are currently
+being used, plus how they should be used/interpreted. You can check the
+what type a certain event is using the GST\_EVENT\_TYPE macro (or if you
+need a string for debugging purposes you can use
+GST\_EVENT\_TYPE\_NAME).
+
+In this chapter, we will discuss the following events:
+
+ - [Stream Start](#stream-start)
+
+ - [Caps](#caps)
+
+ - [Segment](#segment)
+
+ - [Tag (metadata)](#tag-metadata)
+
+ - [End of Stream (EOS)](#end-of-stream-eos)
+
+ - [Table Of Contents](#table-of-contents)
+
+ - [Gap](#gap)
+
+ - [Flush Start](#flush-start)
+
+ - [Flush Stop](#flush-stop)
+
+ - [Quality Of Service (QOS)](#quality-of-service-qos)
+
+ - [Seek Request](#seek-request)
+
+ - [Navigation](#navigation)
+
+For more comprehensive information about events and how they should be
+used correctly in various circumstances please consult the GStreamer
+design documentation. This section only gives a general overview.
+
+### Stream Start
+
+WRITEME
+
+### Caps
+
+The CAPS event contains the format description of the following buffers.
+See [Caps negotiation](pwg-negotiation.md) for more information
+about negotiation.
+
+### Segment
+
+A segment event is sent downstream to announce the range of valid
+timestamps in the stream and how they should be transformed into
+running-time and stream-time. A segment event must always be sent before
+the first buffer of data and after a flush (see above).
+
+The first segment event is created by the element driving the pipeline,
+like a source operating in push-mode or a demuxer/decoder operating
+pull-based. This segment event then travels down the pipeline and may be
+transformed on the way (a decoder, for example, might receive a segment
+event in BYTES format and might transform this into a segment event in
+TIMES format based on the average bitrate).
+
+Depending on the element type, the event can simply be forwarded using
+`gst_pad_event_default ()`, or it should be parsed and a modified event
+should be sent on. The last is true for demuxers, which generally have a
+byte-to-time conversion concept. Their input is usually byte-based, so
+the incoming event will have an offset in byte units
+(`GST_FORMAT_BYTES`), too. Elements downstream, however, expect segment
+events in time units, so that it can be used to synchronize against the
+pipeline clock. Therefore, demuxers and similar elements should not
+forward the event, but parse it, free it and send a segment event (in
+time units, `GST_FORMAT_TIME`) further downstream.
+
+The segment event is created using the function `gst_event_new_segment
+()`. See the API reference and design document for details about its
+parameters.
+
+Elements parsing this event can use gst\_event\_parse\_segment() to
+extract the event details. Elements may find the GstSegment API useful
+to keep track of the current segment (if they want to use it for output
+clipping, for example).
+
+### Tag (metadata)
+
+Tagging events are being sent downstream to indicate the tags as parsed
+from the stream data. This is currently used to preserve tags during
+stream transcoding from one format to the other. Tags are discussed
+extensively in [Tagging (Metadata and
+Streaminfo)](pwg-advanced-tagging.md). Most elements will simply
+forward the event by calling `gst_pad_event_default ()`.
+
+The tag event is created using the function `gst_event_new_tag ()`, but
+more often elements will send a tag event downstream that will be
+converted into a message on the bus by sink elements. All of these
+functions require a filled-in taglist as argument, which they will take
+ownership of.
+
+Elements parsing this event can use the function `gst_event_parse_tag
+()` to acquire the taglist that the event contains.
+
+### End of Stream (EOS)
+
+End-of-stream events are sent if the stream that an element sends out is
+finished. An element receiving this event (from upstream, so it receives
+it on its sinkpad) will generally just process any buffered data (if
+there is any) and then forward the event further downstream. The
+`gst_pad_event_default ()` takes care of all this, so most elements do
+not need to support this event. Exceptions are elements that explicitly
+need to close a resource down on EOS, and N-to-1 elements. Note that the
+stream itself is *not* a resource that should be closed down on EOS\!
+Applications might seek back to a point before EOS and continue playing
+again.
+
+The EOS event has no properties, which makes it one of the simplest
+events in GStreamer. It is created using the `gst_event_new_eos()`
+function.
+
+It is important to note that *only elements driving the pipeline should
+ever send an EOS event*. If your element is chain-based, it is not
+driving the pipeline. Chain-based elements should just return
+GST\_FLOW\_EOS from their chain function at the end of the stream (or
+the configured segment), the upstream element that is driving the
+pipeline will then take care of sending the EOS event (or alternatively
+post a SEGMENT\_DONE message on the bus depending on the mode of
+operation). If you are implementing your own source element, you also do
+not need to ever manually send an EOS event, you should also just return
+GST\_FLOW\_EOS in your create or fill function (assuming your element
+derives from GstBaseSrc or GstPushSrc).
+
+### Table Of Contents
+
+WRITEME
+
+### Gap
+
+WRITEME
+
+### Flush Start
+
+The flush start event is sent downstream (in push mode) or upstream (in
+pull mode) if all buffers and caches in the pipeline should be emptied.
+“Queue” elements will empty their internal list of buffers when they
+receive this event, for example. File sink elements (e.g. “filesink”)
+will flush the kernel-to-disk cache (`fdatasync ()` or `fflush ()`) when
+they receive this event. Normally, elements receiving this event will
+simply just forward it, since most filter or filter-like elements don't
+have an internal cache of data. `gst_pad_event_default ()` does just
+that, so for most elements, it is enough to forward the event using the
+default event handler.
+
+As a side-effect of flushing all data from the pipeline, this event
+unblocks the streaming thread by making all pads reject data until they
+receive a [Flush Stop](#flush-stop) signal (elements trying to push data
+will get a FLUSHING flow return and stop processing data).
+
+The flush-start event is created with the `gst_event_new_flush_start
+()`. Like the EOS event, it has no properties. This event is usually
+only created by elements driving the pipeline, like source elements
+operating in push-mode or pull-range based demuxers/decoders.
+
+### Flush Stop
+
+The flush-stop event is sent by an element driving the pipeline after a
+flush-start and tells pads and elements downstream that they should
+accept events and buffers again (there will be at least a SEGMENT event
+before any buffers first though).
+
+If your element keeps temporary caches of stream data, it should clear
+them when it receives a FLUSH-STOP event (and also whenever its chain
+function receives a buffer with the DISCONT flag set).
+
+The flush-stop event is created with `gst_event_new_flush_stop ()`. It
+has one parameter that controls if the running-time of the pipeline
+should be reset to 0 or not. Normally after a flushing seek, the
+running\_time is set back to 0.
+
+### Quality Of Service (QOS)
+
+The QOS event contains a report about the current real-time performance
+of the stream. See more info in [Quality Of Service
+(QoS)](pwg-advanced-qos.md).
+
+### Seek Request
+
+Seek events are meant to request a new stream position to elements. This
+new position can be set in several formats (time, bytes or “default
+units” \[a term indicating frames for video, channel-independent samples
+for audio, etc.\]). Seeking can be done with respect to the end-of-file
+or start-of-file, and usually happens in upstream direction (downstream
+seeking is done by sending a SEGMENT event with the appropriate offsets
+for elements that support that, like filesink).
+
+Elements receiving seek events should, depending on the element type,
+either just forward it upstream (filters, decoders), change the format
+in which the event is given and then forward it (demuxers), or handle
+the event by changing the file pointer in their internal stream resource
+(file sources, demuxers/decoders driving the pipeline in pull-mode) or
+something else.
+
+Seek events are built up using positions in specified formats (time,
+bytes, units). They are created using the function `gst_event_new_seek
+()`. Note that many plugins do not support seeking from the end of the
+stream. An element not driving the pipeline and forwarding a seek
+request should not assume that the seek succeeded or actually happened,
+it should operate based on the SEGMENT events it receives.
+
+Elements parsing this event can do this using `gst_event_parse_seek()`.
+
+### Navigation
+
+Navigation events are sent upstream by video sinks to inform upstream
+elements of where the mouse pointer is, if and where mouse pointer
+clicks have happened, or if keys have been pressed or released.
+
+All this information is contained in the event structure which can be
+obtained with `gst_event_get_structure ()`.
+
+Check out the navigationtest element in gst-plugins-good for an idea how
+to extract navigation information from this event.
+
--- /dev/null
+---
+title: Interfaces
+...
+
+# Interfaces
+
+Previously, in the chapter [Adding
+Properties](pwg-building-args.md), we have introduced the concept of
+GObject properties of controlling an element's behaviour. This is very
+powerful, but it has two big disadvantages: first of all, it is too
+generic, and second, it isn't dynamic.
+
+The first disadvantage is related to the customizability of the end-user
+interface that will be built to control the element. Some properties are
+more important than others. Some integer properties are better shown in
+a spin-button widget, whereas others would be better represented by a
+slider widget. Such things are not possible because the UI has no actual
+meaning in the application. A UI widget that represents a bitrate
+property is the same as a UI widget that represents the size of a video,
+as long as both are of the same `GParamSpec` type. Another problem, is
+that things like parameter grouping, function grouping, or parameter
+coupling are not really possible.
+
+The second problem with parameters are that they are not dynamic. In
+many cases, the allowed values for a property are not fixed, but depend
+on things that can only be detected at runtime. The names of inputs for
+a TV card in a video4linux source element, for example, can only be
+retrieved from the kernel driver when we've opened the device; this only
+happens when the element goes into the READY state. This means that we
+cannot create an enum property type to show this to the user.
+
+The solution to those problems is to create very specialized types of
+controls for certain often-used controls. We use the concept of
+interfaces to achieve this. The basis of this all is the glib
+`GTypeInterface` type. For each case where we think it's useful, we've
+created interfaces which can be implemented by elements at their own
+will.
+
+One important note: interfaces do *not* replace properties. Rather,
+interfaces should be built *next to* properties. There are two important
+reasons for this. First of all, properties can be more easily
+introspected. Second, properties can be specified on the commandline
+(`gst-launch`).
+
+## How to Implement Interfaces
+
+Implementing interfaces is initiated in the `_get_type ()` of your
+element. You can register one or more interfaces after having registered
+the type itself. Some interfaces have dependencies on other interfaces
+or can only be registered by certain types of elements. You will be
+notified of doing that wrongly when using the element: it will quit with
+failed assertions, which will explain what went wrong. If it does, you
+need to register support for *that* interface before registering support
+for the interface that you're wanting to support. The example below
+explains how to add support for a simple interface with no further
+dependencies.
+
+``` c
+static void gst_my_filter_some_interface_init (GstSomeInterface *iface);
+
+GType
+gst_my_filter_get_type (void)
+{
+ static GType my_filter_type = 0;
+
+ if (!my_filter_type) {
+ static const GTypeInfo my_filter_info = {
+ sizeof (GstMyFilterClass),
+ NULL,
+ NULL,
+ (GClassInitFunc) gst_my_filter_class_init,
+ NULL,
+ NULL,
+ sizeof (GstMyFilter),
+ 0,
+ (GInstanceInitFunc) gst_my_filter_init
+ };
+ static const GInterfaceInfo some_interface_info = {
+ (GInterfaceInitFunc) gst_my_filter_some_interface_init,
+ NULL,
+ NULL
+ };
+
+ my_filter_type =
+ g_type_register_static (GST_TYPE_ELEMENT,
+ "GstMyFilter",
+ &my_filter_info, 0);
+ g_type_add_interface_static (my_filter_type,
+ GST_TYPE_SOME_INTERFACE,
+ &some_interface_info);
+ }
+
+ return my_filter_type;
+}
+
+static void
+gst_my_filter_some_interface_init (GstSomeInterface *iface)
+{
+ /* here, you would set virtual function pointers in the interface */
+}
+
+```
+
+Or more
+conveniently:
+
+``` c
+static void gst_my_filter_some_interface_init (GstSomeInterface *iface);
+
+G_DEFINE_TYPE_WITH_CODE (GstMyFilter, gst_my_filter,GST_TYPE_ELEMENT,
+ G_IMPLEMENT_INTERFACE (GST_TYPE_SOME_INTERFACE,
+ gst_my_filter_some_interface_init));
+
+
+```
+
+## URI interface
+
+WRITEME
+
+## Color Balance Interface
+
+WRITEME
+
+## Video Overlay Interface
+
+The \#GstVideoOverlay interface is used for 2 main purposes :
+
+ - To get a grab on the Window where the video sink element is going to
+ render. This is achieved by either being informed about the Window
+ identifier that the video sink element generated, or by forcing the
+ video sink element to use a specific Window identifier for
+ rendering.
+
+ - To force a redrawing of the latest video frame the video sink
+ element displayed on the Window. Indeed if the \#GstPipeline is in
+ \#GST\_STATE\_PAUSED state, moving the Window around will damage its
+ content. Application developers will want to handle the Expose
+ events themselves and force the video sink element to refresh the
+ Window's content.
+
+A plugin drawing video output in a video window will need to have that
+window at one stage or another. Passive mode simply means that no window
+has been given to the plugin before that stage, so the plugin created
+the window by itself. In that case the plugin is responsible of
+destroying that window when it's not needed any more and it has to tell
+the applications that a window has been created so that the application
+can use it. This is done using the `have-window-handle` message that can
+be posted from the plugin with the `gst_video_overlay_got_window_handle`
+method.
+
+As you probably guessed already active mode just means sending a video
+window to the plugin so that video output goes there. This is done using
+the `gst_video_overlay_set_window_handle` method.
+
+It is possible to switch from one mode to another at any moment, so the
+plugin implementing this interface has to handle all cases. There are
+only 2 methods that plugins writers have to implement and they most
+probably look like that :
+
+``` c
+static void
+gst_my_filter_set_window_handle (GstVideoOverlay *overlay, guintptr handle)
+{
+ GstMyFilter *my_filter = GST_MY_FILTER (overlay);
+
+ if (my_filter->window)
+ gst_my_filter_destroy_window (my_filter->window);
+
+ my_filter->window = handle;
+}
+
+static void
+gst_my_filter_xoverlay_init (GstVideoOverlayClass *iface)
+{
+ iface->set_window_handle = gst_my_filter_set_window_handle;
+}
+
+```
+
+You will also need to use the interface methods to post messages when
+needed such as when receiving a CAPS event where you will know the video
+geometry and maybe create the window.
+
+``` c
+static MyFilterWindow *
+gst_my_filter_window_create (GstMyFilter *my_filter, gint width, gint height)
+{
+ MyFilterWindow *window = g_new (MyFilterWindow, 1);
+ ...
+ gst_video_overlay_got_window_handle (GST_VIDEO_OVERLAY (my_filter), window->win);
+}
+
+/* called from the event handler for CAPS events */
+static gboolean
+gst_my_filter_sink_set_caps (GstMyFilter *my_filter, GstCaps *caps)
+{
+ gint width, height;
+ gboolean ret;
+ ...
+ ret = gst_structure_get_int (structure, "width", &width);
+ ret &= gst_structure_get_int (structure, "height", &height);
+ if (!ret) return FALSE;
+
+ gst_video_overlay_prepare_window_handle (GST_VIDEO_OVERLAY (my_filter));
+
+ if (!my_filter->window)
+ my_filter->window = gst_my_filter_create_window (my_filter, width, height);
+
+ ...
+}
+
+```
+
+## Navigation Interface
+
+WRITEME
+
--- /dev/null
+---
+title: Quality Of Service (QoS)
+...
+
+# Quality Of Service (QoS)
+
+Quality of Service in GStreamer is about measuring and adjusting the
+real-time performance of a pipeline. The real-time performance is always
+measured relative to the pipeline clock and typically happens in the
+sinks when they synchronize buffers against the clock.
+
+When buffers arrive late in the sink, i.e. when their running-time is
+smaller than that of the clock, we say that the pipeline is having a
+quality of service problem. These are a few possible reasons:
+
+ - High CPU load, there is not enough CPU power to handle the stream,
+ causing buffers to arrive late in the sink.
+
+ - Network problems
+
+ - Other resource problems such as disk load, memory bottlenecks etc
+
+The measurements result in QOS events that aim to adjust the datarate in
+one or more upstream elements. Two types of adjustments can be made:
+
+ - Short time "emergency" corrections based on latest observation in
+ the sinks.
+
+ Long term rate corrections based on trends observed in the sinks.
+
+It is also possible for the application to artificially introduce delay
+between synchronized buffers, this is called throttling. It can be used
+to limit or reduce the framerate, for example.
+
+## Measuring QoS
+
+Elements that synchronize buffers on the pipeline clock will usually
+measure the current QoS. They will also need to keep some statistics in
+order to generate the QOS event.
+
+For each buffer that arrives in the sink, the element needs to calculate
+how late or how early it was. This is called the jitter. Negative jitter
+values mean that the buffer was early, positive values mean that the
+buffer was late. the jitter value gives an indication of how early/late
+a buffer was.
+
+A synchronizing element will also need to calculate how much time
+elapsed between receiving two consecutive buffers. We call this the
+processing time because that is the amount of time it takes for the
+upstream element to produce/process the buffer. We can compare this
+processing time to the duration of the buffer to have a measurement of
+how fast upstream can produce data, called the proportion. If, for
+example, upstream can produce a buffer in 0.5 seconds of 1 second long,
+it is operating at twice the required speed. If, on the other hand, it
+takes 2 seconds to produce a buffer with 1 seconds worth of data,
+upstream is producing buffers too slow and we won't be able to keep
+synchronization. Usually, a running average is kept of the proportion.
+
+A synchronizing element also needs to measure its own performance in
+order to figure out if the performance problem is upstream of itself.
+
+These measurements are used to construct a QOS event that is sent
+upstream. Note that a QoS event is sent for each buffer that arrives in
+the sink.
+
+## Handling QoS
+
+An element will have to install an event function on its source pads in
+order to receive QOS events. Usually, the element will need to store the
+value of the QOS event and use them in the data processing function. The
+element will need to use a lock to protect these QoS values as shown in
+the example below. Also make sure to pass the QoS event upstream.
+
+``` c
+
+ [...]
+
+ case GST_EVENT_QOS:
+ {
+ GstQOSType type;
+ gdouble proportion;
+ GstClockTimeDiff diff;
+ GstClockTime timestamp;
+
+ gst_event_parse_qos (event, &type, &proportion, &diff, ×tamp);
+
+ GST_OBJECT_LOCK (decoder);
+ priv->qos_proportion = proportion;
+ priv->qos_timestamp = timestamp;
+ priv->qos_diff = diff;
+ GST_OBJECT_UNLOCK (decoder);
+
+ res = gst_pad_push_event (decoder->sinkpad, event);
+ break;
+ }
+
+ [...]
+
+
+```
+
+With the QoS values, there are two types of corrections that an element
+can do:
+
+### Short term correction
+
+The timestamp and the jitter value in the QOS event can be used to
+perform a short term correction. If the jitter is positive, the previous
+buffer arrived late and we can be sure that a buffer with a timestamp \<
+timestamp + jitter is also going to be late. We can thus drop all
+buffers with a timestamp less than timestamp + jitter.
+
+If the buffer duration is known, a better estimation for the next likely
+timestamp as: timestamp + 2 \* jitter + duration.
+
+A possible algorithm typically looks like this:
+
+``` c
+
+ [...]
+
+ GST_OBJECT_LOCK (dec);
+ qos_proportion = priv->qos_proportion;
+ qos_timestamp = priv->qos_timestamp;
+ qos_diff = priv->qos_diff;
+ GST_OBJECT_UNLOCK (dec);
+
+ /* calculate the earliest valid timestamp */
+ if (G_LIKELY (GST_CLOCK_TIME_IS_VALID (qos_timestamp))) {
+ if (G_UNLIKELY (qos_diff > 0)) {
+ earliest_time = qos_timestamp + 2 * qos_diff + frame_duration;
+ } else {
+ earliest_time = qos_timestamp + qos_diff;
+ }
+ } else {
+ earliest_time = GST_CLOCK_TIME_NONE;
+ }
+
+ /* compare earliest_time to running-time of next buffer */
+ if (earliest_time > timestamp)
+ goto drop_buffer;
+
+ [...]
+
+
+```
+
+### Long term correction
+
+Long term corrections are a bit more difficult to perform. They rely on
+the value of the proportion in the QOS event. Elements should reduce the
+amount of resources they consume by the proportion field in the QoS
+message.
+
+Here are some possible strategies to achieve this:
+
+ - Permanently dropping frames or reducing the CPU or bandwidth
+ requirements of the element. Some decoders might be able to skip
+ decoding of B frames.
+
+ - Switch to lower quality processing or reduce the algorithmic
+ complexity. Care should be taken that this doesn't introduce
+ disturbing visual or audible glitches.
+
+ - Switch to a lower quality source to reduce network bandwidth.
+
+ - Assign more CPU cycles to critical parts of the pipeline. This
+ could, for example, be done by increasing the thread priority.
+
+In all cases, elements should be prepared to go back to their normal
+processing rate when the proportion member in the QOS event approaches
+the ideal proportion of 1.0 again.
+
+## Throttling
+
+Elements synchronizing to the clock should expose a property to
+configure them in throttle mode. In throttle mode, the time distance
+between buffers is kept to a configurable throttle interval. This means
+that effectively the buffer rate is limited to 1 buffer per throttle
+interval. This can be used to limit the framerate, for example.
+
+When an element is configured in throttling mode (this is usually only
+implemented on sinks) it should produce QoS events upstream with the
+jitter field set to the throttle interval. This should instruct upstream
+elements to skip or drop the remaining buffers in the configured
+throttle interval.
+
+The proportion field is set to the desired slowdown needed to get the
+desired throttle interval. Implementations can use the QoS Throttle
+type, the proportion and the jitter member to tune their
+implementations.
+
+The default sink base class, has the “throttle-time” property for this
+feature. You can test this with: `gst-launch-1.0 videotestsrc !
+xvimagesink throttle-time=500000000`
+
+## QoS Messages
+
+In addition to the QOS events that are sent between elements in the
+pipeline, there are also QOS messages posted on the pipeline bus to
+inform the application of QoS decisions. The QOS message contains the
+timestamps of when something was dropped along with the amount of
+dropped vs processed items. Elements must post a QOS message under these
+conditions:
+
+ - The element dropped a buffer because of QoS reasons.
+
+ - An element changes its processing strategy because of QoS reasons
+ (quality). This could include a decoder that decides to drop every B
+ frame to increase its processing speed or an effect element
+ switching to a lower quality algorithm.
+
--- /dev/null
+---
+title: Request and Sometimes pads
+...
+
+# Request and Sometimes pads
+
+Until now, we've only dealt with pads that are always available.
+However, there's also pads that are only being created in some cases, or
+only if the application requests the pad. The first is called a
+*sometimes*; the second is called a *request* pad. The availability of a
+pad (always, sometimes or request) can be seen in a pad's template. This
+chapter will discuss when each of the two is useful, how they are
+created and when they should be disposed.
+
+## Sometimes pads
+
+A “sometimes” pad is a pad that is created under certain conditions, but
+not in all cases. This mostly depends on stream content: demuxers will
+generally parse the stream header, decide what elementary (video, audio,
+subtitle, etc.) streams are embedded inside the system stream, and will
+then create a sometimes pad for each of those elementary streams. At its
+own choice, it can also create more than one instance of each of those
+per element instance. The only limitation is that each newly created pad
+should have a unique name. Sometimes pads are disposed when the stream
+data is disposed, too (i.e. when going from PAUSED to the READY state).
+You should *not* dispose the pad on EOS, because someone might
+re-activate the pipeline and seek back to before the end-of-stream
+point. The stream should still stay valid after EOS, at least until the
+stream data is disposed. In any case, the element is always the owner of
+such a pad.
+
+The example code below will parse a text file, where the first line is a
+number (n). The next lines all start with a number (0 to n-1), which is
+the number of the source pad over which the data should be sent.
+
+```
+3
+0: foo
+1: bar
+0: boo
+2: bye
+
+```
+
+The code to parse this file and create the dynamic “sometimes” pads,
+looks like this:
+
+``` c
+
+typedef struct _GstMyFilter {
+[..]
+ gboolean firstrun;
+ GList *srcpadlist;
+} GstMyFilter;
+
+static GstStaticPadTemplate src_factory =
+GST_STATIC_PAD_TEMPLATE (
+ "src_%u",
+ GST_PAD_SRC,
+ GST_PAD_SOMETIMES,
+ GST_STATIC_CAPS ("ANY")
+);
+
+static void
+gst_my_filter_class_init (GstMyFilterClass *klass)
+{
+ GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
+[..]
+ gst_element_class_add_pad_template (element_class,
+ gst_static_pad_template_get (&src_factory));
+[..]
+}
+
+static void
+gst_my_filter_init (GstMyFilter *filter)
+{
+[..]
+ filter->firstrun = TRUE;
+ filter->srcpadlist = NULL;
+}
+
+/*
+ * Get one line of data - without newline.
+ */
+
+static GstBuffer *
+gst_my_filter_getline (GstMyFilter *filter)
+{
+ guint8 *data;
+ gint n, num;
+
+ /* max. line length is 512 characters - for safety */
+ for (n = 0; n < 512; n++) {
+ num = gst_bytestream_peek_bytes (filter->bs, &data, n + 1);
+ if (num != n + 1)
+ return NULL;
+
+ /* newline? */
+ if (data[n] == '\n') {
+ GstBuffer *buf = gst_buffer_new_allocate (NULL, n + 1, NULL);
+
+ gst_bytestream_peek_bytes (filter->bs, &data, n);
+ gst_buffer_fill (buf, 0, data, n);
+ gst_buffer_memset (buf, n, '\0', 1);
+ gst_bytestream_flush_fast (filter->bs, n + 1);
+
+ return buf;
+ }
+ }
+}
+
+static void
+gst_my_filter_loopfunc (GstElement *element)
+{
+ GstMyFilter *filter = GST_MY_FILTER (element);
+ GstBuffer *buf;
+ GstPad *pad;
+ GstMapInfo map;
+ gint num, n;
+
+ /* parse header */
+ if (filter->firstrun) {
+ gchar *padname;
+ guint8 id;
+
+ if (!(buf = gst_my_filter_getline (filter))) {
+ gst_element_error (element, STREAM, READ, (NULL),
+ ("Stream contains no header"));
+ return;
+ }
+ gst_buffer_extract (buf, 0, &id, 1);
+ num = atoi (id);
+ gst_buffer_unref (buf);
+
+ /* for each of the streams, create a pad */
+ for (n = 0; n < num; n++) {
+ padname = g_strdup_printf ("src_%u", n);
+ pad = gst_pad_new_from_static_template (src_factory, padname);
+ g_free (padname);
+
+ /* here, you would set _event () and _query () functions */
+
+ /* need to activate the pad before adding */
+ gst_pad_set_active (pad, TRUE);
+
+ gst_element_add_pad (element, pad);
+ filter->srcpadlist = g_list_append (filter->srcpadlist, pad);
+ }
+ }
+
+ /* and now, simply parse each line and push over */
+ if (!(buf = gst_my_filter_getline (filter))) {
+ GstEvent *event = gst_event_new (GST_EVENT_EOS);
+ GList *padlist;
+
+ for (padlist = srcpadlist;
+ padlist != NULL; padlist = g_list_next (padlist)) {
+ pad = GST_PAD (padlist->data);
+ gst_pad_push_event (pad, gst_event_ref (event));
+ }
+ gst_event_unref (event);
+ /* pause the task here */
+ return;
+ }
+
+ /* parse stream number and go beyond the ':' in the data */
+ gst_buffer_map (buf, &map, GST_MAP_READ);
+ num = atoi (map.data[0]);
+ if (num >= 0 && num < g_list_length (filter->srcpadlist)) {
+ pad = GST_PAD (g_list_nth_data (filter->srcpadlist, num);
+
+ /* magic buffer parsing foo */
+ for (n = 0; map.data[n] != ':' &&
+ map.data[n] != '\0'; n++) ;
+ if (map.data[n] != '\0') {
+ GstBuffer *sub;
+
+ /* create region copy that starts right past the space. The reason
+ * that we don't just forward the data pointer is because the
+ * pointer is no longer the start of an allocated block of memory,
+ * but just a pointer to a position somewhere in the middle of it.
+ * That cannot be freed upon disposal, so we'd either crash or have
+ * a memleak. Creating a region copy is a simple way to solve that. */
+ sub = gst_buffer_copy_region (buf, GST_BUFFER_COPY_ALL,
+ n + 1, map.size - n - 1);
+ gst_pad_push (pad, sub);
+ }
+ }
+ gst_buffer_unmap (buf, &map);
+ gst_buffer_unref (buf);
+}
+
+
+```
+
+Note that we use a lot of checks everywhere to make sure that the
+content in the file is valid. This has two purposes: first, the file
+could be erroneous, in which case we prevent a crash. The second and
+most important reason is that - in extreme cases - the file could be
+used maliciously to cause undefined behaviour in the plugin, which might
+lead to security issues. *Always* assume that the file could be used to
+do bad things.
+
+## Request pads
+
+“Request” pads are similar to sometimes pads, except that request are
+created on demand of something outside of the element rather than
+something inside the element. This concept is often used in muxers,
+where - for each elementary stream that is to be placed in the output
+system stream - one sink pad will be requested. It can also be used in
+elements with a variable number of input or outputs pads, such as the
+`tee` (multi-output) or `input-selector` (multi-input) elements.
+
+To implement request pads, you need to provide a padtemplate with a
+GST\_PAD\_REQUEST presence and implement the `request_new_pad` virtual
+method in `GstElement`. To clean up, you will need to implement the
+`release_pad` virtual method.
+
+``` c
+
+static GstPad * gst_my_filter_request_new_pad (GstElement *element,
+ GstPadTemplate *templ,
+ const gchar *name,
+ const GstCaps *caps);
+
+static void gst_my_filter_release_pad (GstElement *element,
+ GstPad *pad);
+
+static GstStaticPadTemplate sink_factory =
+GST_STATIC_PAD_TEMPLATE (
+ "sink_%u",
+ GST_PAD_SINK,
+ GST_PAD_REQUEST,
+ GST_STATIC_CAPS ("ANY")
+);
+
+static void
+gst_my_filter_class_init (GstMyFilterClass *klass)
+{
+ GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
+[..]
+ gst_element_class_add_pad_template (klass,
+ gst_static_pad_template_get (&sink_factory));
+[..]
+ element_class->request_new_pad = gst_my_filter_request_new_pad;
+ element_class->release_pad = gst_my_filter_release_pad;
+}
+
+static GstPad *
+gst_my_filter_request_new_pad (GstElement *element,
+ GstPadTemplate *templ,
+ const gchar *name,
+ const GstCaps *caps)
+{
+ GstPad *pad;
+ GstMyFilterInputContext *context;
+
+ context = g_new0 (GstMyFilterInputContext, 1);
+ pad = gst_pad_new_from_template (templ, name);
+ gst_pad_set_element_private (pad, context);
+
+ /* normally, you would set _chain () and _event () functions here */
+
+ gst_element_add_pad (element, pad);
+
+ return pad;
+}
+
+static void
+gst_my_filter_release_pad (GstElement *element,
+ GstPad *pad)
+{
+ GstMyFilterInputContext *context;
+
+ context = gst_pad_get_element_private (pad);
+ g_free (context);
+
+ gst_element_remove_pad (element, pad);
+}
+
+
+
+```
+
--- /dev/null
+---
+title: Tagging (Metadata and Streaminfo)
+...
+
+# Tagging (Metadata and Streaminfo)
+
+## Overview
+
+Tags are pieces of information stored in a stream that are not the
+content itself, but they rather *describe* the content. Most media
+container formats support tagging in one way or another. Ogg uses
+VorbisComment for this, MP3 uses ID3, AVI and WAV use RIFF's INFO list
+chunk, etc. GStreamer provides a general way for elements to read tags
+from the stream and expose this to the user. The tags (at least the
+metadata) will be part of the stream inside the pipeline. The
+consequence of this is that transcoding of files from one format to
+another will automatically preserve tags, as long as the input and
+output format elements both support tagging.
+
+Tags are separated in two categories in GStreamer, even though
+applications won't notice anything of this. The first are called
+*metadata*, the second are called *streaminfo*. Metadata are tags that
+describe the non-technical parts of stream content. They can be changed
+without needing to re-encode the stream completely. Examples are
+“author”, “title” or “album”. The container format might still need
+to be re-written for the tags to fit in, though. Streaminfo, on the
+other hand, are tags that describe the stream contents technically. To
+change them, the stream needs to be re-encoded. Examples are “codec” or
+“bitrate”. Note that some container formats (like ID3) store various
+streaminfo tags as metadata in the file container, which means that they
+can be changed so that they don't match the content in the file any
+more. Still, they are called metadata because *technically*, they can be
+changed without re-encoding the whole stream, even though that makes
+them invalid. Files with such metadata tags will have the same tag
+twice: once as metadata, once as streaminfo.
+
+There is no special name for tag reading elements in GStreamer. There
+are specialised elements (e.g. id3demux) that do nothing besides tag
+reading, but any GStreamer element may extract tags while processing
+data, and most decoders, demuxers and parsers do.
+
+A tag writer is called
+[`TagSetter`](../../gstreamer/html/GstTagSetter.html). An element
+supporting both can be used in a tag editor for quick tag changing
+(note: in-place tag editing is still poorly supported at the time of
+writing and usually requires tag extraction/stripping and remuxing of
+the stream with new tags).
+
+## Reading Tags from Streams
+
+The basic object for tags is a [`GstTagList
+`](../../gstreamer/html/GstTagList.html). An element that is reading
+tags from a stream should create an empty taglist and fill this with
+individual tags. Empty tag lists can be created with `gst_tag_list_new
+()`. Then, the element can fill the list using `gst_tag_list_add ()
+` or `gst_tag_list_add_values ()`. Note that elements often read
+metadata as strings, but the values in the taglist might not necessarily
+be strings - they need to be of the type the tag was registered as (the
+API documentation for each predefined tag should contain the type). Be
+sure to use functions like `gst_value_transform ()` to make sure that
+your data is of the right type. After data reading, you can send the
+tags downstream with the TAG event. When the TAG event reaches the sink,
+it will post the TAG message on the pipeline's GstBus for the
+application to pick up.
+
+We currently require the core to know the GType of tags before they are
+being used, so all tags must be registered first. You can add new tags
+to the list of known tags using `gst_tag_register ()`. If you think the
+tag will be useful in more cases than just your own element, it might be
+a good idea to add it to `gsttag.c` instead. That's up to you to decide.
+If you want to do it in your own element, it's easiest to register the
+tag in one of your class init functions, preferably `_class_init ()`.
+
+``` c
+
+static void
+gst_my_filter_class_init (GstMyFilterClass *klass)
+{
+[..]
+ gst_tag_register ("my_tag_name", GST_TAG_FLAG_META,
+ G_TYPE_STRING,
+ _("my own tag"),
+ _("a tag that is specific to my own element"),
+ NULL);
+[..]
+}
+
+
+```
+
+## Writing Tags to Streams
+
+Tag writers are the opposite of tag readers. Tag writers only take
+metadata tags into account, since that's the only type of tags that have
+to be written into a stream. Tag writers can receive tags in three ways:
+internal, application and pipeline. Internal tags are tags read by the
+element itself, which means that the tag writer is - in that case - a
+tag reader, too. Application tags are tags provided to the element via
+the TagSetter interface (which is just a layer). Pipeline tags are tags
+provided to the element from within the pipeline. The element receives
+such tags via the `GST_EVENT_TAG` event, which means that tags writers
+should implement an event handler. The tag writer is responsible for
+combining all these three into one list and writing them to the output
+stream.
+
+The example below will receive tags from both application and pipeline,
+combine them and write them to the output stream. It implements the tag
+setter so applications can set tags, and retrieves pipeline tags from
+incoming events.
+
+Warning, this example is outdated and doesn't work with the 1.0 version
+of GStreamer anymore.
+
+``` c
+
+GType
+gst_my_filter_get_type (void)
+{
+[..]
+ static const GInterfaceInfo tag_setter_info = {
+ NULL,
+ NULL,
+ NULL
+ };
+[..]
+ g_type_add_interface_static (my_filter_type,
+ GST_TYPE_TAG_SETTER,
+ &tag_setter_info);
+[..]
+}
+
+static void
+gst_my_filter_init (GstMyFilter *filter)
+{
+[..]
+}
+
+/*
+ * Write one tag.
+ */
+
+static void
+gst_my_filter_write_tag (const GstTagList *taglist,
+ const gchar *tagname,
+ gpointer data)
+{
+ GstMyFilter *filter = GST_MY_FILTER (data);
+ GstBuffer *buffer;
+ guint num_values = gst_tag_list_get_tag_size (list, tag_name), n;
+ const GValue *from;
+ GValue to = { 0 };
+
+ g_value_init (&to, G_TYPE_STRING);
+
+ for (n = 0; n < num_values; n++) {
+ guint8 * data;
+ gsize size;
+
+ from = gst_tag_list_get_value_index (taglist, tagname, n);
+ g_value_transform (from, &to);
+
+ data = g_strdup_printf ("%s:%s", tagname,
+ g_value_get_string (&to));
+ size = strlen (data);
+
+ buf = gst_buffer_new_wrapped (data, size);
+ gst_pad_push (filter->srcpad, buf);
+ }
+
+ g_value_unset (&to);
+}
+
+static void
+gst_my_filter_task_func (GstElement *element)
+{
+ GstMyFilter *filter = GST_MY_FILTER (element);
+ GstTagSetter *tagsetter = GST_TAG_SETTER (element);
+ GstData *data;
+ GstEvent *event;
+ gboolean eos = FALSE;
+ GstTagList *taglist = gst_tag_list_new ();
+
+ while (!eos) {
+ data = gst_pad_pull (filter->sinkpad);
+
+ /* We're not very much interested in data right now */
+ if (GST_IS_BUFFER (data))
+ gst_buffer_unref (GST_BUFFER (data));
+ event = GST_EVENT (data);
+
+ switch (GST_EVENT_TYPE (event)) {
+ case GST_EVENT_TAG:
+ gst_tag_list_insert (taglist, gst_event_tag_get_list (event),
+ GST_TAG_MERGE_PREPEND);
+ gst_event_unref (event);
+ break;
+ case GST_EVENT_EOS:
+ eos = TRUE;
+ gst_event_unref (event);
+ break;
+ default:
+ gst_pad_event_default (filter->sinkpad, event);
+ break;
+ }
+ }
+
+ /* merge tags with the ones retrieved from the application */
+ if ((gst_tag_setter_get_tag_list (tagsetter)) {
+ gst_tag_list_insert (taglist,
+ gst_tag_setter_get_tag_list (tagsetter),
+ gst_tag_setter_get_tag_merge_mode (tagsetter));
+ }
+
+ /* write tags */
+ gst_tag_list_foreach (taglist, gst_my_filter_write_tag, filter);
+
+ /* signal EOS */
+ gst_pad_push (filter->srcpad, gst_event_new (GST_EVENT_EOS));
+}
+
+
+```
+
+Note that normally, elements would not read the full stream before
+processing tags. Rather, they would read from each sinkpad until they've
+received data (since tags usually come in before the first data buffer)
+and process that.
+
--- /dev/null
+---
+title: Advanced Filter Concepts
+...
+
+# Advanced Filter Concepts
+
+By now, you should be able to create basic filter elements that can
+receive and send data. This is the simple model that GStreamer stands
+for. But GStreamer can do much more than only this\! In this chapter,
+various advanced topics will be discussed, such as scheduling, special
+pad types, clocking, events, interfaces, tagging and more. These topics
+are the sugar that makes GStreamer so easy to use for applications.
+
--- /dev/null
+---
+title: Memory allocation
+...
+
+# Memory allocation
+
+Memory allocation and management is a very important topic in
+multimedia. High definition video uses many megabytes to store one
+single frame of video. It is important to reuse the memory when possible
+instead of constantly allocating and freeing the memory.
+
+Multimedia systems usually use special purpose chips, such as DSPs or
+GPUs to perform the heavy lifting (especially for video). These special
+purpose chips have usually strict requirements for the memory that they
+can operate on and how the memory is accessed.
+
+This chapter talks about the memory management features that GStreamer
+plugins can use. We will first talk about the lowlevel `GstMemory`
+object that manages access to a piece of memory. We then continue with
+`GstBuffer` that is used to exchange data between plugins (and the
+application) and that uses `GstMemory`. We talk about `GstMeta` that can
+be placed on buffers to give extra info about the buffer and its memory.
+For efficiently managing buffers of the same size, we take a look at
+`GstBufferPool`. To conclude this chapter we take a look at the
+GST\_QUERY\_ALLOCATION query that is used to negotiate memory management
+options between elements.
+
+## GstMemory
+
+`GstMemory` is an object that manages a region of memory. The memory
+object points to a region of memory of “maxsize”. The area in this
+memory starting at “offset” and for “size” bytes is the accessible
+region in the memory. the maxsize of the memory can never be changed
+after the object is created, however, the offset and size can be
+changed.
+
+### GstAllocator
+
+`GstMemory` objects are created by a `GstAllocator` object. Most
+allocators implement the default `gst_allocator_alloc()` method but some
+allocator might implement a different method, for example when
+additional parameters are needed to allocate the specific memory.
+
+Different allocators exist for, for example, system memory, shared
+memory and memory backed by a DMAbuf file descriptor. To implement
+support for a new kind of memory type, you must implement a new
+allocator object as shown below.
+
+### GstMemory API example
+
+Data access to the memory wrapped by the `GstMemory` object is always
+protected with a `gst_memory_map()` and `gst_memory_unmap()` pair. An
+access mode (read/write) must be given when mapping memory. The map
+function returns a pointer to the valid memory region that can then be
+accessed according to the requested access mode.
+
+Below is an example of making a `GstMemory` object and using the
+`gst_memory_map()` to access the memory region.
+
+``` c
+
+[...]
+
+ GstMemory *mem;
+ GstMapInfo info;
+ gint i;
+
+ /* allocate 100 bytes */
+ mem = gst_allocator_alloc (NULL, 100, NULL);
+
+ /* get access to the memory in write mode */
+ gst_memory_map (mem, &info, GST_MAP_WRITE);
+
+ /* fill with pattern */
+ for (i = 0; i < info.size; i++)
+ info.data[i] = i;
+
+ /* release memory */
+ gst_memory_unmap (mem, &info);
+
+[...]
+
+
+```
+
+### Implementing a GstAllocator
+
+WRITEME
+
+## GstBuffer
+
+A `GstBuffer` is an lightweight object that is passed from an upstream
+to a downstream element and contains memory and metadata. It represents
+the multimedia content that is pushed or pull downstream by elements.
+
+The buffer contains one or more `GstMemory` objects that represent the
+data in the buffer.
+
+Metadata in the buffer consists of:
+
+ - DTS and PTS timestamps. These represent the decoding and
+ presentation timestamps of the buffer content and is used by
+ synchronizing elements to schedule buffers. Both these timestamps
+ can be GST\_CLOCK\_TIME\_NONE when unknown/undefined.
+
+ - The duration of the buffer contents. This duration can be
+ GST\_CLOCK\_TIME\_NONE when unknown/undefined.
+
+ - Media specific offsets and offset\_end. For video this is the frame
+ number in the stream and for audio the sample number. Other
+ definitions for other media exist.
+
+ - Arbitrary structures via `GstMeta`, see below.
+
+### GstBuffer writability
+
+A buffer is writable when the refcount of the object is exactly 1,
+meaning that only one object is holding a ref to the buffer. You can
+only modify anything in the buffer when the buffer is writable. This
+means that you need to call `gst_buffer_make_writable()` before changing
+the timestamps, offsets, metadata or adding and removing memory blocks.
+
+### GstBuffer API examples
+
+You can create a buffer with `gst_buffer_new ()` and then add memory
+objects to it or you can use a convenience function
+`gst_buffer_new_allocate ()` which combines the two. It's also possible
+to wrap existing memory with `gst_buffer_new_wrapped_full () ` where you
+can give the function to call when the memory should be freed.
+
+You can access the memory of the buffer by getting and mapping the
+`GstMemory` objects individually or by using `gst_buffer_map ()`. The
+latter merges all the memory into one big block and then gives you a
+pointer to this block.
+
+Below is an example of how to create a buffer and access its memory.
+
+``` c
+
+[...]
+ GstBuffer *buffer;
+ GstMemory *mem;
+ GstMapInfo info;
+
+ /* make empty buffer */
+ buffer = gst_buffer_new ();
+
+ /* make memory holding 100 bytes */
+ mem = gst_allocator_alloc (NULL, 100, NULL);
+
+ /* add the buffer */
+ gst_buffer_append_memory (buffer, mem);
+
+[...]
+
+ /* get WRITE access to the memory and fill with 0xff */
+ gst_buffer_map (buffer, &info, GST_MAP_WRITE);
+ memset (info.data, 0xff, info.size);
+ gst_buffer_unmap (buffer, &info);
+
+[...]
+
+ /* free the buffer */
+ gst_buffer_unref (buffer);
+
+[...]
+
+
+```
+
+## GstMeta
+
+With the `GstMeta` system you can add arbitrary structures on buffers.
+These structures describe extra properties of the buffer such as
+cropping, stride, region of interest etc.
+
+The metadata system separates API specification (what the metadata and
+its API look like) and the implementation (how it works). This makes it
+possible to make different implementations of the same API, for example,
+depending on the hardware you are running on.
+
+### GstMeta API example
+
+After allocating a new buffer, you can add metadata to the buffer with
+the metadata specific API. This means that you will need to link to the
+header file where the metadata is defined to use its API.
+
+By convention, a metadata API with name `FooBar` should provide two
+methods, a `gst_buffer_add_foo_bar_meta ()` and a
+`gst_buffer_get_foo_bar_meta ()`. Both functions should return a pointer
+to a `FooBarMeta` structure that contains the metadata fields. Some of
+the `_add_*_meta ()` can have extra parameters that will usually be used
+to configure the metadata structure for you.
+
+Let's have a look at the metadata that is used to specify a cropping
+region for video frames.
+
+``` c
+
+#include <gst/video/gstvideometa.h>
+
+[...]
+ GstVideoCropMeta *meta;
+
+ /* buffer points to a video frame, add some cropping metadata */
+ meta = gst_buffer_add_video_crop_meta (buffer);
+
+ /* configure the cropping metadata */
+ meta->x = 8;
+ meta->y = 8;
+ meta->width = 120;
+ meta->height = 80;
+[...]
+
+
+```
+
+An element can then use the metadata on the buffer when rendering the
+frame like this:
+
+``` c
+
+#include <gst/video/gstvideometa.h>
+
+[...]
+ GstVideoCropMeta *meta;
+
+ /* buffer points to a video frame, get the cropping metadata */
+ meta = gst_buffer_get_video_crop_meta (buffer);
+
+ if (meta) {
+ /* render frame with cropping */
+ _render_frame_cropped (buffer, meta->x, meta->y, meta->width, meta->height);
+ } else {
+ /* render frame */
+ _render_frame (buffer);
+ }
+[...]
+
+
+
+```
+
+### Implementing new GstMeta
+
+In the next sections we show how you can add new metadata to the system
+and use it on buffers.
+
+#### Define the metadata API
+
+First we need to define what our API will look like and we will have to
+register this API to the system. This is important because this API
+definition will be used when elements negotiate what kind of metadata
+they will exchange. The API definition also contains arbitrary tags that
+give hints about what the metadata contains. This is important when we
+see how metadata is preserved when buffers pass through the pipeline.
+
+If you are making a new implementation of an existing API, you can skip
+this step and move on to the implementation step.
+
+First we start with making the `my-example-meta.h` header file that will
+contain the definition of the API and structure for our metadata.
+
+``` c
+
+#include <gst/gst.h>
+
+typedef struct _MyExampleMeta MyExampleMeta;
+
+struct _MyExampleMeta {
+ GstMeta meta;
+
+ gint age;
+ gchar *name;
+};
+
+GType my_example_meta_api_get_type (void);
+#define MY_EXAMPLE_META_API_TYPE (my_example_meta_api_get_type())
+
+#define gst_buffer_get_my_example_meta(b) \
+ ((MyExampleMeta*)gst_buffer_get_meta((b),MY_EXAMPLE_META_API_TYPE))
+
+
+```
+
+The metadata API definition consists of the definition of the structure
+that holds a gint and a string. The first field in the structure must be
+`GstMeta`.
+
+We also define a `my_example_meta_api_get_type ()` function that will
+register out metadata API definition. We also define a convenience macro
+`gst_buffer_get_my_example_meta ()` that simply finds and returns the
+metadata with our new API.
+
+Next let's have a look at how the `my_example_meta_api_get_type ()`
+function is implemented in the `my-example-meta.c` file.
+
+``` c
+
+#include "my-example-meta.h"
+
+GType
+my_example_meta_api_get_type (void)
+{
+ static volatile GType type;
+ static const gchar *tags[] = { "foo", "bar", NULL };
+
+ if (g_once_init_enter (&type)) {
+ GType _type = gst_meta_api_type_register ("MyExampleMetaAPI", tags);
+ g_once_init_leave (&type, _type);
+ }
+ return type;
+}
+
+
+```
+
+As you can see, it simply uses the `gst_meta_api_type_register ()`
+function to register a name for the api and some tags. The result is a
+new pointer GType that defines the newly registered API.
+
+#### Implementing a metadata API
+
+Next we can make an implementation for a registered metadata API GType.
+The implementation detail of a metadata API are kept in a `GstMetaInfo`
+structure that you will make available to the users of your metadata API
+implementation with a `my_example_meta_get_info ()` function and a
+convenience `MY_EXAMPLE_META_INFO` macro. You will also make a method to
+add your metadata implementation to a `GstBuffer`. Your
+`my-example-meta.h` header file will need these additions:
+
+``` c
+
+[...]
+
+/* implementation */
+const GstMetaInfo *my_example_meta_get_info (void);
+#define MY_EXAMPLE_META_INFO (my_example_meta_get_info())
+
+MyExampleMeta * gst_buffer_add_my_example_meta (GstBuffer *buffer,
+ gint age,
+ const gchar *name);
+
+
+```
+
+Let's have a look at how these functions are implemented in the
+`my-example-meta.c` file.
+
+``` c
+
+[...]
+
+static gboolean
+my_example_meta_init (GstMeta * meta, gpointer params, GstBuffer * buffer)
+{
+ MyExampleMeta *emeta = (MyExampleMeta *) meta;
+
+ emeta->age = 0;
+ emeta->name = NULL;
+
+ return TRUE;
+}
+
+static gboolean
+my_example_meta_transform (GstBuffer * transbuf, GstMeta * meta,
+ GstBuffer * buffer, GQuark type, gpointer data)
+{
+ MyExampleMeta *emeta = (MyExampleMeta *) meta;
+
+ /* we always copy no matter what transform */
+ gst_buffer_add_my_example_meta (transbuf, emeta->age, emeta->name);
+
+ return TRUE;
+}
+
+static void
+my_example_meta_free (GstMeta * meta, GstBuffer * buffer)
+{
+ MyExampleMeta *emeta = (MyExampleMeta *) meta;
+
+ g_free (emeta->name);
+ emeta->name = NULL;
+}
+
+const GstMetaInfo *
+my_example_meta_get_info (void)
+{
+ static const GstMetaInfo *meta_info = NULL;
+
+ if (g_once_init_enter (&meta_info)) {
+ const GstMetaInfo *mi = gst_meta_register (MY_EXAMPLE_META_API_TYPE,
+ "MyExampleMeta",
+ sizeof (MyExampleMeta),
+ my_example_meta_init,
+ my_example_meta_free,
+ my_example_meta_transform);
+ g_once_init_leave (&meta_info, mi);
+ }
+ return meta_info;
+}
+
+MyExampleMeta *
+gst_buffer_add_my_example_meta (GstBuffer *buffer,
+ gint age,
+ const gchar *name)
+{
+ MyExampleMeta *meta;
+
+ g_return_val_if_fail (GST_IS_BUFFER (buffer), NULL);
+
+ meta = (MyExampleMeta *) gst_buffer_add_meta (buffer,
+ MY_EXAMPLE_META_INFO, NULL);
+
+ meta->age = age;
+ meta->name = g_strdup (name);
+
+ return meta;
+}
+
+
+```
+
+`gst_meta_register ()` registers the implementation details, like the
+API that you implement and the size of the metadata structure along with
+methods to initialize and free the memory area. You can also implement a
+transform function that will be called when a certain transformation
+(identified by the quark and quark specific data) is performed on a
+buffer.
+
+Lastly, you implement a `gst_buffer_add_*_meta()` that adds the metadata
+implementation to a buffer and sets the values of the metadata.
+
+## GstBufferPool
+
+The `GstBufferPool` object provides a convenient base class for managing
+lists of reusable buffers. Essential for this object is that all the
+buffers have the same properties such as size, padding, metadata and
+alignment.
+
+A bufferpool object can be configured to manage a minimum and maximum
+amount of buffers of a specific size. A bufferpool can also be
+configured to use a specific `GstAllocator` for the memory of the
+buffers. There is support in the bufferpool to enable bufferpool
+specific options, such as adding `GstMeta` to the buffers in the pool or
+such as enabling specific padding on the memory in the buffers.
+
+A Bufferpool can be inactivate and active. In the inactive state, you
+can configure the pool. In the active state, you can't change the
+configuration anymore but you can acquire and release buffers from/to
+the pool.
+
+In the following sections we take a look at how you can use a
+bufferpool.
+
+### GstBufferPool API example
+
+Many different bufferpool implementations can exist; they are all
+subclasses of the base class `GstBufferPool`. For this example, we will
+assume we somehow have access to a bufferpool, either because we created
+it ourselves or because we were given one as a result of the ALLOCATION
+query as we will see below.
+
+The bufferpool is initially in the inactive state so that we can
+configure it. Trying to configure a bufferpool that is not in the
+inactive state will fail. Likewise, trying to activate a bufferpool that
+is not configured will fail.
+
+``` c
+
+ GstStructure *config;
+
+[...]
+
+ /* get config structure */
+ config = gst_buffer_pool_get_config (pool);
+
+ /* set caps, size, minimum and maximum buffers in the pool */
+ gst_buffer_pool_config_set_params (config, caps, size, min, max);
+
+ /* configure allocator and parameters */
+ gst_buffer_pool_config_set_allocator (config, allocator, ¶ms);
+
+ /* store the updated configuration again */
+ gst_buffer_pool_set_config (pool, config);
+
+[...]
+
+
+```
+
+The configuration of the bufferpool is maintained in a generic
+`GstStructure` that can be obtained with `gst_buffer_pool_get_config()`.
+Convenience methods exist to get and set the configuration options in
+this structure. After updating the structure, it is set as the current
+configuration in the bufferpool again with
+`gst_buffer_pool_set_config()`.
+
+The following options can be configured on a bufferpool:
+
+ - The caps of the buffers to allocate.
+
+ - The size of the buffers. This is the suggested size of the buffers
+ in the pool. The pool might decide to allocate larger buffers to add
+ padding.
+
+ - The minimum and maximum amount of buffers in the pool. When minimum
+ is set to \> 0, the bufferpool will pre-allocate this amount of
+ buffers. When maximum is not 0, the bufferpool will allocate up to
+ maximum amount of buffers.
+
+ - The allocator and parameters to use. Some bufferpools might ignore
+ the allocator and use its internal one.
+
+ - Other arbitrary bufferpool options identified with a string. a
+ bufferpool lists the supported options with
+ `gst_buffer_pool_get_options()` and you can ask if an option is
+ supported with `gst_buffer_pool_has_option()`. The option can be
+ enabled by adding it to the configuration structure with
+ `gst_buffer_pool_config_add_option ()`. These options are used to
+ enable things like letting the pool set metadata on the buffers or
+ to add extra configuration options for padding, for example.
+
+After the configuration is set on the bufferpool, the pool can be
+activated with `gst_buffer_pool_set_active (pool, TRUE)`. From that
+point on you can use `gst_buffer_pool_acquire_buffer ()` to retrieve a
+buffer from the pool, like this:
+
+``` c
+
+ [...]
+
+ GstFlowReturn ret;
+ GstBuffer *buffer;
+
+ ret = gst_buffer_pool_acquire_buffer (pool, &buffer, NULL);
+ if (G_UNLIKELY (ret != GST_FLOW_OK))
+ goto pool_failed;
+
+ [...]
+
+
+```
+
+It is important to check the return value of the acquire function
+because it is possible that it fails: When your element shuts down, it
+will deactivate the bufferpool and then all calls to acquire will return
+GST\_FLOW\_FLUSHNG.
+
+All buffers that are acquired from the pool will have their pool member
+set to the original pool. When the last ref is decremented on the
+buffer, GStreamer will automatically call
+`gst_buffer_pool_release_buffer()` to release the buffer back to the
+pool. You (or any other downstream element) don't need to know if a
+buffer came from a pool, you can just unref it.
+
+### Implementing a new GstBufferPool
+
+WRITEME
+
+## GST\_QUERY\_ALLOCATION
+
+The ALLOCATION query is used to negotiate `GstMeta`, `GstBufferPool` and
+`GstAllocator` between elements. Negotiation of the allocation strategy
+is always initiated and decided by a srcpad after it has negotiated a
+format and before it decides to push buffers. A sinkpad can suggest an
+allocation strategy but it is ultimately the source pad that will decide
+based on the suggestions of the downstream sink pad.
+
+The source pad will do a GST\_QUERY\_ALLOCATION with the negotiated caps
+as a parameter. This is needed so that the downstream element knows what
+media type is being handled. A downstream sink pad can answer the
+allocation query with the following results:
+
+ - An array of possible `GstBufferPool` suggestions with suggested
+ size, minimum and maximum amount of buffers.
+
+ - An array of GstAllocator objects along with suggested allocation
+ parameters such as flags, prefix, alignment and padding. These
+ allocators can also be configured in a bufferpool when this is
+ supported by the bufferpool.
+
+ - An array of supported `GstMeta` implementations along with metadata
+ specific parameters. It is important that the upstream element knows
+ what kind of metadata is supported downstream before it places that
+ metadata on buffers.
+
+When the GST\_QUERY\_ALLOCATION returns, the source pad will select from
+the available bufferpools, allocators and metadata how it will allocate
+buffers.
+
+### ALLOCATION query example
+
+Below is an example of the ALLOCATION query.
+
+``` c
+
+#include <gst/video/video.h>
+#include <gst/video/gstvideometa.h>
+#include <gst/video/gstvideopool.h>
+
+ GstCaps *caps;
+ GstQuery *query;
+ GstStructure *structure;
+ GstBufferPool *pool;
+ GstStructure *config;
+ guint size, min, max;
+
+[...]
+
+ /* find a pool for the negotiated caps now */
+ query = gst_query_new_allocation (caps, TRUE);
+
+ if (!gst_pad_peer_query (scope->srcpad, query)) {
+ /* query failed, not a problem, we use the query defaults */
+ }
+
+ if (gst_query_get_n_allocation_pools (query) > 0) {
+ /* we got configuration from our peer, parse them */
+ gst_query_parse_nth_allocation_pool (query, 0, &pool, &size, &min, &max);
+ } else {
+ pool = NULL;
+ size = 0;
+ min = max = 0;
+ }
+
+ if (pool == NULL) {
+ /* we did not get a pool, make one ourselves then */
+ pool = gst_video_buffer_pool_new ();
+ }
+
+ config = gst_buffer_pool_get_config (pool);
+ gst_buffer_pool_config_add_option (config, GST_BUFFER_POOL_OPTION_VIDEO_META);
+ gst_buffer_pool_config_set_params (config, caps, size, min, max);
+ gst_buffer_pool_set_config (pool, config);
+
+ /* and activate */
+ gst_buffer_pool_set_active (pool, TRUE);
+
+[...]
+
+
+```
+
+This particular implementation will make a custom `GstVideoBufferPool`
+object that is specialized in allocating video buffers. You can also
+enable the pool to put `GstVideoMeta` metadata on the buffers from the
+pool doing `gst_buffer_pool_config_add_option (config,
+GST_BUFFER_POOL_OPTION_VIDEO_META)`.
+
+### The ALLOCATION query in base classes
+
+In many baseclasses you will see the following virtual methods for
+influencing the allocation strategy:
+
+ - `propose_allocation ()` should suggest allocation parameters for the
+ upstream element.
+
+ - `decide_allocation ()` should decide the allocation parameters from
+ the suggestions received from downstream.
+
+Implementors of these methods should modify the given `GstQuery` object
+by updating the pool options and allocation options.
+
--- /dev/null
+---
+title: Appendices
+...
+
+# Appendices
+
+This chapter contains things that don't belong anywhere else.
+
--- /dev/null
+---
+title: Adding Properties
+...
+
+# Adding Properties
+
+The primary and most important way of controlling how an element
+behaves, is through GObject properties. GObject properties are defined
+in the `_class_init ()` function. The element optionally implements a
+`_get_property ()` and a `_set_property ()` function. These functions
+will be notified if an application changes or requests the value of a
+property, and can then fill in the value or take action required for
+that property to change value internally.
+
+You probably also want to keep an instance variable around with the
+currently configured value of the property that you use in the get and
+set functions. Note that `GObject` will not automatically set your
+instance variable to the default value, you will have to do that in the
+`_init ()` function of your element.
+
+``` c
+
+/* properties */
+enum {
+ PROP_0,
+ PROP_SILENT
+ /* FILL ME */
+};
+
+static void gst_my_filter_set_property (GObject *object,
+ guint prop_id,
+ const GValue *value,
+ GParamSpec *pspec);
+static void gst_my_filter_get_property (GObject *object,
+ guint prop_id,
+ GValue *value,
+ GParamSpec *pspec);
+
+static void
+gst_my_filter_class_init (GstMyFilterClass *klass)
+{
+ GObjectClass *object_class = G_OBJECT_CLASS (klass);
+
+ /* define virtual function pointers */
+ object_class->set_property = gst_my_filter_set_property;
+ object_class->get_property = gst_my_filter_get_property;
+
+ /* define properties */
+ g_object_class_install_property (object_class, PROP_SILENT,
+ g_param_spec_boolean ("silent", "Silent",
+ "Whether to be very verbose or not",
+ FALSE, G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
+}
+
+static void
+gst_my_filter_set_property (GObject *object,
+ guint prop_id,
+ const GValue *value,
+ GParamSpec *pspec)
+{
+ GstMyFilter *filter = GST_MY_FILTER (object);
+
+ switch (prop_id) {
+ case PROP_SILENT:
+ filter->silent = g_value_get_boolean (value);
+ g_print ("Silent argument was changed to %s\n",
+ filter->silent ? "true" : "false");
+ break;
+ default:
+ G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
+ break;
+ }
+}
+
+static void
+gst_my_filter_get_property (GObject *object,
+ guint prop_id,
+ GValue *value,
+ GParamSpec *pspec)
+{
+ GstMyFilter *filter = GST_MY_FILTER (object);
+
+ switch (prop_id) {
+ case PROP_SILENT:
+ g_value_set_boolean (value, filter->silent);
+ break;
+ default:
+ G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
+ break;
+ }
+}
+```
+
+The above is a very simple example of how properties are used. Graphical
+applications will use these properties and will display a
+user-controllable widget with which these properties can be changed.
+This means that - for the property to be as user-friendly as possible -
+you should be as exact as possible in the definition of the property.
+Not only in defining ranges in between which valid properties can be
+located (for integers, floats, etc.), but also in using very descriptive
+(better yet: internationalized) strings in the definition of the
+property, and if possible using enums and flags instead of integers. The
+GObject documentation describes these in a very complete way, but below,
+we'll give a short example of where this is useful. Note that using
+integers here would probably completely confuse the user, because they
+make no sense in this context. The example is stolen from videotestsrc.
+
+``` c
+typedef enum {
+ GST_VIDEOTESTSRC_SMPTE,
+ GST_VIDEOTESTSRC_SNOW,
+ GST_VIDEOTESTSRC_BLACK
+} GstVideotestsrcPattern;
+
+[..]
+
+#define GST_TYPE_VIDEOTESTSRC_PATTERN (gst_videotestsrc_pattern_get_type ())
+static GType
+gst_videotestsrc_pattern_get_type (void)
+{
+ static GType videotestsrc_pattern_type = 0;
+
+ if (!videotestsrc_pattern_type) {
+ static GEnumValue pattern_types[] = {
+ { GST_VIDEOTESTSRC_SMPTE, "SMPTE 100% color bars", "smpte" },
+ { GST_VIDEOTESTSRC_SNOW, "Random (television snow)", "snow" },
+ { GST_VIDEOTESTSRC_BLACK, "0% Black", "black" },
+ { 0, NULL, NULL },
+ };
+
+ videotestsrc_pattern_type =
+ g_enum_register_static ("GstVideotestsrcPattern",
+ pattern_types);
+ }
+
+ return videotestsrc_pattern_type;
+}
+
+[..]
+
+static void
+gst_videotestsrc_class_init (GstvideotestsrcClass *klass)
+{
+[..]
+ g_object_class_install_property (G_OBJECT_CLASS (klass), PROP_PATTERN,
+ g_param_spec_enum ("pattern", "Pattern",
+ "Type of test pattern to generate",
+ GST_TYPE_VIDEOTESTSRC_PATTERN, GST_VIDEOTESTSRC_SMPTE,
+ G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
+[..]
+}
+
+```
+
--- /dev/null
+---
+title: Constructing the Boilerplate
+...
+
+# Constructing the Boilerplate
+
+In this chapter you will learn how to construct the bare minimum code
+for a new plugin. Starting from ground zero, you will see how to get the
+GStreamer template source. Then you will learn how to use a few basic
+tools to copy and modify a template plugin to create a new plugin. If
+you follow the examples here, then by the end of this chapter you will
+have a functional audio filter plugin that you can compile and use in
+GStreamer applications.
+
+## Getting the GStreamer Plugin Templates
+
+There are currently two ways to develop a new plugin for GStreamer: You
+can write the entire plugin by hand, or you can copy an existing plugin
+template and write the plugin code you need. The second method is by far
+the simpler of the two, so the first method will not even be described
+here. (Errm, that is, “it is left as an exercise to the reader.”)
+
+The first step is to check out a copy of the `gst-template` git module
+to get an important tool and the source code template for a basic
+GStreamer plugin. To check out the `gst-template` module, make sure you
+are connected to the internet, and type the following commands at a
+command
+console:
+
+```
+shell $ git clone git://anongit.freedesktop.org/gstreamer/gst-template.git
+Initialized empty Git repository in /some/path/gst-template/.git/
+remote: Counting objects: 373, done.
+remote: Compressing objects: 100% (114/114), done.
+remote: Total 373 (delta 240), reused 373 (delta 240)
+Receiving objects: 100% (373/373), 75.16 KiB | 78 KiB/s, done.
+Resolving deltas: 100% (240/240), done.
+
+```
+
+This command will check out a series of files and directories into
+`gst-template`. The template you will be using is in the
+`gst-template/gst-plugin/` directory. You should look over the files in
+that directory to get a general idea of the structure of a source tree
+for a plugin.
+
+If for some reason you can't access the git repository, you can also
+[download a snapshot of the latest
+revision](http://cgit.freedesktop.org/gstreamer/gst-template/commit/)
+via the cgit web interface.
+
+## Using the Project Stamp
+
+The first thing to do when making a new element is to specify some basic
+details about it: what its name is, who wrote it, what version number it
+is, etc. We also need to define an object to represent the element and
+to store the data the element needs. These details are collectively
+known as the *boilerplate*.
+
+The standard way of defining the boilerplate is simply to write some
+code, and fill in some structures. As mentioned in the previous section,
+the easiest way to do this is to copy a template and add functionality
+according to your needs. To help you do so, there is a tool in the
+`./gst-plugin/tools/` directory. This tool, `make_element`, is a command
+line utility that creates the boilerplate code for you.
+
+To use `make_element`, first open up a terminal window. Change to the
+`gst-template/gst-plugin/src` directory, and then run the `make_element`
+command. The arguments to the `make_element` are:
+
+1. the name of the plugin, and
+
+2. the source file that the tool will use. By default, `gstplugin` is
+ used.
+
+For example, the following commands create the MyFilter plugin based on
+the plugin template and put the output files in the
+`gst-template/gst-plugin/src` directory:
+
+```
+shell $ cd gst-template/gst-plugin/src
+shell $ ../tools/make_element MyFilter
+
+```
+
+> **Note**
+>
+> Capitalization is important for the name of the plugin. Keep in mind
+> that under some operating systems, capitalization is also important
+> when specifying directory and file names in general.
+
+The last command creates two files: `gstmyfilter.c` and `gstmyfilter.h`.
+
+> **Note**
+>
+> It is recommended that you create a copy of the `gst-plugin` directory
+> before continuing.
+
+Now one needs to adjust the `Makefile.am` to use the new filenames and
+run `autogen.sh` from the parent directory to bootstrap the build
+environment. After that, the project can be built and installed using
+the well known `make && sudo make install` commands.
+
+> **Note**
+>
+> Be aware that by default `autogen.sh` and `configure` would choose
+> `/usr/local` as a default location. One would need to add
+> `/usr/local/lib/gstreamer-1.0` to `GST_PLUGIN_PATH` in order to make
+> the new plugin show up in a gstreamer that's been installed from
+> packages.
+
+> **Note**
+>
+> FIXME: this section is slightly outdated. gst-template is still useful
+> as an example for a minimal plugin build system skeleton. However, for
+> creating elements the tool gst-element-maker from gst-plugins-bad is
+> recommended these days.
+
+## Examining the Basic Code
+
+First we will examine the code you would be likely to place in a header
+file (although since the interface to the code is entirely defined by
+the plugin system, and doesn't depend on reading a header file, this is
+not crucial.)
+
+``` c
+#include <gst/gst.h>
+
+/* Definition of structure storing data for this element. */
+typedef struct _GstMyFilter {
+ GstElement element;
+
+ GstPad *sinkpad, *srcpad;
+
+ gboolean silent;
+
+
+
+} GstMyFilter;
+
+/* Standard definition defining a class for this element. */
+typedef struct _GstMyFilterClass {
+ GstElementClass parent_class;
+} GstMyFilterClass;
+
+/* Standard macros for defining types for this element. */
+#define GST_TYPE_MY_FILTER (gst_my_filter_get_type())
+#define GST_MY_FILTER(obj) \
+ (G_TYPE_CHECK_INSTANCE_CAST((obj),GST_TYPE_MY_FILTER,GstMyFilter))
+#define GST_MY_FILTER_CLASS(klass) \
+ (G_TYPE_CHECK_CLASS_CAST((klass),GST_TYPE_MY_FILTER,GstMyFilterClass))
+#define GST_IS_MY_FILTER(obj) \
+ (G_TYPE_CHECK_INSTANCE_TYPE((obj),GST_TYPE_MY_FILTER))
+#define GST_IS_MY_FILTER_CLASS(klass) \
+ (G_TYPE_CHECK_CLASS_TYPE((klass),GST_TYPE_MY_FILTER))
+
+/* Standard function returning type information. */
+GType gst_my_filter_get_type (void);
+
+```
+
+Using this header file, you can use the following macro to setup the
+`GObject` basics in your source file so that all functions will be
+called appropriately:
+
+``` c
+#include "filter.h"
+
+G_DEFINE_TYPE (GstMyFilter, gst_my_filter, GST_TYPE_ELEMENT);
+
+```
+
+## Element metadata
+
+The Element metadata provides extra element information. It is
+configured with `gst_element_class_set_metadata` or
+`gst_element_class_set_static_metadata` which takes the following
+parameters:
+
+ - A long, English, name for the element.
+
+ - The type of the element, see the docs/design/draft-klass.txt
+ document in the GStreamer core source tree for details and examples.
+
+ - A brief description of the purpose of the element.
+
+ - The name of the author of the element, optionally followed by a
+ contact email address in angle brackets.
+
+For example:
+
+``` c
+gst_element_class_set_static_metadata (klass,
+ "An example plugin",
+ "Example/FirstExample",
+ "Shows the basic structure of a plugin",
+ "your name <your.name@your.isp>");
+
+```
+
+The element details are registered with the plugin during the
+`_class_init ()` function, which is part of the GObject system. The
+`_class_init ()` function should be set for this GObject in the function
+where you register the type with GLib.
+
+``` c
+static void
+gst_my_filter_class_init (GstMyFilterClass * klass)
+{
+ GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
+
+[..]
+ gst_element_class_set_static_metadata (element_klass,
+ "An example plugin",
+ "Example/FirstExample",
+ "Shows the basic structure of a plugin",
+ "your name <your.name@your.isp>");
+
+}
+
+```
+
+## GstStaticPadTemplate
+
+A GstStaticPadTemplate is a description of a pad that the element will
+(or might) create and use. It contains:
+
+ - A short name for the pad.
+
+ - Pad direction.
+
+ - Existence property. This indicates whether the pad exists always (an
+ “always” pad), only in some cases (a “sometimes” pad) or only if the
+ application requested such a pad (a “request” pad).
+
+ - Supported types by this element (capabilities).
+
+For example:
+
+``` c
+static GstStaticPadTemplate sink_factory =
+GST_STATIC_PAD_TEMPLATE (
+ "sink",
+ GST_PAD_SINK,
+ GST_PAD_ALWAYS,
+ GST_STATIC_CAPS ("ANY")
+);
+
+
+
+```
+
+Those pad templates are registered during the `_class_init ()` function
+with the `gst_element_class_add_pad_template ()`. For this function you
+need a handle the `GstPadTemplate` which you can create from the static
+pad template with `gst_static_pad_template_get ()`. See below for more
+details on this.
+
+Pads are created from these static templates in the element's `_init ()`
+function using `gst_pad_new_from_static_template ()`. In order to create
+a new pad from this template using `gst_pad_new_from_static_template
+()`, you will need to declare the pad template as a global variable.
+More on this subject in [Specifying the pads](pwg-building-pads.md).
+
+ static GstStaticPadTemplate sink_factory = [..],
+ src_factory = [..];
+
+ static void
+ gst_my_filter_class_init (GstMyFilterClass * klass)
+ {
+ GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
+ [..]
+
+ gst_element_class_add_pad_template (element_class,
+ gst_static_pad_template_get (&src_factory));
+ gst_element_class_add_pad_template (element_class,
+ gst_static_pad_template_get (&sink_factory));
+ }
+
+The last argument in a template is its type or list of supported types.
+In this example, we use 'ANY', which means that this element will accept
+all input. In real-life situations, you would set a media type and
+optionally a set of properties to make sure that only supported input
+will come in. This representation should be a string that starts with a
+media type, then a set of comma-separates properties with their
+supported values. In case of an audio filter that supports raw integer
+16-bit audio, mono or stereo at any samplerate, the correct template
+would look like this:
+
+``` c
+
+static GstStaticPadTemplate sink_factory =
+GST_STATIC_PAD_TEMPLATE (
+ "sink",
+ GST_PAD_SINK,
+ GST_PAD_ALWAYS,
+ GST_STATIC_CAPS (
+ "audio/x-raw, "
+ "format = (string) " GST_AUDIO_NE (S16) ", "
+ "channels = (int) { 1, 2 }, "
+ "rate = (int) [ 8000, 96000 ]"
+ )
+);
+
+
+```
+
+Values surrounded by curly brackets (“{” and “}”) are lists, values
+surrounded by square brackets (“\[” and “\]”) are ranges. Multiple sets
+of types are supported too, and should be separated by a semicolon
+(“;”). Later, in the chapter on pads, we will see how to use types
+to know the exact format of a stream: [Specifying the
+pads](pwg-building-pads.md).
+
+## Constructor Functions
+
+Each element has two functions which are used for construction of an
+element. The `_class_init()` function, which is used to initialise the
+class only once (specifying what signals, arguments and virtual
+functions the class has and setting up global state); and the `_init()`
+function, which is used to initialise a specific instance of this type.
+
+## The plugin\_init function
+
+Once we have written code defining all the parts of the plugin, we need
+to write the plugin\_init() function. This is a special function, which
+is called as soon as the plugin is loaded, and should return TRUE or
+FALSE depending on whether it loaded initialized any dependencies
+correctly. Also, in this function, any supported element type in the
+plugin should be registered.
+
+``` c
+
+
+static gboolean
+plugin_init (GstPlugin *plugin)
+{
+ return gst_element_register (plugin, "my_filter",
+ GST_RANK_NONE,
+ GST_TYPE_MY_FILTER);
+}
+
+GST_PLUGIN_DEFINE (
+ GST_VERSION_MAJOR,
+ GST_VERSION_MINOR,
+ my_filter,
+ "My filter plugin",
+ plugin_init,
+ VERSION,
+ "LGPL",
+ "GStreamer",
+ "http://gstreamer.net/"
+)
+
+
+
+```
+
+Note that the information returned by the plugin\_init() function will
+be cached in a central registry. For this reason, it is important that
+the same information is always returned by the function: for example, it
+must not make element factories available based on runtime conditions.
+If an element can only work in certain conditions (for example, if the
+soundcard is not being used by some other process) this must be
+reflected by the element being unable to enter the READY state if
+unavailable, rather than the plugin attempting to deny existence of the
+plugin.
+
--- /dev/null
+---
+title: The chain function
+...
+
+# The chain function
+
+The chain function is the function in which all data processing takes
+place. In the case of a simple filter, `_chain ()` functions are mostly
+linear functions - so for each incoming buffer, one buffer will go out,
+too. Below is a very simple implementation of a chain function:
+
+``` c
+
+static GstFlowReturn gst_my_filter_chain (GstPad *pad,
+ GstObject *parent,
+ GstBuffer *buf);
+
+[..]
+
+static void
+gst_my_filter_init (GstMyFilter * filter)
+{
+[..]
+ /* configure chain function on the pad before adding
+ * the pad to the element */
+ gst_pad_set_chain_function (filter->sinkpad,
+ gst_my_filter_chain);
+[..]
+}
+
+static GstFlowReturn
+gst_my_filter_chain (GstPad *pad,
+ GstObject *parent,
+ GstBuffer *buf)
+{
+ GstMyFilter *filter = GST_MY_FILTER (parent);
+
+ if (!filter->silent)
+ g_print ("Have data of size %" G_GSIZE_FORMAT" bytes!\n",
+ gst_buffer_get_size (buf));
+
+ return gst_pad_push (filter->srcpad, buf);
+}
+```
+
+Obviously, the above doesn't do much useful. Instead of printing that
+the data is in, you would normally process the data there. Remember,
+however, that buffers are not always writeable.
+
+In more advanced elements (the ones that do event processing), you may
+want to additionally specify an event handling function, which will be
+called when stream-events are sent (such as caps, end-of-stream,
+newsegment, tags, etc.).
+
+ static void
+ gst_my_filter_init (GstMyFilter * filter)
+ {
+ [..]
+ gst_pad_set_event_function (filter->sinkpad,
+ gst_my_filter_sink_event);
+ [..]
+ }
+
+
+
+ static gboolean
+ gst_my_filter_sink_event (GstPad *pad,
+ GstObject *parent,
+ GstEvent *event)
+ {
+ GstMyFilter *filter = GST_MY_FILTER (parent);
+
+ switch (GST_EVENT_TYPE (event)) {
+ case GST_EVENT_CAPS:
+ /* we should handle the format here */
+ break;
+ case GST_EVENT_EOS:
+ /* end-of-stream, we should close down all stream leftovers here */
+ gst_my_filter_stop_processing (filter);
+ break;
+ default:
+ break;
+ }
+
+ return gst_pad_event_default (pad, parent, event);
+ }
+
+ static GstFlowReturn
+ gst_my_filter_chain (GstPad *pad,
+ GstObject *parent,
+ GstBuffer *buf)
+ {
+ GstMyFilter *filter = GST_MY_FILTER (parent);
+ GstBuffer *outbuf;
+
+ outbuf = gst_my_filter_process_data (filter, buf);
+ gst_buffer_unref (buf);
+ if (!outbuf) {
+ /* something went wrong - signal an error */
+ GST_ELEMENT_ERROR (GST_ELEMENT (filter), STREAM, FAILED, (NULL), (NULL));
+ return GST_FLOW_ERROR;
+ }
+
+ return gst_pad_push (filter->srcpad, outbuf);
+ }
+
+In some cases, it might be useful for an element to have control over
+the input data rate, too. In that case, you probably want to write a
+so-called *loop-based* element. Source elements (with only source pads)
+can also be *get-based* elements. These concepts will be explained in
+the advanced section of this guide, and in the section that specifically
+discusses source pads.
+
--- /dev/null
+---
+title: The event function
+...
+
+# The event function
+
+The event function notifies you of special events that happen in the
+datastream (such as caps, end-of-stream, newsegment, tags, etc.). Events
+can travel both upstream and downstream, so you can receive them on sink
+pads as well as source pads.
+
+Below follows a very simple event function that we install on the sink
+pad of our element.
+
+``` c
+
+static gboolean gst_my_filter_sink_event (GstPad *pad,
+ GstObject *parent,
+ GstEvent *event);
+
+[..]
+
+static void
+gst_my_filter_init (GstMyFilter * filter)
+{
+[..]
+ /* configure event function on the pad before adding
+ * the pad to the element */
+ gst_pad_set_event_function (filter->sinkpad,
+ gst_my_filter_sink_event);
+[..]
+}
+
+static gboolean
+gst_my_filter_sink_event (GstPad *pad,
+ GstObject *parent,
+ GstEvent *event)
+{
+ gboolean ret;
+ GstMyFilter *filter = GST_MY_FILTER (parent);
+
+ switch (GST_EVENT_TYPE (event)) {
+ case GST_EVENT_CAPS:
+ /* we should handle the format here */
+
+ /* push the event downstream */
+ ret = gst_pad_push_event (filter->srcpad, event);
+ break;
+ case GST_EVENT_EOS:
+ /* end-of-stream, we should close down all stream leftovers here */
+ gst_my_filter_stop_processing (filter);
+
+ ret = gst_pad_event_default (pad, parent, event);
+ break;
+ default:
+ /* just call the default handler */
+ ret = gst_pad_event_default (pad, parent, event);
+ break;
+ }
+ return ret;
+}
+
+
+```
+
+It is a good idea to call the default event handler
+`gst_pad_event_default ()` for unknown events. Depending on the event
+type, the default handler will forward the event or simply unref it. The
+CAPS event is by default not forwarded so we need to do this in the
+event handler ourselves.
+
--- /dev/null
+---
+title: Specifying the pads
+...
+
+# Specifying the pads
+
+As explained before, pads are the port through which data goes in and
+out of your element, and that makes them a very important item in the
+process of element creation. In the boilerplate code, we have seen how
+static pad templates take care of registering pad templates with the
+element class. Here, we will see how to create actual elements, use an
+`_event
+()`-function to configure for a particular format and how to register
+functions to let data flow through the element.
+
+In the element `_init ()` function, you create the pad from the pad
+template that has been registered with the element class in the
+`_class_init ()` function. After creating the pad, you have to set a
+`_chain ()` function pointer that will receive and process the input
+data on the sinkpad. You can optionally also set an `_event ()` function
+pointer and a `_query ()` function pointer. Alternatively, pads can also
+operate in looping mode, which means that they can pull data themselves.
+More on this topic later. After that, you have to register the pad with
+the element. This happens like this:
+
+``` c
+
+
+
+static void
+gst_my_filter_init (GstMyFilter *filter)
+{
+ /* pad through which data comes in to the element */
+ filter->sinkpad = gst_pad_new_from_static_template (
+ &sink_template, "sink");
+ /* pads are configured here with gst_pad_set_*_function () */
+
+
+
+ gst_element_add_pad (GST_ELEMENT (filter), filter->sinkpad);
+
+ /* pad through which data goes out of the element */
+ filter->srcpad = gst_pad_new_from_static_template (
+ &src_template, "src");
+ /* pads are configured here with gst_pad_set_*_function () */
+
+
+
+ gst_element_add_pad (GST_ELEMENT (filter), filter->srcpad);
+
+ /* properties initial value */
+ filter->silent = FALSE;
+}
+
+```
+
--- /dev/null
+---
+title: The query function
+...
+
+# The query function
+
+Through the query function, your element will receive queries that it
+has to reply to. These are queries like position, duration but also
+about the supported formats and scheduling modes your element supports.
+Queries can travel both upstream and downstream, so you can receive them
+on sink pads as well as source pads.
+
+Below follows a very simple query function that we install on the source
+pad of our element.
+
+``` c
+
+static gboolean gst_my_filter_src_query (GstPad *pad,
+ GstObject *parent,
+ GstQuery *query);
+
+[..]
+
+static void
+gst_my_filter_init (GstMyFilter * filter)
+{
+[..]
+ /* configure event function on the pad before adding
+ * the pad to the element */
+ gst_pad_set_query_function (filter->srcpad,
+ gst_my_filter_src_query);
+[..]
+}
+
+static gboolean
+gst_my_filter_src_query (GstPad *pad,
+ GstObject *parent,
+ GstQuery *query)
+{
+ gboolean ret;
+ GstMyFilter *filter = GST_MY_FILTER (parent);
+
+ switch (GST_QUERY_TYPE (query)) {
+ case GST_QUERY_POSITION:
+ /* we should report the current position */
+ [...]
+ break;
+ case GST_QUERY_DURATION:
+ /* we should report the duration here */
+ [...]
+ break;
+ case GST_QUERY_CAPS:
+ /* we should report the supported caps here */
+ [...]
+ break;
+ default:
+ /* just call the default handler */
+ ret = gst_pad_query_default (pad, parent, query);
+ break;
+ }
+ return ret;
+}
+
+
+```
+
+It is a good idea to call the default query handler
+`gst_pad_query_default ()` for unknown queries. Depending on the query
+type, the default handler will forward the query or simply unref it.
+
--- /dev/null
+---
+title: Signals
+...
+
+# Signals
+
+GObject signals can be used to notify applications of events specific to
+this object. Note, however, that the application needs to be aware of
+signals and their meaning, so if you're looking for a generic way for
+application-element interaction, signals are probably not what you're
+looking for. In many cases, however, signals can be very useful. See the
+[GObject documentation](http://library.gnome.org/devel/gobject/stable/)
+for all internals about signals.
+
--- /dev/null
+---
+title: Building a Test Application
+...
+
+# Building a Test Application
+
+Often, you will want to test your newly written plugin in an as small
+setting as possible. Usually, `gst-launch-1.0` is a good first step at
+testing a plugin. If you have not installed your plugin in a directory
+that GStreamer searches, then you will need to set the plugin path.
+Either set GST\_PLUGIN\_PATH to the directory containing your plugin, or
+use the command-line option --gst-plugin-path. If you based your plugin
+off of the gst-plugin template, then this will look something like `
+gst-launch-1.0 --gst-plugin-path=$HOME/gst-template/gst-plugin/src/.libs
+TESTPIPELINE
+` However, you will often need more testing features than gst-launch-1.0
+can provide, such as seeking, events, interactivity and more. Writing
+your own small testing program is the easiest way to accomplish this.
+This section explains - in a few words - how to do that. For a complete
+application development guide, see the [Application Development
+Manual](../../manual/html/index.html).
+
+At the start, you need to initialize the GStreamer core library by
+calling `gst_init ()`. You can alternatively call
+`gst_init_get_option_group ()`, which will return a pointer to
+GOptionGroup. You can then use GOption to handle the initialization, and
+this will finish the GStreamer initialization.
+
+You can create elements using `gst_element_factory_make ()`, where the
+first argument is the element type that you want to create, and the
+second argument is a free-form name. The example at the end uses a
+simple filesource - decoder - soundcard output pipeline, but you can use
+specific debugging elements if that's necessary. For example, an
+`identity` element can be used in the middle of the pipeline to act as a
+data-to-application transmitter. This can be used to check the data for
+misbehaviours or correctness in your test application. Also, you can use
+a `fakesink` element at the end of the pipeline to dump your data to the
+stdout (in order to do this, set the `dump` property to TRUE). Lastly,
+you can use valgrind to check for memory errors.
+
+During linking, your test application can use filtered caps as a way to
+drive a specific type of data to or from your element. This is a very
+simple and effective way of checking multiple types of input and output
+in your element.
+
+Note that during running, you should listen for at least the “error” and
+“eos” messages on the bus and/or your plugin/element to check for
+correct handling of this. Also, you should add events into the pipeline
+and make sure your plugin handles these correctly (with respect to
+clocking, internal caching, etc.).
+
+Never forget to clean up memory in your plugin or your test application.
+When going to the NULL state, your element should clean up allocated
+memory and caches. Also, it should close down any references held to
+possible support libraries. Your application should `unref ()` the
+pipeline and make sure it doesn't crash.
+
+``` c
+#include <gst/gst.h>
+
+static gboolean
+bus_call (GstBus *bus,
+ GstMessage *msg,
+ gpointer data)
+{
+ GMainLoop *loop = data;
+
+ switch (GST_MESSAGE_TYPE (msg)) {
+ case GST_MESSAGE_EOS:
+ g_print ("End-of-stream\n");
+ g_main_loop_quit (loop);
+ break;
+ case GST_MESSAGE_ERROR: {
+ gchar *debug = NULL;
+ GError *err = NULL;
+
+ gst_message_parse_error (msg, &err, &debug);
+
+ g_print ("Error: %s\n", err->message);
+ g_error_free (err);
+
+ if (debug) {
+ g_print ("Debug details: %s\n", debug);
+ g_free (debug);
+ }
+
+ g_main_loop_quit (loop);
+ break;
+ }
+ default:
+ break;
+ }
+
+ return TRUE;
+}
+
+gint
+main (gint argc,
+ gchar *argv[])
+{
+ GstStateChangeReturn ret;
+ GstElement *pipeline, *filesrc, *decoder, *filter, *sink;
+ GstElement *convert1, *convert2, *resample;
+ GMainLoop *loop;
+ GstBus *bus;
+ guint watch_id;
+
+ /* initialization */
+ gst_init (&argc, &argv);
+ loop = g_main_loop_new (NULL, FALSE);
+ if (argc != 2) {
+ g_print ("Usage: %s <mp3 filename>\n", argv[0]);
+ return 01;
+ }
+
+ /* create elements */
+ pipeline = gst_pipeline_new ("my_pipeline");
+
+ /* watch for messages on the pipeline's bus (note that this will only
+ * work like this when a GLib main loop is running) */
+ bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
+ watch_id = gst_bus_add_watch (bus, bus_call, loop);
+ gst_object_unref (bus);
+
+ filesrc = gst_element_factory_make ("filesrc", "my_filesource");
+ decoder = gst_element_factory_make ("mad", "my_decoder");
+
+ /* putting an audioconvert element here to convert the output of the
+ * decoder into a format that my_filter can handle (we are assuming it
+ * will handle any sample rate here though) */
+ convert1 = gst_element_factory_make ("audioconvert", "audioconvert1");
+
+ /* use "identity" here for a filter that does nothing */
+ filter = gst_element_factory_make ("my_filter", "my_filter");
+
+ /* there should always be audioconvert and audioresample elements before
+ * the audio sink, since the capabilities of the audio sink usually vary
+ * depending on the environment (output used, sound card, driver etc.) */
+ convert2 = gst_element_factory_make ("audioconvert", "audioconvert2");
+ resample = gst_element_factory_make ("audioresample", "audioresample");
+ sink = gst_element_factory_make ("pulsesink", "audiosink");
+
+ if (!sink || !decoder) {
+ g_print ("Decoder or output could not be found - check your install\n");
+ return -1;
+ } else if (!convert1 || !convert2 || !resample) {
+ g_print ("Could not create audioconvert or audioresample element, "
+ "check your installation\n");
+ return -1;
+ } else if (!filter) {
+ g_print ("Your self-written filter could not be found. Make sure it "
+ "is installed correctly in $(libdir)/gstreamer-1.0/ or "
+ "~/.gstreamer-1.0/plugins/ and that gst-inspect-1.0 lists it. "
+ "If it doesn't, check with 'GST_DEBUG=*:2 gst-inspect-1.0' for "
+ "the reason why it is not being loaded.");
+ return -1;
+ }
+
+ g_object_set (G_OBJECT (filesrc), "location", argv[1], NULL);
+
+ gst_bin_add_many (GST_BIN (pipeline), filesrc, decoder, convert1, filter,
+ convert2, resample, sink, NULL);
+
+ /* link everything together */
+ if (!gst_element_link_many (filesrc, decoder, convert1, filter, convert2,
+ resample, sink, NULL)) {
+ g_print ("Failed to link one or more elements!\n");
+ return -1;
+ }
+
+ /* run */
+ ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
+ if (ret == GST_STATE_CHANGE_FAILURE) {
+ GstMessage *msg;
+
+ g_print ("Failed to start up pipeline!\n");
+
+ /* check if there is an error message with details on the bus */
+ msg = gst_bus_poll (bus, GST_MESSAGE_ERROR, 0);
+ if (msg) {
+ GError *err = NULL;
+
+ gst_message_parse_error (msg, &err, NULL);
+ g_print ("ERROR: %s\n", err->message);
+ g_error_free (err);
+ gst_message_unref (msg);
+ }
+ return -1;
+ }
+
+ g_main_loop_run (loop);
+
+ /* clean up */
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+ g_source_remove (watch_id);
+ g_main_loop_unref (loop);
+
+ return 0;
+}
+
+```
+
--- /dev/null
+---
+title: Types and Properties
+...
+
+# Types and Properties
+
+There is a very large set of possible types that may be used to pass
+data between elements. Indeed, each new element that is defined may use
+a new data format (though unless at least one other element recognises
+that format, it will be most likely be useless since nothing will be
+able to link with it).
+
+In order for types to be useful, and for systems like autopluggers to
+work, it is necessary that all elements agree on the type definitions,
+and which properties are required for each type. The GStreamer framework
+itself simply provides the ability to define types and parameters, but
+does not fix the meaning of types and parameters, and does not enforce
+standards on the creation of new types. This is a matter for a policy to
+decide, not technical systems to enforce.
+
+For now, the policy is simple:
+
+ - Do not create a new type if you could use one which already exists.
+
+ - If creating a new type, discuss it first with the other GStreamer
+ developers, on at least one of: IRC, mailing lists.
+
+ - Try to ensure that the name for a new format is as unlikely to
+ conflict with anything else created already, and is not a more
+ generalised name than it should be. For example: "audio/compressed"
+ would be too generalised a name to represent audio data compressed
+ with an mp3 codec. Instead "audio/mp3" might be an appropriate name,
+ or "audio/compressed" could exist and have a property indicating the
+ type of compression used.
+
+ - Ensure that, when you do create a new type, you specify it clearly,
+ and get it added to the list of known types so that other developers
+ can use the type correctly when writing their elements.
+
+## Building a Simple Format for Testing
+
+If you need a new format that has not yet been defined in our [List of
+Defined Types](#list-of-defined-types), you will want to have some
+general guidelines on media type naming, properties and such. A media
+type would ideally be equivalent to the Mime-type defined by IANA; else,
+it should be in the form type/x-name, where type is the sort of data
+this media type handles (audio, video, ...) and name should be something
+specific for this specific type. Audio and video media types should try
+to support the general audio/video properties (see the list), and can
+use their own properties, too. To get an idea of what properties we
+think are useful, see (again) the list.
+
+Take your time to find the right set of properties for your type. There
+is no reason to hurry. Also, experimenting with this is generally a good
+idea. Experience learns that theoretically thought-out types are good,
+but they still need practical use to assure that they serve their needs.
+Make sure that your property names do not clash with similar properties
+used in other types. If they match, make sure they mean the same thing;
+properties with different types but the same names are *not* allowed.
+
+## Typefind Functions and Autoplugging
+
+With only *defining* the types, we're not yet there. In order for a
+random data file to be recognized and played back as such, we need a way
+of recognizing their type out of the blue. For this purpose,
+“typefinding” was introduced. Typefinding is the process of detecting
+the type of a data stream. Typefinding consists of two separate parts:
+first, there's an unlimited number of functions that we call *typefind
+functions*, which are each able to recognize one or more types from an
+input stream. Then, secondly, there's a small engine which registers and
+calls each of those functions. This is the typefind core. On top of this
+typefind core, you would normally write an autoplugger, which is able to
+use this type detection system to dynamically build a pipeline around an
+input stream. Here, we will focus only on typefind functions.
+
+A typefind function usually lives in
+`gst-plugins-base/gst/typefind/gsttypefindfunctions.c`, unless there's a
+good reason (like library dependencies) to put it elsewhere. The reason
+for this centralization is to reduce the number of plugins that need to
+be loaded in order to detect a stream's type. Below is an example that
+will recognize AVI files, which start with a “RIFF” tag, then the size
+of the file and then an “AVI” tag:
+
+``` c
+static void
+gst_my_typefind_function (GstTypeFind *tf,
+ gpointer data)
+{
+ guint8 *data = gst_type_find_peek (tf, 0, 12);
+
+ if (data &&
+ GUINT32_FROM_LE (&((guint32 *) data)[0]) == GST_MAKE_FOURCC ('R','I','F','F') &&
+ GUINT32_FROM_LE (&((guint32 *) data)[2]) == GST_MAKE_FOURCC ('A','V','I',' ')) {
+ gst_type_find_suggest (tf, GST_TYPE_FIND_MAXIMUM,
+ gst_caps_new_simple ("video/x-msvideo", NULL));
+ }
+}
+
+static gboolean
+plugin_init (GstPlugin *plugin)
+{
+ if (!gst_type_find_register (plugin, "", GST_RANK_PRIMARY,
+ gst_my_typefind_function, "avi",
+ gst_caps_new_simple ("video/x-msvideo",
+ NULL), NULL))
+ return FALSE;
+}
+
+```
+
+Note that `gst-plugins/gst/typefind/gsttypefindfunctions.c` has some
+simplification macros to decrease the amount of code. Make good use of
+those if you want to submit typefinding patches with new typefind
+functions.
+
+Autoplugging has been discussed in great detail in the Application
+Development Manual.
+
+## List of Defined Types
+
+Below is a list of all the defined types in GStreamer. They are split up
+in separate tables for audio, video, container, subtitle and other
+types, for the sake of readability. Below each table might follow a list
+of notes that apply to that table. In the definition of each type, we
+try to follow the types and rules as defined by
+[IANA](http://www.iana.org/assignments/media-types) for as far as
+possible.
+
+Jump directly to a specific table:
+
+ - [Table of Audio Types](#table-of-audio-types)
+
+ - [Table of Video Types](#table-of-video-types)
+
+ - [Table of Container Types](#table-of-container-types)
+
+ - [Table of Subtitle Types](#table-of-subtitle-types)
+
+ - [Table of Other Types](#table-of-other-types)
+
+Note that many of the properties are not *required*, but rather
+*optional* properties. This means that most of these properties can be
+extracted from the container header, but that - in case the container
+header does not provide these - they can also be extracted by parsing
+the stream header or the stream content. The policy is that your element
+should provide the data that it knows about by only parsing its own
+content, not another element's content. Example: the AVI header provides
+samplerate of the contained audio stream in the header. MPEG system
+streams don't. This means that an AVI stream demuxer would provide
+samplerate as a property for MPEG audio streams, whereas an MPEG demuxer
+would not. A decoder needing this data would require a stream parser in
+between two extract this from the header or calculate it from the
+stream.
+
+<table>
+<caption>Table of Audio Types</caption>
+<colgroup>
+<col width="14%" />
+<col width="85%" />
+</colgroup>
+<thead>
+<tr class="header">
+<th>Media Type</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr class="odd">
+<td><em>All audio types.</em></td>
+</tr>
+<tr class="even">
+<td>audio/*</td>
+<td><em>All audio types</em></td>
+</tr>
+<tr class="odd">
+<td>channels</td>
+<td>integer</td>
+</tr>
+<tr class="even">
+<td>channel-mask</td>
+<td>bitmask</td>
+</tr>
+<tr class="odd">
+<td>format</td>
+<td>string</td>
+</tr>
+<tr class="even">
+<td>layout</td>
+<td>string</td>
+</tr>
+<tr class="odd">
+<td><em>All raw audio types.</em></td>
+</tr>
+<tr class="even">
+<td>audio/x-raw</td>
+<td>Unstructured and uncompressed raw audio data.</td>
+</tr>
+<tr class="odd">
+<td><em>All encoded audio types.</em></td>
+</tr>
+<tr class="even">
+<td>audio/x-ac3</td>
+<td>AC-3 or A52 audio streams.</td>
+</tr>
+<tr class="odd">
+<td>audio/x-adpcm</td>
+<td>ADPCM Audio streams.</td>
+</tr>
+<tr class="even">
+<td>block_align</td>
+<td>integer</td>
+</tr>
+<tr class="odd">
+<td>audio/x-cinepak</td>
+<td>Audio as provided in a Cinepak (Quicktime) stream.</td>
+</tr>
+<tr class="even">
+<td>audio/x-dv</td>
+<td>Audio as provided in a Digital Video stream.</td>
+</tr>
+<tr class="odd">
+<td>audio/x-flac</td>
+<td>Free Lossless Audio codec (FLAC).</td>
+</tr>
+<tr class="even">
+<td>audio/x-gsm</td>
+<td>Data encoded by the GSM codec.</td>
+</tr>
+<tr class="odd">
+<td>audio/x-alaw</td>
+<td>A-Law Audio.</td>
+</tr>
+<tr class="even">
+<td>audio/x-mulaw</td>
+<td>Mu-Law Audio.</td>
+</tr>
+<tr class="odd">
+<td>audio/x-mace</td>
+<td>MACE Audio (used in Quicktime).</td>
+</tr>
+<tr class="even">
+<td>audio/mpeg</td>
+<td>Audio data compressed using the MPEG audio encoding scheme.</td>
+</tr>
+<tr class="odd">
+<td>framed</td>
+<td>boolean</td>
+</tr>
+<tr class="even">
+<td>layer</td>
+<td>integer</td>
+</tr>
+<tr class="odd">
+<td>bitrate</td>
+<td>integer</td>
+</tr>
+<tr class="even">
+<td>audio/x-qdm2</td>
+<td>Data encoded by the QDM version 2 codec.</td>
+</tr>
+<tr class="odd">
+<td>audio/x-pn-realaudio</td>
+<td>Realmedia Audio data.</td>
+</tr>
+<tr class="even">
+<td>audio/x-speex</td>
+<td>Data encoded by the Speex audio codec</td>
+</tr>
+<tr class="odd">
+<td>audio/x-vorbis</td>
+<td>Vorbis audio data</td>
+</tr>
+<tr class="even">
+<td>audio/x-wma</td>
+<td>Windows Media Audio</td>
+</tr>
+<tr class="odd">
+<td>audio/x-paris</td>
+<td>Ensoniq PARIS audio</td>
+</tr>
+<tr class="even">
+<td>audio/x-svx</td>
+<td>Amiga IFF / SVX8 / SV16 audio</td>
+</tr>
+<tr class="odd">
+<td>audio/x-nist</td>
+<td>Sphere NIST audio</td>
+</tr>
+<tr class="even">
+<td>audio/x-voc</td>
+<td>Sound Blaster VOC audio</td>
+</tr>
+<tr class="odd">
+<td>audio/x-ircam</td>
+<td>Berkeley/IRCAM/CARL audio</td>
+</tr>
+<tr class="even">
+<td>audio/x-w64</td>
+<td>Sonic Foundry's 64 bit RIFF/WAV</td>
+</tr>
+</tbody>
+</table>
+
+<table>
+<caption>Table of Video Types</caption>
+<colgroup>
+<col width="14%" />
+<col width="85%" />
+</colgroup>
+<thead>
+<tr class="header">
+<th>Media Type</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr class="odd">
+<td><em>All video types.</em></td>
+</tr>
+<tr class="even">
+<td>video/*</td>
+<td><em>All video types</em></td>
+</tr>
+<tr class="odd">
+<td>height</td>
+<td>integer</td>
+</tr>
+<tr class="even">
+<td>framerate</td>
+<td>fraction</td>
+</tr>
+<tr class="odd">
+<td>max-framerate</td>
+<td>fraction</td>
+</tr>
+<tr class="even">
+<td>views</td>
+<td>integer</td>
+</tr>
+<tr class="odd">
+<td>interlace-mode</td>
+<td>string</td>
+</tr>
+<tr class="even">
+<td>chroma-site</td>
+<td>string</td>
+</tr>
+<tr class="odd">
+<td>colorimetry</td>
+<td>string</td>
+</tr>
+<tr class="even">
+<td>pixel-aspect-ratio</td>
+<td>fraction</td>
+</tr>
+<tr class="odd">
+<td>format</td>
+<td>string</td>
+</tr>
+<tr class="even">
+<td><em>All raw video types.</em></td>
+</tr>
+<tr class="odd">
+<td>video/x-raw</td>
+<td>Unstructured and uncompressed raw video data.</td>
+</tr>
+<tr class="even">
+<td><em>All encoded video types.</em></td>
+</tr>
+<tr class="odd">
+<td>video/x-3ivx</td>
+<td>3ivx video.</td>
+</tr>
+<tr class="even">
+<td>video/x-divx</td>
+<td>DivX video.</td>
+</tr>
+<tr class="odd">
+<td>video/x-dv</td>
+<td>Digital Video.</td>
+</tr>
+<tr class="even">
+<td>video/x-ffv</td>
+<td>FFMpeg video.</td>
+</tr>
+<tr class="odd">
+<td>video/x-h263</td>
+<td>H-263 video.</td>
+</tr>
+<tr class="even">
+<td>h263version</td>
+<td>string</td>
+</tr>
+<tr class="odd">
+<td>video/x-h264</td>
+<td>H-264 video.</td>
+</tr>
+<tr class="even">
+<td>video/x-huffyuv</td>
+<td>Huffyuv video.</td>
+</tr>
+<tr class="odd">
+<td>video/x-indeo</td>
+<td>Indeo video.</td>
+</tr>
+<tr class="even">
+<td>video/x-intel-h263</td>
+<td>H-263 video.</td>
+</tr>
+<tr class="odd">
+<td>video/x-jpeg</td>
+<td>Motion-JPEG video.</td>
+</tr>
+<tr class="even">
+<td>video/mpeg</td>
+<td>MPEG video.</td>
+</tr>
+<tr class="odd">
+<td>systemstream</td>
+<td>boolean</td>
+</tr>
+<tr class="even">
+<td>video/x-msmpeg</td>
+<td>Microsoft MPEG-4 video deviations.</td>
+</tr>
+<tr class="odd">
+<td>video/x-msvideocodec</td>
+<td>Microsoft Video 1 (oldish codec).</td>
+</tr>
+<tr class="even">
+<td>video/x-pn-realvideo</td>
+<td>Realmedia video.</td>
+</tr>
+<tr class="odd">
+<td>video/x-rle</td>
+<td>RLE animation format.</td>
+</tr>
+<tr class="even">
+<td>depth</td>
+<td>integer</td>
+</tr>
+<tr class="odd">
+<td>palette_data</td>
+<td>GstBuffer</td>
+</tr>
+<tr class="even">
+<td>video/x-svq</td>
+<td>Sorensen Video.</td>
+</tr>
+<tr class="odd">
+<td>video/x-tarkin</td>
+<td>Tarkin video.</td>
+</tr>
+<tr class="even">
+<td>video/x-theora</td>
+<td>Theora video.</td>
+</tr>
+<tr class="odd">
+<td>video/x-vp3</td>
+<td>VP-3 video.</td>
+</tr>
+<tr class="even">
+<td>video/x-wmv</td>
+<td>Windows Media Video</td>
+</tr>
+<tr class="odd">
+<td>video/x-xvid</td>
+<td>XviD video.</td>
+</tr>
+<tr class="even">
+<td><em>All image types.</em></td>
+</tr>
+<tr class="odd">
+<td>image/gif</td>
+<td>Graphics Interchange Format.</td>
+</tr>
+<tr class="even">
+<td>image/jpeg</td>
+<td>Joint Picture Expert Group Image.</td>
+</tr>
+<tr class="odd">
+<td>image/png</td>
+<td>Portable Network Graphics Image.</td>
+</tr>
+<tr class="even">
+<td>image/tiff</td>
+<td>Tagged Image File Format.</td>
+</tr>
+</tbody>
+</table>
+
+<table>
+<caption>Table of Container Types</caption>
+<colgroup>
+<col width="14%" />
+<col width="85%" />
+</colgroup>
+<thead>
+<tr class="header">
+<th>Media Type</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr class="odd">
+<td>video/x-ms-asf</td>
+<td>Advanced Streaming Format (ASF).</td>
+</tr>
+<tr class="even">
+<td>video/x-msvideo</td>
+<td>AVI.</td>
+</tr>
+<tr class="odd">
+<td>video/x-dv</td>
+<td>Digital Video.</td>
+</tr>
+<tr class="even">
+<td>video/x-matroska</td>
+<td>Matroska.</td>
+</tr>
+<tr class="odd">
+<td>video/mpeg</td>
+<td>Motion Pictures Expert Group System Stream.</td>
+</tr>
+<tr class="even">
+<td>application/ogg</td>
+<td>Ogg.</td>
+</tr>
+<tr class="odd">
+<td>video/quicktime</td>
+<td>Quicktime.</td>
+</tr>
+<tr class="even">
+<td>application/vnd.rn-realmedia</td>
+<td>RealMedia.</td>
+</tr>
+<tr class="odd">
+<td>audio/x-wav</td>
+<td>WAV.</td>
+</tr>
+</tbody>
+</table>
+
+<table>
+<caption>Table of Subtitle Types</caption>
+<colgroup>
+<col width="14%" />
+<col width="85%" />
+</colgroup>
+<thead>
+<tr class="header">
+<th>Media Type</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr class="odd">
+<td></td>
+<td></td>
+</tr>
+</tbody>
+</table>
+
+<table>
+<caption>Table of Other Types</caption>
+<colgroup>
+<col width="14%" />
+<col width="85%" />
+</colgroup>
+<thead>
+<tr class="header">
+<th>Media Type</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr class="odd">
+<td></td>
+<td></td>
+</tr>
+</tbody>
+</table>
+
--- /dev/null
+---
+title: Building a Plugin
+...
+
+# Building a Plugin
+
+You are now ready to learn how to build a plugin. In this part of the
+guide, you will learn how to apply basic GStreamer programming concepts
+to write a simple plugin. The previous parts of the guide have contained
+no explicit example code, perhaps making things a bit abstract and
+difficult to understand. In contrast, this section will present both
+applications and code by following the development of an example audio
+filter plugin called “MyFilter”.
+
+The example filter element will begin with a single input pad and a
+single output pad. The filter will, at first, simply pass media and
+event data from its sink pad to its source pad without modification. But
+by the end of this part of the guide, you will learn to add some more
+interesting functionality, including properties and signal handlers. And
+after reading the next part of the guide, [Advanced Filter
+Concepts](pwg-advanced.md), you will be able to add even more
+functionality to your plugins.
+
--- /dev/null
+---
+title: Things to check when writing an element
+...
+
+# Things to check when writing an element
+
+This chapter contains a fairly random selection of things to take care
+of when writing an element. It's up to you how far you're going to stick
+to those guidelines. However, keep in mind that when you're writing an
+element and hope for it to be included in the mainstream GStreamer
+distribution, it *has to* meet those requirements. As far as possible,
+we will try to explain why those requirements are set.
+
+## About states
+
+ - Make sure the state of an element gets reset when going to `NULL`.
+ Ideally, this should set all object properties to their original
+ state. This function should also be called from \_init.
+
+ - Make sure an element forgets *everything* about its contained stream
+ when going from `PAUSED` to `READY`. In `READY`, all stream states
+ are reset. An element that goes from `PAUSED` to `READY` and back to
+ `PAUSED` should start reading the stream from the start again.
+
+ - People that use `gst-launch` for testing have the tendency to not
+ care about cleaning up. This is *wrong*. An element should be tested
+ using various applications, where testing not only means to “make
+ sure it doesn't crash”, but also to test for memory leaks using
+ tools such as `valgrind`. Elements have to be reusable in a pipeline
+ after having been reset.
+
+## Debugging
+
+ - Elements should *never* use their standard output for debugging
+ (using functions such as `printf
+ ()` or `g_print ()`). Instead, elements should use the logging
+ functions provided by GStreamer, named `GST_DEBUG ()`, `GST_LOG ()`,
+ `GST_INFO ()`, `GST_WARNING ()` and `GST_ERROR ()`. The various
+ logging levels can be turned on and off at runtime and can thus be
+ used for solving issues as they turn up. Instead of `GST_LOG ()` (as
+ an example), you can also use `GST_LOG_OBJECT
+ ()` to print the object that you're logging output for.
+
+ - Ideally, elements should use their own debugging category. Most
+ elements use the following code to do that:
+
+ ``` c
+ GST_DEBUG_CATEGORY_STATIC (myelement_debug);
+ #define GST_CAT_DEFAULT myelement_debug
+
+ [..]
+
+ static void
+ gst_myelement_class_init (GstMyelementClass *klass)
+ {
+ [..]
+ GST_DEBUG_CATEGORY_INIT (myelement_debug, "myelement",
+ 0, "My own element");
+ }
+
+ ```
+
+ At runtime, you can turn on debugging using the commandline option
+ `--gst-debug=myelement:5`.
+
+ - Elements should use GST\_DEBUG\_FUNCPTR when setting pad functions
+ or overriding element class methods, for example:
+
+ ``` c
+ gst_pad_set_event_func (myelement->srcpad,
+ GST_DEBUG_FUNCPTR (my_element_src_event));
+
+ ```
+
+ This makes debug output much easier to read later on.
+
+ - Elements that are aimed for inclusion into one of the GStreamer
+ modules should ensure consistent naming of the element name,
+ structures and function names. For example, if the element type is
+ GstYellowFooDec, functions should be prefixed with
+ gst\_yellow\_foo\_dec\_ and the element should be registered as
+ 'yellowfoodec'. Separate words should be separate in this scheme, so
+ it should be GstFooDec and gst\_foo\_dec, and not GstFoodec and
+ gst\_foodec.
+
+## Querying, events and the like
+
+ - All elements to which it applies (sources, sinks, demuxers) should
+ implement query functions on their pads, so that applications and
+ neighbour elements can request the current position, the stream
+ length (if known) and so on.
+
+ - Elements should make sure they forward events they do not handle
+ with gst\_pad\_event\_default (pad, parent, event) instead of just
+ dropping them. Events should never be dropped unless specifically
+ intended.
+
+ - Elements should make sure they forward queries they do not handle
+ with gst\_pad\_query\_default (pad, parent, query) instead of just
+ dropping them.
+
+## Testing your element
+
+ - `gst-launch` is *not* a good tool to show that your element is
+ finished. Applications such as Rhythmbox and Totem (for GNOME) or
+ AmaroK (for KDE) *are*. `gst-launch` will not test various things
+ such as proper clean-up on reset, event handling, querying and so
+ on.
+
+ - Parsers and demuxers should make sure to check their input. Input
+ cannot be trusted. Prevent possible buffer overflows and the like.
+ Feel free to error out on unrecoverable stream errors. Test your
+ demuxer using stream corruption elements such as `breakmydata`
+ (included in gst-plugins). It will randomly insert, delete and
+ modify bytes in a stream, and is therefore a good test for
+ robustness. If your element crashes when adding this element, your
+ element needs fixing. If it errors out properly, it's good enough.
+ Ideally, it'd just continue to work and forward data as much as
+ possible.
+
+ - Demuxers should not assume that seeking works. Be prepared to work
+ with unseekable input streams (e.g. network sources) as well.
+
+ - Sources and sinks should be prepared to be assigned another clock
+ then the one they expose themselves. Always use the provided clock
+ for synchronization, else you'll get A/V sync issues.
+
--- /dev/null
+---
+title: Supporting Dynamic Parameters
+...
+
+# Supporting Dynamic Parameters
+
+Warning, this part describes 0.10 and is outdated.
+
+Sometimes object properties are not powerful enough to control the
+parameters that affect the behaviour of your element. When this is the
+case you can mark these parameters as being Controllable. Aware
+applications can use the controller subsystem to dynamically adjust the
+property values over time.
+
+## Getting Started
+
+The controller subsystem is contained within the `gstcontroller`
+library. You need to include the header in your element's source file:
+
+``` c
+...
+#include <gst/gst.h>
+#include <gst/controller/gstcontroller.h>
+...
+
+```
+
+Even though the `gstcontroller` library may be linked into the host
+application, you should make sure it is initialized in your
+`plugin_init` function:
+
+``` c
+ static gboolean
+ plugin_init (GstPlugin *plugin)
+ {
+ ...
+ /* initialize library */
+ gst_controller_init (NULL, NULL);
+ ...
+ }
+
+```
+
+It makes no sense for all GObject parameter to be real-time controlled.
+Therefore the next step is to mark controllable parameters. This is done
+by using the special flag `GST_PARAM_CONTROLLABLE`. when setting up
+GObject params in the `_class_init` method.
+
+``` c
+ g_object_class_install_property (gobject_class, PROP_FREQ,
+ g_param_spec_double ("freq", "Frequency", "Frequency of test signal",
+ 0.0, 20000.0, 440.0,
+ G_PARAM_READWRITE | GST_PARAM_CONTROLLABLE | G_PARAM_STATIC_STRINGS));
+
+```
+
+## The Data Processing Loop
+
+In the last section we learned how to mark GObject params as
+controllable. Application developers can then queue parameter changes
+for these parameters. The approach the controller subsystem takes is to
+make plugins responsible for pulling the changes in. This requires just
+one action:
+
+``` c
+ gst_object_sync_values(element,timestamp);
+
+```
+
+This call makes all parameter-changes for the given timestamp active by
+adjusting the GObject properties of the element. Its up to the element
+to determine the synchronisation rate.
+
+### The Data Processing Loop for Video Elements
+
+For video processing elements it is the best to synchronise for every
+frame. That means one would add the `gst_object_sync_values()` call
+described in the previous section to the data processing function of the
+element.
+
+### The Data Processing Loop for Audio Elements
+
+For audio processing elements the case is not as easy as for video
+processing elements. The problem here is that audio has a much higher
+rate. For PAL video one will e.g. process 25 full frames per second, but
+for standard audio it will be 44100 samples. It is rarely useful to
+synchronise controllable parameters that often. The easiest solution is
+also to have just one synchronisation call per buffer processing. This
+makes the control-rate depend on the buffer size.
+
+Elements that need a specific control-rate need to break their data
+processing loop to synchronise every n-samples.
+
--- /dev/null
+---
+title: Plugin Writer's Guide
+short-description: Complete walkthrough for writing your very own GStreamer plugin
+...
+
+# GStreamer Writer's Guide
--- /dev/null
+---
+title: Foundations
+...
+
+# Foundations
+
+This chapter of the guide introduces the basic concepts of GStreamer.
+Understanding these concepts will help you grok the issues involved in
+extending GStreamer. Many of these concepts are explained in greater
+detail in the *GStreamer Application Development Manual*; the basic
+concepts presented here serve mainly to refresh your memory.
+
+## Elements and Plugins
+
+Elements are at the core of GStreamer. In the context of plugin
+development, an *element* is an object derived from the [`
+GstElement`](../../gstreamer/html/GstElement.html) class. Elements
+provide some sort of functionality when linked with other elements: For
+example, a source element provides data to a stream, and a filter
+element acts on the data in a stream. Without elements, GStreamer is
+just a bunch of conceptual pipe fittings with nothing to link. A large
+number of elements ship with GStreamer, but extra elements can also be
+written.
+
+Just writing a new element is not entirely enough, however: You will
+need to encapsulate your element in a *plugin* to enable GStreamer to
+use it. A plugin is essentially a loadable block of code, usually called
+a shared object file or a dynamically linked library. A single plugin
+may contain the implementation of several elements, or just a single
+one. For simplicity, this guide concentrates primarily on plugins
+containing one element.
+
+A *filter* is an important type of element that processes a stream of
+data. Producers and consumers of data are called *source* and *sink*
+elements, respectively. *Bin* elements contain other elements. One type
+of bin is responsible for synchronization of the elements that they
+contain so that data flows smoothly. Another type of bin, called
+*autoplugger* elements, automatically add other elements to the bin and
+links them together so that they act as a filter between two arbitrary
+stream types.
+
+The plugin mechanism is used everywhere in GStreamer, even if only the
+standard packages are being used. A few very basic functions reside in
+the core library, and all others are implemented in plugins. A plugin
+registry is used to store the details of the plugins in an binary
+registry file. This way, a program using GStreamer does not have to load
+all plugins to determine which are needed. Plugins are only loaded when
+their provided elements are requested.
+
+See the *GStreamer Library Reference* for the current implementation
+details of [`GstElement`](../../gstreamer/html/GstElement.html) and
+[`GstPlugin`](../../gstreamer/html/GstPlugin.html).
+
+## Pads
+
+*Pads* are used to negotiate links and data flow between elements in
+GStreamer. A pad can be viewed as a “place” or “port” on an element
+where links may be made with other elements, and through which data can
+flow to or from those elements. Pads have specific data handling
+capabilities: A pad can restrict the type of data that flows through it.
+Links are only allowed between two pads when the allowed data types of
+the two pads are compatible.
+
+An analogy may be helpful here. A pad is similar to a plug or jack on a
+physical device. Consider, for example, a home theater system consisting
+of an amplifier, a DVD player, and a (silent) video projector. Linking
+the DVD player to the amplifier is allowed because both devices have
+audio jacks, and linking the projector to the DVD player is allowed
+because both devices have compatible video jacks. Links between the
+projector and the amplifier may not be made because the projector and
+amplifier have different types of jacks. Pads in GStreamer serve the
+same purpose as the jacks in the home theater system.
+
+For the most part, all data in GStreamer flows one way through a link
+between elements. Data flows out of one element through one or more
+*source pads*, and elements accept incoming data through one or more
+*sink pads*. Source and sink elements have only source and sink pads,
+respectively.
+
+See the *GStreamer Library Reference* for the current implementation
+details of a [`GstPad`](../../gstreamer/html/GstPad.html).
+
+## GstMiniObject, Buffers and Events
+
+All streams of data in GStreamer are chopped up into chunks that are
+passed from a source pad on one element to a sink pad on another
+element. *GstMiniObject* is the structure used to hold these chunks of
+data.
+
+GstMiniObject contains the following important types:
+
+ - An exact type indicating what type of data (event, buffer, ...) this
+ GstMiniObject is.
+
+ - A reference count indicating the number of elements currently
+ holding a reference to the miniobject. When the reference count
+ falls to zero, the miniobject will be disposed, and its memory will
+ be freed in some sense (see below for more details).
+
+For data transport, there are two types of GstMiniObject defined: events
+(control) and buffers (content).
+
+Buffers may contain any sort of data that the two linked pads know how
+to handle. Normally, a buffer contains a chunk of some sort of audio or
+video data that flows from one element to another.
+
+Buffers also contain metadata describing the buffer's contents. Some of
+the important types of metadata are:
+
+ - Pointers to one or more GstMemory objects. GstMemory objects are
+ refcounted objects that encapsulate a region of memory.
+
+ - A timestamp indicating the preferred display timestamp of the
+ content in the buffer.
+
+Events contain information on the state of the stream flowing between
+the two linked pads. Events will only be sent if the element explicitly
+supports them, else the core will (try to) handle the events
+automatically. Events are used to indicate, for example, a media type,
+the end of a media stream or that the cache should be flushed.
+
+Events may contain several of the following items:
+
+ - A subtype indicating the type of the contained event.
+
+ - The other contents of the event depend on the specific event type.
+
+Events will be discussed extensively in [Events: Seeking, Navigation and
+More](pwg-advanced-events.md). Until then, the only event that will
+be used is the *EOS* event, which is used to indicate the end-of-stream
+(usually end-of-file).
+
+See the *GStreamer Library Reference* for the current implementation
+details of a
+[`GstMiniObject`](../../gstreamer/html/gstreamer-GstMiniObject.html),
+[`GstBuffer`](../../gstreamer/html/GstBuffer.html) and
+[`GstEvent`](../../gstreamer/html/GstEvent.html).
+
+### Buffer Allocation
+
+Buffers are able to store chunks of memory of several different types.
+The most generic type of buffer contains memory allocated by malloc().
+Such buffers, although convenient, are not always very fast, since data
+often needs to be specifically copied into the buffer.
+
+Many specialized elements create buffers that point to special memory.
+For example, the filesrc element usually maps a file into the address
+space of the application (using mmap()), and creates buffers that point
+into that address range. These buffers created by filesrc act exactly
+like generic buffers, except that they are read-only. The buffer freeing
+code automatically determines the correct method of freeing the
+underlying memory. Downstream elements that receive these kinds of
+buffers do not need to do anything special to handle or unreference it.
+
+Another way an element might get specialized buffers is to request them
+from a downstream peer through a GstBufferPool or GstAllocator. Elements
+can ask a GstBufferPool or GstAllocator from the downstream peer
+element. If downstream is able to provide these objects, upstream can
+use them to allocate buffers. See more in [Memory
+allocation](pwg-allocation.md).
+
+Many sink elements have accelerated methods for copying data to
+hardware, or have direct access to hardware. It is common for these
+elements to be able to create a GstBufferPool or GstAllocator for their
+upstream peers. One such example is ximagesink. It creates buffers that
+contain XImages. Thus, when an upstream peer copies data into the
+buffer, it is copying directly into the XImage, enabling ximagesink to
+draw the image directly to the screen instead of having to copy data
+into an XImage first.
+
+Filter elements often have the opportunity to either work on a buffer
+in-place, or work while copying from a source buffer to a destination
+buffer. It is optimal to implement both algorithms, since the GStreamer
+framework can choose the fastest algorithm as appropriate. Naturally,
+this only makes sense for strict filters -- elements that have exactly
+the same format on source and sink pads.
+
+## Media types and Properties
+
+GStreamer uses a type system to ensure that the data passed between
+elements is in a recognized format. The type system is also important
+for ensuring that the parameters required to fully specify a format
+match up correctly when linking pads between elements. Each link that is
+made between elements has a specified type and optionally a set of
+properties. See more about caps negotiation in [Caps
+negotiation](pwg-negotiation.md).
+
+### The Basic Types
+
+GStreamer already supports many basic media types. Following is a table
+of a few of the basic types used for buffers in GStreamer. The table
+contains the name ("media type") and a description of the type, the
+properties associated with the type, and the meaning of each property. A
+full list of supported types is included in [List of Defined
+Types](pwg-building-types.md#list-of-defined-types).
+
+<table>
+<caption>Table of Example Types</caption>
+<thead>
+<tr class="header">
+<th>Media Type</th>
+<th>Description</th>
+<th>Property</th>
+<th>Property Type</th>
+<th>Property Values</th>
+<th>Property Description</th>
+</tr>
+</thead>
+<tbody>
+<tr class="odd">
+<td>audio/*</td>
+<td><em>All audio types</em></td>
+<td>rate</td>
+<td>integer</td>
+<td>greater than 0</td>
+<td>The sample rate of the data, in samples (per channel) per second.</td>
+</tr>
+<tr class="even">
+<td>channels</td>
+<td>integer</td>
+<td>greater than 0</td>
+<td>The number of channels of audio data.</td>
+</tr>
+<tr class="odd">
+<td>audio/x-raw</td>
+<td>Unstructured and uncompressed raw integer audio data.</td>
+<td>format</td>
+<td>string</td>
+<td>S8 U8 S16LE S16BE U16LE U16BE S24_32LE S24_32BE U24_32LE U24_32BE S32LE S32BE U32LE U32BE S24LE S24BE U24LE U24BE S20LE S20BE U20LE U20BE S18LE S18BE U18LE U18BE F32LE F32BE F64LE F64BE</td>
+<td>The format of the sample data.</td>
+</tr>
+<tr class="even">
+<td>audio/mpeg</td>
+<td>Audio data compressed using the MPEG audio encoding scheme.</td>
+<td>mpegversion</td>
+<td>integer</td>
+<td>1, 2 or 4</td>
+<td>The MPEG-version used for encoding the data. The value 1 refers to MPEG-1, -2 and -2.5 layer 1, 2 or 3. The values 2 and 4 refer to the MPEG-AAC audio encoding schemes.</td>
+</tr>
+<tr class="odd">
+<td>framed</td>
+<td>boolean</td>
+<td>0 or 1</td>
+<td>A true value indicates that each buffer contains exactly one frame. A false value indicates that frames and buffers do not necessarily match up.</td>
+</tr>
+<tr class="even">
+<td>layer</td>
+<td>integer</td>
+<td>1, 2, or 3</td>
+<td>The compression scheme layer used to compress the data <em>(only if mpegversion=1)</em>.</td>
+</tr>
+<tr class="odd">
+<td>bitrate</td>
+<td>integer</td>
+<td>greater than 0</td>
+<td>The bitrate, in bits per second. For VBR (variable bitrate) MPEG data, this is the average bitrate.</td>
+</tr>
+<tr class="even">
+<td>audio/x-vorbis</td>
+<td>Vorbis audio data</td>
+<td></td>
+<td></td>
+<td></td>
+<td>There are currently no specific properties defined for this type.</td>
+</tr>
+</tbody>
+</table>
+
--- /dev/null
+---
+title: Preface
+...
+
+# Preface
+
+## What is GStreamer?
+
+GStreamer is a framework for creating streaming media applications. The
+fundamental design comes from the video pipeline at Oregon Graduate
+Institute, as well as some ideas from DirectShow.
+
+GStreamer's development framework makes it possible to write any type of
+streaming multimedia application. The GStreamer framework is designed to
+make it easy to write applications that handle audio or video or both.
+It isn't restricted to audio and video, and can process any kind of data
+flow. The pipeline design is made to have little overhead above what the
+applied filters induce. This makes GStreamer a good framework for
+designing even high-end audio applications which put high demands on
+latency or performance.
+
+One of the most obvious uses of GStreamer is using it to build a media
+player. GStreamer already includes components for building a media
+player that can support a very wide variety of formats, including MP3,
+Ogg/Vorbis, MPEG-1/2, AVI, Quicktime, mod, and more. GStreamer, however,
+is much more than just another media player. Its main advantages are
+that the pluggable components can be mixed and matched into arbitrary
+pipelines so that it's possible to write a full-fledged video or audio
+editing application.
+
+The framework is based on plugins that will provide the various codec
+and other functionality. The plugins can be linked and arranged in a
+pipeline. This pipeline defines the flow of the data.
+
+The GStreamer core function is to provide a framework for plugins, data
+flow, synchronization and media type handling/negotiation. It also
+provides an API to write applications using the various plugins.
+
+## Who Should Read This Guide?
+
+This guide explains how to write new modules for GStreamer. The guide is
+relevant to several groups of people:
+
+ - Anyone who wants to add support for new ways of processing data in
+ GStreamer. For example, a person in this group might want to create
+ a new data format converter, a new visualization tool, or a new
+ decoder or encoder.
+
+ - Anyone who wants to add support for new input and output devices.
+ For example, people in this group might want to add the ability to
+ write to a new video output system or read data from a digital
+ camera or special microphone.
+
+ - Anyone who wants to extend GStreamer in any way. You need to have an
+ understanding of how the plugin system works before you can
+ understand the constraints that the plugin system places on the rest
+ of the code. Also, you might be surprised after reading this at how
+ much can be done with plugins.
+
+This guide is not relevant to you if you only want to use the existing
+functionality of GStreamer, or if you just want to use an application
+that uses GStreamer. If you are only interested in using existing
+plugins to write a new application - and there are quite a lot of
+plugins already - you might want to check the *GStreamer Application
+Development Manual*. If you are just trying to get help with a GStreamer
+application, then you should check with the user manual for that
+particular application.
+
+## Preliminary Reading
+
+This guide assumes that you are somewhat familiar with the basic
+workings of GStreamer. For a gentle introduction to programming concepts
+in GStreamer, you may wish to read the *GStreamer Application
+Development Manual* first. Also check out the other documentation
+available on the [GStreamer web
+site](http://gstreamer.freedesktop.org/documentation/).
+
+In order to understand this manual, you will need to have a basic
+understanding of the C language. Since GStreamer adheres to the GObject
+programming model, this guide also assumes that you understand the
+basics of [GObject](http://developer.gnome.org/gobject/stable/pt01.html)
+programming. You may also want to have a look at Eric Harlow's book
+*Developing Linux Applications with GTK+ and GDK*.
+
+## Structure of This Guide
+
+To help you navigate through this guide, it is divided into several
+large parts. Each part addresses a particular broad topic concerning
+GStreamer plugin development. The parts of this guide are laid out in
+the following order:
+
+ - [Building a Plugin](pwg-building.md) - Introduction to the
+ structure of a plugin, using an example audio filter for
+ illustration.
+
+ This part covers all the basic steps you generally need to perform
+ to build a plugin, such as registering the element with GStreamer
+ and setting up the basics so it can receive data from and send data
+ to neighbour elements. The discussion begins by giving examples of
+ generating the basic structures and registering an element in
+ [Constructing the Boilerplate](pwg-building-boiler.md). Then,
+ you will learn how to write the code to get a basic filter plugin
+ working in [Specifying the pads](pwg-building-pads.md), [The
+ chain function](pwg-building-chainfn.md) and [What are
+ states?](pwg-statemanage-states.md).
+
+ After that, we will show some of the GObject concepts on how to make
+ an element configurable for applications and how to do
+ application-element interaction in [Adding
+ Properties](pwg-building-args.md) and
+ [Signals](pwg-building-signals.md). Next, you will learn to
+ build a quick test application to test all that you've just learned
+ in [Building a Test Application](pwg-building-testapp.md). We
+ will just touch upon basics here. For full-blown application
+ development, you should look at [the Application Development
+ Manual](http://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/index.html).
+
+ - [Advanced Filter Concepts](pwg-advanced.md) - Information on
+ advanced features of GStreamer plugin development.
+
+ After learning about the basic steps, you should be able to create a
+ functional audio or video filter plugin with some nice features.
+ However, GStreamer offers more for plugin writers. This part of the
+ guide includes chapters on more advanced topics, such as scheduling,
+ media type definitions in GStreamer, clocks, interfaces and tagging.
+ Since these features are purpose-specific, you can read them in any
+ order, most of them don't require knowledge from other sections.
+
+ The first chapter, named [Different scheduling
+ modes](pwg-scheduling.md), will explain some of the basics of
+ element scheduling. It is not very in-depth, but is mostly some sort
+ of an introduction on why other things work as they do. Read this
+ chapter if you're interested in GStreamer internals. Next, we will
+ apply this knowledge and discuss another type of data transmission
+ than what you learned in [The chain
+ function](pwg-building-chainfn.md): [Different scheduling
+ modes](pwg-scheduling.md). Loop-based elements will give you
+ more control over input rate. This is useful when writing, for
+ example, muxers or demuxers.
+
+ Next, we will discuss media identification in GStreamer in [Types
+ and Properties](pwg-building-types.md). You will learn how to
+ define new media types and get to know a list of standard media
+ types defined in GStreamer.
+
+ In the next chapter, you will learn the concept of request- and
+ sometimes-pads, which are pads that are created dynamically, either
+ because the application asked for it (request) or because the media
+ stream requires it (sometimes). This will be in [Request and
+ Sometimes pads](pwg-advanced-request.md).
+
+ The next chapter, [Clocking](pwg-advanced-clock.md), will
+ explain the concept of clocks in GStreamer. You need this
+ information when you want to know how elements should achieve
+ audio/video synchronization.
+
+ The next few chapters will discuss advanced ways of doing
+ application-element interaction. Previously, we learned on the
+ GObject-ways of doing this in [Adding
+ Properties](pwg-building-args.md) and
+ [Signals](pwg-building-signals.md). We will discuss dynamic
+ parameters, which are a way of defining element behaviour over time
+ in advance, in [Supporting Dynamic Parameters](pwg-dparams.md).
+ Next, you will learn about interfaces in
+ [Interfaces](pwg-advanced-interfaces.md). Interfaces are very
+ target- specific ways of application-element interaction, based on
+ GObject's GInterface. Lastly, you will learn about how metadata is
+ handled in GStreamer in [Tagging (Metadata and
+ Streaminfo)](pwg-advanced-tagging.md).
+
+ The last chapter, [Events: Seeking, Navigation and
+ More](pwg-advanced-events.md), will discuss the concept of
+ events in GStreamer. Events are, on the one hand, another way of
+ doing application-element interaction. It takes care of seeking, for
+ example. On the other hand, it is also a way in which elements
+ interact with each other, such as letting each other know about
+ media stream discontinuities, forwarding tags inside a pipeline and
+ so on.
+
+ - [Creating special element types](pwg-other.md) - Explanation of
+ writing other plugin types.
+
+ Because the first two parts of the guide use an audio filter as an
+ example, the concepts introduced apply to filter plugins. But many
+ of the concepts apply equally to other plugin types, including
+ sources, sinks, and autopluggers. This part of the guide presents
+ the issues that arise when working on these more specialized plugin
+ types. The chapter starts with a special focus on elements that can
+ be written using a base-class ([Pre-made base
+ classes](pwg-other-base.md)), and later also goes into writing
+ special types of elements in [Writing a Demuxer or
+ Parser](pwg-other-oneton.md), [Writing a N-to-1 Element or
+ Muxer](pwg-other-ntoone.md) and [Writing a
+ Manager](pwg-other-manager.md).
+
+ - [Appendices](pwg-appendix.md) - Further information for plugin
+ developers.
+
+ The appendices contain some information that stubbornly refuses to
+ fit cleanly in other sections of the guide. Most of this section is
+ not yet finished.
+
+The remainder of this introductory part of the guide presents a short
+overview of the basic concepts involved in GStreamer plugin development.
+Topics covered include [Elements and
+Plugins](pwg-intro-basics.md#elements-and-plugins),
+[Pads](pwg-intro-basics.md#pads), [GstMiniObject, Buffers and
+Events](pwg-intro-basics.md#gstminiobject-buffers-and-events) and
+[Media types and
+Properties](pwg-intro-basics.md#media-types-and-properties). If you
+are already familiar with this information, you can use this short
+overview to refresh your memory, or you can skip to [Building a
+Plugin](pwg-building.md).
+
+As you can see, there a lot to learn, so let's get started\!
+
+ - Creating compound and complex elements by extending from a GstBin.
+ This will allow you to create plugins that have other plugins
+ embedded in them.
+
+ - Adding new media types to the registry along with typedetect
+ functions. This will allow your plugin to operate on a completely
+ new media type.
+
--- /dev/null
+---
+title: Introduction
+...
+
+# Introduction
+
+GStreamer is an extremely powerful and versatile framework for creating
+streaming media applications. Many of the virtues of the GStreamer
+framework come from its modularity: GStreamer can seamlessly incorporate
+new plugin modules. But because modularity and power often come at a
+cost of greater complexity (consider, for example,
+[CORBA](http://www.omg.org/)), writing new plugins is not always easy.
+
+This guide is intended to help you understand the GStreamer framework
+ so you can develop new plugins to extend the existing
+functionality. The guide addresses most issues by following the
+development of an example plugin - an audio filter plugin - written in
+C. However, the later parts of the guide also present some issues
+involved in writing other types of plugins, and the end of the guide
+describes some of the Python bindings for GStreamer.
+
--- /dev/null
+---
+title: GStreamer licensing
+...
+
+# GStreamer licensing
+
+## How to license the code you write for GStreamer
+
+GStreamer is a plugin-based framework licensed under the LGPL. The
+reason for this choice in licensing is to ensure that everyone can use
+GStreamer to build applications using licenses of their choice.
+
+To keep this policy viable, the GStreamer community has made a few
+licensing rules for code to be included in GStreamer's core or
+GStreamer's official modules, like our plugin packages. We require that
+all code going into our core package is LGPL. For the plugin code, we
+require the use of the LGPL for all plugins written from scratch or
+linking to external libraries. The only exception to this is when
+plugins contain older code under more liberal licenses (like the MPL or
+BSD). They can use those licenses instead and will still be considered
+for inclusion. We do not accept GPL code to be added to our plugins
+module, but we do accept LGPL-licensed plugins using an external GPL
+library. The reason for demanding plugins be licensed under the LGPL,
+even when using a GPL library, is that other developers might want to
+use the plugin code as a template for plugins linking to non-GPL
+libraries.
+
+We also plan on splitting out the plugins using GPL libraries into a
+separate package eventually and implement a system which makes sure an
+application will not be able to access these plugins unless it uses some
+special code to do so. The point of this is not to block GPL-licensed
+plugins from being used and developed, but to make sure people are not
+unintentionally violating the GPL license of said plugins.
+
+This advisory is part of a bigger advisory with a FAQ which you can find
+on the [GStreamer
+website](http://gstreamer.freedesktop.org/documentation/licensing.html)
+
--- /dev/null
+---
+title: Caps negotiation
+...
+
+# Caps negotiation
+
+Caps negotiation is the act of finding a media format (GstCaps) between
+elements that they can handle. This process in GStreamer can in most
+cases find an optimal solution for the complete pipeline. In this
+section we explain how this works.
+
+## Caps negotiation basics
+
+In GStreamer, negotiation of the media format always follows the
+following simple rules:
+
+ - A downstream element suggest a format on its sinkpad and places the
+ suggestion in the result of the CAPS query performed on the sinkpad.
+ See also [Implementing a CAPS query
+ function](#implementing-a-caps-query-function).
+
+ - An upstream element decides on a format. It sends the selected media
+ format downstream on its source pad with a CAPS event. Downstream
+ elements reconfigure themselves to handle the media type in the CAPS
+ event on the sinkpad.
+
+ - A downstream element can inform upstream that it would like to
+ suggest a new format by sending a RECONFIGURE event upstream. The
+ RECONFIGURE event simply instructs an upstream element to restart
+ the negotiation phase. Because the element that sent out the
+ RECONFIGURE event is now suggesting another format, the format in
+ the pipeline might change.
+
+In addition to the CAPS and RECONFIGURE event and the CAPS query, there
+is an ACCEPT\_CAPS query to quickly check if a certain caps can be
+accepted by an element.
+
+All negotiation follows these simple rules. Let's take a look at some
+typical uses cases and how negotiation happens.
+
+## Caps negotiation use cases
+
+In what follows we will look at some use cases for push-mode scheduling.
+The pull-mode scheduling negotiation phase is discussed in [Pull-mode
+Caps negotiation](#pull-mode-caps-negotiation) and is actually similar
+as we will see.
+
+Since the sink pads only suggest formats and the source pads need to
+decide, the most complicated work is done in the source pads. We can
+identify 3 caps negotiation use cases for the source pads:
+
+ - Fixed negotiation. An element can output one format only. See [Fixed
+ negotiation](#fixed-negotiation).
+
+ - Transform negotiation. There is a (fixed) transform between the
+ input and output format of the element, usually based on some
+ element property. The caps that the element will produce depend on
+ the upstream caps and the caps that the element can accept depend on
+ the downstream caps. See [Transform
+ negotiation](#transform-negotiation).
+
+ - Dynamic negotiation. An element can output many formats. See
+ [Dynamic negotiation](#dynamic-negotiation).
+
+### Fixed negotiation
+
+In this case, the source pad can only produce a fixed format. Usually
+this format is encoded inside the media. No downstream element can ask
+for a different format, the only way that the source pad will
+renegotiate is when the element decides to change the caps itself.
+
+Elements that could implement fixed caps (on their source pads) are, in
+general, all elements that are not renegotiable. Examples include:
+
+ - A typefinder, since the type found is part of the actual data stream
+ and can thus not be re-negotiated. The typefinder will look at the
+ stream of bytes, figure out the type, send a CAPS event with the
+ caps and then push buffers of the type.
+
+ - Pretty much all demuxers, since the contained elementary data
+ streams are defined in the file headers, and thus not renegotiable.
+
+ - Some decoders, where the format is embedded in the data stream and
+ not part of the peercaps *and* where the decoder itself is not
+ reconfigurable, too.
+
+ - Some sources that produce a fixed format.
+
+`gst_pad_use_fixed_caps()` is used on the source pad with fixed caps. As
+long as the pad is not negotiated, the default CAPS query will return
+the caps presented in the padtemplate. As soon as the pad is negotiated,
+the CAPS query will return the negotiated caps (and nothing else). These
+are the relevant code snippets for fixed caps source pads.
+
+``` c
+
+[..]
+ pad = gst_pad_new_from_static_template (..);
+ gst_pad_use_fixed_caps (pad);
+[..]
+
+
+```
+
+The fixed caps can then be set on the pad by calling `gst_pad_set_caps
+()`.
+
+``` c
+
+[..]
+ caps = gst_caps_new_simple ("audio/x-raw",
+ "format", G_TYPE_STRING, GST_AUDIO_NE(F32),
+ "rate", G_TYPE_INT, <samplerate>,
+ "channels", G_TYPE_INT, <num-channels>, NULL);
+ if (!gst_pad_set_caps (pad, caps)) {
+ GST_ELEMENT_ERROR (element, CORE, NEGOTIATION, (NULL),
+ ("Some debug information here"));
+ return GST_FLOW_ERROR;
+ }
+[..]
+
+
+```
+
+These types of elements also don't have a relation between the input
+format and the output format, the input caps simply don't contain the
+information needed to produce the output caps.
+
+All other elements that need to be configured for the format should
+implement full caps negotiation, which will be explained in the next few
+sections.
+
+### Transform negotiation
+
+In this negotiation technique, there is a fixed transform between the
+element input caps and the output caps. This transformation could be
+parameterized by element properties but not by the content of the stream
+(see [Fixed negotiation](#fixed-negotiation) for that use-case).
+
+The caps that the element can accept depend on the (fixed
+transformation) downstream caps. The caps that the element can produce
+depend on the (fixed transformation of) the upstream caps.
+
+This type of element can usually set caps on its source pad from the
+`_event()` function on the sink pad when it received the CAPS event.
+This means that the caps transform function transforms a fixed caps into
+another fixed caps. Examples of elements include:
+
+ - Videobox. It adds configurable border around a video frame depending
+ on object properties.
+
+ - Identity elements. All elements that don't change the format of the
+ data, only the content. Video and audio effects are an example.
+ Other examples include elements that inspect the stream.
+
+ - Some decoders and encoders, where the output format is defined by
+ input format, like mulawdec and mulawenc. These decoders usually
+ have no headers that define the content of the stream. They are
+ usually more like conversion elements.
+
+Below is an example of a negotiation steps of a typical transform
+element. In the sink pad CAPS event handler, we compute the caps for the
+source pad and set those.
+
+``` c
+
+ [...]
+
+static gboolean
+gst_my_filter_setcaps (GstMyFilter *filter,
+ GstCaps *caps)
+{
+ GstStructure *structure;
+ int rate, channels;
+ gboolean ret;
+ GstCaps *outcaps;
+
+ structure = gst_caps_get_structure (caps, 0);
+ ret = gst_structure_get_int (structure, "rate", &rate);
+ ret = ret && gst_structure_get_int (structure, "channels", &channels);
+ if (!ret)
+ return FALSE;
+
+ outcaps = gst_caps_new_simple ("audio/x-raw",
+ "format", G_TYPE_STRING, GST_AUDIO_NE(S16),
+ "rate", G_TYPE_INT, rate,
+ "channels", G_TYPE_INT, channels, NULL);
+ ret = gst_pad_set_caps (filter->srcpad, outcaps);
+ gst_caps_unref (outcaps);
+
+ return ret;
+}
+
+static gboolean
+gst_my_filter_sink_event (GstPad *pad,
+ GstObject *parent,
+ GstEvent *event)
+{
+ gboolean ret;
+ GstMyFilter *filter = GST_MY_FILTER (parent);
+
+ switch (GST_EVENT_TYPE (event)) {
+ case GST_EVENT_CAPS:
+ {
+ GstCaps *caps;
+
+ gst_event_parse_caps (event, &caps);
+ ret = gst_my_filter_setcaps (filter, caps);
+ break;
+ }
+ default:
+ ret = gst_pad_event_default (pad, parent, event);
+ break;
+ }
+ return ret;
+}
+
+ [...]
+
+
+```
+
+### Dynamic negotiation
+
+A last negotiation method is the most complex and powerful dynamic
+negotiation.
+
+Like with the transform negotiation in [Transform
+negotiation](#transform-negotiation), dynamic negotiation will perform a
+transformation on the downstream/upstream caps. Unlike the transform
+negotiation, this transform will convert fixed caps to unfixed caps.
+This means that the sink pad input caps can be converted into unfixed
+(multiple) formats. The source pad will have to choose a format from all
+the possibilities. It would usually like to choose a format that
+requires the least amount of effort to produce but it does not have to
+be. The selection of the format should also depend on the caps that can
+be accepted downstream (see a QUERY\_CAPS function in [Implementing a
+CAPS query function](#implementing-a-caps-query-function)).
+
+A typical flow goes like this:
+
+ - Caps are received on the sink pad of the element.
+
+ - If the element prefers to operate in passthrough mode, check if
+ downstream accepts the caps with the ACCEPT\_CAPS query. If it does,
+ we can complete negotiation and we can operate in passthrough mode.
+
+ - Calculate the possible caps for the source pad.
+
+ - Query the downstream peer pad for the list of possible caps.
+
+ - Select from the downstream list the first caps that you can
+ transform to and set this as the output caps. You might have to
+ fixate the caps to some reasonable defaults to construct fixed caps.
+
+Examples of this type of elements include:
+
+ - Converter elements such as videoconvert, audioconvert,
+ audioresample, videoscale, ...
+
+ - Source elements such as audiotestsrc, videotestsrc, v4l2src,
+ pulsesrc, ...
+
+Let's look at the example of an element that can convert between
+samplerates, so where input and output samplerate don't have to be the
+same:
+
+``` c
+
+static gboolean
+gst_my_filter_setcaps (GstMyFilter *filter,
+ GstCaps *caps)
+{
+ if (gst_pad_set_caps (filter->srcpad, caps)) {
+ filter->passthrough = TRUE;
+ } else {
+ GstCaps *othercaps, *newcaps;
+ GstStructure *s = gst_caps_get_structure (caps, 0), *others;
+
+ /* no passthrough, setup internal conversion */
+ gst_structure_get_int (s, "channels", &filter->channels);
+ othercaps = gst_pad_get_allowed_caps (filter->srcpad);
+ others = gst_caps_get_structure (othercaps, 0);
+ gst_structure_set (others,
+ "channels", G_TYPE_INT, filter->channels, NULL);
+
+ /* now, the samplerate value can optionally have multiple values, so
+ * we "fixate" it, which means that one fixed value is chosen */
+ newcaps = gst_caps_copy_nth (othercaps, 0);
+ gst_caps_unref (othercaps);
+ gst_pad_fixate_caps (filter->srcpad, newcaps);
+ if (!gst_pad_set_caps (filter->srcpad, newcaps))
+ return FALSE;
+
+ /* we are now set up, configure internally */
+ filter->passthrough = FALSE;
+ gst_structure_get_int (s, "rate", &filter->from_samplerate);
+ others = gst_caps_get_structure (newcaps, 0);
+ gst_structure_get_int (others, "rate", &filter->to_samplerate);
+ }
+
+ return TRUE;
+}
+
+static gboolean
+gst_my_filter_sink_event (GstPad *pad,
+ GstObject *parent,
+ GstEvent *event)
+{
+ gboolean ret;
+ GstMyFilter *filter = GST_MY_FILTER (parent);
+
+ switch (GST_EVENT_TYPE (event)) {
+ case GST_EVENT_CAPS:
+ {
+ GstCaps *caps;
+
+ gst_event_parse_caps (event, &caps);
+ ret = gst_my_filter_setcaps (filter, caps);
+ break;
+ }
+ default:
+ ret = gst_pad_event_default (pad, parent, event);
+ break;
+ }
+ return ret;
+}
+
+static GstFlowReturn
+gst_my_filter_chain (GstPad *pad,
+ GstObject *parent,
+ GstBuffer *buf)
+{
+ GstMyFilter *filter = GST_MY_FILTER (parent);
+ GstBuffer *out;
+
+ /* push on if in passthrough mode */
+ if (filter->passthrough)
+ return gst_pad_push (filter->srcpad, buf);
+
+ /* convert, push */
+ out = gst_my_filter_convert (filter, buf);
+ gst_buffer_unref (buf);
+
+ return gst_pad_push (filter->srcpad, out);
+}
+
+
+```
+
+## Upstream caps (re)negotiation
+
+Upstream negotiation's primary use is to renegotiate (part of) an
+already-negotiated pipeline to a new format. Some practical examples
+include to select a different video size because the size of the video
+window changed, and the video output itself is not capable of rescaling,
+or because the audio channel configuration changed.
+
+Upstream caps renegotiation is requested by sending a
+GST\_EVENT\_RECONFIGURE event upstream. The idea is that it will
+instruct the upstream element to reconfigure its caps by doing a new
+query for the allowed caps and then choosing a new caps. The element
+that sends out the RECONFIGURE event would influence the selection of
+the new caps by returning the new preferred caps from its
+GST\_QUERY\_CAPS query function. The RECONFIGURE event will set the
+GST\_PAD\_FLAG\_NEED\_RECONFIGURE on all pads that it travels over.
+
+It is important to note here that different elements actually have
+different responsibilities here:
+
+ - Elements that want to propose a new format upstream need to first
+ check if the new caps are acceptable upstream with an ACCEPT\_CAPS
+ query. Then they would send a RECONFIGURE event and be prepared to
+ answer the CAPS query with the new preferred format. It should be
+ noted that when there is no upstream element that can (or wants) to
+ renegotiate, the element needs to deal with the currently configured
+ format.
+
+ - Elements that operate in transform negotiation according to
+ [Transform negotiation](#transform-negotiation) pass the RECONFIGURE
+ event upstream. Because these elements simply do a fixed transform
+ based on the upstream caps, they need to send the event upstream so
+ that it can select a new format.
+
+ - Elements that operate in fixed negotiation ([Fixed
+ negotiation](#fixed-negotiation)) drop the RECONFIGURE event. These
+ elements can't reconfigure and their output caps don't depend on the
+ upstream caps so the event can be dropped.
+
+ - Elements that can be reconfigured on the source pad (source pads
+ implementing dynamic negotiation in [Dynamic
+ negotiation](#dynamic-negotiation)) should check its
+ NEED\_RECONFIGURE flag with `gst_pad_check_reconfigure ()` and it
+ should start renegotiation when the function returns TRUE.
+
+## Implementing a CAPS query function
+
+A `_query ()`-function with the GST\_QUERY\_CAPS query type is called
+when a peer element would like to know which formats this pad supports,
+and in what order of preference. The return value should be all formats
+that this elements supports, taking into account limitations of peer
+elements further downstream or upstream, sorted by order of preference,
+highest preference first.
+
+``` c
+
+static gboolean
+gst_my_filter_query (GstPad *pad, GstObject * parent, GstQuery * query)
+{
+ gboolean ret;
+ GstMyFilter *filter = GST_MY_FILTER (parent);
+
+ switch (GST_QUERY_TYPE (query)) {
+ case GST_QUERY_CAPS
+ {
+ GstPad *otherpad;
+ GstCaps *temp, *caps, *filt, *tcaps;
+ gint i;
+
+ otherpad = (pad == filter->srcpad) ? filter->sinkpad :
+ filter->srcpad;
+ caps = gst_pad_get_allowed_caps (otherpad);
+
+ gst_query_parse_caps (query, &filt);
+
+ /* We support *any* samplerate, indifferent from the samplerate
+ * supported by the linked elements on both sides. */
+ for (i = 0; i < gst_caps_get_size (caps); i++) {
+ GstStructure *structure = gst_caps_get_structure (caps, i);
+
+ gst_structure_remove_field (structure, "rate");
+ }
+
+ /* make sure we only return results that intersect our
+ * padtemplate */
+ tcaps = gst_pad_get_pad_template_caps (pad);
+ if (tcaps) {
+ temp = gst_caps_intersect (caps, tcaps);
+ gst_caps_unref (caps);
+ gst_caps_unref (tcaps);
+ caps = temp;
+ }
+ /* filter against the query filter when needed */
+ if (filt) {
+ temp = gst_caps_intersect (caps, filt);
+ gst_caps_unref (caps);
+ caps = temp;
+ }
+ gst_query_set_caps_result (query, caps);
+ gst_caps_unref (caps);
+ ret = TRUE;
+ break;
+ }
+ default:
+ ret = gst_pad_query_default (pad, parent, query);
+ break;
+ }
+ return ret;
+}
+
+
+```
+
+## Pull-mode Caps negotiation
+
+WRITEME, the mechanism of pull-mode negotiation is not yet fully
+understood.
+
+Using all the knowledge you've acquired by reading this chapter, you
+should be able to write an element that does correct caps negotiation.
+If in doubt, look at other elements of the same type in our git
+repository to get an idea of how they do what you want to do.
+
--- /dev/null
+---
+title: Pre-made base classes
+...
+
+# Pre-made base classes
+
+So far, we've been looking at low-level concepts of creating any type of
+GStreamer element. Now, let's assume that all you want is to create an
+simple audiosink that works exactly the same as, say, “esdsink”, or a
+filter that simply normalizes audio volume. Such elements are very
+general in concept and since they do nothing special, they should be
+easier to code than to provide your own scheduler activation functions
+and doing complex caps negotiation. For this purpose, GStreamer provides
+base classes that simplify some types of elements. Those base classes
+will be discussed in this chapter.
+
+## Writing a sink
+
+Sinks are special elements in GStreamer. This is because sink elements
+have to take care of *preroll*, which is the process that takes care
+that elements going into the `GST_STATE_PAUSED` state will have buffers
+ready after the state change. The result of this is that such elements
+can start processing data immediately after going into the
+`GST_STATE_PLAYING` state, without requiring to take some time to
+initialize outputs or set up decoders; all that is done already before
+the state-change to `GST_STATE_PAUSED` successfully completes.
+
+Preroll, however, is a complex process that would require the same code
+in many elements. Therefore, sink elements can derive from the
+`GstBaseSink` base-class, which does preroll and a few other utility
+functions automatically. The derived class only needs to implement a
+bunch of virtual functions and will work automatically.
+
+The base class implement much of the synchronization logic that a sink
+has to perform.
+
+The `GstBaseSink` base-class specifies some limitations on elements,
+though:
+
+ - It requires that the sink only has one sinkpad. Sink elements that
+ need more than one sinkpad, must make a manager element with
+ multiple GstBaseSink elements inside.
+
+Sink elements can derive from `GstBaseSink` using the usual `GObject`
+convenience macro `G_DEFINE_TYPE ()`:
+
+``` c
+G_DEFINE_TYPE (GstMySink, gst_my_sink, GST_TYPE_BASE_SINK);
+
+[..]
+
+static void
+gst_my_sink_class_init (GstMySinkClass * klass)
+{
+ klass->set_caps = [..];
+ klass->render = [..];
+[..]
+}
+
+```
+
+The advantages of deriving from `GstBaseSink` are numerous:
+
+ - Derived implementations barely need to be aware of preroll, and do
+ not need to know anything about the technical implementation
+ requirements of preroll. The base-class does all the hard work.
+
+ Less code to write in the derived class, shared code (and thus
+ shared bugfixes).
+
+There are also specialized base classes for audio and video, let's look
+at those a bit.
+
+### Writing an audio sink
+
+Essentially, audio sink implementations are just a special case of a
+general sink. An audio sink has the added complexity that it needs to
+schedule playback of samples. It must match the clock selected in the
+pipeline against the clock of the audio device and calculate and
+compensate for drift and jitter.
+
+There are two audio base classes that you can choose to derive from,
+depending on your needs: `GstAudioBasesink` and `GstAudioSink`. The
+audiobasesink provides full control over how synchronization and
+scheduling is handled, by using a ringbuffer that the derived class
+controls and provides. The audiosink base-class is a derived class of
+the audiobasesink, implementing a standard ringbuffer implementing
+default synchronization and providing a standard audio-sample clock.
+Derived classes of this base class merely need to provide a `_open
+()`, `_close ()` and a `_write
+()` function implementation, and some optional functions. This should
+suffice for many sound-server output elements and even most interfaces.
+More demanding audio systems, such as Jack, would want to implement the
+`GstAudioBaseSink` base-class.
+
+The `GstAudioBaseSink` has little to no limitations and should fit
+virtually every implementation, but is hard to implement. The
+`GstAudioSink`, on the other hand, only fits those systems with a simple
+`open
+()` / `close ()` / `write
+()` API (which practically means pretty much all of them), but has the
+advantage that it is a lot easier to implement. The benefits of this
+second base class are large:
+
+ - Automatic synchronization, without any code in the derived class.
+
+ - Also automatically provides a clock, so that other sinks (e.g. in
+ case of audio/video playback) are synchronized.
+
+ - Features can be added to all audiosinks by making a change in the
+ base class, which makes maintenance easy.
+
+ - Derived classes require only three small functions, plus some
+ `GObject` boilerplate code.
+
+In addition to implementing the audio base-class virtual functions,
+derived classes can (should) also implement the `GstBaseSink` `set_caps
+()` and `get_caps ()` virtual functions for negotiation.
+
+### Writing a video sink
+
+Writing a videosink can be done using the `GstVideoSink` base-class,
+which derives from `GstBaseSink` internally. Currently, it does nothing
+yet but add another compile dependency, so derived classes will need to
+implement all base-sink virtual functions. When they do this correctly,
+this will have some positive effects on the end user experience with the
+videosink:
+
+ - Because of preroll (and the `preroll ()` virtual function), it is
+ possible to display a video frame already when going into the
+ `GST_STATE_PAUSED` state.
+
+ - By adding new features to `GstVideoSink`, it will be possible to add
+ extensions to videosinks that affect all of them, but only need to
+ be coded once, which is a huge maintenance benefit.
+
+## Writing a source
+
+In the previous part, particularly [Providing random
+access](pwg-scheduling.md#providing-random-access), we have learned
+that some types of elements can provide random access. This applies most
+definitely to source elements reading from a randomly seekable location,
+such as file sources. However, other source elements may be better
+described as a live source element, such as a camera source, an audio
+card source and such; those are not seekable and do not provide
+byte-exact access. For all such use cases, GStreamer provides two base
+classes: `GstBaseSrc` for the basic source functionality, and
+`GstPushSrc`, which is a non-byte exact source base-class. The
+pushsource base class itself derives from basesource as well, and thus
+all statements about the basesource apply to the pushsource, too.
+
+The basesrc class does several things automatically for derived classes,
+so they no longer have to worry about it:
+
+ - Fixes to `GstBaseSrc` apply to all derived classes automatically.
+
+ - Automatic pad activation handling, and task-wrapping in case we get
+ assigned to start a task ourselves.
+
+The `GstBaseSrc` may not be suitable for all cases, though; it has
+limitations:
+
+ - There is one and only one sourcepad. Source elements requiring
+ multiple sourcepads must implement a manager bin and use multiple
+ source elements internally or make a manager element that uses a
+ source element and a demuxer inside.
+
+It is possible to use special memory, such as X server memory pointers
+or `mmap ()`'ed memory areas, as data pointers in buffers returned from
+the `create()` virtual function.
+
+### Writing an audio source
+
+An audio source is nothing more but a special case of a pushsource.
+Audio sources would be anything that reads audio, such as a source
+reading from a soundserver, a kernel interface (such as ALSA) or a test
+sound / signal generator. GStreamer provides two base classes, similar
+to the two audiosinks described in [Writing an audio
+sink](#writing-an-audio-sink); one is ringbuffer-based, and requires the
+derived class to take care of its own scheduling, synchronization and
+such. The other is based on this `GstAudioBaseSrc` and is called
+`GstAudioSrc`, and provides a simple `open ()`, `close ()` and `read ()`
+interface, which is rather simple to implement and will suffice for most
+soundserver sources and audio interfaces (e.g. ALSA or OSS) out there.
+
+The `GstAudioSrc` base-class has several benefits for derived classes,
+on top of the benefits of the `GstPushSrc` base-class that it is based
+on:
+
+ - Does syncronization and provides a clock.
+
+ - New features can be added to it and will apply to all derived
+ classes automatically.
+
+## Writing a transformation element
+
+A third base-class that GStreamer provides is the `GstBaseTransform`.
+This is a base class for elements with one sourcepad and one sinkpad
+which act as a filter of some sort, such as volume changing, audio
+resampling, audio format conversion, and so on and so on. There is quite
+a lot of bookkeeping that such elements need to do in order for things
+such as buffer allocation forwarding, passthrough, in-place processing
+and such to all work correctly. This base class does all that for you,
+so that you just need to do the actual processing.
+
+Since the `GstBaseTransform` is based on the 1-to-1 model for filters,
+it may not apply well to elements such as decoders, which may have to
+parse properties from the stream. Also, it will not work for elements
+requiring more than one sourcepad or sinkpad.
+
--- /dev/null
+---
+title: Writing a Manager
+...
+
+# Writing a Manager
+
+Managers are elements that add a function or unify the function of
+another (series of) element(s). Managers are generally a `GstBin` with
+one or more ghostpads. Inside them is/are the actual element(s) that
+matters. There is several cases where this is useful. For example:
+
+ - To add support for private events with custom event handling to
+ another element.
+
+ - To add support for custom pad `_query ()` or `_convert ()` handling
+ to another element.
+
+ - To add custom data handling before or after another element's data
+ handler function (generally its `_chain ()` function).
+
+ - To embed an element, or a series of elements, into something that
+ looks and works like a simple element to the outside world. This is
+ particular handy for implementing sources and sink elements with
+ multiple pads.
+
+Making a manager is about as simple as it gets. You can derive from a
+`GstBin`, and in most cases, you can embed the required elements in the
+`_init ()` already, including setup of ghostpads. If you need any custom
+data handlers, you can connect signals or embed a second element which
+you control.
+
--- /dev/null
+---
+title: Writing a N-to-1 Element or Muxer
+...
+
+# Writing a N-to-1 Element or Muxer
+
+N-to-1 elements have been previously mentioned and discussed in both
+[Request and Sometimes pads](pwg-advanced-request.md) and in
+[Different scheduling modes](pwg-scheduling.md). The main noteworthy
+thing about N-to-1 elements is that each pad is push-based in its own
+thread, and the N-to-1 element synchronizes those streams by
+expected-timestamp-based logic. This means it lets all streams wait
+except for the one that provides the earliest next-expected timestamp.
+When that stream has passed one buffer, the next
+earliest-expected-timestamp is calculated, and we start back where we
+were, until all streams have reached EOS. There is a helper base class,
+called `GstCollectPads`, that will help you to do this.
+
+Note, however, that this helper class will only help you with grabbing a
+buffer from each input and giving you the one with earliest timestamp.
+If you need anything more difficult, such as "don't-grab-a-new-buffer
+until a given timestamp" or something like that, you'll need to do this
+yourself.
+
--- /dev/null
+---
+title: Writing a Demuxer or Parser
+...
+
+# Writing a Demuxer or Parser
+
+Demuxers are the 1-to-N elements that need very special care. They are
+responsible for timestamping raw, unparsed data into elementary video or
+audio streams, and there are many things that you can optimize or do
+wrong. Here, several culprits will be mentioned and common solutions
+will be offered. Parsers are demuxers with only one source pad. Also,
+they only cut the stream into buffers, they don't touch the data
+otherwise.
+
+As mentioned previously in [Caps negotiation](pwg-negotiation.md),
+demuxers should use fixed caps, since their data type will not change.
+
+As discussed in [Different scheduling modes](pwg-scheduling.md),
+demuxer elements can be written in multiple ways:
+
+ - They can be the driving force of the pipeline, by running their own
+ task. This works particularly well for elements that need random
+ access, for example an AVI demuxer.
+
+ - They can also run in push-based mode, which means that an upstream
+ element drives the pipeline. This works particularly well for
+ streams that may come from network, such as Ogg.
+
+In addition, audio parsers with one output can, in theory, also be
+written in random access mode. Although simple playback will mostly work
+if your element only accepts one mode, it may be required to implement
+multiple modes to work in combination with all sorts of applications,
+such as editing. Also, performance may become better if you implement
+multiple modes. See [Different scheduling modes](pwg-scheduling.md)
+to see how an element can accept multiple scheduling modes.
+
--- /dev/null
+---
+title: Creating special element types
+...
+
+# Creating special element types
+
+By now, we have looked at pretty much any feature that can be embedded
+into a GStreamer element. Most of this has been fairly low-level and
+given deep insights in how GStreamer works internally. Fortunately,
+GStreamer contains some easier-to-use interfaces to create such
+elements. In order to do that, we will look closer at the element types
+for which GStreamer provides base classes (sources, sinks and
+transformation elements). We will also look closer at some types of
+elements that require no specific coding such as scheduling-interaction
+or data passing, but rather require specific pipeline control (e.g.
+N-to-1 elements and managers).
+
--- /dev/null
+---
+title: Porting 0.10 plug-ins to 1.0
+...
+
+# Porting 0.10 plug-ins to 1.0
+
+You can find the list of changes in the [Porting
+to 1.0](http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/random/porting-to-1.0.txt)
+document.
+
--- /dev/null
+---
+title: Porting 0.8 plug-ins to 0.10
+...
+
+# Porting 0.8 plug-ins to 0.10
+
+This section of the appendix will discuss shortly what changes to
+plugins will be needed to quickly and conveniently port most
+applications from GStreamer-0.8 to GStreamer-0.10, with references to
+the relevant sections in this Plugin Writer's Guide where needed. With
+this list, it should be possible to port most plugins to GStreamer-0.10
+in less than a day. Exceptions are elements that will require a base
+class in 0.10 (sources, sinks), in which case it may take a lot longer,
+depending on the coder's skills (however, when using the `GstBaseSink`
+and `GstBaseSrc` base-classes, it shouldn't be all too bad), and
+elements requiring the deprecated bytestream interface, which should
+take 1-2 days with random access. The scheduling parts of muxers will
+also need a rewrite, which will take about the same amount of time.
+
+## List of changes
+
+ - Discont events have been replaced by newsegment events. In 0.10, it
+ is essential that you send a newsegment event downstream before you
+ send your first buffer (in 0.8 the scheduler would invent discont
+ events if you forgot them, in 0.10 this is no longer the case).
+
+ - In 0.10, buffers have caps attached to them. Elements should
+ allocate new buffers with `gst_pad_alloc_buffer ()`. See [Caps
+ negotiation](pwg-negotiation.md) for more details.
+
+ - Most functions returning an object or an object property have been
+ changed to return its own reference rather than a constant reference
+ of the one owned by the object itself. The reason for this change is
+ primarily thread-safety. This means effectively that return values
+ of functions such as `gst_element_get_pad ()`, `gst_pad_get_name
+ ()`, `gst_pad_get_parent ()`, `gst_object_get_parent ()`, and many
+ more like these have to be free'ed or unreferenced after use. Check
+ the API references of each function to know for sure whether return
+ values should be free'ed or not.
+
+ - In 0.8, scheduling could happen in any way. Source elements could be
+ `_get ()`-based or `_loop
+ ()`-based, and any other element could be `_chain
+ ()`-based or `_loop ()`-based, with no limitations. Scheduling in
+ 0.10 is simpler for the scheduler, and the element is expected to do
+ some more work. Pads get assigned a scheduling mode, based on which
+ they can either operate in random access-mode, in pipeline driving
+ mode or in push-mode. all this is documented in detail in [Different
+ scheduling modes](pwg-scheduling.md). As a result of this, the
+ bytestream object no longer exists. Elements requiring byte-level
+ access should now use random access on their sinkpads.
+
+ - Negotiation is asynchronous. This means that downstream negotiation
+ is done as data comes in and upstream negotiation is done whenever
+ renegotiation is required. All details are described in [Caps
+ negotiation](pwg-negotiation.md).
+
+ - For as far as possible, elements should try to use existing base
+ classes in 0.10. Sink and source elements, for example, could derive
+ from `GstBaseSrc` and `GstBaseSink`. Audio sinks or sources could
+ even derive from audio-specific base classes. All existing base
+ classes have been discussed in [Pre-made base
+ classes](pwg-other-base.md) and the next few chapters.
+
+ - In 0.10, event handling and buffers are separated once again. This
+ means that in order to receive events, one no longer has to set the
+ `GST_FLAG_EVENT_AWARE` flag, but can simply set an event handling
+ function on the element's sinkpad(s), using the function
+ `gst_pad_set_event_function ()`. The `_chain ()`-function will only
+ receive buffers.
+
+ - Although core will wrap most threading-related locking for you (e.g.
+ it takes the stream lock before calling your data handling
+ functions), you are still responsible for locking around certain
+ functions, e.g. object properties. Be sure to lock properly here,
+ since applications will change those properties in a different
+ thread than the thread which does the actual data passing\! You can
+ use the `GST_OBJECT_LOCK ()` and `GST_OBJECT_UNLOCK
+ ()` helpers in most cases, fortunately, which grabs the default
+ property lock of the element.
+
+ - `GstValueFixedList` and all `*_fixed_list_* ()` functions were
+ renamed to `GstValueArray` and `*_array_*
+ ()`.
+
+ - The semantics of `GST_STATE_PAUSED` and `GST_STATE_PLAYING` have
+ changed for elements that are not sink elements. Non-sink elements
+ need to be able to accept and process data already in the
+ `GST_STATE_PAUSED` state now (i.e. when prerolling the pipeline).
+ More details can be found in [What are
+ states?](pwg-statemanage-states.md).
+
+ - If your plugin's state change function hasn't been superseded by
+ virtual start() and stop() methods of one of the new base classes,
+ then your plugin's state change functions may need to be changed in
+ order to safely handle concurrent access by multiple threads. Your
+ typical state change function will now first handle upwards state
+ changes, then chain up to the state change function of the parent
+ class (usually GstElementClass in these cases), and only then handle
+ downwards state changes. See the vorbis decoder plugin in
+ gst-plugins-base for an example.
+
+ The reason for this is that in the case of downwards state changes
+ you don't want to destroy allocated resources while your plugin's
+ chain function (for example) is still accessing those resources in
+ another thread. Whether your chain function might be running or not
+ depends on the state of your plugin's pads, and the state of those
+ pads is closely linked to the state of the element. Pad states are
+ handled in the GstElement class's state change function, including
+ proper locking, that's why it is essential to chain up before
+ destroying allocated resources.
+
+ As already mentioned above, you should really rewrite your plugin to
+ derive from one of the new base classes though, so you don't have to
+ worry about these things, as the base class will handle it for you.
+ There are no base classes for decoders and encoders yet, so the
+ above paragraphs about state changes definitively apply if your
+ plugin is a decoder or an encoder.
+
+ - `gst_pad_set_link_function ()`, which used to set a function that
+ would be called when a format was negotiated between two `GstPad`s,
+ now sets a function that is called when two elements are linked
+ together in an application. For all practical purposes, you most
+ likely want to use the function `gst_pad_set_setcaps_function ()`,
+ nowadays, which sets a function that is called when the format
+ streaming over a pad changes (so similar to `_set_link_function ()`
+ in GStreamer-0.8).
+
+ If the element is derived from a `GstBase` class, then override the
+ `set_caps ()`.
+
+ - `gst_pad_use_explicit_caps ()` has been replaced by
+ `gst_pad_use_fixed_caps ()`. You can then set the fixed caps to use
+ on a pad with `gst_pad_set_caps ()`.
+
--- /dev/null
+---
+title: Different scheduling modes
+...
+
+# Different scheduling modes
+
+The scheduling mode of a pad defines how data is retrieved from (source)
+or given to (sink) pads. GStreamer can operate in two scheduling mode,
+called push- and pull-mode. GStreamer supports elements with pads in any
+of the scheduling modes where not all pads need to be operating in the
+same mode.
+
+So far, we have only discussed `_chain ()`-operating elements, i.e.
+elements that have a chain-function set on their sink pad and push
+buffers on their source pad(s). We call this the push-mode because a
+peer element will use `gst_pad_push ()` on a srcpad, which will cause
+our `_chain ()`-function to be called, which in turn causes our element
+to push out a buffer on the source pad. The initiative to start the
+dataflow happens somewhere upstream when it pushes out a buffer and all
+downstream elements get scheduled when their `_chain ()`-functions are
+called in turn.
+
+Before we explain pull-mode scheduling, let's first understand how the
+different scheduling modes are selected and activated on a pad.
+
+## The pad activation stage
+
+During the element state change of READY-\>PAUSED, the pads of an
+element will be activated. This happens first on the source pads and
+then on the sink pads of the element. GStreamer calls the `_activate ()`
+of a pad. By default this function will activate the pad in push-mode by
+calling `gst_pad_activate_mode ()` with the GST\_PAD\_MODE\_PUSH
+scheduling mode. It is possible to override the `_activate ()` of a pad
+and decide on a different scheduling mode. You can know in what
+scheduling mode a pad is activated by overriding the `_activate_mode
+()`-function.
+
+GStreamer allows the different pads of an element to operate in
+different scheduling modes. This allows for many different possible
+use-cases. What follows is an overview of some typical use-cases.
+
+ - If all pads of an element are activated in push-mode scheduling, the
+ element as a whole is operating in push-mode. For source elements
+ this means that they will have to start a task that pushes out
+ buffers on the source pad to the downstream elements. Downstream
+ elements will have data pushed to them by upstream elements using
+ the sinkpads `_chain ()`-function which will push out buffers on the
+ source pads. Prerequisites for this scheduling mode are that a
+ chain-function was set for each sinkpad using
+ `gst_pad_set_chain_function ()` and that all downstream elements
+ operate in the same mode.
+
+ - Alternatively, sinkpads can be the driving force behind a pipeline
+ by operating in pull-mode, while the sourcepads of the element still
+ operate in push-mode. In order to be the driving force, those pads
+ start a `GstTask` when they are activated. This task is a thread,
+ which will call a function specified by the element. When called,
+ this function will have random data access (through
+ `gst_pad_pull_range ()`) over all sinkpads, and can push data over
+ the sourcepads, which effectively means that this element controls
+ data flow in the pipeline. Prerequisites for this mode are that all
+ downstream elements can act in push mode, and that all upstream
+ elements operate in pull-mode (see below).
+
+ Source pads can be activated in PULL mode by a downstream element
+ when they return GST\_PAD\_MODE\_PULL from the
+ GST\_QUERY\_SCHEDULING query. Prerequisites for this scheduling mode
+ are that a getrange-function was set for the source pad using
+ `gst_pad_set_getrange_function ()`.
+
+ - Lastly, all pads in an element can be activated in PULL-mode.
+ However, contrary to the above, this does not mean that they start a
+ task on their own. Rather, it means that they are pull slave for the
+ downstream element, and have to provide random data access to it
+ from their `_get_range ()`-function. Requirements are that the a
+ `_get_range
+ ()`-function was set on this pad using the function
+ `gst_pad_set_getrange_function ()`. Also, if the element has any
+ sinkpads, all those pads (and thereby their peers) need to operate
+ in PULL access mode, too.
+
+ When a sink element is activated in PULL mode, it should start a
+ task that calls `gst_pad_pull_range ()` on its sinkpad. It can only
+ do this when the upstream SCHEDULING query returns support for the
+ GST\_PAD\_MODE\_PULL scheduling mode.
+
+In the next two sections, we will go closer into pull-mode scheduling
+(elements/pads driving the pipeline, and elements/pads providing random
+access), and some specific use cases will be given.
+
+## Pads driving the pipeline
+
+Sinkpads operating in pull-mode, with the sourcepads operating in
+push-mode (or it has no sourcepads when it is a sink), can start a task
+that will drive the pipeline data flow. Within this task function, you
+have random access over all of the sinkpads, and push data over the
+sourcepads. This can come in useful for several different kinds of
+elements:
+
+ - Demuxers, parsers and certain kinds of decoders where data comes in
+ unparsed (such as MPEG-audio or video streams), since those will
+ prefer byte-exact (random) access from their input. If possible,
+ however, such elements should be prepared to operate in push-mode
+ mode, too.
+
+ - Certain kind of audio outputs, which require control over their
+ input data flow, such as the Jack sound server.
+
+First you need to perform a SCHEDULING query to check if the upstream
+element(s) support pull-mode scheduling. If that is possible, you can
+activate the sinkpad in pull-mode. Inside the activate\_mode function
+you can then start the task.
+
+``` c
+#include "filter.h"
+#include <string.h>
+
+static gboolean gst_my_filter_activate (GstPad * pad,
+ GstObject * parent);
+static gboolean gst_my_filter_activate_mode (GstPad * pad,
+ GstObject * parent,
+ GstPadMode mode,
+ gboolean active);
+static void gst_my_filter_loop (GstMyFilter * filter);
+
+G_DEFINE_TYPE (GstMyFilter, gst_my_filter, GST_TYPE_ELEMENT);
+
+
+static void
+gst_my_filter_init (GstMyFilter * filter)
+{
+
+[..]
+
+ gst_pad_set_activate_function (filter->sinkpad, gst_my_filter_activate);
+ gst_pad_set_activatemode_function (filter->sinkpad,
+ gst_my_filter_activate_mode);
+
+
+[..]
+}
+
+[..]
+
+static gboolean
+gst_my_filter_activate (GstPad * pad, GstObject * parent)
+{
+ GstQuery *query;
+ gboolean pull_mode;
+
+ /* first check what upstream scheduling is supported */
+ query = gst_query_new_scheduling ();
+
+ if (!gst_pad_peer_query (pad, query)) {
+ gst_query_unref (query);
+ goto activate_push;
+ }
+
+ /* see if pull-mode is supported */
+ pull_mode = gst_query_has_scheduling_mode_with_flags (query,
+ GST_PAD_MODE_PULL, GST_SCHEDULING_FLAG_SEEKABLE);
+ gst_query_unref (query);
+
+ if (!pull_mode)
+ goto activate_push;
+
+ /* now we can activate in pull-mode. GStreamer will also
+ * activate the upstream peer in pull-mode */
+ return gst_pad_activate_mode (pad, GST_PAD_MODE_PULL, TRUE);
+
+activate_push:
+ {
+ /* something not right, we fallback to push-mode */
+ return gst_pad_activate_mode (pad, GST_PAD_MODE_PUSH, TRUE);
+ }
+}
+
+static gboolean
+gst_my_filter_activate_pull (GstPad * pad,
+ GstObject * parent,
+ GstPadMode mode,
+ gboolean active)
+{
+ gboolean res;
+ GstMyFilter *filter = GST_MY_FILTER (parent);
+
+ switch (mode) {
+ case GST_PAD_MODE_PUSH:
+ res = TRUE;
+ break;
+ case GST_PAD_MODE_PULL:
+ if (active) {
+ filter->offset = 0;
+ res = gst_pad_start_task (pad,
+ (GstTaskFunction) gst_my_filter_loop, filter, NULL);
+ } else {
+ res = gst_pad_stop_task (pad);
+ }
+ break;
+ default:
+ /* unknown scheduling mode */
+ res = FALSE;
+ break;
+ }
+ return res;
+}
+
+```
+
+Once started, your task has full control over input and output. The most
+simple case of a task function is one that reads input and pushes that
+over its source pad. It's not all that useful, but provides some more
+flexibility than the old push-mode case that we've been looking at so
+far.
+
+``` c
+ #define BLOCKSIZE 2048
+
+ static void
+ gst_my_filter_loop (GstMyFilter * filter)
+ {
+ GstFlowReturn ret;
+ guint64 len;
+ GstFormat fmt = GST_FORMAT_BYTES;
+ GstBuffer *buf = NULL;
+
+ if (!gst_pad_query_duration (filter->sinkpad, fmt, &len)) {
+ GST_DEBUG_OBJECT (filter, "failed to query duration, pausing");
+ goto stop;
+ }
+
+ if (filter->offset >= len) {
+ GST_DEBUG_OBJECT (filter, "at end of input, sending EOS, pausing");
+ gst_pad_push_event (filter->srcpad, gst_event_new_eos ());
+ goto stop;
+ }
+
+ /* now, read BLOCKSIZE bytes from byte offset filter->offset */
+ ret = gst_pad_pull_range (filter->sinkpad, filter->offset,
+ BLOCKSIZE, &buf);
+
+ if (ret != GST_FLOW_OK) {
+ GST_DEBUG_OBJECT (filter, "pull_range failed: %s", gst_flow_get_name (ret));
+ goto stop;
+ }
+
+ /* now push buffer downstream */
+ ret = gst_pad_push (filter->srcpad, buf);
+
+ buf = NULL; /* gst_pad_push() took ownership of buffer */
+
+ if (ret != GST_FLOW_OK) {
+ GST_DEBUG_OBJECT (filter, "pad_push failed: %s", gst_flow_get_name (ret));
+ goto stop;
+ }
+
+ /* everything is fine, increase offset and wait for us to be called again */
+ filter->offset += BLOCKSIZE;
+ return;
+
+ stop:
+ GST_DEBUG_OBJECT (filter, "pausing task");
+ gst_pad_pause_task (filter->sinkpad);
+ }
+```
+
+## Providing random access
+
+In the previous section, we have talked about how elements (or pads)
+that are activated to drive the pipeline using their own task, must use
+pull-mode scheduling on their sinkpads. This means that all pads linked
+to those pads need to be activated in pull-mode. Source pads activated
+in pull-mode must implement a `_get_range ()`-function set using
+`gst_pad_set_getrange_function ()`, and that function will be called
+when the peer pad requests some data with `gst_pad_pull_range ()`. The
+element is then responsible for seeking to the right offset and
+providing the requested data. Several elements can implement random
+access:
+
+ - Data sources, such as a file source, that can provide data from any
+ offset with reasonable low latency.
+
+ - Filters that would like to provide a pull-mode scheduling over the
+ whole pipeline.
+
+ - Parsers who can easily provide this by skipping a small part of
+ their input and are thus essentially "forwarding" getrange requests
+ literally without any own processing involved. Examples include tag
+ readers (e.g. ID3) or single output parsers, such as a WAVE parser.
+
+The following example will show how a `_get_range
+()`-function can be implemented in a source element:
+
+ #include "filter.h"
+ static GstFlowReturn
+ gst_my_filter_get_range (GstPad * pad,
+ GstObject * parent,
+ guint64 offset,
+ guint length,
+ GstBuffer ** buf);
+
+ G_DEFINE_TYPE (GstMyFilter, gst_my_filter, GST_TYPE_ELEMENT);
+
+
+
+ static void
+ gst_my_filter_init (GstMyFilter * filter)
+ {
+
+ [..]
+
+ gst_pad_set_getrange_function (filter->srcpad,
+ gst_my_filter_get_range);
+
+ [..]
+ }
+
+ static GstFlowReturn
+ gst_my_filter_get_range (GstPad * pad,
+ GstObject * parent,
+ guint64 offset,
+ guint length,
+ GstBuffer ** buf)
+ {
+
+ GstMyFilter *filter = GST_MY_FILTER (parent);
+
+ [.. here, you would fill *buf ..]
+
+ return GST_FLOW_OK;
+ }
+
+In practice, many elements that could theoretically do random access,
+may in practice often be activated in push-mode scheduling anyway, since
+there is no downstream element able to start its own task. Therefore, in
+practice, those elements should implement both a `_get_range
+()`-function and a `_chain
+()`-function (for filters and parsers) or a `_get_range
+()`-function and be prepared to start their own task by providing
+`_activate_* ()`-functions (for source elements).
+
--- /dev/null
+---
+title: What are states?
+...
+
+# What are states?
+
+A state describes whether the element instance is initialized, whether
+it is ready to transfer data and whether it is currently handling data.
+There are four states defined in GStreamer:
+
+ - `GST_STATE_NULL`
+
+ - `GST_STATE_READY`
+
+ - `GST_STATE_PAUSED`
+
+ - `GST_STATE_PLAYING`
+
+which will from now on be referred to simply as “NULL”, “READY”,
+“PAUSED” and “PLAYING”.
+
+`GST_STATE_NULL` is the default state of an element. In this state, it
+has not allocated any runtime resources, it has not loaded any runtime
+libraries and it can obviously not handle data.
+
+`GST_STATE_READY` is the next state that an element can be in. In the
+READY state, an element has all default resources (runtime-libraries,
+runtime-memory) allocated. However, it has not yet allocated or defined
+anything that is stream-specific. When going from NULL to READY state
+(`GST_STATE_CHANGE_NULL_TO_READY`), an element should allocate any
+non-stream-specific resources and should load runtime-loadable libraries
+(if any). When going the other way around (from READY to NULL,
+`GST_STATE_CHANGE_READY_TO_NULL`), an element should unload these
+libraries and free all allocated resources. Examples of such resources
+are hardware devices. Note that files are generally streams, and these
+should thus be considered as stream-specific resources; therefore, they
+should *not* be allocated in this state.
+
+`GST_STATE_PAUSED` is the state in which an element is ready to accept
+and handle data. For most elements this state is the same as PLAYING.
+The only exception to this rule are sink elements. Sink elements only
+accept one single buffer of data and then block. At this point the
+pipeline is 'prerolled' and ready to render data immediately.
+
+`GST_STATE_PLAYING` is the highest state that an element can be in. For
+most elements this state is exactly the same as PAUSED, they accept and
+process events and buffers with data. Only sink elements need to
+differentiate between PAUSED and PLAYING state. In PLAYING state, sink
+elements actually render incoming data, e.g. output audio to a sound
+card or render video pictures to an image sink.
+
+## Managing filter state
+
+If at all possible, your element should derive from one of the new base
+classes ([Pre-made base classes](pwg-other-base.md)). There are
+ready-made general purpose base classes for different types of sources,
+sinks and filter/transformation elements. In addition to those,
+specialised base classes exist for audio and video elements and others.
+
+If you use a base class, you will rarely have to handle state changes
+yourself. All you have to do is override the base class's start() and
+stop() virtual functions (might be called differently depending on the
+base class) and the base class will take care of everything for you.
+
+If, however, you do not derive from a ready-made base class, but from
+GstElement or some other class not built on top of a base class, you
+will most likely have to implement your own state change function to be
+notified of state changes. This is definitively necessary if your plugin
+is a demuxer or a muxer, as there are no base classes for muxers or
+demuxers yet.
+
+An element can be notified of state changes through a virtual function
+pointer. Inside this function, the element can initialize any sort of
+specific data needed by the element, and it can optionally fail to go
+from one state to another.
+
+Do not g\_assert for unhandled state changes; this is taken care of by
+the GstElement base class.
+
+ static GstStateChangeReturn
+ gst_my_filter_change_state (GstElement *element, GstStateChange transition);
+
+ static void
+ gst_my_filter_class_init (GstMyFilterClass *klass)
+ {
+ GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
+
+ element_class->change_state = gst_my_filter_change_state;
+ }
+
+
+
+ static GstStateChangeReturn
+ gst_my_filter_change_state (GstElement *element, GstStateChange transition)
+ {
+ GstStateChangeReturn ret = GST_STATE_CHANGE_SUCCESS;
+ GstMyFilter *filter = GST_MY_FILTER (element);
+
+ switch (transition) {
+ case GST_STATE_CHANGE_NULL_TO_READY:
+ if (!gst_my_filter_allocate_memory (filter))
+ return GST_STATE_CHANGE_FAILURE;
+ break;
+ default:
+ break;
+ }
+
+ ret = GST_ELEMENT_CLASS (parent_class)->change_state (element, transition);
+ if (ret == GST_STATE_CHANGE_FAILURE)
+ return ret;
+
+ switch (transition) {
+ case GST_STATE_CHANGE_READY_TO_NULL:
+ gst_my_filter_free_memory (filter);
+ break;
+ default:
+ break;
+ }
+
+ return ret;
+ }
+
+Note that upwards (NULL=\>READY, READY=\>PAUSED, PAUSED=\>PLAYING) and
+downwards (PLAYING=\>PAUSED, PAUSED=\>READY, READY=\>NULL) state changes
+are handled in two separate blocks with the downwards state change
+handled only after we have chained up to the parent class's state change
+function. This is necessary in order to safely handle concurrent access
+by multiple threads.
+
+The reason for this is that in the case of downwards state changes you
+don't want to destroy allocated resources while your plugin's chain
+function (for example) is still accessing those resources in another
+thread. Whether your chain function might be running or not depends on
+the state of your plugin's pads, and the state of those pads is closely
+linked to the state of the element. Pad states are handled in the
+GstElement class's state change function, including proper locking,
+that's why it is essential to chain up before destroying allocated
+resources.
+
--- /dev/null
+# Android tutorial 5: A Complete media player
+
+## Goal!
+
+![screenshot]
+
+This tutorial wants to be the “demo application” that showcases what can
+be done with GStreamer in the Android platform.
+
+It is intended to be downloaded in final, compiled, form rather than
+analyzed for its pedagogical value, since it adds very little GStreamer
+knowledge over what has already been shown in [](sdk-android-tutorial-media-player.md).
+
+
+**FIXME: Do we want to provide a binary of the app?**
+
+## Introduction
+
+The previous tutorial already implemented a basic media player. This one
+simply adds a few finishing touches. In particular, it adds the
+capability to choose the media to play, and disables the screensaver
+during media playback.
+
+These are not features directly related to GStreamer, and are therefore
+outside the scope of these tutorials. Only a few implementation pointers
+are given here.
+
+## Registering as a media player
+
+The `AndroidManifest.xml` tells the Android system the capabilities of
+the application. By specifying in the `intent-filter` of the activity
+that it understands the `audio/*`, `video/*` and `image/*` MIME types,
+the tutorial will be offered as an option whenever an application
+requires such medias to be viewed.
+
+“Unfortunately”, GStreamer knows more file formats than Android does,
+so, for some files, Android will not provide a MIME type. For these
+cases, a new `intent-filter` has to be provided which ignores MIME types
+and focuses only in the filename extension. This is inconvenient because
+the list of extensions can be large, but there does not seem to be
+another option. In this tutorial, only a very short list of extensions
+is provided, for simplicity.
+
+Finally, GStreamer can also playback remote files, so URI schemes like
+`http` are supported in another `intent-filter`. Android does not
+provide MIME types for remote files, so the filename extension list has
+to be provided again.
+
+Once we have informed the system of our capabilities, it will start
+sending
+[Intents](http://developer.android.com/reference/android/content/Intent.html)
+to invoke our activity, which will contain the desired URI to play. In
+the `onCreate()` method the intent that invoked the activity is
+retrieved and checked for such URI.
+
+## Implementing a file chooser dialog
+
+The UI includes a new button . It
+invokes a file chooser dialog (based on the [Android File
+Dialog](http://code.google.com/p/android-file-dialog/) project) that
+allows you to choose a local media file, no matter what extension or
+MIME type it has.
+
+If a new media is selected, it is passed onto the native code (which
+will set the pipeline to READY, pass the URI onto `playbin`, and bring
+the pipeline back to the previous state). The current position is also
+reset, so the new clip does not start in the previous position.
+
+## Preventing the screen from turning off
+
+While watching a movie, there is typically no user activity. After a
+short period of such inactivity, Android will dim the screen, and then
+turn it off completely. To prevent this, a [Wake
+Lock](http://developer.android.com/reference/android/os/PowerManager.WakeLock.html)
+is used. The application acquires the lock when the Play button is
+pressed, so the screen is never turned off, and releases it when the
+Pause button is pressed.
+
+## Conclusion
+
+This finishes the series of Android tutorials. Each one of the
+preceding tutorials has evolved on top of the previous one, showing
+how to implement a particular set of features, and concluding in this
+tutorial 5. The goal of tutorial 5 is to build a complete media player
+which can already be used to showcase the integration of GStreamer and
+Android.
+
+It has been a pleasure having you here, and see you soon!
+
+ [screenshot]: images/sdk-android-tutorial-a-complete-media-player-screenshot.png
+ [media-next]: images/media-next.png
--- /dev/null
+# Android tutorial 2: A running pipeline
+
+### Goal
+
+![screenshot]
+
+The tutorials seen in the [Basic](sdk-basic-tutorials.md) and
+[Playback](sdk-playback-tutorials.md) sections are intended for Desktop
+platforms and, therefore, their main thread is allowed to block (using
+`gst_bus_pop_filtered()`) or relinquish control to a GLib main loop. On
+Android this would lead to the application being tagged as
+non-responsive and probably closed.
+
+This tutorial shows how to overcome this problem. In particular, we will
+learn:
+
+ - How to move the native code to its own thread
+ - How to allow threads created from C code to communicate with Java
+ - How to access Java code from C
+ - How to allocate a `CustomData` structure from C and have Java host
+ it
+
+### Introduction
+
+When using a Graphical User Interface (UI), if the application waits for
+GStreamer calls to complete the user experience will suffer. The usual
+approach, with the [GTK+ toolkit](http://www.gtk.org) for example, is to
+relinquish control to a GLib `GMainLoop` and let it control the events
+coming from the UI or GStreamer.
+
+This approach can be very cumbersome when GStreamer and the Android UI
+communicate through the JNI interface, so we take a cleaner route: We
+use a GLib main loop, and move it to its own thread, so it does not
+block the application. This simplifies the GStreamer-Android
+integration, and we only need to worry about a few inter-process
+synchronization bits, which are detailed in this tutorial.
+
+Additionally, this tutorial shows how to obtain, from any thread, the
+[JNI Environment
+pointer](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/design.html#wp16696)
+required to make JNI calls. This is necessary, for example, to call Java
+code from callbacks in threads spawned deep within GStreamer, which
+never received this pointer directly.
+
+Finally, this tutorial explains how to call Java methods from native C
+code, which involves locating the desired method’s ID in the class.
+These IDs never change, so they are cached as global variables in the C
+code and obtained in the static initializer of the class.
+
+The code below builds a pipeline with an `audiotestsrc` and an
+`autoaudiosink` (it plays an audible tone). Two buttons in the UI allow
+setting the pipeline to PLAYING or PAUSED. A TextView in the UI shows
+messages sent from the C code (for errors and state changes).
+
+### A pipeline on Android \[Java code\]
+
+**src/org/freedesktop/gstreamer/tutorials/tutorial\_2/Tutorial2.java**
+
+``` java
+package org.freedesktop.gstreamer.tutorials.tutorial_2;
+
+import android.app.Activity;
+import android.os.Bundle;
+import android.util.Log;
+import android.view.View;
+import android.view.View.OnClickListener;
+import android.widget.ImageButton;
+import android.widget.TextView;
+import android.widget.Toast;
+
+import org.freedesktop.gstreamer.GStreamer;
+
+public class Tutorial2 extends Activity {
+ private native void nativeInit(); // Initialize native code, build pipeline, etc
+ private native void nativeFinalize(); // Destroy pipeline and shutdown native code
+ private native void nativePlay(); // Set pipeline to PLAYING
+ private native void nativePause(); // Set pipeline to PAUSED
+ private static native boolean nativeClassInit(); // Initialize native class: cache Method IDs for callbacks
+ private long native_custom_data; // Native code will use this to keep private data
+
+ private boolean is_playing_desired; // Whether the user asked to go to PLAYING
+
+ // Called when the activity is first created.
+ @Override
+ public void onCreate(Bundle savedInstanceState)
+ {
+ super.onCreate(savedInstanceState);
+
+ // Initialize GStreamer and warn if it fails
+ try {
+ GStreamer.init(this);
+ } catch (Exception e) {
+ Toast.makeText(this, e.getMessage(), Toast.LENGTH_LONG).show();
+ finish();
+ return;
+ }
+
+ setContentView(R.layout.main);
+
+ ImageButton play = (ImageButton) this.findViewById(R.id.button_play);
+ play.setOnClickListener(new OnClickListener() {
+ public void onClick(View v) {
+ is_playing_desired = true;
+ nativePlay();
+ }
+ });
+
+ ImageButton pause = (ImageButton) this.findViewById(R.id.button_stop);
+ pause.setOnClickListener(new OnClickListener() {
+ public void onClick(View v) {
+ is_playing_desired = false;
+ nativePause();
+ }
+ });
+
+ if (savedInstanceState != null) {
+ is_playing_desired = savedInstanceState.getBoolean("playing");
+ Log.i ("GStreamer", "Activity created. Saved state is playing:" + is_playing_desired);
+ } else {
+ is_playing_desired = false;
+ Log.i ("GStreamer", "Activity created. There is no saved state, playing: false");
+ }
+
+ // Start with disabled buttons, until native code is initialized
+ this.findViewById(R.id.button_play).setEnabled(false);
+ this.findViewById(R.id.button_stop).setEnabled(false);
+
+ nativeInit();
+ }
+
+ protected void onSaveInstanceState (Bundle outState) {
+ Log.d ("GStreamer", "Saving state, playing:" + is_playing_desired);
+ outState.putBoolean("playing", is_playing_desired);
+ }
+
+ protected void onDestroy() {
+ nativeFinalize();
+ super.onDestroy();
+ }
+
+ // Called from native code. This sets the content of the TextView from the UI thread.
+ private void setMessage(final String message) {
+ final TextView tv = (TextView) this.findViewById(R.id.textview_message);
+ runOnUiThread (new Runnable() {
+ public void run() {
+ tv.setText(message);
+ }
+ });
+ }
+
+ // Called from native code. Native code calls this once it has created its pipeline and
+ // the main loop is running, so it is ready to accept commands.
+ private void onGStreamerInitialized () {
+ Log.i ("GStreamer", "Gst initialized. Restoring state, playing:" + is_playing_desired);
+ // Restore previous playing state
+ if (is_playing_desired) {
+ nativePlay();
+ } else {
+ nativePause();
+ }
+
+ // Re-enable buttons, now that GStreamer is initialized
+ final Activity activity = this;
+ runOnUiThread(new Runnable() {
+ public void run() {
+ activity.findViewById(R.id.button_play).setEnabled(true);
+ activity.findViewById(R.id.button_stop).setEnabled(true);
+ }
+ });
+ }
+
+ static {
+ System.loadLibrary("gstreamer_android");
+ System.loadLibrary("tutorial-2");
+ nativeClassInit();
+ }
+
+}
+```
+
+As usual, the first bit that gets executed is the static initializer of
+the class:
+
+``` java
+static {
+ System.loadLibrary("gstreamer_android");
+ System.loadLibrary("tutorial-2");
+ nativeClassInit();
+}
+```
+
+As explained in the previous tutorial, the two native libraries are
+loaded and their `JNI_OnLoad()` methods are executed. Here, we also call
+the native method `nativeClassInit()`, previously declared with the
+`native` keyword in line 19. We will later see what its purpose is
+
+In the `onCreate()` method GStreamer is initialized as in the previous
+tutorial with `GStreamer.init(this)`, and then the layout is inflated
+and listeners are setup for the two UI buttons:
+
+``` java
+ImageButton play = (ImageButton) this.findViewById(R.id.button_play);
+play.setOnClickListener(new OnClickListener() {
+ public void onClick(View v) {
+ is_playing_desired = true;
+ nativePlay();
+ }
+});
+ImageButton pause = (ImageButton) this.findViewById(R.id.button_stop);
+pause.setOnClickListener(new OnClickListener() {
+ public void onClick(View v) {
+ is_playing_desired = false;
+ nativePause();
+ }
+});
+```
+
+Each button instructs the native code to set the pipeline to the desired
+state, and also remembers this state in the
+`is_playing_desired` variable. This is required so, when the
+application is restarted (for example, due to an orientation change), it
+can set the pipeline again to the desired state. This approach is easier
+and safer than tracking the actual pipeline state, because orientation
+changes can happen before the pipeline has moved to the desired state,
+for example.
+
+``` java
+if (savedInstanceState != null) {
+ is_playing_desired = savedInstanceState.getBoolean("playing");
+ Log.i ("GStreamer", "Activity created. Saved state is playing:" + is_playing_desired);
+} else {
+ is_playing_desired = false;
+ Log.i ("GStreamer", "Activity created. There is no saved state, playing: false");
+}
+```
+
+Restore the previous playing state (if any) from `savedInstanceState`.
+We will first build the GStreamer pipeline (below) and only when the
+native code reports itself as initialized we will use
+`is_playing_desired`.
+
+``` java
+nativeInit();
+```
+
+As will be shown in the C code, `nativeInit()` creates a dedicated
+thread, a GStreamer pipeline, a GLib main loop, and, right before
+calling `g_main_loop_run()` and going to sleep, it warns the Java code
+that the native code is initialized and ready to accept commands.
+
+This finishes the `onCreate()` method and the Java initialization. The
+UI buttons are disabled, so nothing will happen until native code is
+ready and `onGStreamerInitialized()` is called:
+
+``` java
+private void onGStreamerInitialized () {
+ Log.i ("GStreamer", "Gst initialized. Restoring state, playing:" + is_playing_desired);
+```
+
+This is called by the native code when its main loop is finally running.
+We first retrieve the desired playing state from `is_playing_desired`,
+and then set that state:
+
+``` java
+// Restore previous playing state
+if (is_playing_desired) {
+ nativePlay();
+} else {
+ nativePause();
+}
+```
+
+Here comes the first caveat, when re-enabling the UI buttons:
+
+``` java
+// Re-enable buttons, now that GStreamer is initialized
+final Activity activity = this;
+runOnUiThread(new Runnable() {
+ public void run() {
+ activity.findViewById(R.id.button_play).setEnabled(true);
+ activity.findViewById(R.id.button_stop).setEnabled(true);
+ }
+});
+```
+
+This method is being called from the thread that the native code created
+to run its main loop, and is not allowed to issue UI-altering commands:
+Only the UI thread can do that. The solution is easy though: Android
+Activities have a handy
+[runOnUiThread()](http://developer.android.com/reference/android/app/Activity.html#runOnUiThread\(java.lang.Runnable\))
+method which lets bits of code to be executed from the correct thread. A
+[Runnable](http://developer.android.com/reference/java/lang/Runnable.html)
+instance has to be constructed and any parameter can be passed either by
+sub-classing
+[Runnable](http://developer.android.com/reference/java/lang/Runnable.html)
+and adding a dedicated constructor, or by using the `final` modifier, as
+shown in the above snippet.
+
+The same problem exists when the native code wants to output a string in
+our TextView using the `setMessage()` method: it has to be done from the
+UI thread. The solution is the same:
+
+``` java
+private void setMessage(final String message) {
+ final TextView tv = (TextView) this.findViewById(R.id.textview_message);
+ runOnUiThread (new Runnable() {
+ public void run() {
+ tv.setText(message);
+ }
+ });
+}
+```
+
+Finally, a few remaining bits:
+
+``` java
+protected void onSaveInstanceState (Bundle outState) {
+ Log.d ("GStreamer", "Saving state, playing:" + is_playing_desired);
+ outState.putBoolean("playing", is_playing_desired);
+}
+```
+
+This method stores the currently desired playing state when Android is
+about to shut us down, so next time it restarts (after an orientation
+change, for example), it can restore the same state.
+
+``` java
+protected void onDestroy() {
+ nativeFinalize();
+ super.onDestroy();
+}
+```
+
+And this is called before Android destroys our application. We call the
+`nativeFinalize()`method to exit the main loop, destroy its thread and
+all allocated resources.
+
+This concludes the UI part of the tutorial.
+
+### A pipeline on Android \[C code\]
+
+**jni/tutorial-2.c**
+
+``` c
+#include <string.h>
+#include <jni.h>
+#include <android/log.h>
+#include <gst/gst.h>
+#include <pthread.h>
+
+GST_DEBUG_CATEGORY_STATIC (debug_category);
+#define GST_CAT_DEFAULT debug_category
+
+/*
+ * These macros provide a way to store the native pointer to CustomData, which might be 32 or 64 bits, into
+ * a jlong, which is always 64 bits, without warnings.
+ */
+#if GLIB_SIZEOF_VOID_P == 8
+## define GET_CUSTOM_DATA(env, thiz, fieldID) (CustomData *)(*env)->GetLongField (env, thiz, fieldID)
+## define SET_CUSTOM_DATA(env, thiz, fieldID, data) (*env)->SetLongField (env, thiz, fieldID, (jlong)data)
+#else
+## define GET_CUSTOM_DATA(env, thiz, fieldID) (CustomData *)(jint)(*env)->GetLongField (env, thiz, fieldID)
+## define SET_CUSTOM_DATA(env, thiz, fieldID, data) (*env)->SetLongField (env, thiz, fieldID, (jlong)(jint)data)
+#endif
+
+/* Structure to contain all our information, so we can pass it to callbacks */
+typedef struct _CustomData {
+ jobject app; /* Application instance, used to call its methods. A global reference is kept. */
+ GstElement *pipeline; /* The running pipeline */
+ GMainContext *context; /* GLib context used to run the main loop */
+ GMainLoop *main_loop; /* GLib main loop */
+ gboolean initialized; /* To avoid informing the UI multiple times about the initialization */
+} CustomData;
+
+/* These global variables cache values which are not changing during execution */
+static pthread_t gst_app_thread;
+static pthread_key_t current_jni_env;
+static JavaVM *java_vm;
+static jfieldID custom_data_field_id;
+static jmethodID set_message_method_id;
+static jmethodID on_gstreamer_initialized_method_id;
+
+/*
+ * Private methods
+ */
+
+/* Register this thread with the VM */
+static JNIEnv *attach_current_thread (void) {
+ JNIEnv *env;
+ JavaVMAttachArgs args;
+
+ GST_DEBUG ("Attaching thread %p", g_thread_self ());
+ args.version = JNI_VERSION_1_4;
+ args.name = NULL;
+ args.group = NULL;
+
+ if ((*java_vm)->AttachCurrentThread (java_vm, &env, &args) < 0) {
+ GST_ERROR ("Failed to attach current thread");
+ return NULL;
+ }
+
+ return env;
+}
+
+/* Unregister this thread from the VM */
+static void detach_current_thread (void *env) {
+ GST_DEBUG ("Detaching thread %p", g_thread_self ());
+ (*java_vm)->DetachCurrentThread (java_vm);
+}
+
+/* Retrieve the JNI environment for this thread */
+static JNIEnv *get_jni_env (void) {
+ JNIEnv *env;
+
+ if ((env = pthread_getspecific (current_jni_env)) == NULL) {
+ env = attach_current_thread ();
+ pthread_setspecific (current_jni_env, env);
+ }
+
+ return env;
+}
+
+/* Change the content of the UI's TextView */
+static void set_ui_message (const gchar *message, CustomData *data) {
+ JNIEnv *env = get_jni_env ();
+ GST_DEBUG ("Setting message to: %s", message);
+ jstring jmessage = (*env)->NewStringUTF(env, message);
+ (*env)->CallVoidMethod (env, data->app, set_message_method_id, jmessage);
+ if ((*env)->ExceptionCheck (env)) {
+ GST_ERROR ("Failed to call Java method");
+ (*env)->ExceptionClear (env);
+ }
+ (*env)->DeleteLocalRef (env, jmessage);
+}
+
+/* Retrieve errors from the bus and show them on the UI */
+static void error_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ GError *err;
+ gchar *debug_info;
+ gchar *message_string;
+
+ gst_message_parse_error (msg, &err, &debug_info);
+ message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);
+ g_clear_error (&err);
+ g_free (debug_info);
+ set_ui_message (message_string, data);
+ g_free (message_string);
+ gst_element_set_state (data->pipeline, GST_STATE_NULL);
+}
+
+/* Notify UI about pipeline state changes */
+static void state_changed_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ /* Only pay attention to messages coming from the pipeline, not its children */
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->pipeline)) {
+ gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));
+ set_ui_message(message, data);
+ g_free (message);
+ }
+}
+
+/* Check if all conditions are met to report GStreamer as initialized.
+ * These conditions will change depending on the application */
+static void check_initialization_complete (CustomData *data) {
+ JNIEnv *env = get_jni_env ();
+ if (!data->initialized && data->main_loop) {
+ GST_DEBUG ("Initialization complete, notifying application. main_loop:%p", data->main_loop);
+ (*env)->CallVoidMethod (env, data->app, on_gstreamer_initialized_method_id);
+ if ((*env)->ExceptionCheck (env)) {
+ GST_ERROR ("Failed to call Java method");
+ (*env)->ExceptionClear (env);
+ }
+ data->initialized = TRUE;
+ }
+}
+
+/* Main method for the native code. This is executed on its own thread. */
+static void *app_function (void *userdata) {
+ JavaVMAttachArgs args;
+ GstBus *bus;
+ CustomData *data = (CustomData *)userdata;
+ GSource *bus_source;
+ GError *error = NULL;
+
+ GST_DEBUG ("Creating pipeline in CustomData at %p", data);
+
+ /* Create our own GLib Main Context and make it the default one */
+ data->context = g_main_context_new ();
+ g_main_context_push_thread_default(data->context);
+
+ /* Build pipeline */
+ data->pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);
+ if (error) {
+ gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
+ g_clear_error (&error);
+ set_ui_message(message, data);
+ g_free (message);
+ return NULL;
+ }
+
+ /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
+ bus = gst_element_get_bus (data->pipeline);
+ bus_source = gst_bus_create_watch (bus);
+ g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
+ g_source_attach (bus_source, data->context);
+ g_source_unref (bus_source);
+ g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, data);
+ g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, data);
+ gst_object_unref (bus);
+
+ /* Create a GLib Main Loop and set it to run */
+ GST_DEBUG ("Entering main loop... (CustomData:%p)", data);
+ data->main_loop = g_main_loop_new (data->context, FALSE);
+ check_initialization_complete (data);
+ g_main_loop_run (data->main_loop);
+ GST_DEBUG ("Exited main loop");
+ g_main_loop_unref (data->main_loop);
+ data->main_loop = NULL;
+
+ /* Free resources */
+ g_main_context_pop_thread_default(data->context);
+ g_main_context_unref (data->context);
+ gst_element_set_state (data->pipeline, GST_STATE_NULL);
+ gst_object_unref (data->pipeline);
+
+ return NULL;
+}
+
+/*
+ * Java Bindings
+ */
+
+/* Instruct the native code to create its internal data structure, pipeline and thread */
+static void gst_native_init (JNIEnv* env, jobject thiz) {
+ CustomData *data = g_new0 (CustomData, 1);
+ SET_CUSTOM_DATA (env, thiz, custom_data_field_id, data);
+ GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-2", 0, "Android tutorial 2");
+ gst_debug_set_threshold_for_name("tutorial-2", GST_LEVEL_DEBUG);
+ GST_DEBUG ("Created CustomData at %p", data);
+ data->app = (*env)->NewGlobalRef (env, thiz);
+ GST_DEBUG ("Created GlobalRef for app object at %p", data->app);
+ pthread_create (&gst_app_thread, NULL, &app_function, data);
+}
+
+/* Quit the main loop, remove the native thread and free resources */
+static void gst_native_finalize (JNIEnv* env, jobject thiz) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ GST_DEBUG ("Quitting main loop...");
+ g_main_loop_quit (data->main_loop);
+ GST_DEBUG ("Waiting for thread to finish...");
+ pthread_join (gst_app_thread, NULL);
+ GST_DEBUG ("Deleting GlobalRef for app object at %p", data->app);
+ (*env)->DeleteGlobalRef (env, data->app);
+ GST_DEBUG ("Freeing CustomData at %p", data);
+ g_free (data);
+ SET_CUSTOM_DATA (env, thiz, custom_data_field_id, NULL);
+ GST_DEBUG ("Done finalizing");
+}
+
+/* Set pipeline to PLAYING state */
+static void gst_native_play (JNIEnv* env, jobject thiz) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ GST_DEBUG ("Setting state to PLAYING");
+ gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
+}
+
+/* Set pipeline to PAUSED state */
+static void gst_native_pause (JNIEnv* env, jobject thiz) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ GST_DEBUG ("Setting state to PAUSED");
+ gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
+}
+
+/* Static class initializer: retrieve method and field IDs */
+static jboolean gst_native_class_init (JNIEnv* env, jclass klass) {
+ custom_data_field_id = (*env)->GetFieldID (env, klass, "native_custom_data", "J");
+ set_message_method_id = (*env)->GetMethodID (env, klass, "setMessage", "(Ljava/lang/String;)V");
+ on_gstreamer_initialized_method_id = (*env)->GetMethodID (env, klass, "onGStreamerInitialized", "()V");
+
+ if (!custom_data_field_id || !set_message_method_id || !on_gstreamer_initialized_method_id) {
+ /* We emit this message through the Android log instead of the GStreamer log because the later
+ * has not been initialized yet.
+ */
+ __android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "The calling class does not implement all necessary interface methods");
+ return JNI_FALSE;
+ }
+ return JNI_TRUE;
+}
+
+/* List of implemented native methods */
+static JNINativeMethod native_methods[] = {
+ { "nativeInit", "()V", (void *) gst_native_init},
+ { "nativeFinalize", "()V", (void *) gst_native_finalize},
+ { "nativePlay", "()V", (void *) gst_native_play},
+ { "nativePause", "()V", (void *) gst_native_pause},
+ { "nativeClassInit", "()Z", (void *) gst_native_class_init}
+};
+
+/* Library initializer */
+jint JNI_OnLoad(JavaVM *vm, void *reserved) {
+ JNIEnv *env = NULL;
+
+ java_vm = vm;
+
+ if ((*vm)->GetEnv(vm, (void**) &env, JNI_VERSION_1_4) != JNI_OK) {
+ __android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "Could not retrieve JNIEnv");
+ return 0;
+ }
+ jclass klass = (*env)->FindClass (env, "org/freedesktop/gstreamer/tutorials/tutorial_2/Tutorial2");
+ (*env)->RegisterNatives (env, klass, native_methods, G_N_ELEMENTS(native_methods));
+
+ pthread_key_create (¤t_jni_env, detach_current_thread);
+
+ return JNI_VERSION_1_4;
+}
+```
+
+Let’s start with the `CustomData` structure. We have seen it in most of
+the basic tutorials, and it is used to hold all our information in one
+place, so we can easily pass it around to
+callbacks:
+
+``` c
+/* Structure to contain all our information, so we can pass it to callbacks */
+typedef struct _CustomData {
+ jobject app; /* Application instance, used to call its methods. A global reference is kept. */
+ GstElement *pipeline; /* The running pipeline */
+ GMainContext *context; /* GLib context used to run the main loop */
+ GMainLoop *main_loop; /* GLib main loop */
+ gboolean initialized; /* To avoid informing the UI multiple times about the initialization */
+} CustomData;
+```
+
+We will see the meaning of each member as we go. What is interesting now
+is that `CustomData` belongs to the application, so a pointer is kept in
+the Tutorial2 Java class in the `private long
+native_custom_data` attribute. Java only holds this pointer for us; it
+is completely handled in C code.
+
+From C, this pointer can be set and retrieved with the
+[SetLongField()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#wp16613)
+and
+[GetLongField()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#wp16572)
+JNI functions, but two convenience macros have been defined,
+`SET_CUSTOM_DATA` and `GET_CUSTOM_DATA`. These macros are handy because
+the `long` type used in Java is always 64 bits wide, but the pointer
+used in C can be either 32 or 64 bits wide. The macros take care of the
+conversion without warnings.
+
+``` c
+/* Library initializer */
+jint JNI_OnLoad(JavaVM *vm, void *reserved) {
+ JNIEnv *env = NULL;
+
+ java_vm = vm;
+
+ if ((*vm)->GetEnv(vm, (void**) &env, JNI_VERSION_1_4) != JNI_OK) {
+ __android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "Could not retrieve JNIEnv");
+ return 0;
+ }
+ jclass klass = (*env)->FindClass (env, "org/freedesktop/gstreamer/tutorials/tutorial_2/Tutorial2");
+ (*env)->RegisterNatives (env, klass, native_methods, G_N_ELEMENTS(native_methods));
+
+ pthread_key_create (¤t_jni_env, detach_current_thread);
+
+ return JNI_VERSION_1_4;
+}
+```
+
+The `JNI_OnLoad` function is almost the same as the previous tutorial.
+It registers the list of native methods (which is longer in this
+tutorial). It also
+uses [pthread\_key\_create()](http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_key_create.html)
+to be able to store per-thread information, which is crucial to properly
+manage the JNI Environment, as shown later.
+
+``` c
+/* Static class initializer: retrieve method and field IDs */
+static jboolean gst_native_class_init (JNIEnv* env, jclass klass) {
+ custom_data_field_id = (*env)->GetFieldID (env, klass, "native_custom_data", "J");
+ set_message_method_id = (*env)->GetMethodID (env, klass, "setMessage", "(Ljava/lang/String;)V");
+ on_gstreamer_initialized_method_id = (*env)->GetMethodID (env, klass, "onGStreamerInitialized", "()V");
+
+ if (!custom_data_field_id || !set_message_method_id || !on_gstreamer_initialized_method_id) {
+ /* We emit this message through the Android log instead of the GStreamer log because the later
+ * has not been initialized yet.
+ */
+ __android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "The calling class does not implement all necessary interface methods");
+ return JNI_FALSE;
+ }
+ return JNI_TRUE;
+}
+```
+
+This method is called from the static initializer of the Java class,
+which is passed as a parameter (since this is called from a static
+method, it receives a class object instead of an instance object). In
+order for C code to be able to call a Java method, it needs to know the
+method’s
+[MethodID](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/types.html#wp1064).
+This ID is obtained from the method’s name and signature and can be
+cached. The purpose of the `gst_native_class_init()` function is to
+obtain the IDs of all the methods and fields that the C code will need.
+If some ID cannot be retrieved, the calling Java class does not offer
+the expected interface and execution should halt (which is not currently
+done for simplicity).
+
+Let’s review now the first native method which can be directly called
+from Java:
+
+#### `gst_native_init()` (`nativeInit()` from Java)
+
+This method is called at the end of Java's `onCreate()`.
+
+``` c
+static void gst_native_init (JNIEnv* env, jobject thiz) {
+ CustomData *data = g_new0 (CustomData, 1);
+ SET_CUSTOM_DATA (env, thiz, custom_data_field_id, data);
+```
+
+It first allocates memory for the `CustomData` structure and passes the
+pointer to the Java class with `SET_CUSTOM_DATA`, so it is remembered.
+
+``` c
+data->app = (*env)->NewGlobalRef (env, thiz);
+```
+
+A pointer to the application class (the `Tutorial2` class) is also kept
+in `CustomData` (a [Global
+Reference](http://developer.android.com/guide/practices/jni.html#local_and_global_references)
+is used) so its methods can be called later.
+
+``` c
+pthread_create (&gst_app_thread, NULL, &app_function, data);
+```
+
+Finally, a thread is created and it starts running the
+`app_function()` method.
+
+#### `app_function()`
+
+``` c
+/* Main method for the native code. This is executed on its own thread. */
+static void *app_function (void *userdata) {
+ JavaVMAttachArgs args;
+ GstBus *bus;
+ CustomData *data = (CustomData *)userdata;
+ GSource *bus_source;
+ GError *error = NULL;
+
+ GST_DEBUG ("Creating pipeline in CustomData at %p", data);
+
+ /* Create our own GLib Main Context and make it the default one */
+ data->context = g_main_context_new ();
+ g_main_context_push_thread_default(data->context);
+```
+
+It first creates a GLib context so all `GSource` are kept in the same
+place. This also helps cleaning after GSources created by other
+libraries which might not have been properly disposed of. A new context
+is created with `g_main_context_new()` and then it is made the default
+one for the thread with
+`g_main_context_push_thread_default()`.
+
+``` c
+data->pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);
+if (error) {
+ gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
+ g_clear_error (&error);
+ set_ui_message(message, data);
+ g_free (message);
+ return NULL;
+}
+```
+
+It then creates a pipeline the easy way, with `gst-parse-launch()`. In
+this case, it is simply an `audiotestsrc` (which produces a continuous
+tone) and an `autoaudiosink`, with accompanying adapter elements.
+
+``` c
+bus = gst_element_get_bus (data->pipeline);
+bus_source = gst_bus_create_watch (bus);
+g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
+g_source_attach (bus_source, data->context);
+g_source_unref (bus_source);
+g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, data);
+g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, data);
+gst_object_unref (bus);
+```
+
+These lines create a bus signal watch and connect to some interesting
+signals, just like we have been doing in the basic tutorials. The
+creation of the watch is done step by step instead of using
+`gst_bus_add_signal_watch()` to exemplify how to use a custom GLib
+context.
+
+``` c
+GST_DEBUG ("Entering main loop... (CustomData:%p)", data);
+data->main_loop = g_main_loop_new (data->context, FALSE);
+check_initialization_complete (data);
+g_main_loop_run (data->main_loop);
+GST_DEBUG ("Exited main loop");
+g_main_loop_unref (data->main_loop);
+data->main_loop = NULL;
+```
+
+Finally, the main loop is created and set to run. When it exits (because
+somebody else calls `g_main_loop_quit()`) the main loop is disposed of.
+Before entering the main loop, though,
+`check_initialization_complete()` is called. This method checks if all
+conditions are met to consider the native code “ready” to accept
+commands. Since having a running main loop is one of the conditions,
+`check_initialization_complete()` is called here. This method is
+reviewed below.
+
+Once the main loop has quit, all resources are freed in lines 178 to
+181.
+
+#### `check_initialization_complete()`
+
+``` c
+static void check_initialization_complete (CustomData *data) {
+ JNIEnv *env = get_jni_env ();
+ if (!data->initialized && data->main_loop) {
+ GST_DEBUG ("Initialization complete, notifying application. main_loop:%p", data->main_loop);
+ (*env)->CallVoidMethod (env, data->app, on_gstreamer_initialized_method_id);
+ if ((*env)->ExceptionCheck (env)) {
+ GST_ERROR ("Failed to call Java method");
+ (*env)->ExceptionClear (env);
+ }
+ data->initialized = TRUE;
+ }
+}
+```
+
+This method does not do much in this tutorial, but it will also be used
+in the next ones, with progressively more complex functionality. Its
+purpose is to check if the native code is ready to accept commands, and,
+if so, notify the UI code.
+
+In tutorial 2, the only conditions are 1) the code is not already
+initialized and 2) the main loop is running. If these two are met, the
+Java `onGStreamerInitialized()` method is called via the
+[CallVoidMethod()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#wp4256)
+JNI call.
+
+Here comes a tricky bit. JNI calls require a JNI Environment, **which is
+different for every thread**. C methods called from Java receive a
+`JNIEnv` pointer as a parameter, but this is not the situation with
+`check_initialization_complete()`. Here, we are in a thread which has
+never been called from Java, so we have no `JNIEnv`. We need to use the
+`JavaVM` pointer (passed to us in the `JNI_OnLoad()` method, and shared
+among all threads) to attach this thread to the Java Virtual Machine and
+obtain a `JNIEnv`. This `JNIEnv` is stored in the [Thread-Local
+Storage](http://en.wikipedia.org/wiki/Thread-local_storage) (TLS) using
+the pthread key we created in `JNI_OnLoad()`, so we do not need to
+attach the thread anymore.
+
+This behavior is implemented in the `get_jni_env()` method, used for
+example in `check_initialization_complete()` as we have just seen. Let’s
+see how it works, step by step:
+
+#### `get_jni_env()`
+
+``` c
+static JNIEnv *get_jni_env (void) {
+ JNIEnv *env;
+ if ((env = pthread_getspecific (current_jni_env)) == NULL) {
+ env = attach_current_thread ();
+ pthread_setspecific (current_jni_env, env);
+ }
+ return env;
+}
+```
+
+It first retrieves the current `JNIEnv` from the TLS using
+[pthread\_getspecific()](http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_getspecific.html)
+and the key we obtained from
+[pthread\_key\_create()](http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_key_create.html).
+If it returns NULL, we never attached this thread, so we do now with
+`attach_current_thread()` and then store the new `JNIEnv` into the TLS
+with
+[pthread\_setspecific()](http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_setspecific.html).
+
+#### `attach_current_thread()`
+
+This method is simply a convenience wrapper around
+[AttachCurrentThread()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/invocation.html#attach_current_thread)
+to deal with its parameters.
+
+#### `detach_current_thread()`
+
+This method is called by the pthreads library when a TLS key is deleted,
+meaning that the thread is about to be destroyed. We simply detach the
+thread from the JavaVM with
+[DetachCurrentThread()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/invocation.html#detach_current_thread).
+
+Let's now review the rest of the native methods accessible from Java:
+
+#### `gst_native_finalize()` (`nativeFinalize()` from Java)
+
+``` c
+static void gst_native_finalize (JNIEnv* env, jobject thiz) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ GST_DEBUG ("Quitting main loop...");
+ g_main_loop_quit (data->main_loop);
+ GST_DEBUG ("Waiting for thread to finish...");
+ pthread_join (gst_app_thread, NULL);
+ GST_DEBUG ("Deleting GlobalRef for app object at %p", data->app);
+ (*env)->DeleteGlobalRef (env, data->app);
+ GST_DEBUG ("Freeing CustomData at %p", data);
+ g_free (data);
+ SET_CUSTOM_DATA (env, thiz, custom_data_field_id, NULL);
+ GST_DEBUG ("Done finalizing");
+}
+```
+
+This method is called from Java in `onDestroy()`, when the activity is
+about to be destroyed. Here, we:
+
+ - Instruct the GLib main loop to quit with `g_main_loop_quit()`. This
+ call returns immediately, and the main loop will terminate at its
+ earliest convenience.
+ - Wait for the thread to finish with
+ [pthread\_join()](http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_join.html).
+ This call blocks until the `app_function()` method returns, meaning
+ that the main loop has exited, and the thread has been destroyed.
+ - Dispose of the global reference we kept for the Java application
+ class (`Tutorial2`) in `CustomData`.
+ - Free `CustomData` and set the Java pointer inside the
+ `Tutorial2` class to NULL with
+`SET_CUSTOM_DATA()`.
+
+#### `gst_native_play` and `gst_native_pause()` (`nativePlay` and `nativePause()` from Java)
+
+These two simple methods retrieve `CustomData` from the passed-in object
+with `GET_CUSTOM_DATA()` and set the pipeline found inside `CustomData`
+to the desired state, returning immediately.
+
+Finally, let’s see how the GStreamer callbacks are handled:
+
+#### `error_cb` and `state_changed_cb`
+
+This tutorial does not do much in these callbacks. They simply parse the
+error or state changed message and display a message in the UI using the
+`set_ui_message()` method:
+
+#### `set_ui_message()`
+
+``` c
+static void set_ui_message (const gchar *message, CustomData *data) {
+ JNIEnv *env = get_jni_env ();
+ GST_DEBUG ("Setting message to: %s", message);
+ jstring jmessage = (*env)->NewStringUTF(env, message);
+ (*env)->CallVoidMethod (env, data->app, set_message_method_id, jmessage);
+ if ((*env)->ExceptionCheck (env)) {
+ GST_ERROR ("Failed to call Java method");
+ (*env)->ExceptionClear (env);
+ }
+ (*env)->DeleteLocalRef (env, jmessage);
+}
+```
+
+
+
+This is the other method (besides `check_initialization_complete()`)
+that needs to call a Java function from a thread which never received an
+`JNIEnv` pointer directly. Notice how all the complexities of attaching
+the thread to the JavaVM and storing the JNI environment in the TLS are
+hidden in the simple call to `get_jni_env()`.
+
+The desired message (received in
+[ASCII](http://en.wikipedia.org/wiki/ASCII), or modified
+[UTF8](http://en.wikipedia.org/wiki/Modified_UTF-8#Modified_UTF-8)), is
+converted to [UTF16](http://en.wikipedia.org/wiki/UTF-16) as required by
+Java using the
+[NewStringUTF()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#wp17220)
+JNI call.
+
+The `setMessage()` Java method is called via the JNI
+[CallVoidMethod()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#wp4256)
+using the global reference to the class we are keeping in
+`CustomData` (`data->app`) and the `set_message_method_id` we cached in
+`gst_native_class_init()`.
+
+We check for exceptions with the JNI
+[ExceptionCheck()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#exception_check)
+method and free the UTF16 message with
+[DeleteLocalRef()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#DeleteLocalRef).
+
+### A pipeline on Android \[Android.mk\]
+
+**jni/Android.mk**
+
+``` ruby
+LOCAL_PATH := $(call my-dir)
+
+include $(CLEAR_VARS)
+
+LOCAL_MODULE := tutorial-2
+LOCAL_SRC_FILES := tutorial-2.c
+LOCAL_SHARED_LIBRARIES := gstreamer_android
+LOCAL_LDLIBS := -llog
+include $(BUILD_SHARED_LIBRARY)
+
+ifndef GSTREAMER_ROOT
+ifndef GSTREAMER_ROOT_ANDROID
+$(error GSTREAMER_ROOT_ANDROID is not defined!)
+endif
+GSTREAMER_ROOT := $(GSTREAMER_ROOT_ANDROID)
+endif
+GSTREAMER_NDK_BUILD_PATH := $(GSTREAMER_ROOT)/share/gst-android/ndk-build/
+include $(GSTREAMER_NDK_BUILD_PATH)/plugins.mk
+GSTREAMER_PLUGINS := $(GSTREAMER_PLUGINS_CORE) $(GSTREAMER_PLUGINS_SYS)
+include $(GSTREAMER_NDK_BUILD_PATH)/gstreamer-1.0.mk
+```
+
+Notice how the required `GSTREAMER_PLUGINS` are now
+`$(GSTREAMER_PLUGINS_CORE)` (For the test source and converter elements)
+and `$(GSTREAMER_PLUGINS_SYS)` (for the audio sink).
+
+And this is it\! This has been a rather long tutorial, but we covered a
+lot of territory. Building on top of this one, the following ones are
+shorter and focus only on the new topics.
+
+### Conclusion
+
+This tutorial has shown:
+
+ - How to manage multiple threads from C code and have them interact
+ with java.
+ - How to access Java code from any C thread
+ using [AttachCurrentThread()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/invocation.html#attach_current_thread).
+ - How to allocate a CustomData structure from C and have Java host it,
+ so it is available to all threads.
+
+Most of the methods introduced in this tutorial, like `get_jni_env()`,
+`check_initialization_complete()`, `app_function()` and the API methods
+`gst_native_init()`, `gst_native_finalize()` and
+`gst_native_class_init()` will continue to be used in the following
+tutorials with minimal modifications, so better get used to them\!
+
+As usual, it has been a pleasure having you here, and see you soon\!
+
+ [screenshot]: images/sdk-android-tutorial-a-running-pipeline-screenshot.png
\ No newline at end of file
--- /dev/null
+# Android tutorial 1: Link against GStreamer
+
+## Goal!
+
+![screenshot]
+
+This first Android tutorial is extremely simple: it just retrieves the
+GStreamer version and displays it on the screen. It exemplifies how to
+access GStreamer C code from Java and verifies that there have been no
+linkage problems.
+
+## Hello GStreamer \[Java code\]
+
+At **FIXME: add path** folder you should find an `android-tutorial-1` directory,
+with the usual Android NDK structure: a `src` folder for the Java code,
+a `jni` folder for the C code and a `res` folder for UI resources.
+
+We recommend that you open this project in Eclipse (as explained
+in [](sdk-installing-for-android-development.md)) so you can
+easily see how all the pieces fit together.
+
+Let’s first introduce the Java code, then the C code and finally the
+makefile that allows GStreamer integration.
+
+**src/org/freedesktop/gstreamer/tutorials/tutorial_1/Tutorial1.java**
+
+``` java
+package org.freedesktop.gstreamer.tutorials.tutorial_1;
+
+import android.app.Activity;
+import android.os.Bundle;
+import android.widget.TextView;
+import android.widget.Toast;
+
+import org.freedesktop.gstreamer.GStreamer;
+
+public class Tutorial1 extends Activity {
+ private native String nativeGetGStreamerInfo();
+
+ // Called when the activity is first created.
+ @Override
+ public void onCreate(Bundle savedInstanceState)
+ {
+ super.onCreate(savedInstanceState);
+
+ try {
+ GStreamer.init(this);
+ } catch (Exception e) {
+ Toast.makeText(this, e.getMessage(), Toast.LENGTH_LONG).show();
+ finish();
+ return;
+ }
+
+ setContentView(R.layout.main);
+
+ TextView tv = (TextView)findViewById(R.id.textview_info);
+ tv.setText("Welcome to " + nativeGetGStreamerInfo() + " !");
+ }
+
+ static {
+ System.loadLibrary("gstreamer_android");
+ System.loadLibrary("tutorial-1");
+ }
+
+}
+```
+
+Calls from Java to C happen through native methods, like the one
+declared here:
+
+``` java
+private native String nativeGetGStreamerInfo();
+```
+
+This tells Java that there exists a method with this signature somewhere
+so it compiles happily. It is your responsibility to ensure that, **at
+runtime**, this method is accessible. This is accomplished by the C code
+shown later.
+
+The first bit of code that gets actually executed is the static
+initializer of the class:
+
+``` java
+static {
+ System.loadLibrary("gstreamer_android");
+ System.loadLibrary("tutorial-1");
+}
+```
+
+It loads `libgstreamer_android.so`, which contains all GStreamer
+methods, and `libtutorial-1.so`, which contains the C part of this
+tutorial, explained below.
+
+Upon loading, each of these libraries’ `JNI_OnLoad()` method is
+executed. It basically registers the native methods that these libraries
+expose. The GStreamer library only exposes a `init()` method, which
+initializes GStreamer and registers all plugins (The tutorial library is
+explained later below).
+
+``` java
+try {
+ GStreamer.init(this);
+} catch (Exception e) {
+ Toast.makeText(this, e.getMessage(), Toast.LENGTH_LONG).show();
+ finish();
+ return;
+}
+```
+
+Next, in the `OnCreate()` method of the
+[Activity](http://developer.android.com/reference/android/app/Activity.html)
+we actually initialize GStreamer by calling `GStreamer.init()`. This
+method requires a
+[Context](http://developer.android.com/reference/android/content/Context.html)
+so it cannot be called from the static initializer, but there is no
+danger in calling it multiple times, as all but the first time the calls
+will be ignored.
+
+Should initialization fail, the `init()` method would throw an
+[Exception](http://developer.android.com/reference/java/lang/Exception.html)
+with the details provided by the GStreamer library.
+
+``` java
+TextView tv = (TextView)findViewById(R.id.textview_info);
+tv.setText("Welcome to " + nativeGetGStreamerInfo() + " !");
+```
+
+Then, the native method `nativeGetGStreamerInfo()` is called and a
+string is retrieved, which is used to format the content of the
+[TextView](http://developer.android.com/reference/android/widget/TextView.html)
+in the UI.
+
+This finishes the UI part of this tutorial. Let’s take a look at the C
+code:
+
+## Hello GStreamer \[C code\]
+
+**jni/tutorial-1.c**
+
+``` c
+#include <string.h>
+#include <jni.h>
+#include <android/log.h>
+#include <gst/gst.h>
+
+/*
+ * Java Bindings
+ */
+static jstring gst_native_get_gstreamer_info (JNIEnv* env, jobject thiz) {
+ char *version_utf8 = gst_version_string();
+ jstring *version_jstring = (*env)->NewStringUTF(env, version_utf8);
+ g_free (version_utf8);
+ return version_jstring;
+}
+
+static JNINativeMethod native_methods[] = {
+ { "nativeGetGStreamerInfo", "()Ljava/lang/String;", (void *) gst_native_get_gstreamer_info}
+};
+
+jint JNI_OnLoad(JavaVM *vm, void *reserved) {
+ JNIEnv *env = NULL;
+
+ if ((*vm)->GetEnv(vm, (void**) &env, JNI_VERSION_1_4) != JNI_OK) {
+ __android_log_print (ANDROID_LOG_ERROR, "tutorial-1", "Could not retrieve JNIEnv");
+ return 0;
+ }
+ jclass klass = (*env)->FindClass (env, "org/freedesktop/gstreamer/tutorials/tutorial_1/Tutorial1");
+ (*env)->RegisterNatives (env, klass, native_methods, G_N_ELEMENTS(native_methods));
+
+ return JNI_VERSION_1_4;
+}
+```
+
+The `JNI_OnLoad()` method is executed every time the Java Virtual
+Machine (VM) loads a library.
+
+Here, we retrieve the JNI environment needed to make calls that interact
+with Java:
+
+``` c
+JNIEnv *env = NULL;
+
+if ((*vm)->GetEnv(vm, (void**) &env, JNI_VERSION_1_4) != JNI_OK) {
+ __android_log_print (ANDROID_LOG_ERROR, "tutorial-1", "Could not retrieve JNIEnv");
+ return 0;
+}
+```
+
+And then locate the class containing the UI part of this tutorial using
+`
+FindClass()`:
+
+``` c
+jclass klass = (*env)->FindClass (env, "org/freedesktop/gstreamer/tutorials/tutorial_1/Tutorial1");
+```
+
+Finally, we register our native methods with `RegisterNatives()`, this
+is, we provide the code for the methods we advertised in Java using the
+**`native`**
+ keyword:
+
+``` c
+(*env)->RegisterNatives (env, klass, native_methods, G_N_ELEMENTS(native_methods));
+```
+
+The `native_methods` array describes each one of the methods to register
+(only one in this tutorial). For each method, it provides its Java
+name, its [type
+signature](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/types.html#wp276)
+and a pointer to the C function implementing it:
+
+``` c
+static JNINativeMethod native_methods[] = {
+ { "nativeGetGStreamerInfo", "()Ljava/lang/String;", (void *) gst_native_get_gstreamer_info}
+};
+```
+
+The only native method used in this tutorial
+is `nativeGetGStreamerInfo()`:
+
+``` c
+jstring gst_native_get_gstreamer_info (JNIEnv* env, jobject thiz) {
+ char *version_utf8 = gst_version_string();
+ jstring *version_jstring = (*env)->NewStringUTF(env, version_utf8);
+ g_free (version_utf8);
+ return version_jstring;
+}
+```
+
+It simply calls `gst_version_string()` to obtain a string describing
+this version of GStreamer. This [Modified
+UTF8](http://en.wikipedia.org/wiki/UTF-8#Modified_UTF-8) string is then
+converted to [UTF16](http://en.wikipedia.org/wiki/UTF-16) by `
+NewStringUTF()` as required by Java and returned. Java will be
+responsible for freeing the memory used by the new UTF16 String, but we
+must free the `char *` returned by `gst_version_string()`.
+
+## Hello GStreamer \[Android.mk\]
+
+**jni/Android.mk**
+
+``` ruby
+LOCAL_PATH := $(call my-dir)
+
+include $(CLEAR_VARS)
+
+LOCAL_MODULE := tutorial-1
+LOCAL_SRC_FILES := tutorial-1.c
+LOCAL_SHARED_LIBRARIES := gstreamer_android
+LOCAL_LDLIBS := -llog
+include $(BUILD_SHARED_LIBRARY)
+
+ifndef GSTREAMER_ROOT
+ifndef GSTREAMER_ROOT_ANDROID
+$(error GSTREAMER_ROOT_ANDROID is not defined!)
+endif
+GSTREAMER_ROOT := $(GSTREAMER_ROOT_ANDROID)
+endif
+GSTREAMER_NDK_BUILD_PATH := $(GSTREAMER_ROOT)/share/gst-android/ndk-build/
+GSTREAMER_PLUGINS := coreelements
+include $(GSTREAMER_NDK_BUILD_PATH)/gstreamer-1.0.mk
+```
+
+This is a barebones makefile for a project with GStreamer support. It
+simply states that it depends on the `libgstreamer_android.so` library
+(line 7), and requires the `coreelements` plugin (line 18). More complex
+applications will probably add more libraries and plugins
+to `Android.mk`
+
+## Conclusion
+
+This ends the first Android tutorial. It has shown that, besides the
+interconnection between Java and C (which abides to the standard JNI
+procedure), adding GStreamer support to an Android application is not
+any more complicated than adding it to a desktop application.
+
+The following tutorials detail the few places in which care has to be
+taken when developing specifically for the Android platform.
+
+As usual, it has been a pleasure having you here, and see you soon\!
+
+ [screenshot]: images/sdk-android-tutorial-link-against-gstreamer-screenshot.png
\ No newline at end of file
--- /dev/null
+# Android tutorial 4: A basic media player
+
+### Goal
+
+![screenshot]
+
+Enough testing with synthetic images and audio tones! This tutorial
+finally plays actual media, streamed directly from the Internet, in your
+Android device. It shows:
+
+ - How to keep the User Interface regularly updated with the current
+ playback position and duration
+ - How to implement a [Seek
+ Bar](http://developer.android.com/reference/android/widget/SeekBar.html)
+ - How to report the media size to adapt the display surface
+
+It also uses the knowledge gathered in the [](sdk-basic-tutorials.md) regarding:
+
+ - How to use `playbin` to play any kind of media
+ - How to handle network resilience problems
+
+### Introduction
+
+From the previous tutorials, we already have almost all necessary pieces
+to build a media player. The most complex part is assembling a pipeline
+which retrieves, decodes and displays the media, but we already know
+that the `playbin` element can take care of all that for us. We only
+need to replace the manual pipeline we used in
+[](sdk-android-tutorial-video.md) with a single-element
+`playbin` pipeline and we are good to go!
+
+However, we can do better than. We will add a [Seek
+Bar](http://developer.android.com/reference/android/widget/SeekBar.html),
+with a moving thumb that will advance as our current position in the
+media advances. We will also allow the user to drag the thumb, to jump
+(or *seek*) to a different position.
+
+And finally, we will make the video surface adapt to the media size, so
+the video sink is not forced to draw black borders around the clip.
+ This also allows the Android layout to adapt more nicely to the actual
+media content. You can still force the video surface to have a specific
+size if you really want to.
+
+### A basic media player \[Java code\]
+
+**src/com/gst\_sdk\_tutorials/tutorial\_4/Tutorial4.java**
+
+``` java
+package com.gst_sdk_tutorials.tutorial_4;
+
+import java.text.SimpleDateFormat;
+import java.util.Date;
+import java.util.TimeZone;
+
+import android.app.Activity;
+import android.os.Bundle;
+import android.util.Log;
+import android.view.SurfaceHolder;
+import android.view.SurfaceView;
+import android.view.View;
+import android.view.View.OnClickListener;
+import android.widget.ImageButton;
+import android.widget.SeekBar;
+import android.widget.SeekBar.OnSeekBarChangeListener;
+import android.widget.TextView;
+import android.widget.Toast;
+
+import org.freedesktop.gstreamer.GStreamer;
+
+public class Tutorial4 extends Activity implements SurfaceHolder.Callback, OnSeekBarChangeListener {
+ private native void nativeInit(); // Initialize native code, build pipeline, etc
+ private native void nativeFinalize(); // Destroy pipeline and shutdown native code
+ private native void nativeSetUri(String uri); // Set the URI of the media to play
+ private native void nativePlay(); // Set pipeline to PLAYING
+ private native void nativeSetPosition(int milliseconds); // Seek to the indicated position, in milliseconds
+ private native void nativePause(); // Set pipeline to PAUSED
+ private static native boolean nativeClassInit(); // Initialize native class: cache Method IDs for callbacks
+ private native void nativeSurfaceInit(Object surface); // A new surface is available
+ private native void nativeSurfaceFinalize(); // Surface about to be destroyed
+ private long native_custom_data; // Native code will use this to keep private data
+
+ private boolean is_playing_desired; // Whether the user asked to go to PLAYING
+ private int position; // Current position, reported by native code
+ private int duration; // Current clip duration, reported by native code
+ private boolean is_local_media; // Whether this clip is stored locally or is being streamed
+ private int desired_position; // Position where the users wants to seek to
+ private String mediaUri; // URI of the clip being played
+
+ private final String defaultMediaUri = "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-368p.ogv";
+
+ // Called when the activity is first created.
+ @Override
+ public void onCreate(Bundle savedInstanceState)
+ {
+ super.onCreate(savedInstanceState);
+
+ // Initialize GStreamer and warn if it fails
+ try {
+ GStreamer.init(this);
+ } catch (Exception e) {
+ Toast.makeText(this, e.getMessage(), Toast.LENGTH_LONG).show();
+ finish();
+ return;
+ }
+
+ setContentView(R.layout.main);
+
+ ImageButton play = (ImageButton) this.findViewById(R.id.button_play);
+ play.setOnClickListener(new OnClickListener() {
+ public void onClick(View v) {
+ is_playing_desired = true;
+ nativePlay();
+ }
+ });
+
+ ImageButton pause = (ImageButton) this.findViewById(R.id.button_stop);
+ pause.setOnClickListener(new OnClickListener() {
+ public void onClick(View v) {
+ is_playing_desired = false;
+ nativePause();
+ }
+ });
+
+ SurfaceView sv = (SurfaceView) this.findViewById(R.id.surface_video);
+ SurfaceHolder sh = sv.getHolder();
+ sh.addCallback(this);
+
+ SeekBar sb = (SeekBar) this.findViewById(R.id.seek_bar);
+ sb.setOnSeekBarChangeListener(this);
+
+ // Retrieve our previous state, or initialize it to default values
+ if (savedInstanceState != null) {
+ is_playing_desired = savedInstanceState.getBoolean("playing");
+ position = savedInstanceState.getInt("position");
+ duration = savedInstanceState.getInt("duration");
+ mediaUri = savedInstanceState.getString("mediaUri");
+ Log.i ("GStreamer", "Activity created with saved state:");
+ } else {
+ is_playing_desired = false;
+ position = duration = 0;
+ mediaUri = defaultMediaUri;
+ Log.i ("GStreamer", "Activity created with no saved state:");
+ }
+ is_local_media = false;
+ Log.i ("GStreamer", " playing:" + is_playing_desired + " position:" + position +
+ " duration: " + duration + " uri: " + mediaUri);
+
+ // Start with disabled buttons, until native code is initialized
+ this.findViewById(R.id.button_play).setEnabled(false);
+ this.findViewById(R.id.button_stop).setEnabled(false);
+
+ nativeInit();
+ }
+
+ protected void onSaveInstanceState (Bundle outState) {
+ Log.d ("GStreamer", "Saving state, playing:" + is_playing_desired + " position:" + position +
+ " duration: " + duration + " uri: " + mediaUri);
+ outState.putBoolean("playing", is_playing_desired);
+ outState.putInt("position", position);
+ outState.putInt("duration", duration);
+ outState.putString("mediaUri", mediaUri);
+ }
+
+ protected void onDestroy() {
+ nativeFinalize();
+ super.onDestroy();
+ }
+
+ // Called from native code. This sets the content of the TextView from the UI thread.
+ private void setMessage(final String message) {
+ final TextView tv = (TextView) this.findViewById(R.id.textview_message);
+ runOnUiThread (new Runnable() {
+ public void run() {
+ tv.setText(message);
+ }
+ });
+ }
+
+ // Set the URI to play, and record whether it is a local or remote file
+ private void setMediaUri() {
+ nativeSetUri (mediaUri);
+ is_local_media = mediaUri.startsWith("file://");
+ }
+
+ // Called from native code. Native code calls this once it has created its pipeline and
+ // the main loop is running, so it is ready to accept commands.
+ private void onGStreamerInitialized () {
+ Log.i ("GStreamer", "GStreamer initialized:");
+ Log.i ("GStreamer", " playing:" + is_playing_desired + " position:" + position + " uri: " + mediaUri);
+
+ // Restore previous playing state
+ setMediaUri ();
+ nativeSetPosition (position);
+ if (is_playing_desired) {
+ nativePlay();
+ } else {
+ nativePause();
+ }
+
+ // Re-enable buttons, now that GStreamer is initialized
+ final Activity activity = this;
+ runOnUiThread(new Runnable() {
+ public void run() {
+ activity.findViewById(R.id.button_play).setEnabled(true);
+ activity.findViewById(R.id.button_stop).setEnabled(true);
+ }
+ });
+ }
+
+ // The text widget acts as an slave for the seek bar, so it reflects what the seek bar shows, whether
+ // it is an actual pipeline position or the position the user is currently dragging to.
+ private void updateTimeWidget () {
+ final TextView tv = (TextView) this.findViewById(R.id.textview_time);
+ final SeekBar sb = (SeekBar) this.findViewById(R.id.seek_bar);
+ final int pos = sb.getProgress();
+
+ SimpleDateFormat df = new SimpleDateFormat("HH:mm:ss");
+ df.setTimeZone(TimeZone.getTimeZone("UTC"));
+ final String message = df.format(new Date (pos)) + " / " + df.format(new Date (duration));
+ tv.setText(message);
+ }
+
+ // Called from native code
+ private void setCurrentPosition(final int position, final int duration) {
+ final SeekBar sb = (SeekBar) this.findViewById(R.id.seek_bar);
+
+ // Ignore position messages from the pipeline if the seek bar is being dragged
+ if (sb.isPressed()) return;
+
+ runOnUiThread (new Runnable() {
+ public void run() {
+ sb.setMax(duration);
+ sb.setProgress(position);
+ updateTimeWidget();
+ }
+ });
+ this.position = position;
+ this.duration = duration;
+ }
+
+ static {
+ System.loadLibrary("gstreamer_android");
+ System.loadLibrary("tutorial-4");
+ nativeClassInit();
+ }
+
+ public void surfaceChanged(SurfaceHolder holder, int format, int width,
+ int height) {
+ Log.d("GStreamer", "Surface changed to format " + format + " width "
+ + width + " height " + height);
+ nativeSurfaceInit (holder.getSurface());
+ }
+
+ public void surfaceCreated(SurfaceHolder holder) {
+ Log.d("GStreamer", "Surface created: " + holder.getSurface());
+ }
+
+ public void surfaceDestroyed(SurfaceHolder holder) {
+ Log.d("GStreamer", "Surface destroyed");
+ nativeSurfaceFinalize ();
+ }
+
+ // Called from native code when the size of the media changes or is first detected.
+ // Inform the video surface about the new size and recalculate the layout.
+ private void onMediaSizeChanged (int width, int height) {
+ Log.i ("GStreamer", "Media size changed to " + width + "x" + height);
+ final GStreamerSurfaceView gsv = (GStreamerSurfaceView) this.findViewById(R.id.surface_video);
+ gsv.media_width = width;
+ gsv.media_height = height;
+ runOnUiThread(new Runnable() {
+ public void run() {
+ gsv.requestLayout();
+ }
+ });
+ }
+
+ // The Seek Bar thumb has moved, either because the user dragged it or we have called setProgress()
+ public void onProgressChanged(SeekBar sb, int progress, boolean fromUser) {
+ if (fromUser == false) return;
+ desired_position = progress;
+ // If this is a local file, allow scrub seeking, this is, seek as soon as the slider is moved.
+ if (is_local_media) nativeSetPosition(desired_position);
+ updateTimeWidget();
+ }
+
+ // The user started dragging the Seek Bar thumb
+ public void onStartTrackingTouch(SeekBar sb) {
+ nativePause();
+ }
+
+ // The user released the Seek Bar thumb
+ public void onStopTrackingTouch(SeekBar sb) {
+ // If this is a remote file, scrub seeking is probably not going to work smoothly enough.
+ // Therefore, perform only the seek when the slider is released.
+ if (!is_local_media) nativeSetPosition(desired_position);
+ if (is_playing_desired) nativePlay();
+ }
+}
+```
+
+#### Supporting arbitrary media URIs
+
+The C code provides the `nativeSetUri()` method so we can indicate the
+URI of the media to play. Since `playbin` will be taking care of
+retrieving the media, we can use local or remote URIs indistinctly
+(`file://` or `http://`, for example). From Java, though, we want to
+keep track of whether the file is local or remote, because we will not
+offer the same functionalities. We keep track of this in the
+`is_local_media` variable, and update it every time we change the media
+URI:
+
+``` java
+private void setMediaUri() {
+ nativeSetUri (mediaUri);
+ is_local_media = mediaUri.startsWith("file://");
+}
+```
+
+We call `setMediaUri()` in the `onGStreamerInitialized()` callback, once
+the pipeline is ready to accept commands.
+
+#### Reporting media size
+
+Every time the size of the media changes (which could happen mid-stream,
+for some kind of streams), or when it is first detected, C code calls
+our `onMediaSizeChanged()` callback:
+
+``` java
+private void onMediaSizeChanged (int width, int height) {
+ Log.i ("GStreamer", "Media size changed to " + width + "x" + height);
+ final GStreamerSurfaceView gsv = (GStreamerSurfaceView) this.findViewById(R.id.surface_video);
+ gsv.media_width = width;
+ gsv.media_height = height;
+ runOnUiThread(new Runnable() {
+ public void run() {
+ gsv.requestLayout();
+ }
+ });
+}
+```
+
+Here we simply pass the new size onto the `GStreamerSurfaceView` in
+charge of displaying the media, and ask the Android layout to be
+recalculated. Eventually, the `onMeasure()` method in
+GStreamerSurfaceView will be called and the new size will be taken
+into account. As we have already seen in
+[](sdk-android-tutorial-a-running-pipeline.md), methods which change
+the UI must be called from the main thread, and we are now in a
+callback from some GStreamer internal thread. Hence, the usage of
+[runOnUiThread()](http://developer.android.com/reference/android/app/Activity.html#runOnUiThread\(java.lang.Runnable\)).
+
+#### Refreshing the Seek Bar
+
+[](sdk-basic-tutorial-toolkit-integration.md)
+has already shown how to implement a [Seek
+Bar](http://developer.android.com/reference/android/widget/SeekBar.html) using
+the GTK+ toolkit. The implementation on Android is very similar.
+
+The Seek Bar accomplishes to functions: First, it moves on its own to
+reflect the current playback position in the media. Second, it can be
+dragged by the user to seek to a different position.
+
+To realize the first function, C code will periodically call our
+`setCurrentPosition()` method so we can update the position of the thumb
+in the Seek Bar. Again we do so from the UI thread, using
+`RunOnUiThread()`.
+
+``` java
+private void setCurrentPosition(final int position, final int duration) {
+ final SeekBar sb = (SeekBar) this.findViewById(R.id.seek_bar);
+
+ // Ignore position messages from the pipeline if the seek bar is being dragged
+ if (sb.isPressed()) return;
+
+ runOnUiThread (new Runnable() {
+ public void run() {
+ sb.setMax(duration);
+ sb.setProgress(position);
+ updateTimeWidget();
+ }
+ });
+ this.position = position;
+ this.duration = duration;
+}
+```
+
+To the left of the Seek Bar (refer to the screenshot at the top of this
+page), there is a
+[TextView](http://developer.android.com/reference/android/widget/TextView.html)
+widget which we will use to display the current position and duration in
+`HH:mm:ss / HH:mm:ss` textual format. The `updateTimeWidget()` method
+takes care of it, and must be called every time the Seek Bar is updated:
+
+``` java
+private void updateTimeWidget () {
+ final TextView tv = (TextView) this.findViewById(R.id.textview_time);
+ final SeekBar sb = (SeekBar) this.findViewById(R.id.seek_bar);
+ final int pos = sb.getProgress();
+
+ SimpleDateFormat df = new SimpleDateFormat("HH:mm:ss");
+ df.setTimeZone(TimeZone.getTimeZone("UTC"));
+ final String message = df.format(new Date (pos)) + " / " + df.format(new Date (duration));
+ tv.setText(message);
+}
+```
+
+#### Seeking with the Seek Bar
+
+To perform the second function of the [Seek
+Bar](http://developer.android.com/reference/android/widget/SeekBar.html) (allowing
+the user to seek by dragging the thumb), we implement the
+[OnSeekBarChangeListener](http://developer.android.com/reference/android/widget/SeekBar.OnSeekBarChangeListener.html)
+interface in the
+Activity:
+
+``` java
+public class Tutorial4 extends Activity implements SurfaceHolder.Callback, OnSeekBarChangeListener {
+```
+
+And we register the Activity as the listener for the [Seek
+Bar](http://developer.android.com/reference/android/widget/SeekBar.html)’s
+events in the `onCreate()` method:
+
+``` java
+SeekBar sb = (SeekBar) this.findViewById(R.id.seek_bar);
+sb.setOnSeekBarChangeListener(this);
+```
+
+We will now be notified of three events: When the user starts dragging
+the thumb, every time the thumb moves and when the thumb is released by
+the user:
+
+``` java
+public void onStartTrackingTouch(SeekBar sb) {
+ nativePause();
+}
+```
+
+[onStartTrackingTouch()](http://developer.android.com/reference/android/widget/SeekBar.OnSeekBarChangeListener.html#onStartTrackingTouch\(android.widget.SeekBar\))
+is called when the user starts dragging, and the only thing we do is
+pause the pipeline. If the user is searching for a particular scene, we
+do not want it to keep
+moving.
+
+``` java
+public void onProgressChanged(SeekBar sb, int progress, boolean fromUser) {
+ if (fromUser == false) return;
+ desired_position = progress;
+ // If this is a local file, allow scrub seeking, this is, seek soon as the slider is moved.
+ if (is_local_media) nativeSetPosition(desired_position);
+ updateTimeWidget();
+}
+```
+
+[onProgressChanged()](http://developer.android.com/reference/android/widget/SeekBar.OnSeekBarChangeListener.html#onProgressChanged\(android.widget.SeekBar,%20int,%20boolean\)) is
+called every time the thumb moves, be it because the user dragged it, or
+because we called `setProgress()` on the Seek Bar. We discard the latter
+case with the handy `fromUser` parameter.
+
+As the comment says, if this is a local media, we allow scrub seeking,
+this is, we jump to the indicated position as soon as the thumb moves.
+Otherwise, the seek will be performed when the thumb is released, and
+the only thing we do here is update the textual time widget.
+
+``` java
+public void onStopTrackingTouch(SeekBar sb) {
+ // If this is a remote file, scrub seeking is probably not going to work smoothly enough.
+ // Therefore, perform only the seek when the slider is released.
+ if (!is_local_media) nativeSetPosition(desired_position);
+ if (is_playing_desired) nativePlay();
+}
+```
+
+Finally, [onStopTrackingTouch()](http://developer.android.com/reference/android/widget/SeekBar.OnSeekBarChangeListener.html#onStopTrackingTouch\(android.widget.SeekBar\))
+is called when the thumb is released. We simply perform the seek
+operation if the file was non-local, and restore the pipeline to the
+desired playing state.
+
+This concludes the User interface part of this tutorial. Let’s review
+now the under-the-hood C code that allows this to work.
+
+### A basic media player \[C code\]
+
+**jni/tutorial-4.c**
+
+``` c
+#include <string.h>
+#include <jni.h>
+#include <android/log.h>
+#include <android/native_window.h>
+#include <android/native_window_jni.h>
+#include <gst/gst.h>
+#include <gst/interfaces/xoverlay.h>
+#include <gst/video/video.h>
+#include <pthread.h>
+
+GST_DEBUG_CATEGORY_STATIC (debug_category);
+#define GST_CAT_DEFAULT debug_category
+
+/*
+ * These macros provide a way to store the native pointer to CustomData, which might be 32 or 64 bits, into
+ * a jlong, which is always 64 bits, without warnings.
+ */
+#if GLIB_SIZEOF_VOID_P == 8
+## define GET_CUSTOM_DATA(env, thiz, fieldID) (CustomData *)(*env)->GetLongField (env, thiz, fieldID)
+## define SET_CUSTOM_DATA(env, thiz, fieldID, data) (*env)->SetLongField (env, thiz, fieldID, (jlong)data)
+#else
+## define GET_CUSTOM_DATA(env, thiz, fieldID) (CustomData *)(jint)(*env)->GetLongField (env, thiz, fieldID)
+## define SET_CUSTOM_DATA(env, thiz, fieldID, data) (*env)->SetLongField (env, thiz, fieldID, (jlong)(jint)data)
+#endif
+
+/* Do not allow seeks to be performed closer than this distance. It is visually useless, and will probably
+ * confuse some demuxers. */
+#define SEEK_MIN_DELAY (500 * GST_MSECOND)
+
+/* Structure to contain all our information, so we can pass it to callbacks */
+typedef struct _CustomData {
+ jobject app; /* Application instance, used to call its methods. A global reference is kept. */
+ GstElement *pipeline; /* The running pipeline */
+ GMainContext *context; /* GLib context used to run the main loop */
+ GMainLoop *main_loop; /* GLib main loop */
+ gboolean initialized; /* To avoid informing the UI multiple times about the initialization */
+ ANativeWindow *native_window; /* The Android native window where video will be rendered */
+ GstState state; /* Current pipeline state */
+ GstState target_state; /* Desired pipeline state, to be set once buffering is complete */
+ gint64 duration; /* Cached clip duration */
+ gint64 desired_position; /* Position to seek to, once the pipeline is running */
+ GstClockTime last_seek_time; /* For seeking overflow prevention (throttling) */
+ gboolean is_live; /* Live streams do not use buffering */
+} CustomData;
+
+/* playbin flags */
+typedef enum {
+ GST_PLAY_FLAG_TEXT = (1 << 2) /* We want subtitle output */
+} GstPlayFlags;
+
+/* These global variables cache values which are not changing during execution */
+static pthread_t gst_app_thread;
+static pthread_key_t current_jni_env;
+static JavaVM *java_vm;
+static jfieldID custom_data_field_id;
+static jmethodID set_message_method_id;
+static jmethodID set_current_position_method_id;
+static jmethodID on_gstreamer_initialized_method_id;
+static jmethodID on_media_size_changed_method_id;
+
+/*
+ * Private methods
+ */
+
+/* Register this thread with the VM */
+static JNIEnv *attach_current_thread (void) {
+ JNIEnv *env;
+ JavaVMAttachArgs args;
+
+ GST_DEBUG ("Attaching thread %p", g_thread_self ());
+ args.version = JNI_VERSION_1_4;
+ args.name = NULL;
+ args.group = NULL;
+
+ if ((*java_vm)->AttachCurrentThread (java_vm, &env, &args) < 0) {
+ GST_ERROR ("Failed to attach current thread");
+ return NULL;
+ }
+
+ return env;
+}
+
+/* Unregister this thread from the VM */
+static void detach_current_thread (void *env) {
+ GST_DEBUG ("Detaching thread %p", g_thread_self ());
+ (*java_vm)->DetachCurrentThread (java_vm);
+}
+
+/* Retrieve the JNI environment for this thread */
+static JNIEnv *get_jni_env (void) {
+ JNIEnv *env;
+
+ if ((env = pthread_getspecific (current_jni_env)) == NULL) {
+ env = attach_current_thread ();
+ pthread_setspecific (current_jni_env, env);
+ }
+
+ return env;
+}
+
+/* Change the content of the UI's TextView */
+static void set_ui_message (const gchar *message, CustomData *data) {
+ JNIEnv *env = get_jni_env ();
+ GST_DEBUG ("Setting message to: %s", message);
+ jstring jmessage = (*env)->NewStringUTF(env, message);
+ (*env)->CallVoidMethod (env, data->app, set_message_method_id, jmessage);
+ if ((*env)->ExceptionCheck (env)) {
+ GST_ERROR ("Failed to call Java method");
+ (*env)->ExceptionClear (env);
+ }
+ (*env)->DeleteLocalRef (env, jmessage);
+}
+
+/* Tell the application what is the current position and clip duration */
+static void set_current_ui_position (gint position, gint duration, CustomData *data) {
+ JNIEnv *env = get_jni_env ();
+ (*env)->CallVoidMethod (env, data->app, set_current_position_method_id, position, duration);
+ if ((*env)->ExceptionCheck (env)) {
+ GST_ERROR ("Failed to call Java method");
+ (*env)->ExceptionClear (env);
+ }
+}
+
+/* If we have pipeline and it is running, query the current position and clip duration and inform
+ * the application */
+static gboolean refresh_ui (CustomData *data) {
+ GstFormat fmt = GST_FORMAT_TIME;
+ gint64 current = -1;
+ gint64 position;
+
+ /* We do not want to update anything unless we have a working pipeline in the PAUSED or PLAYING state */
+ if (!data || !data->pipeline || data->state < GST_STATE_PAUSED)
+ return TRUE;
+
+ /* If we didn't know it yet, query the stream duration */
+ if (!GST_CLOCK_TIME_IS_VALID (data->duration)) {
+ if (!gst_element_query_duration (data->pipeline, &fmt, &data->duration)) {
+ GST_WARNING ("Could not query current duration");
+ }
+ }
+
+ if (gst_element_query_position (data->pipeline, &fmt, &position)) {
+ /* Java expects these values in milliseconds, and GStreamer provides nanoseconds */
+ set_current_ui_position (position / GST_MSECOND, data->duration / GST_MSECOND, data);
+ }
+ return TRUE;
+}
+
+/* Forward declaration for the delayed seek callback */
+static gboolean delayed_seek_cb (CustomData *data);
+
+/* Perform seek, if we are not too close to the previous seek. Otherwise, schedule the seek for
+ * some time in the future. */
+static void execute_seek (gint64 desired_position, CustomData *data) {
+ gint64 diff;
+
+ if (desired_position == GST_CLOCK_TIME_NONE)
+ return;
+
+ diff = gst_util_get_timestamp () - data->last_seek_time;
+
+ if (GST_CLOCK_TIME_IS_VALID (data->last_seek_time) && diff < SEEK_MIN_DELAY) {
+ /* The previous seek was too close, delay this one */
+ GSource *timeout_source;
+
+ if (data->desired_position == GST_CLOCK_TIME_NONE) {
+ /* There was no previous seek scheduled. Setup a timer for some time in the future */
+ timeout_source = g_timeout_source_new ((SEEK_MIN_DELAY - diff) / GST_MSECOND);
+ g_source_set_callback (timeout_source, (GSourceFunc)delayed_seek_cb, data, NULL);
+ g_source_attach (timeout_source, data->context);
+ g_source_unref (timeout_source);
+ }
+ /* Update the desired seek position. If multiple requests are received before it is time
+ * to perform a seek, only the last one is remembered. */
+ data->desired_position = desired_position;
+ GST_DEBUG ("Throttling seek to %" GST_TIME_FORMAT ", will be in %" GST_TIME_FORMAT,
+ GST_TIME_ARGS (desired_position), GST_TIME_ARGS (SEEK_MIN_DELAY - diff));
+ } else {
+ /* Perform the seek now */
+ GST_DEBUG ("Seeking to %" GST_TIME_FORMAT, GST_TIME_ARGS (desired_position));
+ data->last_seek_time = gst_util_get_timestamp ();
+ gst_element_seek_simple (data->pipeline, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT, desired_position);
+ data->desired_position = GST_CLOCK_TIME_NONE;
+ }
+}
+
+/* Delayed seek callback. This gets called by the timer setup in the above function. */
+static gboolean delayed_seek_cb (CustomData *data) {
+ GST_DEBUG ("Doing delayed seek to %" GST_TIME_FORMAT, GST_TIME_ARGS (data->desired_position));
+ execute_seek (data->desired_position, data);
+ return FALSE;
+}
+
+/* Retrieve errors from the bus and show them on the UI */
+static void error_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ GError *err;
+ gchar *debug_info;
+ gchar *message_string;
+
+ gst_message_parse_error (msg, &err, &debug_info);
+ message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);
+ g_clear_error (&err);
+ g_free (debug_info);
+ set_ui_message (message_string, data);
+ g_free (message_string);
+ data->target_state = GST_STATE_NULL;
+ gst_element_set_state (data->pipeline, GST_STATE_NULL);
+}
+
+/* Called when the End Of the Stream is reached. Just move to the beginning of the media and pause. */
+static void eos_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ data->target_state = GST_STATE_PAUSED;
+ data->is_live = (gst_element_set_state (data->pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_NO_PREROLL);
+ execute_seek (0, data);
+}
+
+/* Called when the duration of the media changes. Just mark it as unknown, so we re-query it in the next UI refresh. */
+static void duration_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ data->duration = GST_CLOCK_TIME_NONE;
+}
+
+/* Called when buffering messages are received. We inform the UI about the current buffering level and
+ * keep the pipeline paused until 100% buffering is reached. At that point, set the desired state. */
+static void buffering_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ gint percent;
+
+ if (data->is_live)
+ return;
+
+ gst_message_parse_buffering (msg, &percent);
+ if (percent < 100 && data->target_state >= GST_STATE_PAUSED) {
+ gchar * message_string = g_strdup_printf ("Buffering %d%%", percent);
+ gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
+ set_ui_message (message_string, data);
+ g_free (message_string);
+ } else if (data->target_state >= GST_STATE_PLAYING) {
+ gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
+ } else if (data->target_state >= GST_STATE_PAUSED) {
+ set_ui_message ("Buffering complete", data);
+ }
+}
+
+/* Called when the clock is lost */
+static void clock_lost_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ if (data->target_state >= GST_STATE_PLAYING) {
+ gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
+ gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
+ }
+}
+
+/* Retrieve the video sink's Caps and tell the application about the media size */
+static void check_media_size (CustomData *data) {
+ JNIEnv *env = get_jni_env ();
+ GstElement *video_sink;
+ GstPad *video_sink_pad;
+ GstCaps *caps;
+ GstVideoFormat fmt;
+ int width;
+ int height;
+
+ /* Retrieve the Caps at the entrance of the video sink */
+ g_object_get (data->pipeline, "video-sink", &video_sink, NULL);
+ video_sink_pad = gst_element_get_static_pad (video_sink, "sink");
+ caps = gst_pad_get_negotiated_caps (video_sink_pad);
+
+ if (gst_video_format_parse_caps(caps, &fmt, &width, &height)) {
+ int par_n, par_d;
+ if (gst_video_parse_caps_pixel_aspect_ratio (caps, &par_n, &par_d)) {
+ width = width * par_n / par_d;
+ }
+ GST_DEBUG ("Media size is %dx%d, notifying application", width, height);
+
+ (*env)->CallVoidMethod (env, data->app, on_media_size_changed_method_id, (jint)width, (jint)height);
+ if ((*env)->ExceptionCheck (env)) {
+ GST_ERROR ("Failed to call Java method");
+ (*env)->ExceptionClear (env);
+ }
+ }
+
+ gst_caps_unref(caps);
+ gst_object_unref (video_sink_pad);
+ gst_object_unref(video_sink);
+}
+
+/* Notify UI about pipeline state changes */
+static void state_changed_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ /* Only pay attention to messages coming from the pipeline, not its children */
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->pipeline)) {
+ data->state = new_state;
+ gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));
+ set_ui_message(message, data);
+ g_free (message);
+
+ /* The Ready to Paused state change is particularly interesting: */
+ if (old_state == GST_STATE_READY && new_state == GST_STATE_PAUSED) {
+ /* By now the sink already knows the media size */
+ check_media_size(data);
+
+ /* If there was a scheduled seek, perform it now that we have moved to the Paused state */
+ if (GST_CLOCK_TIME_IS_VALID (data->desired_position))
+ execute_seek (data->desired_position, data);
+ }
+ }
+}
+
+/* Check if all conditions are met to report GStreamer as initialized.
+ * These conditions will change depending on the application */
+static void check_initialization_complete (CustomData *data) {
+ JNIEnv *env = get_jni_env ();
+ if (!data->initialized && data->native_window && data->main_loop) {
+ GST_DEBUG ("Initialization complete, notifying application. native_window:%p main_loop:%p", data->native_window, data->main_loop);
+
+ /* The main loop is running and we received a native window, inform the sink about it */
+ gst_x_overlay_set_window_handle (GST_X_OVERLAY (data->pipeline), (guintptr)data->native_window);
+
+ (*env)->CallVoidMethod (env, data->app, on_gstreamer_initialized_method_id);
+ if ((*env)->ExceptionCheck (env)) {
+ GST_ERROR ("Failed to call Java method");
+ (*env)->ExceptionClear (env);
+ }
+ data->initialized = TRUE;
+ }
+}
+
+/* Main method for the native code. This is executed on its own thread. */
+static void *app_function (void *userdata) {
+ JavaVMAttachArgs args;
+ GstBus *bus;
+ CustomData *data = (CustomData *)userdata;
+ GSource *timeout_source;
+ GSource *bus_source;
+ GError *error = NULL;
+ guint flags;
+
+ GST_DEBUG ("Creating pipeline in CustomData at %p", data);
+
+ /* Create our own GLib Main Context and make it the default one */
+ data->context = g_main_context_new ();
+ g_main_context_push_thread_default(data->context);
+
+ /* Build pipeline */
+ data->pipeline = gst_parse_launch("playbin", &error);
+ if (error) {
+ gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
+ g_clear_error (&error);
+ set_ui_message(message, data);
+ g_free (message);
+ return NULL;
+ }
+
+ /* Disable subtitles */
+ g_object_get (data->pipeline, "flags", &flags, NULL);
+ flags &= ~GST_PLAY_FLAG_TEXT;
+ g_object_set (data->pipeline, "flags", flags, NULL);
+
+ /* Set the pipeline to READY, so it can already accept a window handle, if we have one */
+ data->target_state = GST_STATE_READY;
+ gst_element_set_state(data->pipeline, GST_STATE_READY);
+
+ /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
+ bus = gst_element_get_bus (data->pipeline);
+ bus_source = gst_bus_create_watch (bus);
+ g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
+ g_source_attach (bus_source, data->context);
+ g_source_unref (bus_source);
+ g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, data);
+ g_signal_connect (G_OBJECT (bus), "message::eos", (GCallback)eos_cb, data);
+ g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, data);
+ g_signal_connect (G_OBJECT (bus), "message::duration", (GCallback)duration_cb, data);
+ g_signal_connect (G_OBJECT (bus), "message::buffering", (GCallback)buffering_cb, data);
+ g_signal_connect (G_OBJECT (bus), "message::clock-lost", (GCallback)clock_lost_cb, data);
+ gst_object_unref (bus);
+
+ /* Register a function that GLib will call 4 times per second */
+ timeout_source = g_timeout_source_new (250);
+ g_source_set_callback (timeout_source, (GSourceFunc)refresh_ui, data, NULL);
+ g_source_attach (timeout_source, data->context);
+ g_source_unref (timeout_source);
+
+ /* Create a GLib Main Loop and set it to run */
+ GST_DEBUG ("Entering main loop... (CustomData:%p)", data);
+ data->main_loop = g_main_loop_new (data->context, FALSE);
+ check_initialization_complete (data);
+ g_main_loop_run (data->main_loop);
+ GST_DEBUG ("Exited main loop");
+ g_main_loop_unref (data->main_loop);
+ data->main_loop = NULL;
+
+ /* Free resources */
+ g_main_context_pop_thread_default(data->context);
+ g_main_context_unref (data->context);
+ data->target_state = GST_STATE_NULL;
+ gst_element_set_state (data->pipeline, GST_STATE_NULL);
+ gst_object_unref (data->pipeline);
+
+ return NULL;
+}
+
+/*
+ * Java Bindings
+ */
+
+/* Instruct the native code to create its internal data structure, pipeline and thread */
+static void gst_native_init (JNIEnv* env, jobject thiz) {
+ CustomData *data = g_new0 (CustomData, 1);
+ data->desired_position = GST_CLOCK_TIME_NONE;
+ data->last_seek_time = GST_CLOCK_TIME_NONE;
+ SET_CUSTOM_DATA (env, thiz, custom_data_field_id, data);
+ GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-4", 0, "Android tutorial 4");
+ gst_debug_set_threshold_for_name("tutorial-4", GST_LEVEL_DEBUG);
+ GST_DEBUG ("Created CustomData at %p", data);
+ data->app = (*env)->NewGlobalRef (env, thiz);
+ GST_DEBUG ("Created GlobalRef for app object at %p", data->app);
+ pthread_create (&gst_app_thread, NULL, &app_function, data);
+}
+
+/* Quit the main loop, remove the native thread and free resources */
+static void gst_native_finalize (JNIEnv* env, jobject thiz) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ GST_DEBUG ("Quitting main loop...");
+ g_main_loop_quit (data->main_loop);
+ GST_DEBUG ("Waiting for thread to finish...");
+ pthread_join (gst_app_thread, NULL);
+ GST_DEBUG ("Deleting GlobalRef for app object at %p", data->app);
+ (*env)->DeleteGlobalRef (env, data->app);
+ GST_DEBUG ("Freeing CustomData at %p", data);
+ g_free (data);
+ SET_CUSTOM_DATA (env, thiz, custom_data_field_id, NULL);
+ GST_DEBUG ("Done finalizing");
+}
+
+/* Set playbin's URI */
+void gst_native_set_uri (JNIEnv* env, jobject thiz, jstring uri) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data || !data->pipeline) return;
+ const jbyte *char_uri = (*env)->GetStringUTFChars (env, uri, NULL);
+ GST_DEBUG ("Setting URI to %s", char_uri);
+ if (data->target_state >= GST_STATE_READY)
+ gst_element_set_state (data->pipeline, GST_STATE_READY);
+ g_object_set(data->pipeline, "uri", char_uri, NULL);
+ (*env)->ReleaseStringUTFChars (env, uri, char_uri);
+ data->duration = GST_CLOCK_TIME_NONE;
+ data->is_live = (gst_element_set_state (data->pipeline, data->target_state) == GST_STATE_CHANGE_NO_PREROLL);
+}
+
+/* Set pipeline to PLAYING state */
+static void gst_native_play (JNIEnv* env, jobject thiz) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ GST_DEBUG ("Setting state to PLAYING");
+ data->target_state = GST_STATE_PLAYING;
+ data->is_live = (gst_element_set_state (data->pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_NO_PREROLL);
+}
+
+/* Set pipeline to PAUSED state */
+static void gst_native_pause (JNIEnv* env, jobject thiz) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ GST_DEBUG ("Setting state to PAUSED");
+ data->target_state = GST_STATE_PAUSED;
+ data->is_live = (gst_element_set_state (data->pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_NO_PREROLL);
+}
+
+/* Instruct the pipeline to seek to a different position */
+void gst_native_set_position (JNIEnv* env, jobject thiz, int milliseconds) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ gint64 desired_position = (gint64)(milliseconds * GST_MSECOND);
+ if (data->state >= GST_STATE_PAUSED) {
+ execute_seek(desired_position, data);
+ } else {
+ GST_DEBUG ("Scheduling seek to %" GST_TIME_FORMAT " for later", GST_TIME_ARGS (desired_position));
+ data->desired_position = desired_position;
+ }
+}
+
+/* Static class initializer: retrieve method and field IDs */
+static jboolean gst_native_class_init (JNIEnv* env, jclass klass) {
+ custom_data_field_id = (*env)->GetFieldID (env, klass, "native_custom_data", "J");
+ set_message_method_id = (*env)->GetMethodID (env, klass, "setMessage", "(Ljava/lang/String;)V");
+ set_current_position_method_id = (*env)->GetMethodID (env, klass, "setCurrentPosition", "(II)V");
+ on_gstreamer_initialized_method_id = (*env)->GetMethodID (env, klass, "onGStreamerInitialized", "()V");
+ on_media_size_changed_method_id = (*env)->GetMethodID (env, klass, "onMediaSizeChanged", "(II)V");
+
+ if (!custom_data_field_id || !set_message_method_id || !on_gstreamer_initialized_method_id ||
+ !on_media_size_changed_method_id || !set_current_position_method_id) {
+ /* We emit this message through the Android log instead of the GStreamer log because the later
+ * has not been initialized yet.
+ */
+ __android_log_print (ANDROID_LOG_ERROR, "tutorial-4", "The calling class does not implement all necessary interface methods");
+ return JNI_FALSE;
+ }
+ return JNI_TRUE;
+}
+
+static void gst_native_surface_init (JNIEnv *env, jobject thiz, jobject surface) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ ANativeWindow *new_native_window = ANativeWindow_fromSurface(env, surface);
+ GST_DEBUG ("Received surface %p (native window %p)", surface, new_native_window);
+
+ if (data->native_window) {
+ ANativeWindow_release (data->native_window);
+ if (data->native_window == new_native_window) {
+ GST_DEBUG ("New native window is the same as the previous one", data->native_window);
+ if (data->pipeline) {
+ gst_x_overlay_expose(GST_X_OVERLAY (data->pipeline));
+ gst_x_overlay_expose(GST_X_OVERLAY (data->pipeline));
+ }
+ return;
+ } else {
+ GST_DEBUG ("Released previous native window %p", data->native_window);
+ data->initialized = FALSE;
+ }
+ }
+ data->native_window = new_native_window;
+
+ check_initialization_complete (data);
+}
+
+static void gst_native_surface_finalize (JNIEnv *env, jobject thiz) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ GST_DEBUG ("Releasing Native Window %p", data->native_window);
+
+ if (data->pipeline) {
+ gst_x_overlay_set_window_handle (GST_X_OVERLAY (data->pipeline), (guintptr)NULL);
+ gst_element_set_state (data->pipeline, GST_STATE_READY);
+ }
+
+ ANativeWindow_release (data->native_window);
+ data->native_window = NULL;
+ data->initialized = FALSE;
+}
+
+/* List of implemented native methods */
+static JNINativeMethod native_methods[] = {
+ { "nativeInit", "()V", (void *) gst_native_init},
+ { "nativeFinalize", "()V", (void *) gst_native_finalize},
+ { "nativeSetUri", "(Ljava/lang/String;)V", (void *) gst_native_set_uri},
+ { "nativePlay", "()V", (void *) gst_native_play},
+ { "nativePause", "()V", (void *) gst_native_pause},
+ { "nativeSetPosition", "(I)V", (void*) gst_native_set_position},
+ { "nativeSurfaceInit", "(Ljava/lang/Object;)V", (void *) gst_native_surface_init},
+ { "nativeSurfaceFinalize", "()V", (void *) gst_native_surface_finalize},
+ { "nativeClassInit", "()Z", (void *) gst_native_class_init}
+};
+
+/* Library initializer */
+jint JNI_OnLoad(JavaVM *vm, void *reserved) {
+ JNIEnv *env = NULL;
+
+ java_vm = vm;
+
+ if ((*vm)->GetEnv(vm, (void**) &env, JNI_VERSION_1_4) != JNI_OK) {
+ __android_log_print (ANDROID_LOG_ERROR, "tutorial-4", "Could not retrieve JNIEnv");
+ return 0;
+ }
+ jclass klass = (*env)->FindClass (env, "com/gst_sdk_tutorials/tutorial_4/Tutorial4");
+ (*env)->RegisterNatives (env, klass, native_methods, G_N_ELEMENTS(native_methods));
+
+ pthread_key_create (¤t_jni_env, detach_current_thread);
+
+ return JNI_VERSION_1_4;
+}
+```
+
+#### Supporting arbitrary media URIs
+
+Java code will call `gst_native_set_uri()` whenever it wants to change
+the playing URI (in this tutorial the URI never changes, but it could):
+
+``` c
+void gst_native_set_uri (JNIEnv* env, jobject thiz, jstring uri) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data || !data->pipeline) return;
+ const jbyte *char_uri = (*env)->GetStringUTFChars (env, uri, NULL);
+ GST_DEBUG ("Setting URI to %s", char_uri);
+ if (data->target_state >= GST_STATE_READY)
+ gst_element_set_state (data->pipeline, GST_STATE_READY);
+ g_object_set(data->pipeline, "uri", char_uri, NULL);
+ (*env)->ReleaseStringUTFChars (env, uri, char_uri);
+ data->duration = GST_CLOCK_TIME_NONE;
+ data->is_live = (gst_element_set_state (data->pipeline, data->target_state) == GST_STATE_CHANGE_NO_PREROLL);
+}
+```
+
+We first need to convert between the
+[UTF16](http://en.wikipedia.org/wiki/UTF-16) encoding used by Java and
+the [Modified
+UTF8](http://en.wikipedia.org/wiki/UTF-8#Modified_UTF-8) used by
+GStreamer with
+[GetStringUTFChars()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#wp17265)
+and
+[ReleaseStringUTFChars()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#wp17294).
+
+`playbin` will only care about URI changes in the READY to PAUSED state
+change, because the new URI might need a completely different playback
+pipeline (think about switching from a local Matroska file to a remote
+OGG file: this would require, at least, different source and demuxing
+elements). Thus, before passing the new URI to `playbin` we set its
+state to READY (if we were in PAUSED or PLAYING).
+
+`playbin`’s URI is exposed as a common GObject property, so we simply
+set it with `g_object_set()`.
+
+We then reset the clip duration, so it is re-queried later, and bring
+the pipeline to the playing state it had before. In this last step, we
+also take note of whether the new URI corresponds to a live source or
+not. Live sources must not use buffering (otherwise latency is
+introduced which is inacceptable for them), so we keep track of this
+information in the `is_live` variable.
+
+#### Reporting media size
+
+Some codecs allow the media size (width and height of the video) to
+change during playback. For simplicity, this tutorial assumes that they
+do not. Therefore, in the READY to PAUSED state change, once the Caps of
+the decoded media are known, we inspect them in `check_media_size()`:
+
+``` c
+static void check_media_size (CustomData *data) {
+ JNIEnv *env = get_jni_env ();
+ GstElement *video_sink;
+ GstPad *video_sink_pad;
+ GstCaps *caps;
+ GstVideoFormat fmt;
+ int width;
+ int height;
+
+ /* Retrieve the Caps at the entrance of the video sink */
+ g_object_get (data->pipeline, "video-sink", &video_sink, NULL);
+ video_sink_pad = gst_element_get_static_pad (video_sink, "sink");
+ caps = gst_pad_get_negotiated_caps (video_sink_pad);
+
+ if (gst_video_format_parse_caps(caps, &fmt, &width, &height)) {
+ int par_n, par_d;
+ if (gst_video_parse_caps_pixel_aspect_ratio (caps, &par_n, &par_d)) {
+ width = width * par_n / par_d;
+ }
+ GST_DEBUG ("Media size is %dx%d, notifying application", width, height);
+
+ (*env)->CallVoidMethod (env, data->app, on_media_size_changed_method_id, (jint)width, (jint)height);
+ if ((*env)->ExceptionCheck (env)) {
+ GST_ERROR ("Failed to call Java method");
+ (*env)->ExceptionClear (env);
+ }
+ }
+
+ gst_caps_unref(caps);
+ gst_object_unref (video_sink_pad);
+ gst_object_unref(video_sink);
+}
+```
+
+We first retrieve the video sink element from the pipeline, using the
+`video-sink` property of `playbin`, and then its sink Pad. The
+negotiated Caps of this Pad, which we recover using
+`gst_pad_get_negotiated_caps()`, are the Caps of the decoded media.
+
+The helper functions `gst_video_format_parse_caps()` and
+`gst_video_parse_caps_pixel_aspect_ratio()` turn the Caps into
+manageable integers, which we pass to Java through
+its `onMediaSizeChanged()` callback.
+
+#### Refreshing the Seek Bar
+
+To keep the UI updated, a GLib timer is installed in the
+`app_function()` that fires 4 times per second (or every 250ms), right
+before entering the main loop:
+
+``` c
+timeout_source = g_timeout_source_new (250);
+g_source_set_callback (timeout_source, (GSourceFunc)refresh_ui, data, NULL);
+g_source_attach (timeout_source, data->context);
+g_source_unref (timeout_source);
+```
+
+Then, in the refresh\_ui method:
+
+``` c
+static gboolean refresh_ui (CustomData *data) {
+ GstFormat fmt = GST_FORMAT_TIME;
+ gint64 current = -1;
+ gint64 position;
+
+ /* We do not want to update anything unless we have a working pipeline in the PAUSED or PLAYING state */
+ if (!data || !data->pipeline || data->state < GST_STATE_PAUSED)
+ return TRUE;
+
+ /* If we didn't know it yet, query the stream duration */
+ if (!GST_CLOCK_TIME_IS_VALID (data->duration)) {
+ if (!gst_element_query_duration (data->pipeline, &fmt, &data->duration)) {
+ GST_WARNING ("Could not query current duration");
+ }
+ }
+
+ if (gst_element_query_position (data->pipeline, &fmt, &position)) {
+ /* Java expects these values in milliseconds, and GStreamer provides nanoseconds */
+ set_current_ui_position (position / GST_MSECOND, data->duration / GST_MSECOND, data);
+ }
+ return TRUE;
+}
+```
+
+If it is unknown, the clip duration is retrieved, as explained in
+[](sdk-basic-tutorial-time-management.md). The current position is
+retrieved next, and the UI is informed of both through its
+`setCurrentPosition()` callback.
+
+Bear in mind that all time-related measures returned by GStreamer are in
+nanoseconds, whereas, for simplicity, we decided to make the UI code
+work in milliseconds.
+
+#### Seeking with the Seek Bar
+
+The Java UI code already takes care of most of the complexity of seeking
+by dragging the thumb of the Seek Bar. From C code, we just need to
+honor the calls to `nativeSetPosition()` and instruct the pipeline to
+jump to the indicated position.
+
+There are, though, a couple of caveats. Firstly, seeks are only possible
+when the pipeline is in the PAUSED or PLAYING state, and we might
+receive seek requests before that happens. Secondly, dragging the Seek
+Bar can generate a very high number of seek requests in a short period
+of time, which is visually useless and will impair responsiveness. Let’s
+see how to overcome these problems.
+
+##### Delayed seeks
+
+In
+`gst_native_set_position()`:
+
+``` c
+void gst_native_set_position (JNIEnv* env, jobject thiz, int milliseconds) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ gint64 desired_position = (gint64)(milliseconds * GST_MSECOND);
+ if (data->state >= GST_STATE_PAUSED) {
+ execute_seek(desired_position, data);
+ } else {
+ GST_DEBUG ("Scheduling seek to %" GST_TIME_FORMAT " for later", GST_TIME_ARGS (desired_position));
+ data->desired_position = desired_position;
+ }
+}
+```
+
+If we are already in the correct state for seeking, execute it right
+away; otherwise, store the desired position in the
+`desired_position` variable. Then, in the
+`state_changed_cb()` callback:
+
+``` c
+if (old_state == GST_STATE_READY && new_state == GST_STATE_PAUSED) {
+ /* By now the sink already knows the media size */
+ check_media_size(data);
+
+ /* If there was a scheduled seek, perform it now that we have moved to the Paused state */
+ if (GST_CLOCK_TIME_IS_VALID (data->desired_position))
+ execute_seek (data->desired_position, data);
+ }
+}
+```
+
+Once the pipeline moves from the READY to the PAUSED state, we check if
+there is a pending seek operation and execute it. The
+`desired_position` variable is reset inside `execute_seek()`.
+
+##### Seek throttling
+
+A seek is potentially a lengthy operation. The demuxer (the element
+typically in charge of seeking) needs to estimate the appropriate byte
+offset inside the media file that corresponds to the time position to
+jump to. Then, it needs to start decoding from that point until the
+desired position is reached. If the initial estimate is accurate, this
+will not take long, but, on some container formats, or when indexing
+information is missing, it can take up to several seconds.
+
+If a demuxer is in the process of performing a seek and receives a
+second one, it is up to it to finish the first one, start the second one
+or abort both, which is a bad thing. A simple method to avoid this issue
+is *throttling*, which means that we will only allow one seek every half
+a second (for example): after performing a seek, only the last seek
+request received during the next 500ms is stored, and will be honored
+once this period elapses.
+
+To achieve this, all seek requests are routed through the
+`execute_seek()` method:
+
+``` c
+static void execute_seek (gint64 desired_position, CustomData *data) {
+ gint64 diff;
+
+ if (desired_position == GST_CLOCK_TIME_NONE)
+ return;
+
+ diff = gst_util_get_timestamp () - data->last_seek_time;
+
+ if (GST_CLOCK_TIME_IS_VALID (data->last_seek_time) && diff < SEEK_MIN_DELAY) {
+ /* The previous seek was too close, delay this one */
+ GSource *timeout_source;
+
+ if (data->desired_position == GST_CLOCK_TIME_NONE) {
+ /* There was no previous seek scheduled. Setup a timer for some time in the future */
+ timeout_source = g_timeout_source_new ((SEEK_MIN_DELAY - diff) / GST_MSECOND);
+ g_source_set_callback (timeout_source, (GSourceFunc)delayed_seek_cb, data, NULL);
+ g_source_attach (timeout_source, data->context);
+ g_source_unref (timeout_source);
+ }
+ /* Update the desired seek position. If multiple requests are received before it is time
+ * to perform a seek, only the last one is remembered. */
+ data->desired_position = desired_position;
+ GST_DEBUG ("Throttling seek to %" GST_TIME_FORMAT ", will be in %" GST_TIME_FORMAT,
+ GST_TIME_ARGS (desired_position), GST_TIME_ARGS (SEEK_MIN_DELAY - diff));
+ } else {
+ /* Perform the seek now */
+ GST_DEBUG ("Seeking to %" GST_TIME_FORMAT, GST_TIME_ARGS (desired_position));
+ data->last_seek_time = gst_util_get_timestamp ();
+ gst_element_seek_simple (data->pipeline, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT, desired_position);
+ data->desired_position = GST_CLOCK_TIME_NONE;
+ }
+}
+```
+
+The time at which the last seek was performed is stored in the
+`last_seek_time` variable. This is wall clock time, not to be confused
+with the stream time carried in the media time stamps, and is obtained
+with `gst_util_get_timestamp()`.
+
+If enough time has passed since the last seek operation, the new one is
+directly executed and `last_seek_time` is updated. Otherwise, the new
+seek is scheduled for later. If there is no previously scheduled seek, a
+one-shot timer is setup to trigger 500ms after the last seek operation.
+If another seek was already scheduled, its desired position is simply
+updated with the new one.
+
+The one-shot timer calls `delayed_seek_cb()`, which simply calls
+`execute_seek()` again.
+
+> ![information]
+> Ideally, `execute_seek()` will now find that enough time has indeed passed since the last seek and the scheduled one will proceed. It might happen, though, that after 500ms of the previous seek, and before the timer wakes up, yet another seek comes through and is executed. `delayed_seek_cb()` needs to check for this condition to avoid performing two very close seeks, and therefore calls `execute_seek()` instead of performing it itself.
+>
+> This is not a complete solution: the scheduled seek will still be executed, even though a more-recent seek has already been executed that should have cancelled it. However, it is a good tradeoff between functionality and simplicity.
+
+#### Network resilience
+
+[](sdk-basic-tutorial-streaming.md) has already
+shown how to adapt to the variable nature of the network bandwidth by
+using buffering. The same procedure is used here, by listening to the
+buffering
+messages:
+
+``` c
+g_signal_connect (G_OBJECT (bus), "message::buffering", (GCallback)buffering_cb, data);
+```
+
+And pausing the pipeline until buffering is complete (unless this is a
+live
+source):
+
+``` c
+static void buffering_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ gint percent;
+
+ if (data->is_live)
+ return;
+
+ gst_message_parse_buffering (msg, &percent);
+ if (percent < 100 && data->target_state >= GST_STATE_PAUSED) {
+ gchar * message_string = g_strdup_printf ("Buffering %d%%", percent);
+ gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
+ set_ui_message (message_string, data);
+ g_free (message_string);
+ } else if (data->target_state >= GST_STATE_PLAYING) {
+ gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
+ } else if (data->target_state >= GST_STATE_PAUSED) {
+ set_ui_message ("Buffering complete", data);
+ }
+}
+```
+
+`target_state` is the state in which we have been instructed to set the
+pipeline, which might be different to the current state, because
+buffering forces us to go to PAUSED. Once buffering is complete we set
+the pipeline to the `target_state`.
+
+### A basic media player \[Android.mk\]
+
+The only line worth mentioning in the makefile
+is `GSTREAMER_PLUGINS`:
+
+**jni/Android.mk**
+
+```
+GSTREAMER_PLUGINS := $(GSTREAMER_PLUGINS_CORE) $(GSTREAMER_PLUGINS_PLAYBACK) $(GSTREAMER_PLUGINS_CODECS) $(GSTREAMER_PLUGINS_NET) $(GSTREAMER_PLUGINS_SYS)
+```
+
+In which all plugins required for playback are loaded, because it is not
+known at build time what would be needed for an unspecified URI (again,
+in this tutorial the URI does not change, but it will in the next one).
+
+### Conclusion
+
+This tutorial has shown how to embed a `playbin` pipeline into an
+Android application. This, effectively, turns such application into a
+basic media player, capable of streaming and decoding all the formats
+GStreamer understands. More particularly, it has shown:
+
+ - How to keep the User Interface regularly updated by using a timer,
+ querying the pipeline position and calling a UI code method.
+ - How to implement a Seek Bar which follows the current position and
+ transforms thumb motion into reliable seek events.
+ - How to report the media size to adapt the display surface, by
+ reading the sink Caps at the appropriate moment and telling the UI
+ about it.
+
+The next tutorial adds the missing bits to turn the application built
+here into an acceptable Android media player.
+
+As usual, it has been a pleasure having you here, and see you soon!
+
+ [screenshot]: images/sdk-android-tutorial-media-player-screenshot.png
+ [information]: images/icons/emoticons/information.png
\ No newline at end of file
--- /dev/null
+# Android tutorial 3: Video
+
+### Goal
+
+![screenshot]
+
+Except for [](sdk-basic-tutorial-toolkit-integration.md),
+which embedded a video window on a GTK application, all tutorials so far
+relied on GStreamer video sinks to create a window to display their
+contents. The video sink on Android is not capable of creating its own
+window, so a drawing surface always needs to be provided. This tutorial
+shows:
+
+ - How to allocate a drawing surface on the Android layout and pass it
+ to GStreamer
+ - How to keep GStreamer posted on changes to the surface
+
+### Introduction
+
+Since Android does not provide a windowing system, a GStreamer video
+sink cannot create pop-up windows as it would do on a Desktop platform.
+Fortunately, the `VideoOverlay` interface allows providing video sinks with
+an already created window onto which they can draw, as we have seen in
+[](sdk-basic-tutorial-toolkit-integration.md).
+
+In this tutorial, a
+[SurfaceView](http://developer.android.com/reference/android/view/SurfaceView.html)
+widget (actually, a subclass of it) is placed on the main layout. When
+Android informs the application that a surface has been created for this
+widget, we pass it to the C code which stores it. The
+`check_initialization_complete()` method explained in the previous
+tutorial is extended so that GStreamer is not considered initialized
+until a main loop is running and a drawing surface has been received.
+
+### A video surface on Android \[Java code\]
+
+**src/org/freedesktop/gstreamer/tutorials/tutorial\_3/Tutorial3.java**
+
+``` java
+package org.freedesktop.gstreamer.tutorials.tutorial_3;
+
+import android.app.Activity;
+import android.os.Bundle;
+import android.util.Log;
+import android.view.SurfaceHolder;
+import android.view.SurfaceView;
+import android.view.View;
+import android.view.View.OnClickListener;
+import android.widget.ImageButton;
+import android.widget.TextView;
+import android.widget.Toast;
+
+import org.freedesktop.gstreamer.GStreamer;
+
+public class Tutorial3 extends Activity implements SurfaceHolder.Callback {
+ private native void nativeInit(); // Initialize native code, build pipeline, etc
+ private native void nativeFinalize(); // Destroy pipeline and shutdown native code
+ private native void nativePlay(); // Set pipeline to PLAYING
+ private native void nativePause(); // Set pipeline to PAUSED
+ private static native boolean nativeClassInit(); // Initialize native class: cache Method IDs for callbacks
+ private native void nativeSurfaceInit(Object surface);
+ private native void nativeSurfaceFinalize();
+ private long native_custom_data; // Native code will use this to keep private data
+
+ private boolean is_playing_desired; // Whether the user asked to go to PLAYING
+
+ // Called when the activity is first created.
+ @Override
+ public void onCreate(Bundle savedInstanceState)
+ {
+ super.onCreate(savedInstanceState);
+
+ // Initialize GStreamer and warn if it fails
+ try {
+ GStreamer.init(this);
+ } catch (Exception e) {
+ Toast.makeText(this, e.getMessage(), Toast.LENGTH_LONG).show();
+ finish();
+ return;
+ }
+
+ setContentView(R.layout.main);
+
+ ImageButton play = (ImageButton) this.findViewById(R.id.button_play);
+ play.setOnClickListener(new OnClickListener() {
+ public void onClick(View v) {
+ is_playing_desired = true;
+ nativePlay();
+ }
+ });
+
+ ImageButton pause = (ImageButton) this.findViewById(R.id.button_stop);
+ pause.setOnClickListener(new OnClickListener() {
+ public void onClick(View v) {
+ is_playing_desired = false;
+ nativePause();
+ }
+ });
+
+ SurfaceView sv = (SurfaceView) this.findViewById(R.id.surface_video);
+ SurfaceHolder sh = sv.getHolder();
+ sh.addCallback(this);
+
+ if (savedInstanceState != null) {
+ is_playing_desired = savedInstanceState.getBoolean("playing");
+ Log.i ("GStreamer", "Activity created. Saved state is playing:" + is_playing_desired);
+ } else {
+ is_playing_desired = false;
+ Log.i ("GStreamer", "Activity created. There is no saved state, playing: false");
+ }
+
+ // Start with disabled buttons, until native code is initialized
+ this.findViewById(R.id.button_play).setEnabled(false);
+ this.findViewById(R.id.button_stop).setEnabled(false);
+
+ nativeInit();
+ }
+
+ protected void onSaveInstanceState (Bundle outState) {
+ Log.d ("GStreamer", "Saving state, playing:" + is_playing_desired);
+ outState.putBoolean("playing", is_playing_desired);
+ }
+
+ protected void onDestroy() {
+ nativeFinalize();
+ super.onDestroy();
+ }
+
+ // Called from native code. This sets the content of the TextView from the UI thread.
+ private void setMessage(final String message) {
+ final TextView tv = (TextView) this.findViewById(R.id.textview_message);
+ runOnUiThread (new Runnable() {
+ public void run() {
+ tv.setText(message);
+ }
+ });
+ }
+
+ // Called from native code. Native code calls this once it has created its pipeline and
+ // the main loop is running, so it is ready to accept commands.
+ private void onGStreamerInitialized () {
+ Log.i ("GStreamer", "Gst initialized. Restoring state, playing:" + is_playing_desired);
+ // Restore previous playing state
+ if (is_playing_desired) {
+ nativePlay();
+ } else {
+ nativePause();
+ }
+
+ // Re-enable buttons, now that GStreamer is initialized
+ final Activity activity = this;
+ runOnUiThread(new Runnable() {
+ public void run() {
+ activity.findViewById(R.id.button_play).setEnabled(true);
+ activity.findViewById(R.id.button_stop).setEnabled(true);
+ }
+ });
+ }
+
+ static {
+ System.loadLibrary("gstreamer_android");
+ System.loadLibrary("tutorial-3");
+ nativeClassInit();
+ }
+
+ public void surfaceChanged(SurfaceHolder holder, int format, int width,
+ int height) {
+ Log.d("GStreamer", "Surface changed to format " + format + " width "
+ + width + " height " + height);
+ nativeSurfaceInit (holder.getSurface());
+ }
+
+ public void surfaceCreated(SurfaceHolder holder) {
+ Log.d("GStreamer", "Surface created: " + holder.getSurface());
+ }
+
+ public void surfaceDestroyed(SurfaceHolder holder) {
+ Log.d("GStreamer", "Surface destroyed");
+ nativeSurfaceFinalize ();
+ }
+
+}
+```
+
+This tutorial continues where the previous one left, adding a video
+surface to the layout and changing the GStreamer pipeline to produce
+video instead of audio. Only the parts of the code that are new will be
+discussed.
+
+``` java
+private native void nativeSurfaceInit(Object surface);
+private native void nativeSurfaceFinalize();
+```
+
+Two new entry points to the C code are defined,
+`nativeSurfaceInit()` and `nativeSurfaceFinalize()`, which we will call
+when the video surface becomes available and when it is about to be
+destroyed, respectively.
+
+``` java
+SurfaceView sv = (SurfaceView) this.findViewById(R.id.surface_video);
+SurfaceHolder sh = sv.getHolder();
+sh.addCallback(this);
+```
+
+In `onCreate()`, we retrieve the
+[SurfaceView](http://developer.android.com/reference/android/view/SurfaceView.html),
+and then register ourselves to receive notifications about the surface
+state through the
+[SurfaceHolder](http://developer.android.com/reference/android/view/SurfaceHolder.html)
+interface. This is why we declared this Activity as implementing the
+[SurfaceHolder.Callback](http://developer.android.com/reference/android/view/SurfaceHolder.Callback.html)
+interface in line 16.
+
+``` java
+public void surfaceChanged(SurfaceHolder holder, int format, int width,
+ int height) {
+ Log.d("GStreamer", "Surface changed to format " + format + " width "
+ + width + " height " + height);
+ nativeSurfaceInit (holder.getSurface());
+}
+
+public void surfaceCreated(SurfaceHolder holder) {
+ Log.d("GStreamer", "Surface created: " + holder.getSurface());
+}
+
+public void surfaceDestroyed(SurfaceHolder holder) {
+ Log.d("GStreamer", "Surface destroyed");
+ nativeSurfaceFinalize ();
+}
+```
+
+This interface is composed of the three methods above, which get called
+when the geometry of the surface changes, when the surface is created
+and when it is about to be destroyed. `surfaceChanged()` always gets
+called at least once, right after `surfaceCreated()`, so we will use it
+to notify GStreamer about the new surface. We use
+`surfaceDestroyed()` to tell GStreamer to stop using this surface.
+
+Let’s review the C code to see what these functions do.
+
+### A video surface on Android \[C code\]
+
+**jni/tutorial-3.c**
+
+``` c
+#include <string.h>
+#include <stdint.h>
+#include <jni.h>
+#include <android/log.h>
+#include <android/native_window.h>
+#include <android/native_window_jni.h>
+#include <gst/gst.h>
+#include <gst/video/video.h>
+#include <pthread.h>
+
+GST_DEBUG_CATEGORY_STATIC (debug_category);
+#define GST_CAT_DEFAULT debug_category
+
+/*
+ * These macros provide a way to store the native pointer to CustomData, which might be 32 or 64 bits, into
+ * a jlong, which is always 64 bits, without warnings.
+ */
+#if GLIB_SIZEOF_VOID_P == 8
+## define GET_CUSTOM_DATA(env, thiz, fieldID) (CustomData *)(*env)->GetLongField (env, thiz, fieldID)
+## define SET_CUSTOM_DATA(env, thiz, fieldID, data) (*env)->SetLongField (env, thiz, fieldID, (jlong)data)
+#else
+## define GET_CUSTOM_DATA(env, thiz, fieldID) (CustomData *)(jint)(*env)->GetLongField (env, thiz, fieldID)
+## define SET_CUSTOM_DATA(env, thiz, fieldID, data) (*env)->SetLongField (env, thiz, fieldID, (jlong)(jint)data)
+#endif
+
+/* Structure to contain all our information, so we can pass it to callbacks */
+typedef struct _CustomData {
+ jobject app; /* Application instance, used to call its methods. A global reference is kept. */
+ GstElement *pipeline; /* The running pipeline */
+ GMainContext *context; /* GLib context used to run the main loop */
+ GMainLoop *main_loop; /* GLib main loop */
+ gboolean initialized; /* To avoid informing the UI multiple times about the initialization */
+ GstElement *video_sink; /* The video sink element which receives VideoOverlay commands */
+ ANativeWindow *native_window; /* The Android native window where video will be rendered */
+} CustomData;
+
+/* These global variables cache values which are not changing during execution */
+static pthread_t gst_app_thread;
+static pthread_key_t current_jni_env;
+static JavaVM *java_vm;
+static jfieldID custom_data_field_id;
+static jmethodID set_message_method_id;
+static jmethodID on_gstreamer_initialized_method_id;
+
+/*
+ * Private methods
+ */
+
+/* Register this thread with the VM */
+static JNIEnv *attach_current_thread (void) {
+ JNIEnv *env;
+ JavaVMAttachArgs args;
+
+ GST_DEBUG ("Attaching thread %p", g_thread_self ());
+ args.version = JNI_VERSION_1_4;
+ args.name = NULL;
+ args.group = NULL;
+
+ if ((*java_vm)->AttachCurrentThread (java_vm, &env, &args) < 0) {
+ GST_ERROR ("Failed to attach current thread");
+ return NULL;
+ }
+
+ return env;
+}
+
+/* Unregister this thread from the VM */
+static void detach_current_thread (void *env) {
+ GST_DEBUG ("Detaching thread %p", g_thread_self ());
+ (*java_vm)->DetachCurrentThread (java_vm);
+}
+
+/* Retrieve the JNI environment for this thread */
+static JNIEnv *get_jni_env (void) {
+ JNIEnv *env;
+
+ if ((env = pthread_getspecific (current_jni_env)) == NULL) {
+ env = attach_current_thread ();
+ pthread_setspecific (current_jni_env, env);
+ }
+
+ return env;
+}
+
+/* Change the content of the UI's TextView */
+static void set_ui_message (const gchar *message, CustomData *data) {
+ JNIEnv *env = get_jni_env ();
+ GST_DEBUG ("Setting message to: %s", message);
+ jstring jmessage = (*env)->NewStringUTF(env, message);
+ (*env)->CallVoidMethod (env, data->app, set_message_method_id, jmessage);
+ if ((*env)->ExceptionCheck (env)) {
+ GST_ERROR ("Failed to call Java method");
+ (*env)->ExceptionClear (env);
+ }
+ (*env)->DeleteLocalRef (env, jmessage);
+}
+
+/* Retrieve errors from the bus and show them on the UI */
+static void error_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ GError *err;
+ gchar *debug_info;
+ gchar *message_string;
+
+ gst_message_parse_error (msg, &err, &debug_info);
+ message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);
+ g_clear_error (&err);
+ g_free (debug_info);
+ set_ui_message (message_string, data);
+ g_free (message_string);
+ gst_element_set_state (data->pipeline, GST_STATE_NULL);
+}
+
+/* Notify UI about pipeline state changes */
+static void state_changed_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ /* Only pay attention to messages coming from the pipeline, not its children */
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->pipeline)) {
+ gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));
+ set_ui_message(message, data);
+ g_free (message);
+ }
+}
+
+/* Check if all conditions are met to report GStreamer as initialized.
+ * These conditions will change depending on the application */
+static void check_initialization_complete (CustomData *data) {
+ JNIEnv *env = get_jni_env ();
+ if (!data->initialized && data->native_window && data->main_loop) {
+ GST_DEBUG ("Initialization complete, notifying application. native_window:%p main_loop:%p", data->native_window, data->main_loop);
+
+ /* The main loop is running and we received a native window, inform the sink about it */
+ gst_video_overlay_set_window_handle (GST_VIDEO_OVERLAY (data->video_sink), (guintptr)data->native_window);
+
+ (*env)->CallVoidMethod (env, data->app, on_gstreamer_initialized_method_id);
+ if ((*env)->ExceptionCheck (env)) {
+ GST_ERROR ("Failed to call Java method");
+ (*env)->ExceptionClear (env);
+ }
+ data->initialized = TRUE;
+ }
+}
+
+/* Main method for the native code. This is executed on its own thread. */
+static void *app_function (void *userdata) {
+ JavaVMAttachArgs args;
+ GstBus *bus;
+ CustomData *data = (CustomData *)userdata;
+ GSource *bus_source;
+ GError *error = NULL;
+
+ GST_DEBUG ("Creating pipeline in CustomData at %p", data);
+
+ /* Create our own GLib Main Context and make it the default one */
+ data->context = g_main_context_new ();
+ g_main_context_push_thread_default(data->context);
+
+ /* Build pipeline */
+ data->pipeline = gst_parse_launch("videotestsrc ! warptv ! videoconvert ! autovideosink", &error);
+ if (error) {
+ gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
+ g_clear_error (&error);
+ set_ui_message(message, data);
+ g_free (message);
+ return NULL;
+ }
+
+ /* Set the pipeline to READY, so it can already accept a window handle, if we have one */
+ gst_element_set_state(data->pipeline, GST_STATE_READY);
+
+ data->video_sink = gst_bin_get_by_interface(GST_BIN(data->pipeline), GST_TYPE_VIDEO_OVERLAY);
+ if (!data->video_sink) {
+ GST_ERROR ("Could not retrieve video sink");
+ return NULL;
+ }
+
+ /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
+ bus = gst_element_get_bus (data->pipeline);
+ bus_source = gst_bus_create_watch (bus);
+ g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
+ g_source_attach (bus_source, data->context);
+ g_source_unref (bus_source);
+ g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, data);
+ g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, data);
+ gst_object_unref (bus);
+
+ /* Create a GLib Main Loop and set it to run */
+ GST_DEBUG ("Entering main loop... (CustomData:%p)", data);
+ data->main_loop = g_main_loop_new (data->context, FALSE);
+ check_initialization_complete (data);
+ g_main_loop_run (data->main_loop);
+ GST_DEBUG ("Exited main loop");
+ g_main_loop_unref (data->main_loop);
+ data->main_loop = NULL;
+
+ /* Free resources */
+ g_main_context_pop_thread_default(data->context);
+ g_main_context_unref (data->context);
+ gst_element_set_state (data->pipeline, GST_STATE_NULL);
+ gst_object_unref (data->video_sink);
+ gst_object_unref (data->pipeline);
+
+ return NULL;
+}
+
+/*
+ * Java Bindings
+ */
+
+/* Instruct the native code to create its internal data structure, pipeline and thread */
+static void gst_native_init (JNIEnv* env, jobject thiz) {
+ CustomData *data = g_new0 (CustomData, 1);
+ SET_CUSTOM_DATA (env, thiz, custom_data_field_id, data);
+ GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-3", 0, "Android tutorial 3");
+ gst_debug_set_threshold_for_name("tutorial-3", GST_LEVEL_DEBUG);
+ GST_DEBUG ("Created CustomData at %p", data);
+ data->app = (*env)->NewGlobalRef (env, thiz);
+ GST_DEBUG ("Created GlobalRef for app object at %p", data->app);
+ pthread_create (&gst_app_thread, NULL, &app_function, data);
+}
+
+/* Quit the main loop, remove the native thread and free resources */
+static void gst_native_finalize (JNIEnv* env, jobject thiz) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ GST_DEBUG ("Quitting main loop...");
+ g_main_loop_quit (data->main_loop);
+ GST_DEBUG ("Waiting for thread to finish...");
+ pthread_join (gst_app_thread, NULL);
+ GST_DEBUG ("Deleting GlobalRef for app object at %p", data->app);
+ (*env)->DeleteGlobalRef (env, data->app);
+ GST_DEBUG ("Freeing CustomData at %p", data);
+ g_free (data);
+ SET_CUSTOM_DATA (env, thiz, custom_data_field_id, NULL);
+ GST_DEBUG ("Done finalizing");
+}
+
+/* Set pipeline to PLAYING state */
+static void gst_native_play (JNIEnv* env, jobject thiz) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ GST_DEBUG ("Setting state to PLAYING");
+ gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
+}
+
+/* Set pipeline to PAUSED state */
+static void gst_native_pause (JNIEnv* env, jobject thiz) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ GST_DEBUG ("Setting state to PAUSED");
+ gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
+}
+
+/* Static class initializer: retrieve method and field IDs */
+static jboolean gst_native_class_init (JNIEnv* env, jclass klass) {
+ custom_data_field_id = (*env)->GetFieldID (env, klass, "native_custom_data", "J");
+ set_message_method_id = (*env)->GetMethodID (env, klass, "setMessage", "(Ljava/lang/String;)V");
+ on_gstreamer_initialized_method_id = (*env)->GetMethodID (env, klass, "onGStreamerInitialized", "()V");
+
+ if (!custom_data_field_id || !set_message_method_id || !on_gstreamer_initialized_method_id) {
+ /* We emit this message through the Android log instead of the GStreamer log because the later
+ * has not been initialized yet.
+ */
+ __android_log_print (ANDROID_LOG_ERROR, "tutorial-3", "The calling class does not implement all necessary interface methods");
+ return JNI_FALSE;
+ }
+ return JNI_TRUE;
+}
+
+static void gst_native_surface_init (JNIEnv *env, jobject thiz, jobject surface) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ ANativeWindow *new_native_window = ANativeWindow_fromSurface(env, surface);
+ GST_DEBUG ("Received surface %p (native window %p)", surface, new_native_window);
+
+ if (data->native_window) {
+ ANativeWindow_release (data->native_window);
+ if (data->native_window == new_native_window) {
+ GST_DEBUG ("New native window is the same as the previous one", data->native_window);
+ if (data->video_sink) {
+ gst_video_overlay_expose(GST_VIDEO_OVERLAY (data->video_sink));
+ gst_video_overlay_expose(GST_VIDEO_OVERLAY (data->video_sink));
+ }
+ return;
+ } else {
+ GST_DEBUG ("Released previous native window %p", data->native_window);
+ data->initialized = FALSE;
+ }
+ }
+ data->native_window = new_native_window;
+
+ check_initialization_complete (data);
+}
+
+static void gst_native_surface_finalize (JNIEnv *env, jobject thiz) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ GST_DEBUG ("Releasing Native Window %p", data->native_window);
+
+ if (data->video_sink) {
+ gst_video_overlay_set_window_handle (GST_VIDEO_OVERLAY (data->video_sink), (guintptr)NULL);
+ gst_element_set_state (data->pipeline, GST_STATE_READY);
+ }
+
+ ANativeWindow_release (data->native_window);
+ data->native_window = NULL;
+ data->initialized = FALSE;
+}
+
+/* List of implemented native methods */
+static JNINativeMethod native_methods[] = {
+ { "nativeInit", "()V", (void *) gst_native_init},
+ { "nativeFinalize", "()V", (void *) gst_native_finalize},
+ { "nativePlay", "()V", (void *) gst_native_play},
+ { "nativePause", "()V", (void *) gst_native_pause},
+ { "nativeSurfaceInit", "(Ljava/lang/Object;)V", (void *) gst_native_surface_init},
+ { "nativeSurfaceFinalize", "()V", (void *) gst_native_surface_finalize},
+ { "nativeClassInit", "()Z", (void *) gst_native_class_init}
+};
+
+/* Library initializer */
+jint JNI_OnLoad(JavaVM *vm, void *reserved) {
+ JNIEnv *env = NULL;
+
+ java_vm = vm;
+
+ if ((*vm)->GetEnv(vm, (void**) &env, JNI_VERSION_1_4) != JNI_OK) {
+ __android_log_print (ANDROID_LOG_ERROR, "tutorial-3", "Could not retrieve JNIEnv");
+ return 0;
+ }
+ jclass klass = (*env)->FindClass (env, "org/freedesktop/gstreamer/tutorials/tutorial_3/Tutorial3");
+ (*env)->RegisterNatives (env, klass, native_methods, G_N_ELEMENTS(native_methods));
+
+ pthread_key_create (¤t_jni_env, detach_current_thread);
+
+ return JNI_VERSION_1_4;
+}
+```
+
+First, our `CustomData` structure is augmented to keep a pointer to the
+video sink element and the native window
+handle:
+
+``` c
+GstElement *video_sink; /* The video sink element which receives VideoOverlay commands */
+ANativeWindow *native_window; /* The Android native window where video will be rendered */
+```
+
+The `check_initialization_complete()` method is also augmented so that
+it requires a native window before considering GStreamer to be
+initialized:
+
+``` c
+static void check_initialization_complete (CustomData *data) {
+ JNIEnv *env = get_jni_env ();
+ if (!data->initialized && data->native_window && data->main_loop) {
+ GST_DEBUG ("Initialization complete, notifying application. native_window:%p main_loop:%p", data->native_window, data->main_loop);
+
+ /* The main loop is running and we received a native window, inform the sink about it */
+ gst_video_overlay_set_window_handle (GST_VIDEO_OVERLAY (data->video_sink), (guintptr)data->native_window);
+
+ (*env)->CallVoidMethod (env, data->app, on_gstreamer_initialized_method_id);
+ if ((*env)->ExceptionCheck (env)) {
+ GST_ERROR ("Failed to call Java method");
+ (*env)->ExceptionClear (env);
+ }
+ data->initialized = TRUE;
+ }
+}
+```
+
+Also, once the pipeline has been built and a native window has been
+received, we inform the video sink of the window handle to use via the
+`gst_video_overlay_set_window_handle()` method.
+
+The GStreamer pipeline for this tutorial involves a `videotestsrc`, a
+`warptv` psychedelic distorter effect (check out other cool video
+effects in the `GSTREAMER_PLUGINS_EFFECTS` package), and an
+`autovideosink` which will instantiate the adequate video sink for the
+platform:
+
+``` c
+data->pipeline = gst_parse_launch("videotestsrc ! warptv ! videoconvert ! autovideosink ", &error);
+```
+
+Here things start to get more
+interesting:
+
+``` c
+/* Set the pipeline to READY, so it can already accept a window handle, if we have one */
+gst_element_set_state(data->pipeline, GST_STATE_READY);
+
+data->video_sink = gst_bin_get_by_interface(GST_BIN(data->pipeline), GST_TYPE_VIDEO_OVERLAY);
+if (!data->video_sink) {
+ GST_ERROR ("Could not retrieve video sink");
+ return NULL;
+}
+```
+
+We start by setting the pipeline to the READY state. No data flow occurs
+yet, but the `autovideosink` will instantiate the actual sink so we can
+ask for it immediately.
+
+The `gst_bin_get_by_interface()` method will examine the whole pipeline
+and return a pointer to an element which supports the requested
+interface. We are asking for the `VideoOverlay` interface, explained in
+[](sdk-basic-tutorial-toolkit-integration.md),
+which controls how to perform rendering into foreign (non-GStreamer)
+windows. The internal video sink instantiated by `autovideosink` is the
+only element in this pipeline implementing it, so it will be returned.
+
+Now we will implement the two native functions called by the Java code
+when the drawing surface becomes available or is about to be
+destroyed:
+
+``` c
+static void gst_native_surface_init (JNIEnv *env, jobject thiz, jobject surface) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ ANativeWindow *new_native_window = ANativeWindow_fromSurface(env, surface);
+ GST_DEBUG ("Received surface %p (native window %p)", surface, new_native_window);
+
+ if (data->native_window) {
+ ANativeWindow_release (data->native_window);
+ if (data->native_window == new_native_window) {
+ GST_DEBUG ("New native window is the same as the previous one", data->native_window);
+ if (data->video_sink) {
+ gst_video_overlay_expose(GST_VIDEO_OVERLAY (data->video_sink));
+ gst_video_overlay_expose(GST_VIDEO_OVERLAY (data->video_sink));
+ }
+ return;
+ } else {
+ GST_DEBUG ("Released previous native window %p", data->native_window);
+ data->initialized = FALSE;
+ }
+ }
+ data->native_window = new_native_window;
+
+ check_initialization_complete (data);
+}
+```
+
+This method is responsible for providing the video sink with the window
+handle coming from the Java code. We are passed a
+[Surface](http://developer.android.com/reference/android/view/Surface.html)
+object, and we use `ANativeWindow_fromSurface()` to obtain the
+underlying native window pointer. There is no official online
+documentation for the NDK, but fortunately the header files are well
+commented. Native window management functions can be found in
+`$(ANDROID_NDK_ROOT)\platforms\android-9\arch-arm\usr\include\android\native_window.h` and `native_window_jni.h`
+
+If we had already stored a native window, the one we just received can
+either be a new one, or just an update of the one we have. If the
+pointers are the same, we assume the geometry of the surface has
+changed, and simply instruct the video sink to redraw itself, via the
+`gst_video_overlay_expose()` method. The video sink will recover the new
+size from the surface itself, so we do not need to bother about it
+here. We need to call `gst_video_overlay_expose()` twice because of the way
+the surface changes propagate down the OpenGL ES / EGL pipeline (The
+only video sink available for Android in GStreamer uses OpenGL
+ES). By the time we call the first expose, the surface that the sink
+will pick up still contains the old size.
+
+On the other hand, if the pointers are different, we mark GStreamer as
+not being initialized. Next time we call
+`check_initialization_complete()`, the video sink will be informed of
+the new window handle.
+
+We finally store the new window handle and call
+`check_initialization_complete()` to inform the Java code that
+everything is set up, if that is the case.
+
+``` c
+static void gst_native_surface_finalize (JNIEnv *env, jobject thiz) {
+ CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
+ if (!data) return;
+ GST_DEBUG ("Releasing Native Window %p", data->native_window);
+
+ if (data->video_sink) {
+ gst_video_overlay_set_window_handle (GST_VIDEO_OVERLAY (data->video_sink), (guintptr)NULL);
+ gst_element_set_state (data->pipeline, GST_STATE_READY);
+ }
+
+ ANativeWindow_release (data->native_window);
+ data->native_window = NULL;
+ data->initialized = FALSE;
+}
+```
+
+The complementary function, `gst_native_surface_finalize()` is called
+when a surface is about to be destroyed and should not be used anymore.
+Here, we simply instruct the video sink to stop using the window handle
+and set the pipeline to READY so no rendering occurs. We release the
+window pointer we had stored with `ANativeWindow_release()`, and mark
+GStreamer as not being initialized anymore.
+
+And this is all there is to it, regarding the main code. Only a couple
+of details remain, the subclass we made for SurfaceView and the
+`Android.mk` file.
+
+### GStreamerSurfaceView, a convenient SurfaceView wrapper \[Java code\]
+
+By default,
+[SurfaceView](http://developer.android.com/reference/android/view/SurfaceView.html) does
+not have any particular size, so it expands to use all the space the
+layout can give it. While this might be convenient sometimes, it does
+not allow a great deal of control. In particular, when the surface does
+not have the same aspect ratio as the media, the sink will add black
+borders (the known “letterbox” or “pillarbox” effect), which is an
+unnecessary work (and a waste of battery).
+
+The subclass of
+[SurfaceView](http://developer.android.com/reference/android/view/SurfaceView.html) presented
+here overrides the
+[onMeasure()](http://developer.android.com/reference/android/view/SurfaceView.html#onMeasure\(int,%20int\)) method
+to report the actual media size, so the surface can adapt to any layout
+while preserving the media aspect ratio.
+
+Since in this tutorial the media size is known beforehand, it is
+hardcoded in the GStreamerSurfaceView class for simplicity. The next
+tutorial shows how it can be recovered at runtime and passed onto the
+surface.
+
+**src/org/freedesktop/gstreamer/tutorials/tutorial\_3/GStreamerSurfaceView.java**
+
+``` java
+package org.freedesktop.gstreamer.tutorials.tutorial_3;
+
+import android.content.Context;
+import android.util.AttributeSet;
+import android.util.Log;
+import android.view.SurfaceView;
+import android.view.View;
+
+// A simple SurfaceView whose width and height can be set from the outside
+public class GStreamerSurfaceView extends SurfaceView {
+ public int media_width = 320;
+ public int media_height = 240;
+
+ // Mandatory constructors, they do not do much
+ public GStreamerSurfaceView(Context context, AttributeSet attrs,
+ int defStyle) {
+ super(context, attrs, defStyle);
+ }
+
+ public GStreamerSurfaceView(Context context, AttributeSet attrs) {
+ super(context, attrs);
+ }
+
+ public GStreamerSurfaceView (Context context) {
+ super(context);
+ }
+
+ // Called by the layout manager to find out our size and give us some rules.
+ // We will try to maximize our size, and preserve the media's aspect ratio if
+ // we are given the freedom to do so.
+ @Override
+ protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
+ int width = 0, height = 0;
+ int wmode = View.MeasureSpec.getMode(widthMeasureSpec);
+ int hmode = View.MeasureSpec.getMode(heightMeasureSpec);
+ int wsize = View.MeasureSpec.getSize(widthMeasureSpec);
+ int hsize = View.MeasureSpec.getSize(heightMeasureSpec);
+
+ Log.i ("GStreamer", "onMeasure called with " + media_width + "x" + media_height);
+ // Obey width rules
+ switch (wmode) {
+ case View.MeasureSpec.AT_MOST:
+ if (hmode == View.MeasureSpec.EXACTLY) {
+ width = Math.min(hsize * media_width / media_height, wsize);
+ break;
+ }
+ case View.MeasureSpec.EXACTLY:
+ width = wsize;
+ break;
+ case View.MeasureSpec.UNSPECIFIED:
+ width = media_width;
+ }
+
+ // Obey height rules
+ switch (hmode) {
+ case View.MeasureSpec.AT_MOST:
+ if (wmode == View.MeasureSpec.EXACTLY) {
+ height = Math.min(wsize * media_height / media_width, hsize);
+ break;
+ }
+ case View.MeasureSpec.EXACTLY:
+ height = hsize;
+ break;
+ case View.MeasureSpec.UNSPECIFIED:
+ height = media_height;
+ }
+
+ // Finally, calculate best size when both axis are free
+ if (hmode == View.MeasureSpec.AT_MOST && wmode == View.MeasureSpec.AT_MOST) {
+ int correct_height = width * media_height / media_width;
+ int correct_width = height * media_width / media_height;
+
+ if (correct_height < height)
+ height = correct_height;
+ else
+ width = correct_width;
+ }
+
+ // Obey minimum size
+ width = Math.max (getSuggestedMinimumWidth(), width);
+ height = Math.max (getSuggestedMinimumHeight(), height);
+ setMeasuredDimension(width, height);
+ }
+
+}
+```
+
+### A video surface on Android \[Android.mk\]
+
+**/jni/Android.mk**
+
+``` ruby
+LOCAL_PATH := $(call my-dir)
+
+include $(CLEAR_VARS)
+
+LOCAL_MODULE := tutorial-3
+LOCAL_SRC_FILES := tutorial-3.c
+LOCAL_SHARED_LIBRARIES := gstreamer_android
+LOCAL_LDLIBS := -llog -landroid
+include $(BUILD_SHARED_LIBRARY)
+
+ifndef GSTREAMER_ROOT
+ifndef GSTREAMER_ROOT_ANDROID
+$(error GSTREAMER_ROOT_ANDROID is not defined!)
+endif
+GSTREAMER_ROOT := $(GSTREAMER_ROOT_ANDROID)
+endif
+GSTREAMER_NDK_BUILD_PATH := $(GSTREAMER_ROOT)/share/gst-android/ndk-build/
+include $(GSTREAMER_NDK_BUILD_PATH)/plugins.mk
+GSTREAMER_PLUGINS := $(GSTREAMER_PLUGINS_CORE) $(GSTREAMER_PLUGINS_SYS) $(GSTREAMER_PLUGINS_EFFECTS)
+GSTREAMER_EXTRA_DEPS := gstreamer-video-1.0
+include $(GSTREAMER_NDK_BUILD_PATH)/gstreamer.mk
+```
+
+Worth mentioning is the `-landroid` library being used to allow
+interaction with the native windows, and the different plugin
+packages: `GSTREAMER_PLUGINS_SYS` for the system-dependent video sink
+and `GSTREAMER_PLUGINS_EFFECTS` for the `warptv` element. This tutorial
+requires the `gstreamer-video` library to use the
+`VideoOverlay` interface and the video helper methods.
+
+### Conclusion
+
+This tutorial has shown:
+
+ - How to display video on Android using a
+ [SurfaceView](http://developer.android.com/reference/android/view/SurfaceView.html) and
+ the `VideoOverlay` interface.
+ - How to be aware of changes in the surface’s size using
+ [SurfaceView](http://developer.android.com/reference/android/view/SurfaceView.html)’s
+ callbacks.
+ - How to report the media size to the Android layout engine.
+
+The following tutorial plays an actual clip and adds a few more controls
+to this tutorial in order to build a simple media player.
+
+It has been a pleasure having you here, and see you soon\!
+
+
+ [screenshot]: images/sdk-android-tutorial-video-screenshot.png
--- /dev/null
+# Android tutorials
+
+## Welcome to the GStreamer Android tutorials
+
+These tutorials describe Android-specific topics. General GStreamer
+concepts will not be explained in these tutorials, so the
+[](sdk-basic-tutorials.md) should be reviewed first. The reader should
+also be familiar with basic Android programming techniques.
+
+Each Android tutorial builds on top of the previous one and adds
+progressively more functionality, until a working media player
+application is obtained in [](sdk-android-tutorial-a-complete-media-player.md).
+This is the same media player application used to advertise
+GStreamer on Android, and the download link can be found in
+the [](sdk-android-tutorial-a-complete-media-player.md) page.
+
+Make sure to have read the instructions in
+[](sdk-installing-for-android-development.md) before jumping into the
+Android tutorials.
+
+### A note on the documentation
+
+All Java methods, both Android-specific and generic, are documented in
+the [Android reference
+site](http://developer.android.com/reference/packages.html).
+
+Unfortunately, there is no official online documentation for the NDK.
+The header files, though, are well commented. If you installed the
+Android NDK in the `$(ANDROID_NDK_ROOT)` folder, you can find the header
+files
+in `$(ANDROID_NDK_ROOT)\platforms\android-9\arch-arm\usr\include\android`.
--- /dev/null
+# Basic Media Player
+
+## Goal
+
+This tutorial shows how to create a basic media player with
+[Qt](http://qt-project.org/) and
+[QtGStreamer](http://gstreamer.freedesktop.org/data/doc/gstreamer/head/qt-gstreamer/html/index.html).
+It assumes that you are already familiar with the basics of Qt and
+GStreamer. If not, please refer to the other tutorials in this
+documentation.
+
+In particular, you will learn:
+
+ - How to create a basic pipeline
+ - How to create a video output
+ - Updating the GUI based on playback time
+
+## A media player with Qt
+
+These files are located in the qt-gstreamer SDK's `examples/` directory.
+
+Due to the length of these samples, they are initially hidden. Click on
+each file to expand.
+
+CMakeLists.txt
+
+**CMakeLists.txt**
+
+```
+project(qtgst-example-player)
+find_package(QtGStreamer REQUIRED)
+## automoc is now a built-in tool since CMake 2.8.6.
+if (${CMAKE_VERSION} VERSION_LESS "2.8.6")
+ find_package(Automoc4 REQUIRED)
+else()
+ set(CMAKE_AUTOMOC TRUE)
+ macro(automoc4_add_executable)
+ add_executable(${ARGV})
+ endmacro()
+endif()
+include_directories(${QTGSTREAMER_INCLUDES} ${CMAKE_CURRENT_BINARY_DIR} ${QT_QTWIDGETS_INCLUDE_DIRS})
+add_definitions(${QTGSTREAMER_DEFINITIONS})
+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${QTGSTREAMER_FLAGS}")
+set(player_SOURCES main.cpp player.cpp mediaapp.cpp)
+automoc4_add_executable(player ${player_SOURCES})
+target_link_libraries(player ${QTGSTREAMER_UI_LIBRARIES} ${QT_QTOPENGL_LIBRARIES} ${QT_QTWIDGETS_LIBRARIES})
+```
+
+main.cpp
+
+**main.cpp**
+
+``` c
+#include "mediaapp.h"
+#include <QtWidgets/QApplication>
+#include <QGst/Init>
+int main(int argc, char *argv[])
+{
+ QApplication app(argc, argv);
+ QGst::init(&argc, &argv);
+ MediaApp media;
+ media.show();
+ if (argc == 2) {
+ media.openFile(argv[1]);
+ }
+ return app.exec();
+}
+```
+
+mediaapp.h
+
+**mediaapp.h**
+
+``` c
+#ifndef MEDIAAPP_H
+#define MEDIAAPP_H
+#include <QtCore/QTimer>
+#include <QtWidgets/QWidget>
+#include <QtWidgets/QStyle>
+class Player;
+class QBoxLayout;
+class QLabel;
+class QSlider;
+class QToolButton;
+class QTimer;
+class MediaApp : public QWidget
+{
+ Q_OBJECT
+public:
+ MediaApp(QWidget *parent = 0);
+ ~MediaApp();
+ void openFile(const QString & fileName);
+private Q_SLOTS:
+ void open();
+ void toggleFullScreen();
+ void onStateChanged();
+ void onPositionChanged();
+ void setPosition(int position);
+ void showControls(bool show = true);
+ void hideControls() { showControls(false); }
+protected:
+ void mouseMoveEvent(QMouseEvent *event);
+private:
+ QToolButton *initButton(QStyle::StandardPixmap icon, const QString & tip,
+ QObject *dstobj, const char *slot_method, QLayout *layout);
+ void createUI(QBoxLayout *appLayout);
+ QString m_baseDir;
+ Player *m_player;
+ QToolButton *m_openButton;
+ QToolButton *m_fullScreenButton;
+ QToolButton *m_playButton;
+ QToolButton *m_pauseButton;
+ QToolButton *m_stopButton;
+ QSlider *m_positionSlider;
+ QSlider *m_volumeSlider;
+ QLabel *m_positionLabel;
+ QLabel *m_volumeLabel;
+ QTimer m_fullScreenTimer;
+};
+#endif
+```
+
+mediaapp.cpp
+
+**mediaapp.cpp**
+
+``` c
+#include "mediaapp.h"
+#include "player.h"
+#if (QT_VERSION >= QT_VERSION_CHECK(5, 0, 0))
+#include <QtWidgets/QBoxLayout>
+#include <QtWidgets/QFileDialog>
+#include <QtWidgets/QToolButton>
+#include <QtWidgets/QLabel>
+#include <QtWidgets/QSlider>
+#else
+#include <QtGui/QBoxLayout>
+#include <QtGui/QFileDialog>
+#include <QtGui/QToolButton>
+#include <QtGui/QLabel>
+#include <QtGui/QSlider>
+#include <QtGui/QMouseEvent>
+#endif
+MediaApp::MediaApp(QWidget *parent)
+ : QWidget(parent)
+{
+ //create the player
+ m_player = new Player(this);
+ connect(m_player, SIGNAL(positionChanged()), this, SLOT(onPositionChanged()));
+ connect(m_player, SIGNAL(stateChanged()), this, SLOT(onStateChanged()));
+ //m_baseDir is used to remember the last directory that was used.
+ //defaults to the current working directory
+ m_baseDir = QLatin1String(".");
+ //this timer (re-)hides the controls after a few seconds when we are in fullscreen mode
+ m_fullScreenTimer.setSingleShot(true);
+ connect(&m_fullScreenTimer, SIGNAL(timeout()), this, SLOT(hideControls()));
+ //create the UI
+ QVBoxLayout *appLayout = new QVBoxLayout;
+ appLayout->setContentsMargins(0, 0, 0, 0);
+ createUI(appLayout);
+ setLayout(appLayout);
+ onStateChanged(); //set the controls to their default state
+ setWindowTitle(tr("QtGStreamer example player"));
+ resize(400, 400);
+}
+MediaApp::~MediaApp()
+{
+ delete m_player;
+}
+void MediaApp::openFile(const QString & fileName)
+{
+ m_baseDir = QFileInfo(fileName).path();
+ m_player->stop();
+ m_player->setUri(fileName);
+ m_player->play();
+}
+void MediaApp::open()
+{
+ QString fileName = QFileDialog::getOpenFileName(this, tr("Open a Movie"), m_baseDir);
+ if (!fileName.isEmpty()) {
+ openFile(fileName);
+ }
+}
+void MediaApp::toggleFullScreen()
+{
+ if (isFullScreen()) {
+ setMouseTracking(false);
+ m_player->setMouseTracking(false);
+ m_fullScreenTimer.stop();
+ showControls();
+ showNormal();
+ } else {
+ setMouseTracking(true);
+ m_player->setMouseTracking(true);
+ hideControls();
+ showFullScreen();
+ }
+}
+void MediaApp::onStateChanged()
+{
+ QGst::State newState = m_player->state();
+ m_playButton->setEnabled(newState != QGst::StatePlaying);
+ m_pauseButton->setEnabled(newState == QGst::StatePlaying);
+ m_stopButton->setEnabled(newState != QGst::StateNull);
+ m_positionSlider->setEnabled(newState != QGst::StateNull);
+ m_volumeSlider->setEnabled(newState != QGst::StateNull);
+ m_volumeLabel->setEnabled(newState != QGst::StateNull);
+ m_volumeSlider->setValue(m_player->volume());
+ //if we are in Null state, call onPositionChanged() to restore
+ //the position of the slider and the text on the label
+ if (newState == QGst::StateNull) {
+ onPositionChanged();
+ }
+}
+/* Called when the positionChanged() is received from the player */
+void MediaApp::onPositionChanged()
+{
+ QTime length(0,0);
+ QTime curpos(0,0);
+ if (m_player->state() != QGst::StateReady &&
+ m_player->state() != QGst::StateNull)
+ {
+ length = m_player->length();
+ curpos = m_player->position();
+ }
+ m_positionLabel->setText(curpos.toString("hh:mm:ss.zzz")
+ + "/" +
+ length.toString("hh:mm:ss.zzz"));
+ if (length != QTime(0,0)) {
+ m_positionSlider->setValue(curpos.msecsTo(QTime(0,0)) * 1000 / length.msecsTo(QTime(0,0)));
+ } else {
+ m_positionSlider->setValue(0);
+ }
+ if (curpos != QTime(0,0)) {
+ m_positionLabel->setEnabled(true);
+ m_positionSlider->setEnabled(true);
+ }
+}
+/* Called when the user changes the slider's position */
+void MediaApp::setPosition(int value)
+{
+ uint length = -m_player->length().msecsTo(QTime(0,0));
+ if (length != 0 && value > 0) {
+ QTime pos(0,0);
+ pos = pos.addMSecs(length * (value / 1000.0));
+ m_player->setPosition(pos);
+ }
+}
+void MediaApp::showControls(bool show)
+{
+ m_openButton->setVisible(show);
+ m_playButton->setVisible(show);
+ m_pauseButton->setVisible(show);
+ m_stopButton->setVisible(show);
+ m_fullScreenButton->setVisible(show);
+ m_positionSlider->setVisible(show);
+ m_volumeSlider->setVisible(show);
+ m_volumeLabel->setVisible(show);
+ m_positionLabel->setVisible(show);
+}
+void MediaApp::mouseMoveEvent(QMouseEvent *event)
+{
+ Q_UNUSED(event);
+ if (isFullScreen()) {
+ showControls();
+ m_fullScreenTimer.start(3000); //re-hide controls after 3s
+ }
+}
+QToolButton *MediaApp::initButton(QStyle::StandardPixmap icon, const QString & tip,
+ QObject *dstobj, const char *slot_method, QLayout *layout)
+{
+ QToolButton *button = new QToolButton;
+ button->setIcon(style()->standardIcon(icon));
+ button->setIconSize(QSize(36, 36));
+ button->setToolTip(tip);
+ connect(button, SIGNAL(clicked()), dstobj, slot_method);
+ layout->addWidget(button);
+ return button;
+}
+void MediaApp::createUI(QBoxLayout *appLayout)
+{
+ appLayout->addWidget(m_player);
+ m_positionLabel = new QLabel();
+ m_positionSlider = new QSlider(Qt::Horizontal);
+ m_positionSlider->setTickPosition(QSlider::TicksBelow);
+ m_positionSlider->setTickInterval(10);
+ m_positionSlider->setMaximum(1000);
+ connect(m_positionSlider, SIGNAL(sliderMoved(int)), this, SLOT(setPosition(int)));
+ m_volumeSlider = new QSlider(Qt::Horizontal);
+ m_volumeSlider->setTickPosition(QSlider::TicksLeft);
+ m_volumeSlider->setTickInterval(2);
+ m_volumeSlider->setMaximum(10);
+ m_volumeSlider->setMaximumSize(64,32);
+ connect(m_volumeSlider, SIGNAL(sliderMoved(int)), m_player, SLOT(setVolume(int)));
+ QGridLayout *posLayout = new QGridLayout;
+ posLayout->addWidget(m_positionLabel, 1, 0);
+ posLayout->addWidget(m_positionSlider, 1, 1, 1, 2);
+ appLayout->addLayout(posLayout);
+ QHBoxLayout *btnLayout = new QHBoxLayout;
+ btnLayout->addStretch();
+ m_openButton = initButton(QStyle::SP_DialogOpenButton, tr("Open File"),
+ this, SLOT(open()), btnLayout);
+ m_playButton = initButton(QStyle::SP_MediaPlay, tr("Play"),
+ m_player, SLOT(play()), btnLayout);
+ m_pauseButton = initButton(QStyle::SP_MediaPause, tr("Pause"),
+ m_player, SLOT(pause()), btnLayout);
+ m_stopButton = initButton(QStyle::SP_MediaStop, tr("Stop"),
+ m_player, SLOT(stop()), btnLayout);
+ m_fullScreenButton = initButton(QStyle::SP_TitleBarMaxButton, tr("Fullscreen"),
+ this, SLOT(toggleFullScreen()), btnLayout);
+ btnLayout->addStretch();
+ m_volumeLabel = new QLabel();
+ m_volumeLabel->setPixmap(
+ style()->standardIcon(QStyle::SP_MediaVolume).pixmap(QSize(32, 32),
+ QIcon::Normal, QIcon::On));
+ btnLayout->addWidget(m_volumeLabel);
+ btnLayout->addWidget(m_volumeSlider);
+ appLayout->addLayout(btnLayout);
+}
+#include "moc_mediaapp.cpp"
+```
+
+player.h
+
+**player.h**
+
+``` c
+#ifndef PLAYER_H
+#define PLAYER_H
+#include <QtCore/QTimer>
+#include <QtCore/QTime>
+#include <QGst/Pipeline>
+#include <QGst/Ui/VideoWidget>
+
+class Player : public QGst::Ui::VideoWidget
+{
+ Q_OBJECT
+public:
+ Player(QWidget *parent = 0);
+ ~Player();
+
+ void setUri(const QString &uri);
+
+ QTime position() const;
+ void setPosition(const QTime &pos);
+ int volume() const;
+ QTime length() const;
+ QGst::State state() const;
+
+public Q_SLOTS:
+ void play();
+ void pause();
+ void stop();
+ void setVolume(int volume);
+
+Q_SIGNALS:
+ void positionChanged();
+ void stateChanged();
+
+private:
+ void onBusMessage(const QGst::MessagePtr &message);
+ void handlePipelineStateChange(const QGst::StateChangedMessagePtr &scm);
+
+ QGst::PipelinePtr m_pipeline;
+ QTimer m_positionTimer;
+};
+
+#endif //PLAYER_H
+```
+
+player.cpp
+
+**player.cpp**
+
+``` c
+#include "player.h"
+#include <QtCore/QDir>
+#include <QtCore/QUrl>
+#include <QGlib/Connect>
+#include <QGlib/Error>
+#include <QGst/Pipeline>
+#include <QGst/ElementFactory>
+#include <QGst/Bus>
+#include <QGst/Message>
+#include <QGst/Query>
+#include <QGst/ClockTime>
+#include <QGst/Event>
+#include <QGst/StreamVolume>
+Player::Player(QWidget *parent)
+ : QGst::Ui::VideoWidget(parent)
+{
+ //this timer is used to tell the ui to change its position slider & label
+ //every 100 ms, but only when the pipeline is playing
+ connect(&m_positionTimer, SIGNAL(timeout()), this, SIGNAL(positionChanged()));
+}
+Player::~Player()
+{
+ if (m_pipeline) {
+ m_pipeline->setState(QGst::StateNull);
+ stopPipelineWatch();
+ }
+}
+void Player::setUri(const QString & uri)
+{
+ QString realUri = uri;
+ //if uri is not a real uri, assume it is a file path
+ if (realUri.indexOf("://") < 0) {
+ realUri = QUrl::fromLocalFile(realUri).toEncoded();
+ }
+ if (!m_pipeline) {
+ m_pipeline = QGst::ElementFactory::make("playbin").dynamicCast<QGst::Pipeline>();
+ if (m_pipeline) {
+ //let the video widget watch the pipeline for new video sinks
+ watchPipeline(m_pipeline);
+ //watch the bus for messages
+ QGst::BusPtr bus = m_pipeline->bus();
+ bus->addSignalWatch();
+ QGlib::connect(bus, "message", this, &Player::onBusMessage);
+ } else {
+ qCritical() << "Failed to create the pipeline";
+ }
+ }
+ if (m_pipeline) {
+ m_pipeline->setProperty("uri", realUri);
+ }
+}
+QTime Player::position() const
+{
+ if (m_pipeline) {
+ //here we query the pipeline about its position
+ //and we request that the result is returned in time format
+ QGst::PositionQueryPtr query = QGst::PositionQuery::create(QGst::FormatTime);
+ m_pipeline->query(query);
+ return QGst::ClockTime(query->position()).toTime();
+ } else {
+ return QTime(0,0);
+ }
+}
+void Player::setPosition(const QTime & pos)
+{
+ QGst::SeekEventPtr evt = QGst::SeekEvent::create(
+ 1.0, QGst::FormatTime, QGst::SeekFlagFlush,
+ QGst::SeekTypeSet, QGst::ClockTime::fromTime(pos),
+ QGst::SeekTypeNone, QGst::ClockTime::None
+ );
+ m_pipeline->sendEvent(evt);
+}
+int Player::volume() const
+{
+ if (m_pipeline) {
+ QGst::StreamVolumePtr svp =
+ m_pipeline.dynamicCast<QGst::StreamVolume>();
+ if (svp) {
+ return svp->volume(QGst::StreamVolumeFormatCubic) * 10;
+ }
+ }
+ return 0;
+}
+
+void Player::setVolume(int volume)
+{
+ if (m_pipeline) {
+ QGst::StreamVolumePtr svp =
+ m_pipeline.dynamicCast<QGst::StreamVolume>();
+ if(svp) {
+ svp->setVolume((double)volume / 10, QGst::StreamVolumeFormatCubic);
+ }
+ }
+}
+QTime Player::length() const
+{
+ if (m_pipeline) {
+ //here we query the pipeline about the content's duration
+ //and we request that the result is returned in time format
+ QGst::DurationQueryPtr query = QGst::DurationQuery::create(QGst::FormatTime);
+ m_pipeline->query(query);
+ return QGst::ClockTime(query->duration()).toTime();
+ } else {
+ return QTime(0,0);
+ }
+}
+QGst::State Player::state() const
+{
+ return m_pipeline ? m_pipeline->currentState() : QGst::StateNull;
+}
+void Player::play()
+{
+ if (m_pipeline) {
+ m_pipeline->setState(QGst::StatePlaying);
+ }
+}
+void Player::pause()
+{
+ if (m_pipeline) {
+ m_pipeline->setState(QGst::StatePaused);
+ }
+}
+void Player::stop()
+{
+ if (m_pipeline) {
+ m_pipeline->setState(QGst::StateNull);
+ //once the pipeline stops, the bus is flushed so we will
+ //not receive any StateChangedMessage about this.
+ //so, to inform the ui, we have to emit this signal manually.
+ Q_EMIT stateChanged();
+ }
+}
+void Player::onBusMessage(const QGst::MessagePtr & message)
+{
+ switch (message->type()) {
+ case QGst::MessageEos: //End of stream. We reached the end of the file.
+ stop();
+ break;
+ case QGst::MessageError: //Some error occurred.
+ qCritical() << message.staticCast<QGst::ErrorMessage>()->error();
+ stop();
+ break;
+ case QGst::MessageStateChanged: //The element in message->source() has changed state
+ if (message->source() == m_pipeline) {
+ handlePipelineStateChange(message.staticCast<QGst::StateChangedMessage>());
+ }
+ break;
+ default:
+ break;
+ }
+}
+void Player::handlePipelineStateChange(const QGst::StateChangedMessagePtr & scm)
+{
+ switch (scm->newState()) {
+ case QGst::StatePlaying:
+ //start the timer when the pipeline starts playing
+ m_positionTimer.start(100);
+ break;
+ case QGst::StatePaused:
+ //stop the timer when the pipeline pauses
+ if(scm->oldState() == QGst::StatePlaying) {
+ m_positionTimer.stop();
+ }
+ break;
+ default:
+ break;
+ }
+ Q_EMIT stateChanged();
+}
+#include "moc_player.cpp"
+```
+
+## Walkthrough
+
+### Setting up GStreamer
+
+We begin by looking at `main()`:
+
+**main.cpp**
+
+``` c
+int main(int argc, char *argv[])
+{
+ QApplication app(argc, argv);
+ QGst::init(&argc, &argv);
+ MediaApp media;
+ media.show();
+ if (argc == 2) {
+ media.openFile(argv[1]);
+ }
+ return app.exec();
+}
+```
+
+We first initialize QtGStreamer by calling `QGst::init()`, passing
+`argc` and `argv`. Internally, this ensures that the GLib type system
+and GStreamer plugin registry is configured and initialized, along with
+handling helpful environment variables such as `GST_DEBUG` and common
+command line options. Please see the [Running GStreamer
+Applications](http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gst-running.html)
+section of the core reference manual for details.
+
+Construction of the `MediaApp` (derived from
+[`QApplication`](http://qt-project.org/doc/qt-5.0/qtwidgets/qapplication.html))
+involves constructing the `Player` object and connecting its signals to
+the UI:
+
+**MediaApp::MediaApp()**
+
+``` c
+ //create the player
+ m_player = new Player(this);
+ connect(m_player, SIGNAL(positionChanged()), this, SLOT(onPositionChanged()));
+ connect(m_player, SIGNAL(stateChanged()), this, SLOT(onStateChanged()));
+```
+
+Next, we instruct the `MediaApp` to open the file given on the command
+line, if any:
+
+**MediaApp::openFile()**
+
+``` c
+void MediaApp::openFile(const QString & fileName)
+{
+ m_baseDir = QFileInfo(fileName).path();
+ m_player->stop();
+ m_player->setUri(fileName);
+ m_player->play();
+}
+```
+
+This in turn instructs the `Player` to construct our GStreamer pipeline:
+
+**Player::setUri()**
+
+``` c
+void Player::setUri(const QString & uri)
+{
+ QString realUri = uri;
+ //if uri is not a real uri, assume it is a file path
+ if (realUri.indexOf("://") < 0) {
+ realUri = QUrl::fromLocalFile(realUri).toEncoded();
+ }
+ if (!m_pipeline) {
+ m_pipeline = QGst::ElementFactory::make("playbin").dynamicCast<QGst::Pipeline>();
+ if (m_pipeline) {
+ //let the video widget watch the pipeline for new video sinks
+ watchPipeline(m_pipeline);
+ //watch the bus for messages
+ QGst::BusPtr bus = m_pipeline->bus();
+ bus->addSignalWatch();
+ QGlib::connect(bus, "message", this, &Player::onBusMessage);
+ } else {
+ qCritical() << "Failed to create the pipeline";
+ }
+ }
+ if (m_pipeline) {
+ m_pipeline->setProperty("uri", realUri);
+ }
+}
+```
+
+Here, we first ensure that the pipeline will receive a proper URI. If
+`Player::setUri()` is called with `/home/user/some/file.mp3`, the path
+is modified to `file:///home/user/some/file.mp3`. `playbin` only
+accepts complete URIs.
+
+The pipeline is created via `QGst::ElementFactory::make()`. The
+`Player` object inherits from the `QGst::Ui::VideoWidget` class, which
+includes a function to watch for the `prepare-xwindow-id` message, which
+associates the underlying video sink with a Qt widget used for
+rendering. For clarity, here is a portion of the implementation:
+
+**prepare-xwindow-id handling**
+
+``` c
+ QGlib::connect(pipeline->bus(), "sync-message",
+ this, &PipelineWatch::onBusSyncMessage);
+...
+void PipelineWatch::onBusSyncMessage(const MessagePtr & msg)
+{
+...
+ if (msg->internalStructure()->name() == QLatin1String("prepare-xwindow-id")) {
+ XOverlayPtr overlay = msg->source().dynamicCast<XOverlay>();
+ m_renderer->setVideoSink(overlay);
+ }
+```
+
+Once the pipeline is created, we connect to the bus' message signal (via
+`QGlib::connect()`) to dispatch state change signals:
+
+``` c
+void Player::onBusMessage(const QGst::MessagePtr & message)
+{
+ switch (message->type()) {
+ case QGst::MessageEos: //End of stream. We reached the end of the file.
+ stop();
+ break;
+ case QGst::MessageError: //Some error occurred.
+ qCritical() << message.staticCast<QGst::ErrorMessage>()->error();
+ stop();
+ break;
+ case QGst::MessageStateChanged: //The element in message->source() has changed state
+ if (message->source() == m_pipeline) {
+ handlePipelineStateChange(message.staticCast<QGst::StateChangedMessage>());
+ }
+ break;
+ default:
+ break;
+ }
+}
+void Player::handlePipelineStateChange(const QGst::StateChangedMessagePtr & scm)
+{
+ switch (scm->newState()) {
+ case QGst::StatePlaying:
+ //start the timer when the pipeline starts playing
+ m_positionTimer.start(100);
+ break;
+ case QGst::StatePaused:
+ //stop the timer when the pipeline pauses
+ if(scm->oldState() == QGst::StatePlaying) {
+ m_positionTimer.stop();
+ }
+ break;
+ default:
+ break;
+ }
+ Q_EMIT stateChanged();
+}
+```
+
+Finally, we tell `playbin` what to play by setting the `uri` property:
+
+``` c
+m_pipeline->setProperty("uri", realUri);
+```
+
+### Starting Playback
+
+After `Player::setUri()` is called, `MediaApp::openFile()` calls
+`play()` on the `Player` object:
+
+**Player::play()**
+
+``` c
+void Player::play()
+{
+ if (m_pipeline) {
+ m_pipeline->setState(QGst::StatePlaying);
+ }
+}
+```
+
+The other state control methods are equally simple:
+
+**Player state functions**
+
+``` c
+void Player::pause()
+{
+ if (m_pipeline) {
+ m_pipeline->setState(QGst::StatePaused);
+ }
+}
+void Player::stop()
+{
+ if (m_pipeline) {
+ m_pipeline->setState(QGst::StateNull);
+ //once the pipeline stops, the bus is flushed so we will
+ //not receive any StateChangedMessage about this.
+ //so, to inform the ui, we have to emit this signal manually.
+ Q_EMIT stateChanged();
+ }
+}
+```
+
+Once the pipeline has entered the playing state, a state change message
+is emitted on the GStreamer bus which gets picked up by the `Player`:
+
+**Player::onBusMessage()**
+
+``` c
+void Player::onBusMessage(const QGst::MessagePtr & message)
+{
+ switch (message->type()) {
+ case QGst::MessageEos: //End of stream. We reached the end of the file.
+ stop();
+ break;
+ case QGst::MessageError: //Some error occurred.
+ qCritical() << message.staticCast<QGst::ErrorMessage>()->error();
+ stop();
+ break;
+ case QGst::MessageStateChanged: //The element in message->source() has changed state
+ if (message->source() == m_pipeline) {
+ handlePipelineStateChange(message.staticCast<QGst::StateChangedMessage>());
+ }
+ break;
+ default:
+ break;
+ }
+}
+```
+
+The `stateChanged` signal we connected to earlier is emitted and
+handled:
+
+**MediaApp::onStateChanged()**
+
+``` c
+void MediaApp::onStateChanged()
+{
+ QGst::State newState = m_player->state();
+ m_playButton->setEnabled(newState != QGst::StatePlaying);
+ m_pauseButton->setEnabled(newState == QGst::StatePlaying);
+ m_stopButton->setEnabled(newState != QGst::StateNull);
+ m_positionSlider->setEnabled(newState != QGst::StateNull);
+ m_volumeSlider->setEnabled(newState != QGst::StateNull);
+ m_volumeLabel->setEnabled(newState != QGst::StateNull);
+ m_volumeSlider->setValue(m_player->volume());
+ //if we are in Null state, call onPositionChanged() to restore
+ //the position of the slider and the text on the label
+ if (newState == QGst::StateNull) {
+ onPositionChanged();
+ }
+}
+```
+
+This updates the UI to reflect the current state of the player's
+pipeline.
+
+Driven by a
+[`QTimer`](http://qt-project.org/doc/qt-5.0/qtcore/qtimer.html), the
+`Player` emits the `positionChanged` signal at regular intervals for the
+UI to handle:
+
+**MediaApp::onPositionChanged()**
+
+``` c
+void MediaApp::onPositionChanged()
+{
+ QTime length(0,0);
+ QTime curpos(0,0);
+ if (m_player->state() != QGst::StateReady &&
+ m_player->state() != QGst::StateNull)
+ {
+ length = m_player->length();
+ curpos = m_player->position();
+ }
+ m_positionLabel->setText(curpos.toString("hh:mm:ss.zzz")
+ + "/" +
+ length.toString("hh:mm:ss.zzz"));
+ if (length != QTime(0,0)) {
+ m_positionSlider->setValue(curpos.msecsTo(QTime(0,0)) * 1000 / length.msecsTo(QTime(0,0)));
+ } else {
+ m_positionSlider->setValue(0);
+ }
+ if (curpos != QTime(0,0)) {
+ m_positionLabel->setEnabled(true);
+ m_positionSlider->setEnabled(true);
+ }
+}
+```
+
+The `MediaApp` queries the pipeline via the `Player`'s
+`position()` method, which submits a position query. This is analogous
+to `gst_element_query_position()`:
+
+**Player::position()**
+
+``` c
+QTime Player::position() const
+{
+ if (m_pipeline) {
+ //here we query the pipeline about its position
+ //and we request that the result is returned in time format
+ QGst::PositionQueryPtr query = QGst::PositionQuery::create(QGst::FormatTime);
+ m_pipeline->query(query);
+ return QGst::ClockTime(query->position()).toTime();
+ } else {
+ return QTime(0,0);
+ }
+}
+```
+
+Due to the way Qt handles signals that cross threads, there is no need
+to worry about calling UI functions from outside the UI thread in this
+example.
+
+## Conclusion
+
+This tutorial has shown:
+
+ - How to create a basic pipeline
+ - How to create a video output
+ - Updating the GUI based on playback time
+
+It has been a pleasure having you here, and see you soon\!
--- /dev/null
+# Basic tutorial 2: GStreamer concepts
+
+## Goal
+
+The previous tutorial showed how to build a pipeline automatically. Now
+we are going to build a pipeline manually by instantiating each element
+and linking them all together. In the process, we will learn:
+
+ - What is a GStreamer element and how to create one.
+
+ - How to connect elements to each other.
+
+ - How to customize an element's behavior.
+
+ - How to watch the bus for error conditions and extract information
+ from GStreamer messages.
+
+## Manual Hello World
+
+Copy this code into a text file named `basic-tutorial-2.c` (or find it
+in the SDK installation).
+
+**basic-tutorial-2.c**
+
+``` c
+#include <gst/gst.h>
+
+int main(int argc, char *argv[]) {
+ GstElement *pipeline, *source, *sink;
+ GstBus *bus;
+ GstMessage *msg;
+ GstStateChangeReturn ret;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Create the elements */
+ source = gst_element_factory_make ("videotestsrc", "source");
+ sink = gst_element_factory_make ("autovideosink", "sink");
+
+ /* Create the empty pipeline */
+ pipeline = gst_pipeline_new ("test-pipeline");
+
+ if (!pipeline || !source || !sink) {
+ g_printerr ("Not all elements could be created.\n");
+ return -1;
+ }
+
+ /* Build the pipeline */
+ gst_bin_add_many (GST_BIN (pipeline), source, sink, NULL);
+ if (gst_element_link (source, sink) != TRUE) {
+ g_printerr ("Elements could not be linked.\n");
+ gst_object_unref (pipeline);
+ return -1;
+ }
+
+ /* Modify the source's properties */
+ g_object_set (source, "pattern", 0, NULL);
+
+ /* Start playing */
+ ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
+ if (ret == GST_STATE_CHANGE_FAILURE) {
+ g_printerr ("Unable to set the pipeline to the playing state.\n");
+ gst_object_unref (pipeline);
+ return -1;
+ }
+
+ /* Wait until error or EOS */
+ bus = gst_element_get_bus (pipeline);
+ msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
+
+ /* Parse message */
+ if (msg != NULL) {
+ GError *err;
+ gchar *debug_info;
+
+ switch (GST_MESSAGE_TYPE (msg)) {
+ case GST_MESSAGE_ERROR:
+ gst_message_parse_error (msg, &err, &debug_info);
+ g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
+ g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
+ g_clear_error (&err);
+ g_free (debug_info);
+ break;
+ case GST_MESSAGE_EOS:
+ g_print ("End-Of-Stream reached.\n");
+ break;
+ default:
+ /* We should not reach here because we only asked for ERRORs and EOS */
+ g_printerr ("Unexpected message received.\n");
+ break;
+ }
+ gst_message_unref (msg);
+ }
+
+ /* Free resources */
+ gst_object_unref (bus);
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+ return 0;
+}
+```
+
+> 
+> Need help?
+>
+> If you need help to compile this code, refer to the **Building the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Build), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Build) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Build), or use this specific command on Linux:
+>
+> `` gcc basic-tutorial-2.c -o basic-tutorial-2 `pkg-config --cflags --libs gstreamer-1.0` ``
+>
+>If you need help to run this code, refer to the **Running the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Run), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Run) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Run).
+>
+>This tutorial opens a window and displays a test pattern, without audio
+>
+>Required libraries: `gstreamer-1.0`
+
+## Walkthrough
+
+The basic construction block of GStreamer are the elements, which
+process the data as it flows *downstream* from the source elements (the
+producers of data) to the sink elements (the consumers of data), passing
+through filter elements.
+
+
+
+**Figure 1**. Example pipeline
+
+### Element creation
+
+We will skip GStreamer initialization, since it is the same as the
+previous tutorial:
+
+``` c
+/* Create the elements */
+source = gst_element_factory_make ("videotestsrc", "source");
+sink = gst_element_factory_make ("autovideosink", "sink");
+```
+
+As seen in this code, new elements can be created
+with `gst_element_factory_make()`. The first parameter is the type of
+element to create ([Basic tutorial 14: Handy
+elements](sdk-basic-tutorial-handy-elements.md) shows a
+few common types, and [Basic tutorial 10: GStreamer
+tools](sdk-basic-tutorial-gstreamer-tools.md) shows how to
+obtain the list of all available types). The second parameter is the
+name we want to give to this particular instance. Naming your elements
+is useful to retrieve them later if you didn't keep a pointer (and for
+more meaningful debug output). If you pass NULL for the name, however,
+GStreamer will provide a unique name for you.
+
+For this tutorial we create two elements: a `videotestsrc` and
+an `autovideosink`.
+
+`videotestsrc` is a source element (it produces data), which creates a
+test video pattern. This element is useful for debugging purposes (and
+tutorials) and is not usually found in real applications.
+
+`autovideosink` is a sink element (it consumes data), which displays on
+a window the images it receives. There exist several video sinks,
+depending on the operating system, with a varying range of capabilities.
+`autovideosink` automatically selects and instantiates the best one, so
+you do not have to worry with the details, and your code is more
+platform-independent.
+
+### Pipeline creation
+
+``` c
+/* Create the empty pipeline */
+pipeline = gst_pipeline_new ("test-pipeline");
+```
+
+All elements in GStreamer must typically be contained inside a pipeline
+before they can be used, because it takes care of some clocking and
+messaging functions. We create the pipeline with `gst_pipeline_new()`.
+
+``` c
+/* Build the pipeline */
+gst_bin_add_many (GST_BIN (pipeline), source, sink, NULL);
+if (gst_element_link (source, sink) != TRUE) {
+ g_printerr ("Elements could not be linked.\n");
+ gst_object_unref (pipeline);
+ return -1;
+}
+```
+
+A pipeline is a particular type of `bin`, which is the element used to
+contain other elements. Therefore all methods which apply to bins also
+apply to pipelines. In our case, we call `gst_bin_add_many()` to add the
+elements to the pipeline (mind the cast). This function accepts a list
+of elements to be added, ending with NULL. Individual elements can be
+added with `gst_bin_add()`.
+
+These elements, however, are not linked with each other yet. For this,
+we need to use `gst_element_link()`. Its first parameter is the source,
+and the second one the destination. The order counts, because links must
+be established following the data flow (this is, from source elements to
+sink elements). Keep in mind that only elements residing in the same bin
+can be linked together, so remember to add them to the pipeline before
+trying to link them!
+
+### Properties
+
+``` c
+/* Modify the source's properties */
+g_object_set (source, "pattern", 0, NULL);
+```
+
+Most GStreamer elements have customizable properties: named attributes
+that can be modified to change the element's behavior (writable
+properties) or inquired to find out about the element's internal state
+(readable properties).
+
+Properties are read from with `g_object_get()` and written to
+with `g_object_set()`.
+
+`g_object_set()` accepts a NULL-terminated list of property-name,
+property-value pairs, so multiple properties can be changed in one go
+(GStreamer elements are all a particular kind of `GObject`, which is the
+entity offering property facilities: This is why the property handling
+methods have the `g_` prefix).
+
+The line of code above changes the “pattern” property of `videotestsrc`,
+which controls the type of test video the element outputs. Try different
+values!
+
+The names and possible values of all the properties an element exposes
+can be found using the gst-inspect-1.0 tool described in [Basic tutorial 10:
+GStreamer tools](sdk-basic-tutorial-gstreamer-tools.md).
+
+### Error checking
+
+At this point, we have the whole pipeline built and setup, and the rest
+of the tutorial is very similar to the previous one, but we are going to
+add more error checking:
+
+``` c
+/* Start playing */
+ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
+if (ret == GST_STATE_CHANGE_FAILURE) {
+ g_printerr ("Unable to set the pipeline to the playing state.\n");
+ gst_object_unref (pipeline);
+ return -1;
+}
+```
+
+We call `gst_element_set_state()`, but this time we check its return
+value for errors. Changing states is a delicate process and a few more
+details are given in [Basic tutorial 3: Dynamic
+pipelines](sdk-basic-tutorial-dynamic-pipelines.md).
+
+``` c
+/* Wait until error or EOS */
+bus = gst_element_get_bus (pipeline);
+msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
+
+/* Parse message */
+if (msg != NULL) {
+ GError *err;
+ gchar *debug_info;
+
+ switch (GST_MESSAGE_TYPE (msg)) {
+ case GST_MESSAGE_ERROR:
+ gst_message_parse_error (msg, &err, &debug_info);
+ g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
+ g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
+ g_clear_error (&err);
+ g_free (debug_info);
+ break;
+ case GST_MESSAGE_EOS:
+ g_print ("End-Of-Stream reached.\n");
+ break;
+ default:
+ /* We should not reach here because we only asked for ERRORs and EOS */
+ g_printerr ("Unexpected message received.\n");
+ break;
+ }
+ gst_message_unref (msg);
+}
+```
+
+`gst_bus_timed_pop_filtered()` waits for execution to end and returns
+with a `GstMessage` which we previously ignored. We
+asked `gst_bus_timed_pop_filtered()` to return when GStreamer
+encountered either an error condition or an EOS, so we need to check
+which one happened, and print a message on screen (Your application will
+probably want to undertake more complex actions).
+
+`GstMessage` is a very versatile structure which can deliver virtually
+any kind of information. Fortunately, GStreamer provides a series of
+parsing functions for each kind of message.
+
+In this case, once we know the message contains an error (by using the
+`GST_MESSAGE_TYPE()` macro), we can use
+`gst_message_parse_error()` which returns a GLib `GError` error
+structure and a string useful for debugging. Examine the code to see how
+these are used and freed afterward.
+
+### The GStreamer bus
+
+At this point it is worth introducing the GStreamer bus a bit more
+formally. It is the object responsible for delivering to the application
+the `GstMessage`s generated by the elements, in order and to the
+application thread. This last point is important, because the actual
+streaming of media is done in another thread than the application.
+
+Messages can be extracted from the bus synchronously with
+`gst_bus_timed_pop_filtered()` and its siblings, or asynchronously,
+using signals (shown in the next tutorial). Your application should
+always keep an eye on the bus to be notified of errors and other
+playback-related issues.
+
+The rest of the code is the cleanup sequence, which is the same as
+in [Basic tutorial 1: Hello
+world!](sdk-basic-tutorial-hello-world.md).
+
+## Exercise
+
+If you feel like practicing, try this exercise: Add a video filter
+element in between the source and the sink of this pipeline. Use
+`vertigotv` for a nice effect. You will need to create it, add it to the
+pipeline, and link it with the other elements.
+
+Depending on your platform and available plugins, you might get a
+“negotiation” error, because the sink does not understand what the
+filter is producing (more about negotiation in [Basic tutorial 6: Media
+formats and Pad
+Capabilities](sdk-basic-tutorial-media-formats-and-pad-capabilities.md)).
+In this case, try to add an element called `videoconvert` after the
+filter (this is, build a pipeline of 4 elements. More on
+`videoconvert` in [Basic tutorial 14: Handy
+elements](sdk-basic-tutorial-handy-elements.md)).
+
+## Conclusion
+
+This tutorial showed:
+
+ - How to create elements with `gst_element_factory_make()`
+
+ - How to create an empty pipeline with `gst_pipeline_new()`
+
+ - How to add elements to the pipeline with `gst_bin_add_many()`
+
+ - How to link the elements with each other with `gst_element_link()`
+
+This concludes the first of the two tutorials devoted to basic GStreamer
+concepts. The second one comes next.
+
+Remember that attached to this page you should find the complete source
+code of the tutorial and any accessory files needed to build it.
+
+It has been a pleasure having you here, and see you soon!
--- /dev/null
+# Basic tutorial 11: Debugging tools
+
+## Goal
+
+Sometimes things won’t go as expected and the error messages retrieved
+from the bus (if any) just don’t provide enough information. Luckily,
+GStreamer ships with massive amounts of debug information, which usually
+hint what the problem might be. This tutorial shows:
+
+ - How to get more debug information from GStreamer.
+
+ - How to print your own debug information into the GStreamer log.
+
+ - How to get pipeline graphs
+
+## Printing debug information
+
+### The debug log
+
+GStreamer and its plugins are full of debug traces, this is, places in
+the code where a particularly interesting piece of information is
+printed to the console, along with time stamping, process, category,
+source code file, function and element information.
+
+The debug output is controlled with the `GST_DEBUG` environment
+variable. Here’s an example with
+`GST_DEBUG=2`:
+
+```
+0:00:00.868050000 1592 09F62420 WARN filesrc gstfilesrc.c:1044:gst_file_src_start:<filesrc0> error: No such file "non-existing-file.webm"
+```
+
+As you can see, this is quite a bit of information. In fact, the
+GStreamer debug log is so verbose, that when fully enabled it can render
+applications unresponsive (due to the console scrolling) or fill up
+megabytes of text files (when redirected to a file). For this reason,
+the logs are categorized, and you seldom need to enable all categories
+at once.
+
+The first category is the Debug Level, which is a number specifying the
+amount of desired output:
+
+| # | Name | Description |
+|---|---------|---|
+| 0 | none | No debug information is output. |
+| 1 | ERROR | Logs all fatal errors. These are errors that do not allow the core or elements to perform the requested action. The application can still recover if programmed to handle the conditions that triggered the error. |
+| 2 | WARNING | Logs all warnings. Typically these are non-fatal, but user-visible problems are expected to happen. |
+| 3 | FIXME | Logs all "fixme" messages. Those typically that a codepath that is known to be incomplete has been triggered. It may work in most cases, but mauy cause problems in specific instances. |
+| 4 | INFO | Logs all informational messages. These are typically used for events in the system that only happen once, or are important and rare enough to be logged at this level. |
+| 5 | DEBUG | Logs all debug messages. These are general debug messages for events that happen only a limited number of times during an object's lifetime; these include setup, teardown, change of parameters, ... |
+| 6 | LOG | Logs all log messages. These are messages for events that happen repeatedly during an object's lifetime; these include streaming and steady-state conditions. This is used for log messages that happen on every buffer in an element for example. |
+| 7 | TRACE | Logs all trace messages. Those are message that happen very very often. This is for example is each each time the reference count of a GstMiniObject, such as a GstBuffer or GstEvent, is modified. |
+| 8 | MEMDUMP | Logs all memory dump messages. This is the heaviest logging and may include dumping the content of blocks of memory. |
+
+To enable debug output, set the `GST_DEBUG` environment variable to the
+desired debug level. All levels below that will also be shown (i.e., if
+you set `GST_DEBUG=2`, you will get both `ERROR` and
+`WARNING` messages).
+
+Furthermore, each plugin or part of the GStreamer defines its own
+category, so you can specify a debug level for each individual category.
+For example, `GST_DEBUG=2,audiotestsrc:6`, will use Debug Level 6 for
+the `audiotestsrc` element, and 2 for all the others.
+
+The `GST_DEBUG` environment variable, then, is a comma-separated list of
+*category*:*level* pairs, with an optional *level* at the beginning,
+representing the default debug level for all categories.
+
+The `'*'` wildcard is also available. For example
+`GST_DEBUG=2,audio*:6` will use Debug Level 5 for all categories
+starting with the word `audio`. `GST_DEBUG=*:2` is equivalent to
+`GST_DEBUG=2`.
+
+Use `gst-launch-1.0 --gst-debug-help` to obtain the list of all
+registered categories. Bear in mind that each plugin registers its own
+categories, so, when installing or removing plugins, this list can
+change.
+
+Use `GST_DEBUG` when the error information posted on the GStreamer bus
+does not help you nail down a problem. It is common practice to redirect
+the output log to a file, and then examine it later, searching for
+specific messages.
+
+The content of each line in the debug output
+is:
+
+```
+0:00:00.868050000 1592 09F62420 WARN filesrc gstfilesrc.c:1044:gst_file_src_start:<filesrc0> error: No such file "non-existing-file.webm"
+```
+
+
+| Example | Explained |
+|---------------------|-----------|
+| `0:00:00.868050000` | Time stamp in HH:MM:SS.sssssssss format since the start of the program |
+| `1592` | Process ID from which the message was issued. Useful when your problem involves multiple processes |
+| `09F62420` | Thread ID from which the message was issued. Useful when your problem involves multiple threads |
+| `WARN` | Debug level of the message |
+| `filesrc` | Debug Category of the message |
+| `gstfilesrc.c:1044` | Source file and line in the GStreamer source code where this message is printed |
+| `gst_file_src_start`| Function from which the message was issued |
+| `<filesrc0>` | Name of the object that issued the message. It can be an element, a Pad, or something else. Useful when you have multiple elements of the same kind and need to distinguish among them. Naming your elements with the name property will make this debug output more readable (otherwise, GStreamer assigns each new element a unique name). |
+| `error: No such file "non-existing-file.webm"` | The actual message.|
+
+### Adding your own debug information
+
+In the parts of your code that interact with GStreamer, it is
+interesting to use GStreamer’s debugging facilities. In this way, you
+have all debug output in the same file and the temporal relationship
+between different messages is preserved.
+
+To do so, use the `GST_ERROR()`, `GST_WARNING()`, `GST_INFO()`,
+`GST_LOG()` and `GST_DEBUG()` macros. They accept the same parameters as
+`printf`, and they use the `default` category (`default` will be shown
+as the Debug category in the output log).
+
+To change the category to something more meaningful, add these two lines
+at the top of your code:
+
+``` c
+GST_DEBUG_CATEGORY_STATIC (my_category);
+#define GST_CAT_DEFAULT my_category
+```
+
+And then this one after you have initialized GStreamer with
+`gst_init()`:
+
+``` c
+GST_DEBUG_CATEGORY_INIT (my_category, "my category", 0, "This is my very own");
+```
+
+This registers a new category (this is, for the duration of your
+application: it is not stored in any file), and sets it as the default
+category for your code. See the documentation
+for `GST_DEBUG_CATEGORY_INIT()`.
+
+### Getting pipeline graphs
+
+For those cases where your pipeline starts to grow too large and you
+lose track of what is connected with what, GStreamer has the capability
+to output graph files. These are `.dot` files, readable with free
+programs like [GraphViz](http://www.graphviz.org), that describe the
+topology of your pipeline, along with the caps negotiated in each link.
+
+This is also very handy when using all-in-one elements like `playbin`
+ or `uridecodebin`, which instantiate several elements inside them. Use
+the `.dot` files to learn what pipeline they have created inside (and
+learn a bit of GStreamer along the way).
+
+To obtain `.dot` files, simply set
+the `GST_DEBUG_DUMP_DOT_DIR` environment variable to point to the
+folder where you want the files to be placed. `gst-launch-1.0` will create
+a `.dot` file at each state change, so you can see the evolution of the
+caps negotiation. Unset the variable to disable this facility. From
+within your application, you can use the
+`GST_DEBUG_BIN_TO_DOT_FILE()` and
+`GST_DEBUG_BIN_TO_DOT_FILE_WITH_TS()` macros to generate `.dot` files
+at your convenience.
+
+Here you have an example of the kind of pipelines that playbin
+generates. It is very complex because `playbin` can handle many
+different cases: Your manual pipelines normally do not need to be this
+long. If your manual pipeline is starting to get very big, consider
+using `playbin`.
+
+
+
+To download the full-size picture, use the attachments link at the top
+of this page (It's the paperclip icon).
+
+## Conclusion
+
+This tutorial has shown:
+
+ - How to get more debug information from GStreamer using the
+ `GST_DEBUG` environment variable.
+ - How to print your own debug information into the GStreamer log with
+ the `GST_ERROR()` macro and relatives.
+ - How to get pipeline graphs with the
+ `GST_DEBUG_DUMP_DOT_DIR` environment variable.
+
+It has been a pleasure having you here, and see you soon!
--- /dev/null
+# Basic tutorial 3: Dynamic pipelines
+
+## Goal
+
+This tutorial shows the rest of the basic concepts required to use
+GStreamer, which allow building the pipeline "on the fly", as
+information becomes available, instead of having a monolithic pipeline
+defined at the beginning of your application.
+
+After this tutorial, you will have the necessary knowledge to start the
+[Playback tutorials](sdk-playback-tutorials.md). The points reviewed
+here will be:
+
+ - How to attain finer control when linking elements.
+
+ - How to be notified of interesting events so you can react in time.
+
+ - The various states in which an element can be.
+
+## Introduction
+
+As you are about to see, the pipeline in this tutorial is not
+completely built before it is set to the playing state. This is OK. If
+we did not take further action, data would reach the end of the
+pipeline and the pipeline would produce an error message and stop. But
+we are going to take further action...
+
+In this example we are opening a file which is multiplexed (or *muxed)*,
+this is, audio and video are stored together inside a *container* file.
+The elements responsible for opening such containers are called
+*demuxers*, and some examples of container formats are Matroska (MKV),
+Quick Time (QT, MOV), Ogg, or Advanced Systems Format (ASF, WMV, WMA).
+
+If a container embeds multiple streams (one video and two audio tracks,
+for example), the demuxer will separate them and expose them through
+different output ports. In this way, different branches can be created
+in the pipeline, dealing with different types of data.
+
+The ports through which GStreamer elements communicate with each other
+are called pads (`GstPad`). There exists sink pads, through which data
+enters an element, and source pads, through which data exits an element.
+It follows naturally that source elements only contain source pads, sink
+elements only contain sink pads, and filter elements contain
+both.
+
+  
+
+**Figure 1**. GStreamer elements with their pads.
+
+A demuxer contains one sink pad, through which the muxed data arrives,
+and multiple source pads, one for each stream found in the container:
+
+
+
+**Figure 2**. A demuxer with two source pads.
+
+For completeness, here you have a simplified pipeline containing a
+demuxer and two branches, one for audio and one for video. This is
+**NOT** the pipeline that will be built in this example:
+
+
+
+**Figure 3**. Example pipeline with two branches.
+
+The main complexity when dealing with demuxers is that they cannot
+produce any information until they have received some data and have had
+a chance to look at the container to see what is inside. This is,
+demuxers start with no source pads to which other elements can link, and
+thus the pipeline must necessarily terminate at them.
+
+The solution is to build the pipeline from the source down to the
+demuxer, and set it to run (play). When the demuxer has received enough
+information to know about the number and kind of streams in the
+container, it will start creating source pads. This is the right time
+for us to finish building the pipeline and attach it to the newly added
+demuxer pads.
+
+For simplicity, in this example, we will only link to the audio pad and
+ignore the video.
+
+## Dynamic Hello World
+
+Copy this code into a text file named `basic-tutorial-3.c` (or find it
+in the SDK installation).
+
+**basic-tutorial-3.c**
+
+``` c
+#include <gst/gst.h>
+
+/* Structure to contain all our information, so we can pass it to callbacks */
+typedef struct _CustomData {
+ GstElement *pipeline;
+ GstElement *source;
+ GstElement *convert;
+ GstElement *sink;
+} CustomData;
+
+/* Handler for the pad-added signal */
+static void pad_added_handler (GstElement *src, GstPad *pad, CustomData *data);
+
+int main(int argc, char *argv[]) {
+ CustomData data;
+ GstBus *bus;
+ GstMessage *msg;
+ GstStateChangeReturn ret;
+ gboolean terminate = FALSE;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Create the elements */
+ data.source = gst_element_factory_make ("uridecodebin", "source");
+ data.convert = gst_element_factory_make ("audioconvert", "convert");
+ data.sink = gst_element_factory_make ("autoaudiosink", "sink");
+
+ /* Create the empty pipeline */
+ data.pipeline = gst_pipeline_new ("test-pipeline");
+
+ if (!data.pipeline || !data.source || !data.convert || !data.sink) {
+ g_printerr ("Not all elements could be created.\n");
+ return -1;
+ }
+
+ /* Build the pipeline. Note that we are NOT linking the source at this
+ * point. We will do it later. */
+ gst_bin_add_many (GST_BIN (data.pipeline), data.source, data.convert , data.sink, NULL);
+ if (!gst_element_link (data.convert, data.sink)) {
+ g_printerr ("Elements could not be linked.\n");
+ gst_object_unref (data.pipeline);
+ return -1;
+ }
+
+ /* Set the URI to play */
+ g_object_set (data.source, "uri", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
+
+ /* Connect to the pad-added signal */
+ g_signal_connect (data.source, "pad-added", G_CALLBACK (pad_added_handler), &data);
+
+ /* Start playing */
+ ret = gst_element_set_state (data.pipeline, GST_STATE_PLAYING);
+ if (ret == GST_STATE_CHANGE_FAILURE) {
+ g_printerr ("Unable to set the pipeline to the playing state.\n");
+ gst_object_unref (data.pipeline);
+ return -1;
+ }
+
+ /* Listen to the bus */
+ bus = gst_element_get_bus (data.pipeline);
+ do {
+ msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE,
+ GST_MESSAGE_STATE_CHANGED | GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
+
+ /* Parse message */
+ if (msg != NULL) {
+ GError *err;
+ gchar *debug_info;
+
+ switch (GST_MESSAGE_TYPE (msg)) {
+ case GST_MESSAGE_ERROR:
+ gst_message_parse_error (msg, &err, &debug_info);
+ g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
+ g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
+ g_clear_error (&err);
+ g_free (debug_info);
+ terminate = TRUE;
+ break;
+ case GST_MESSAGE_EOS:
+ g_print ("End-Of-Stream reached.\n");
+ terminate = TRUE;
+ break;
+ case GST_MESSAGE_STATE_CHANGED:
+ /* We are only interested in state-changed messages from the pipeline */
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data.pipeline)) {
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ g_print ("Pipeline state changed from %s to %s:\n",
+ gst_element_state_get_name (old_state), gst_element_state_get_name (new_state));
+ }
+ break;
+ default:
+ /* We should not reach here */
+ g_printerr ("Unexpected message received.\n");
+ break;
+ }
+ gst_message_unref (msg);
+ }
+ } while (!terminate);
+
+ /* Free resources */
+ gst_object_unref (bus);
+ gst_element_set_state (data.pipeline, GST_STATE_NULL);
+ gst_object_unref (data.pipeline);
+ return 0;
+}
+
+/* This function will be called by the pad-added signal */
+static void pad_added_handler (GstElement *src, GstPad *new_pad, CustomData *data) {
+ GstPad *sink_pad = gst_element_get_static_pad (data->convert, "sink");
+ GstPadLinkReturn ret;
+ GstCaps *new_pad_caps = NULL;
+ GstStructure *new_pad_struct = NULL;
+ const gchar *new_pad_type = NULL;
+
+ g_print ("Received new pad '%s' from '%s':\n", GST_PAD_NAME (new_pad), GST_ELEMENT_NAME (src));
+
+ /* If our converter is already linked, we have nothing to do here */
+ if (gst_pad_is_linked (sink_pad)) {
+ g_print (" We are already linked. Ignoring.\n");
+ goto exit;
+ }
+
+ /* Check the new pad's type */
+ new_pad_caps = gst_pad_query_caps (new_pad, NULL);
+ new_pad_struct = gst_caps_get_structure (new_pad_caps, 0);
+ new_pad_type = gst_structure_get_name (new_pad_struct);
+ if (!g_str_has_prefix (new_pad_type, "audio/x-raw")) {
+ g_print (" It has type '%s' which is not raw audio. Ignoring.\n", new_pad_type);
+ goto exit;
+ }
+
+ /* Attempt the link */
+ ret = gst_pad_link (new_pad, sink_pad);
+ if (GST_PAD_LINK_FAILED (ret)) {
+ g_print (" Type is '%s' but link failed.\n", new_pad_type);
+ } else {
+ g_print (" Link succeeded (type '%s').\n", new_pad_type);
+ }
+
+exit:
+ /* Unreference the new pad's caps, if we got them */
+ if (new_pad_caps != NULL)
+ gst_caps_unref (new_pad_caps);
+
+ /* Unreference the sink pad */
+ gst_object_unref (sink_pad);
+}
+```
+
+> 
+> Need help?
+>
+> If you need help to compile this code, refer to the **Building the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Build), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Build) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Build), or use this specific command on Linux:
+> ``gcc basic-tutorial-3.c -o basic-tutorial-3 `pkg-config --cflags --libs gstreamer-1.0` ``
+>
+>If you need help to run this code, refer to the **Running the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Run), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Run) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Run).
+>
+> This tutorial only plays audio. The media is fetched from the Internet, so it might take a few seconds to start, depending on your connection speed.
+>
+>Required libraries: `gstreamer-1.0`
+
+## Walkthrough
+
+``` c
+/* Structure to contain all our information, so we can pass it to callbacks */
+typedef struct _CustomData {
+ GstElement *pipeline;
+ GstElement *source;
+ GstElement *convert;
+ GstElement *sink;
+} CustomData;
+```
+
+So far we have kept all the information we needed (pointers
+to `GstElement`s, basically) as local variables. Since this tutorial
+(and most real applications) involves callbacks, we will group all our
+data in a structure for easier handling.
+
+``` c
+/* Handler for the pad-added signal */
+static void pad_added_handler (GstElement *src, GstPad *pad, CustomData *data);
+```
+
+This is a forward reference, to be used later.
+
+``` c
+/* Create the elements */
+data.source = gst_element_factory_make ("uridecodebin", "source");
+data.convert = gst_element_factory_make ("audioconvert", "convert");
+data.sink = gst_element_factory_make ("autoaudiosink", "sink");
+```
+
+We create the elements as usual. `uridecodebin` will internally
+instantiate all the necessary elements (sources, demuxers and decoders)
+to turn a URI into raw audio and/or video streams. It does half the work
+that `playbin` does. Since it contains demuxers, its source pads are
+not initially available and we will need to link to them on the fly.
+
+`audioconvert` is useful for converting between different audio formats,
+making sure that this example will work on any platform, since the
+format produced by the audio decoder might not be the same that the
+audio sink expects.
+
+The `autoaudiosink` is the equivalent of `autovideosink` seen in the
+previous tutorial, for audio. It will render the audio stream to the
+audio card.
+
+``` c
+if (!gst_element_link (data.convert, data.sink)) {
+ g_printerr ("Elements could not be linked.\n");
+ gst_object_unref (data.pipeline);
+ return -1;
+}
+```
+
+Here we link the converter element to the sink, but we **DO NOT** link
+them with the source, since at this point it contains no source pads. We
+just leave this branch (converter + sink) unlinked, until later on.
+
+``` c
+/* Set the URI to play */
+g_object_set (data.source, "uri", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
+```
+
+We set the URI of the file to play via a property, just like we did in
+the previous tutorial.
+
+### Signals
+
+``` c
+/* Connect to the pad-added signal */
+g_signal_connect (data.source, "pad-added", G_CALLBACK (pad_added_handler), &data);
+```
+
+`GSignals` are a crucial point in GStreamer. They allow you to be
+notified (by means of a callback) when something interesting has
+happened. Signals are identified by a name, and each `GObject` has its
+own signals.
+
+In this line, we are *attaching* to the “pad-added” signal of our source
+(an `uridecodebin` element). To do so, we use `g_signal_connect()` and
+provide the callback function to be used (`pad_added_handler`) and a
+data pointer. GStreamer does nothing with this data pointer, it just
+forwards it to the callback so we can share information with it. In this
+case, we pass a pointer to the `CustomData` structure we built specially
+for this purpose.
+
+The signals that a `GstElement` generates can be found in its
+documentation or using the `gst-inspect-1.0` tool as described in [Basic
+tutorial 10: GStreamer
+tools](sdk-basic-tutorial-gstreamer-tools.md).
+
+We are now ready to go! Just set the pipeline to the PLAYING state and
+start listening to the bus for interesting messages (like ERROR or EOS),
+just like in the previous tutorials.
+
+### The callback
+
+When our source element finally has enough information to start
+producing data, it will create source pads, and trigger the “pad-added”
+signal. At this point our callback will be
+called:
+
+``` c
+static void pad_added_handler (GstElement *src, GstPad *new_pad, CustomData *data) {
+```
+
+`src` is the `GstElement` which triggered the signal. In this example,
+it can only be the `uridecodebin`, since it is the only signal to which
+we have attached. The first parameter of a signal handler is always the object
+that has triggered it.
+
+`new_pad` is the `GstPad` that has just been added to the `src` element.
+This is usually the pad to which we want to link.
+
+`data` is the pointer we provided when attaching to the signal. In this
+example, we use it to pass the `CustomData` pointer.
+
+``` c
+GstPad *sink_pad = gst_element_get_static_pad (data->convert, "sink");
+```
+
+From `CustomData` we extract the converter element, and then retrieve
+its sink pad using `gst_element_get_static_pad ()`. This is the pad to
+which we want to link `new_pad`. In the previous tutorial we linked
+element against element, and let GStreamer choose the appropriate pads.
+Now we are going to link the pads directly.
+
+``` c
+/* If our converter is already linked, we have nothing to do here */
+if (gst_pad_is_linked (sink_pad)) {
+ g_print (" We are already linked. Ignoring.\n");
+ goto exit;
+}
+```
+
+`uridecodebin` can create as many pads as it sees fit, and for each one,
+this callback will be called. These lines of code will prevent us from
+trying to link to a new pad once we are already linked.
+
+``` c
+/* Check the new pad's type */
+new_pad_caps = gst_pad_query_caps (new_pad, NULL);
+new_pad_struct = gst_caps_get_structure (new_pad_caps, 0);
+new_pad_type = gst_structure_get_name (new_pad_struct);
+if (!g_str_has_prefix (new_pad_type, "audio/x-raw")) {
+ g_print (" It has type '%s' which is not raw audio. Ignoring.\n", new_pad_type);
+ goto exit;
+}
+```
+
+Now we will check the type of data this new pad is going to output,
+because we are only interested in pads producing audio. We have
+previously created a piece of pipeline which deals with audio (an
+`audioconvert` linked with an `autoaudiosink`), and we will not be able
+to link it to a pad producing video, for example.
+
+`gst_pad_query_caps()` retrieves the *capabilities* of the pad (this is,
+the kind of data it supports), wrapped in a `GstCaps` structure. A pad
+can offer many capabilities, and hence `GstCaps` can contain many
+`GstStructure`, each representing a different capability.
+
+Since, in this case, we know that the pad we want only had one
+capability (audio), we retrieve the first `GstStructure` with
+`gst_caps_get_structure()`.
+
+Finally, with `gst_structure_get_name()` we recover the name of the
+structure, which contains the main description of the format (its *media
+type*, actually).
+
+If the name is not `audio/x-raw`, this is not a decoded
+audio pad, and we are not interested in it.
+
+Otherwise, attempt the link:
+
+``` c
+/* Attempt the link */
+ret = gst_pad_link (new_pad, sink_pad);
+if (GST_PAD_LINK_FAILED (ret)) {
+ g_print (" Type is '%s' but link failed.\n", new_pad_type);
+} else {
+ g_print (" Link succeeded (type '%s').\n", new_pad_type);
+}
+```
+
+`gst_pad_link()` tries to link two pads. As it was the case
+with `gst_element_link()`, the link must be specified from source to
+sink, and both pads must be owned by elements residing in the same bin
+(or pipeline).
+
+And we are done! When a pad of the right kind appears, it will be
+linked to the rest of the audio-processing pipeline and execution will
+continue until ERROR or EOS. However, we will squeeze a bit more content
+from this tutorial by also introducing the concept of State.
+
+### GStreamer States
+
+We already talked a bit about states when we said that playback does not
+start until you bring the pipeline to the PLAYING state. We will
+introduce here the rest of states and their meaning. There are 4 states
+in GStreamer:
+
+| State | Description |
+|-----------|--------------------|
+| `NULL` | the NULL state or initial state of an element. |
+| `READY` | the element is ready to go to PAUSED. |
+| `PAUSED` | the element is PAUSED, it is ready to accept and process data. Sink elements however only accept one buffer and then block. |
+| `PLAYING` | the element is PLAYING, the clock is running and the data is flowing. |
+
+You can only move between adjacent ones, this is, you can't go from NULL
+to PLAYING, you have to go through the intermediate READY and PAUSED
+states. If you set the pipeline to PLAYING, though, GStreamer will make
+the intermediate transitions for you.
+
+``` c
+case GST_MESSAGE_STATE_CHANGED:
+ /* We are only interested in state-changed messages from the pipeline */
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data.pipeline)) {
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ g_print ("Pipeline state changed from %s to %s:\n",
+ gst_element_state_get_name (old_state), gst_element_state_get_name (new_state));
+ }
+ break;
+```
+
+We added this piece of code that listens to bus messages regarding state
+changes and prints them on screen to help you understand the
+transitions. Every element puts messages on the bus regarding its
+current state, so we filter them out and only listen to messages coming
+from the pipeline.
+
+Most applications only need to worry about going to PLAYING to start
+playback, then to PAUSE to perform a pause, and then back to NULL at
+program exit to free all resources.
+
+## Exercise
+
+Dynamic pad linking has traditionally been a difficult topic for a lot
+of programmers. Prove that you have achieved its mastery by
+instantiating an `autovideosink` (probably with an `videoconvert` in
+front) and link it to the demuxer when the right pad appears. Hint: You
+are already printing on screen the type of the video pads.
+
+You should now see (and hear) the same movie as in [Basic tutorial 1:
+Hello world!](sdk-basic-tutorial-hello-world.md). In
+that tutorial you used `playbin`, which is a handy element that
+automatically takes care of all the demuxing and pad linking for you.
+Most of the [Playback tutorials](sdk-playback-tutorials.md) are devoted
+to `playbin`.
+
+## Conclusion
+
+In this tutorial, you learned:
+
+ - How to be notified of events using `GSignals`
+ - How to connect `GstPad`s directly instead of their parent elements.
+ - The various states of a GStreamer element.
+
+You also combined these items to build a dynamic pipeline, which was not
+defined at program start, but was created as information regarding the
+media was available.
+
+You can now continue with the basic tutorials and learn about performing
+seeks and time-related queries in [Basic tutorial 4: Time
+management](sdk-basic-tutorial-time-management.md) or move
+to the [Playback tutorials](sdk-playback-tutorials.md), and gain more
+insight about the `playbin` element.
+
+Remember that attached to this page you should find the complete source
+code of the tutorial and any accessory files needed to build it.
+It has been a pleasure having you here, and see you soon!
--- /dev/null
+# Basic tutorial 10: GStreamer tools
+
+## Goal
+
+GStreamer comes with a set of tools which range from handy to
+absolutely essential. There is no code in this tutorial, just sit back
+and relax, and we will teach you:
+
+ - How to build and run GStreamer pipelines from the command line,
+ without using C at all!
+ - How to find out what GStreamer elements you have available and their
+ capabilities.
+ - How to discover the internal structure of media files.
+
+## Introduction
+
+These tools are available in the bin directory of the GStreamer
+binaries. You need to move to this directory to execute them, because
+it is not added to the system’s `PATH` environment variable (to avoid
+polluting it too much).
+
+Just open a terminal (or console window) and go to the `bin` directory
+of your GStreamer installation (Read again the [Installing
+GStreamer](sdk-installing.md) section to find our where this is),
+and you are ready to start typing the commands given in this tutorial.
+
+
+> 
+>
+>On Linux, though, you can use the provided
+>`/opt/gstreamer-sdk/bin/gst-sdk-shell` script to enter the
+>GStreamer SDK shell environment, in which the <code>bin</code>
+>directory is in the path. In this environment, you can use the
+>GStreamer tools from any folder.
+
+**FIXME: What is this now? Just refer to /usr/bin of the distro??**
+
+In order to allow for multiple versions of GStreamer to coexists in the
+same system, these tools are versioned, this is, a GStreamer version
+number is appended to their name. This version is based on
+GStreamer 1.0, so the tools are called `gst-launch-1.0`,
+`gst-inspect-1.0` and `gst-discoverer-1.0`
+
+## `gst-launch-1.0`
+
+This tool accepts a textual description of a pipeline, instantiates it,
+and sets it to the PLAYING state. It allows you to quickly check if a
+given pipeline works, before going through the actual implementation
+using GStreamer API calls.
+
+Bear in mind that it can only create simple pipelines. In particular, it
+can only simulate the interaction of the pipeline with the application
+up to a certain level. In any case, it is extremely handy to test
+pipelines quickly, and is used by GStreamer developers around the world
+on a daily basis.
+
+Please note that `gst-launch-1.0` is primarily a debugging tool for
+developers. You should not build applications on top of it. Instead, use
+the `gst_parse_launch()` function of the GStreamer API as an easy way to
+construct pipelines from pipeline descriptions.
+
+Although the rules to construct pipeline descriptions are very simple,
+the concatenation of multiple elements can quickly make such
+descriptions resemble black magic. Fear not, for everyone learns the
+`gst-launch-1.0` syntax, eventually.
+
+The command line for gst-launch-1.0 consists of a list of options followed
+by a PIPELINE-DESCRIPTION. Some simplified instructions are given next,
+se the complete documentation at [the reference page](gst-launch.md)
+for `gst-launch-1.0`.
+
+### Elements
+
+In simple form, a PIPELINE-DESCRIPTION is a list of element types
+separated by exclamation marks (!). Go ahead and type in the following
+command:
+
+```
+gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink
+```
+
+You should see a windows with an animated video pattern. Use CTRL+C on
+the terminal to stop the program.
+
+This instantiates a new element of type `videotestsrc` (an element which
+generates a sample video pattern), an `videoconvert` (an element
+which does raw video format conversion, making sure other elements can
+understand each other), and an `autovideosink` (a window to which video
+is rendered). Then, GStreamer tries to link the output of each element
+to the input of the element appearing on its right in the description.
+If more than one input or output Pad is available, the Pad Caps are used
+to find two compatible Pads.
+
+### Properties
+
+Properties may be appended to elements, in the form
+*property=value *(multiple properties can be specified, separated by
+spaces). Use the `gst-inspect-1.0` tool (explained next) to find out the
+available properties for an
+element.
+
+```
+gst-launch-1.0 videotestsrc pattern=11 ! videoconvert ! autovideosink
+```
+
+You should see a static video pattern, made of circles.
+
+### Named elements
+
+Elements can be named using the `name` property, in this way complex
+pipelines involving branches can be created. Names allow linking to
+elements created previously in the description, and are indispensable to
+use elements with multiple output pads, like demuxers or tees, for
+example.
+
+Named elements are referred to using their name followed by a
+dot.
+
+```
+gst-launch-1.0 videotestsrc ! videoconvert ! tee name=t ! queue ! autovideosink t. ! queue ! autovideosink
+```
+
+You should see two video windows, showing the same sample video pattern.
+If you see only one, try to move it, since it is probably on top of the
+second window.
+
+This example instantiates a `videotestsrc`, linked to a
+`videoconvert`, linked to a `tee` (Remember from [](sdk-basic-tutorial-multithreading-and-pad-availability.md) that
+a `tee` copies to each of its output pads everything coming through its
+input pad). The `tee` is named simply ‘t’ (using the `name` property)
+and then linked to a `queue` and an `autovideosink`. The same `tee` is
+referred to using ‘t.’ (mind the dot) and then linked to a second
+`queue` and a second `autovideosink`.
+
+To learn why the queues are necessary read [](sdk-basic-tutorial-multithreading-and-pad-availability.md).
+
+### Pads
+
+Instead of letting GStreamer choose which Pad to use when linking two
+elements, you may want to specify the Pads directly. You can do this by
+adding a dot plus the Pad name after the name of the element (it must be
+a named element). Learn the names of the Pads of an element by using
+the `gst-inspect-1.0` tool.
+
+This is useful, for example, when you want to retrieve one particular
+stream out of a
+demuxer:
+
+```
+gst-launch-1.0 souphttpsrc location=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm ! matroskademux name=d d.video_00 ! matroskamux ! filesink location=sintel_video.mkv
+```
+
+This fetches a media file from the internet using `souphttpsrc`, which
+is in webm format (a special kind of Matroska container, see [](sdk-basic-tutorial-concepts.md)). We
+then open the container using `matroskademux`. This media contains both
+audio and video, so `matroskademux` will create two output Pads, named
+`video_00` and `audio_00`. We link `video_00` to a `matroskamux` element
+to re-pack the video stream into a new container, and finally link it to
+a `filesink`, which will write the stream into a file named
+"sintel\_video.mkv" (the `location` property specifies the name of the
+file).
+
+All in all, we took a webm file, stripped it of audio, and generated a
+new matroska file with the video. If we wanted to keep only the
+audio:
+
+```
+gst-launch-1.0 souphttpsrc location=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm ! matroskademux name=d d.audio_00 ! vorbisparse ! matroskamux ! filesink location=sintel_audio.mka
+```
+
+The `vorbisparse` element is required to extract some information from
+the stream and put it in the Pad Caps, so the next element,
+`matroskamux`, knows how to deal with the stream. In the case of video
+this was not necessary, because `matroskademux` already extracted this
+information and added it to the Caps.
+
+Note that in the above two examples no media has been decoded or played.
+We have just moved from one container to another (demultiplexing and
+re-multiplexing again).
+
+### Caps filters
+
+When an element has more than one output pad, it might happen that the
+link to the next element is ambiguous: the next element may have more
+than one compatible input pad, or its input pad may be compatible with
+the Pad Caps of all the output pads. In these cases GStreamer will link
+using the first pad that is available, which pretty much amounts to
+saying that GStreamer will choose one output pad at random.
+
+Consider the following
+pipeline:
+
+```
+gst-launch-1.0 souphttpsrc location=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm ! matroskademux ! filesink location=test
+```
+
+This is the same media file and demuxer as in the previous example. The
+input Pad Caps of `filesink` are `ANY`, meaning that it can accept any
+kind of media. Which one of the two output pads of `matroskademux` will
+be linked against the filesink? `video_00` or `audio_00`? You cannot
+know.
+
+You can remove this ambiguity, though, by using named pads, as in the
+previous sub-section, or by using **Caps
+Filters**:
+
+```
+gst-launch-1.0 souphttpsrc location=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm ! matroskademux ! video/x-vp8 ! matroskamux ! filesink location=sintel_video.mkv
+```
+
+A Caps Filter behaves like a pass-through element which does nothing and
+only accepts media with the given Caps, effectively resolving the
+ambiguity. In this example, between `matroskademux` and `matroskamux` we
+added a `video/x-vp8` Caps Filter to specify that we are interested in
+the output pad of `matroskademux` which can produce this kind of video.
+
+To find out the Caps an element accepts and produces, use the
+`gst-inspect-1.0` tool. To find out the Caps contained in a particular file,
+use the `gst-discoverer-1.0` tool. To find out the Caps an element is
+producing for a particular pipeline, run `gst-launch-1.0` as usual, with the
+`–v` option to print Caps information.
+
+### Examples
+
+Play a media file using `playbin` (as in [](sdk-basic-tutorial-hello-world.md)):
+
+```
+gst-launch-1.0 playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm
+```
+
+A fully operation playback pipeline, with audio and video (more or less
+the same pipeline that `playbin` will create
+internally):
+
+```
+gst-launch-1.0 souphttpsrc location=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm ! matroskademux name=d ! queue ! vp8dec ! videoconvert ! autovideosink d. ! queue ! vorbisdec ! audioconvert ! audioresample ! autoaudiosink
+```
+
+A transcoding pipeline, which opens the webm container and decodes both
+streams (via uridecodebin), then re-encodes the audio and video branches
+with a different codec, and puts them back together in an Ogg container
+(just for the sake of
+it).
+
+```
+gst-launch-1.0 uridecodebin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm name=d ! queue ! theoraenc ! oggmux name=m ! filesink location=sintel.ogg d. ! queue ! audioconvert ! audioresample ! flacenc ! m.
+```
+
+A rescaling pipeline. The `videoscale` element performs a rescaling
+operation whenever the frame size is different in the input and the
+output caps. The output caps are set by the Caps Filter to
+320x200.
+
+```
+gst-launch-1.0 uridecodebin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm ! queue ! videoscale ! video/x-raw-yuv,width=320,height=200 ! videoconvert ! autovideosink
+```
+
+This short description of `gst-launch-1.0` should be enough to get you
+started. Remember that you have the [complete documentation available
+here](gst-launch.md).
+
+## `gst-inspect-1.0`
+
+This tool has three modes of operation:
+
+ - Without arguments, it lists all available elements types, this is,
+ the types you can use to instantiate new elements.
+ - With a file name as an argument, it treats the file as a GStreamer
+ plugin, tries to open it, and lists all the elements described
+ inside.
+ - With a GStreamer element name as an argument, it lists all
+ information regarding that element.
+
+Let's see an example of the third mode:
+
+```
+gst-inspect-1.0 vp8dec
+
+Factory Details:
+ Rank primary (256)
+ Long-name On2 VP8 Decoder
+ Klass Codec/Decoder/Video
+ Description Decode VP8 video streams
+ Author David Schleef <ds@entropywave.com>, Sebastian Dröge <sebastian.droege@collabora.co.uk>
+
+Plugin Details:
+ Name vpx
+ Description VP8 plugin
+ Filename /usr/lib64/gstreamer-1.0/libgstvpx.so
+ Version 1.6.4
+ License LGPL
+ Source module gst-plugins-good
+ Source release date 2016-04-14
+ Binary package Fedora GStreamer-plugins-good package
+ Origin URL http://download.fedoraproject.org
+
+GObject
+ +----GInitiallyUnowned
+ +----GstObject
+ +----GstElement
+ +----GstVideoDecoder
+ +----GstVP8Dec
+
+Pad Templates:
+ SINK template: 'sink'
+ Availability: Always
+ Capabilities:
+ video/x-vp8
+
+ SRC template: 'src'
+ Availability: Always
+ Capabilities:
+ video/x-raw
+ format: I420
+ width: [ 1, 2147483647 ]
+ height: [ 1, 2147483647 ]
+ framerate: [ 0/1, 2147483647/1 ]
+
+
+Element Flags:
+ no flags set
+
+Element Implementation:
+ Has change_state() function: gst_video_decoder_change_state
+
+Element has no clocking capabilities.
+Element has no URI handling capabilities.
+
+Pads:
+ SINK: 'sink'
+ Pad Template: 'sink'
+ SRC: 'src'
+ Pad Template: 'src'
+
+Element Properties:
+ name : The name of the object
+ flags: readable, writable
+ String. Default: "vp8dec0"
+ parent : The parent of the object
+ flags: readable, writable
+ Object of type "GstObject"
+ post-processing : Enable post processing
+ flags: readable, writable
+ Boolean. Default: false
+ post-processing-flags: Flags to control post processing
+ flags: readable, writable
+ Flags "GstVP8DecPostProcessingFlags" Default: 0x00000403, "mfqe+demacroblock+deblock"
+ (0x00000001): deblock - Deblock
+ (0x00000002): demacroblock - Demacroblock
+ (0x00000004): addnoise - Add noise
+ (0x00000400): mfqe - Multi-frame quality enhancement
+ deblocking-level : Deblocking level
+ flags: readable, writable
+ Unsigned Integer. Range: 0 - 16 Default: 4
+ noise-level : Noise level
+ flags: readable, writable
+ Unsigned Integer. Range: 0 - 16 Default: 0
+ threads : Maximum number of decoding threads
+ flags: readable, writable
+ Unsigned Integer. Range: 1 - 16 Default: 0
+```
+
+The most relevant sections are:
+
+ - Pad Templates: This lists all the kinds of Pads this
+ element can have, along with their capabilities. This is where you
+ look to find out if an element can link with another one. In this
+ case, it has only one sink pad template, accepting only
+ `video/x-vp8` (encoded video data in VP8 format) and only one source
+ pad template, producing `video/x-raw` (decoded video data).
+ - Element Properties: This lists the properties of the
+ element, along with their type and accepted values.
+
+For more information, you can check the [documentation
+page](gst-inspect.md) of `gst-inspect-1.0`.
+
+## `gst-discoverer-1.0`
+
+This tool is a wrapper around the `GstDiscoverer` object shown in [](sdk-basic-tutorial-media-information-gathering.md).
+It accepts a URI from the command line and prints all information
+regarding the media that GStreamer can extract. It is useful to find out
+what container and codecs have been used to produce the media, and
+therefore what elements you need to put in a pipeline to play it.
+
+Use `gst-discoverer-1.0 --help` to obtain the list of available options,
+which basically control the amount of verbosity of the output.
+
+Let's see an
+example:
+
+```
+gst-discoverer-1.0 https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm -v
+
+Analyzing https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm
+Done discovering https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm
+Topology:
+ container: video/webm
+ audio: audio/x-vorbis, channels=(int)2, rate=(int)48000
+ Codec:
+ audio/x-vorbis, channels=(int)2, rate=(int)48000
+ Additional info:
+ None
+ Language: en
+ Channels: 2
+ Sample rate: 48000
+ Depth: 0
+ Bitrate: 80000
+ Max bitrate: 0
+ Tags:
+ taglist, language-code=(string)en, container-format=(string)Matroska, audio-codec=(string)Vorbis, application-name=(string)ffmpeg2theora-0.24, encoder=(string)"Xiph.Org\ libVorbis\ I\ 20090709", encoder-version=(uint)0, nominal-bitrate=(uint)80000, bitrate=(uint)80000;
+ video: video/x-vp8, width=(int)854, height=(int)480, framerate=(fraction)25/1
+ Codec:
+ video/x-vp8, width=(int)854, height=(int)480, framerate=(fraction)25/1
+ Additional info:
+ None
+ Width: 854
+ Height: 480
+ Depth: 0
+ Frame rate: 25/1
+ Pixel aspect ratio: 1/1
+ Interlaced: false
+ Bitrate: 0
+ Max bitrate: 0
+ Tags:
+ taglist, video-codec=(string)"VP8\ video", container-format=(string)Matroska;
+
+Properties:
+ Duration: 0:00:52.250000000
+ Seekable: yes
+ Tags:
+ video codec: VP8 video
+ language code: en
+ container format: Matroska
+ application name: ffmpeg2theora-0.24
+ encoder: Xiph.Org libVorbis I 20090709
+ encoder version: 0
+ audio codec: Vorbis
+ nominal bitrate: 80000
+ bitrate: 80000
+```
+
+## Conclusion
+
+This tutorial has shown:
+
+ - How to build and run GStreamer pipelines from the command line using
+ the `gst-launch-1.0` tool.
+ - How to find out what GStreamer elements you have available and their
+ capabilities, using the `gst-inspect-1.0` tool.
+ - How to discover the internal structure of media files, using
+ `gst-discoverer-1.0`.
+
+It has been a pleasure having you here, and see you soon!
--- /dev/null
+# Basic tutorial 14: Handy elements
+
+## Goal
+
+This tutorial gives a list of handy GStreamer elements that are worth
+knowing. They range from powerful all-in-one elements that allow you to
+build complex pipelines easily (like `playbin`), to little helper
+elements which are extremely useful when debugging.
+
+For simplicity, the following examples are given using the
+`gst-launch-1.0` tool (Learn about it in
+[](sdk-basic-tutorial-gstreamer-tools.md)). Use the `-v` command line
+parameter if you want to see the Pad Caps that are being negotiated.
+
+## Bins
+
+These are Bin elements which you treat as a single element and they take
+care of instantiating all the necessary internal pipeline to accomplish
+their task.
+
+### `playbin`
+
+This element has been extensively used throughout the tutorials. It
+manages all aspects of media playback, from source to display, passing
+through demuxing and decoding. It is so flexible and has so many options
+that a whole set of tutorials are devoted to it. See the [](sdk-playback-tutorials.md) for more details.
+
+### `uridecodebin`
+
+This element decodes data from a URI into raw media. It selects a source
+element that can handle the given URI scheme and connects it to
+a `decodebin` element. It acts like a demuxer, so it offers as many
+source pads as streams are found in the
+media.
+
+``` bash
+gst-launch-1.0 uridecodebin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm ! videoconvert ! autovideosink
+```
+
+``` bash
+gst-launch-1.0 uridecodebin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm ! audioconvert ! autoaudiosink
+```
+
+### `decodebin`
+
+This element automatically constructs a decoding pipeline using
+available decoders and demuxers via auto-plugging until raw media is
+obtained. It is used internally by `uridecodebin` which is often more
+convenient to use, as it creates a suitable source element as well. It
+replaces the old `decodebin` element. It acts like a demuxer, so it
+offers as many source pads as streams are found in the
+media.
+
+``` bash
+gst-launch-1.0 souphttpsrc location=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm ! decodebin ! autovideosink
+```
+
+## File input/output
+
+### `filesrc`
+
+This element reads a local file and produces media with `ANY` Caps. If
+you want to obtain the correct Caps for the media, explore the stream by
+using a `typefind` element or by setting the `typefind` property
+of `filesrc` to
+`TRUE`.
+
+``` c
+gst-launch-1.0 filesrc location=f:\\media\\sintel\\sintel_trailer-480p.webm ! decodebin ! autovideosink
+```
+
+### `filesink`
+
+This element writes to a file all the media it receives. Use the
+`location` property to specify the file
+name.
+
+```
+gst-launch-1.0 audiotestsrc ! vorbisenc ! oggmux ! filesink location=test.ogg
+```
+
+## Network
+
+### `souphttpsrc`
+
+This element receives data as a client over the network via HTTP using
+the [libsoup](https://wiki.gnome.org/Projects/libsoup) library. Set the URL to retrieve through the `location`
+property.
+
+``` bash
+gst-launch-1.0 souphttpsrc location=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm ! decodebin ! autovideosink
+```
+
+## Test media generation
+
+These elements are very useful to check if other parts of the pipeline
+are working, by replacing the source by one of these test sources which
+are “guaranteed” to work.
+
+### `videotestsrc`
+
+This element produces a video pattern (selectable among many different
+options with the `pattern` property). Use it to test video pipelines.
+
+``` bash
+gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink
+```
+
+### `audiotestsrc`
+
+This element produces an audio wave (selectable among many different
+options with the `wave` property). Use it to test video pipelines.
+
+``` bash
+gst-launch-1.0 audiotestsrc ! audioconvert ! autoaudiosink
+```
+
+## Video adapters
+
+### `videoconvert`
+
+This element converts from one color space (e.g. RGB) to another one
+(e.g. YUV). It can also convert between different YUV formats (e.g.
+I420, NV12, YUY2 …) or RGB format arrangements (e.g. RGBA, ARGB, BGRA…).
+
+This is normally your first choice when solving negotiation problems.
+When not needed, because its upstream and downstream elements can
+already understand each other, it acts in pass-through mode having
+minimal impact on the performance.
+
+As a rule of thumb, always use `videoconvert` whenever you use
+elements whose Caps are unknown at design time, like `autovideosink`, or
+that can vary depending on external factors, like decoding a
+user-provided file.
+
+``` bash
+gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink
+```
+
+### `videorate`
+
+This element takes an incoming stream of time-stamped video frames and
+produces a stream that matches the source pad's frame rate. The
+correction is performed by dropping and duplicating frames, no fancy
+algorithm is used to interpolate frames.
+
+This is useful to allow elements requiring different frame rates to
+link. As with the other adapters, if it is not needed (because there is
+a frame rate on which both Pads can agree), it acts in pass-through mode
+and does not impact performance.
+
+It is therefore a good idea to always use it whenever the actual frame
+rate is unknown at design time, just in
+case.
+
+``` c
+gst-launch-1.0 videotestsrc ! video/x-raw,framerate=30/1 ! videorate ! video/x-raw,framerate=1/1 ! videoconvert ! autovideosink
+```
+
+### `videoscale`
+
+This element resizes video frames. By default the element tries to
+negotiate to the same size on the source and sink Pads so that no
+scaling is needed. It is therefore safe to insert this element in a
+pipeline to get more robust behavior without any cost if no scaling is
+needed.
+
+This element supports a wide range of color spaces including various YUV
+and RGB formats and is therefore generally able to operate anywhere in a
+pipeline.
+
+If the video is to be output to a window whose size is controlled by the
+user, it is a good idea to use a `videoscale` element, since not all
+video sinks are capable of performing scaling
+operations.
+
+``` bash
+gst-launch-1.0 uridecodebin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm ! videoscale ! video/x-raw,width=178,height=100 ! videoconvert ! autovideosink
+```
+
+## Audio adapters
+
+### `audioconvert`
+
+This element converts raw audio buffers between various possible
+formats. It supports integer to float conversion, width/depth
+conversion, signedness and endianness conversion and channel
+transformations.
+
+Like `videoconvert` does for video, you use this to solve
+negotiation problems with audio, and it is generally safe to use it
+liberally, since this element does nothing if it is not needed.
+
+``` bash
+gst-launch-1.0 audiotestsrc ! audioconvert ! autoaudiosink
+```
+
+### `audioresample`
+
+This element resamples raw audio buffers to different sampling rates
+using a configurable windowing function to enhance quality
+
+Again, use it to solve negotiation problems regarding sampling rates and
+do not fear to use it
+generously.
+
+``` bash
+gst-launch-1.0 uridecodebin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm ! audioresample ! audio/x-raw-float,rate=4000 ! audioconvert ! autoaudiosink
+```
+
+### `audiorate`
+
+This element takes an incoming stream of time-stamped raw audio frames
+and produces a perfect stream by inserting or dropping samples as
+needed. It does not allow the sample rate to be changed
+as `videorate` does, it just fills gaps and removes overlapped samples
+so the output stream is continuous and “clean”.
+
+It is useful in situations where the timestamps are going to be lost
+(when storing into certain file formats, for example) and the receiver
+will require all samples to be present. It is cumbersome to exemplify
+this, so no example is given.
+
+
+Most of the time, `audiorate` is not what you want.
+
+## Multithreading
+
+### `queue`
+
+Queues have been explained in [](sdk-basic-tutorial-multithreading-and-pad-availability.md). Basically, a queue performs two tasks:
+
+ - Data is queued until a selected limit is reached. Any attempt to
+ push more buffers into the queue blocks the pushing thread until
+ more space becomes available.
+ - The queue creates a new thread on the source Pad to decouple the
+ processing on sink and source Pads.
+
+Additionally, `queue` triggers signals when it is about to become empty
+or full (according to some configurable thresholds), and can be
+instructed to drop buffers instead of blocking when it is full.
+
+As a rule of thumb, prefer the simpler `queue` element
+over `queue2` whenever network buffering is not a concern to you.
+See [](sdk-basic-tutorial-multithreading-and-pad-availability.md)
+for an example.
+
+### `queue2`
+
+This element is not an evolution of `queue`. It has the same design
+goals but follows a different implementation approach, which results in
+different features. Unfortunately, it is often not easy to tell which
+queue is the best choice.
+
+`queue2` performs the two tasks listed above for `queue`, and,
+additionally, is able to store the received data (or part of it) on a
+disk file, for later retrieval. It also replaces the signals with the
+more general and convenient buffering messages described in
+[](sdk-basic-tutorial-streaming.md).
+
+As a rule of thumb, prefer `queue2` over `queue` whenever network
+buffering is a concern to you. See [](sdk-basic-tutorial-streaming.md)
+for an example (`queue2` is hidden inside `playbin`).
+
+### `multiqueue`
+
+This element provides queues for multiple streams simultaneously, and
+eases their management, by allowing some queues to grow if no data is
+being received on other streams, or by allowing some queues to drop data
+if they are not connected to anything (instead of returning an error, as
+a simpler queue would do). Additionally, it synchronizes the different
+streams, ensuring that none of them goes too far ahead of the others.
+
+This is an advanced element. It is found inside `decodebin`, but you
+will rarely need to instantiate it yourself in a normal playback
+application.
+
+### `tee`
+
+[](sdk-basic-tutorial-multithreading-and-pad-availability.md) already
+showed how to use a `tee` element, which splits data to multiple pads.
+Splitting the data flow is useful, for example, when capturing a video
+where the video is shown on the screen and also encoded and written to a
+file. Another example is playing music and hooking up a visualization
+module.
+
+One needs to use separate `queue` elements in each branch to provide
+separate threads for each branch. Otherwise a blocked dataflow in one
+branch would stall the other
+branches.
+
+```
+gst-launch-1.0 audiotestsrc ! tee name=t ! queue ! audioconvert ! autoaudiosink t. ! queue ! wavescope ! videoconvert ! autovideosink
+```
+
+## Capabilities
+
+### `capsfilter`
+[](sdk-basic-tutorial-gstreamer-tools.md) already
+explained how to use Caps filters with `gst-launch-1.0`. When building a
+pipeline programmatically, Caps filters are implemented with
+the `capsfilter` element. This element does not modify data as such,
+but enforces limitations on the data format.
+
+``` bash
+gst-launch-1.0 videotestsrc ! video/x-raw, format=GRAY8 ! videoconvert ! autovideosink
+```
+
+### `typefind`
+
+This element determines the type of media a stream contains. It applies
+typefind functions in the order of their rank. Once the type has been
+detected it sets its source Pad Caps to the found media type and emits
+the `have-type` signal.
+
+It is instantiated internally by `decodebin`, and you can use it too to
+find the media type, although you can normally use the
+`GstDiscoverer` which provides more information (as seen in
+[](sdk-basic-tutorial-media-information-gathering.md)).
+
+## Debugging
+
+### `fakesink`
+
+This sink element simply swallows any data fed to it. It is useful when
+debugging, to replace your normal sinks and rule them out of the
+equation. It can be very verbose when combined with the `-v` switch
+of `gst-launch-1.0`, so use the `silent` property to remove any unwanted
+noise.
+
+```
+gst-launch-1.0 audiotestsrc num-buffers=1000 ! fakesink sync=false
+```
+
+### `identity`
+
+This is a dummy element that passes incoming data through unmodified. It
+has several useful diagnostic functions, such as offset and timestamp
+checking, or buffer dropping. Read its documentation to learn all the
+things this seemingly harmless element can
+do.
+
+```
+gst-launch-1.0 audiotestsrc ! identity drop-probability=0.1 ! audioconvert ! autoaudiosink
+```
+
+## Conclusion
+
+This tutorial has listed a few elements which are worth knowing, due to
+their usefulness in the day-to-day work with GStreamer. Some are
+valuable for production pipelines, whereas others are only needed for
+debugging purposes.
+
+It has been a pleasure having you here, and see you soon!
--- /dev/null
+---
+short-description: The mandatory 'Hello world' example
+...
+
+# Basic tutorial 1: Hello world!
+
+## Goal
+
+Nothing better to get a first impression about a software library than
+to print “Hello World” on the screen!
+
+But since we are dealing with multimedia frameworks, we are going to
+play a video instead.
+
+Do not be scared by the amount of code below: there are only 4 lines
+which do *real* work. The rest is cleanup code, and, in C, this is
+always a bit verbose.
+
+Without further ado, get ready for your first GStreamer application...
+
+## Hello world
+
+Copy this code into a text file named `basic-tutorial-1.c` (or find it
+in the SDK installation).
+
+**basic-tutorial-1.c**
+
+{{ examples/basic-tutorial-1.c }}
+
+Compile it as described in [Installing on Linux], [Installing on Mac OS
+X] or [Installing on Windows]. If you get compilation errors,
+double-check the instructions given in those sections.
+
+If everything built fine, fire up the executable! You should see a
+window pop up, containing a video being played straight from the
+Internet, along with audio. Congratulations!
+
+> ![Information] Need help?
+>
+> If you need help to compile this code, refer to the **Building the
+> tutorials** section for your platform: [Linux], [Mac OS X] or
+> [Windows], or use this specific command on Linux:
+>
+> `` gcc basic-tutorial-1.c -o basic-tutorial-1 `pkg-config --cflags --libs gstreamer-1.0` ``
+>
+> If you need help to run this code, refer to the **Running the
+> tutorials** section for your platform: [Linux][1], [Mac OS X][2] or
+> [Windows][3].
+>
+> This tutorial opens a window and displays a movie, with accompanying
+> audio. The media is fetched from the Internet, so the window might
+> take a few seconds to appear, depending on your connection speed.
+> Also, there is no latency management (buffering), so on slow
+> connections, the movie might stop after a few seconds. See how [Basic
+> tutorial 12: Streaming] solves this issue.
+>
+> Required libraries: `gstreamer-1.0`
+
+## Walkthrough
+
+Let's review these lines of code and see what they do:
+
+``` c
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+```
+
+This must always be your first GStreamer command. Among other things,
+`gst_init()`:
+
+- Initializes all internal structures
+
+- Checks what plug-ins are available
+
+- Executes any command-line option intended for GStreamer
+
+If you always pass your command-line parameters
+`argc` and `argv` to `gst_init()` your application will automatically
+benefit from the GStreamer standard command-line options (more on this
+in [Basic tutorial 10: GStreamer tools])
+
+``` c
+ /* Build the pipeline */
+ pipeline = gst_parse_launch ("playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
+```
+
+This line is the heart of this tutorial, and exemplifies **two** key
+points: `gst_parse_launch()` and `playbin`.
+
+### gst\_parse\_launch
+
+GStreamer is a framework designed to handle multimedia flows. Media
+travels from the “source” elements (the producers), down to the “sink”
+elements (the consumers), passing through a series of intermediate
+elements performing all kinds of tasks. The set of all the
+interconnected elements is called a “pipeline”.
+
+In GStreamer you usually build the pipeline by manually assembling the
+individual elements, but, when the pipeline is easy enough, and you do
+not need any advanced features, you can take the shortcut:
+`gst_parse_launch()`.
+
+This function takes a textual representation of a pipeline and turns it
+into an actual pipeline, which is very handy. In fact, this function is
+so handy there is a tool built completely around it which you will get
+very acquainted with (see [Basic tutorial 10: GStreamer tools][Basic
+tutorial 10: GStreamer tools] to learn about `gst-launch-1.0` and the
+`gst-launch-1.0` syntax).
+
+### playbin
+
+So, what kind of pipeline are we asking `gst_parse_launch()`to build for
+us? Here enters the second key point: We are building a pipeline
+composed of a single element called `playbin`.
+
+`playbin` is a special element which acts as a source and as a sink, and
+is a whole pipeline. Internally, it creates and connects all the
+necessary elements to play your media, so you do not have to worry about
+it.
+
+It does not allow the control granularity that a manual pipeline does,
+but, still, it permits enough customization to suffice for a wide range
+of applications. Including this tutorial.
+
+In this example, we are only passing one parameter to `playbin`, which
+is the URI of the media we want to play. Try changing it to something
+else! Whether it is an `http://` or `file://` URI, `playbin` will
+instantiate the appropriate GStreamer source transparently!
+
+If you mistype the URI, or the file does not exist, or you are missing a
+plug-in, GStreamer provides several notification mechanisms, but the
+only thing we are doing in this example is exiting on error, so do not
+expect much feedback.
+
+``` c
+ /* Start playing */
+ gst_element_set_state (pipeline, GST_STATE_PLAYING);
+
+This line highlights another interesting concept: the state. Every
+GStreamer element has an associated state, which you can more or less
+think of as the Play/Pause button in your regular DVD player. For now,
+suffice to say that playback will not start unless you set the pipeline
+to the PLAYING state.
+
+In this line, `gst_element_set_state()` is setting `pipeline` (our only
+element, remember) to the PLAYING state, thus initiating playback.
+```
+
+``` c
+ /* Wait until error or EOS */
+ bus = gst_element_get_bus (pipeline);
+ gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
+```
+
+These lines will wait until an error occurs or the end of the stream is
+found. `gst_element_get_bus()` retrieves the pipeline's bus, and
+`gst_bus_timed_pop_filtered()` will block until you receive either an
+ERROR or an EOS (End-Of-Stream) through that bus. Do not worry much
+about this line, the GStreamer bus is explained in [Basic tutorial 2:
+GStreamer concepts].
+
+And that's it! From this point onwards, GStreamer takes care of
+everything. Execution will end when the media reaches its end (EOS) or
+an error is encountered (try closing the video window, or unplugging the
+network cable). The application can always be stopped by pressing
+control-C in the console.
+
+### Cleanup
+
+Before terminating the application, though, there is a couple of things
+we need to do to tidy up correctly after ourselves.
+
+``` c
+ /* Free resources */
+ if (msg != NULL)
+ gst_message_unref (msg);
+ gst_object_unref (bus);
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+```
+
+Always read the documentation of the functions you use, to know if you
+should free the objects they return after using them.
+
+In this case, `gst_bus_timed_pop_filtered()` returned a message which
+needs to be freed with `gst_message_unref()` (more about messages in
+[Basic tutorial 2: GStreamer concepts][Basic tutorial 2: GStreamer
+concepts]).
+
+`gst_element_get_bus()` added a reference to the bus that must be freed
+with `gst_object_unref()`. Setting the pipeline to the NULL state will
+make sure it frees any resources it has allocated (More about states in
+[Basic tutorial 3: Dynamic pipelines]). Finally, unreferencing the
+pipeline will destroy it, and all its contents.
+
+_______________________________________________________________________________
+
+## Conclusion
+
+And so ends your first tutorial with GStreamer. We hope its brevity
+serves as an example of how powerful this framework is!
+
+Let's recap a bit. Today we have learned:
+
+- How to initialize GStreamer using `gst_init()`.
+
+- How to quickly build a pipeline from a textual description using
+ `gst_parse_launch()`.
+
+- How to create an automatic playback pipeline using `playbin`.
+
+- How to signal GStreamer to start playback using
+ `gst_element_set_state()`.
+
+- How to sit back and relax, while GStreamer takes care of everything,
+ using `gst_element_get_bus()` and `gst_bus_timed_pop_filtered()`.
+
+The next tutorial will keep introducing more basic GStreamer elements,
+and show you how to build a pipeline manually.
+
+It has been a pleasure having you here, and see you soon!
+
+ [Installing on Linux]: sdk-installing-on-linux.md
+ [Installing on Mac OS X]: sdk-installing-on-mac-osx.md
+ [Installing on Windows]: sdk-installing-on-windows.md
+ [Information]: images/icons/emoticons/information.png
+ [Linux]: sdk-installing-on-linux.md#InstallingonLinux-Build
+ [Mac OS X]: sdk-installing-on-mac-osx.md#InstallingonMacOSX-Build
+ [Windows]: sdk-installing-on-windows.md#InstallingonWindows-Build
+ [1]: sdk-installing-on-linux.md#InstallingonLinux-Run
+ [2]: sdk-installing-on-mac-osx.md#InstallingonMacOSX-Run
+ [3]: sdk-installing-on-windows.md#InstallingonWindows-Run
+ [Basic tutorial 12: Streaming]: sdk-basic-tutorial-streaming.md
+ [Basic tutorial 10: GStreamer tools]: sdk-basic-tutorial-gstreamer-tools.md
+ [Basic tutorial 2: GStreamer concepts]: sdk-basic-tutorial-concepts.md
+ [Basic tutorial 3: Dynamic pipelines]: sdk-basic-tutorial-dynamic-pipelines.md
--- /dev/null
+# Basic tutorial 6: Media formats and Pad Capabilities
+
+## Goal
+
+Pad Capabilities are a fundamental element of GStreamer, although most
+of the time they are invisible because the framework handles them
+automatically. This somewhat theoretical tutorial shows:
+
+ - What are Pad Capabilities.
+
+ - How to retrieve them.
+
+ - When to retrieve them.
+
+ - Why you need to know about them.
+
+## Introduction
+
+### Pads
+
+As it has already been shown, Pads allow information to enter and leave
+an element. The *Capabilities* (or *Caps*, for short) of a Pad, then,
+specify what kind of information can travel through the Pad. For
+example, “RGB video with a resolution of 320x200 pixels and 30 frames
+per second”, or “16-bits per sample audio, 5.1 channels at 44100 samples
+per second”, or even compressed formats like mp3 or h264.
+
+Pads can support multiple Capabilities (for example, a video sink can
+support video in different types of RGB or YUV formats) and Capabilities can be
+specified as *ranges* (for example, an audio sink can support samples
+rates from 1 to 48000 samples per second). However, the actual
+information traveling from Pad to Pad must have only one well-specified
+type. Through a process known as *negotiation*, two linked Pads agree on
+a common type, and thus the Capabilities of the Pads become *fixed*
+(they only have one type and do not contain ranges). The walkthrough of
+the sample code below should make all this clear.
+
+**In order for two elements to be linked together, they must share a
+common subset of Capabilities** (Otherwise they could not possibly
+understand each other). This is the main goal of Capabilities.
+
+As an application developer, you will usually build pipelines by linking
+elements together (to a lesser extent if you use all-in-all elements
+like `playbin`). In this case, you need to know the *Pad Caps* (as they
+are familiarly referred to) of your elements, or, at least, know what
+they are when GStreamer refuses to link two elements with a negotiation
+error.
+
+### Pad templates
+
+Pads are created from *Pad Templates*, which indicate all possible
+Capabilities a Pad could ever have. Templates are useful to create several
+similar Pads, and also allow early refusal of connections between
+elements: If the Capabilities of their Pad Templates do not have a
+common subset (their *intersection* is empty), there is no need to
+negotiate further.
+
+Pad Templates can be viewed as the first step in the negotiation
+process. As the process evolves, actual Pads are instantiated and their
+Capabilities refined until they are fixed (or negotiation fails).
+
+### Capabilities examples
+
+```
+SINK template: 'sink'
+ Availability: Always
+ Capabilities:
+ audio/x-raw
+ format: S16LE
+ rate: [ 1, 2147483647 ]
+ channels: [ 1, 2 ]
+ audio/x-raw
+ format: U8
+ rate: [ 1, 2147483647 ]
+ channels: [ 1, 2 ]
+```
+
+This pad is a sink which is always available on the element (we will not
+talk about availability for now). It supports two kinds of media, both
+raw audio in integer format (`audio/x-raw`): signed, 16-bit little endian and
+unsigned 8-bit. The square brackets indicate a range: for instance, the
+number of channels varies from 1 to 2.
+
+```
+SRC template: 'src'
+ Availability: Always
+ Capabilities:
+ video/x-raw
+ width: [ 1, 2147483647 ]
+ height: [ 1, 2147483647 ]
+ framerate: [ 0/1, 2147483647/1 ]
+ format: { I420, NV12, NV21, YV12, YUY2, Y42B, Y444, YUV9, YVU9, Y41B, Y800, Y8, GREY, Y16 , UYVY, YVYU, IYU1, v308, AYUV, A420 }
+```
+
+`video/x-raw` indicates that this source pad outputs raw video. It
+supports a wide range of dimensions and framerates, and a set of YUV
+formats (The curly braces indicate a *list*). All these formats
+indicate different packing and subsampling of the image planes.
+
+### Last remarks
+
+You can use the `gst-inspect-1.0` tool described in [Basic tutorial 10:
+GStreamer tools](sdk-basic-tutorial-gstreamer-tools.md) to
+learn about the Caps of any GStreamer element.
+
+Bear in mind that some elements query the underlying hardware for
+supported formats and offer their Pad Caps accordingly (They usually do
+this when entering the READY state or higher). Therefore, the shown caps
+can vary from platform to platform, or even from one execution to the
+next (even though this case is rare).
+
+This tutorial instantiates two elements (this time, through their
+factories), shows their Pad Templates, links them and sets the pipeline
+to play. On each state change, the Capabilities of the sink element's
+Pad are shown, so you can observe how the negotiation proceeds until the
+Pad Caps are fixed.
+
+## A trivial Pad Capabilities Example
+
+Copy this code into a text file named `basic-tutorial-6.c` (or find it
+in the SDK installation).
+
+**basic-tutorial-6.c**
+
+``` c
+#include <gst/gst.h>
+
+/* Functions below print the Capabilities in a human-friendly format */
+static gboolean print_field (GQuark field, const GValue * value, gpointer pfx) {
+ gchar *str = gst_value_serialize (value);
+
+ g_print ("%s %15s: %s\n", (gchar *) pfx, g_quark_to_string (field), str);
+ g_free (str);
+ return TRUE;
+}
+
+static void print_caps (const GstCaps * caps, const gchar * pfx) {
+ guint i;
+
+ g_return_if_fail (caps != NULL);
+
+ if (gst_caps_is_any (caps)) {
+ g_print ("%sANY\n", pfx);
+ return;
+ }
+ if (gst_caps_is_empty (caps)) {
+ g_print ("%sEMPTY\n", pfx);
+ return;
+ }
+
+ for (i = 0; i < gst_caps_get_size (caps); i++) {
+ GstStructure *structure = gst_caps_get_structure (caps, i);
+
+ g_print ("%s%s\n", pfx, gst_structure_get_name (structure));
+ gst_structure_foreach (structure, print_field, (gpointer) pfx);
+ }
+}
+
+/* Prints information about a Pad Template, including its Capabilities */
+static void print_pad_templates_information (GstElementFactory * factory) {
+ const GList *pads;
+ GstStaticPadTemplate *padtemplate;
+
+ g_print ("Pad Templates for %s:\n", gst_element_factory_get_longname (factory));
+ if (!gst_element_factory_get_num_pad_templates (factory)) {
+ g_print (" none\n");
+ return;
+ }
+
+ pads = gst_element_factory_get_static_pad_templates (factory);
+ while (pads) {
+ padtemplate = pads->data
+ pads = g_list_next (pads);
+
+ if (padtemplate->direction == GST_PAD_SRC)
+ g_print (" SRC template: '%s'\n", padtemplate->name_template);
+ else if (padtemplate->direction == GST_PAD_SINK)
+ g_print (" SINK template: '%s'\n", padtemplate->name_template);
+ else
+ g_print (" UNKNOWN!!! template: '%s'\n", padtemplate->name_template);
+
+ if (padtemplate->presence == GST_PAD_ALWAYS)
+ g_print (" Availability: Always\n");
+ else if (padtemplate->presence == GST_PAD_SOMETIMES)
+ g_print (" Availability: Sometimes\n");
+ else if (padtemplate->presence == GST_PAD_REQUEST) {
+ g_print (" Availability: On request\n");
+ } else
+ g_print (" Availability: UNKNOWN!!!\n");
+
+ if (padtemplate->static_caps.string) {
+ GstCaps *caps;
+ g_print (" Capabilities:\n");
+ caps = gst_static_caps_get (&padtemplate->static_caps);
+ print_caps (caps, " ");
+ gst_caps_unref (caps);
+
+ }
+
+ g_print ("\n");
+ }
+}
+
+/* Shows the CURRENT capabilities of the requested pad in the given element */
+static void print_pad_capabilities (GstElement *element, gchar *pad_name) {
+ GstPad *pad = NULL;
+ GstCaps *caps = NULL;
+
+ /* Retrieve pad */
+ pad = gst_element_get_static_pad (element, pad_name);
+ if (!pad) {
+ g_printerr ("Could not retrieve pad '%s'\n", pad_name);
+ return;
+ }
+
+ /* Retrieve negotiated caps (or acceptable caps if negotiation is not finished yet) */
+ caps = gst_pad_get_current_caps (pad);
+ if (!caps)
+ caps = gst_pad_query_caps (pad, NULL);
+
+ /* Print and free */
+ g_print ("Caps for the %s pad:\n", pad_name);
+ print_caps (caps, " ");
+ gst_caps_unref (caps);
+ gst_object_unref (pad);
+}
+
+int main(int argc, char *argv[]) {
+ GstElement *pipeline, *source, *sink;
+ GstElementFactory *source_factory, *sink_factory;
+ GstBus *bus;
+ GstMessage *msg;
+ GstStateChangeReturn ret;
+ gboolean terminate = FALSE;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Create the element factories */
+ source_factory = gst_element_factory_find ("audiotestsrc");
+ sink_factory = gst_element_factory_find ("autoaudiosink");
+ if (!source_factory || !sink_factory) {
+ g_printerr ("Not all element factories could be created.\n");
+ return -1;
+ }
+
+ /* Print information about the pad templates of these factories */
+ print_pad_templates_information (source_factory);
+ print_pad_templates_information (sink_factory);
+
+ /* Ask the factories to instantiate actual elements */
+ source = gst_element_factory_create (source_factory, "source");
+ sink = gst_element_factory_create (sink_factory, "sink");
+
+ /* Create the empty pipeline */
+ pipeline = gst_pipeline_new ("test-pipeline");
+
+ if (!pipeline || !source || !sink) {
+ g_printerr ("Not all elements could be created.\n");
+ return -1;
+ }
+
+ /* Build the pipeline */
+ gst_bin_add_many (GST_BIN (pipeline), source, sink, NULL);
+ if (gst_element_link (source, sink) != TRUE) {
+ g_printerr ("Elements could not be linked.\n");
+ gst_object_unref (pipeline);
+ return -1;
+ }
+
+ /* Print initial negotiated caps (in NULL state) */
+ g_print ("In NULL state:\n");
+ print_pad_capabilities (sink, "sink");
+
+ /* Start playing */
+ ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
+ if (ret == GST_STATE_CHANGE_FAILURE) {
+ g_printerr ("Unable to set the pipeline to the playing state (check the bus for error messages).\n");
+ }
+
+ /* Wait until error, EOS or State Change */
+ bus = gst_element_get_bus (pipeline);
+ do {
+ msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS |
+ GST_MESSAGE_STATE_CHANGED);
+
+ /* Parse message */
+ if (msg != NULL) {
+ GError *err;
+ gchar *debug_info;
+
+ switch (GST_MESSAGE_TYPE (msg)) {
+ case GST_MESSAGE_ERROR:
+ gst_message_parse_error (msg, &err, &debug_info);
+ g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
+ g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
+ g_clear_error (&err);
+ g_free (debug_info);
+ terminate = TRUE;
+ break;
+ case GST_MESSAGE_EOS:
+ g_print ("End-Of-Stream reached.\n");
+ terminate = TRUE;
+ break;
+ case GST_MESSAGE_STATE_CHANGED:
+ /* We are only interested in state-changed messages from the pipeline */
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (pipeline)) {
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ g_print ("\nPipeline state changed from %s to %s:\n",
+ gst_element_state_get_name (old_state), gst_element_state_get_name (new_state));
+ /* Print the current capabilities of the sink element */
+ print_pad_capabilities (sink, "sink");
+ }
+ break;
+ default:
+ /* We should not reach here because we only asked for ERRORs, EOS and STATE_CHANGED */
+ g_printerr ("Unexpected message received.\n");
+ break;
+ }
+ gst_message_unref (msg);
+ }
+ } while (!terminate);
+
+ /* Free resources */
+ gst_object_unref (bus);
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+ gst_object_unref (source_factory);
+ gst_object_unref (sink_factory);
+ return 0;
+}
+```
+
+> 
+> Need help?
+>
+> If you need help to compile this code, refer to the **Building the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Build), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Build) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Build), or use this specific command on Linux:
+>
+> `` gcc basic-tutorial-6.c -o basic-tutorial-6 `pkg-config --cflags --libs gstreamer-1.0` ``
+>
+>If you need help to run this code, refer to the **Running the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Run), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Run) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Run).
+>
+> This tutorial simply displays information regarding the Pad Capabilities in different time instants.
+>
+> Required libraries: `gstreamer-1.0`
+
+## Walkthrough
+
+The `print_field`, `print_caps` and `print_pad_templates` simply
+display, in a human-friendly format, the capabilities structures. If you
+want to learn about the internal organization of the
+`GstCaps` structure, read the `GStreamer Documentation` regarding Pad
+Caps.
+
+``` c
+/* Shows the CURRENT capabilities of the requested pad in the given element */
+static void print_pad_capabilities (GstElement *element, gchar *pad_name) {
+ GstPad *pad = NULL;
+ GstCaps *caps = NULL;
+
+ /* Retrieve pad */
+ pad = gst_element_get_static_pad (element, pad_name);
+ if (!pad) {
+ g_printerr ("Could not retrieve pad '%s'\n", pad_name);
+ return;
+ }
+
+ /* Retrieve negotiated caps (or acceptable caps if negotiation is not finished yet) */
+ caps = gst_pad_get_current_caps (pad);
+ if (!caps)
+ caps = gst_pad_query_caps (pad, NULL);
+
+ /* Print and free */
+ g_print ("Caps for the %s pad:\n", pad_name);
+ print_caps (caps, " ");
+ gst_caps_unref (caps);
+ gst_object_unref (pad);
+}
+```
+
+`gst_element_get_static_pad()` retrieves the named Pad from the given
+element. This Pad is *static* because it is always present in the
+element. To know more about Pad availability read the `GStreamer
+documentation` about Pads.
+
+Then we call `gst_pad_get_current_caps()` to retrieve the Pad's
+current Capabilities, which can be fixed or not, depending on the state
+of the negotiation process. They could even be non-existent, in which
+case, we call `gst_pad_query_caps()` to retrieve the currently
+acceptable Pad Capabilities. The currently acceptable Caps will be the
+Pad Template's Caps in the NULL state, but might change in later states,
+as the actual hardware Capabilities might be queried.
+
+We then print these Capabilities.
+
+``` c
+/* Create the element factories */
+source_factory = gst_element_factory_find ("audiotestsrc");
+sink_factory = gst_element_factory_find ("autoaudiosink");
+if (!source_factory || !sink_factory) {
+ g_printerr ("Not all element factories could be created.\n");
+ return -1;
+}
+
+/* Print information about the pad templates of these factories */
+print_pad_templates_information (source_factory);
+print_pad_templates_information (sink_factory);
+
+/* Ask the factories to instantiate actual elements */
+source = gst_element_factory_create (source_factory, "source");
+sink = gst_element_factory_create (sink_factory, "sink");
+```
+
+In the previous tutorials we created the elements directly using
+`gst_element_factory_make()` and skipped talking about factories, but we
+will do now. A `GstElementFactory` is in charge of instantiating a
+particular type of element, identified by its factory name.
+
+You can use `gst_element_factory_find()` to create a factory of type
+“videotestsrc”, and then use it to instantiate multiple “videotestsrc”
+elements using `gst_element_factory_create()`.
+`gst_element_factory_make()` is really a shortcut for
+`gst_element_factory_find()`+ `gst_element_factory_create()`.
+
+The Pad Templates can already be accessed through the factories, so they
+are printed as soon as the factories are created.
+
+We skip the pipeline creation and start, and go to the State-Changed
+message handling:
+
+``` c
+case GST_MESSAGE_STATE_CHANGED:
+ /* We are only interested in state-changed messages from the pipeline */
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (pipeline)) {
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ g_print ("\nPipeline state changed from %s to %s:\n",
+ gst_element_state_get_name (old_state), gst_element_state_get_name (new_state));
+ /* Print the current capabilities of the sink element */
+ print_pad_capabilities (sink, "sink");
+ }
+ break;
+```
+
+This simply prints the current Pad Caps every time the state of the
+pipeline changes. You should see, in the output, how the initial caps
+(the Pad Template's Caps) are progressively refined until they are
+completely fixed (they contain a single type with no ranges).
+
+## Conclusion
+
+This tutorial has shown:
+
+ - What are Pad Capabilities and Pad Template Capabilities.
+
+ - How to retrieve them
+ with `gst_pad_get_current_caps()` or `gst_pad_query_caps()`.
+
+ - That they have different meaning depending on the state of the
+ pipeline (initially they indicate all the possible Capabilities,
+ later they indicate the currently negotiated Caps for the Pad).
+
+ - That Pad Caps are important to know beforehand if two elements can
+ be linked together.
+
+ - That Pad Caps can be found using the `gst-inspect-1.0` tool described
+ in [Basic tutorial 10: GStreamer
+ tools](sdk-basic-tutorial-gstreamer-tools.md).
+
+Next tutorial shows how data can be manually injected into and extracted
+from the GStreamer pipeline.
+
+Remember that attached to this page you should find the complete source
+code of the tutorial and any accessory files needed to build it.
+It has been a pleasure having you here, and see you soon!
--- /dev/null
+# Basic tutorial 9: Media information gathering
+
+## Goal
+
+Sometimes you might want to quickly find out what kind of media a file
+(or URI) contains, or if you will be able to play the media at all. You
+can build a pipeline, set it to run, and watch the bus messages, but
+GStreamer has a utility that does just that for you. This tutorial
+shows:
+
+ - How to recover information regarding a URI
+
+ - How to find out if a URI is playable
+
+## Introduction
+
+`GstDiscoverer` is a utility object found in the `pbutils` library
+(Plug-in Base utilities) that accepts a URI or list of URIs, and returns
+information about them. It can work in synchronous or asynchronous
+modes.
+
+In synchronous mode, there is only a single function to call,
+`gst_discoverer_discover_uri()`, which blocks until the information is
+ready. Due to this blocking, it is usually less interesting for
+GUI-based applications and the asynchronous mode is used, as described
+in this tutorial.
+
+The recovered information includes codec descriptions, stream topology
+(number of streams and sub-streams) and available metadata (like the
+audio language).
+
+As an example, this is the result
+of discovering https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel\_trailer-480p.webm
+
+ Duration: 0:00:52.250000000
+ Tags:
+ video codec: On2 VP8
+ language code: en
+ container format: Matroska
+ application name: ffmpeg2theora-0.24
+ encoder: Xiph.Org libVorbis I 20090709
+ encoder version: 0
+ audio codec: Vorbis
+ nominal bitrate: 80000
+ bitrate: 80000
+ Seekable: yes
+ Stream information:
+ container: WebM
+ audio: Vorbis
+ Tags:
+ language code: en
+ container format: Matroska
+ audio codec: Vorbis
+ application name: ffmpeg2theora-0.24
+ encoder: Xiph.Org libVorbis I 20090709
+ encoder version: 0
+ nominal bitrate: 80000
+ bitrate: 80000
+ video: VP8
+ Tags:
+ video codec: VP8 video
+ container format: Matroska
+
+The following code tries to discover the URI provided through the
+command line, and outputs the retrieved information (If no URI is
+provided it uses a default one).
+
+This is a simplified version of what the `gst-discoverer-1.0` tool does
+([](sdk-basic-tutorial-gstreamer-tools.md)), which is
+an application that only displays data, but does not perform any
+playback.
+
+## The GStreamer Discoverer
+
+Copy this code into a text file named `basic-tutorial-9.c` (or find it
+in the SDK installation).
+
+**basic-tutorial-9.c**
+
+``` c
+#include <string.h>
+#include <gst/gst.h>
+#include <gst/pbutils/pbutils.h>
+
+/* Structure to contain all our information, so we can pass it around */
+typedef struct _CustomData {
+ GstDiscoverer *discoverer;
+ GMainLoop *loop;
+} CustomData;
+
+/* Print a tag in a human-readable format (name: value) */
+static void print_tag_foreach (const GstTagList *tags, const gchar *tag, gpointer user_data) {
+ GValue val = { 0, };
+ gchar *str;
+ gint depth = GPOINTER_TO_INT (user_data);
+
+ gst_tag_list_copy_value (&val, tags, tag);
+
+ if (G_VALUE_HOLDS_STRING (&val))
+ str = g_value_dup_string (&val);
+ else
+ str = gst_value_serialize (&val);
+
+ g_print ("%*s%s: %s\n", 2 * depth, " ", gst_tag_get_nick (tag), str);
+ g_free (str);
+
+ g_value_unset (&val);
+}
+
+/* Print information regarding a stream */
+static void print_stream_info (GstDiscovererStreamInfo *info, gint depth) {
+ gchar *desc = NULL;
+ GstCaps *caps;
+ const GstTagList *tags;
+
+ caps = gst_discoverer_stream_info_get_caps (info);
+
+ if (caps) {
+ if (gst_caps_is_fixed (caps))
+ desc = gst_pb_utils_get_codec_description (caps);
+ else
+ desc = gst_caps_to_string (caps);
+ gst_caps_unref (caps);
+ }
+
+ g_print ("%*s%s: %s\n", 2 * depth, " ", gst_discoverer_stream_info_get_stream_type_nick (info), (desc ? desc : ""));
+
+ if (desc) {
+ g_free (desc);
+ desc = NULL;
+ }
+
+ tags = gst_discoverer_stream_info_get_tags (info);
+ if (tags) {
+ g_print ("%*sTags:\n", 2 * (depth + 1), " ");
+ gst_tag_list_foreach (tags, print_tag_foreach, GINT_TO_POINTER (depth + 2));
+ }
+}
+
+/* Print information regarding a stream and its substreams, if any */
+static void print_topology (GstDiscovererStreamInfo *info, gint depth) {
+ GstDiscovererStreamInfo *next;
+
+ if (!info)
+ return;
+
+ print_stream_info (info, depth);
+
+ next = gst_discoverer_stream_info_get_next (info);
+ if (next) {
+ print_topology (next, depth + 1);
+ gst_discoverer_stream_info_unref (next);
+ } else if (GST_IS_DISCOVERER_CONTAINER_INFO (info)) {
+ GList *tmp, *streams;
+
+ streams = gst_discoverer_container_info_get_streams (GST_DISCOVERER_CONTAINER_INFO (info));
+ for (tmp = streams; tmp; tmp = tmp->next) {
+ GstDiscovererStreamInfo *tmpinf = (GstDiscovererStreamInfo *) tmp->data;
+ print_topology (tmpinf, depth + 1);
+ }
+ gst_discoverer_stream_info_list_free (streams);
+ }
+}
+
+/* This function is called every time the discoverer has information regarding
+ * one of the URIs we provided.*/
+static void on_discovered_cb (GstDiscoverer *discoverer, GstDiscovererInfo *info, GError *err, CustomData *data) {
+ GstDiscovererResult result;
+ const gchar *uri;
+ const GstTagList *tags;
+ GstDiscovererStreamInfo *sinfo;
+
+ uri = gst_discoverer_info_get_uri (info);
+ result = gst_discoverer_info_get_result (info);
+ switch (result) {
+ case GST_DISCOVERER_URI_INVALID:
+ g_print ("Invalid URI '%s'\n", uri);
+ break;
+ case GST_DISCOVERER_ERROR:
+ g_print ("Discoverer error: %s\n", err->message);
+ break;
+ case GST_DISCOVERER_TIMEOUT:
+ g_print ("Timeout\n");
+ break;
+ case GST_DISCOVERER_BUSY:
+ g_print ("Busy\n");
+ break;
+ case GST_DISCOVERER_MISSING_PLUGINS:{
+ const GstStructure *s;
+ gchar *str;
+
+ s = gst_discoverer_info_get_misc (info);
+ str = gst_structure_to_string (s);
+
+ g_print ("Missing plugins: %s\n", str);
+ g_free (str);
+ break;
+ }
+ case GST_DISCOVERER_OK:
+ g_print ("Discovered '%s'\n", uri);
+ break;
+ }
+
+ if (result != GST_DISCOVERER_OK) {
+ g_printerr ("This URI cannot be played\n");
+ return;
+ }
+
+ /* If we got no error, show the retrieved information */
+
+ g_print ("\nDuration: %" GST_TIME_FORMAT "\n", GST_TIME_ARGS (gst_discoverer_info_get_duration (info)));
+
+ tags = gst_discoverer_info_get_tags (info);
+ if (tags) {
+ g_print ("Tags:\n");
+ gst_tag_list_foreach (tags, print_tag_foreach, GINT_TO_POINTER (1));
+ }
+
+ g_print ("Seekable: %s\n", (gst_discoverer_info_get_seekable (info) ? "yes" : "no"));
+
+ g_print ("\n");
+
+ sinfo = gst_discoverer_info_get_stream_info (info);
+ if (!sinfo)
+ return;
+
+ g_print ("Stream information:\n");
+
+ print_topology (sinfo, 1);
+
+ gst_discoverer_stream_info_unref (sinfo);
+
+ g_print ("\n");
+}
+
+/* This function is called when the discoverer has finished examining
+ * all the URIs we provided.*/
+static void on_finished_cb (GstDiscoverer *discoverer, CustomData *data) {
+ g_print ("Finished discovering\n");
+
+ g_main_loop_quit (data->loop);
+}
+
+int main (int argc, char **argv) {
+ CustomData data;
+ GError *err = NULL;
+ gchar *uri = "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm";
+
+ /* if a URI was provided, use it instead of the default one */
+ if (argc > 1) {
+ uri = argv[1];
+ }
+
+ /* Initialize cumstom data structure */
+ memset (&data, 0, sizeof (data));
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ g_print ("Discovering '%s'\n", uri);
+
+ /* Instantiate the Discoverer */
+ data.discoverer = gst_discoverer_new (5 * GST_SECOND, &err);
+ if (!data.discoverer) {
+ g_print ("Error creating discoverer instance: %s\n", err->message);
+ g_clear_error (&err);
+ return -1;
+ }
+
+ /* Connect to the interesting signals */
+ g_signal_connect (data.discoverer, "discovered", G_CALLBACK (on_discovered_cb), &data);
+ g_signal_connect (data.discoverer, "finished", G_CALLBACK (on_finished_cb), &data);
+
+ /* Start the discoverer process (nothing to do yet) */
+ gst_discoverer_start (data.discoverer);
+
+ /* Add a request to process asynchronously the URI passed through the command line */
+ if (!gst_discoverer_discover_uri_async (data.discoverer, uri)) {
+ g_print ("Failed to start discovering URI '%s'\n", uri);
+ g_object_unref (data.discoverer);
+ return -1;
+ }
+
+ /* Create a GLib Main Loop and set it to run, so we can wait for the signals */
+ data.loop = g_main_loop_new (NULL, FALSE);
+ g_main_loop_run (data.loop);
+
+ /* Stop the discoverer process */
+ gst_discoverer_stop (data.discoverer);
+
+ /* Free resources */
+ g_object_unref (data.discoverer);
+ g_main_loop_unref (data.loop);
+
+ return 0;
+}
+```
+
+
+> 
+> Need help?
+>
+> If you need help to compile this code, refer to the **Building the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Build), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Build) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Build), or use this specific command on Linux:
+>
+> ``gcc basic-tutorial-9.c -o basic-tutorial-9 `pkg-config --cflags --libs gstreamer-1.0 gstreamer-pbutils-1.0` ``
+>
+>If you need help to run this code, refer to the **Running the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Run), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Run) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Run).
+>
+> This tutorial opens the URI passed as the first parameter in the command line (or a default URI if none is provided) and outputs information about it on the screen. If the media is located on the Internet, the application might take a bit to react depending on your connection speed.
+>
+> Required libraries: `gstreamer-pbutils-1.0` `gstreamer-1.0`
+
+
+## Walkthrough
+
+These are the main steps to use the `GstDiscoverer`:
+
+``` c
+/* Instantiate the Discoverer */
+data.discoverer = gst_discoverer_new (5 * GST_SECOND, &err);
+if (!data.discoverer) {
+ g_print ("Error creating discoverer instance: %s\n", err->message);
+ g_clear_error (&err);
+ return -1;
+}
+```
+
+`gst_discoverer_new()` creates a new Discoverer object. The first
+parameter is the timeout per file, in nanoseconds (use the
+`GST_SECOND` macro for simplicity).
+
+``` c
+/* Connect to the interesting signals */
+g_signal_connect (data.discoverer, "discovered", G_CALLBACK (on_discovered_cb), &data);
+g_signal_connect (data.discoverer, "finished", G_CALLBACK (on_finished_cb), &data);
+```
+
+Connect to the interesting signals, as usual. We discuss them in the
+snippet for their callbacks.
+
+``` c
+/* Start the discoverer process (nothing to do yet) */
+gst_discoverer_start (data.discoverer);
+```
+
+`gst_discoverer_start()` launches the discovering process, but we have
+not provided any URI to discover yet. This is done
+next:
+
+``` c
+/* Add a request to process asynchronously the URI passed through the command line */
+if (!gst_discoverer_discover_uri_async (data.discoverer, uri)) {
+ g_print ("Failed to start discovering URI '%s'\n", uri);
+ g_object_unref (data.discoverer);
+ return -1;
+}
+```
+
+`gst_discoverer_discover_uri_async()` enqueues the provided URI for
+discovery. Multiple URIs can be enqueued with this function. As the
+discovery process for each of them finishes, the registered callback
+functions will be fired
+up.
+
+``` c
+/* Create a GLib Main Loop and set it to run, so we can wait for the signals */
+data.loop = g_main_loop_new (NULL, FALSE);
+g_main_loop_run (data.loop);
+```
+
+The usual GLib main loop is instantiated and executed. We will get out
+of it when `g_main_loop_quit()` is called from the
+`on_finished_cb` callback.
+
+``` c
+/* Stop the discoverer process */
+gst_discoverer_stop (data.discoverer);
+```
+
+Once we are done with the discoverer, we stop it with
+`gst_discoverer_stop()` and unref it with `g_object_unref()`.
+
+Let's review now the callbacks we have
+registered:
+
+``` c
+/* This function is called every time the discoverer has information regarding
+ * one of the URIs we provided.*/
+static void on_discovered_cb (GstDiscoverer *discoverer, GstDiscovererInfo *info, GError *err, CustomData *data) {
+ GstDiscovererResult result;
+ const gchar *uri;
+ const GstTagList *tags;
+ GstDiscovererStreamInfo *sinfo;
+
+ uri = gst_discoverer_info_get_uri (info);
+ result = gst_discoverer_info_get_result (info);
+```
+
+We got here because the Discoverer has finished working on one URI, and
+provides us a `GstDiscovererInfo` structure with all the information.
+
+The first step is to retrieve the particular URI this call refers to (in
+case we had multiple discover process running, which is not the case in
+this example) with `gst_discoverer_info_get_uri()` and the discovery
+result with `gst_discoverer_info_get_result()`.
+
+``` c
+switch (result) {
+ case GST_DISCOVERER_URI_INVALID:
+ g_print ("Invalid URI '%s'\n", uri);
+ break;
+ case GST_DISCOVERER_ERROR:
+ g_print ("Discoverer error: %s\n", err->message);
+ break;
+ case GST_DISCOVERER_TIMEOUT:
+ g_print ("Timeout\n");
+ break;
+ case GST_DISCOVERER_BUSY:
+ g_print ("Busy\n");
+ break;
+ case GST_DISCOVERER_MISSING_PLUGINS:{
+ const GstStructure *s;
+ gchar *str;
+
+ s = gst_discoverer_info_get_misc (info);
+ str = gst_structure_to_string (s);
+
+ g_print ("Missing plugins: %s\n", str);
+ g_free (str);
+ break;
+ }
+ case GST_DISCOVERER_OK:
+ g_print ("Discovered '%s'\n", uri);
+ break;
+}
+
+if (result != GST_DISCOVERER_OK) {
+ g_printerr ("This URI cannot be played\n");
+ return;
+}
+```
+
+As the code shows, any result other than `GST_DISCOVERER_OK` means that
+there has been some kind of problem, and this URI cannot be played. The
+reasons can vary, but the enum values are quite explicit
+(`GST_DISCOVERER_BUSY` can only happen when in synchronous mode, which
+is not used in this example).
+
+If no error happened, information can be retrieved from the
+`GstDiscovererInfo` structure with the different
+`gst_discoverer_info_get_*` methods (like,
+`gst_discoverer_info_get_duration()`, for example).
+
+Bits of information which are made of lists, like tags and stream info,
+needs some extra parsing:
+
+``` c
+tags = gst_discoverer_info_get_tags (info);
+if (tags) {
+ g_print ("Tags:\n");
+ gst_tag_list_foreach (tags, print_tag_foreach, GINT_TO_POINTER (1));
+}
+```
+
+Tags are metadata (labels) attached to the media. They can be examined
+with `gst_tag_list_foreach()`, which will call `print_tag_foreach` for
+each tag found (the list could also be traversed manually, for example,
+or a specific tag could be searched for with
+`gst_tag_list_get_string()`). The code for `print_tag_foreach` is pretty
+much self-explicative.
+
+``` c
+sinfo = gst_discoverer_info_get_stream_info (info);
+if (!sinfo)
+ return;
+
+g_print ("Stream information:\n");
+
+print_topology (sinfo, 1);
+
+gst_discoverer_stream_info_unref (sinfo);
+```
+
+`gst_discoverer_info_get_stream_info()` returns
+a `GstDiscovererStreamInfo` structure that is parsed in
+the `print_topology` function, and then discarded
+with `gst_discoverer_stream_info_unref()`.
+
+``` c
+/* Print information regarding a stream and its substreams, if any */
+static void print_topology (GstDiscovererStreamInfo *info, gint depth) {
+ GstDiscovererStreamInfo *next;
+
+ if (!info)
+ return;
+
+ print_stream_info (info, depth);
+
+ next = gst_discoverer_stream_info_get_next (info);
+ if (next) {
+ print_topology (next, depth + 1);
+ gst_discoverer_stream_info_unref (next);
+ } else if (GST_IS_DISCOVERER_CONTAINER_INFO (info)) {
+ GList *tmp, *streams;
+
+ streams = gst_discoverer_container_info_get_streams (GST_DISCOVERER_CONTAINER_INFO (info));
+ for (tmp = streams; tmp; tmp = tmp->next) {
+ GstDiscovererStreamInfo *tmpinf = (GstDiscovererStreamInfo *) tmp->data;
+ print_topology (tmpinf, depth + 1);
+ }
+ gst_discoverer_stream_info_list_free (streams);
+ }
+}
+```
+
+The `print_stream_info` function's code is also pretty much
+self-explicative: it prints the stream's capabilities and then the
+associated caps, using `print_tag_foreach` too.
+
+Then, `print_topology` looks for the next element to display. If
+`gst_discoverer_stream_info_get_next()` returns a non-NULL stream info,
+it refers to our descendant and that should be displayed. Otherwise, if
+we are a container, recursively call `print_topology` on each of our
+children obatined with `gst_discoverer_container_info_get_streams()`.
+Otherwise, we are a final stream, and do not need to recurse (This part
+of the Discoverer API is admittedly a bit obscure).
+
+## Conclusion
+
+This tutorial has shown:
+
+ - How to recover information regarding a URI using the `GstDiscoverer`
+
+ - How to find out if a URI is playable by looking at the return code
+ obtained with `gst_discoverer_info_get_result()`.
+
+It has been a pleasure having you here, and see you soon!
--- /dev/null
+# Basic tutorial 7: Multithreading and Pad Availability
+
+## Goal
+
+GStreamer handles multithreading automatically, but, under some
+circumstances, you might need to decouple threads manually. This
+tutorial shows how to do this and, in addition, completes the exposition
+about Pad Availability. More precisely, this document explains:
+
+ - How to create new threads of execution for some parts of the
+ pipeline
+
+ - What is the Pad Availability
+
+ - How to replicate streams
+
+## Introduction
+
+### Multithreading
+
+GStreamer is a multithreaded framework. This means that, internally, it
+creates and destroys threads as it needs them, for example, to decouple
+streaming from the application thread. Moreover, plugins are also free
+to create threads for their own processing, for example, a video decoder
+could create 4 threads to take full advantage of a CPU with 4 cores.
+
+On top of this, when building the pipeline an application can specify
+explicitly that a *branch* (a part of the pipeline) runs on a different
+thread (for example, to have the audio and video decoders executing
+simultaneously).
+
+This is accomplished using the `queue` element, which works as follows.
+The sink pad just enqueues data and returns control. On a different
+thread, data is dequeued and pushed downstream. This element is also
+used for buffering, as seen later in the streaming tutorials. The size
+of the queue can be controlled through properties.
+
+### The example pipeline
+
+This example builds the following pipeline:
+
+
+
+The source is a synthetic audio signal (a continuous tone) which is
+split using a `tee` element (it sends through its source pads everything
+it receives through its sink pad). One branch then sends the signal to
+the audio card, and the other renders a video of the waveform and sends
+it to the screen.
+
+As seen in the picture, queues create a new thread, so this pipeline
+runs in 3 threads. Pipelines with more than one sink usually need to be
+multithreaded, because, to be synchronized, sinks usually block
+execution until all other sinks are ready, and they cannot get ready if
+there is only one thread, being blocked by the first sink.
+
+### Request pads
+
+In [Basic tutorial 3: Dynamic
+pipelines](sdk-basic-tutorial-dynamic-pipelines.md) we saw
+an element (`uridecodebin`) which had no pads to begin with, and they
+appeared as data started to flow and the element learned about the
+media. These are called **Sometimes Pads**, and contrast with the
+regular pads which are always available and are called **Always Pads**.
+
+The third kind of pad is the **Request Pad**, which is created on
+demand. The classical example is the `tee` element, which has one sink
+pad and no initial source pads: they need to be requested and then
+`tee` adds them. In this way, an input stream can be replicated any
+number of times. The disadvantage is that linking elements with Request
+Pads is not as automatic, as linking Always Pads, as the walkthrough for
+this example will show.
+
+Also, to request (or release) pads in the PLAYING or PAUSED states, you
+need to take additional cautions (Pad blocking) which are not described
+in this tutorial. It is safe to request (or release) pads in the NULL or
+READY states, though.
+
+Without further delay, let's see the code.
+
+## Simple multithreaded example
+
+Copy this code into a text file named `basic-tutorial-7.c` (or find it
+in the SDK installation).
+
+**basic-tutorial-7.c**
+
+``` c
+#include <gst/gst.h>
+
+int main(int argc, char *argv[]) {
+ GstElement *pipeline, *audio_source, *tee, *audio_queue, *audio_convert, *audio_resample, *audio_sink;
+ GstElement *video_queue, *visual, *video_convert, *video_sink;
+ GstBus *bus;
+ GstMessage *msg;
+ GstPadTemplate *tee_src_pad_template;
+ GstPad *tee_audio_pad, *tee_video_pad;
+ GstPad *queue_audio_pad, *queue_video_pad;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Create the elements */
+ audio_source = gst_element_factory_make ("audiotestsrc", "audio_source");
+ tee = gst_element_factory_make ("tee", "tee");
+ audio_queue = gst_element_factory_make ("queue", "audio_queue");
+ audio_convert = gst_element_factory_make ("audioconvert", "audio_convert");
+ audio_resample = gst_element_factory_make ("audioresample", "audio_resample");
+ audio_sink = gst_element_factory_make ("autoaudiosink", "audio_sink");
+ video_queue = gst_element_factory_make ("queue", "video_queue");
+ visual = gst_element_factory_make ("wavescope", "visual");
+ video_convert = gst_element_factory_make ("videoconvert", "csp");
+ video_sink = gst_element_factory_make ("autovideosink", "video_sink");
+
+ /* Create the empty pipeline */
+ pipeline = gst_pipeline_new ("test-pipeline");
+
+ if (!pipeline || !audio_source || !tee || !audio_queue || !audio_convert || !audio_resample || !audio_sink ||
+ !video_queue || !visual || !video_convert || !video_sink) {
+ g_printerr ("Not all elements could be created.\n");
+ return -1;
+ }
+
+ /* Configure elements */
+ g_object_set (audio_source, "freq", 215.0f, NULL);
+ g_object_set (visual, "shader", 0, "style", 1, NULL);
+
+ /* Link all elements that can be automatically linked because they have "Always" pads */
+ gst_bin_add_many (GST_BIN (pipeline), audio_source, tee, audio_queue, audio_convert, audio_resample, audio_sink,
+ video_queue, visual, video_convert, video_sink, NULL);
+ if (gst_element_link_many (audio_source, tee, NULL) != TRUE ||
+ gst_element_link_many (audio_queue, audio_convert, audio_resample, audio_sink, NULL) != TRUE ||
+ gst_element_link_many (video_queue, visual, video_convert, video_sink, NULL) != TRUE) {
+ g_printerr ("Elements could not be linked.\n");
+ gst_object_unref (pipeline);
+ return -1;
+ }
+
+ /* Manually link the Tee, which has "Request" pads */
+ tee_src_pad_template = gst_element_class_get_pad_template (GST_ELEMENT_GET_CLASS (tee), "src_%d");
+ tee_audio_pad = gst_element_request_pad (tee, tee_src_pad_template, NULL, NULL);
+ g_print ("Obtained request pad %s for audio branch.\n", gst_pad_get_name (tee_audio_pad));
+ queue_audio_pad = gst_element_get_static_pad (audio_queue, "sink");
+ tee_video_pad = gst_element_request_pad (tee, tee_src_pad_template, NULL, NULL);
+ g_print ("Obtained request pad %s for video branch.\n", gst_pad_get_name (tee_video_pad));
+ queue_video_pad = gst_element_get_static_pad (video_queue, "sink");
+ if (gst_pad_link (tee_audio_pad, queue_audio_pad) != GST_PAD_LINK_OK ||
+ gst_pad_link (tee_video_pad, queue_video_pad) != GST_PAD_LINK_OK) {
+ g_printerr ("Tee could not be linked.\n");
+ gst_object_unref (pipeline);
+ return -1;
+ }
+ gst_object_unref (queue_audio_pad);
+ gst_object_unref (queue_video_pad);
+
+ /* Start playing the pipeline */
+ gst_element_set_state (pipeline, GST_STATE_PLAYING);
+
+ /* Wait until error or EOS */
+ bus = gst_element_get_bus (pipeline);
+ msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
+
+ /* Release the request pads from the Tee, and unref them */
+ gst_element_release_request_pad (tee, tee_audio_pad);
+ gst_element_release_request_pad (tee, tee_video_pad);
+ gst_object_unref (tee_audio_pad);
+ gst_object_unref (tee_video_pad);
+
+ /* Free resources */
+ if (msg != NULL)
+ gst_message_unref (msg);
+ gst_object_unref (bus);
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+
+ gst_object_unref (pipeline);
+ return 0;
+}
+```
+
+> 
+> Need help?
+>
+> If you need help to compile this code, refer to the **Building the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Build), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Build) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Build), or use this specific command on Linux:
+>
+> ``gcc basic-tutorial-7.c -o basic-tutorial-7 `pkg-config --cflags --libs gstreamer-1.0` ``
+>
+>If you need help to run this code, refer to the **Running the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Run), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Run) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Run).
+>
+> This tutorial plays an audible tone through the audio card and opens a window with a waveform representation of the tone. The waveform should be a sinusoid, but due to the refreshing of the window might not appear so.
+>
+> Required libraries: `gstreamer-1.0`
+
+## Walkthrough
+
+``` c
+/* Create the elements */
+audio_source = gst_element_factory_make ("audiotestsrc", "audio_source");
+tee = gst_element_factory_make ("tee", "tee");
+audio_queue = gst_element_factory_make ("queue", "audio_queue");
+audio_convert = gst_element_factory_make ("audioconvert", "audio_convert");
+ audio_resample = gst_element_factory_make ("audioresample", "audio_resample");
+audio_sink = gst_element_factory_make ("autoaudiosink", "audio_sink");
+video_queue = gst_element_factory_make ("queue", "video_queue");
+visual = gst_element_factory_make ("wavescope", "visual");
+video_convert = gst_element_factory_make ("videoconvert", "video_convert");
+video_sink = gst_element_factory_make ("autovideosink", "video_sink");
+```
+
+All the elements in the above picture are instantiated here:
+
+`audiotestsrc` produces a synthetic tone. `wavescope` consumes an audio
+signal and renders a waveform as if it was an (admittedly cheap)
+oscilloscope. We have already worked with the `autoaudiosink` and
+`autovideosink`.
+
+The conversion elements (`audioconvert`, `audioresample` and
+`videoconvert`) are necessary to guarantee that the pipeline can be
+linked. Indeed, the Capabilities of the audio and video sinks depend on
+the hardware, and you do not know at design time if they will match the
+Caps produced by the `audiotestsrc` and `wavescope`. If the Caps
+matched, though, these elements act in “pass-through” mode and do not
+modify the signal, having negligible impact on performance.
+
+``` c
+/* Configure elements */
+g_object_set (audio_source, "freq", 215.0f, NULL);
+g_object_set (visual, "shader", 0, "style", 1, NULL);
+```
+
+Small adjustments for better demonstration: The “freq” property of
+`audiotestsrc` controls the frequency of the wave (215Hz makes the wave
+appear almost stationary in the window), and this style and shader for
+`wavescope` make the wave continuous. Use the `gst-inspect-1.0` tool
+described in [Basic tutorial 10: GStreamer
+tools](sdk-basic-tutorial-gstreamer-tools.md) to learn all
+the properties of these
+elements.
+
+``` c
+/* Link all elements that can be automatically linked because they have "Always" pads */
+gst_bin_add_many (GST_BIN (pipeline), audio_source, tee, audio_queue, audio_convert, audio_sink,
+ video_queue, visual, video_convert, video_sink, NULL);
+if (gst_element_link_many (audio_source, tee, NULL) != TRUE ||
+ gst_element_link_many (audio_queue, audio_convert, audio_sink, NULL) != TRUE ||
+ gst_element_link_many (video_queue, visual, video_convert, video_sink, NULL) != TRUE) {
+ g_printerr ("Elements could not be linked.\n");
+ gst_object_unref (pipeline);
+ return -1;
+}
+```
+
+This code block adds all elements to the pipeline and then links the
+ones that can be automatically linked (the ones with Always Pads, as the
+comment says).
+
+> 
+> `gst_element_link_many()` can actually link elements with Request Pads. It internally requests the Pads so you do not have worry about the elements being linked having Always or Request Pads. Strange as it might seem, this is actually inconvenient, because you still need to release the requested Pads afterwards, and, if the Pad was requested automatically by `gst_element_link_many()`, it is easy to forget. Stay out of trouble by always requesting Request Pads manually, as shown in the next code block.
+
+``` c
+/* Manually link the Tee, which has "Request" pads */
+tee_src_pad_template = gst_element_class_get_pad_template (GST_ELEMENT_GET_CLASS (tee), "src_%d");
+tee_audio_pad = gst_element_request_pad (tee, tee_src_pad_template, NULL, NULL);
+g_print ("Obtained request pad %s for audio branch.\n", gst_pad_get_name (tee_audio_pad));
+queue_audio_pad = gst_element_get_static_pad (audio_queue, "sink");
+tee_video_pad = gst_element_request_pad (tee, tee_src_pad_template, NULL, NULL);
+g_print ("Obtained request pad %s for video branch.\n", gst_pad_get_name (tee_video_pad));
+queue_video_pad = gst_element_get_static_pad (video_queue, "sink");
+if (gst_pad_link (tee_audio_pad, queue_audio_pad) != GST_PAD_LINK_OK ||
+ gst_pad_link (tee_video_pad, queue_video_pad) != GST_PAD_LINK_OK) {
+ g_printerr ("Tee could not be linked.\n");
+ gst_object_unref (pipeline);
+ return -1;
+}
+gst_object_unref (queue_audio_pad);
+gst_object_unref (queue_video_pad);
+```
+
+To link Request Pads, they need to be obtained by “requesting” them to
+the element. An element might be able to produce different kinds of
+Request Pads, so, when requesting them, the desired Pad Template must be
+provided. Pad templates are obtained with
+`gst_element_class_get_pad_template()` and are identified by their name.
+In the documentation for the `tee` element we see that it has two pad
+templates named “sink” (for its sink Pads) and “src_%d” (for the Request
+Pads).
+
+Once we have the Pad template, we request two Pads from the tee (for the
+audio and video branches) with `gst_element_request_pad()`.
+
+We then obtain the Pads from the downstream elements to which these
+Request Pads need to be linked. These are normal Always Pads, so we
+obtain them with `gst_element_get_static_pad()`.
+
+Finally, we link the pads with `gst_pad_link()`. This is the function
+that `gst_element_link()` and `gst_element_link_many()` use internally.
+
+The sink Pads we have obtained need to be released with
+`gst_object_unref()`. The Request Pads will be released when we no
+longer need them, at the end of the program.
+
+We then set the pipeline to playing as usual, and wait until an error
+message or an EOS is produced. The only thing left to so is cleanup the
+requested Pads:
+
+``` c
+/* Release the request pads from the Tee, and unref them */
+gst_element_release_request_pad (tee, tee_audio_pad);
+gst_element_release_request_pad (tee, tee_video_pad);
+gst_object_unref (tee_audio_pad);
+gst_object_unref (tee_video_pad);
+```
+
+`gst_element_release_request_pad()` releases the pad from the `tee`, but
+it still needs to be unreferenced (freed) with `gst_object_unref()`.
+
+## Conclusion
+
+ This tutorial has shown:
+
+ - How to make parts of a pipeline run on a different thread by using
+ `queue` elements.
+
+ - What is a Request Pad and how to link elements with request pads,
+ with `gst_element_class_get_pad_template()`, `gst_element_request_pad()`, `gst_pad_link()` and
+ `gst_element_release_request_pad()`.
+
+ - How to have the same stream available in different branches by using
+ `tee` elements.
+
+The next tutorial builds on top of this one to show how data can be
+manually injected into and extracted from a running pipeline.
+
+It has been a pleasure having you here, and see you soon!
--- /dev/null
+# Basic tutorial 16: Platform-specific elements
+
+## Goal
+
+Even though GStreamer is a multiplatform framework, not all the elements
+are available on all platforms. For example, the video sinks
+depend heavily on the underlying windowing system, and a different one
+needs to be selected depending on the platform. You normally do not need
+to worry about this when using elements like `playbin` or
+`autovideosink`, but, for those cases when you need to use one of the
+sinks that are only available on specific platforms, this tutorial hints
+you some of their peculiarities.
+
+## Cross Platform
+
+### `glimagesink`
+
+This video sink is based on
+[OpenGL](http://en.wikipedia.org/wiki/OpenGL) or [OpenGL ES](https://en.wikipedia.org/wiki/OpenGL_ES). It supports rescaling
+and filtering of the scaled image to alleviate aliasing. It implements
+the VideoOverlay interface, so the video window can be re-parented
+(embedded inside other windows). This is the video sink recommended on
+most platforms. In particular, on Android and iOS, it is the only
+available video sink. It can be decomposed into
+`glupload ! glcolorconvert ! glimagesinkelement` to insert further OpenGL
+hardware accelerated processing into the pipeline.
+
+## Linux
+
+### `ximagesink`
+
+A standard RGB only X-based video sink. It implements the VideoOverlay
+interface, so the video window can be re-parented (embedded inside
+other windows). It does not support scaling or color formats other
+than RGB; it has to be performed by different means (using the
+`videoscale` element, for example).
+
+### `xvimagesink`
+
+An X-based video sink, using the [X Video
+Extension](http://en.wikipedia.org/wiki/X_video_extension) (Xv). It
+implements the VideoOverlay interface, so the video window can be
+re-parented (embedded inside other windows). It can perform scaling
+efficiently, on the GPU. It is only available if the hardware and
+corresponding drivers support the Xv extension.
+
+### `alsasink`
+
+This audio sink outputs to the sound card via
+[ALSA](http://www.alsa-project.org/) (Advanced Linux Sound
+Architecture). This sink is available on almost every Linux platform. It
+is often seen as a “low level” interface to the sound card, and can be
+complicated to configure (See the comment on
+[](sdk-playback-tutorial-digital-audio-pass-through.md)).
+
+### `pulsesink`
+
+This sink plays audio to a [PulseAudio](http://www.pulseaudio.org/)
+server. It is a higher level abstraction of the sound card than ALSA,
+and is therefore easier to use and offers more advanced features. It has
+been known to be unstable on some older Linux distributions, though.
+
+## Mac OS X
+
+### `osxvideosink`
+
+This is the video sink available to GStreamer on Mac OS X. It is also
+possible to draw using `glimagesink` using OpenGL.
+
+### `osxaudiosink`
+
+This is the only audio sink available to GStreamer on Mac OS X.
+
+## Windows
+
+### `directdrawsink`
+
+This is the oldest of the Windows video sinks, based on [Direct
+Draw](http://en.wikipedia.org/wiki/DirectDraw). It requires DirectX 7,
+so it is available on almost every current Windows platform. It supports
+rescaling and filtering of the scaled image to alleviate aliasing.
+
+### `dshowvideosink`
+
+This video sink is based on [Direct
+Show](http://en.wikipedia.org/wiki/Direct_Show). It can use different
+rendering back-ends, like
+[EVR](http://en.wikipedia.org/wiki/Enhanced_Video_Renderer),
+[VMR9](http://en.wikipedia.org/wiki/Direct_Show#Video_rendering_filters)
+or
+[VMR7](http://en.wikipedia.org/wiki/Direct_Show#Video_rendering_filters),
+EVR only being available on Windows Vista or more recent. It supports
+rescaling and filtering of the scaled image to alleviate aliasing. It
+implements the VideoOverlay interface, so the video window can be
+re-parented (embedded inside other windows).
+
+### `d3dvideosink`
+
+This video sink is based on
+[Direct3D](http://en.wikipedia.org/wiki/Direct3D) and it’s the most
+recent Windows video sink. It supports rescaling and filtering of the
+scaled image to alleviate aliasing. It implements the VideoOverlay
+interface, so the video window can be re-parented (embedded inside other
+windows).
+
+### `directsoundsink`
+
+This is the default audio sink for Windows, based on [Direct
+Sound](http://en.wikipedia.org/wiki/DirectSound), which is available in
+all Windows versions.
+
+### `dshowdecwrapper`
+
+[Direct Show](http://en.wikipedia.org/wiki/Direct_Show) is a multimedia
+framework similar to GStreamer. They are different enough, though, so
+that their pipelines cannot be interconnected. However, through this
+element, GStreamer can benefit from the decoding elements present in
+Direct Show. `dshowdecwrapper` wraps multiple Direct Show decoders so
+they can be embedded in a GStreamer pipeline. Use the `gst-inspect-1.0` tool
+(see [](sdk-basic-tutorial-gstreamer-tools.md)) to see the
+available decoders.
+
+## Android
+
+### `openslessink`
+
+This is the only audio sink available to GStreamer on Android. It is
+based on [OpenSL ES](http://en.wikipedia.org/wiki/OpenSL_ES).
+
+### `openslessrc`
+
+This is the only audio source available to GStreamer on Android. It is
+based on [OpenSL ES](http://en.wikipedia.org/wiki/OpenSL_ES).
+
+### `androidmedia`
+
+[android.media.MediaCodec](http://developer.android.com/reference/android/media/MediaCodec.html)
+is an Android specific API to access the codecs that are available on
+the device, including hardware codecs. It is available since API level
+16 (JellyBean) and GStreamer can use it via the androidmedia plugin
+for audio and video decoding. On Android, attaching the hardware
+decoder to the `glimagesink` element can produce a high performance
+zero-copy decodebin pipeline.
+
+### `ahcsrc`
+
+This video source can capture from the cameras on Android devices, it is part
+of the androidmedia plugin and uses the [android.hardware.Camera API](https://developer.android.com/reference/android/hardware/Camera.html).
+
+## iOS
+
+### `osxaudiosink`
+
+This is the only audio sink available to GStreamer on iOS.
+
+### `iosassetsrc`
+
+Source element to read iOS assets, this is, documents stored in the
+Library (like photos, music and videos). It can be instantiated
+automatically by `playbin` when URIs use the
+`assets-library://` scheme.
+
+### `iosavassetsrc`
+
+Source element to read and decode iOS audiovisual assets, this is,
+documents stored in the Library (like photos, music and videos). It can
+be instantiated automatically by `playbin` when URIs use the
+`ipod-library://` scheme. Decoding is performed by the system, so
+dedicated hardware will be used if available.
+
+## Conclusion
+
+This tutorial has shown a few specific details about some GStreamer
+elements which are not available on all platforms. You do not have to
+worry about them when using multiplatform elements like `playbin` or
+`autovideosink`, but it is good to know their personal quirks if
+instancing them manually.
+
+It has been a pleasure having you here, and see you soon!
--- /dev/null
+# Basic tutorial 13: Playback speed
+
+## Goal
+
+Fast-forward, reverse-playback and slow-motion are all techniques
+collectively known as *trick modes* and they all have in common that
+modify the normal playback rate. This tutorial shows how to achieve
+these effects and adds frame-stepping into the deal. In particular, it
+shows:
+
+ - How to change the playback rate, faster and slower than normal,
+ forward and backwards.
+ - How to advance a video frame-by-frame
+
+## Introduction
+
+Fast-forward is the technique that plays a media at a speed higher than
+its normal (intended) speed; whereas slow-motion uses a speed lower than
+the intended one. Reverse playback does the same thing but backwards,
+from the end of the stream to the beginning.
+
+All these techniques do is change the playback rate, which is a variable
+equal to 1.0 for normal playback, greater than 1.0 (in absolute value)
+for fast modes, lower than 1.0 (in absolute value) for slow modes,
+positive for forward playback and negative for reverse playback.
+
+GStreamer provides two mechanisms to change the playback rate: Step
+Events and Seek Events. Step Events allow skipping a given amount of
+media besides changing the subsequent playback rate (only to positive
+values). Seek Events, additionally, allow jumping to any position in the
+stream and set positive and negative playback rates.
+
+In [](sdk-basic-tutorial-time-management.md) seek
+events have already been shown, using a helper function to hide their
+complexity. This tutorial explains a bit more how to use these events.
+
+Step Events are a more convenient way of changing the playback rate, due
+to the reduced number of parameters needed to create them; however,
+their implementation in GStreamer still needs a bit more polishing
+so Seek Events are used in this tutorial instead.
+**FIXME: Is that even true ???**
+
+To use these events, they are created and then passed onto the pipeline,
+where they propagate upstream until they reach an element that can
+handle them. If an event is passed onto a bin element like `playbin`,
+it will simply feed the event to all its sinks, which will result in
+multiple seeks being performed. The common approach is to retrieve one
+of `playbin`’s sinks through the `video-sink` or
+`audio-sink` properties and feed the event directly into the sink.
+
+Frame stepping is a technique that allows playing a video frame by
+frame. It is implemented by pausing the pipeline, and then sending Step
+Events to skip one frame each time.
+
+## A trick mode player
+
+Copy this code into a text file named `basic-tutorial-13.c`.
+
+**basic-tutorial-13.c**
+
+``` c
+#include <string.h>
+#include <stdio.h>
+#include <gst/gst.h>
+
+typedef struct _CustomData {
+ GstElement *pipeline;
+ GstElement *video_sink;
+ GMainLoop *loop;
+
+ gboolean playing; /* Playing or Paused */
+ gdouble rate; /* Current playback rate (can be negative) */
+} CustomData;
+
+/* Send seek event to change rate */
+static void send_seek_event (CustomData *data) {
+ gint64 position;
+ GstFormat format = GST_FORMAT_TIME;
+ GstEvent *seek_event;
+
+ /* Obtain the current position, needed for the seek event */
+ if (!gst_element_query_position (data->pipeline, &format, &position)) {
+ g_printerr ("Unable to retrieve current position.\n");
+ return;
+ }
+
+ /* Create the seek event */
+ if (data->rate > 0) {
+ seek_event = gst_event_new_seek (data->rate, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE,
+ GST_SEEK_TYPE_SET, position, GST_SEEK_TYPE_NONE, 0);
+ } else {
+ seek_event = gst_event_new_seek (data->rate, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE,
+ GST_SEEK_TYPE_SET, 0, GST_SEEK_TYPE_SET, position);
+ }
+
+ if (data->video_sink == NULL) {
+ /* If we have not done so, obtain the sink through which we will send the seek events */
+ g_object_get (data->pipeline, "video-sink", &data->video_sink, NULL);
+ }
+
+ /* Send the event */
+ gst_element_send_event (data->video_sink, seek_event);
+
+ g_print ("Current rate: %g\n", data->rate);
+}
+
+/* Process keyboard input */
+static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data) {
+ gchar *str = NULL;
+
+ if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) != G_IO_STATUS_NORMAL) {
+ return TRUE;
+ }
+
+ switch (g_ascii_tolower (str[0])) {
+ case 'p':
+ data->playing = !data->playing;
+ gst_element_set_state (data->pipeline, data->playing ? GST_STATE_PLAYING : GST_STATE_PAUSED);
+ g_print ("Setting state to %s\n", data->playing ? "PLAYING" : "PAUSE");
+ break;
+ case 's':
+ if (g_ascii_isupper (str[0])) {
+ data->rate *= 2.0;
+ } else {
+ data->rate /= 2.0;
+ }
+ send_seek_event (data);
+ break;
+ case 'd':
+ data->rate *= -1.0;
+ send_seek_event (data);
+ break;
+ case 'n':
+ if (data->video_sink == NULL) {
+ /* If we have not done so, obtain the sink through which we will send the step events */
+ g_object_get (data->pipeline, "video-sink", &data->video_sink, NULL);
+ }
+
+ gst_element_send_event (data->video_sink,
+ gst_event_new_step (GST_FORMAT_BUFFERS, 1, data->rate, TRUE, FALSE));
+ g_print ("Stepping one frame\n");
+ break;
+ case 'q':
+ g_main_loop_quit (data->loop);
+ break;
+ default:
+ break;
+ }
+
+ g_free (str);
+
+ return TRUE;
+}
+
+int main(int argc, char *argv[]) {
+ CustomData data;
+ GstStateChangeReturn ret;
+ GIOChannel *io_stdin;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Initialize our data structure */
+ memset (&data, 0, sizeof (data));
+
+ /* Print usage map */
+ g_print (
+ "USAGE: Choose one of the following options, then press enter:\n"
+ " 'P' to toggle between PAUSE and PLAY\n"
+ " 'S' to increase playback speed, 's' to decrease playback speed\n"
+ " 'D' to toggle playback direction\n"
+ " 'N' to move to next frame (in the current direction, better in PAUSE)\n"
+ " 'Q' to quit\n");
+
+ /* Build the pipeline */
+ data.pipeline = gst_parse_launch ("playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
+
+ /* Add a keyboard watch so we get notified of keystrokes */
+#ifdef G_OS_WIN32
+ io_stdin = g_io_channel_win32_new_fd (fileno (stdin));
+#else
+ io_stdin = g_io_channel_unix_new (fileno (stdin));
+#endif
+ g_io_add_watch (io_stdin, G_IO_IN, (GIOFunc)handle_keyboard, &data);
+
+ /* Start playing */
+ ret = gst_element_set_state (data.pipeline, GST_STATE_PLAYING);
+ if (ret == GST_STATE_CHANGE_FAILURE) {
+ g_printerr ("Unable to set the pipeline to the playing state.\n");
+ gst_object_unref (data.pipeline);
+ return -1;
+ }
+ data.playing = TRUE;
+ data.rate = 1.0;
+
+ /* Create a GLib Main Loop and set it to run */
+ data.loop = g_main_loop_new (NULL, FALSE);
+ g_main_loop_run (data.loop);
+
+ /* Free resources */
+ g_main_loop_unref (data.loop);
+ g_io_channel_unref (io_stdin);
+ gst_element_set_state (data.pipeline, GST_STATE_NULL);
+ if (data.video_sink != NULL)
+ gst_object_unref (data.video_sink);
+ gst_object_unref (data.pipeline);
+ return 0;
+}
+```
+
+
+> 
+> Need help?
+>
+> If you need help to compile this code, refer to the **Building the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Build), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Build) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Build), or use this specific command on Linux:
+>
+> `` gcc basic-tutorial-13.c -o basic-tutorial-13 `pkg-config --cflags --libs gstreamer-1.0` ``
+>
+>If you need help to run this code, refer to the **Running the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Run), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Run) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Run).
+>
+> This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed. The console shows the available commands, composed of a single upper-case or lower-case letter, which you should input followed by the Enter key.
+>
+> Required libraries: `gstreamer-1.0`
+
+## Walkthrough
+
+There is nothing new in the initialization code in the main function: a
+`playbin` pipeline is instantiated, an I/O watch is installed to track
+keystrokes and a GLib main loop is executed.
+
+Then, in the keyboard handler function:
+
+``` c
+/* Process keyboard input */
+static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data) {
+ gchar *str = NULL;
+
+ if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) != G_IO_STATUS_NORMAL) {
+ return TRUE;
+ }
+
+ switch (g_ascii_tolower (str[0])) {
+ case 'p':
+ data->playing = !data->playing;
+ gst_element_set_state (data->pipeline, data->playing ? GST_STATE_PLAYING : GST_STATE_PAUSED);
+ g_print ("Setting state to %s\n", data->playing ? "PLAYING" : "PAUSE");
+ break;
+```
+
+Pause / Playing toggle is handled with `gst_element_set_state()` as in
+previous tutorials.
+
+``` c
+case 's':
+ if (g_ascii_isupper (str[0])) {
+ data->rate *= 2.0;
+ } else {
+ data->rate /= 2.0;
+ }
+ send_seek_event (data);
+ break;
+case 'd':
+ data->rate *= -1.0;
+ send_seek_event (data);
+ break;
+```
+
+Use ‘S’ and ‘s’ to double or halve the current playback rate, and ‘d’ to
+reverse the current playback direction. In both cases, the
+`rate` variable is updated and `send_seek_event` is called. Let’s
+review this function.
+
+``` c
+/* Send seek event to change rate */
+static void send_seek_event (CustomData *data) {
+ gint64 position;
+ GstEvent *seek_event;
+
+ /* Obtain the current position, needed for the seek event */
+ if (!gst_element_query_position (data->pipeline, GST_FORMAT_TIME, &position)) {
+ g_printerr ("Unable to retrieve current position.\n");
+ return;
+ }
+```
+
+This function creates a new Seek Event and sends it to the pipeline to
+update the rate. First, the current position is recovered with
+`gst_element_query_position()`. This is needed because the Seek Event
+jumps to another position in the stream, and, since we do not actually
+want to move, we jump to the current position. Using a Step Event would
+be simpler, but this event is not currently fully functional, as
+explained in the Introduction.
+
+``` c
+/* Create the seek event */
+if (data->rate > 0) {
+ seek_event = gst_event_new_seek (data->rate, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE,
+ GST_SEEK_TYPE_SET, position, GST_SEEK_TYPE_NONE, 0);
+} else {
+ seek_event = gst_event_new_seek (data->rate, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE,
+ GST_SEEK_TYPE_SET, 0, GST_SEEK_TYPE_SET, position);
+}
+```
+
+The Seek Event is created with `gst_event_new_seek()`. Its parameters
+are, basically, the new rate, the new start position and the new stop
+position. Regardless of the playback direction, the start position must
+be smaller than the stop position, so the two playback directions are
+treated differently.
+
+``` c
+if (data->video_sink == NULL) {
+ /* If we have not done so, obtain the sink through which we will send the seek events */
+ g_object_get (data->pipeline, "video-sink", &data->video_sink, NULL);
+}
+```
+
+As explained in the Introduction, to avoid performing multiple Seeks,
+the Event is sent to only one sink, in this case, the video sink. It is
+obtained from `playbin` through the `video-sink` property. It is read
+at this time instead at initialization time because the actual sink may
+change depending on the media contents, and this won’t be known until
+the pipeline is PLAYING and some media has been read.
+
+``` c
+/* Send the event */
+gst_element_send_event (data->video_sink, seek_event);
+```
+
+The new Event is finally sent to the selected sink with
+`gst_element_send_event()`.
+
+Back to the keyboard handler, we still miss the frame stepping code,
+which is really simple:
+
+``` c
+case 'n':
+ if (data->video_sink == NULL) {
+ /* If we have not done so, obtain the sink through which we will send the step events */
+ g_object_get (data->pipeline, "video-sink", &data->video_sink, NULL);
+ }
+
+ gst_element_send_event (data->video_sink,
+ gst_event_new_step (GST_FORMAT_BUFFERS, 1, data->rate, TRUE, FALSE));
+ g_print ("Stepping one frame\n");
+ break;
+```
+
+A new Step Event is created with `gst_event_new_step()`, whose
+parameters basically specify the amount to skip (1 frame in the example)
+and the new rate (which we do not change).
+
+The video sink is grabbed from `playbin` in case we didn’t have it yet,
+just like before.
+
+And with this we are done. When testing this tutorial, keep in mind that
+backward playback is not optimal in many elements.
+
+> 
+>
+>Changing the playback rate might only work with local files. If you cannot modify it, try changing the URI passed to `playbin` in line 114 to a local URI, starting with `file:///`
+</table>
+
+## Conclusion
+
+This tutorial has shown:
+
+ - How to change the playback rate using a Seek Event, created with
+ `gst_event_new_seek()` and fed to the pipeline
+ with `gst_element_send_event()`.
+ - How to advance a video frame-by-frame by using Step Events, created
+ with `gst_event_new_step()`.
+
+It has been a pleasure having you here, and see you soon!
\ No newline at end of file
--- /dev/null
+# Basic tutorial 8: Short-cutting the pipeline
+
+## Goal
+
+Pipelines constructed with GStreamer do not need to be completely
+closed. Data can be injected into the pipeline and extracted from it at
+any time, in a variety of ways. This tutorial shows:
+
+ - How to inject external data into a general GStreamer pipeline.
+
+ - How to extract data from a general GStreamer pipeline.
+
+ - How to access and manipulate this data.
+
+[](sdk-playback-tutorial-short-cutting-the-pipeline.md) explains
+how to achieve the same goals in a playbin-based pipeline.
+
+## Introduction
+
+Applications can interact with the data flowing through a GStreamer
+pipeline in several ways. This tutorial describes the easiest one, since
+it uses elements that have been created for this sole purpose.
+
+The element used to inject application data into a GStreamer pipeline is
+`appsrc`, and its counterpart, used to extract GStreamer data back to
+the application is `appsink`. To avoid confusing the names, think of it
+from GStreamer's point of view: `appsrc` is just a regular source, that
+provides data magically fallen from the sky (provided by the
+application, actually). `appsink` is a regular sink, where the data
+flowing through a GStreamer pipeline goes to die (it is recovered by the
+application, actually).
+
+`appsrc` and `appsink` are so versatile that they offer their own API
+(see their documentation), which can be accessed by linking against the
+`gstreamer-app` library. In this tutorial, however, we will use a
+simpler approach and control them through signals.
+
+`appsrc` can work in a variety of modes: in **pull** mode, it requests
+data from the application every time it needs it. In **push** mode, the
+application pushes data at its own pace. Furthermore, in push mode, the
+application can choose to be blocked in the push function when enough
+data has already been provided, or it can listen to the
+`enough-data` and `need-data` signals to control flow. This example
+implements the latter approach. Information regarding the other methods
+can be found in the `appsrc` documentation.
+
+### Buffers
+
+Data travels through a GStreamer pipeline in chunks called **buffers**.
+Since this example produces and consumes data, we need to know about
+`GstBuffer`s.
+
+Source Pads produce buffers, that are consumed by Sink Pads; GStreamer
+takes these buffers and passes them from element to element.
+
+A buffer simply represents a unit of data, do not assume that all
+buffers will have the same size, or represent the same amount of time.
+Neither should you assume that if a single buffer enters an element, a
+single buffer will come out. Elements are free to do with the received
+buffers as they please. `GstBuffer`s may also contain more than one
+actual memory buffer. Actual memory buffers are abstracted away using
+`GstMemory` objects, and a `GstBuffer` can contain multiple `GstMemory` objects.
+
+Every buffer has attached time-stamps and duration, that describe in
+which moment the content of the buffer should be decoded, rendered or
+displayed. Time stamping is a very complex and delicate subject, but
+this simplified vision should suffice for now.
+
+As an example, a `filesrc` (a GStreamer element that reads files)
+produces buffers with the “ANY” caps and no time-stamping information.
+After demuxing (see [](sdk-basic-tutorial-dynamic-pipelines.md))
+buffers can have some specific caps, for example “video/x-h264”. After
+decoding, each buffer will contain a single video frame with raw caps
+(for example, “video/x-raw-yuv”) and very precise time stamps indicating
+when should that frame be displayed.
+
+### This tutorial
+
+This tutorial expands [](sdk-basic-tutorial-multithreading-and-pad-availability.md) in
+two ways: firstly, the `audiotestsrc` is replaced by an `appsrc` that
+will generate the audio data. Secondly, a new branch is added to the
+`tee` so data going into the audio sink and the wave display is also
+replicated into an `appsink`. The `appsink` uploads the information back
+into the application, which then just notifies the user that data has
+been received, but it could obviously perform more complex tasks.
+
+
+
+## A crude waveform generator
+
+Copy this code into a text file named `basic-tutorial-8.c` (or find it
+in the SDK installation).
+
+``` c
+#include <gst/gst.h>
+#include <gst/audio/audio.h>
+#include <string.h>
+
+#define CHUNK_SIZE 1024 /* Amount of bytes we are sending in each buffer */
+#define SAMPLE_RATE 44100 /* Samples per second we are sending */
+
+/* Structure to contain all our information, so we can pass it to callbacks */
+typedef struct _CustomData {
+ GstElement *pipeline, *app_source, *tee, *audio_queue, *audio_convert1, *audio_resample, *audio_sink;
+ GstElement *video_queue, *audio_convert2, *visual, *video_convert, *video_sink;
+ GstElement *app_queue, *app_sink;
+
+ guint64 num_samples; /* Number of samples generated so far (for timestamp generation) */
+ gfloat a, b, c, d; /* For waveform generation */
+
+ guint sourceid; /* To control the GSource */
+
+ GMainLoop *main_loop; /* GLib's Main Loop */
+} CustomData;
+
+/* This method is called by the idle GSource in the mainloop, to feed CHUNK_SIZE bytes into appsrc.
+ * The ide handler is added to the mainloop when appsrc requests us to start sending data (need-data signal)
+ * and is removed when appsrc has enough data (enough-data signal).
+ */
+static gboolean push_data (CustomData *data) {
+ GstBuffer *buffer;
+ GstFlowReturn ret;
+ int i;
+ GstMapInfo map;
+ gint16 *raw;
+ gint num_samples = CHUNK_SIZE / 2; /* Because each sample is 16 bits */
+ gfloat freq;
+
+ /* Create a new empty buffer */
+ buffer = gst_buffer_new_and_alloc (CHUNK_SIZE);
+
+ /* Set its timestamp and duration */
+ GST_BUFFER_TIMESTAMP (buffer) = gst_util_uint64_scale (data->num_samples, GST_SECOND, SAMPLE_RATE);
+ GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale (CHUNK_SIZE, GST_SECOND, SAMPLE_RATE);
+
+ /* Generate some psychodelic waveforms */
+ gst_buffer_map (buffer, &map, GST_MAP_WRITE);
+ raw = (gint16 *)map.data;
+ data->c += data->d;
+ data->d -= data->c / 1000;
+ freq = 1100 + 1000 * data->d;
+ for (i = 0; i < num_samples; i++) {
+ data->a += data->b;
+ data->b -= data->a / freq;
+ raw[i] = (gint16)(500 * data->a);
+ }
+ gst_buffer_unmap (buffer, &map);
+ data->num_samples += num_samples;
+
+ /* Push the buffer into the appsrc */
+ g_signal_emit_by_name (data->app_source, "push-buffer", buffer, &ret);
+
+ /* Free the buffer now that we are done with it */
+ gst_buffer_unref (buffer);
+
+ if (ret != GST_FLOW_OK) {
+ /* We got some error, stop sending data */
+ return FALSE;
+ }
+
+ return TRUE;
+}
+
+/* This signal callback triggers when appsrc needs data. Here, we add an idle handler
+ * to the mainloop to start pushing data into the appsrc */
+static void start_feed (GstElement *source, guint size, CustomData *data) {
+ if (data->sourceid == 0) {
+ g_print ("Start feeding\n");
+ data->sourceid = g_idle_add ((GSourceFunc) push_data, data);
+ }
+}
+
+/* This callback triggers when appsrc has enough data and we can stop sending.
+ * We remove the idle handler from the mainloop */
+static void stop_feed (GstElement *source, CustomData *data) {
+ if (data->sourceid != 0) {
+ g_print ("Stop feeding\n");
+ g_source_remove (data->sourceid);
+ data->sourceid = 0;
+ }
+}
+
+/* The appsink has received a buffer */
+static void new_sample (GstElement *sink, CustomData *data) {
+ GstSample *sample;
+
+ /* Retrieve the buffer */
+ g_signal_emit_by_name (sink, "pull-sample", &sample);
+ if (sample) {
+ /* The only thing we do in this example is print a * to indicate a received buffer */
+ g_print ("*");
+ gst_buffer_unref (sample);
+ }
+}
+
+/* This function is called when an error message is posted on the bus */
+static void error_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ GError *err;
+ gchar *debug_info;
+
+ /* Print error details on the screen */
+ gst_message_parse_error (msg, &err, &debug_info);
+ g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
+ g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
+ g_clear_error (&err);
+ g_free (debug_info);
+
+ g_main_loop_quit (data->main_loop);
+}
+
+int main(int argc, char *argv[]) {
+ CustomData data;
+ GstPadTemplate *tee_src_pad_template;
+ GstPad *tee_audio_pad, *tee_video_pad, *tee_app_pad;
+ GstPad *queue_audio_pad, *queue_video_pad, *queue_app_pad;
+ GstAudioInfo info;
+ GstCaps *audio_caps;
+ GstBus *bus;
+
+ /* Initialize cumstom data structure */
+ memset (&data, 0, sizeof (data));
+ data.b = 1; /* For waveform generation */
+ data.d = 1;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Create the elements */
+ data.app_source = gst_element_factory_make ("appsrc", "audio_source");
+ data.tee = gst_element_factory_make ("tee", "tee");
+ data.audio_queue = gst_element_factory_make ("queue", "audio_queue");
+ data.audio_convert1 = gst_element_factory_make ("audioconvert", "audio_convert1");
+ data.audio_resample = gst_element_factory_make ("audioresample", "audio_resample");
+ data.audio_sink = gst_element_factory_make ("autoaudiosink", "audio_sink");
+ data.video_queue = gst_element_factory_make ("queue", "video_queue");
+ data.audio_convert2 = gst_element_factory_make ("audioconvert", "audio_convert2");
+ data.visual = gst_element_factory_make ("wavescope", "visual");
+ data.video_convert = gst_element_factory_make ("videoconvert", "csp");
+ data.video_sink = gst_element_factory_make ("autovideosink", "video_sink");
+ data.app_queue = gst_element_factory_make ("queue", "app_queue");
+ data.app_sink = gst_element_factory_make ("appsink", "app_sink");
+
+ /* Create the empty pipeline */
+ data.pipeline = gst_pipeline_new ("test-pipeline");
+
+ if (!data.pipeline || !data.app_source || !data.tee || !data.audio_queue || !data.audio_convert1 ||
+ !data.audio_resample || !data.audio_sink || !data.video_queue || !data.audio_convert2 || !data.visual ||
+ !data.video_convert || !data.video_sink || !data.app_queue || !data.app_sink) {
+ g_printerr ("Not all elements could be created.\n");
+ return -1;
+ }
+
+ /* Configure wavescope */
+ g_object_set (data.visual, "shader", 0, "style", 0, NULL);
+
+ /* Configure appsrc */
+ gst_audio_info_set_format (&info, GST_AUDIO_FORMAT_S16, SAMPLE_RATE, 1, NULL);
+ audio_caps = gst_audio_info_to_caps (&info);
+ g_object_set (data.app_source, "caps", audio_caps, "format", GST_FORMAT_TIME, NULL);
+ g_signal_connect (data.app_source, "need-data", G_CALLBACK (start_feed), &data);
+ g_signal_connect (data.app_source, "enough-data", G_CALLBACK (stop_feed), &data);
+
+ /* Configure appsink */
+ g_object_set (data.app_sink, "emit-signals", TRUE, "caps", audio_caps, NULL);
+ g_signal_connect (data.app_sink, "new-sample", G_CALLBACK (new_sample), &data);
+ gst_caps_unref (audio_caps);
+ g_free (audio_caps_text);
+
+ /* Link all elements that can be automatically linked because they have "Always" pads */
+ gst_bin_add_many (GST_BIN (data.pipeline), data.app_source, data.tee, data.audio_queue, data.audio_convert1, data.audio_resample,
+ data.audio_sink, data.video_queue, data.audio_convert2, data.visual, data.video_convert, data.video_sink, data.app_queue,
+ data.app_sink, NULL);
+ if (gst_element_link_many (data.app_source, data.tee, NULL) != TRUE ||
+ gst_element_link_many (data.audio_queue, data.audio_convert1, data.audio_resample, data.audio_sink, NULL) != TRUE ||
+ gst_element_link_many (data.video_queue, data.audio_convert2, data.visual, data.video_convert, data.video_sink, NULL) != TRUE ||
+ gst_element_link_many (data.app_queue, data.app_sink, NULL) != TRUE) {
+ g_printerr ("Elements could not be linked.\n");
+ gst_object_unref (data.pipeline);
+ return -1;
+ }
+
+ /* Manually link the Tee, which has "Request" pads */
+ tee_src_pad_template = gst_element_class_get_pad_template (GST_ELEMENT_GET_CLASS (data.tee), "src_%d");
+ tee_audio_pad = gst_element_request_pad (data.tee, tee_src_pad_template, NULL, NULL);
+ g_print ("Obtained request pad %s for audio branch.\n", gst_pad_get_name (tee_audio_pad));
+ queue_audio_pad = gst_element_get_static_pad (data.audio_queue, "sink");
+ tee_video_pad = gst_element_request_pad (data.tee, tee_src_pad_template, NULL, NULL);
+ g_print ("Obtained request pad %s for video branch.\n", gst_pad_get_name (tee_video_pad));
+ queue_video_pad = gst_element_get_static_pad (data.video_queue, "sink");
+ tee_app_pad = gst_element_request_pad (data.tee, tee_src_pad_template, NULL, NULL);
+ g_print ("Obtained request pad %s for app branch.\n", gst_pad_get_name (tee_app_pad));
+ queue_app_pad = gst_element_get_static_pad (data.app_queue, "sink");
+ if (gst_pad_link (tee_audio_pad, queue_audio_pad) != GST_PAD_LINK_OK ||
+ gst_pad_link (tee_video_pad, queue_video_pad) != GST_PAD_LINK_OK ||
+ gst_pad_link (tee_app_pad, queue_app_pad) != GST_PAD_LINK_OK) {
+ g_printerr ("Tee could not be linked\n");
+ gst_object_unref (data.pipeline);
+ return -1;
+ }
+ gst_object_unref (queue_audio_pad);
+ gst_object_unref (queue_video_pad);
+ gst_object_unref (queue_app_pad);
+
+ /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
+ bus = gst_element_get_bus (data.pipeline);
+ gst_bus_add_signal_watch (bus);
+ g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, &data);
+ gst_object_unref (bus);
+
+ /* Start playing the pipeline */
+ gst_element_set_state (data.pipeline, GST_STATE_PLAYING);
+
+ /* Create a GLib Main Loop and set it to run */
+ data.main_loop = g_main_loop_new (NULL, FALSE);
+ g_main_loop_run (data.main_loop);
+
+ /* Release the request pads from the Tee, and unref them */
+ gst_element_release_request_pad (data.tee, tee_audio_pad);
+ gst_element_release_request_pad (data.tee, tee_video_pad);
+ gst_element_release_request_pad (data.tee, tee_app_pad);
+ gst_object_unref (tee_audio_pad);
+ gst_object_unref (tee_video_pad);
+ gst_object_unref (tee_app_pad);
+
+ /* Free resources */
+ gst_element_set_state (data.pipeline, GST_STATE_NULL);
+ gst_object_unref (data.pipeline);
+ return 0;
+}
+```
+
+> 
+> Need help?
+>
+> If you need help to compile this code, refer to the **Building the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Build), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Build) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Build), or use this specific command on Linux:
+>
+> `` gcc basic-tutorial-8.c -o basic-tutorial-8 `pkg-config --cflags --libs gstreamer-1.0 gst-audio-1.0` ``
+>
+>If you need help to run this code, refer to the **Running the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Run), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Run) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Run).
+>
+> This tutorial plays an audible tone for varying frequency through the audio card and opens a window with a waveform representation of the tone. The waveform should be a sinusoid, but due to the refreshing of the window might not appear so.
+>
+> Required libraries: `gstreamer-1.0`
+
+## Walkthrough
+
+The code to create the pipeline (Lines 131 to 205) is an enlarged
+version of [Basic tutorial 7: Multithreading and Pad
+Availability](sdk-basic-tutorial-multithreading-and-pad-availability.md).
+It involves instantiating all the elements, link the elements with
+Always Pads, and manually link the Request Pads of the `tee` element.
+
+Regarding the configuration of the `appsrc` and `appsink` elements:
+
+``` c
+/* Configure appsrc */
+audio_caps_text = g_strdup_printf (AUDIO_CAPS, SAMPLE_RATE);
+audio_caps = gst_caps_from_string (audio_caps_text);
+g_object_set (data.app_source, "caps", audio_caps, NULL);
+g_signal_connect (data.app_source, "need-data", G_CALLBACK (start_feed), &data);
+g_signal_connect (data.app_source, "enough-data", G_CALLBACK (stop_feed), &data);
+```
+
+The first property that needs to be set on the `appsrc` is `caps`. It
+specifies the kind of data that the element is going to produce, so
+GStreamer can check if linking with downstream elements is possible
+(this is, if the downstream elements will understand this kind of data).
+This property must be a `GstCaps` object, which is easily built from a
+string with `gst_caps_from_string()`.
+
+We then connect to the `need-data` and `enough-data` signals. These are
+fired by `appsrc` when its internal queue of data is running low or
+almost full, respectively. We will use these signals to start and stop
+(respectively) our signal generation process.
+
+``` c
+/* Configure appsink */
+g_object_set (data.app_sink, "emit-signals", TRUE, "caps", audio_caps, NULL);
+g_signal_connect (data.app_sink, "new-sample", G_CALLBACK (new_sample), &data);
+gst_caps_unref (audio_caps);
+g_free (audio_caps_text);
+```
+
+Regarding the `appsink` configuration, we connect to the
+`new-sample` signal, which is emitted every time the sink receives a
+buffer. Also, the signal emission needs to be enabled through the
+`emit-signals` property, because, by default, it is disabled.
+
+Starting the pipeline, waiting for messages and final cleanup is done as
+usual. Let's review the callbacks we have just
+registered:
+
+``` c
+/* This signal callback triggers when appsrc needs data. Here, we add an idle handler
+ * to the mainloop to start pushing data into the appsrc */
+static void start_feed (GstElement *source, guint size, CustomData *data) {
+ if (data->sourceid == 0) {
+ g_print ("Start feeding\n");
+ data->sourceid = g_idle_add ((GSourceFunc) push_data, data);
+ }
+}
+```
+
+This function is called when the internal queue of `appsrc` is about to
+starve (run out of data). The only thing we do here is register a GLib
+idle function with `g_idle_add()` that feeds data to `appsrc` until it
+is full again. A GLib idle function is a method that GLib will call from
+its main loop whenever it is “idle”, this is, when it has no
+higher-priority tasks to perform. It requires a GLib `GMainLoop` to be
+instantiated and running, obviously.
+
+This is only one of the multiple approaches that `appsrc` allows. In
+particular, buffers do not need to be fed into `appsrc` from the main
+thread using GLib, and you do not need to use the `need-data` and
+`enough-data` signals to synchronize with `appsrc` (although this is
+allegedly the most convenient).
+
+We take note of the sourceid that `g_idle_add()` returns, so we can
+disable it
+later.
+
+``` c
+/* This callback triggers when appsrc has enough data and we can stop sending.
+ * We remove the idle handler from the mainloop */
+static void stop_feed (GstElement *source, CustomData *data) {
+ if (data->sourceid != 0) {
+ g_print ("Stop feeding\n");
+ g_source_remove (data->sourceid);
+ data->sourceid = 0;
+ }
+}
+```
+
+This function is called when the internal queue of `appsrc` is full
+enough so we stop pushing data. Here we simply remove the idle function
+by using `g_source_remove()` (The idle function is implemented as a
+`GSource`).
+
+``` c
+/* This method is called by the idle GSource in the mainloop, to feed CHUNK_SIZE bytes into appsrc.
+ * The ide handler is added to the mainloop when appsrc requests us to start sending data (need-data signal)
+ * and is removed when appsrc has enough data (enough-data signal).
+ */
+static gboolean push_data (CustomData *data) {
+ GstBuffer *buffer;
+ GstFlowReturn ret;
+ int i;
+ gint16 *raw;
+ gint num_samples = CHUNK_SIZE / 2; /* Because each sample is 16 bits */
+ gfloat freq;
+
+ /* Create a new empty buffer */
+ buffer = gst_buffer_new_and_alloc (CHUNK_SIZE);
+
+ /* Set its timestamp and duration */
+ GST_BUFFER_TIMESTAMP (buffer) = gst_util_uint64_scale (data->num_samples, GST_SECOND, SAMPLE_RATE);
+ GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale (CHUNK_SIZE, GST_SECOND, SAMPLE_RATE);
+
+ /* Generate some psychodelic waveforms */
+ raw = (gint16 *)GST_BUFFER_DATA (buffer);
+```
+
+This is the function that feeds `appsrc`. It will be called by GLib at
+times and rates which are out of our control, but we know that we will
+disable it when its job is done (when the queue in `appsrc` is full).
+
+Its first task is to create a new buffer with a given size (in this
+example, it is arbitrarily set to 1024 bytes) with
+`gst_buffer_new_and_alloc()`.
+
+We count the number of samples that we have generated so far with the
+`CustomData.num_samples` variable, so we can time-stamp this buffer
+using the `GST_BUFFER_TIMESTAMP` macro in `GstBuffer`.
+
+Since we are producing buffers of the same size, their duration is the
+same and is set using the `GST_BUFFER_DURATION` in `GstBuffer`.
+
+`gst_util_uint64_scale()` is a utility function that scales (multiply
+and divide) numbers which can be large, without fear of overflows.
+
+The bytes that for the buffer can be accessed with GST\_BUFFER\_DATA in
+`GstBuffer` (Be careful not to write past the end of the buffer: you
+allocated it, so you know its size).
+
+We will skip over the waveform generation, since it is outside the scope
+of this tutorial (it is simply a funny way of generating a pretty
+psychedelic wave).
+
+``` c
+/* Push the buffer into the appsrc */
+g_signal_emit_by_name (data->app_source, "push-buffer", buffer, &ret);
+
+/* Free the buffer now that we are done with it */
+gst_buffer_unref (buffer);
+```
+
+Once we have the buffer ready, we pass it to `appsrc` with the
+`push-buffer` action signal (see information box at the end of [](sdk-playback-tutorial-playbin-usage.md)), and then
+`gst_buffer_unref()` it since we no longer need it.
+
+``` c
+/* The appsink has received a buffer */
+static void new_sample (GstElement *sink, CustomData *data) {
+ GstSample *sample;
+ /* Retrieve the buffer */
+ g_signal_emit_by_name (sink, "pull-sample", &sample);
+ if (sample) {
+ /* The only thing we do in this example is print a * to indicate a received buffer */
+ g_print ("*");
+ gst_sample_unref (sample);
+ }
+}
+```
+
+Finally, this is the function that gets called when the
+`appsink` receives a buffer. We use the `pull-sample` action signal to
+retrieve the buffer and then just print some indicator on the screen. We
+can retrieve the data pointer using the `GST_BUFFER_DATA` macro and the
+data size using the `GST_BUFFER_SIZE` macro in `GstBuffer`. Remember
+that this buffer does not have to match the buffer that we produced in
+the `push_data` function, any element in the path could have altered the
+buffers in any way (Not in this example: there is only a `tee` in the
+path between `appsrc` and `appsink`, and it does not change the content
+of the buffers).
+
+We then `gst_buffer_unref()` the buffer, and this tutorial is done.
+
+## Conclusion
+
+This tutorial has shown how applications can:
+
+ - Inject data into a pipeline using the `appsrc`element.
+ - Retrieve data from a pipeline using the `appsink` element.
+ - Manipulate this data by accessing the `GstBuffer`.
+
+In a playbin-based pipeline, the same goals are achieved in a slightly
+different way. [](sdk-playback-tutorial-short-cutting-the-pipeline.md) shows
+how to do it.
+
+It has been a pleasure having you here, and see you soon\!
--- /dev/null
+# Basic tutorial 12: Streaming
+
+## Goal
+
+Playing media straight from the Internet without storing it locally is
+known as Streaming. We have been doing it throughout the tutorials
+whenever we used a URI starting with `http://`. This tutorial shows a
+couple of additional points to keep in mind when streaming. In
+particular:
+
+ - How to enable buffering (to alleviate network problems)
+ - How to recover from interruptions (lost clock)
+
+## Introduction
+
+When streaming, media chunks are decoded and queued for presentation as
+soon as they arrive form the network. This means that if a chunk is
+delayed (which is not an uncommon situation at all on the Internet) the
+presentation queue might run dry and media playback could stall.
+
+The universal solution is to build a “buffer”, this is, allow a certain
+number of media chunks to be queued before starting playback. In this
+way, playback start is delayed a bit, but, if some chunks are late,
+reproduction is not impacted as there are more chunks in the queue,
+waiting.
+
+As it turns out, this solution is already implemented in GStreamer, but
+the previous tutorials have not been benefiting from it. Some elements,
+like the `queue2` and `multiqueue` found inside `playbin`, are capable
+of building this buffer and post bus messages regarding the buffer level
+(the state of the queue). An application wanting to have more network
+resilience, then, should listen to these messages and pause playback if
+the buffer level is not high enough (usually, whenever it is below
+100%).
+
+To achieve synchronization among multiple sinks (for example and audio
+and a video sink) a global clock is used. This clock is selected by
+GStreamer among all elements which can provide one. Under some
+circumstances, for example, an RTP source switching streams or changing
+the output device, this clock can be lost and a new one needs to be
+selected. This happens mostly when dealing with streaming, so the
+process is explained in this tutorial.
+
+When the clock is lost, the application receives a message on the bus;
+to select a new one, the application just needs to set the pipeline to
+PAUSED and then to PLAYING again.
+
+## A network-resilient example
+
+Copy this code into a text file named `basic-tutorial-12.c`.
+
+**basic-tutorial-12.c**
+
+``` c
+#include <gst/gst.h>
+#include <string.h>
+
+typedef struct _CustomData {
+ gboolean is_live;
+ GstElement *pipeline;
+ GMainLoop *loop;
+} CustomData;
+
+static void cb_message (GstBus *bus, GstMessage *msg, CustomData *data) {
+
+ switch (GST_MESSAGE_TYPE (msg)) {
+ case GST_MESSAGE_ERROR: {
+ GError *err;
+ gchar *debug;
+
+ gst_message_parse_error (msg, &err, &debug);
+ g_print ("Error: %s\n", err->message);
+ g_error_free (err);
+ g_free (debug);
+
+ gst_element_set_state (data->pipeline, GST_STATE_READY);
+ g_main_loop_quit (data->loop);
+ break;
+ }
+ case GST_MESSAGE_EOS:
+ /* end-of-stream */
+ gst_element_set_state (data->pipeline, GST_STATE_READY);
+ g_main_loop_quit (data->loop);
+ break;
+ case GST_MESSAGE_BUFFERING: {
+ gint percent = 0;
+
+ /* If the stream is live, we do not care about buffering. */
+ if (data->is_live) break;
+
+ gst_message_parse_buffering (msg, &percent);
+ g_print ("Buffering (%3d%%)\r", percent);
+ /* Wait until buffering is complete before start/resume playing */
+ if (percent < 100)
+ gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
+ else
+ gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
+ break;
+ }
+ case GST_MESSAGE_CLOCK_LOST:
+ /* Get a new clock */
+ gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
+ gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
+ break;
+ default:
+ /* Unhandled message */
+ break;
+ }
+}
+
+int main(int argc, char *argv[]) {
+ GstElement *pipeline;
+ GstBus *bus;
+ GstStateChangeReturn ret;
+ GMainLoop *main_loop;
+ CustomData data;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Initialize our data structure */
+ memset (&data, 0, sizeof (data));
+
+ /* Build the pipeline */
+ pipeline = gst_parse_launch ("playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
+ bus = gst_element_get_bus (pipeline);
+
+ /* Start playing */
+ ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
+ if (ret == GST_STATE_CHANGE_FAILURE) {
+ g_printerr ("Unable to set the pipeline to the playing state.\n");
+ gst_object_unref (pipeline);
+ return -1;
+ } else if (ret == GST_STATE_CHANGE_NO_PREROLL) {
+ data.is_live = TRUE;
+ }
+
+ main_loop = g_main_loop_new (NULL, FALSE);
+ data.loop = main_loop;
+ data.pipeline = pipeline;
+
+ gst_bus_add_signal_watch (bus);
+ g_signal_connect (bus, "message", G_CALLBACK (cb_message), &data);
+
+ g_main_loop_run (main_loop);
+
+ /* Free resources */
+ g_main_loop_unref (main_loop);
+ gst_object_unref (bus);
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+ return 0;
+}
+```
+
+> 
+> Need help?
+>
+> If you need help to compile this code, refer to the **Building the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Build), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Build) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Build), or use this specific command on Linux:
+>
+> `` gcc basic-tutorial-12.c -o basic-tutorial-12 `pkg-config --cflags --libs gstreamer-1.0` ``
+>
+>If you need help to run this code, refer to the **Running the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Run), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Run) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Run).
+>
+> This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed. In the console window, you should see a buffering message, and playback should only start when the buffering reaches 100%. This percentage might not change at all if your connection is fast enough and buffering is not required.
+>
+> Required libraries: `gstreamer-1.0`
+
+## Walkthrough
+
+The only special thing this tutorial does is react to certain messages;
+therefore, the initialization code is very simple and should be
+self-explanative by now. The only new bit is the detection of live
+streams:
+
+``` c
+/* Start playing */
+ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
+if (ret == GST_STATE_CHANGE_FAILURE) {
+ g_printerr ("Unable to set the pipeline to the playing state.\n");
+ gst_object_unref (pipeline);
+ return -1;
+} else if (ret == GST_STATE_CHANGE_NO_PREROLL) {
+ data.is_live = TRUE;
+}
+```
+
+Live streams cannot be paused, so they behave in PAUSED state as if they
+were in the PLAYING state. Setting live streams to PAUSED succeeds, but
+returns `GST_STATE_CHANGE_NO_PREROLL`, instead of
+`GST_STATE_CHANGE_SUCCESS` to indicate that this is a live stream. We
+are receiving the NO\_PROROLL return code even though we are trying to
+set the pipeline to PLAYING, because state changes happen progressively
+(from NULL to READY, to PAUSED and then to PLAYING).
+
+We care about live streams because we want to disable buffering for
+them, so we take note of the result of `gst_element_set_state()` in the
+`is_live` variable.
+
+Let’s now review the interesting parts of the message parsing callback:
+
+``` c
+case GST_MESSAGE_BUFFERING: {
+ gint percent = 0;
+
+ /* If the stream is live, we do not care about buffering. */
+ if (data->is_live) break;
+
+ gst_message_parse_buffering (msg, &percent);
+ g_print ("Buffering (%3d%%)\r", percent);
+ /* Wait until buffering is complete before start/resume playing */
+ if (percent < 100)
+ gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
+ else
+ gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
+ break;
+}
+```
+
+First, if this is a live source, ignore buffering messages.
+
+We parse the buffering message with `gst_message_parse_buffering()` to
+retrieve the buffering level.
+
+Then, we print the buffering level on the console and set the pipeline
+to PAUSED if it is below 100%. Otherwise, we set the pipeline to
+PLAYING.
+
+At startup, we will see the buffering level rise up to 100% before
+playback starts, which is what we wanted to achieve. If, later on, the
+network becomes slow or unresponsive and our buffer depletes, we will
+receive new buffering messages with levels below 100% so we will pause
+the pipeline again until enough buffer has been built up.
+
+``` c
+case GST_MESSAGE_CLOCK_LOST:
+ /* Get a new clock */
+ gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
+ gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
+ break;
+```
+
+For the second network issue, the loss of clock, we simply set the
+pipeline to PAUSED and back to PLAYING, so a new clock is selected,
+waiting for new media chunks to be received if necessary.
+
+## Conclusion
+
+This tutorial has described how to add network resilience to your
+application with two very simple precautions:
+
+ - Taking care of buffering messages sent by the pipeline
+ - Taking care of clock loss
+
+Handling these messages improves the application’s response to network
+problems, increasing the overall playback smoothness.
+
+It has been a pleasure having you here, and see you soon!
\ No newline at end of file
--- /dev/null
+# Basic tutorial 4: Time management
+
+## Goal
+
+This tutorial shows how to use GStreamer time-related facilities. In
+particular:
+
+ - How to query the pipeline for information like stream position or
+ duration.
+
+ - How to seek (jump) to a different position (time instant) inside the
+ stream.
+
+## Introduction
+
+`GstQuery` is a mechanism that allows asking an element or pad for a
+piece of information. In this example we ask the pipeline if seeking is
+allowed (some sources, like live streams, do not allow seeking). If it
+is allowed, then, once the movie has been running for ten seconds, we
+skip to a different position using a seek.
+
+In the previous tutorials, once we had the pipeline setup and running,
+our main function just sat and waited to receive an ERROR or an EOS
+through the bus. Here we modify this function to periodically wake up
+and query the pipeline for the stream position, so we can print it on
+screen. This is similar to what a media player would do, updating the
+User Interface on a periodic basis.
+
+Finally, the stream duration is queried and updated whenever it changes.
+
+## Seeking example
+
+Copy this code into a text file named `basic-tutorial-4.c` (or find it
+in the SDK installation).
+
+**basic-tutorial-4.c**
+
+``` c
+#include <gst/gst.h>
+
+/* Structure to contain all our information, so we can pass it around */
+typedef struct _CustomData {
+ GstElement *playbin; /* Our one and only element */
+ gboolean playing; /* Are we in the PLAYING state? */
+ gboolean terminate; /* Should we terminate execution? */
+ gboolean seek_enabled; /* Is seeking enabled for this media? */
+ gboolean seek_done; /* Have we performed the seek already? */
+ gint64 duration; /* How long does this media last, in nanoseconds */
+} CustomData;
+
+/* Forward definition of the message processing function */
+static void handle_message (CustomData *data, GstMessage *msg);
+
+int main(int argc, char *argv[]) {
+ CustomData data;
+ GstBus *bus;
+ GstMessage *msg;
+ GstStateChangeReturn ret;
+
+ data.playing = FALSE;
+ data.terminate = FALSE;
+ data.seek_enabled = FALSE;
+ data.seek_done = FALSE;
+ data.duration = GST_CLOCK_TIME_NONE;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Create the elements */
+ data.playbin = gst_element_factory_make ("playbin", "playbin");
+
+ if (!data.playbin) {
+ g_printerr ("Not all elements could be created.\n");
+ return -1;
+ }
+
+ /* Set the URI to play */
+ g_object_set (data.playbin, "uri", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
+
+ /* Start playing */
+ ret = gst_element_set_state (data.playbin, GST_STATE_PLAYING);
+ if (ret == GST_STATE_CHANGE_FAILURE) {
+ g_printerr ("Unable to set the pipeline to the playing state.\n");
+ gst_object_unref (data.playbin);
+ return -1;
+ }
+
+ /* Listen to the bus */
+ bus = gst_element_get_bus (data.playbin);
+ do {
+ msg = gst_bus_timed_pop_filtered (bus, 100 * GST_MSECOND,
+ GST_MESSAGE_STATE_CHANGED | GST_MESSAGE_ERROR | GST_MESSAGE_EOS | GST_MESSAGE_DURATION);
+
+ /* Parse message */
+ if (msg != NULL) {
+ handle_message (&data, msg);
+ } else {
+ /* We got no message, this means the timeout expired */
+ if (data.playing) {
+ gint64 current = -1;
+
+ /* Query the current position of the stream */
+ if (!gst_element_query_position (data.playbin, GST_TIME_FORMAT, ¤t)) {
+ g_printerr ("Could not query current position.\n");
+ }
+
+ /* If we didn't know it yet, query the stream duration */
+ if (!GST_CLOCK_TIME_IS_VALID (data.duration)) {
+ if (!gst_element_query_duration (data.playbin, GST_TIME_FORMAT, &data.duration)) {
+ g_printerr ("Could not query current duration.\n");
+ }
+ }
+
+ /* Print current position and total duration */
+ g_print ("Position %" GST_TIME_FORMAT " / %" GST_TIME_FORMAT "\r",
+ GST_TIME_ARGS (current), GST_TIME_ARGS (data.duration));
+
+ /* If seeking is enabled, we have not done it yet, and the time is right, seek */
+ if (data.seek_enabled && !data.seek_done && current > 10 * GST_SECOND) {
+ g_print ("\nReached 10s, performing seek...\n");
+ gst_element_seek_simple (data.playbin, GST_FORMAT_TIME,
+ GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT, 30 * GST_SECOND);
+ data.seek_done = TRUE;
+ }
+ }
+ }
+ } while (!data.terminate);
+
+ /* Free resources */
+ gst_object_unref (bus);
+ gst_element_set_state (data.playbin, GST_STATE_NULL);
+ gst_object_unref (data.playbin);
+ return 0;
+}
+
+static void handle_message (CustomData *data, GstMessage *msg) {
+ GError *err;
+ gchar *debug_info;
+
+ switch (GST_MESSAGE_TYPE (msg)) {
+ case GST_MESSAGE_ERROR:
+ gst_message_parse_error (msg, &err, &debug_info);
+ g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
+ g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
+ g_clear_error (&err);
+ g_free (debug_info);
+ data->terminate = TRUE;
+ break;
+ case GST_MESSAGE_EOS:
+ g_print ("End-Of-Stream reached.\n");
+ data->terminate = TRUE;
+ break;
+ case GST_MESSAGE_DURATION:
+ /* The duration has changed, mark the current one as invalid */
+ data->duration = GST_CLOCK_TIME_NONE;
+ break;
+ case GST_MESSAGE_STATE_CHANGED: {
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->playbin)) {
+ g_print ("Pipeline state changed from %s to %s:\n",
+ gst_element_state_get_name (old_state), gst_element_state_get_name (new_state));
+
+ /* Remember whether we are in the PLAYING state or not */
+ data->playing = (new_state == GST_STATE_PLAYING);
+
+ if (data->playing) {
+ /* We just moved to PLAYING. Check if seeking is possible */
+ GstQuery *query;
+ gint64 start, end;
+ query = gst_query_new_seeking (GST_FORMAT_TIME);
+ if (gst_element_query (data->playbin, query)) {
+ gst_query_parse_seeking (query, NULL, &data->seek_enabled, &start, &end);
+ if (data->seek_enabled) {
+ g_print ("Seeking is ENABLED from %" GST_TIME_FORMAT " to %" GST_TIME_FORMAT "\n",
+ GST_TIME_ARGS (start), GST_TIME_ARGS (end));
+ } else {
+ g_print ("Seeking is DISABLED for this stream.\n");
+ }
+ }
+ else {
+ g_printerr ("Seeking query failed.");
+ }
+ gst_query_unref (query);
+ }
+ }
+ } break;
+ default:
+ /* We should not reach here */
+ g_printerr ("Unexpected message received.\n");
+ break;
+ }
+ gst_message_unref (msg);
+}
+```
+
+> 
+> Need help?
+>
+> If you need help to compile this code, refer to the **Building the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Build), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Build) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Build), or use this specific command on Linux:
+>
+> ``gcc basic-tutorial-4.c -o basic-tutorial-4 `pkg-config --cflags --libs gstreamer-1.0` ``
+>
+>If you need help to run this code, refer to the **Running the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Run), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Run) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Run).
+>
+> This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed. 10 seconds into the movie it skips to a new position
+>
+>Required libraries: `gstreamer-1.0`
+
+## Walkthrough
+
+```
+/* Structure to contain all our information, so we can pass it around */
+typedef struct _CustomData {
+ GstElement *playbin; /* Our one and only element */
+ gboolean playing; /* Are we in the PLAYING state? */
+ gboolean terminate; /* Should we terminate execution? */
+ gboolean seek_enabled; /* Is seeking enabled for this media? */
+ gboolean seek_done; /* Have we performed the seek already? */
+ gint64 duration; /* How long does this media last, in nanoseconds */
+} CustomData;
+
+/* Forward definition of the message processing function */
+static void handle_message (CustomData *data, GstMessage *msg);
+```
+
+We start by defining a structure to contain all our information, so we
+can pass it around to other functions. In particular, in this example we
+move the message handling code to its own function
+`handle_message` because it is growing a bit too big.
+
+We would then build a pipeline composed of a single element, a
+`playbin`, which we already saw in [Basic tutorial 1: Hello
+world!](sdk-basic-tutorial-hello-world.md). However,
+`playbin` is in itself a pipeline, and in this case it is the only
+element in the pipeline, so we use directly the `playbin` element. We
+will skip the details: the URI of the clip is given to `playbin` via
+the URI property and the pipeline is set to the playing state.
+
+```
+msg = gst_bus_timed_pop_filtered (bus, 100 * GST_MSECOND,
+ GST_MESSAGE_STATE_CHANGED | GST_MESSAGE_ERROR | GST_MESSAGE_EOS | GST_MESSAGE_DURATION);
+```
+
+Previously we did not provide a timeout to
+`gst_bus_timed_pop_filtered()`, meaning that it didn't return until a
+message was received. Now we use a timeout of 100 milliseconds, so, if
+no message is received, 10 times per second the function will return
+with a NULL instead of a `GstMessage`. We are going to use this to
+update our “UI”. Note that the timeout period is specified in
+nanoseconds, so usage of the `GST_SECOND` or `GST_MSECOND` macros is
+highly recommended.
+
+If we got a message, we process it in the `handle_message`` `function
+(next subsection), otherwise:
+
+### User interface resfreshing
+
+```
+/* We got no message, this means the timeout expired */
+if (data.playing) {
+```
+
+First off, if we are not in the PLAYING state, we do not want to do
+anything here, since most queries would fail. Otherwise, it is time to
+refresh the screen.
+
+We get here approximately 10 times per second, a good enough refresh
+rate for our UI. We are going to print on screen the current media
+position, which we can learn be querying the pipeline. This involves a
+few steps that will be shown in the next subsection, but, since position
+and duration are common enough queries, `GstElement` offers easier,
+ready-made alternatives:
+
+```
+/* Query the current position of the stream */
+if (!gst_element_query_position (data.pipeline, GST_FORMAT_TIME, ¤t)) {
+ g_printerr ("Could not query current position.\n");
+}
+```
+
+`gst_element_query_position()` hides the management of the query object
+and directly provides us with the result.
+
+```
+/* If we didn't know it yet, query the stream duration */
+if (!GST_CLOCK_TIME_IS_VALID (data.duration)) {
+ if (!gst_element_query_duration (data.pipeline, GST_TIME_FORMAT, &data.duration)) {
+ g_printerr ("Could not query current duration.\n");
+ }
+}
+```
+
+Now is a good moment to know the length of the stream, with
+another `GstElement` helper function: `gst_element_query_duration()`
+
+```
+/* Print current position and total duration */
+g_print ("Position %" GST_TIME_FORMAT " / %" GST_TIME_FORMAT "\r",
+ GST_TIME_ARGS (current), GST_TIME_ARGS (data.duration));
+```
+
+Note the usage of the `GST_TIME_FORMAT` and `GST_TIME_ARGS` macros to
+provide user-friendly representation of GStreamer
+times.
+
+```
+/* If seeking is enabled, we have not done it yet, and the time is right, seek */
+if (data.seek_enabled && !data.seek_done && current > 10 * GST_SECOND) {
+ g_print ("\nReached 10s, performing seek...\n");
+ gst_element_seek_simple (data.pipeline, GST_FORMAT_TIME,
+ GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT, 30 * GST_SECOND);
+ data.seek_done = TRUE;
+}
+```
+
+Now we perform the seek, “simply” by
+calling `gst_element_seek_simple()` on the pipeline. A lot of the
+intricacies of seeking are hidden in this method, which is a good
+thing!
+
+Let's review the parameters:
+
+`GST_FORMAT_TIME` indicates that we are specifying the destination in
+time, as opposite to bytes (and other more obscure mechanisms).
+
+Then come the GstSeekFlags, let's review the most common:
+
+`GST_SEEK_FLAG_FLUSH`: This discards all data currently in the pipeline
+before doing the seek. Might pause a bit while the pipeline is refilled
+and the new data starts to show up, but greatly increases the
+“responsiveness” of the application. If this flag is not provided,
+“stale” data might be shown for a while until the new position appears
+at the end of the pipeline.
+
+`GST_SEEK_FLAG_KEY_UNIT`: Most encoded video streams cannot seek to
+arbitrary positions, only to certain frames called Key Frames. When this
+flag is used, the seek will actually move to the closest key frame and
+start producing data straight away. If this flag is not used, the
+pipeline will move internally to the closest key frame (it has no other
+alternative) but data will not be shown until it reaches the requested
+position. Not providing the flag is more accurate, but might take longer
+to react.
+
+`GST_SEEK_FLAG_ACCURATE`: Some media clips do not provide enough
+indexing information, meaning that seeking to arbitrary positions is
+time-consuming. In these cases, GStreamer usually estimates the position
+to seek to, and usually works just fine. If this precision is not good
+enough for your case (you see seeks not going to the exact time you
+asked for), then provide this flag. Be warned that it might take longer
+to calculate the seeking position (very long, on some files).
+
+And finally we provide the position to seek to. Since we asked
+for `GST_FORMAT_TIME` , this position is in nanoseconds, so we use
+the `GST_SECOND` macro for simplicity.
+
+### Message Pump
+
+The `handle_message` function processes all messages received through
+the pipeline's bus. ERROR and EOS handling is the same as in previous
+tutorials, so we skip to the interesting part:
+
+```
+case GST_MESSAGE_DURATION:
+ /* The duration has changed, mark the current one as invalid */
+ data->duration = GST_CLOCK_TIME_NONE;
+ break;
+```
+
+This message is posted on the bus whenever the duration of the stream
+changes. Here we simply mark the current duration as invalid, so it gets
+re-queried later.
+
+```
+case GST_MESSAGE_STATE_CHANGED: {
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->pipeline)) {
+ g_print ("Pipeline state changed from %s to %s:\n",
+ gst_element_state_get_name (old_state), gst_element_state_get_name (new_state));
+
+ /* Remember whether we are in the PLAYING state or not */
+ data->playing = (new_state == GST_STATE_PLAYING);
+```
+
+Seeks and time queries generally only get a valid reply when in the
+PAUSED or PLAYING state, since all elements have had a chance to
+receive information and configure themselves. Here we take note of
+whether we are in the PLAYING state or not with the `playing`
+variable.
+
+Also, if we have just entered the PLAYING state, we do our first query.
+We ask the pipeline if seeking is allowed on this stream:
+
+```
+if (data->playing) {
+ /* We just moved to PLAYING. Check if seeking is possible */
+ GstQuery *query;
+ gint64 start, end;
+ query = gst_query_new_seeking (GST_FORMAT_TIME);
+ if (gst_element_query (data->pipeline, query)) {
+ gst_query_parse_seeking (query, NULL, &data->seek_enabled, &start, &end);
+ if (data->seek_enabled) {
+ g_print ("Seeking is ENABLED from %" GST_TIME_FORMAT " to %" GST_TIME_FORMAT "\n",
+ GST_TIME_ARGS (start), GST_TIME_ARGS (end));
+ } else {
+ g_print ("Seeking is DISABLED for this stream.\n");
+ }
+ }
+ else {
+ g_printerr ("Seeking query failed.");
+ }
+ gst_query_unref (query);
+}
+```
+
+`gst_query_new_seeking()` creates a new query object of the "seeking"
+type, with `GST_FORMAT_TIME` format. This indicates that we are
+interested in seeking by specifying the new time to which we want to
+move. We could also ask for `GST_FORMAT_BYTES`, and then seek to a
+particular byte position inside the source file, but this is normally
+less useful.
+
+This query object is then passed to the pipeline with
+`gst_element_query()`. The result is stored in the same query, and can
+be easily retrieved with `gst_query_parse_seeking()`. It extracts a
+boolean indicating if seeking is allowed, and the range in which seeking
+is possible.
+
+Don't forget to unref the query object when you are done with it.
+
+And that's it! With this knowledge a media player can be built which
+periodically updates a slider based on the current stream position and
+allows seeking by moving the slider!
+
+## Conclusion
+
+This tutorial has shown:
+
+ - How to query the pipeline for information using `GstQuery`
+
+ - How to obtain common information like position and duration
+ using `gst_element_query_position()` and `gst_element_query_duration()`
+
+ - How to seek to an arbitrary position in the stream
+ using `gst_element_seek_simple()`
+
+ - In which states all these operations can be performed.
+
+The next tutorial shows how to integrate GStreamer with a Graphical User
+Interface toolkit.
+
+Remember that attached to this page you should find the complete source
+code of the tutorial and any accessory files needed to build it.
+
+It has been a pleasure having you here, and see you soon!
--- /dev/null
+# Basic tutorial 5: GUI toolkit integration
+
+## Goal
+
+This tutorial shows how to integrate GStreamer in a Graphical User
+Interface (GUI) toolkit like [GTK+](http://www.gtk.org). Basically,
+GStreamer takes care of media playback while the GUI toolkit handles
+user interaction. The most interesting parts are those in which both
+libraries have to interact: Instructing GStreamer to output video to a
+GTK+ window and forwarding user actions to GStreamer.
+
+In particular, you will learn:
+
+ - How to tell GStreamer to output video to a particular window
+ (instead of creating its own window).
+
+ - How to continuously refresh the GUI with information from GStreamer.
+
+ - How to update the GUI from the multiple threads of GStreamer, an
+ operation forbidden on most GUI toolkits.
+
+ - A mechanism to subscribe only to the messages you are interested in,
+ instead of being notified of all of them.
+
+## Introduction
+
+We are going to build a media player using the
+[GTK+](http://www.gtk.org/) toolkit, but the concepts apply to other
+toolkits like [QT](http://qt-project.org/), for example. A minimum
+knowledge of [GTK+](http://www.gtk.org/) will help understand this
+tutorial.
+
+The main point is telling GStreamer to output the video to a window of
+our choice. The specific mechanism depends on the operating system (or
+rather, on the windowing system), but GStreamer provides a layer of
+abstraction for the sake of platform independence. This independence
+comes through the `GstVideoOverlay` interface, that allows the application to
+tell a video sink the handler of the window that should receive the
+rendering.
+
+> 
+> **GObject interfaces**
+>
+> A GObject *interface* (which GStreamer uses) is a set of functions that an element can implement. If it does, then it is said to support that particular interface. For example, video sinks usually create their own windows to display video, but, if they are also capable of rendering to an external window, they can choose to implement the `GstVideoOverlay` interface and provide functions to specify this external window. From the application developer point of view, if a certain interface is supported, you can use it and forget about which kind of element is implementing it. Moreover, if you are using `playbin`, it will automatically expose some of the interfaces supported by its internal elements: You can use your interface functions directly on `playbin` without knowing who is implementing them!
+
+Another issue is that GUI toolkits usually only allow manipulation of
+the graphical “widgets” through the main (or application) thread,
+whereas GStreamer usually spawns multiple threads to take care of
+different tasks. Calling [GTK+](http://www.gtk.org/) functions from
+within callbacks will usually fail, because callbacks execute in the
+calling thread, which does not need to be the main thread. This problem
+can be solved by posting a message on the GStreamer bus in the callback:
+The messages will be received by the main thread which will then react
+accordingly.
+
+Finally, so far we have registered a `handle_message` function that got
+called every time a message appeared on the bus, which forced us to
+parse every message to see if it was of interest to us. In this tutorial
+a different method is used that registers a callback for each kind of
+message, so there is less parsing and less code overall.
+
+## A media player in GTK+
+
+Let's write a very simple media player based on playbin, this time,
+with a GUI!
+
+Copy this code into a text file named `basic-tutorial-5.c` (or find it
+in the SDK installation).
+
+**basic-tutorial-5.c**
+
+``` c
+#include <string.h>
+
+#include <gtk/gtk.h>
+#include <gst/gst.h>
+#include <gst/video/videooverlay.h>
+
+#include <gdk/gdk.h>
+#if defined (GDK_WINDOWING_X11)
+#include <gdk/gdkx.h>
+#elif defined (GDK_WINDOWING_WIN32)
+#include <gdk/gdkwin32.h>
+#elif defined (GDK_WINDOWING_QUARTZ)
+#include <gdk/gdkquartz.h>
+#endif
+
+/* Structure to contain all our information, so we can pass it around */
+typedef struct _CustomData {
+ GstElement *playbin; /* Our one and only pipeline */
+
+ GtkWidget *slider; /* Slider widget to keep track of current position */
+ GtkWidget *streams_list; /* Text widget to display info about the streams */
+ gulong slider_update_signal_id; /* Signal ID for the slider update signal */
+
+ GstState state; /* Current state of the pipeline */
+ gint64 duration; /* Duration of the clip, in nanoseconds */
+} CustomData;
+
+/* This function is called when the GUI toolkit creates the physical window that will hold the video.
+ * At this point we can retrieve its handler (which has a different meaning depending on the windowing system)
+ * and pass it to GStreamer through the GstVideoOverlay interface. */
+static void realize_cb (GtkWidget *widget, CustomData *data) {
+ GdkWindow *window = gtk_widget_get_window (widget);
+ guintptr window_handle;
+
+ if (!gdk_window_ensure_native (window))
+ g_error ("Couldn't create native window needed for GstVideoOverlay!");
+
+ /* Retrieve window handler from GDK */
+#if defined (GDK_WINDOWING_WIN32)
+ window_handle = (guintptr)GDK_WINDOW_HWND (window);
+#elif defined (GDK_WINDOWING_QUARTZ)
+ window_handle = gdk_quartz_window_get_nsview (window);
+#elif defined (GDK_WINDOWING_X11)
+ window_handle = GDK_WINDOW_XID (window);
+#endif
+ /* Pass it to playbin, which implements GstVideoOverlay and will forward it to the video sink */
+ gst_video_overlay_set_window_handle (GST_VIDEO_OVERLAY (data->playbin), window_handle);
+}
+
+/* This function is called when the PLAY button is clicked */
+static void play_cb (GtkButton *button, CustomData *data) {
+ gst_element_set_state (data->playbin, GST_STATE_PLAYING);
+}
+
+/* This function is called when the PAUSE button is clicked */
+static void pause_cb (GtkButton *button, CustomData *data) {
+ gst_element_set_state (data->playbin, GST_STATE_PAUSED);
+}
+
+/* This function is called when the STOP button is clicked */
+static void stop_cb (GtkButton *button, CustomData *data) {
+ gst_element_set_state (data->playbin, GST_STATE_READY);
+}
+
+/* This function is called when the main window is closed */
+static void delete_event_cb (GtkWidget *widget, GdkEvent *event, CustomData *data) {
+ stop_cb (NULL, data);
+ gtk_main_quit ();
+}
+
+/* This function is called everytime the video window needs to be redrawn (due to damage/exposure,
+ * rescaling, etc). GStreamer takes care of this in the PAUSED and PLAYING states, otherwise,
+ * we simply draw a black rectangle to avoid garbage showing up. */
+static gboolean draw_cb (GtkWidget *widget, cairo_t *cr, CustomData *data) {
+ if (data->state < GST_STATE_PAUSED) {
+ GtkAllocation allocation;
+
+ /* Cairo is a 2D graphics library which we use here to clean the video window.
+ * It is used by GStreamer for other reasons, so it will always be available to us. */
+ gtk_widget_get_allocation (widget, &allocation);
+ cairo_set_source_rgb (cr, 0, 0, 0);
+ cairo_rectangle (cr, 0, 0, allocation.width, allocation.height);
+ cairo_fill (cr);
+ cairo_destroy (cr);
+ }
+
+ return FALSE;
+}
+
+/* This function is called when the slider changes its position. We perform a seek to the
+ * new position here. */
+static void slider_cb (GtkRange *range, CustomData *data) {
+ gdouble value = gtk_range_get_value (GTK_RANGE (data->slider));
+ gst_element_seek_simple (data->playbin, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT,
+ (gint64)(value * GST_SECOND));
+}
+
+/* This creates all the GTK+ widgets that compose our application, and registers the callbacks */
+static void create_ui (CustomData *data) {
+ GtkWidget *main_window; /* The uppermost window, containing all other windows */
+ GtkWidget *video_window; /* The drawing area where the video will be shown */
+ GtkWidget *main_box; /* VBox to hold main_hbox and the controls */
+ GtkWidget *main_hbox; /* HBox to hold the video_window and the stream info text widget */
+ GtkWidget *controls; /* HBox to hold the buttons and the slider */
+ GtkWidget *play_button, *pause_button, *stop_button; /* Buttons */
+
+ main_window = gtk_window_new (GTK_WINDOW_TOPLEVEL);
+ g_signal_connect (G_OBJECT (main_window), "delete-event", G_CALLBACK (delete_event_cb), data);
+
+ video_window = gtk_drawing_area_new ();
+ gtk_widget_set_double_buffered (video_window, FALSE);
+ g_signal_connect (video_window, "realize", G_CALLBACK (realize_cb), data);
+ g_signal_connect (video_window, "draw", G_CALLBACK (draw_cb), data);
+
+ play_button = gtk_button_new_from_stock (GTK_STOCK_MEDIA_PLAY);
+ g_signal_connect (G_OBJECT (play_button), "clicked", G_CALLBACK (play_cb), data);
+
+ pause_button = gtk_button_new_from_stock (GTK_STOCK_MEDIA_PAUSE);
+ g_signal_connect (G_OBJECT (pause_button), "clicked", G_CALLBACK (pause_cb), data);
+
+ stop_button = gtk_button_new_from_stock (GTK_STOCK_MEDIA_STOP);
+ g_signal_connect (G_OBJECT (stop_button), "clicked", G_CALLBACK (stop_cb), data);
+
+ data->slider = gtk_hscale_new_with_range (0, 100, 1);
+ gtk_scale_set_draw_value (GTK_SCALE (data->slider), 0);
+ data->slider_update_signal_id = g_signal_connect (G_OBJECT (data->slider), "value-changed", G_CALLBACK (slider_cb), data);
+
+ data->streams_list = gtk_text_view_new ();
+ gtk_text_view_set_editable (GTK_TEXT_VIEW (data->streams_list), FALSE);
+
+ controls = gtk_box_new (GTK_ORIENTATION_HORIZONTAL,, 0);
+ gtk_box_pack_start (GTK_BOX (controls), play_button, FALSE, FALSE, 2);
+ gtk_box_pack_start (GTK_BOX (controls), pause_button, FALSE, FALSE, 2);
+ gtk_box_pack_start (GTK_BOX (controls), stop_button, FALSE, FALSE, 2);
+ gtk_box_pack_start (GTK_BOX (controls), data->slider, TRUE, TRUE, 2);
+
+ main_hbox = gtk_box_new (GTK_ORIENTATION_HORIZONTAL,, 0);
+ gtk_box_pack_start (GTK_BOX (main_hbox), video_window, TRUE, TRUE, 0);
+ gtk_box_pack_start (GTK_BOX (main_hbox), data->streams_list, FALSE, FALSE, 2);
+
+ main_box = gtk_box_new (GTK_ORIENTATION_VERTICAL,, 0);
+ gtk_box_pack_start (GTK_BOX (main_box), main_hbox, TRUE, TRUE, 0);
+ gtk_box_pack_start (GTK_BOX (main_box), controls, FALSE, FALSE, 0);
+ gtk_container_add (GTK_CONTAINER (main_window), main_box);
+ gtk_window_set_default_size (GTK_WINDOW (main_window), 640, 480);
+
+ gtk_widget_show_all (main_window);
+}
+
+/* This function is called periodically to refresh the GUI */
+static gboolean refresh_ui (CustomData *data) {
+ gint64 current = -1;
+
+ /* We do not want to update anything unless we are in the PAUSED or PLAYING states */
+ if (data->state < GST_STATE_PAUSED)
+ return TRUE;
+
+ /* If we didn't know it yet, query the stream duration */
+ if (!GST_CLOCK_TIME_IS_VALID (data->duration)) {
+ if (!gst_element_query_duration (data->playbin, GST_FORMAT_TIME, &data->duration)) {
+ g_printerr ("Could not query current duration.\n");
+ } else {
+ /* Set the range of the slider to the clip duration, in SECONDS */
+ gtk_range_set_range (GTK_RANGE (data->slider), 0, (gdouble)data->duration / GST_SECOND);
+ }
+ }
+
+ if (gst_element_query_position (data->playbin, GST_FORMAT_TIME, ¤t)) {
+ /* Block the "value-changed" signal, so the slider_cb function is not called
+ * (which would trigger a seek the user has not requested) */
+ g_signal_handler_block (data->slider, data->slider_update_signal_id);
+ /* Set the position of the slider to the current pipeline positoin, in SECONDS */
+ gtk_range_set_value (GTK_RANGE (data->slider), (gdouble)current / GST_SECOND);
+ /* Re-enable the signal */
+ g_signal_handler_unblock (data->slider, data->slider_update_signal_id);
+ }
+ return TRUE;
+}
+
+/* This function is called when new metadata is discovered in the stream */
+static void tags_cb (GstElement *playbin, gint stream, CustomData *data) {
+ /* We are possibly in a GStreamer working thread, so we notify the main
+ * thread of this event through a message in the bus */
+ gst_element_post_message (playbin,
+ gst_message_new_application (GST_OBJECT (playbin),
+ gst_structure_new ("tags-changed", NULL)));
+}
+
+/* This function is called when an error message is posted on the bus */
+static void error_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ GError *err;
+ gchar *debug_info;
+
+ /* Print error details on the screen */
+ gst_message_parse_error (msg, &err, &debug_info);
+ g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
+ g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
+ g_clear_error (&err);
+ g_free (debug_info);
+
+ /* Set the pipeline to READY (which stops playback) */
+ gst_element_set_state (data->playbin, GST_STATE_READY);
+}
+
+/* This function is called when an End-Of-Stream message is posted on the bus.
+ * We just set the pipeline to READY (which stops playback) */
+static void eos_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ g_print ("End-Of-Stream reached.\n");
+ gst_element_set_state (data->playbin, GST_STATE_READY);
+}
+
+/* This function is called when the pipeline changes states. We use it to
+ * keep track of the current state. */
+static void state_changed_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->playbin)) {
+ data->state = new_state;
+ g_print ("State set to %s\n", gst_element_state_get_name (new_state));
+ if (old_state == GST_STATE_READY && new_state == GST_STATE_PAUSED) {
+ /* For extra responsiveness, we refresh the GUI as soon as we reach the PAUSED state */
+ refresh_ui (data);
+ }
+ }
+}
+
+/* Extract metadata from all the streams and write it to the text widget in the GUI */
+static void analyze_streams (CustomData *data) {
+ gint i;
+ GstTagList *tags;
+ gchar *str, *total_str;
+ guint rate;
+ gint n_video, n_audio, n_text;
+ GtkTextBuffer *text;
+
+ /* Clean current contents of the widget */
+ text = gtk_text_view_get_buffer (GTK_TEXT_VIEW (data->streams_list));
+ gtk_text_buffer_set_text (text, "", -1);
+
+ /* Read some properties */
+ g_object_get (data->playbin, "n-video", &n_video, NULL);
+ g_object_get (data->playbin, "n-audio", &n_audio, NULL);
+ g_object_get (data->playbin, "n-text", &n_text, NULL);
+
+ for (i = 0; i < n_video; i++) {
+ tags = NULL;
+ /* Retrieve the stream's video tags */
+ g_signal_emit_by_name (data->playbin, "get-video-tags", i, &tags);
+ if (tags) {
+ total_str = g_strdup_printf ("video stream %d:\n", i);
+ gtk_text_buffer_insert_at_cursor (text, total_str, -1);
+ g_free (total_str);
+ gst_tag_list_get_string (tags, GST_TAG_VIDEO_CODEC, &str);
+ total_str = g_strdup_printf (" codec: %s\n", str ? str : "unknown");
+ gtk_text_buffer_insert_at_cursor (text, total_str, -1);
+ g_free (total_str);
+ g_free (str);
+ gst_tag_list_free (tags);
+ }
+ }
+
+ for (i = 0; i < n_audio; i++) {
+ tags = NULL;
+ /* Retrieve the stream's audio tags */
+ g_signal_emit_by_name (data->playbin, "get-audio-tags", i, &tags);
+ if (tags) {
+ total_str = g_strdup_printf ("\naudio stream %d:\n", i);
+ gtk_text_buffer_insert_at_cursor (text, total_str, -1);
+ g_free (total_str);
+ if (gst_tag_list_get_string (tags, GST_TAG_AUDIO_CODEC, &str)) {
+ total_str = g_strdup_printf (" codec: %s\n", str);
+ gtk_text_buffer_insert_at_cursor (text, total_str, -1);
+ g_free (total_str);
+ g_free (str);
+ }
+ if (gst_tag_list_get_string (tags, GST_TAG_LANGUAGE_CODE, &str)) {
+ total_str = g_strdup_printf (" language: %s\n", str);
+ gtk_text_buffer_insert_at_cursor (text, total_str, -1);
+ g_free (total_str);
+ g_free (str);
+ }
+ if (gst_tag_list_get_uint (tags, GST_TAG_BITRATE, &rate)) {
+ total_str = g_strdup_printf (" bitrate: %d\n", rate);
+ gtk_text_buffer_insert_at_cursor (text, total_str, -1);
+ g_free (total_str);
+ }
+ gst_tag_list_free (tags);
+ }
+ }
+
+ for (i = 0; i < n_text; i++) {
+ tags = NULL;
+ /* Retrieve the stream's subtitle tags */
+ g_signal_emit_by_name (data->playbin, "get-text-tags", i, &tags);
+ if (tags) {
+ total_str = g_strdup_printf ("\nsubtitle stream %d:\n", i);
+ gtk_text_buffer_insert_at_cursor (text, total_str, -1);
+ g_free (total_str);
+ if (gst_tag_list_get_string (tags, GST_TAG_LANGUAGE_CODE, &str)) {
+ total_str = g_strdup_printf (" language: %s\n", str);
+ gtk_text_buffer_insert_at_cursor (text, total_str, -1);
+ g_free (total_str);
+ g_free (str);
+ }
+ gst_tag_list_free (tags);
+ }
+ }
+}
+
+/* This function is called when an "application" message is posted on the bus.
+ * Here we retrieve the message posted by the tags_cb callback */
+static void application_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ if (g_strcmp0 (gst_structure_get_name (gst_message_get_structure(msg)), "tags-changed") == 0) {
+ /* If the message is the "tags-changed" (only one we are currently issuing), update
+ * the stream info GUI */
+ analyze_streams (data);
+ }
+}
+
+int main(int argc, char *argv[]) {
+ CustomData data;
+ GstStateChangeReturn ret;
+ GstBus *bus;
+
+ /* Initialize GTK */
+ gtk_init (&argc, &argv);
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Initialize our data structure */
+ memset (&data, 0, sizeof (data));
+ data.duration = GST_CLOCK_TIME_NONE;
+
+ /* Create the elements */
+ data.playbin = gst_element_factory_make ("playbin", "playbin");
+
+ if (!data.playbin) {
+ g_printerr ("Not all elements could be created.\n");
+ return -1;
+ }
+
+ /* Set the URI to play */
+ g_object_set (data.playbin, "uri", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
+
+ /* Connect to interesting signals in playbin */
+ g_signal_connect (G_OBJECT (data.playbin), "video-tags-changed", (GCallback) tags_cb, &data);
+ g_signal_connect (G_OBJECT (data.playbin), "audio-tags-changed", (GCallback) tags_cb, &data);
+ g_signal_connect (G_OBJECT (data.playbin), "text-tags-changed", (GCallback) tags_cb, &data);
+
+ /* Create the GUI */
+ create_ui (&data);
+
+ /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
+ bus = gst_element_get_bus (data.playbin);
+ gst_bus_add_signal_watch (bus);
+ g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, &data);
+ g_signal_connect (G_OBJECT (bus), "message::eos", (GCallback)eos_cb, &data);
+ g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, &data);
+ g_signal_connect (G_OBJECT (bus), "message::application", (GCallback)application_cb, &data);
+ gst_object_unref (bus);
+
+ /* Start playing */
+ ret = gst_element_set_state (data.playbin, GST_STATE_PLAYING);
+ if (ret == GST_STATE_CHANGE_FAILURE) {
+ g_printerr ("Unable to set the pipeline to the playing state.\n");
+ gst_object_unref (data.playbin);
+ return -1;
+ }
+
+ /* Register a function that GLib will call every second */
+ g_timeout_add_seconds (1, (GSourceFunc)refresh_ui, &data);
+
+ /* Start the GTK main loop. We will not regain control until gtk_main_quit is called. */
+ gtk_main ();
+
+ /* Free resources */
+ gst_element_set_state (data.playbin, GST_STATE_NULL);
+ gst_object_unref (data.playbin);
+ return 0;
+}
+```
+
+> 
+> Need help?
+>
+> If you need help to compile this code, refer to the **Building the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Build), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Build) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Build), or use this specific command on Linux:
+>
+> ``gcc basic-tutorial-5.c -o basic-tutorial-5 `pkg-config --cflags --libs gstreamer-interfaces-1.0 gtk+-3.0 gstreamer-1.0``
+>
+>If you need help to run this code, refer to the **Running the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Run), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Run) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Run).
+>
+> This tutorial opens a GTK+ window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed. The Window has some GTK+ buttons to Pause, Stop and Play the movie, and a slider to show the current position of the stream, which can be dragged to change it. Also, information about the stream is shown on a column at the right edge of the window.
+>
+>
+> Bear in mind that there is no latency management (buffering), so on slow connections, the movie might stop after a few seconds. See how [](sdk-basic-tutorial-streaming.md) solves this issue.
+>
+> Required libraries: `gstreamer-video-1.0 gtk+-3.0 gstreamer-1.0`
+
+## Walkthrough
+
+Regarding this tutorial's structure, we are not going to use forward
+function definitions anymore: Functions will be defined before they are
+used. Also, for clarity of explanation, the order in which the snippets
+of code are presented will not always match the program order. Use the
+line numbers to locate the snippets in the complete code.
+
+``` c
+#include <gdk/gdk.h>
+#if defined (GDK_WINDOWING_X11)
+#include <gdk/gdkx.h>
+#elif defined (GDK_WINDOWING_WIN32)
+#include <gdk/gdkwin32.h>
+#elif defined (GDK_WINDOWING_QUARTZ)
+#include <gdk/gdkquartzwindow.h>
+#endif
+```
+
+The first thing worth noticing is that we are no longer completely
+platform-independent. We need to include the appropriate GDK headers for
+the windowing system we are going to use. Fortunately, there are not
+that many supported windowing systems, so these three lines often
+suffice: X11 for Linux, Win32 for Windows and Quartz for Mac OSX.
+
+This tutorial is composed mostly of callback functions, which will be
+called from GStreamer or GTK+, so let's review the `main` function,
+which registers all these callbacks.
+
+``` c
+int main(int argc, char *argv[]) {
+ CustomData data;
+ GstStateChangeReturn ret;
+ GstBus *bus;
+
+ /* Initialize GTK */
+ gtk_init (&argc, &argv);
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Initialize our data structure */
+ memset (&data, 0, sizeof (data));
+ data.duration = GST_CLOCK_TIME_NONE;
+
+ /* Create the elements */
+ data.playbin = gst_element_factory_make ("playbin", "playbin");
+
+ if (!data.playbin) {
+ g_printerr ("Not all elements could be created.\n");
+ return -1;
+ }
+
+ /* Set the URI to play */
+ g_object_set (data.playbin, "uri", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
+```
+
+Standard GStreamer initialization and playbin pipeline creation, along
+with GTK+ initialization. Not much new.
+
+``` c
+/* Connect to interesting signals in playbin */
+g_signal_connect (G_OBJECT (data.playbin), "video-tags-changed", (GCallback) tags_cb, &data);
+g_signal_connect (G_OBJECT (data.playbin), "audio-tags-changed", (GCallback) tags_cb, &data);
+g_signal_connect (G_OBJECT (data.playbin), "text-tags-changed", (GCallback) tags_cb, &data);
+```
+
+We are interested in being notified when new tags (metadata) appears on
+the stream. For simplicity, we are going to handle all kinds of tags
+(video, audio and text) from the same callback `tags_cb`.
+
+``` c
+/* Create the GUI */
+create_ui (&data);
+```
+
+All GTK+ widget creation and signal registration happens in this
+function. It contains only GTK-related function calls, so we will skip
+over its definition. The signals to which it registers convey user
+commands, as shown below when reviewing the
+callbacks.
+
+``` c
+/* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
+bus = gst_element_get_bus (data.playbin);
+gst_bus_add_signal_watch (bus);
+g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, &data);
+g_signal_connect (G_OBJECT (bus), "message::eos", (GCallback)eos_cb, &data);
+g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, &data);
+g_signal_connect (G_OBJECT (bus), "message::application", (GCallback)application_cb, &data);
+gst_object_unref (bus);
+```
+
+In [](sdk-playback-tutorial-playbin-usage.md), `gst_bus_add_watch()` is
+used to register a function that receives every message posted to the
+GStreamer bus. We can achieve a finer granularity by using signals
+instead, which allow us to register only to the messages we are
+interested in. By calling `gst_bus_add_signal_watch()` we instruct the
+bus to emit a signal every time it receives a message. This signal has
+the name `message::detail` where *`detail`* is the message that
+triggered the signal emission. For example, when the bus receives the
+EOS message, it emits a signal with the name `message::eos`.
+
+This tutorial is using the `Signals`'s details to register only to the
+messages we care about. If we had registered to the `message` signal, we
+would be notified of every single message, just like
+`gst_bus_add_watch()` would do.
+
+Keep in mind that, in order for the bus watches to work (be it a
+`gst_bus_add_watch()` or a `gst_bus_add_signal_watch()`), there must be
+GLib `Main Loop` running. In this case, it is hidden inside the
+[GTK+](http://www.gtk.org/) main loop.
+
+``` c
+/* Register a function that GLib will call every second */
+g_timeout_add_seconds (1, (GSourceFunc)refresh_ui, &data);
+```
+
+Before transferring control to GTK+, we use `g_timeout_add_seconds
+()` to register yet another callback, this time with a timeout, so it
+gets called every second. We are going to use it to refresh the GUI from
+the `refresh_ui` function.
+
+After this, we are done with the setup and can start the GTK+ main loop.
+We will regain control from our callbacks when interesting things
+happen. Let's review the callbacks. Each callback has a different
+signature, depending on who will call it. You can look up the signature
+(the meaning of the parameters and the return value) in the
+documentation of the signal.
+
+``` c
+/* This function is called when the GUI toolkit creates the physical window that will hold the video.
+ * At this point we can retrieve its handler (which has a different meaning depending on the windowing system)
+ * and pass it to GStreamer through the GstVideoOverlay interface. */
+static void realize_cb (GtkWidget *widget, CustomData *data) {
+ GdkWindow *window = gtk_widget_get_window (widget);
+ guintptr window_handle;
+
+ if (!gdk_window_ensure_native (window))
+ g_error ("Couldn't create native window needed for GstVideoOverlay!");
+
+ /* Retrieve window handler from GDK */
+#if defined (GDK_WINDOWING_WIN32)
+ window_handle = (guintptr)GDK_WINDOW_HWND (window);
+#elif defined (GDK_WINDOWING_QUARTZ)
+ window_handle = gdk_quartz_window_get_nsview (window);
+#elif defined (GDK_WINDOWING_X11)
+ window_handle = GDK_WINDOW_XID (window);
+#endif
+ /* Pass it to playbin, which implements GstVideoOverlay and will forward it to the video sink */
+ gst_video_overlay_set_window_handle (GST_VIDEO_OVERLAY (data->playbin), window_handle);
+}
+```
+
+The code comments talks by itself. At this point in the life cycle of
+the application, we know the handle (be it an X11's `XID`, a Window's
+`HWND` or a Quartz's `NSView`) of the window where GStreamer should
+render the video. We simply retrieve it from the windowing system and
+pass it to `playbin` through the `GstVideoOverlay` interface using
+`gst_video_overlay_set_window_handle()`. `playbin` will locate the video
+sink and pass the handler to it, so it does not create its own window
+and uses this one.
+
+Not much more to see here; `playbin` and the `GstVideoOverlay` really simplify
+this process a lot!
+
+``` c
+/* This function is called when the PLAY button is clicked */
+static void play_cb (GtkButton *button, CustomData *data) {
+ gst_element_set_state (data->playbin, GST_STATE_PLAYING);
+}
+
+/* This function is called when the PAUSE button is clicked */
+static void pause_cb (GtkButton *button, CustomData *data) {
+ gst_element_set_state (data->playbin, GST_STATE_PAUSED);
+}
+
+/* This function is called when the STOP button is clicked */
+static void stop_cb (GtkButton *button, CustomData *data) {
+ gst_element_set_state (data->playbin, GST_STATE_READY);
+}
+```
+
+These three little callbacks are associated with the PLAY, PAUSE and
+STOP buttons in the GUI. They simply set the pipeline to the
+corresponding state. Note that in the STOP state we set the pipeline to
+`READY`. We could have brought the pipeline all the way down to the
+`NULL` state, but, the transition would then be a little slower, since some
+resources (like the audio device) would need to be released and
+re-acquired.
+
+``` c
+/* This function is called when the main window is closed */
+static void delete_event_cb (GtkWidget *widget, GdkEvent *event, CustomData *data) {
+ stop_cb (NULL, data);
+ gtk_main_quit ();
+}
+```
+
+gtk_main_quit() will eventually make the call to to gtk_main_run()
+in `main` to terminate, which, in this case, finishes the program. Here,
+we call it when the main window is closed, after stopping the pipeline
+(just for the sake of tidiness).
+
+``` c
+/* This function is called everytime the video window needs to be redrawn (due to damage/exposure,
+ * rescaling, etc). GStreamer takes care of this in the PAUSED and PLAYING states, otherwise,
+ * we simply draw a black rectangle to avoid garbage showing up. */
+static gboolean expose_cb (GtkWidget *widget, GdkEventExpose *event, CustomData *data) {
+ if (data->state < GST_STATE_PAUSED) {
+ GtkAllocation allocation;
+ GdkWindow *window = gtk_widget_get_window (widget);
+ cairo_t *cr;
+
+ /* Cairo is a 2D graphics library which we use here to clean the video window.
+ * It is used by GStreamer for other reasons, so it will always be available to us. */
+ gtk_widget_get_allocation (widget, &allocation);
+ cr = gdk_cairo_create (window);
+ cairo_set_source_rgb (cr, 0, 0, 0);
+ cairo_rectangle (cr, 0, 0, allocation.width, allocation.height);
+ cairo_fill (cr);
+ cairo_destroy (cr);
+ }
+
+ return FALSE;
+}
+```
+
+When there is data flow (in the `PAUSED` and `PLAYING` states) the video
+sink takes care of refreshing the content of the video window. In the
+other cases, however, it will not, so we have to do it. In this example,
+we just fill the window with a black
+rectangle.
+
+``` c
+/* This function is called when the slider changes its position. We perform a seek to the
+ * new position here. */
+static void slider_cb (GtkRange *range, CustomData *data) {
+ gdouble value = gtk_range_get_value (GTK_RANGE (data->slider));
+ gst_element_seek_simple (data->playbin, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT,
+ (gint64)(value * GST_SECOND));
+}
+```
+
+This is an example of how a complex GUI element like a seeker bar (or
+slider that allows seeking) can be very easily implemented thanks to
+GStreamer and GTK+ collaborating. If the slider has been dragged to a
+new position, tell GStreamer to seek to that position
+with `gst_element_seek_simple()` (as seen in [Basic tutorial 4: Time
+management](sdk-basic-tutorial-time-management.md)). The
+slider has been setup so its value represents seconds.
+
+It is worth mentioning that some performance (and responsiveness) can be
+gained by doing some throttling, this is, not responding to every single
+user request to seek. Since the seek operation is bound to take some
+time, it is often nicer to wait half a second (for example) after a seek
+before allowing another one. Otherwise, the application might look
+unresponsive if the user drags the slider frantically, which would not
+allow any seek to complete before a new one is queued.
+
+``` c
+/* This function is called periodically to refresh the GUI */
+static gboolean refresh_ui (CustomData *data) {
+ gint64 current = -1;
+
+ /* We do not want to update anything unless we are in the PAUSED or PLAYING states */
+ if (data->state < GST_STATE_PAUSED)
+ return TRUE;
+```
+
+This function will move the slider to reflect the current position of
+the media. First off, if we are not in the `PLAYING` state, we have
+nothing to do here (plus, position and duration queries will normally
+fail).
+
+``` c
+/* If we didn't know it yet, query the stream duration */
+if (!GST_CLOCK_TIME_IS_VALID (data->duration)) {
+ if (!gst_element_query_duration (data->playbin, GST_FORMAT_TIME, &data->duration)) {
+ g_printerr ("Could not query current duration.\n");
+ } else {
+ /* Set the range of the slider to the clip duration, in SECONDS */
+ gtk_range_set_range (GTK_RANGE (data->slider), 0, (gdouble)data->duration / GST_SECOND);
+ }
+}
+```
+
+We recover the duration of the clip if we didn't know it, so we can set
+the range for the slider.
+
+``` c
+if (gst_element_query_position (data->playbin, GST_FORMAT_TIME, ¤t)) {
+ /* Block the "value-changed" signal, so the slider_cb function is not called
+ * (which would trigger a seek the user has not requested) */
+ g_signal_handler_block (data->slider, data->slider_update_signal_id);
+ /* Set the position of the slider to the current pipeline positoin, in SECONDS */
+ gtk_range_set_value (GTK_RANGE (data->slider), (gdouble)current / GST_SECOND);
+ /* Re-enable the signal */
+ g_signal_handler_unblock (data->slider, data->slider_update_signal_id);
+}
+return TRUE;
+```
+
+We query the current pipeline position, and set the position of the
+slider accordingly. This would trigger the emission of the
+`value-changed` signal, which we use to know when the user is dragging
+the slider. Since we do not want seeks happening unless the user
+requested them, we disable the `value-changed` signal emission during
+this operation with `g_signal_handler_block()` and
+`g_signal_handler_unblock()`.
+
+Returning TRUE from this function will keep it called in the future. If
+we return FALSE, the timer will be
+removed.
+
+``` c
+/* This function is called when new metadata is discovered in the stream */
+static void tags_cb (GstElement *playbin, gint stream, CustomData *data) {
+ /* We are possibly in a GStreamer working thread, so we notify the main
+ * thread of this event through a message in the bus */
+ gst_element_post_message (playbin,
+ gst_message_new_application (GST_OBJECT (playbin),
+ gst_structure_new ("tags-changed", NULL)));
+}
+```
+
+This is one of the key points of this tutorial. This function will be
+called when new tags are found in the media, **from a streaming
+thread**, this is, from a thread other than the application (or main)
+thread. What we want to do here is to update a GTK+ widget to reflect
+this new information, but **GTK+ does not allow operating from threads
+other than the main one**.
+
+The solution is to make `playbin` post a message on the bus and return
+to the calling thread. When appropriate, the main thread will pick up
+this message and update GTK.
+
+`gst_element_post_message()` makes a GStreamer element post the given
+message to the bus. `gst_message_new_application()` creates a new
+message of the `APPLICATION` type. GStreamer messages have different
+types, and this particular type is reserved to the application: it will
+go through the bus unaffected by GStreamer. The list of types can be
+found in the `GstMessageType` documentation.
+
+Messages can deliver additional information through their embedded
+`GstStructure`, which is a very flexible data container. Here, we create
+a new structure with `gst_structure_new`, and name it `tags-changed`, to
+avoid confusion in case we wanted to send other application messages.
+
+Later, once in the main thread, the bus will receive this message and
+emit the `message::application` signal, which we have associated to the
+`application_cb` function:
+
+``` c
+/* This function is called when an "application" message is posted on the bus.
+ * Here we retrieve the message posted by the tags_cb callback */
+static void application_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ if (g_strcmp0 (gst_structure_get_name (gst_message_get_structure (msg)), "tags-changed") == 0) {
+ /* If the message is the "tags-changed" (only one we are currently issuing), update
+ * the stream info GUI */
+ analyze_streams (data);
+ }
+}
+```
+
+Once me made sure it is the `tags-changed` message, we call the
+`analyze_streams` function, which is also used in [](sdk-playback-tutorial-playbin-usage.md) and is
+more detailed there. It basically recovers the tags from the stream and
+writes them in a text widget in the GUI.
+
+The `error_cb`, `eos_cb` and `state_changed_cb` are not really worth
+explaining, since they do the same as in all previous tutorials, but
+from their own function now.
+
+And this is it! The amount of code in this tutorial might seem daunting
+but the required concepts are few and easy. If you have followed the
+previous tutorials and have a little knowledge of GTK, you probably
+understood this one can now enjoy your very own media player!
+
+
+
+## Exercise
+
+If this media player is not good enough for you, try to change the text
+widget that displays the information about the streams into a proper
+list view (or tree view). Then, when the user selects a different
+stream, make GStreamer switch streams! To switch streams, you will need
+to read [](sdk-playback-tutorial-playbin-usage.md).
+
+## Conclusion
+
+This tutorial has shown:
+
+ - How to output the video to a particular window handle
+ using `gst_video_overlay_set_window_handle()`.
+
+ - How to refresh the GUI periodically by registering a timeout
+ callback with `g_timeout_add_seconds ()`.
+
+ - How to convey information to the main thread by means of application
+ messages through the bus with `gst_element_post_message()`.
+
+ - How to be notified only of interesting messages by making the bus
+ emit signals with `gst_bus_add_signal_watch()` and discriminating
+ among all message types using the signal details.
+
+This allows you to build a somewhat complete media player with a proper
+Graphical User Interface.
+
+The following basic tutorials keep focusing on other individual
+GStreamer topics
+
+It has been a pleasure having you here, and see you soon!
--- /dev/null
+---
+short-description: General topics required to understand the rest of the tutorials
+...
+
+# Basic tutorials
+
+These tutorials describe general topics required to understand the rest
+of tutorials.
--- /dev/null
+# Building from source using Cerbero
+
+> ![Warning] This section is intended for advanced users.
+
+## Build requirements
+
+The GStreamer build system provides bootstrapping facilities for all
+platforms, but it still needs a minimum base to bootstrap:
+
+- python > 2.6 and python's `argparse` module, which is already
+ included in python2.7.
+- git
+
+### Windows users
+
+Cerbero can be used on Windows using the Msys/MinGW shell (a Unix-like
+shell for Windows). There is a bit of setup that you need to do before
+Cerbero can take control.
+
+You need to install the following programs:
+
+- [Python 2.7]
+- [Git] (Select the install option "Checkout as-is, Commit as-is" and
+ install it in a path without spaces, eg: c:\Git)
+- [Msys/MinGW] (Install it with all the options enabled)
+- [CMake] (Select the option "Add CMake in system path for the
+ current user")
+- [Yasm] (Download the win32 or win64 version for your platform, name
+ it `yasm.exe`, and place it in your MinGW `bin` directory,
+ typically, `C:\MinGW\bin`)
+- [WiX 3.5]
+- [Microsoft SDK 7.1] (Install the SDK samples and the Visual C++
+ Compilers, required to build the DirectShow base classes. Might need
+ installing the .NET 4 Framework first if the SDK installer doesn't
+ find it)
+- [Windows Driver Kit 7.1.0]
+
+Your user ID can't have spaces (eg: John Smith). Paths with spaces are
+not correctly handled in the build system and msys uses the user ID for
+the home folder.
+
+Cerbero must be run in the MinGW shell, which is accessible from the
+main menu once MinGW is installed.
+
+The last step is making `python` and `git` available from the shell, for
+which you will need to create a `.profile` file. Issue this command from
+within the MinGW shell:
+
+`echo "export PATH=\"\$PATH:/c/Python27:/c/Git/bin\"" >> ~/.profile`
+
+Using the appropriate paths to where you installed `python` and `git`
+
+(Note that inside the shell, / is mapped to c:\Mingw\msys\1.0 )
+
+### OS X users
+
+To use cerbero on OS X you need to install the "Command Line Tools" from
+XCode. They are available from the "Preferences" dialog under
+"Downloads".
+
+### iOS developers
+
+If you want to build GStreamer for iOS, you also need the iOS
+SDK. The minimum required iOS SDK version is 6.0 and is included in
+[XCode] since version 4.
+
+## Download the sources
+
+To build GStreamer, you first need to download **Cerbero**.
+Cerbero is a multi-platform build system for Open Source projects that
+builds and creates native packages for different platforms,
+architectures and distributions.
+
+Get a copy of Cerbero by cloning the git repository:
+
+ git clone git://anongit.freedesktop.org/gstreamer/cerbero
+
+Cerbero can be run uninstalled and for convenience you can create an
+alias in your `.bashrc` file*. *If you prefer to skip this step,
+remember that you need to replace the calls to `cerbero` with
+`./cerbero-uninstalled` in the next steps.
+
+ echo "alias cerbero='~/git/cerbero/cerbero-uninstalled'" >> ~/.bashrc
+
+## Setup environment
+
+After Cerbero and the base requirements are in place, you need to setup
+the build environment.
+
+Cerbero reads the configuration file `$HOME/.cerbero/cerbero.cbc` to
+determine the build options. This file is a python code which allows
+overriding/defining some options.
+
+If the file does not exist, Cerbero will try to determine the distro you
+are running and will use default build options such as the default build
+directory. The default options should work fine on the supported
+distributions.
+
+An example configuration file with detailed comments can be found [here]
+
+To fire up the bootstrapping process, go to the directory where you
+cloned/unpacked Cerbero and type:
+
+ cerbero bootstrap
+
+Enter the superuser/root password when prompted.
+
+The bootstrap process will then install all packages required to build
+GStreamer.
+
+## Build GSTreamer
+
+To generate GStreamer binaries, use the following command:
+
+ cerbero package gstreamer-1.0
+
+This should build all required GStreamer components and create packages for
+your distribution at the Cerbero source directory.
+
+A list of supported packages to build can be retrieved using:
+
+ cerbero list-packages
+
+Packages are composed of 0 (in case of a meta package) or more
+components that can be built separately if desired. The components are
+defined as individual recipes and can be listed with:
+
+ cerbero list
+
+To build an individual recipe and its dependencies, do the following:
+
+ cerbero build <recipe_name>
+
+Or to build or force a rebuild of a recipe without building its
+dependencies use:
+
+ cerbero buildone <recipe_name>
+
+To wipe everything and start from scratch:
+
+ cerbero wipe
+
+Once built, the output of the recipes will be installed at the prefix
+defined in the Cerbero configuration file `$HOME/.cerbero/cerbero.cbc`
+or at `$HOME/cerbero/dist` if no prefix is defined.
+
+### Build a single project with GStreamer
+
+Rebuilding the whole GStreamer is relatively fast on Linux and OS X, but it
+can be very slow on Windows, so if you only need to rebuild a single
+project (eg: gst-plugins-good to patch qtdemux) there is a much faster
+way of doing it. You will need to follow the steps detailed in this
+page, but skipping the step "**Build GStreamer**", and installing the
+GStreamer's development files as explained in [Installing GStreamer].
+
+By default, Cerbero uses as prefix a folder in the user directory with
+the following schema \~/cerbero/dist/$platform\_$arch, but for GStreamer
+we must change this prefix to use its installation directory. This can
+be done with a custom configuration file named *custom.cbc*:
+
+ # For Windows x86
+ prefix='/c/gstreamer/1.0/x86/'
+
+ # For Windows x86_64
+ #prefix='/c/gstreamer/1.0/x86_64'
+
+ # For Linux
+ #prefix='/opt/gstreamer'
+
+ # For OS X
+ #prefix='/Library/Frameworks/GStreamer.framework/Versions/1.0'
+
+The prefix path might not be writable by your current user. Make sure
+you fix it before, for instance with:
+
+ $ sudo chown -R <username> /Library/Frameworks/GStreamer.framework/
+
+Cerbero has a shell command that starts a new shell with all the
+environment set up to target GStreamer. You can start a new shell using
+the installation prefix defined in *custom.cbc *with the following
+command:
+
+ $ cerbero -c custom.cbc shell
+
+Once you are in Cerbero's shell you can compile new projects targeting
+GStreamer using the regular build process:
+
+ $ git clone git://anongit.freedesktop.org/gstreamer/gst-plugins-good; cd gst-plugins-good
+ $ sh autogen.sh --disable-gtk-doc --prefix=<prefix>
+ $ make -C gst/isomp4
+
+### Cross-compilation of GStreamer
+
+Cerbero can be used to cross-compile GStreamer to other platforms like
+Android or Windows. You only need to use a configuration file that sets
+the target platform, but we also provide a set of of pre-defined
+configuration files for the supported platforms (you will find them in
+the `config` folder with the `.cbc` extension
+
+#### Android
+
+You can cross-compile GStreamer for Android from a Linux host using the
+configuration file `config/cross-android.cbc`. Replace all the previous
+commands with:
+
+ cerbero -c config/cross-android.cbc <command>
+
+#### Windows
+
+GStreamer can also be cross-compiled to Windows from Linux, but you should
+only use it for testing purpose. The DirectShow plugins cannot be
+cross-compiled yet and WiX can't be used with Wine yet, so packages can
+only be created from Windows.
+
+Replace all the above commands for Windows 32bits with:
+
+ cerbero -c config/cross-win32.cbc <command>
+
+Or with using the following for Windows 64bits:
+
+ cerbero -c config/cross-win64.cbc <command>
+
+#### iOS
+
+To cross compile for iOS from OS X, use the configuration file
+`config/cross-ios-universal.cbc`. Replace all previous commands with:
+
+ cerbero -c config/cross-ios-universal.cbc <command>
+
+ [Warning]: images/icons/emoticons/warning.png
+ [Python 2.7]: http://www.python.org/getit/releases/2.7/
+ [Git]: http://code.google.com/p/msysgit/downloads/list?q=full+installer+official+git
+ [Msys/MinGW]: https://sourceforge.net/projects/mingw/files/Installer/mingw-get-inst/
+ [CMake]: http://www.cmake.org/cmake/resources/software.htm
+ [Yasm]: http://yasm.tortall.net/Download.html
+ [WiX 3.5]: http://wix.codeplex.com/releases/view/60102
+ [Microsoft SDK 7.1]: http://www.microsoft.com/en-us/download/details.aspx?id=8279
+ [Windows Driver Kit 7.1.0]: http://msdn.microsoft.com/en-us/windows/hardware/hh852365
+ [XCode]: https://developer.apple.com/devcenter/ios/index.action#downloads
+ [here]: http://www.freedesktop.org/software/gstreamer-sdk/cerbero.cbc.template
+ [Installing GStreamer]: sdk-installing.md
--- /dev/null
+---
+short-description: Deploy GStreamer with your application
+...
+
+# Deploying your application
+
+Once the development of your application is finished, you will need to
+deploy it to the target machine, usually in the form of a package or
+installer. You have several options here, and, even though this subject
+is not really in the scope of this documentation, we will give some
+hints to try to help you.
+
+## Multiplatform vs. single-platform packaging system
+
+The first choice you need to make is whether you want to deploy your
+application to more than one platform. If yes, then you have the choice
+to use a different packaging system for each platform, or use one that
+can deliver to all platforms simultaneously. This table summarizes the
+pros and cons of each option.
+
+<table>
+<colgroup>
+<col width="33%" />
+<col width="33%" />
+<col width="33%" />
+</colgroup>
+<thead>
+<tr class="header">
+<th> </th>
+<th>Pros</th>
+<th>Cons</th>
+</tr>
+</thead>
+<tbody>
+<tr class="odd">
+<td><p><strong>Multiplatform packaging system</strong></p>
+<p>The same system is used to package your application for all platforms</p></td>
+<td><ul>
+<li><p>You only need to develop your packaging system once, and it works for all supported platforms.</p></li>
+</ul></td>
+<td><ul>
+<li>On some platforms, the packaging system might impose artificial restrictions inherited from the other platforms.</li>
+</ul></td>
+</tr>
+<tr class="even">
+<td><p><strong>Single-platform packaging system</strong></p>
+<p>Your application is packaged using a different system on each platform.</p></td>
+<td><ul>
+<li><p>You can make use of all the advantages each packaging system can offer.</p>
+</li>
+</ul></td>
+<td><ul>
+<li><p>You need to develop a new packaging system for each supported platform.</p></li>
+</ul></td>
+</tr>
+</tbody>
+</table>
+
+GStreamer itself supports many different platforms (Linux, iOS, Android, Mac
+OS X, Windows, etc) and has been built using a multiplatform packaging
+system named **Cerbero**, which is available for you to use, should you
+choose to go down this route.
+
+## Shared vs. private GStreamer deployment
+
+You can install GStreamer in the target machine in the same way
+you installed it in your development machine, you can deploy it
+privately, or you can even customize it before deploying. Here you have
+a few options:
+
+<table>
+<colgroup>
+<col width="33%" />
+<col width="33%" />
+<col width="33%" />
+</colgroup>
+<thead>
+<tr class="header">
+<th></th>
+<th>Pros</th>
+<th>Cons</th>
+</tr>
+</thead>
+<tbody>
+<tr class="odd">
+<td><p><strong>Shared GStreamer</strong></p>
+<p>GStreamer is installed independently of your application, as a prerequisite, in a common place in the target computer (<code>C:\Program Files</code>, for example). You application uses an environment variable to locate it.</p></td>
+<td><ul>
+<li>If more than one application in the target computer uses GStreamer, it is installed only once and shared, reducing disk usage.</li>
+</ul></td>
+<td><ul>
+<li>Tampering or corruption of the shared GStreamer installation can make your application fail.</li>
+</ul></td>
+</tr>
+<tr class="even">
+<td><p><strong>Private GStreamer with dynamic linking</strong></p>
+<p>Your application deploys GStreamer to a private folder.</p></td>
+<td><ul>
+<li>Your GStreamer is independent of other applications, so it does not get corrupted if other applications mess with their installations.</li>
+</ul></td>
+<td><ul>
+<li>If multiple applications in the target computer use GStreamer, it won’t be shared, consuming more disk space.</li>
+</ul></td>
+</tr>
+<tr class="odd">
+<td><p><strong>Private GStreamer with static linking</strong></p>
+<p>Your application links statically against GStreamer, so it effectively becomes part of your application binary.</p></td>
+<td><ul>
+<li>Your GStreamer is independent of other applications, so it does not get corrupted if other applications mess with their installations.</li>
+<li>Deployment for ordinary users is easier as you have fewer files.</li>
+<li>This is your only choice on iOS.</li>
+</ul></td>
+<td><ul>
+<li>If multiple applications in the target computer use GStreamer, it won’t be shared, consuming more disk space.</li>
+<li>You need to provide the required files for your users to re-link your application against a modified GStreamer as required by the license.</li>
+</ul></td>
+</tr>
+</tbody>
+</table>
+
+The following pages give further directions for some of the above
+options.
+
+ - Platform-specific packaging methods:
+ - For [Mac OS X](sdk-mac-osx-deployment.md)
+ - For [Windows](sdk-windows-deployment.md)
+ - [Multiplatform deployment using
+ Cerbero](sdk-multiplatform-deployment-using-cerbero.md)
--- /dev/null
+# Installing for Android development
+
+![information] All versions starting from 2.3.1 Gingerbread are supported
+
+## Prerequisites
+
+The development machine is where you will develop your Android
+application, which then you will deploy on the target machine, which
+should obviously be an Android device.
+
+The development machine can either be a Linux, Mac OS X or Windows, and
+needs to have installed:
+
+- The latest version of the [Android SDK]
+- The latest version of the [Android NDK]
+- GStreamer for Android is targeted at API version 9 (Android
+ 2.3.1, Gingerbread) or higher. Use the SDK Manager tool to make sure
+ you have at least one Android SDK platform installed with API
+ version 9 or higher.
+
+<!-- Optionally, you can use the [Android Studio](https://developer.android.com/studio/index.html). As stated in
+the Android documentation, *developing in Android Studio is highly
+recommended and is the fastest way to get started*. -->
+
+Before continuing, make sure you can compile and run the samples
+included in the Android NDK, and that you understand how the integration
+of C and Java works via the [Java Native Interface] (JNI). Besides the
+[Android NDK] documentation, you can find some useful [Android JNI tips
+here].
+
+## Download and install GStreamer binaries
+
+The GStreamer project provides [prebuilt binaries] you should download
+the latest version and unzip it into any folder of your choice.
+
+In the process of building GStreamer-enabled Android applications,
+some tools will need to know where you installed the GStreamer
+binaries. You must define an environment variable called
+`GSTREAMER_ROOT_ANDROID` and point it to the folder where you
+extracted the GStreamer binaries. This environment variable must be available at
+build time, so maybe you want to make it available system-wide by
+adding it to your `~/.profile` file (on Linux and Mac) or to the
+Environment Variables in the System Properties dialog (on Windows).
+
+Point `GSTREAMER_ROOT_ANDROID` to the folder where you unzipped the binaries.
+
+> ![information] If you plan to use Android Studio and do not want to define this
+> environment variable globally, you can set it inside the build.gradle.
+
+> ![information] If you plan to use Eclipse, and do not want to define this
+> environment variable globally, you can set it inside Eclipse. Go to
+> Window → Preferences → C/C++ → Build → Build Variables and define
+> `GSTREAMER_ROOT_ANDROID` there.
+
+> ![warning] The NDK support in the Gradle build system used by
+> Android Studio is still in beta, so the recommended way to build
+> using the GStreamer SDK is still to use "ndk-build".
+
+## Configure your development environment
+
+There are two routes to use GStreamer in an Android application: Either
+writing your GStreamer code in Java or in C.
+
+Android applications are mainly written in Java, so adding GStreamer
+code to them in the same language is a huge advantage. However, this
+requires using [language bindings] for the GStreamer API which are not
+complete yet. In the meantime, this documentation will use Java for the
+User Interface (UI) part and C for the GStreamer code. Both parts
+interact through [JNI][Java Native Interface].
+
+### Building the tutorials
+
+There are a few Android-specific tutorials in the
+`$GSTREAMER_ROOT_ANDROID/share/gst-sdk/tutorials` folder. Each
+tutorial is a folder containing source code (in Java and C) and the
+resource files required to build a complete Android application.
+
+The rest of the GStreamer tutorials (basic and playback tutorials)
+cannot be run on Android without modification.
+
+Android projects with GStreamer support are built like conventional
+Android NDK projects, so the instructions at the [Android NDK] home can
+be followed:
+
+<!--
+#### Using Android Studio
+
+> ![warning] To be completed!!
+-->
+
+#### Using Eclipse
+
+Make sure you have installed the ADT and NDK plugins listed in the
+prerequisites section, and that they are both aware of the location of
+the Android SDK and NDK respectively.
+
+Import a tutorial into the Eclipse workspace:
+File → New → Project… → Android Project from Existing Code, and select
+the folder called `android-tutorial-1`.
+
+After reading in the project and generating some extra files and
+folders, Eclipse might complain about missing files. **This is normal**,
+we are not finished yet.
+
+Provide native development support by activating the NDK plugin:
+Right-click on the project in the Project Explorer (this should be the
+top-most folder,
+called `com.gst_sdk_tutorials.tutorial_1.Tutorial1`) → Android
+tools → Add Native Support… Here the NDK plugin asks for a library name.
+This is irrelevant and any valid file name will do. Accept.
+
+Eclipse will still complain about errors in the code. **This is
+normal**. Some files are missing because they are generated during the
+first build run.
+
+Build the project: Project → Build Project. If you bring up the Eclipse
+Console, you should see some progress messages. Once finished, the
+missing files will appear and all error messages should be gone. The
+project is now ready to run. Hit Run → Run.
+
+A new application called “Android tutorial 1” should now be available on
+your device, with the GStreamer logo. If you want to run the
+tutorial in an Android Virtual Device (AVD), make sure to create the
+device with support for audio playback and GPU Emulation (to enable
+OpenGL ES).
+
+#### Using the command line
+
+> ![warning] Note that, on Windows, this procedure requires a working Cygwin
+> shell, as explained in the [Android NDK System Requirements]
+
+For each tutorial, move to its folder and run:
+
+ android update project -p . -s --target X
+
+Where `X` is one of the targets available in your system (the ones you
+installed with the SDK manager). Make sure to use a target with at least
+API level 9.
+
+To get a list of all available targets in your system issue this
+command:
+
+ android list
+
+The “update project” command generates the `build.xml` file needed by
+the build system. You only need to perform this action once per project.
+
+To build the C part, just call:
+
+ ndk-build
+
+A few lines in the `Android.mk` file (reviewed later) pull up the
+necessary machinery to compile the GStreamer bits and generate the
+Shared Object libraries (.so) that the Java code can use as native
+methods.
+
+Finally, compile the Java code with:
+
+ ant debug
+
+And install on the device with:
+
+ adb install -r bin/Tutorial1-debug.apk
+
+The `-r` switch allows the installer to overwrite previous versions.
+Otherwise, you need to manually uninstall previous versions of your
+application.
+
+A new application called “Android tutorial 1” should now be available on
+your device, with the GStreamer logo. If you want to run the
+tutorial in an Android Virtual Device (AVD), make sure to create the
+device with support for audio playback and GPU Emulation (to enable
+OpenGL ES).
+
+> ![warning] Windows linkage problems
+>
+> Due to problems related to the standard linker, Google’s
+> <a href="http://en.wikipedia.org/wiki/Gold_(linker)" class="external-link">Gold
+> Linker</a> is used to build GStreamer applications. Unfortunately,
+> the Android NDK toolchain for Windows does not include the gold linker
+> and the standard one has to be used.
+>
+> If you observe linkage problems, you can replace the linker in your
+> Android NDK with the gold one from [this project]. Download the
+> `android-ndk-r8b-ma-windows.7z` file, extract
+> `\android-ndk-r8b\toolchains\arm-linux-androideabi-4.6\prebuilt\windows\arm-linux-androideabi\bin\ld.exe`
+> (only this file is needed) and overwrite the one in the same folder in
+> your Android NDK installation. You might need the free [7-Zip
+> archiving utility]
+
+### Creating new projects
+
+Create a normal NDK project, either from the command line as described
+in the [Android NDK][2] home, or use Eclipse: File → New → Project…
+→ Android Application Project, and, once the wizard is complete, right
+click on the project → Android Tools → Add Native Support …
+
+To add GStreamer support you only need to modify the
+`jni/Android.mk` file. This file describes the native files in your
+project, and its barebones structure (as auto-generated by Eclipse) is:
+
+**Android.mk**
+
+ LOCAL_PATH := $(call my-dir)
+
+ include $(CLEAR_VARS)
+
+ LOCAL_MODULE := NativeApplication
+ LOCAL_SRC_FILES := NativeApplication.c
+
+ include $(BUILD_SHARED_LIBRARY)
+
+Where line 5 specifies the name of the `.so` file that will contain your
+native code and line 6 states all source files that compose your native
+code, separated by spaces.
+
+Adding GStreamer support only requires adding these lines:
+
+**Android.mk with GStreamer support**
+
+ LOCAL_PATH := $(call my-dir)
+
+ include $(CLEAR_VARS)
+
+ LOCAL_MODULE := NativeApplication
+ LOCAL_SRC_FILES := NativeApplication.c
+ LOCAL_SHARED_LIBRARIES := gstreamer_android
+ LOCAL_LDLIBS := -landroid
+
+ include $(BUILD_SHARED_LIBRARY)
+
+ ifndef GSTREAMER_ROOT
+ ifndef GSTREAMER_ROOT_ANDROID
+ $(error GSTREAMER_ROOT_ANDROID is not defined!)
+ endif
+ GSTREAMER_ROOT := $(GSTREAMER_ROOT_ANDROID)
+ endif
+
+ GSTREAMER_NDK_BUILD_PATH := $(GSTREAMER_ROOT)/share/gst-android/ndk-build/
+ GSTREAMER_PLUGINS := coreelements ogg theora vorbis videoconvert audioconvert audioresample playback glimagesink soup opensles
+ G_IO_MODULES := gnutls
+ GSTREAMER_EXTRA_DEPS := gstreamer-video-1.0
+
+ include $(GSTREAMER_NDK_BUILD_PATH)/gstreamer.mk
+
+Where line 7 specifies an extra library to be included in the project:
+`libgstreamer_android.so`. This library contains all GStreamer code,
+tailored for your application’s needs, as shown below.
+
+Line 8 specifies additional system libraries, in this case, in order to
+access android-specific functionality.
+
+Lines 12 and 13 simply define some convenient macros.
+
+Line 20 lists the plugins you want statically linked into
+`libgstreamer_android.so`. Listing only the ones you need makes your
+application smaller.
+
+Line 21 is required to have HTTPS/TLS support from GStreamer, through the
+`souphttpsrc` element.
+
+Line 22 defines which GStreamer libraries your application requires.
+
+Finally, line 24 includes the make files which perform the rest of the
+magic.
+
+Listing all desired plugins can be cumbersome, so they have been grouped
+into categories, which can be used by including the `plugins.mk` file,
+and used as follows:
+
+ include $(GSTREAMER_NDK_BUILD_PATH)/plugins.mk
+ GSTREAMER_PLUGINS := $(GSTREAMER_PLUGINS_CORE) $(GSTREAMER_PLUGINS_CODECS) playbin souphttpsrc
+
+#### List of categories and included plugins
+
+| Category | Included plugins |
+|--------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `GSTREAMER_PLUGINS_CORE` | coreelements coreindexers adder app audioconvert audiorate audioresample audiotestsrc videoconvert gdp gio pango typefindfunctions videorate videoscale videotestsrc volume autodetect videofilter |
+| `GSTREAMER_PLUGINS_PLAYBACK` | decodebin playbin |
+| `GSTREAMER_PLUGINS_CODECS` | subparse ogg theora vorbis alaw annodex apetag audioparsers auparse avi flac flv flxdec icydemux id3demux isomp4 jpeg matroska mulaw multipart png speex taglib wavenc wavpack wavparse y4menc adpcmdec adpcmenc aiff cdxaparse dtmf dvbsuboverlay dvdspu fragmented hdvparse id3tag ivfparse jp2k kate mve mxf nsf nuvdemux opus pcapparse pnm schro siren subenc tta videoparsersbad vmnc vp8 y4mdec |
+| `GSTREAMER_PLUGINS_VIS` | libvisual goom goom2k1 audiovisualizers |
+| `GSTREAMER_PLUGINS_EFFECTS` | alpha alphacolor audiofx cutter debug deinterlace effectv equalizer gdkpixbuf imagefreeze interleave level multifile replaygain shapewipe smpte spectrum videobox videocrop videomixer autoconvert bayer coloreffects faceoverlay fieldanalysis freeverb frei0r gaudieffects geometrictransform interlace jp2kdecimator liveadder rawparse removesilence scaletempoplugin segmentclip smooth speed stereo videofiltersbad videomeasure videosignal |
+| `GSTREAMER_PLUGINS_NET` | rtsp rtp rtpmanager souphttpsrc udp dataurisrc rtpmux rtpvp8 sdpelem |
+| `GSTREAMER_PLUGINS_CODECS_GPL` | assrender |
+| `GSTREAMER_PLUGINS_SYS` | glimagesink opensles amc |
+
+Build and run your application as explained in the [Building the tutorial](sdk-installing-for-android-development.md#building-the-tutorials) section.
+
+ [information]: images/icons/emoticons/information.png
+ [Android SDK]: http://developer.android.com/sdk/index.html
+ [Android NDK]: http://developer.android.com/tools/sdk/ndk/index.html
+ [Java Native Interface]: http://en.wikipedia.org/wiki/Java_Native_Interface
+ [Android JNI tips here]: http://developer.android.com/guide/practices/jni.html
+ [prebuilt binaries]: https://gstreamer.freedesktop.org/data/pkg/android/
+ [language bindings]: http://en.wikipedia.org/wiki/Language_binding
+ [warning]: images/icons/emoticons/warning.png
+ [Android NDK System Requirements]: http://developer.android.com/tools/sdk/ndk/index.html#Reqs
+ [this project]: http://code.google.com/p/mingw-and-ndk/downloads/detail?name=android-ndk-r8b-ma-windows.7z&can=2&q=
+ [7-Zip archiving utility]: http://www.7-zip.org/
+ [2]: http://developer.android.com/tools/sdk/ndk/index.html#GetStarted
--- /dev/null
+# Installing for iOS development
+
+ All versions starting from iOS 6 are supported
+
+### Prerequisites
+
+For iOS development you need to download Xcode and the iOS SDK. Xcode
+can be found at the App Store or
+[here](https://developer.apple.com/devcenter/ios/index.action#downloads)
+and the iOS SDK, if it is not already included in your version of Xcode,
+can be downloaded from Xcode's preferences menu under the downloads tab.
+The minimum required iOS version is 6.0. The minimum required version of
+Xcode is 4, but 7.3 is recommended.
+
+In case you are not familiar with iOS, Objective-C or Xcode, we
+recommend taking a look at the available documentation at Apple's
+website.
+[This](http://developer.apple.com/library/ios/#DOCUMENTATION/iPhone/Conceptual/iPhone101/Articles/00_Introduction.html) can be a good starting point.
+
+## Download and install GStreamer binaries
+
+GStreamer binary installer can be found at:
+
+[https://gstreamer.freedesktop.org/data/pkg/ios/](https://gstreamer.freedesktop.org/data/pkg/ios/)
+
+Double click the package file and follow the instructions presented by
+the install wizard. In case the system complains about the package not
+being signed, you can control-click it and open to start the
+installation. When you do this, it will warn you, but there is an option
+to install anyway. Otherwise you can go to System Preferences → Security
+and Privacy → General and select the option to allow installation of
+packages from "anywhere".
+
+The GStreamer SDK installs itself in your home directory, so it is
+available only to the user that installed it. The SDK library is
+installed to `~/Library/Developer/GStreamer/iPhone.sdk`. Inside this
+directory there is the GStreamer.framework that contains the libs,
+headers and resources, and there is a `Templates` directory that has
+Xcode application templates for GStreamer development. Those templates
+are also copied to `~/Library/Developer/Xcode/Templates` during
+installation so that Xcode can find them.
+
+### Configure your development environment
+
+GStreamer is written in C, and the iOS API uses mostly Objective-C (and
+C for some parts), but this should cause no problems as those languages
+interoperate freely. You can mix both in the same source code, for
+example.
+
+#### Building the tutorials
+
+GStreamer provides a few tutorials in the `xcode iOS` folder inside
+the `.dmg` file. Copy them out of the package and into a more suitable
+place. We recommend that you open the project in Xcode, take a look
+at the sources and build them. This should confirm that the installation
+works and give some insight on how simple it is to mix Objective-C and C
+code.
+
+#### Creating new projects
+
+After installation, when creating a new Xcode project, you should see
+the GStreamer project templates under the `Templates` category. OS X and
+iOS have a different way of organizing libraries headers and binaries.
+They are grouped into Frameworks, and that's how we ship GStreamer and
+its dependencies for iOS (and OS X). Due to this difference from
+traditional linux development, we strongly recommend using the SDK
+templates, as they set a few variables on your project that allows Xcode
+to find, use and link GStreamer just like in traditional linux
+development. For example, if you don't use the templates, you'll have to
+use:
+
+```
+#include <GStreamer/gst/gst.h>
+```
+
+instead of the usual:
+
+```
+#include <gst/gst.h>
+```
+
+Among some other things the template does, this was a decision made to
+keep development consistent across all the platforms the SDK supports.
+
+Once a project has been created using a GStreamer SDK Template, it is
+ready to build and run. All necessary infrastructure is already in
+place. To understand what files have been created and how they interact,
+take a look at the [iOS tutorials](sdk-ios-tutorials.md).
--- /dev/null
+# Installing on Linux
+
+## Prerequisites
+
+GStreamer is included in all Linux distributions. We recommend using the latest version of a fast moving distribution such as Fedora, Ubuntu (non-LTS), Debian sid or OpenSuse to get a recent GStreamer release.
+
+All the commands given in this section are intended to be typed in from
+a terminal.
+
+> 
+Make sure you have superuser (root) access rights to install the GStreamer SDK.
+
+## Install GStreamer on Fedora
+
+Run the following command:
+
+```
+dnf install gstreamer1-devel gstreamer1-plugins-base-tools gstreamer1-devel-docs gstreamer1-plugins-base-devel gstreamer1-plugins-base-devel-docs gstreamer1-plugins-good gstreamer1-plugins-good-extras gstreamer1-plugins-ugly gstreamer1-plugins-ugly-devel-docs gstreamer1-plugins-bad-free gstreamer1-plugins-bad-free-devel gstreamer1-plugins-bad-free-extras
+```
+
+## Install GStreamre on Ubuntu or Debian
+
+Run the following command:
+
+` apt-get install .... `
+
+## Building applications using GStreamer
+
+The only other “development environment” that is required is
+the `gcc` compiler and a text editor. In order to compile code that
+requires the GStreamer SDK and uses the GStreamer core library, remember
+to add this string to your `gcc` command:
+
+```
+pkg-config --cflags --libs gstreamer-1.0
+```
+
+If you're using other GStreamer libraries, e.g. the video library, you
+have to add additional packages after gstreamer-1.0 in the above string
+(gstreamer-video-1.0 for the video library, for example).
+
+If your application is built with the help of libtool, e.g. when using
+automake/autoconf as a build system, you have to run
+the `configure` script from inside the `gst-sdk-shell` environment.
+
+#### Getting the tutorial's source code
+
+The source code for the tutorials can be copied and pasted from the
+tutorial pages into a text file, but, for convenience, it is also
+available in a GIT repository and distributed with the SDK.
+
+The GIT repository can be cloned with:
+
+```
+git clone git://anongit.freedesktop.org/gstreamer/gst-sdk-tutorials
+```
+
+#### Building the tutorials
+
+```
+gcc basic-tutorial-1.c -o basic-tutorial-1 `pkg-config --cflags --libs gstreamer-1.0`
+```
+
+Using the file name of the tutorial you are interested in
+(`basic-tutorial-1` in this example).
+
+>  Depending on the GStreamer libraries you need to use, you will have to add more packages to the `pkg-config` command, besides `gstreamer-1.0`
+> At the bottom of each tutorial's source code you will find the command for that specific tutorial, including the required libraries, in the required order.
+> When developing your own applications, the GStreamer documentation will tell you what library a function belongs to.
+
+#### Running the tutorials
+
+To run the tutorials, simply execute the desired tutorial:
+
+``` c
+./basic-tutorial-1
+```
--- /dev/null
+# Installing on Mac OS X
+
+## Supported platforms
+
+ * 10.6 (Snow Leopard)
+ * 10.7 (Lion)
+ * 10.8 (Mountain Lion)
+ * 10.9 (Mavericks)
+ * 10.10 (Yosemite)
+ * 10.11 (El Capitan)
+
+## Prerequisites
+
+**FIXME: Those all need updating**
+
+To develop applications using the GStreamer SDK for OS X you will need
+OS X Snow Leopard (10.6) or later and
+[XCode 3.2.6](https://developer.apple.com/devcenter/mac/index.action) or
+later.
+
+The recommended system is [Mac OS X
+El Capitan](http://www.apple.com/macosx/) with
+[XCode 4](https://developer.apple.com/xcode/)
+
+## Download and install the SDK
+
+There are 3 sets of files in the SDK:
+
+ - The runtime files are needed to run GStreamer applications. You
+ probably want to distribute these files with your application (or
+ the installer below).
+ - The development files are **additional** files you need to create
+ GStreamer applications.
+ - Mac OS X packages that you can use
+ with [PackageMaker](https://developer.apple.com/library/mac/#documentation/DeveloperTools/Conceptual/PackageMakerUserGuide/Introduction/Introduction.html)
+ to deploy GStreamer with your application
+
+Get **both the runtime and the development installers** from [here](https://gstreamer.freedesktop.org/data/pkg/osx/).
+
+
+> 
+> On Mac OS X 10.6 (Snow Leopard) you have to install Python 2.7 manually. It is included in later versions of OS X already. You can get it from [here](http://www.python.org/getit).
+
+The downloads are
+[Installer Packages
+(.pkg)](http://en.wikipedia.org/wiki/Installer_%28Mac_OS_X%29).
+
+Double click the package file and follow the instructions presented by
+the install wizard. In case the system complains about the package not
+being signed, you can control-click it and open to start the
+installation. When you do this, it will warn you, but there is an option
+to install anyway. Otherwise you can go to System Preferences → Security
+and Privacy → General and select the option to allow installation of
+packages from "anywhere".
+
+
+These are some paths of the GStreamer framework that you might find
+useful:
+
+ - /Library/Frameworks/GStreamer.framework/: Framework's root path
+ - /Library/Frameworks/GStreamer.framework/Versions: path with all the
+ versions of the framework
+ - /Library/Frameworks/GStreamer.framework/Versions/Current: link to
+ the current version of the framework
+ - /Library/Frameworks/GStreamer.framework/Headers: path with the
+ development headers
+ - /Library/Frameworks/GStreamer.framework/Commands: link to the
+ commands provided by the framework, such as gst-inspect-1.0 or
+ gst-launch-1.0
+
+For more information on OS X Frameworks anatomy, you can consult the
+following [link](https://developer.apple.com/library/mac/#documentation/MacOSX/Conceptual/BPFrameworks/Concepts/FrameworkAnatomy.html)
+
+## Configure your development environment
+
+### Building the tutorials
+
+The tutorial's code, along with project files and a solution file for
+them all are included in the SDK. The source code and the XCode project
+files are located
+in `/Library/Frameworks/GStreamer.framework/Current/share/gst-sdk/tutorials.`
+
+To start building the tutorials, create a new folder in your Documents
+directory and copy the
+folder `/Library/Frameworks/GStreamer.framework/Current/share/gst-sdk/tutorials`.
+
+You can fire up XCode and load the project file.
+
+Press the **Run** button to build and run the first tutorial. You can
+switch the tutorial to build selecting one of the available schemes.
+
+### Creating new projects
+
+The GStreamer binaries provides a
+[framework](https://developer.apple.com/library/mac/#documentation/MacOSX/Conceptual/BPFrameworks/Tasks/IncludingFrameworks.html)
+that you can drag and drop to XCode to start using it, or using the
+linker option ***-framework GStreamer****.*
+
+There is a small exception to the regular use of frameworks, and you
+will need to manually include the headers search
+path `/Library/Frameworks/GStreamer.framework/Headers`
+
+ - XCode: Add the headers path to **Search Paths → Header Search
+ Paths**
+ - GCC: Using the compiler
+ option **-I/Library/Frameworks/GStreamer.framework/Headers**
--- /dev/null
+# Installing on Windows
+
+## Supported platforms
+
+* Windows XP
+* Windows Vista
+* Windows 7
+* Windows 8
+
+## Prerequisites
+
+To develop applications using GStreamer for Windows you will
+need [Windows
+XP](http://windows.microsoft.com/en-US/windows/products/windows-xp) or
+later.
+
+GStreamer binaries includes C headers (`.h`) and library files (`.lib`)
+valid for any version of [Microsoft Visual
+Studio](http://www.microsoft.com/visualstudio). For convenience,
+property pages (`.props`) are also included which extremely simplify
+creating new projects. These property pages, though, only work with
+[Microsoft Visual
+Studio 2010](http://www.microsoft.com/visualstudio/en-us/products/2010-editions)
+(including the free [Visual C++ Express
+edition](http://www.microsoft.com/visualstudio/en-us/products/2010-editions/visual-cpp-express)).
+
+The recommended system is
+[Windows 7](http://windows.microsoft.com/en-us/windows7/products/home)
+with [Microsoft Visual
+Studio 2010](http://www.microsoft.com/visualstudio/en-us/products/2010-editions) (Take
+a look at its [system
+requirements](http://www.microsoft.com/visualstudio/en-us/products/2010-editions/visual-cpp-express)).
+
+Installing GStreamer for 32-bits platforms requires approximately 286MB of
+free disk space for the runtime and 207MB for the development files.
+
+Installing GStreamer for 64-bits platforms requires approximately 340MB of
+free disk space for the runtime and 216MB for the development files.
+
+## Download and install GStreamer binaries
+
+There are 3 sets of files in GStreamer binaries:
+
+ - The runtime files are needed to run GStreamer applications. You
+ probably want to distribute these files with your application (or
+ the installer below).
+ - The development files are **additional** files you need to create
+ GStreamer applications.
+ - The [Merge
+ Modules](http://msdn.microsoft.com/en-us/library/windows/desktop/aa369820%28v=vs.85%29.aspx)
+ files are **additional** files you can use to deploy GStreamer binaries
+ alongside your application (see [Windows
+ deployment](sdk-windows-deployment.md)).
+
+Get **the Runtime and Development files** installers appropriate for
+your architecture from here:
+
+**FIXME: Add links **
+
+Execute the installers and choose an installation folder. The suggested
+default is usually OK.
+
+> 
+>`If you plan to use Visual Studio, **close it before installing GStreamer**. The installer will define new environment variables which will not be picked up by Visual Studio if it is open.
+
+> On **Windows 8** and **Windows 10**, it might be necessary to log out and log back in to your account after the installation for the newly defined environment variables to be picked up by Visual Studio.
+
+It is the application's responsibility to ensure that, at runtime,
+GStreamer can access its libraries and plugins. It can be done by adding
+`%GSTREAMER_ROOT_X86%\bin` to the `%PATH%` environment variable, or
+by running the application from this same folder.
+
+At runtime, GStreamer will look for its plugins in the following
+folders:
+
+ - `%HOMEDRIVE%%HOMEFOLDER%/.gstreamer-1.0/plugins`
+ - `C:\gstreamer\1.0\x86\lib\gstreamer-1.0`
+ - `<location of libgstreamer-1.0-0.dll>\..\lib\gstreamer-1.0`
+ - `%GST_PLUGIN_PATH%`
+
+So, typically, if your application can find `libgstreamer-1.0-0.dll`,
+it will find the GStreamer plugins, as long as the installation folder
+structure is unmodified. If you do change this structure in your
+application, then you can use the `%GST_PLUGIN_PATH%` environment
+variable to point GStreamer to its plugins. The plugins are initially
+found at `%GSTREAMER_ROOT_X86%\lib\gstreamer-1.0`.
+
+Additionally, if you want to prevent GStreamer from looking in all the
+default folders listed above, you can set the
+`%GST_PLUGIN_SYSTEM_PATH%` environment variable to point where the
+plugins are located.
+
+## Configure your development environment
+
+### Building the tutorials
+
+The tutorial's code, along with project files and a solution file for
+Visual Studio 2010 are all included in the GStreamer binaries, in
+the `%GSTREAMER_ROOT_X86%``\share\gst-sdk\tutorials` folder.
+
+`%GSTREAMER_ROOT_X86%` is an environment variable that the installer
+defined for you, and points to the installation folder of GStreamer binaries.
+
+In order to prevent accidental modification of the original code, and to
+make sure Visual Studio has the necessary permissions to write the
+output files, copy the entire `tutorials` folder to a place of your
+liking, and work from there.
+
+> 
+> **64-bit Users**
+>
+>Use `%GSTREAMER_ROOT_X86_64%` if you have installed GStreamer binaries for 64-bit platforms. Both GStreamer binariess (32 and 64-bit) can be installed simultaneously, and hence the separate environment variables.
+>
+>Make sure you select the Solution Configuration that matches GStreamer that you have installed: `Win32` for 32 bits or `x64` for 64 bits.
+>
+> 
+
+You can fire up Visual Studio 2010 and load your copy of the
+`tutorials.sln` solution file (Click on the screen shots to enlarge
+them).
+
+
+
+
+
+Hit **F7**, press the Build Solution button
+ or go to Build →
+Build Solution. All projects should build without problems.
+
+### Running the tutorials
+
+In order to run the tutorials, we will set the current working directory
+to `%GSTREAMER_ROOT_X86%`\\`bin` in the Debugging section of the
+project properties. **This property is not stored in the project files,
+so you will need to manually add it to every tutorial you want to run
+from within Visual Studio**. Right click on a project in the Solution
+Explorer, Properties → Debugging → Working Directory, and type
+`$(GSTREAMER_ROOT_X86)`\\`bin`
+
+(The `$(...)` notation is required to access environment variables
+from within Visual Studio. You use the `%...%` notation from Windows
+Explorer)
+
+You should now be able to run the tutorials.
+
+### Creating new projects manually
+
+**If you want to create 64-bit applications, remember also to create x64
+Solution and Project configurations as
+explained [here](http://msdn.microsoft.com/en-us/library/9yb4317s\(v=vs.100\).aspx).**
+
+#### Include the necessary GStreamer Property Sheet
+
+The included property sheets make creating new projects extremely easy.
+In Visual Studio 2010 create a new project (Normally a `Win32
+Console` or `Win32 Application`). Then go to the Property Manager
+(View→Property Manager), right-click on your project and select “Add
+Existing Property Sheet...”. Navigate to
+`%GSTREAMER_ROOT_X86%`\\`share\vs\2010\libs` and
+load `gstreamer-1.0.props `
+
+This property sheet contains the directories where the headers and
+libraries are located, and the necessary options for the compiler and
+linker, so you do not need to change anything else in your project.
+
+If you cannot find the Property Manager, you might need to enable Expert
+Settings. Go to Tools → Settings → Expert Settings. Upon first
+installation of Visual Studio, Expert Settings are disabled by
+default.
+
+
+
+> 
+> **Depending on the GStreamer libraries you need to use, you will have to add more property pages, besides `gstreamer-1.0`** (each property page corresponds to one GStreamer library).
+>
+> The tutorial's project files already contain all necessary property pages. When developing your own applications, the GStreamer documentation will tell you what library a function belongs to, and therefore, what property pages you need to add.
+
+#### Remove the dependency with the Visual Studio runtime
+
+At this point, you have a working environment, which you can test by
+running the tutorials. However, there is a last step remaining.
+
+Applications built with Visual C++ 2010 depend on the Visual C++ 2010
+Runtime, which is a DLL that gets installed when you install Visual
+Studio. If you were to distribute your application, you would need to
+distribute this DLL with it (What is known as the [Visual C++ 2010
+Redistributable
+Package](http://www.microsoft.com/download/en/details.aspx?id=5555)).
+This happens with every version of Visual Studio, and the Runtime DLL is
+different for every version of Visual Studio.
+
+Furthermore, GStreamer itself is built using a “basic” C runtime which
+comes in every Windows system since Windows XP, and is named
+`MSVCRT.DLL`. If your application and GStreamer do not use the same C
+Runtime, problems are bound to crop out.
+
+In order to avoid these issues you must instruct your application to use
+the system's C Runtime. First install the [Windows Device Driver Kit
+Version 7.1.0](http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=11800) (DDK).
+When the installer asks about the features, select only “Build
+Environments”. Accept the suggested location for the installation, which
+is usually `C:\WinDDK\7600.16385.1`. This download is an ISO file, you
+can either burn a DVD with it (as recommended in the Microsoft site. You
+will need DVD burning software), mount the file in a virtual DVD device
+(you will need DVD virtualization software) or unpack the file as if it
+was a regular compressed file (you will need decompression software that
+understands the ISO format).
+
+Then, add the `x86.props` or `x86_64.props` (for 32 or 64 bits) property
+sheet found in `%GSTREAMER_ROOT_X86%``\``share\vs\2010\msvc` to your
+project. This will make your application use the ubiquitous
+`MSVCRT.DLL` saving you some troubles in the future.
+
+> 
+> If you did not install the WinDDK to the standard path `C:\WinDDK\7600.16385.1`, you will need to tell Visual Studio where it is. Unfortunately, there is no automated way to do this. Once you have added the `x86.props` or `x86_64.props` to your project, go to the Property Manager, expand your project and its subfolders until you find the property sheet called `config`. Double click to edit it, and select the section called “User Macros” in the list on the left. You should see a macro called `WINDOWS_DRIVER_KIT`. Double click to edit it, and set its value to the root folder where you installed the DDK. This is the folder containing a file called `samples.txt`.
+>
+>That's it. Accept the changes, right click on the `config` property sheet and select “Save”. The path to the DDK is now stored in `config.props` and you do not need to perform this operation anymore.
+
+### Creating new projects using the wizard
+
+Go to File → New → Project… and you should find a template
+named **GStreamer Project**. It takes no parameters, and sets all
+necessary project settings, both for 32 and 64 bits architectures.
+
+The generated project file includes the two required Property Sheets
+described in the previous section, so, in order to link to the correct
+`MSVCRT.DLL`, **you still need to install the Windows Device Driver
+Kit** and change the appropriate property sheets.
--- /dev/null
+---
+short-description: Download and install GStreamer
+...
+
+# Installing GStreamer
+
+## Choose your platform by clicking on the corresponding logo
+
+[](sdk-installing-on-mac-osx.md)
+[](sdk-installing-on-windows.md)
+[](sdk-installing-for-android-development.md)
+[](sdk-installing-for-ios-development.md)
+
+## Linux
+
+The GStreamer community does not provide binaries for GNU/Linux
+platforms as it will always be available through package managers on
+all distribution. It is also always installed by default on desktop
+environments, you will just need to make sure you have the development
+packages installed (refer to your distribution documentation for more
+information). If you really want to run upstream style binaries on
+Linux, you can always follow the instructions to [build from source
+using cerbero](sdk-building-from-source-using-cerbero.md).
--- /dev/null
+# iOS tutorial 4: A basic media player
+
+## Goal
+
+![screenshot]
+
+Enough testing with synthetic images and audio tones! This tutorial
+finally plays actual media, streamed directly from the Internet, in your
+iOS device. It shows:
+
+ - How to keep the User Interface regularly updated with the current
+ playback position and duration
+ - How to implement a [Time
+ Slider](http://developer.apple.com/library/ios/#documentation/UIKit/Reference/UISlider_Class/Reference/Reference.html)
+ - How to report the media size to adapt the display surface
+
+It also uses the knowledge gathered in the [](sdk-basic-tutorials.md) regarding:
+
+ - How to use `playbin` to play any kind of media
+ - How to handle network resilience problems
+
+## Introduction
+
+From the previous tutorials, we already have almost all necessary
+pieces to build a media player. The most complex part is assembling a
+pipeline which retrieves, decodes and displays the media, but we
+already know that the `playbin` element can take care of all that for
+us. We only need to replace the manual pipeline we used in
+[](sdk-ios-tutorial-video.md) with a single-element `playbin` pipeline
+and we are good to go!
+
+However, we can do better than. We will add a [Time
+Slider](http://developer.apple.com/library/ios/#documentation/UIKit/Reference/UISlider_Class/Reference/Reference.html),
+with a moving thumb that will advance as our current position in the
+media advances. We will also allow the user to drag the thumb, to jump
+(or *seek*) to a different position.
+
+And finally, we will make the video surface adapt to the media size, so
+the video sink is not forced to draw black borders around the clip.
+ This also allows the iOS layout to adapt more nicely to the actual
+media content. You can still force the video surface to have a specific
+size if you really want to.
+
+## The User Interface
+
+The User Interface from the previous tutorial is expanded again. A
+`UISlider` has been added to the toolbar, to keep track of the current
+position in the clip, and allow the user to change it. Also, a
+(read-only) `UITextField` is added to show the exact clip position and
+duration.
+
+**VideoViewController.h**
+
+```
+#import <UIKit/UIKit.h>
+#import "GStreamerBackendDelegate.h"
+
+@interface VideoViewController : UIViewController <GStreamerBackendDelegate> {
+ IBOutlet UILabel *message_label;
+ IBOutlet UIBarButtonItem *play_button;
+ IBOutlet UIBarButtonItem *pause_button;
+ IBOutlet UIView *video_view;
+ IBOutlet UIView *video_container_view;
+ IBOutlet NSLayoutConstraint *video_width_constraint;
+ IBOutlet NSLayoutConstraint *video_height_constraint;
+ IBOutlet UIToolbar *toolbar;
+ IBOutlet UITextField *time_label;
+ IBOutlet UISlider *time_slider;
+}
+
+@property (retain,nonatomic) NSString *uri;
+
+-(IBAction) play:(id)sender;
+-(IBAction) pause:(id)sender;
+-(IBAction) sliderValueChanged:(id)sender;
+-(IBAction) sliderTouchDown:(id)sender;
+-(IBAction) sliderTouchUp:(id)sender;
+
+/* From GStreamerBackendDelegate */
+-(void) gstreamerInitialized;
+-(void) gstreamerSetUIMessage:(NSString *)message;
+
+@end
+```
+
+Note how we register callbacks for some of the Actions the
+[UISlider](http://developer.apple.com/library/ios/#documentation/UIKit/Reference/UISlider_Class/Reference/Reference.html) generates.
+Also note that the class has been renamed from `ViewController` to
+`VideoViewController`, since the next tutorial adds another
+`ViewController` and we will need to differentiate.
+
+## The Video View Controller
+
+The `ViewController `class manages the UI, instantiates
+the `GStreamerBackend` and also performs some UI-related tasks on its
+behalf:
+
+Due to the extension of this code,
+this view is collapsed by default. Click here to expand…
+
+**VideoViewController.m**
+
+```
+#import "VideoViewController.h"
+#import "GStreamerBackend.h"
+#import <UIKit/UIKit.h>
+
+@interface VideoViewController () {
+ GStreamerBackend *gst_backend;
+ int media_width; /* Width of the clip */
+ int media_height; /* height of the clip */
+ Boolean dragging_slider; /* Whether the time slider is being dragged or not */
+ Boolean is_local_media; /* Whether this clip is stored locally or is being streamed */
+ Boolean is_playing_desired; /* Whether the user asked to go to PLAYING */
+}
+
+@end
+
+@implementation VideoViewController
+
+@synthesize uri;
+
+/*
+ * Private methods
+ */
+
+/* The text widget acts as an slave for the seek bar, so it reflects what the seek bar shows, whether
+ * it is an actual pipeline position or the position the user is currently dragging to. */
+- (void) updateTimeWidget
+{
+ NSInteger position = time_slider.value / 1000;
+ NSInteger duration = time_slider.maximumValue / 1000;
+ NSString *position_txt = @" -- ";
+ NSString *duration_txt = @" -- ";
+
+ if (duration > 0) {
+ NSUInteger hours = duration / (60 * 60);
+ NSUInteger minutes = (duration / 60) % 60;
+ NSUInteger seconds = duration % 60;
+
+ duration_txt = [NSString stringWithFormat:@"%02u:%02u:%02u", hours, minutes, seconds];
+ }
+ if (position > 0) {
+ NSUInteger hours = position / (60 * 60);
+ NSUInteger minutes = (position / 60) % 60;
+ NSUInteger seconds = position % 60;
+
+ position_txt = [NSString stringWithFormat:@"%02u:%02u:%02u", hours, minutes, seconds];
+ }
+
+ NSString *text = [NSString stringWithFormat:@"%@ / %@",
+ position_txt, duration_txt];
+
+ time_label.text = text;
+}
+
+/*
+ * Methods from UIViewController
+ */
+
+- (void)viewDidLoad
+{
+ [super viewDidLoad];
+
+ play_button.enabled = FALSE;
+ pause_button.enabled = FALSE;
+
+ /* As soon as the GStreamer backend knows the real values, these ones will be replaced */
+ media_width = 320;
+ media_height = 240;
+
+ uri = @"https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-368p.ogv";
+
+ gst_backend = [[GStreamerBackend alloc] init:self videoView:video_view];
+}
+
+- (void)viewDidDisappear:(BOOL)animated
+{
+ if (gst_backend)
+ {
+ [gst_backend deinit];
+ }
+}
+
+- (void)didReceiveMemoryWarning
+{
+ [super didReceiveMemoryWarning];
+ // Dispose of any resources that can be recreated.
+}
+
+/* Called when the Play button is pressed */
+-(IBAction) play:(id)sender
+{
+ [gst_backend play];
+ is_playing_desired = YES;
+}
+
+/* Called when the Pause button is pressed */
+-(IBAction) pause:(id)sender
+{
+ [gst_backend pause];
+ is_playing_desired = NO;
+}
+
+/* Called when the time slider position has changed, either because the user dragged it or
+ * we programmatically changed its position. dragging_slider tells us which one happened */
+- (IBAction)sliderValueChanged:(id)sender {
+ if (!dragging_slider) return;
+ // If this is a local file, allow scrub seeking, this is, seek as soon as the slider is moved.
+ if (is_local_media)
+ [gst_backend setPosition:time_slider.value];
+ [self updateTimeWidget];
+}
+
+/* Called when the user starts to drag the time slider */
+- (IBAction)sliderTouchDown:(id)sender {
+ [gst_backend pause];
+ dragging_slider = YES;
+}
+
+/* Called when the user stops dragging the time slider */
+- (IBAction)sliderTouchUp:(id)sender {
+ dragging_slider = NO;
+ // If this is a remote file, scrub seeking is probably not going to work smoothly enough.
+ // Therefore, perform only the seek when the slider is released.
+ if (!is_local_media)
+ [gst_backend setPosition:time_slider.value];
+ if (is_playing_desired)
+ [gst_backend play];
+}
+
+/* Called when the size of the main view has changed, so we can
+ * resize the sub-views in ways not allowed by storyboarding. */
+- (void)viewDidLayoutSubviews
+{
+ CGFloat view_width = video_container_view.bounds.size.width;
+ CGFloat view_height = video_container_view.bounds.size.height;
+
+ CGFloat correct_height = view_width * media_height / media_width;
+ CGFloat correct_width = view_height * media_width / media_height;
+
+ if (correct_height < view_height) {
+ video_height_constraint.constant = correct_height;
+ video_width_constraint.constant = view_width;
+ } else {
+ video_width_constraint.constant = correct_width;
+ video_height_constraint.constant = view_height;
+ }
+
+ time_slider.frame = CGRectMake(time_slider.frame.origin.x, time_slider.frame.origin.y, toolbar.frame.size.width - time_slider.frame.origin.x - 8, time_slider.frame.size.height);
+}
+
+/*
+ * Methods from GstreamerBackendDelegate
+ */
+
+-(void) gstreamerInitialized
+{
+ dispatch_async(dispatch_get_main_queue(), ^{
+ play_button.enabled = TRUE;
+ pause_button.enabled = TRUE;
+ message_label.text = @"Ready";
+ [gst_backend setUri:uri];
+ is_local_media = [uri hasPrefix:@"file://"];
+ is_playing_desired = NO;
+ });
+}
+
+-(void) gstreamerSetUIMessage:(NSString *)message
+{
+ dispatch_async(dispatch_get_main_queue(), ^{
+ message_label.text = message;
+ });
+}
+
+-(void) mediaSizeChanged:(NSInteger)width height:(NSInteger)height
+{
+ media_width = width;
+ media_height = height;
+ dispatch_async(dispatch_get_main_queue(), ^{
+ [self viewDidLayoutSubviews];
+ [video_view setNeedsLayout];
+ [video_view layoutIfNeeded];
+ });
+}
+
+-(void) setCurrentPosition:(NSInteger)position duration:(NSInteger)duration
+{
+ /* Ignore messages from the pipeline if the time sliders is being dragged */
+ if (dragging_slider) return;
+
+ dispatch_async(dispatch_get_main_queue(), ^{
+ time_slider.maximumValue = duration;
+ time_slider.value = position;
+ [self updateTimeWidget];
+ });
+}
+
+@end
+```
+
+Supporting arbitrary media URIs
+
+The `GStreamerBackend` provides the `setUri()` method so we can
+indicate the URI of the media to play. Since `playbin` will be taking
+care of retrieving the media, we can use local or remote URIs
+indistinctly (`file://` or `http://`, for example). From the UI code,
+though, we want to keep track of whether the file is local or remote,
+because we will not offer the same functionalities. We keep track of
+this in the `is_local_media` variable, which is set when the URI is set,
+in the `gstreamerInitialized` method:
+
+```
+-(void) gstreamerInitialized
+{
+ dispatch_async(dispatch_get_main_queue(), ^{
+ play_button.enabled = TRUE;
+ pause_button.enabled = TRUE;
+ message_label.text = @"Ready";
+ [gst_backend setUri:uri];
+ is_local_media = [uri hasPrefix:@"file://"];
+ is_playing_desired = NO;
+ });
+}
+```
+
+Reporting media size
+
+Every time the size of the media changes (which could happen mid-stream,
+for some kind of streams), or when it is first detected,
+`GStreamerBackend` calls our `mediaSizeChanged()` callback:
+
+```
+-(void) mediaSizeChanged:(NSInteger)width height:(NSInteger)height
+{
+ media_width = width;
+ media_height = height;
+ dispatch_async(dispatch_get_main_queue(), ^{
+ [self viewDidLayoutSubviews];
+ [video_view setNeedsLayout];
+ [video_view layoutIfNeeded];
+ });
+}
+```
+
+Here we simply store the new size and ask the layout to be recalculated.
+As we have already seen in [](sdk-ios-tutorial-a-running-pipeline.md),
+methods which change the UI must be called from the main thread, and we
+are now in a callback from some GStreamer internal thread. Hence, the
+usage
+of `dispatch_async()`[.](http://developer.android.com/reference/android/app/Activity.html#runOnUiThread\(java.lang.Runnable\))
+
+### Refreshing the Time Slider
+
+[](sdk-basic-tutorial-toolkit-integration.md) has
+already shown how to implement a Seek Bar (or [Time
+Slider](http://developer.apple.com/library/ios/#documentation/UIKit/Reference/UISlider_Class/Reference/Reference.html)
+in this tutorial) using the GTK+ toolkit. The implementation on iOS is
+very similar.
+
+The Seek Bar accomplishes to functions: First, it moves on its own to
+reflect the current playback position in the media. Second, it can be
+dragged by the user to seek to a different position.
+
+To realize the first function, `GStreamerBackend` will periodically
+call our `setCurrentPosition` method so we can update the position of
+the thumb in the Seek Bar. Again we do so from the UI thread, using
+`dispatch_async()`.
+
+```
+-(void) setCurrentPosition:(NSInteger)position duration:(NSInteger)duration
+{
+ /* Ignore messages from the pipeline if the time sliders is being dragged */
+ if (dragging_slider) return;
+
+ dispatch_async(dispatch_get_main_queue(), ^{
+ time_slider.maximumValue = duration;
+ time_slider.value = position;
+ [self updateTimeWidget];
+ });
+}
+```
+
+Also note that if the user is currently dragging the slider (the
+`dragging_slider` variable is explained below) we ignore
+`setCurrentPosition` calls from `GStreamerBackend`, as they would
+interfere with the user’s actions.
+
+To the left of the Seek Bar (refer to the screenshot at the top of this
+page), there is
+a [TextField](https://developer.apple.com/library/ios/#documentation/UIKit/Reference/UITextField_Class/Reference/UITextField.html) widget
+which we will use to display the current position and duration in
+"`HH:mm:ss / HH:mm:ss"` textual format. The `updateTimeWidget` method
+takes care of it, and must be called every time the Seek Bar is
+updated:
+
+```
+/* The text widget acts as an slave for the seek bar, so it reflects what the seek bar shows, whether
+ * it is an actual pipeline position or the position the user is currently dragging to. */
+- (void) updateTimeWidget
+{
+ NSInteger position = time_slider.value / 1000;
+ NSInteger duration = time_slider.maximumValue / 1000;
+ NSString *position_txt = @" -- ";
+ NSString *duration_txt = @" -- ";
+
+ if (duration > 0) {
+ NSUInteger hours = duration / (60 * 60);
+ NSUInteger minutes = (duration / 60) % 60;
+ NSUInteger seconds = duration % 60;
+
+ duration_txt = [NSString stringWithFormat:@"%02u:%02u:%02u", hours, minutes, seconds];
+ }
+ if (position > 0) {
+ NSUInteger hours = position / (60 * 60);
+ NSUInteger minutes = (position / 60) % 60;
+ NSUInteger seconds = position % 60;
+
+ position_txt = [NSString stringWithFormat:@"%02u:%02u:%02u", hours, minutes, seconds];
+ }
+
+ NSString *text = [NSString stringWithFormat:@"%@ / %@",
+ position_txt, duration_txt];
+
+ time_label.text = text;
+}
+```
+
+Seeking with the Seek Bar
+
+To perform the second function of the Seek Bar (allowing the user to
+seek by dragging the thumb), we register some callbacks through IBAction
+outlets. Refer to the storyboard in this tutorial’s project to see which
+outlets are connected. We will be notified when the user starts dragging
+the Slider, when the Slider position changes and when the users releases
+the Slider.
+
+```
+/* Called when the user starts to drag the time slider */
+- (IBAction)sliderTouchDown:(id)sender {
+ [gst_backend pause];
+ dragging_slider = YES;
+}
+```
+
+`sliderTouchDown` is called when the user starts dragging. Here we pause
+the pipeline because if the user is searching for a particular scene, we
+do not want it to keep moving. We also mark that a drag operation is in
+progress in the
+`dragging_slider` variable.
+
+```
+/* Called when the time slider position has changed, either because the user dragged it or
+ * we programmatically changed its position. dragging_slider tells us which one happened */
+- (IBAction)sliderValueChanged:(id)sender {
+ if (!dragging_slider) return;
+ // If this is a local file, allow scrub seeking, this is, seek as soon as the slider is moved.
+ if (is_local_media)
+ [gst_backend setPosition:time_slider.value];
+ [self updateTimeWidget];
+}
+```
+
+`sliderValueChanged` is called every time the Slider’s thumb moves, be
+it because the user dragged it, or because we changed its value form the
+program. We discard the latter case using the
+`dragging_slider` variable.
+
+As the comment says, if this is a local media, we allow scrub seeking,
+this is, we jump to the indicated position as soon as the thumb moves.
+Otherwise, the seek operation will be performed when the thumb is
+released, and the only thing we do here is update the textual time
+widget.
+
+```
+/* Called when the user stops dragging the time slider */
+- (IBAction)sliderTouchUp:(id)sender {
+ dragging_slider = NO;
+ // If this is a remote file, scrub seeking is probably not going to work smoothly enough.
+ // Therefore, perform only the seek when the slider is released.
+ if (!is_local_media)
+ [gst_backend setPosition:time_slider.value];
+ if (is_playing_desired)
+ [gst_backend play];
+}
+```
+
+Finally, `sliderTouchUp` is called when the thumb is released. We
+perform the seek operation if the file was non-local, restore the
+pipeline to the desired playing state and end the dragging operation by
+setting `dragging_slider` to NO.
+
+This concludes the User interface part of this tutorial. Let’s review
+now the `GStreamerBackend` class that allows this to work.
+
+## The GStreamer Backend
+
+The `GStreamerBackend` class performs all GStreamer-related tasks and
+offers a simplified interface to the application, which does not need to
+deal with all the GStreamer details. When it needs to perform any UI
+action, it does so through a delegate, which is expected to adhere to
+the `GStreamerBackendDelegate` protocol.
+
+**GStreamerBackend.m**
+
+```
+#import "GStreamerBackend.h"
+
+#include <gst/gst.h>
+#include <gst/video/video.h>
+
+GST_DEBUG_CATEGORY_STATIC (debug_category);
+#define GST_CAT_DEFAULT debug_category
+
+/* Do not allow seeks to be performed closer than this distance. It is visually useless, and will probably
+ * confuse some demuxers. */
+#define SEEK_MIN_DELAY (500 * GST_MSECOND)
+
+@interface GStreamerBackend()
+-(void)setUIMessage:(gchar*) message;
+-(void)app_function;
+-(void)check_initialization_complete;
+@end
+
+@implementation GStreamerBackend {
+ id ui_delegate; /* Class that we use to interact with the user interface */
+ GstElement *pipeline; /* The running pipeline */
+ GstElement *video_sink; /* The video sink element which receives VideoOverlay commands */
+ GMainContext *context; /* GLib context used to run the main loop */
+ GMainLoop *main_loop; /* GLib main loop */
+ gboolean initialized; /* To avoid informing the UI multiple times about the initialization */
+ UIView *ui_video_view; /* UIView that holds the video */
+ GstState state; /* Current pipeline state */
+ GstState target_state; /* Desired pipeline state, to be set once buffering is complete */
+ gint64 duration; /* Cached clip duration */
+ gint64 desired_position; /* Position to seek to, once the pipeline is running */
+ GstClockTime last_seek_time; /* For seeking overflow prevention (throttling) */
+ gboolean is_live; /* Live streams do not use buffering */
+}
+
+/*
+ * Interface methods
+ */
+
+-(id) init:(id) uiDelegate videoView:(UIView *)video_view
+{
+ if (self = [super init])
+ {
+ self->ui_delegate = uiDelegate;
+ self->ui_video_view = video_view;
+ self->duration = GST_CLOCK_TIME_NONE;
+
+ GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-4", 0, "iOS tutorial 4");
+ gst_debug_set_threshold_for_name("tutorial-4", GST_LEVEL_DEBUG);
+
+ /* Start the bus monitoring task */
+ dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
+ [self app_function];
+ });
+ }
+
+ return self;
+}
+
+-(void) deinit
+{
+ if (main_loop) {
+ g_main_loop_quit(main_loop);
+ }
+}
+
+-(void) play
+{
+ target_state = GST_STATE_PLAYING;
+ is_live = (gst_element_set_state (pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_NO_PREROLL);
+}
+
+-(void) pause
+{
+ target_state = GST_STATE_PAUSED;
+ is_live = (gst_element_set_state (pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_NO_PREROLL);
+}
+
+-(void) setUri:(NSString*)uri
+{
+ const char *char_uri = [uri UTF8String];
+ g_object_set(pipeline, "uri", char_uri, NULL);
+ GST_DEBUG ("URI set to %s", char_uri);
+}
+
+-(void) setPosition:(NSInteger)milliseconds
+{
+ gint64 position = (gint64)(milliseconds * GST_MSECOND);
+ if (state >= GST_STATE_PAUSED) {
+ execute_seek(position, self);
+ } else {
+ GST_DEBUG ("Scheduling seek to %" GST_TIME_FORMAT " for later", GST_TIME_ARGS (position));
+ self->desired_position = position;
+ }
+}
+
+/*
+ * Private methods
+ */
+
+/* Change the message on the UI through the UI delegate */
+-(void)setUIMessage:(gchar*) message
+{
+ NSString *string = [NSString stringWithUTF8String:message];
+ if(ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerSetUIMessage:)])
+ {
+ [ui_delegate gstreamerSetUIMessage:string];
+ }
+}
+
+/* Tell the application what is the current position and clip duration */
+-(void) setCurrentUIPosition:(gint)pos duration:(gint)dur
+{
+ if(ui_delegate && [ui_delegate respondsToSelector:@selector(setCurrentPosition:duration:)])
+ {
+ [ui_delegate setCurrentPosition:pos duration:dur];
+ }
+}
+
+/* If we have pipeline and it is running, query the current position and clip duration and inform
+ * the application */
+static gboolean refresh_ui (GStreamerBackend *self) {
+ gint64 position;
+
+ /* We do not want to update anything unless we have a working pipeline in the PAUSED or PLAYING state */
+ if (!self || !self->pipeline || self->state < GST_STATE_PAUSED)
+ return TRUE;
+
+ /* If we didn't know it yet, query the stream duration */
+ if (!GST_CLOCK_TIME_IS_VALID (self->duration)) {
+ gst_element_query_duration (self->pipeline, GST_FORMAT_TIME, &self->duration);
+ }
+
+ if (gst_element_query_position (self->pipeline, GST_FORMAT_TIME, &position)) {
+ /* The UI expects these values in milliseconds, and GStreamer provides nanoseconds */
+ [self setCurrentUIPosition:position / GST_MSECOND duration:self->duration / GST_MSECOND];
+ }
+ return TRUE;
+}
+
+/* Forward declaration for the delayed seek callback */
+static gboolean delayed_seek_cb (GStreamerBackend *self);
+
+/* Perform seek, if we are not too close to the previous seek. Otherwise, schedule the seek for
+ * some time in the future. */
+static void execute_seek (gint64 position, GStreamerBackend *self) {
+ gint64 diff;
+
+ if (position == GST_CLOCK_TIME_NONE)
+ return;
+
+ diff = gst_util_get_timestamp () - self->last_seek_time;
+
+ if (GST_CLOCK_TIME_IS_VALID (self->last_seek_time) && diff < SEEK_MIN_DELAY) {
+ /* The previous seek was too close, delay this one */
+ GSource *timeout_source;
+
+ if (self->desired_position == GST_CLOCK_TIME_NONE) {
+ /* There was no previous seek scheduled. Setup a timer for some time in the future */
+ timeout_source = g_timeout_source_new ((SEEK_MIN_DELAY - diff) / GST_MSECOND);
+ g_source_set_callback (timeout_source, (GSourceFunc)delayed_seek_cb, (__bridge void *)self, NULL);
+ g_source_attach (timeout_source, self->context);
+ g_source_unref (timeout_source);
+ }
+ /* Update the desired seek position. If multiple requests are received before it is time
+ * to perform a seek, only the last one is remembered. */
+ self->desired_position = position;
+ GST_DEBUG ("Throttling seek to %" GST_TIME_FORMAT ", will be in %" GST_TIME_FORMAT,
+ GST_TIME_ARGS (position), GST_TIME_ARGS (SEEK_MIN_DELAY - diff));
+ } else {
+ /* Perform the seek now */
+ GST_DEBUG ("Seeking to %" GST_TIME_FORMAT, GST_TIME_ARGS (position));
+ self->last_seek_time = gst_util_get_timestamp ();
+ gst_element_seek_simple (self->pipeline, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT, position);
+ self->desired_position = GST_CLOCK_TIME_NONE;
+ }
+}
+
+/* Delayed seek callback. This gets called by the timer setup in the above function. */
+static gboolean delayed_seek_cb (GStreamerBackend *self) {
+ GST_DEBUG ("Doing delayed seek to %" GST_TIME_FORMAT, GST_TIME_ARGS (self->desired_position));
+ execute_seek (self->desired_position, self);
+ return FALSE;
+}
+
+/* Retrieve errors from the bus and show them on the UI */
+static void error_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
+{
+ GError *err;
+ gchar *debug_info;
+ gchar *message_string;
+
+ gst_message_parse_error (msg, &err, &debug_info);
+ message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);
+ g_clear_error (&err);
+ g_free (debug_info);
+ [self setUIMessage:message_string];
+ g_free (message_string);
+ gst_element_set_state (self->pipeline, GST_STATE_NULL);
+}
+
+/* Called when the End Of the Stream is reached. Just move to the beginning of the media and pause. */
+static void eos_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self) {
+ self->target_state = GST_STATE_PAUSED;
+ self->is_live = (gst_element_set_state (self->pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_NO_PREROLL);
+ execute_seek (0, self);
+}
+
+/* Called when the duration of the media changes. Just mark it as unknown, so we re-query it in the next UI refresh. */
+static void duration_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self) {
+ self->duration = GST_CLOCK_TIME_NONE;
+}
+
+/* Called when buffering messages are received. We inform the UI about the current buffering level and
+ * keep the pipeline paused until 100% buffering is reached. At that point, set the desired state. */
+static void buffering_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self) {
+ gint percent;
+
+ if (self->is_live)
+ return;
+
+ gst_message_parse_buffering (msg, &percent);
+ if (percent < 100 && self->target_state >= GST_STATE_PAUSED) {
+ gchar * message_string = g_strdup_printf ("Buffering %d%%", percent);
+ gst_element_set_state (self->pipeline, GST_STATE_PAUSED);
+ [self setUIMessage:message_string];
+ g_free (message_string);
+ } else if (self->target_state >= GST_STATE_PLAYING) {
+ gst_element_set_state (self->pipeline, GST_STATE_PLAYING);
+ } else if (self->target_state >= GST_STATE_PAUSED) {
+ [self setUIMessage:"Buffering complete"];
+ }
+}
+
+/* Called when the clock is lost */
+static void clock_lost_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self) {
+ if (self->target_state >= GST_STATE_PLAYING) {
+ gst_element_set_state (self->pipeline, GST_STATE_PAUSED);
+ gst_element_set_state (self->pipeline, GST_STATE_PLAYING);
+ }
+}
+
+/* Retrieve the video sink's Caps and tell the application about the media size */
+static void check_media_size (GStreamerBackend *self) {
+ GstElement *video_sink;
+ GstPad *video_sink_pad;
+ GstCaps *caps;
+ GstVideoInfo info;
+
+ /* Retrieve the Caps at the entrance of the video sink */
+ g_object_get (self->pipeline, "video-sink", &video_sink, NULL);
+
+ /* Do nothing if there is no video sink (this might be an audio-only clip */
+ if (!video_sink) return;
+
+ video_sink_pad = gst_element_get_static_pad (video_sink, "sink");
+ caps = gst_pad_get_current_caps (video_sink_pad);
+
+ if (gst_video_info_from_caps(&info, caps)) {
+ info.width = info.width * info.par_n / info.par_d
+ GST_DEBUG ("Media size is %dx%d, notifying application", info.width, info.height);
+
+ if (self->ui_delegate && [self->ui_delegate respondsToSelector:@selector(mediaSizeChanged:info.height:)])
+ {
+ [self->ui_delegate mediaSizeChanged:info.width height:info.height];
+ }
+ }
+
+ gst_caps_unref(caps);
+ gst_object_unref (video_sink_pad);
+ gst_object_unref(video_sink);
+}
+
+/* Notify UI about pipeline state changes */
+static void state_changed_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
+{
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ /* Only pay attention to messages coming from the pipeline, not its children */
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (self->pipeline)) {
+ self->state = new_state;
+ gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));
+ [self setUIMessage:message];
+ g_free (message);
+
+ if (old_state == GST_STATE_READY && new_state == GST_STATE_PAUSED)
+ {
+ check_media_size(self);
+
+ /* If there was a scheduled seek, perform it now that we have moved to the Paused state */
+ if (GST_CLOCK_TIME_IS_VALID (self->desired_position))
+ execute_seek (self->desired_position, self);
+ }
+ }
+}
+
+/* Check if all conditions are met to report GStreamer as initialized.
+ * These conditions will change depending on the application */
+-(void) check_initialization_complete
+{
+ if (!initialized && main_loop) {
+ GST_DEBUG ("Initialization complete, notifying application.");
+ if (ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerInitialized)])
+ {
+ [ui_delegate gstreamerInitialized];
+ }
+ initialized = TRUE;
+ }
+}
+
+/* Main method for the bus monitoring code */
+-(void) app_function
+{
+ GstBus *bus;
+ GSource *timeout_source;
+ GSource *bus_source;
+ GError *error = NULL;
+
+ GST_DEBUG ("Creating pipeline");
+
+ /* Create our own GLib Main Context and make it the default one */
+ context = g_main_context_new ();
+ g_main_context_push_thread_default(context);
+
+ /* Build pipeline */
+ pipeline = gst_parse_launch("playbin", &error);
+ if (error) {
+ gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
+ g_clear_error (&error);
+ [self setUIMessage:message];
+ g_free (message);
+ return;
+ }
+
+ /* Set the pipeline to READY, so it can already accept a window handle */
+ gst_element_set_state(pipeline, GST_STATE_READY);
+
+ video_sink = gst_bin_get_by_interface(GST_BIN(pipeline), GST_TYPE_VIDEO_OVERLAY);
+ if (!video_sink) {
+ GST_ERROR ("Could not retrieve video sink");
+ return;
+ }
+ gst_video_overlay_set_window_handle(GST_VIDEO_OVERLAY(video_sink), (guintptr) (id) ui_video_view);
+
+ /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
+ bus = gst_element_get_bus (pipeline);
+ bus_source = gst_bus_create_watch (bus);
+ g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
+ g_source_attach (bus_source, context);
+ g_source_unref (bus_source);
+ g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, (__bridge void *)self);
+ g_signal_connect (G_OBJECT (bus), "message::eos", (GCallback)eos_cb, (__bridge void *)self);
+ g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, (__bridge void *)self);
+ g_signal_connect (G_OBJECT (bus), "message::duration", (GCallback)duration_cb, (__bridge void *)self);
+ g_signal_connect (G_OBJECT (bus), "message::buffering", (GCallback)buffering_cb, (__bridge void *)self);
+ g_signal_connect (G_OBJECT (bus), "message::clock-lost", (GCallback)clock_lost_cb, (__bridge void *)self);
+ gst_object_unref (bus);
+
+ /* Register a function that GLib will call 4 times per second */
+ timeout_source = g_timeout_source_new (250);
+ g_source_set_callback (timeout_source, (GSourceFunc)refresh_ui, (__bridge void *)self, NULL);
+ g_source_attach (timeout_source, context);
+ g_source_unref (timeout_source);
+
+ /* Create a GLib Main Loop and set it to run */
+ GST_DEBUG ("Entering main loop...");
+ main_loop = g_main_loop_new (context, FALSE);
+ [self check_initialization_complete];
+ g_main_loop_run (main_loop);
+ GST_DEBUG ("Exited main loop");
+ g_main_loop_unref (main_loop);
+ main_loop = NULL;
+
+ /* Free resources */
+ g_main_context_pop_thread_default(context);
+ g_main_context_unref (context);
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+ pipeline = NULL;
+
+ ui_delegate = NULL;
+ ui_video_view = NULL;
+
+ return;
+}
+
+@end
+```
+
+Supporting arbitrary media URIs
+
+The UI code will call `setUri` whenever it wants to change the playing
+URI (in this tutorial the URI never changes, but it does in the next
+one):
+
+```
+-(void) setUri:(NSString*)uri
+{
+ const char *char_uri = [uri UTF8String];
+ g_object_set(pipeline, "uri", char_uri, NULL);
+ GST_DEBUG ("URI set to %s", char_uri);
+}
+```
+
+We first need to obtain a plain `char *` from within the `NSString *` we
+get, using the `UTF8String` method.
+
+`playbin`’s URI is exposed as a common GObject property, so we simply
+set it with `g_object_set()`.
+
+### Reporting media size
+
+Some codecs allow the media size (width and height of the video) to
+change during playback. For simplicity, this tutorial assumes that they
+do not. Therefore, in the READY to PAUSED state change, once the Caps of
+the decoded media are known, we inspect them
+in `check_media_size()`:
+
+```
+/* Retrieve the video sink's Caps and tell the application about the media size */
+static void check_media_size (GStreamerBackend *self) {
+ GstElement *video_sink;
+ GstPad *video_sink_pad;
+ GstCaps *caps;
+ GstVideoInfo info;
+
+ /* Retrieve the Caps at the entrance of the video sink */
+ g_object_get (self->pipeline, "video-sink", &video_sink, NULL);
+
+ /* Do nothing if there is no video sink (this might be an audio-only clip */
+ if (!video_sink) return;
+
+ video_sink_pad = gst_element_get_static_pad (video_sink, "sink");
+ caps = gst_pad_get_current_caps (video_sink_pad);
+
+ if (gst_video_info_from_caps(&info, caps)) {
+ info.width = info.width * info.par_n / info.par_d;
+ GST_DEBUG ("Media size is %dx%d, notifying application", info.width, info.height);
+
+ if (self->ui_delegate && [self->ui_delegate respondsToSelector:@selector(mediaSizeChanged:info.height:)])
+ {
+ [self->ui_delegate mediaSizeChanged:info.width height:info.height];
+ }
+ }
+
+ gst_caps_unref(caps);
+ gst_object_unref (video_sink_pad);
+ gst_object_unref(video_sink);
+}
+```
+
+We first retrieve the video sink element from the pipeline, using
+the `video-sink` property of `playbin`, and then its sink Pad. The
+negotiated Caps of this Pad, which we recover using
+`gst_pad_get_current_caps()`, are the Caps of the decoded media.
+
+The helper functions `gst_video_format_parse_caps()` and
+`gst_video_parse_caps_pixel_aspect_ratio()` turn the Caps into
+manageable integers, which we pass to the application through
+its `mediaSizeChanged` callback.
+
+### Refreshing the Seek Bar
+
+To keep the UI updated, a GLib timer is installed in
+the `app_function` that fires 4 times per second (or every 250ms),
+right before entering the main loop:
+
+```
+ /* Register a function that GLib will call 4 times per second */
+ timeout_source = g_timeout_source_new (250);
+ g_source_set_callback (timeout_source, (GSourceFunc)refresh_ui, (__bridge void *)self, NULL);
+ g_source_attach (timeout_source, context);
+ g_source_unref (timeout_source);
+```
+
+Then, in the refresh\_ui
+method:
+
+```
+/* If we have pipeline and it is running, query the current position and clip duration and inform
+ * the application */
+static gboolean refresh_ui (GStreamerBackend *self) {
+ gint64 position;
+
+ /* We do not want to update anything unless we have a working pipeline in the PAUSED or PLAYING state */
+ if (!self || !self->pipeline || self->state < GST_STATE_PAUSED)
+ return TRUE;
+
+ /* If we didn't know it yet, query the stream duration */
+ if (!GST_CLOCK_TIME_IS_VALID (self->duration)) {
+ gst_element_query_duration (self->pipeline, GST_FORMAT_TIME, &self->duration);
+ }
+
+ if (gst_element_query_position (self->pipeline, GST_FORMAT_TIME, &position)) {
+ /* The UI expects these values in milliseconds, and GStreamer provides nanoseconds */
+ [self setCurrentUIPosition:position / GST_MSECOND duration:self->duration / GST_MSECOND];
+ }
+ return TRUE;
+}
+```
+
+If it is unknown, the clip duration is retrieved, as explained in
+[](sdk-basic-tutorial-time-management.md). The current position is
+retrieved next, and the UI is informed of both through its
+`setCurrentUIPosition` callback.
+
+Bear in mind that all time-related measures returned by GStreamer are in
+nanoseconds, whereas, for simplicity, we decided to make the UI code
+work in milliseconds.
+
+### Seeking with the Seek Bar
+
+The UI code already takes care of most of the complexity of seeking by
+dragging the thumb of the Seek Bar. From the `GStreamerBackend`, we just
+need to honor the calls to `setPosition` and instruct the pipeline to
+jump to the indicated position.
+
+There are, though, a couple of caveats. Firstly, seeks are only possible
+when the pipeline is in the PAUSED or PLAYING state, and we might
+receive seek requests before that happens. Secondly, dragging the Seek
+Bar can generate a very high number of seek requests in a short period
+of time, which is visually useless and will impair responsiveness. Let’s
+see how to overcome these problems.
+
+#### Delayed seeks
+
+In `setPosition`:
+
+```
+-(void) setPosition:(NSInteger)milliseconds
+{
+ gint64 position = (gint64)(milliseconds * GST_MSECOND);
+ if (state >= GST_STATE_PAUSED) {
+ execute_seek(position, self);
+ } else {
+ GST_DEBUG ("Scheduling seek to %" GST_TIME_FORMAT " for later", GST_TIME_ARGS (position));
+ self->desired_position = position;
+ }
+}
+```
+
+If we are already in the correct state for seeking, execute it right
+away; otherwise, store the desired position in
+the `desired_position` variable. Then, in
+the `state_changed_cb()` callback:
+
+```
+ if (old_state == GST_STATE_READY && new_state == GST_STATE_PAUSED)
+ {
+ check_media_size(self);
+
+ /* If there was a scheduled seek, perform it now that we have moved to the Paused state */
+ if (GST_CLOCK_TIME_IS_VALID (self->desired_position))
+ execute_seek (self->desired_position, self);
+ }
+```
+
+Once the pipeline moves from the READY to the PAUSED state, we check if
+there is a pending seek operation and execute it.
+The `desired_position` variable is reset inside `execute_seek()`.
+
+#### Seek throttling
+
+A seek is potentially a lengthy operation. The demuxer (the element
+typically in charge of seeking) needs to estimate the appropriate byte
+offset inside the media file that corresponds to the time position to
+jump to. Then, it needs to start decoding from that point until the
+desired position is reached. If the initial estimate is accurate, this
+will not take long, but, on some container formats, or when indexing
+information is missing, it can take up to several seconds.
+
+If a demuxer is in the process of performing a seek and receives a
+second one, it is up to it to finish the first one, start the second one
+or abort both, which is a bad thing. A simple method to avoid this issue
+is *throttling*, which means that we will only allow one seek every half
+a second (for example): after performing a seek, only the last seek
+request received during the next 500ms is stored, and will be honored
+once this period elapses.
+
+To achieve this, all seek requests are routed through
+the `execute_seek()` method:
+
+```
+/* Perform seek, if we are not too close to the previous seek. Otherwise, schedule the seek for
+ * some time in the future. */
+static void execute_seek (gint64 position, GStreamerBackend *self) {
+ gint64 diff;
+
+ if (position == GST_CLOCK_TIME_NONE)
+ return;
+
+ diff = gst_util_get_timestamp () - self->last_seek_time;
+
+ if (GST_CLOCK_TIME_IS_VALID (self->last_seek_time) && diff < SEEK_MIN_DELAY) {
+ /* The previous seek was too close, delay this one */
+ GSource *timeout_source;
+
+ if (self->desired_position == GST_CLOCK_TIME_NONE) {
+ /* There was no previous seek scheduled. Setup a timer for some time in the future */
+ timeout_source = g_timeout_source_new ((SEEK_MIN_DELAY - diff) / GST_MSECOND);
+ g_source_set_callback (timeout_source, (GSourceFunc)delayed_seek_cb, (__bridge void *)self, NULL);
+ g_source_attach (timeout_source, self->context);
+ g_source_unref (timeout_source);
+ }
+ /* Update the desired seek position. If multiple requests are received before it is time
+ * to perform a seek, only the last one is remembered. */
+ self->desired_position = position;
+ GST_DEBUG ("Throttling seek to %" GST_TIME_FORMAT ", will be in %" GST_TIME_FORMAT,
+ GST_TIME_ARGS (position), GST_TIME_ARGS (SEEK_MIN_DELAY - diff));
+ } else {
+ /* Perform the seek now */
+ GST_DEBUG ("Seeking to %" GST_TIME_FORMAT, GST_TIME_ARGS (position));
+ self->last_seek_time = gst_util_get_timestamp ();
+ gst_element_seek_simple (self->pipeline, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT, position);
+ self->desired_position = GST_CLOCK_TIME_NONE;
+ }
+}
+```
+
+The time at which the last seek was performed is stored in
+the `last_seek_time` variable. This is wall clock time, not to be
+confused with the stream time carried in the media time stamps, and is
+obtained with `gst_util_get_timestamp()`.
+
+If enough time has passed since the last seek operation, the new one is
+directly executed and `last_seek_time` is updated. Otherwise, the new
+seek is scheduled for later. If there is no previously scheduled seek, a
+one-shot timer is setup to trigger 500ms after the last seek operation.
+If another seek was already scheduled, its desired position is simply
+updated with the new one.
+
+The one-shot timer calls `delayed_seek_cb()`, which simply
+calls `execute_seek()` again.
+
+> ![information]
+> Ideally, `execute_seek()` will now find that enough time has indeed passed since the last seek and the scheduled one will proceed. It might happen, though, that after 500ms of the previous seek, and before the timer wakes up, yet another seek comes through and is executed. `delayed_seek_cb()` needs to check for this condition to avoid performing two very close seeks, and therefore calls `execute_seek()` instead of performing the seek itself.
+>
+>This is not a complete solution: the scheduled seek will still be executed, even though a more-recent seek has already been executed that should have cancelled it. However, it is a good tradeoff between functionality and simplicity.
+
+### Network resilience
+
+[](sdk-basic-tutorial-streaming.md) has already
+shown how to adapt to the variable nature of the network bandwidth by
+using buffering. The same procedure is used here, by listening to the
+buffering
+messages:
+
+```
+g_signal_connect (G_OBJECT (bus), "message::buffering", (GCallback)buffering_cb, (__bridge void *)self);
+```
+
+And pausing the pipeline until buffering is complete (unless this is a
+live
+source):
+
+
+
+```
+/* Called when buffering messages are received. We inform the UI about the current buffering level and
+ * keep the pipeline paused until 100% buffering is reached. At that point, set the desired state. */
+static void buffering_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self) {
+ gint percent;
+
+ if (self->is_live)
+ return;
+
+ gst_message_parse_buffering (msg, &percent);
+ if (percent < 100 && self->target_state >= GST_STATE_PAUSED) {
+ gchar * message_string = g_strdup_printf ("Buffering %d%%", percent);
+ gst_element_set_state (self->pipeline, GST_STATE_PAUSED);
+ [self setUIMessage:message_string];
+ g_free (message_string);
+ } else if (self->target_state >= GST_STATE_PLAYING) {
+ gst_element_set_state (self->pipeline, GST_STATE_PLAYING);
+ } else if (self->target_state >= GST_STATE_PAUSED) {
+ [self setUIMessage:"Buffering complete"];
+ }
+}
+```
+
+`target_state` is the state in which we have been instructed to set the
+pipeline, which might be different to the current state, because
+buffering forces us to go to PAUSED. Once buffering is complete we set
+the pipeline to the `target_state`.
+
+## Conclusion
+
+This tutorial has shown how to embed a `playbin` pipeline into an iOS
+application. This, effectively, turns such application into a basic
+media player, capable of streaming and decoding all the formats
+GStreamer understands. More particularly, it has shown:
+
+ - How to keep the User Interface regularly updated by using a timer,
+ querying the pipeline position and calling a UI code method.
+ - How to implement a Seek Bar which follows the current position and
+ transforms thumb motion into reliable seek events.
+ - How to report the media size to adapt the display surface, by
+ reading the sink Caps at the appropriate moment and telling the UI
+ about it.
+
+The next tutorial adds the missing bits to turn the application built
+here into an acceptable iOS media player.
+
+ [information]: images/icons/emoticons/information.png
+ [screenshot]: images/sdk-ios-tutorial-a-basic-media-player-screenshot.png
\ No newline at end of file
--- /dev/null
+# iOS tutorial 5: A Complete media player
+
+## Goal
+
+![screenshot0]
+![screenshot1]
+
+This tutorial wants to be the “demo application” that showcases what can
+be done with GStreamer on the iOS platform.
+
+It is intended to be built and run, rather than analyzed for its
+pedagogical value, since it adds very little GStreamer knowledge over
+what has already been shown in [](sdk-ios-tutorial-a-basic-media-player.md).
+
+It demonstrates the main functionality that a conventional media player
+has, but it is not a complete application yet, therefore it has not been
+uploaded to the AppStore.
+
+## Introduction
+
+The previous tutorial already implemented a basic media player. This one
+simply adds a few finishing touches. In particular, it adds the
+capability to choose the media to play, and disables the screensaver
+during media playback.
+
+These are not features directly related to GStreamer, and are therefore
+outside the scope of these tutorials. Only a few implementation pointers
+are given here.
+
+## Selecting the media to play
+
+A new `UIView` has been added, derived from `UITableViewController`
+which shows a list of clips. When one is selected, the
+`VideoViewController` from [](sdk-ios-tutorial-a-basic-media-player.md) appears
+and its URI property is set to the URI of the selected clip.
+
+The list of clips is populated from three sources: Media from the
+device’s Photo library, Media from the application’s Documents folder
+(accessible through iTunes file sharing) and a list of hardcoded
+Internet addresses, selected to showcase different container and codec
+formats, and a couple of bogus ones, to illustrate error reporting.
+
+## Preventing the screen from turning off
+
+While watching a movie, there is typically no user activity. After a
+short period of such inactivity, iOS will dim the screen, and then turn
+it off completely. To prevent this, the `idleTimerDisabled` property of
+the `UIApplication` class is used. The application sets it to YES
+(screen locking disabled) when the Play button is pressed, so the screen
+is never turned off, and sets it back to NO when the Pause button is
+pressed.
+
+## Conclusion
+
+This finishes the series of iOS tutorials. Each one of the preceding
+tutorials has evolved on top of the previous one, showing how to
+implement a particular set of features, and concluding in this Tutorial
+5. The goal of Tutorial 5 is to build a complete media player which can
+already be used to showcase the integration of GStreamer and iOS.
+
+It has been a pleasure having you here, and see you soon!
+
+ [screenshot0]: images/sdk-ios-tutorial-a-complete-media-player-screenshot-0.png
+ [screenshot1]: images/sdk-ios-tutorial-a-complete-media-player-screenshot-1.png
--- /dev/null
+# iOS tutorial 2: A running pipeline
+
+## Goal
+
+![screenshot]
+
+As seen in the [Basic](sdk-basic-tutorials.md) and
+[Playback](sdk-playback-tutorials.md) tutorials, GStreamer integrates
+nicely with GLib’s main loops, so pipeline operation and user interface
+can be monitored simultaneously in a very simple way. However, platforms
+like iOS or Android do not use GLib and therefore extra care must be
+taken to keep track of the pipeline progress without blocking the user
+interface (UI).
+
+This tutorial shows:
+
+ - How to move the GStreamer-handling code to a separate Dispatch Queue
+ whereas UI managing still happens from the Main Dispatch Queue
+ - How to communicate between the Objective-C UI code and the C
+ GStreamer code
+
+## Introduction
+
+When using a Graphical User Interface (UI), if the application waits for
+GStreamer calls to complete the user experience will suffer. The usual
+approach, with the [GTK+ toolkit](http://www.gtk.org/) for example, is
+to relinquish control to a GLib `GMainLoop` and let it control the
+events coming from the UI or GStreamer.
+
+Other graphical toolkits that are not based on GLib, like the [Cocoa
+Touch](https://developer.apple.com/library/ios/documentation/General/Conceptual/DevPedia-CocoaCore/Cocoa.html)
+framework used on iOS devices, cannot use this option, though. The
+solution used in this tutorial uses a GLib `GMainLoop` for its
+simplicity, but moves it to a separate thread (a [Dispatch
+Queue](http://developer.apple.com/library/ios/#documentation/General/Conceptual/ConcurrencyProgrammingGuide/OperationQueues/OperationQueues.html)
+different than the main one) so it does not block the user interface
+operation.
+
+Additionally, this tutorial shows a few places where caution has to be
+taken when calling from Objective-C to C and vice versa.
+
+The code below builds a pipeline with an `audiotestsrc` and
+an `autoaudiosink` (it plays an audible tone). Two buttons in the UI
+allow setting the pipeline to PLAYING or PAUSED. A Label in the UI shows
+messages sent from the C code (for errors and state changes).
+
+## The User Interface
+
+A toolbar at the bottom of the screen contains a Play and a Pause
+button. Over the toolbar there is a Label used to display messages from
+GStreamer. This tutorial does not require more elements, but the
+following lessons will build their User Interfaces on top of this one,
+adding more components.
+
+## The View Controller
+
+The `ViewController` class manages the UI, instantiates
+the `GStreamerBackend` and also performs some UI-related tasks on its
+behalf:
+
+**ViewController.m**
+
+```
+#import "ViewController.h"
+#import "GStreamerBackend.h"
+#import <UIKit/UIKit.h>
+
+@interface ViewController () {
+ GStreamerBackend *gst_backend;
+}
+
+@end
+
+@implementation ViewController
+
+/*
+ * Methods from UIViewController
+ */
+
+- (void)viewDidLoad
+{
+ [super viewDidLoad];
+
+ play_button.enabled = FALSE;
+ pause_button.enabled = FALSE;
+
+ gst_backend = [[GStreamerBackend alloc] init:self];
+}
+
+- (void)didReceiveMemoryWarning
+{
+ [super didReceiveMemoryWarning];
+ // Dispose of any resources that can be recreated.
+}
+
+/* Called when the Play button is pressed */
+-(IBAction) play:(id)sender
+{
+ [gst_backend play];
+}
+
+/* Called when the Pause button is pressed */
+-(IBAction) pause:(id)sender
+{
+ [gst_backend pause];
+}
+
+/*
+ * Methods from GstreamerBackendDelegate
+ */
+
+-(void) gstreamerInitialized
+{
+ dispatch_async(dispatch_get_main_queue(), ^{
+ play_button.enabled = TRUE;
+ pause_button.enabled = TRUE;
+ message_label.text = @"Ready";
+ });
+}
+
+-(void) gstreamerSetUIMessage:(NSString *)message
+{
+ dispatch_async(dispatch_get_main_queue(), ^{
+ message_label.text = message;
+ });
+}
+
+@end
+```
+
+An instance of the `GStreamerBackend` in stored inside the class:
+
+```
+@interface ViewController () {
+ GStreamerBackend *gst_backend;
+}
+```
+
+This instance is created in the `viewDidLoad` function through a custom
+`init:` method in the `GStreamerBackend`:
+
+```
+- (void)viewDidLoad
+{
+ [super viewDidLoad];
+
+ play_button.enabled = FALSE;
+ pause_button.enabled = FALSE;
+
+ gst_backend = [[GStreamerBackend alloc] init:self];
+}
+```
+
+This custom method is required to pass the object that has to be used as
+the UI delegate (in this case, ourselves, the `ViewController`).
+
+The Play and Pause buttons are also disabled in the
+`viewDidLoad` function, and they are not re-enabled until the
+`GStreamerBackend` reports that it is initialized and ready.
+
+```
+/* Called when the Play button is pressed */
+-(IBAction) play:(id)sender
+{
+ [gst_backend play];
+}
+
+/* Called when the Pause button is pressed */
+-(IBAction) pause:(id)sender
+{
+ [gst_backend pause];
+}
+```
+
+These two methods are called when the user presses the Play or Pause
+buttons, and simply forward the call to the appropriate method in the
+`GStreamerBackend`.
+
+```
+-(void) gstreamerInitialized
+{
+ dispatch_async(dispatch_get_main_queue(), ^{
+ play_button.enabled = TRUE;
+ pause_button.enabled = TRUE;
+ message_label.text = @"Ready";
+ });
+}
+```
+
+The `gstreamerInitialized` method is defined in the
+`GStreamerBackendDelegate` protocol and indicates that the backend is
+ready to accept commands. In this case, the Play and Pause buttons are
+re-enabled and the Label text is set to “Ready”. This method is called
+from a Dispatch Queue other than the Main one; therefore the need for
+the
+[dispatch_async()](https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man3/dispatch_async.3.html) call
+wrapping all UI code.
+
+```
+-(void) gstreamerSetUIMessage:(NSString *)message
+{
+ dispatch_async(dispatch_get_main_queue(), ^{
+ message_label.text = message;
+ });
+}
+```
+
+The `gstreamerSetUIMessage:` method also belongs to the
+`GStreamerBackendDelegate` protocol. It is called when the backend wants
+to report some message to the user. In this case, the message is copied
+onto the Label in the UI, again, from within a
+[dispatch_async()](https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man3/dispatch_async.3.html) call.
+
+## The GStreamer Backend
+
+The `GStreamerBackend` class performs all GStreamer-related tasks and
+offers a simplified interface to the application, which does not need to
+deal with all the GStreamer details. When it needs to perform any UI
+action, it does so through a delegate, which is expected to adhere to
+the `GStreamerBackendDelegate` protocol:
+
+**GStreamerBackend.m**
+
+```
+#import "GStreamerBackend.h"
+
+#include <gst/gst.h>
+
+GST_DEBUG_CATEGORY_STATIC (debug_category);
+#define GST_CAT_DEFAULT debug_category
+
+@interface GStreamerBackend()
+-(void)setUIMessage:(gchar*) message;
+-(void)app_function;
+-(void)check_initialization_complete;
+@end
+
+@implementation GStreamerBackend {
+ id ui_delegate; /* Class that we use to interact with the user interface */
+ GstElement *pipeline; /* The running pipeline */
+ GMainContext *context; /* GLib context used to run the main loop */
+ GMainLoop *main_loop; /* GLib main loop */
+ gboolean initialized; /* To avoid informing the UI multiple times about the initialization */
+}
+
+/*
+ * Interface methods
+ */
+
+-(id) init:(id) uiDelegate
+{
+ if (self = [super init])
+ {
+ self->ui_delegate = uiDelegate;
+
+ GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-2", 0, "iOS tutorial 2");
+ gst_debug_set_threshold_for_name("tutorial-2", GST_LEVEL_DEBUG);
+
+ /* Start the bus monitoring task */
+ dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
+ [self app_function];
+ });
+ }
+
+ return self;
+}
+
+-(void) dealloc
+{
+ if (pipeline) {
+ GST_DEBUG("Setting the pipeline to NULL");
+ gst_element_set_state(pipeline, GST_STATE_NULL);
+ gst_object_unref(pipeline);
+ pipeline = NULL;
+ }
+}
+
+-(void) play
+{
+ if(gst_element_set_state(pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE) {
+ [self setUIMessage:"Failed to set pipeline to playing"];
+ }
+}
+
+-(void) pause
+{
+ if(gst_element_set_state(pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_FAILURE) {
+ [self setUIMessage:"Failed to set pipeline to paused"];
+ }
+}
+
+/*
+ * Private methods
+ */
+
+/* Change the message on the UI through the UI delegate */
+-(void)setUIMessage:(gchar*) message
+{
+ NSString *string = [NSString stringWithUTF8String:message];
+ if(ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerSetUIMessage:)])
+ {
+ [ui_delegate gstreamerSetUIMessage:string];
+ }
+}
+
+/* Retrieve errors from the bus and show them on the UI */
+static void error_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
+{
+ GError *err;
+ gchar *debug_info;
+ gchar *message_string;
+
+ gst_message_parse_error (msg, &err, &debug_info);
+ message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);
+ g_clear_error (&err);
+ g_free (debug_info);
+ [self setUIMessage:message_string];
+ g_free (message_string);
+ gst_element_set_state (self->pipeline, GST_STATE_NULL);
+}
+
+/* Notify UI about pipeline state changes */
+static void state_changed_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
+{
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ /* Only pay attention to messages coming from the pipeline, not its children */
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (self->pipeline)) {
+ gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));
+ [self setUIMessage:message];
+ g_free (message);
+ }
+}
+
+/* Check if all conditions are met to report GStreamer as initialized.
+ * These conditions will change depending on the application */
+-(void) check_initialization_complete
+{
+ if (!initialized && main_loop) {
+ GST_DEBUG ("Initialization complete, notifying application.");
+ if (ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerInitialized)])
+ {
+ [ui_delegate gstreamerInitialized];
+ }
+ initialized = TRUE;
+ }
+}
+
+/* Main method for the bus monitoring code */
+-(void) app_function
+{
+ GstBus *bus;
+ GSource *bus_source;
+ GError *error = NULL;
+
+ GST_DEBUG ("Creating pipeline");
+
+ /* Create our own GLib Main Context and make it the default one */
+ context = g_main_context_new ();
+ g_main_context_push_thread_default(context);
+
+ /* Build pipeline */
+ pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);
+ if (error) {
+ gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
+ g_clear_error (&error);
+ [self setUIMessage:message];
+ g_free (message);
+ return;
+ }
+
+ /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
+ bus = gst_element_get_bus (pipeline);
+ bus_source = gst_bus_create_watch (bus);
+ g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
+ g_source_attach (bus_source, context);
+ g_source_unref (bus_source);
+ g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, (__bridge void *)self);
+ g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, (__bridge void *)self);
+ gst_object_unref (bus);
+
+ /* Create a GLib Main Loop and set it to run */
+ GST_DEBUG ("Entering main loop...");
+ main_loop = g_main_loop_new (context, FALSE);
+ [self check_initialization_complete];
+ g_main_loop_run (main_loop);
+ GST_DEBUG ("Exited main loop");
+ g_main_loop_unref (main_loop);
+ main_loop = NULL;
+
+ /* Free resources */
+ g_main_context_pop_thread_default(context);
+ g_main_context_unref (context);
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+
+ return;
+}
+
+@end
+```
+
+
+
+#### Interface methods:
+
+```
+-(id) init:(id) uiDelegate
+{
+ if (self = [super init])
+ {
+ self->ui_delegate = uiDelegate;
+
+ GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-2", 0, "iOS tutorial 2");
+ gst_debug_set_threshold_for_name("tutorial-2", GST_LEVEL_DEBUG);
+
+ /* Start the bus monitoring task */
+ dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
+ [self app_function];
+ });
+ }
+
+ return self;
+}
+```
+
+The `init:` method creates the instance by calling `[super init]`,
+stores the delegate object that will handle the UI interaction and
+launches the `app_function`, from a separate, concurrent, Dispatch
+Queue. The `app_function` monitors the GStreamer bus for messages and
+warns the application when interesting things happen.
+
+`init:` also registers a new GStreamer debug category and sets its
+threshold, so we can see the debug output from within Xcode and keep
+track of our application progress.
+
+```
+-(void) dealloc
+{
+ if (pipeline) {
+ GST_DEBUG("Setting the pipeline to NULL");
+ gst_element_set_state(pipeline, GST_STATE_NULL);
+ gst_object_unref(pipeline);
+ pipeline = NULL;
+ }
+}
+```
+
+The `dealloc` method takes care of bringing the pipeline to the NULL
+state and releasing it.
+
+```
+-(void) play
+{
+ if(gst_element_set_state(pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE) {
+ [self setUIMessage:"Failed to set pipeline to playing"];
+ }
+}
+
+-(void) pause
+{
+ if(gst_element_set_state(pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_FAILURE) {
+ [self setUIMessage:"Failed to set pipeline to paused"];
+ }
+}
+```
+
+The `play` and `pause` methods simply try to set the pipeline to the
+desired state and warn the application if something fails.
+
+#### Private methods:
+
+```
+/* Change the message on the UI through the UI delegate */
+-(void)setUIMessage:(gchar*) message
+{
+ NSString *string = [NSString stringWithUTF8String:message];
+ if(ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerSetUIMessage:)])
+ {
+ [ui_delegate gstreamerSetUIMessage:string];
+ }
+}
+```
+
+`setUIMessage:` turns the C strings that GStreamer uses (UTF8 `char *`)
+into `NSString *` and displays them through the
+`gstreamerSetUIMessage` method of the `GStreamerBackendDelegate`. The
+implementation of this method is marked as `@optional`, and hence the
+check for its existence in the delegate with `respondsToSelector:`
+
+```
+/* Retrieve errors from the bus and show them on the UI */
+static void error_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
+{
+ GError *err;
+ gchar *debug_info;
+ gchar *message_string;
+
+ gst_message_parse_error (msg, &err, &debug_info);
+ message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);
+ g_clear_error (&err);
+ g_free (debug_info);
+ [self setUIMessage:message_string];
+ g_free (message_string);
+ gst_element_set_state (self->pipeline, GST_STATE_NULL);
+}
+
+/* Notify UI about pipeline state changes */
+static void state_changed_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
+{
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ /* Only pay attention to messages coming from the pipeline, not its children */
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (self->pipeline)) {
+ gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));
+ [self setUIMessage:message];
+ g_free (message);
+ }
+}
+```
+
+The `error_cb()` and `state_changed_cb()` are callbacks registered to
+the `error` and `state-changed` events in GStreamer, and their goal is
+to inform the user about these events. These callbacks have been widely
+used in the [Basic tutorials](sdk-basic-tutorials.md) and their
+implementation is very similar, except for two points:
+
+Firstly, the messages are conveyed to the user through the
+`setUIMessage:` private method discussed above.
+
+Secondly, they require an instance of a `GStreamerBackend` object in
+order to call its instance method `setUIMessage:`, which is passed
+through the `userdata` pointer of the callbacks (the `self` pointer in
+these implementations). This is discussed below when registering the
+callbacks in the `app_function`.
+
+```
+/* Check if all conditions are met to report GStreamer as initialized.
+ * These conditions will change depending on the application */
+-(void) check_initialization_complete
+{
+ if (!initialized && main_loop) {
+ GST_DEBUG ("Initialization complete, notifying application.");
+ if (ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerInitialized)])
+ {
+ [ui_delegate gstreamerInitialized];
+ }
+ initialized = TRUE;
+ }
+}
+```
+
+`check_initialization_complete()` verifies that all conditions are met
+to consider the backend ready to accept commands and tell the
+application if so. In this simple tutorial the only conditions are that
+the main loop exists and that we have not already told the application
+about this fact. Later (more complex) tutorials include additional
+conditions.
+
+Finally, most of the GStreamer work is performed in the app_function.
+It exists with almost identical content in the Android tutorial, which
+exemplifies how the same code can run on both platforms with little
+change.
+
+```
+ /* Create our own GLib Main Context and make it the default one */
+ context = g_main_context_new ();
+ g_main_context_push_thread_default(context);
+```
+
+It first creates a GLib context so all `GSource`s are kept in the same
+place. This also helps cleaning after GSources created by other
+libraries which might not have been properly disposed of. A new context
+is created with `g_main_context_new()` and then it is made the default
+one for the thread with `g_main_context_push_thread_default()`.
+
+```
+ /* Build pipeline */
+ pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);
+ if (error) {
+ gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
+ g_clear_error (&error);
+ [self setUIMessage:message];
+ g_free (message);
+ return;
+ }
+```
+
+It then creates a pipeline the easy way, with `gst_parse_launch()`. In
+this case, it is simply an `audiotestsrc` (which produces a continuous
+tone) and an `autoaudiosink`, with accompanying adapter
+elements.
+
+```
+ /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
+ bus = gst_element_get_bus (pipeline);
+ bus_source = gst_bus_create_watch (bus);
+ g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
+ g_source_attach (bus_source, context);
+ g_source_unref (bus_source);
+ g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, (__bridge void *)self);
+ g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, (__bridge void *)self);
+ gst_object_unref (bus);
+```
+
+These lines create a bus signal watch and connect to some interesting
+signals, just like we have been doing in the [Basic
+tutorials](sdk-basic-tutorials.md). The creation of the watch is done
+step by step instead of using `gst_bus_add_signal_watch()` to exemplify
+how to use a custom GLib context. The interesting bit here is the usage
+of a
+[__bridge](http://clang.llvm.org/docs/AutomaticReferenceCounting.html#bridged-casts)
+cast to convert an Objective-C object into a plain C pointer. In this
+case we do not worry much about transferal of ownership of the object,
+because it travels through C-land untouched. It re-emerges at the
+different callbacks through the userdata pointer and cast again to a
+`GStreamerBackend *`.
+
+```
+ /* Create a GLib Main Loop and set it to run */
+ GST_DEBUG ("Entering main loop...");
+ main_loop = g_main_loop_new (context, FALSE);
+ [self check_initialization_complete];
+ g_main_loop_run (main_loop);
+ GST_DEBUG ("Exited main loop");
+ g_main_loop_unref (main_loop);
+ main_loop = NULL;
+```
+
+Finally, the main loop is created and set to run. Before entering the
+main loop, though, `check_initialization_complete()` is called. Upon
+exit, the main loop is disposed of.
+
+And this is it! This has been a rather long tutorial, but we covered a
+lot of territory. Building on top of this one, the following ones are
+shorter and focus only on the new topics.
+
+## Conclusion
+
+This tutorial has shown:
+
+ - How to handle GStreamer code from a separate thread using a
+ [Dispatch
+ Queue](http://developer.apple.com/library/ios/#documentation/General/Conceptual/ConcurrencyProgrammingGuide/OperationQueues/OperationQueues.html) other
+ than the Main one.
+ - How to pass objects between the Objective-C UI code and the C
+ GStreamer code.
+
+Most of the methods introduced in this tutorial,
+like `check_initialization_complete()`and `app_function()`, and the
+interface methods `init:`, `play:`, `pause:`,
+`gstreamerInitialized:` and `setUIMessage:` will continue to be used in
+the following tutorials with minimal modifications, so better get used
+to them!
+
+It has been a pleasure having you here, and see you soon!
+
+
+ [screenshot]: images/sdk-ios-tutorial-a-running-pipeline-screenshot.png
\ No newline at end of file
--- /dev/null
+# iOS tutorial 1: Link against GStreamer
+
+## Goal
+
+![screenshot]
+
+The first iOS tutorial is simple. The objective is to get the GStreamer
+version and display it on screen. It exemplifies how to link against the
+GStreamer library from Xcode using objective-C.
+
+## Hello GStreamer!
+
+The code for this project can be found in the tutorials folder of
+**FIXME: where**. It was created using the GStreamer Single View
+Application template. The view contains only a `UILabel` that will be
+used to display the GStreamer's version to the user.
+
+## The User Interface
+
+The UI uses storyboards and contains a single `View` with a centered
+`UILabel`. The `ViewController` for the `View` links its
+`label` variable to this `UILabel` as an `IBOutlet`.
+
+**ViewController.h**
+
+```
+#import <UIKit/UIKit.h>
+
+@interface ViewController : UIViewController {
+ IBOutlet UILabel *label;
+}
+
+@property (retain,nonatomic) UILabel *label;
+
+@end
+```
+
+## The GStreamer backend
+
+All GStreamer-handling code is kept in a single Objective-C class called
+`GStreamerBackend`. In successive tutorials it will get expanded, but,
+for now, it only contains a method to retrieve the GStreamer version.
+
+The `GStreamerBackend` is made in Objective-C so it can take care of the
+few C-to-Objective-C conversions that might be necessary (like `char
+*` to `NSString *`, for example). This eases the usage of this class by
+the UI code, which is typically made in pure Objective-C.
+`GStreamerBackend` serves exactly the same purpose as the JNI code in
+the [](sdk-android-tutorials.md).
+
+**GStreamerBackend.m**
+
+```
+#import "GStreamerBackend.h"
+
+#include <gst/gst.h>
+
+@implementation GStreamerBackend
+
+-(NSString*) getGStreamerVersion
+{
+ char *version_utf8 = gst_version_string();
+ NSString *version_string = [NSString stringWithUTF8String:version_utf8];
+ g_free(version_utf8);
+ return version_string;
+}
+
+@end
+```
+
+The `getGStreamerVersion()` method simply calls
+`gst_version_string()` to obtain a string describing this version of
+GStreamer. This [Modified
+UTF8](http://en.wikipedia.org/wiki/UTF-8#Modified_UTF-8) string is then
+converted to a `NSString *` by ` NSString:stringWithUTF8String `and
+returned. Objective-C will take care of freeing the memory used by the
+new `NSString *`, but we need to free the `char *` returned
+by `gst_version_string()`.
+
+## The View Controller
+
+The view controller instantiates the GStremerBackend and asks it for the
+GStreamer version to display at the label. That's it!
+
+**ViewController.m**
+
+```
+#import "ViewController.h"
+#import "GStreamerBackend.h"
+
+@interface ViewController () {
+ GStreamerBackend *gst_backend;
+}
+
+@end
+
+@implementation ViewController
+
+@synthesize label;
+
+- (void)viewDidLoad
+{
+ [super viewDidLoad];
+ // Do any additional setup after loading the view, typically from a nib.
+ gst_backend = [[GStreamerBackend alloc] init];
+
+ label.text = [NSString stringWithFormat:@"Welcome to %@!", [gst_backend getGStreamerVersion]];
+}
+
+- (void)didReceiveMemoryWarning
+{
+ [super didReceiveMemoryWarning];
+ // Dispose of any resources that can be recreated.
+}
+
+@end
+```
+
+## Conclusion
+
+This ends the first iOS tutorial. It has shown that, due to the
+compatibility of C and Objective-C, adding GStreamer support to an iOS
+app is as easy as it is on a Desktop application. An extra Objective-C
+wrapper has been added (the `GStreamerBackend` class) for clarity, but
+calls to the GStreamer framework are valid from any part of the
+application code.
+
+The following tutorials detail the few places in which care has to be
+taken when developing specifically for the iOS platform.
+
+It has been a pleasure having you here, and see you soon!
+
+ [screenshot]: images/sdk-ios-tutorial-link-against-gstreamer-screenshot.png
\ No newline at end of file
--- /dev/null
+# iOS tutorial 3: Video
+
+## Goal
+
+![screenshot]
+
+Except for [](sdk-basic-tutorial-toolkit-integration.md),
+which embedded a video window on a GTK application, all tutorials so far
+relied on GStreamer video sinks to create a window to display their
+contents. The video sink on iOS is not capable of creating its own
+window, so a drawing surface always needs to be provided. This tutorial
+shows:
+
+ - How to allocate a drawing surface on the Xcode Interface Builder and
+ pass it to GStreamer
+
+## Introduction
+
+Since iOS does not provide a windowing system, a GStreamer video sink
+cannot create pop-up windows as it would do on a Desktop platform.
+Fortunately, the `VideoOverlay` interface allows providing video sinks with
+an already created window onto which they can draw, as we have seen
+in [](sdk-basic-tutorial-toolkit-integration.md).
+
+In this tutorial, a `UIView` widget (actually, a subclass of it) is
+placed on the main storyboard. In the `viewDidLoad` method of the
+`ViewController`, we pass a pointer to this `UIView `to the instance of
+the `GStreamerBackend`, so it can tell the video sink where to draw.
+
+## The User Interface
+
+The storyboard from the previous tutorial is expanded: A `UIView `is
+added over the toolbar and pinned to all sides so it takes up all
+available space (`video_container_view` outlet). Inside it, another
+`UIView `is added (`video_view` outlet) which contains the actual video,
+centered to its parent, and with a size that adapts to the media size
+(through the `video_width_constraint` and `video_height_constraint`
+outlets):
+
+**ViewController.h**
+
+```
+#import <UIKit/UIKit.h>
+#import "GStreamerBackendDelegate.h"
+
+@interface ViewController : UIViewController <GStreamerBackendDelegate> {
+ IBOutlet UILabel *message_label;
+ IBOutlet UIBarButtonItem *play_button;
+ IBOutlet UIBarButtonItem *pause_button;
+ IBOutlet UIView *video_view;
+ IBOutlet UIView *video_container_view;
+ IBOutlet NSLayoutConstraint *video_width_constraint;
+ IBOutlet NSLayoutConstraint *video_height_constraint;
+}
+
+-(IBAction) play:(id)sender;
+-(IBAction) pause:(id)sender;
+
+/* From GStreamerBackendDelegate */
+-(void) gstreamerInitialized;
+-(void) gstreamerSetUIMessage:(NSString *)message;
+
+@end
+```
+
+## The View Controller
+
+The `ViewController `class manages the UI, instantiates
+the `GStreamerBackend` and also performs some UI-related tasks on its
+behalf:
+
+**ViewController.m**
+
+```
+#import "ViewController.h"
+#import "GStreamerBackend.h"
+#import <UIKit/UIKit.h>
+
+@interface ViewController () {
+ GStreamerBackend *gst_backend;
+ int media_width;
+ int media_height;
+}
+
+@end
+
+@implementation ViewController
+
+/*
+ * Methods from UIViewController
+ */
+
+- (void)viewDidLoad
+{
+ [super viewDidLoad];
+
+ play_button.enabled = FALSE;
+ pause_button.enabled = FALSE;
+
+ /* Make these constant for now, later tutorials will change them */
+ media_width = 320;
+ media_height = 240;
+
+ gst_backend = [[GStreamerBackend alloc] init:self videoView:video_view];
+}
+
+- (void)didReceiveMemoryWarning
+{
+ [super didReceiveMemoryWarning];
+ // Dispose of any resources that can be recreated.
+}
+
+/* Called when the Play button is pressed */
+-(IBAction) play:(id)sender
+{
+ [gst_backend play];
+}
+
+/* Called when the Pause button is pressed */
+-(IBAction) pause:(id)sender
+{
+ [gst_backend pause];
+}
+
+- (void)viewDidLayoutSubviews
+{
+ CGFloat view_width = video_container_view.bounds.size.width;
+ CGFloat view_height = video_container_view.bounds.size.height;
+
+ CGFloat correct_height = view_width * media_height / media_width;
+ CGFloat correct_width = view_height * media_width / media_height;
+
+ if (correct_height < view_height) {
+ video_height_constraint.constant = correct_height;
+ video_width_constraint.constant = view_width;
+ } else {
+ video_width_constraint.constant = correct_width;
+ video_height_constraint.constant = view_height;
+ }
+}
+
+/*
+ * Methods from GstreamerBackendDelegate
+ */
+
+-(void) gstreamerInitialized
+{
+ dispatch_async(dispatch_get_main_queue(), ^{
+ play_button.enabled = TRUE;
+ pause_button.enabled = TRUE;
+ message_label.text = @"Ready";
+ });
+}
+
+-(void) gstreamerSetUIMessage:(NSString *)message
+{
+ dispatch_async(dispatch_get_main_queue(), ^{
+ message_label.text = message;
+ });
+}
+
+@end
+```
+
+We expand the class to remember the width and height of the media we are
+currently playing:
+
+```
+@interface ViewController () {
+ GStreamerBackend *gst_backend;
+ int media_width;
+ int media_height;
+}
+```
+
+In later tutorials this data is retrieved from the GStreamer pipeline,
+but in this tutorial, for simplicity’s sake, the width and height of the
+media is constant and initialized in `viewDidLoad`:
+
+```
+- (void)viewDidLoad
+{
+ [super viewDidLoad];
+
+ play_button.enabled = FALSE;
+ pause_button.enabled = FALSE;
+
+ /* Make these constant for now, later tutorials will change them */
+ media_width = 320;
+ media_height = 240;
+
+ gst_backend = [[GStreamerBackend alloc] init:self videoView:video_view];
+}
+```
+
+As shown below, the `GStreamerBackend` constructor has also been
+expanded to accept another parameter: the `UIView *` where the video
+sink should draw.
+
+The rest of the `ViewController `code is the same as the previous
+tutorial, except for the code that adapts the `video_view` size to the
+media size, respecting its aspect ratio:
+
+```
+- (void)viewDidLayoutSubviews
+{
+ CGFloat view_width = video_container_view.bounds.size.width;
+ CGFloat view_height = video_container_view.bounds.size.height;
+
+ CGFloat correct_height = view_width * media_height / media_width;
+ CGFloat correct_width = view_height * media_width / media_height;
+
+ if (correct_height < view_height) {
+ video_height_constraint.constant = correct_height;
+ video_width_constraint.constant = view_width;
+ } else {
+ video_width_constraint.constant = correct_width;
+ video_height_constraint.constant = view_height;
+ }
+}
+```
+
+The `viewDidLayoutSubviews` method is called every time the main view
+size has changed (for example, due to a device orientation change) and
+the entire layout has been recalculated. At this point, we can access
+the `bounds` property of the `video_container_view` to retrieve its new
+size and change the `video_view` size accordingly.
+
+The simple algorithm above maximizes either the width or the height of
+the `video_view`, while changing the other axis so the aspect ratio of
+the media is preserved. The goal is to provide the GStreamer video sink
+with a surface of the correct proportions, so it does not need to add
+black borders (*letterboxing*), which is a waste of processing power.
+
+The final size is reported to the layout engine by changing the
+`constant` field in the width and height `Constraints` of the
+`video_view`. These constraints have been created in the storyboard and
+are accessible to the `ViewController `through IBOutlets, as is usually
+done with other widgets.
+
+## The GStreamer Backend
+
+The `GStreamerBackend` class performs all GStreamer-related tasks and
+offers a simplified interface to the application, which does not need to
+deal with all the GStreamer details. When it needs to perform any UI
+action, it does so through a delegate, which is expected to adhere to
+the `GStreamerBackendDelegate` protocol:
+
+**GStreamerBackend.m**
+
+```
+#import "GStreamerBackend.h"
+
+#include <gst/gst.h>
+#include <gst/video/video.h>
+
+GST_DEBUG_CATEGORY_STATIC (debug_category);
+#define GST_CAT_DEFAULT debug_category
+
+@interface GStreamerBackend()
+-(void)setUIMessage:(gchar*) message;
+-(void)app_function;
+-(void)check_initialization_complete;
+@end
+
+@implementation GStreamerBackend {
+ id ui_delegate; /* Class that we use to interact with the user interface */
+ GstElement *pipeline; /* The running pipeline */
+ GstElement *video_sink;/* The video sink element which receives VideoOverlay commands */
+ GMainContext *context; /* GLib context used to run the main loop */
+ GMainLoop *main_loop; /* GLib main loop */
+ gboolean initialized; /* To avoid informing the UI multiple times about the initialization */
+ UIView *ui_video_view; /* UIView that holds the video */
+}
+
+/*
+ * Interface methods
+ */
+
+-(id) init:(id) uiDelegate videoView:(UIView *)video_view
+{
+ if (self = [super init])
+ {
+ self->ui_delegate = uiDelegate;
+ self->ui_video_view = video_view;
+
+ GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-3", 0, "iOS tutorial 3");
+ gst_debug_set_threshold_for_name("tutorial-3", GST_LEVEL_DEBUG);
+
+ /* Start the bus monitoring task */
+ dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
+ [self app_function];
+ });
+ }
+
+ return self;
+}
+
+-(void) dealloc
+{
+ if (pipeline) {
+ GST_DEBUG("Setting the pipeline to NULL");
+ gst_element_set_state(pipeline, GST_STATE_NULL);
+ gst_object_unref(pipeline);
+ pipeline = NULL;
+ }
+}
+
+-(void) play
+{
+ if(gst_element_set_state(pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE) {
+ [self setUIMessage:"Failed to set pipeline to playing"];
+ }
+}
+
+-(void) pause
+{
+ if(gst_element_set_state(pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_FAILURE) {
+ [self setUIMessage:"Failed to set pipeline to paused"];
+ }
+}
+
+/*
+ * Private methods
+ */
+
+/* Change the message on the UI through the UI delegate */
+-(void)setUIMessage:(gchar*) message
+{
+ NSString *string = [NSString stringWithUTF8String:message];
+ if(ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerSetUIMessage:)])
+ {
+ [ui_delegate gstreamerSetUIMessage:string];
+ }
+}
+
+/* Retrieve errors from the bus and show them on the UI */
+static void error_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
+{
+ GError *err;
+ gchar *debug_info;
+ gchar *message_string;
+
+ gst_message_parse_error (msg, &err, &debug_info);
+ message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);
+ g_clear_error (&err);
+ g_free (debug_info);
+ [self setUIMessage:message_string];
+ g_free (message_string);
+ gst_element_set_state (self->pipeline, GST_STATE_NULL);
+}
+
+/* Notify UI about pipeline state changes */
+static void state_changed_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
+{
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ /* Only pay attention to messages coming from the pipeline, not its children */
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (self->pipeline)) {
+ gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));
+ [self setUIMessage:message];
+ g_free (message);
+ }
+}
+
+/* Check if all conditions are met to report GStreamer as initialized.
+ * These conditions will change depending on the application */
+-(void) check_initialization_complete
+{
+ if (!initialized && main_loop) {
+ GST_DEBUG ("Initialization complete, notifying application.");
+ if (ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerInitialized)])
+ {
+ [ui_delegate gstreamerInitialized];
+ }
+ initialized = TRUE;
+ }
+}
+
+/* Main method for the bus monitoring code */
+-(void) app_function
+{
+ GstBus *bus;
+ GSource *bus_source;
+ GError *error = NULL;
+
+ GST_DEBUG ("Creating pipeline");
+
+ /* Create our own GLib Main Context and make it the default one */
+ context = g_main_context_new ();
+ g_main_context_push_thread_default(context);
+
+ /* Build pipeline */
+ pipeline = gst_parse_launch("videotestsrc ! warptv ! videoconvert ! autovideosink", &error);
+ if (error) {
+ gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
+ g_clear_error (&error);
+ [self setUIMessage:message];
+ g_free (message);
+ return;
+ }
+
+ /* Set the pipeline to READY, so it can already accept a window handle */
+ gst_element_set_state(pipeline, GST_STATE_READY);
+
+ video_sink = gst_bin_get_by_interface(GST_BIN(pipeline), GST_TYPE_VIDEO_OVERLAY);
+ if (!video_sink) {
+ GST_ERROR ("Could not retrieve video sink");
+ return;
+ }
+ gst_video_overlay_set_window_handle(GST_VIDEO_OVERLAY(video_sink), (guintptr) (id) ui_video_view);
+
+ /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
+ bus = gst_element_get_bus (pipeline);
+ bus_source = gst_bus_create_watch (bus);
+ g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
+ g_source_attach (bus_source, context);
+ g_source_unref (bus_source);
+ g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, (__bridge void *)self);
+ g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, (__bridge void *)self);
+ gst_object_unref (bus);
+
+ /* Create a GLib Main Loop and set it to run */
+ GST_DEBUG ("Entering main loop...");
+ main_loop = g_main_loop_new (context, FALSE);
+ [self check_initialization_complete];
+ g_main_loop_run (main_loop);
+ GST_DEBUG ("Exited main loop");
+ g_main_loop_unref (main_loop);
+ main_loop = NULL;
+
+ /* Free resources */
+ g_main_context_pop_thread_default(context);
+ g_main_context_unref (context);
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+
+ return;
+}
+
+@end
+```
+
+The main differences with the previous tutorial are related to the
+handling of the `VideoOverlay` interface:
+
+```
+@implementation GStreamerBackend {
+ id ui_delegate; /* Class that we use to interact with the user interface */
+ GstElement *pipeline; /* The running pipeline */
+ GstElement *video_sink;/* The video sink element which receives VideoOverlay commands */
+ GMainContext *context; /* GLib context used to run the main loop */
+ GMainLoop *main_loop; /* GLib main loop */
+ gboolean initialized; /* To avoid informing the UI multiple times about the initialization */
+ UIView *ui_video_view; /* UIView that holds the video */
+}
+```
+
+The class is expanded to keep track of the video sink element in the
+pipeline and the `UIView *` onto which rendering is to occur.
+
+```
+-(id) init:(id) uiDelegate videoView:(UIView *)video_view
+{
+ if (self = [super init])
+ {
+ self->ui_delegate = uiDelegate;
+ self->ui_video_view = video_view;
+
+ GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-3", 0, "iOS tutorial 3");
+ gst_debug_set_threshold_for_name("tutorial-3", GST_LEVEL_DEBUG);
+
+ /* Start the bus monitoring task */
+ dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
+ [self app_function];
+ });
+ }
+
+ return self;
+}
+```
+
+The constructor accepts the `UIView *` as a new parameter, which, at
+this point, is simply remembered in `ui_video_view`.
+
+```
+/* Build pipeline */
+pipeline = gst_parse_launch("videotestsrc ! warptv ! videoconvert ! autovideosink", &error);
+```
+
+Then, in the `app_function`, the pipeline is constructed. This time we
+build a video pipeline using a simple `videotestsrc` element with a
+`warptv` to add some spice. The video sink is `autovideosink`, which
+choses the appropriate sink for the platform (currently,
+`glimagesink` is the only option for
+iOS).
+
+```
+/* Set the pipeline to READY, so it can already accept a window handle */
+gst_element_set_state(pipeline, GST_STATE_READY);
+
+video_sink = gst_bin_get_by_interface(GST_BIN(pipeline), GST_TYPE_VIDEO_OVERLAY);
+if (!video_sink) {
+ GST_ERROR ("Could not retrieve video sink");
+ return;
+}
+gst_video_overlay_set_window_handle(GST_VIDEO_OVERLAY(video_sink), (guintptr) (id) ui_video_view);
+```
+
+Once the pipeline is built, we set it to READY. In this state, dataflow
+has not started yet, but the caps of adjacent elements have been
+verified to be compatible and their pads have been linked. Also, the
+`autovideosink` has already instantiated the actual video sink so we can
+ask for it immediately.
+
+The `gst_bin_get_by_interface()` method will examine the whole pipeline
+and return a pointer to an element which supports the requested
+interface. We are asking for the `VideoOverlay` interface, explained in
+[](sdk-basic-tutorial-toolkit-integration.md),
+which controls how to perform rendering into foreign (non-GStreamer)
+windows. The internal video sink instantiated by `autovideosink` is the
+only element in this pipeline implementing it, so it will be returned.
+
+Once we have the video sink, we inform it of the `UIView` to use for
+rendering, through the `gst_video_overlay_set_window_handle()` method.
+
+## EaglUIView
+
+One last detail remains. In order for `glimagesink` to be able to draw
+on the
+[`UIView`](http://developer.apple.com/library/ios/#documentation/UIKit/Reference/UIView_Class/UIView/UIView.html),
+the
+[`Layer`](http://developer.apple.com/library/ios/#documentation/GraphicsImaging/Reference/CALayer_class/Introduction/Introduction.html#//apple_ref/occ/cl/CALayer) associated
+with this view must be of the
+[`CAEAGLLayer`](http://developer.apple.com/library/ios/#documentation/QuartzCore/Reference/CAEAGLLayer_Class/CAEGLLayer/CAEGLLayer.html#//apple_ref/occ/cl/CAEAGLLayer) class.
+To this avail, we create the `EaglUIView` class, derived from
+`UIView `and overriding the `layerClass` method:
+
+**EaglUIView.m**
+
+```
+#import "EaglUIVIew.h"
+
+#import <QuartzCore/QuartzCore.h>
+
+@implementation EaglUIView
+
++ (Class) layerClass
+{
+ return [CAEAGLLayer class];
+}
+
+@end
+```
+
+When creating storyboards, bear in mind that the `UIView `which should
+contain the video must have `EaglUIView` as its custom class. This is
+easy to setup from the Xcode interface builder. Take a look at the
+tutorial storyboard to see how to achieve this.
+
+And this is it, using GStreamer to output video onto an iOS application
+is as simple as it seems.
+
+## Conclusion
+
+This tutorial has shown:
+
+ - How to display video on iOS using a `UIView `and
+ the `VideoOverlay` interface.
+ - How to report the media size to the iOS layout engine through
+ runtime manipulation of width and height constraints.
+
+The following tutorial plays an actual clip and adds a few more controls
+to this tutorial in order to build a simple media player.
+
+It has been a pleasure having you here, and see you soon!
+
+ [screenshot]: images/sdk-ios-tutorial-video-screenshot.png
\ No newline at end of file
--- /dev/null
+# iOS tutorials
+
+## Welcome to the GStreamer iOS tutorials
+
+These tutorials describe iOS-specific topics. General GStreamer
+concepts will not be explained in these tutorials, so the
+[](sdk-basic-tutorials.md) should be reviewed first. The reader should
+also be familiar with basic iOS programming techniques.
+
+The iOS tutorials have the same structure as the
+[](sdk-android-tutorials.md): Each one builds on top of the previous
+one and adds progressively more functionality, until a working media
+player application is obtained in
+[](sdk-ios-tutorial-a-complete-media-player.md).
+
+Make sure to have read the instructions in
+[](sdk-installing-for-ios-development.md) before jumping into the iOS
+tutorials.
+
+All iOS tutorials are split into the following classes:
+
+ - The `GStreamerBackend` class performs all GStreamer-related tasks
+ and offers a simplified interface to the application, which does not
+ need to deal with all the GStreamer details. When it needs to
+ perform any UI action, it does so through a delegate, which is
+ expected to adhere to the `GStreamerBackendDelegate` protocol.
+ - The `ViewController` class manages the UI, instantiates the
+ `GStreamerBackend` and also performs some UI-related tasks on its
+ behalf.
+ - The `GStreamerBackendDelegate` protocol defines which methods a
+ class can implement in order to serve as a UI delegate for the
+ `GStreamerBackend`.
--- /dev/null
+---
+short-description: Patents, Licenses and legal F.A.Q.
+...
+
+# Legal information
+
+## Installer, default installation
+
+The installer (Microsoft Windows and MacOSX) and the default
+installation (GNU/Linux) contain and install the minimal default
+installation. At install time or later, the downloading of optional
+components is also possible, but read on for certain legal cautions you
+might want to take. All downloads are from the
+[gstreamer.freedesktop.org](http://gstreamer.freedesktop.org) website.
+
+## Licensing of GStreamer
+
+GStreamer minimal default installation only contains packages which
+are licensed under the [GNU LGPL license
+v2.1](http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html). This
+license gives you the Freedom to use, modify, make copies of the
+software either in the original or in a modified form, provided that the
+software you redistribute is licensed under the same licensing terms.
+This only extends to the software itself and modified versions of it,
+but you are free to link the LGPL software as a library used by other
+software under whichever license. In other words, it is a weak copyleft
+license.
+
+Therefore, it is possible to use GStreamer to build applications that are
+then distributed under a different license, including a proprietary one,
+provided that reverse engineering is not prohibited for debugging
+modifications purposes. Only the pieces of GStreamer that are under the
+LGPL need to be kept under the LGPL, and the corresponding source code
+must be distributed along with the application (or an irrevocable offer
+to do so for at least three years from distribution). Please consult
+section 6 of the
+[LGPL](http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html) for
+further details as to what the corresponding source code must contain.
+
+Some portions of the minimal default installation may be under
+different licenses, which are both more liberal than the LGPL (they are
+less strict conditions for granting the license) and compatible with the
+LGPL. This is advised locally.
+
+## Optional packages
+
+There are two types of optional packages (GPL and Patented), which are
+under a different license or have other issues concerning patentability
+(or both).
+
+#### GPL code
+
+Part of the optional packages are under the GNU GPL
+[v2](http://www.gnu.org/licenses/old-licenses/gpl-2.0.html) or
+[v3](http://www.gnu.org/licenses/gpl-3.0.html). This means that you
+cannot link the GPL software in a program unless the same program is
+also under the GPL, but you are invited to seek competent advice on how
+this works in your precise case and design choices. GPL is called
+“strong copyleft” because the condition to distributed under the same
+license has the largest possible scope and extends to all derivative
+works.
+
+#### Patents
+
+Certain software, and in particular software that implements
+multimedia standard formats such as MP3, MPEG 2 video and audio, h.264,
+MPEG 4 audio and video, AC3, etc, can have patent issues. In certain
+countries patents are granted on software and even software-only
+solution are by and large considered patentable and are patented (such
+as in the United States). In certain others, patents on pure software
+solutions are formally prohibited, but granted (this is the case in many
+European countries), and in others again are neither allowed nor granted.
+
+It is up to you to make sure that in the countries where GStreamer is
+used, products are made using it and product are distributed, a license
+from the applicable patent holders is required or not. Receiving GStreamer
+– or links to other downloadable software – does not provide any license
+expressed or implied over these patents, except in very limited
+conditions where the license so provides. No representation is made.
+
+In certain cases, the optional packages are distributed only as source
+code. It is up to the receiver to make sure that in the applicable
+circumstances compiling the same code for a given platform or
+distributing the object code is not an act that infringes one or more
+patents.
+
+## Software is as-is
+
+All software and the entire GStreamer binaries areprovided as-is, without any
+warranty whatsoever. The individual licenses have particular language
+disclaiming liability: we invite you to read all of them. Should you
+need a warranty on the fact that software works as intended or have any
+kind of indemnification, you have the option to subscribe a software
+maintenance agreement with a company or entity that is in that business.
+Fluendo and Collabora, as well as some other companies, provide software
+maintenance agreements under certain conditions, you are invited to
+contact them in order to receive further details and discuss of the
+commercial terms.
+
+## Data protection
+
+This website might use cookies and HTTP logs for statistical analysis
+and on an aggregate basis only.
+
+## Frequently Asked Questions
+
+##### What licenses are there?
+
+GStreamer binaries containst software under various licenses. See above.
+
+##### How does this relate to the packaging system?
+
+The packaging is only a more convenient way to install software and
+decide what's good for you. GStreamer is meant to be modular, making use
+of different modules, or plugins, that perform different activities.
+
+We provide some of them by default. Others are provided as an additional
+download, should you elect to do so. You could do the same by finding
+and downloading the same packages for your own platform. So it is
+entirely up to you to decide what to do.
+
+Also, we note that GStreamer elements are divided into different packages,
+roughly following the licensing conditions attached to the same. For
+instance, the codecs-gpl package contains GPL licensed codecs. All the
+packages installed by default, conversely, are licensed under the LGPL
+or a more liberal license. This division is provided only for ease of
+reference, but we cannot guarantee that our selection is 100% correct,
+so it is up to the user to verify the actual licensing conditions before
+distributing works that utilize GStreamer.
+
+##### Can I / must I distribute GStreamer along with my application?
+
+You surely can. All software is Free/Open Source software, and can be
+distributed freely. You are not **required** to distribute it. Only,
+be reminded that one of the conditions for you to use software under
+certain licenses to make a work containing such software, is that you
+also distribute the complete source code of the original code (or of
+the modified code, if you have modified it). There are alternative
+ways to comply with this obligation, some of them do not require any
+actual distribution of source code, but since GStreamer contains the
+entire source code, you might want to include it (or the directories
+containing the source code) with your application as a safe way to
+comply with this requirement of the license.
+
+##### What happens when I modify the GStreamer's source code?
+
+You are invited to do so, as the licenses (unless you are dealing with
+proprietary bits, but in that case you will not find the corresponding
+source code) so permit. Be reminded though that in that case you need
+to also provide the complete corresponding source code (and to
+preserve the same license, of course). You might also consider to push
+your modifications upstream, so that they are merged into the main
+branch of development if they are worth it and will be maintained by
+the GStreamer project and not by you individually. We invite you not
+to fork the code, if at all possible. he Cerbero build system has a
+"bundle-source" command that can help you create a source bundle
+containing all of the complete corresponding machine readable source
+code that you are required to provide.
+
+##### How does licensing relate to software patents? What about software patents in general?
+
+This is a tricky question. We believe software patents should not exist,
+so that by distributing and using software on a general purpose machine
+you would not violate any of them. But the inconvenient truth is that
+they do exist.
+
+Software patents are widely available in the USA. Despite they are
+formally prohibited in the European Union, they indeed are granted by
+the thousand by the European Patent Office, and also some national
+patent offices follow the same path. In other countries they are not
+available.
+
+Since patent protection is a national state-granted monopoly,
+distributing software that violates patents in a given country could be
+entirely safe if done in another country. Fair use exceptions also
+exist. So we cannot advice you whether the software we provide would be
+considered violating patents in your country or in any other country,
+but that can be said for virtually all kinds of sofware. Only, since we
+deal with audio-video standards, and these standards are by and large
+designed to use certain patented technologies, it is common wisdom that
+the pieces of software that implement these standards are sensitive in
+this respect.
+
+This is why GStreamer has taken a modular approach, so that you can use
+a Free plugins or a proprietary, patent royalty bearing, plugin for a
+given standard.
+
+##### What about static vs. dynamic linking and copyleft?
+
+We cannot provide one single answer to that question. Since copyright in
+software works as copyright in literature, static linking means
+basically that the programmer includes bits of code of the original
+library in the bytecode at compile time. This amounts to make derivative
+code of the library without conceivable exceptions, so you need a
+permission from the copyright holders of the library to do this.
+
+A widespread line of thinking says that dynamic linking is conversely
+not relevant to the copyleft effect, since the mingling of code in a
+larger work is done at runtime. However, another equally authoritative
+line of thought says that only certain type of dynamic linking is not
+copyright relevant. Therefore, using a library that is specifically
+designed to be loaded into a particular kind of software, even through
+API, requires permission by the copyright holder of the library when
+the two pieces are distributed together.
+
+In all cases, since most of the software we include in GStreamer is under
+the LGPL, this permission is granted once for all, subject to compliance
+with the conditions set out by it. Therefore, the problem only arises
+when you want to use GPL libraries to make non-GPL applications, and you
+need to audit your software in that case to make sure that what you do
+is not an infringement. This is why we have put the GPL libraries in a
+separate set of optional components, so you have a clearer view of what
+is safely clear for use, and what might need better investigation on a
+case-by-case basis.
+
+Please be reminded that even for LGPL, the recipient of the software
+must be in a position to replace the current library with a modified
+one, and to that effect some conditions apply, among which that for
+static linking you must also provide the complete toolchain required to
+relink the library (“any data and utility programs needed for
+reproducing the executable from it”, except the “major components”) and
+that the license of the conditions of the resulting program must allow
+decompilation to debug modifications to the library.
--- /dev/null
+# Mac OS X deployment
+
+This page explains how to deploy GStreamer along your application. There
+are different mechanisms, which have been reviewed in [](sdk-deploying-your-application.md). The details for some
+of the mechanisms are given here, and more options might be added to
+this documentation in the future.
+
+**FIXME: PackageMaker is dead we need a new solution **
+
+## Shared GStreamer
+
+This is the easiest way to deploy GStreamer, although most of the time
+it installs unnecessary files which grow the size of the installer and
+the target drive free space requirements. Since GStreamer might be shared
+among all applications that use it, though, the extra space requirements
+are somewhat blurred.
+
+With PackageMaker, simply add GStreamer **runtime ** disk image
+ ([the same one you used to install the runtime in your development
+machine](sdk-installing-on-mac-osx.md)) inside your installer
+package and create a post-install script that mounts the disk image and
+installs GStreamer package. You can use the following example, where you
+should replace `$INSTALL_PATH` with the path where your installer copied
+GStreamer's disk image files (the `/tmp` directory is good place to
+install it as it will be removed at the end of the installation):
+
+``` bash
+hdiutil attach $INSTALL_PATH/gstreamer-1.0-1.8.1-x86_64-packages.dmg
+cd /Volumes/gstreamer-1.0-1.8.1-x86_64/
+installer -pkg gstreamer-1.0-1.8.1-x86_64.pkg -target "/"
+hdiutil detach /Volumes/gstreamer-1.0-1.8.1-x86_64/
+rm $INSTALL_PATH/gstreamer-1.0-1.8.1-x86_64-packages.dmg
+```
+
+## Private deployment of GStreamer
+
+You can decide to distribute a private copy of GStreamer with your
+application, although it's not the recommended method. In this case,
+simply copy the framework to the application's Frameworks folder as
+defined in the [bundle programming
+guide](https://developer.apple.com/library/mac/documentation/CoreFoundation/Conceptual/CFBundles/BundleTypes/BundleTypes.html#//apple_ref/doc/uid/10000123i-CH101-SW19):
+
+``` bash
+cp -r /Library/Frameworks/GStreamer.framework ~/MyApp.app/Contents/Frameworks
+```
+
+Note that you can have several versions of GStreamer, and targeting
+different architectures, installed in the system. Make sure you only
+copy the version you need and that you update accordingly the link
+`GStreamer.framework/Version/Current`:
+
+``` bash
+$ ls -l Frameworks/GStreamer.framework/Version/Current
+lrwxr-xr-x 1 fluendo staff 21 Jun 5 18:46 Frameworks/GStreamer.framework/Versions/Current -> ../Versions/0.10/x86
+```
+
+Since GStreamer will be relocated, you will need to follow the
+instructions on how to relocate GStreamer at the end of this page.
+
+## Deploy only necessary files, by manually picking them
+
+On the other side of the spectrum, if you want to reduce the space
+requirements (and installer size) to the maximum, you can manually
+choose which GStreamer libraries to deploy. Unfortunately, you are on
+your own on this road, besides using the object file displaying tool:
+[otool](https://developer.apple.com/library/mac/#documentation/darwin/reference/manpages/man1/otool.1.html).
+Being a similar technique to deploying a private copy of GStreamer, keep
+in mind that you should relocate GStreamer too, as explained at the end of
+this page.
+
+Bear also in mind that GStreamer is modular in nature. Plug-ins are
+loaded depending on the media that is being played, so, if you do not
+know in advance what files you are going to play, you do not know which
+plugins and shared libraries you need to deploy.
+
+## Deploy only necessary packages, using the provided ones
+
+This will produce a smaller installer than deploying complete
+GStreamer, without the added burden of having to manually pick each
+library. You just need to know which packages your application requires.
+
+| Package name | Dependencies | Licenses | Description |
+|--------------|--------------|----------|-------------|
+| base-system-1.0 | |JPEG, FreeType, BSD-like, LGPL, LGPL-2+, LGPL-2.1, LibPNG and MIT | Base system dependencies |
+| gstreamer-1.0-capture | gstreamer-1.0-core, gstreamer-1.0-encoding | LGPL and LGPL-2+ | GStreamer plugins for capture |
+| gstreamer-1.0-codecs | base-crypto, gstreamer-1.0-core | BSD, Jasper-2.0, BSD-like, LGPL, LGPL-2, LGPL-2+, LGPL-2.1 and LGPL-2.1+ | GStreamer codecs |
+| gstreamer-1.0-codecs-gpl | gstreamer-1.0-core | BSD-like, LGPL, LGPL-2+ and LGPL-2.1+ | GStreamer codecs under the GPL license and/or with patents issues |
+| gstreamer-1.0-core | base-system-1.0 | LGPL and LGPL-2+ | GStreamer core |
+| gstreamer-1.0-dvd | gstreamer-1.0-core | GPL-2+, LGPL and LGPL-2+ | GStreamer DVD support |
+| gstreamer-1.0-effects | gstreamer-1.0-core | LGPL and LGPL-2+ | GStreamer effects and instrumentation plugins |
+| gstreamer-1.0-net | base-crypto, gstreamer-1.0-core | GPL-3, LGPL, LGPL-2+, LGPL-2.1+ and LGPL-3+ | GStreamer plugins for network protocols |
+| gstreamer-1.0-playback | gstreamer-1.0-core | LGPL and LGPL-2+ | GStreamer plugins for playback |
+| gstreamer-1.0-system | gstreamer-1.0-core | LGPL, LGPL-2+ and LGPL-2.1+ | GStreamer system plugins |
+| gstreamer-1.0-visualizers | gstreamer-1.0-core | LGPL and LGPL-2+ | GStreamer visualization plugins |
+| gstreamer-1.0-encoding | gstreamer-1.0-core, gstreamer-1.0-playback | LGPL and LGPL2+ | GStreamer plugins for encoding |
+| gstreamer-1.0-editing | gstreamer-1.0-core, gstreamer-1.0-devtools | LGPL and LGPL2+ | GStreamer libraries and plugins for non linear editing |
+| gstreamer-1.0-devtools | gstreamer-1.0-core | LGPL and LGPL2+ | GStreamer developers tools |
+| gstreamer-1.0-libav | gstreamer-1.0-core | LGPL and LGPL2+ | GStreamer plugins wrapping ffmpeg |
+| gstreamer-1.0-net-restricted | base-crypto, gstreamer-1.0-core | LGPL and LGPL2+ | GStreamer plugins for network protocols with potential patent issues in some countries |
+| gstreamer-1.0-codecs-restricted | gstreamer-1.0-core | LGPL and LGPL2+ | GStreamer restricted codecs with potential patent issues in some countries |
+| base-crypto | base-system-1.0 | LGPL and LGPL2+ | Cryptographic libraries |
+
+
+
+## Relocation of GStreamer in OS X
+
+In some situations we might need to relocate GStreamer, moving it to a
+different place in the file system, like for instance when we are
+shipping a private copy of GStreamer with our application.
+
+### Location of dependent dynamic libraries.
+
+On Darwin operating systems, the dynamic linker doesn't locate dependent
+dynamic libraries using their leaf name, but instead it uses full paths,
+which makes it harder to relocate them as explained in the DYNAMIC
+LIBRARY LOADING section of
+[dyld](https://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man1/dyld.1.html)'s
+man page:
+
+> Unlike many other operating systems, Darwin does not locate dependent
+> dynamic libraries via their leaf file name. Instead the full path to
+> each dylib is used (e.g. /usr/lib/libSystem.B.dylib). But there are
+> times when a full path is not appropriate; for instance, may want your
+> binaries to be installable in anywhere on the disk.
+
+We can get the list of paths used by an object file to locate its
+dependent dynamic libraries
+using [otool](https://developer.apple.com/library/mac/#documentation/darwin/reference/manpages/man1/otool.1.html):
+
+``` bash
+$ otool -L /Library/Frameworks/GStreamer.framework/Commands/gst-launch-1.0
+/Library/Frameworks/GStreamer.framework/Commands/gst-launch-1.0:
+ /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 550.43.0)
+ /Library/Frameworks/GStreamer.framework/Versions/0.10/x86/lib/libgstreamer-1.0.0.dylib (compatibility version 31.0.0, current version 31.0.0)
+ /Library/Frameworks/GStreamer.framework/Versions/0.10/x86/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.8.0)
+...
+```
+
+As you might have already noticed, if we move GStreamer to a different
+folder, it will stop working because the runtime linker won't be able to
+find `gstreamer-1.0` in the previous location
+`/Library/Frameworks/GStreamer.framework/Versions/0.10/x86/lib/libgstreamer-1.0.0.dylib`.
+
+This full path is extracted from the dynamic library ***install name***
+, a path that is used by the linker to determine its location. The
+install name of a library can be retrieved with
+[otool](https://developer.apple.com/library/mac/#documentation/darwin/reference/manpages/man1/otool.1.html) too:
+
+``` bash
+$ otool -D /Library/Frameworks/GStreamer.framework/Libraries/libgstreamer-1.0.dylib
+/Library/Frameworks/GStreamer.framework/Libraries/libgstreamer-1.0.dylib:
+/Library/Frameworks/GStreamer.framework/Versions/0.10/x86/lib/libgstreamer-1.0.0.dylib
+```
+
+Any object file that links to the dynamic library `gstreamer-1.0` will
+use the
+path `/Library/Frameworks/GStreamer.framework/Versions/0.10/x86/lib/libgstreamer-1.0.0.dylib` to
+locate it, as we saw previously with `gst-launch-1.0`.
+
+Since working exclusively with full paths wouldn't let us install our
+binaries anywhere in the path, the linker provides a mechanism of string
+substitution, adding three variables that can be used as a path prefix.
+At runtime the linker will replace them with the generated path for the
+prefix. These variables are `@executable_path`,
+`@loader_path` and `@rpath`, described in depth in the DYNAMIC LIBRARY
+LOADING section
+of [dyld](https://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man1/dyld.1.html)'s
+man page.
+
+For our purpose we will use the `@executable_path` variable, which is
+replaced with a fixed path, the path to the directory containing the
+main executable: `/Applications/MyApp.app/Contents/MacOS`.
+The `@loader_path` variable can't be used in our scope, because it will
+be replaced with the path to the directory containing the mach-o binary
+that loaded the dynamic library, which can vary.
+
+Therefore, in order to relocate GStreamer we will need to replace all
+paths
+containing `/Library/Frameworks/GStreamer.framework/` with `@executable_path/../Frameworks/GStreamer.framework/`, which
+can be done using
+the [install\_name\_tool](http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man1/install_name_tool.1.html)
+utility
+
+### Relocation of the binaries
+
+As mentioned in the previous section, we can use
+the `install_name_tool` in combination with `otool` to list all paths
+for dependant dynamic libraries and modify them to use the new location.
+However GStreamer has a huge list of binaries and doing it manually would
+be a painful task. That's why a simple relocation script is provided
+which you can find in cerbero's repository
+(`cerbero/tools/osxrelocator.py`). This scripts takes 3 parameters:
+
+1. `directory`: the directory to parse looking for binaries
+2. `old_prefix`: the old prefix we want to change (eg:
+ `/Library/Frameworks/GStreamer.framework`)
+3. `new_prefix`: the new prefix we want to use
+ (eg: `@executable_path/../Frameworks/GStreamer.framework/`)
+
+When looking for binaries to fix, we will run the script in the
+following
+directories:
+
+``` bash
+$ osxrelocator.py MyApp.app/Contents/Frameworks/GStreamer.framework/Versions/Current/lib /Library/Frameworks/GStreamer.framework/ @executable_path/../Frameworks/GStreamer.framework/ -r
+$ osxrelocator.py MyApp.app/Contents/Frameworks/GStreamer.framework/Versions/Current/libexec /Library/Frameworks/GStreamer.framework/ @executable_path/../Frameworks/GStreamer.framework/ -r
+$ osxrelocator.py MyApp.app/Contents/Frameworks/GStreamer.framework/Versions/Current/bin /Library/Frameworks/GStreamer.framework/ @executable_path/../Frameworks/GStreamer.framework/ -r
+$ osxrelocator.py MyApp.app/Contents/MacOS /Library/Frameworks/GStreamer.framework/ @executable_path/../Frameworks/GStreamer.framework/ -r
+```
+
+### Adjusting environment variables with the new paths
+
+The application also needs to set the following environment variables to
+help other libraries finding resources in the new
+ path:
+
+ - `GST_PLUGIN_SYSTEM_PATH=/Applications/MyApp.app/Contents/Frameworks/GStreamer.framework/Versions/Current/lib/gstreamer-1.0`
+ - `GST_PLUGIN_SCANNER=/Applications/MyApp.app/Contents/Frameworks/GStreamer.framework/Versions/Current/libexec/gstreamer-1.0/gst-plugin-scanner`
+ - `GTK_PATH=/Applications/MyApp.app/Contents/Frameworks/GStreamer.framework/Versions/Current/`
+ - `GIO_EXTRA_MODULES=/Applications/MyApp.app/Contents/Frameworks/GStreamer.framework/Versions/Current/lib/gio/modules`
+
+You can use the following functions:
+
+ - C: [putenv("VAR=/foo/bar")](http://linux.die.net/man/3/putenv)
+
+ - Python: [os.environ\['VAR'\] =
+ '/foo/var'](http://docs.python.org/library/os.html)
--- /dev/null
+# Multiplatform deployment using Cerbero
+
+Cerbero is the build and packaging system used to construct
+GStreamer. It uses “recipe” files that indicate how to build
+particular projects, and on what other projects they depend.
+Moreover, the built projects can be combined into packages for
+distribution. These packages are, depending on the target platform,
+Windows or OS X installers or Linux packages.
+
+To use Cerbero to build and package your application, you just need to
+add a recipe explaining how to build you application and make it depend
+on the `gstreamer-sdk` project. Then Cerbero can take care of building
+your application and its dependencies and package them all together.
+
+Read [](sdk-building-from-source-using-cerbero.md) to learn how
+to install and use Cerbero.
+
+At this point, after reading the Build from source section in
+[](sdk-building-from-source-using-cerbero.md), you should be able to
+build GStreamer from source and are ready to create recipe and package
+files for your application.
+
+In the Cerbero installation directory you will find the
+`cerbero-uninstalled` script. Execute it without parameters to see the
+list of commands it accepts:
+
+``` bash
+./cerbero-uninstalled
+```
+
+## Adding a recipe for your application
+
+The first step is to create an empty recipe that you can then tailor to
+your needs:
+
+``` bash
+./cerbero-uninstalled add-recipe my-app 1.0
+```
+
+This will create an initial recipe file in `recipes/my-app.recipe`,
+which contains the smallest necessary recipe. This file is a Python
+script; set the following attributes to describe your application:
+
+| Attribute Name | Description | Required | Example |
+|----------------|-------------|----------|---------|
+| `name` | The recipe name. | Yes | *name = 'my-app'* |
+| `version` | The software version. | Yes | *version = '1.0'* |
+| `licenses` | A list of licenses of the software (see `cerbero/enums.py:License` for allowed licenses). | Yes | *licenses = \[License.LGPLv2Plus\]* |
+| `deps` | A list of build dependencies of the software as recipe names. | No | *deps = \['other', 'recipe', 'names'\]* |
+| `platform_deps` | Platform specific build dependencies (see `cerbero/enums.py:Platform` for allowed platforms). | No | *platform\_deps = {Platform.LINUX: \['some-recipe'\], Platform.WINDOWS: \['another-recipe'\]}* |
+| `remotes` | A dictionary specifying the git remote urls where sources are pulled from. | No | *remotes = {'origin': '<git://somewhere>'}* |
+| `commit` | The git commit, tag or branch to use, defaulting to "sdk-*`version`*"*.* | No | *commit = 'my-app-branch'* |
+| `config_sh` | Used to select the configuration script. | No | *config\_sh = 'autoreconf -fiv && sh ./configure'* |
+| `configure_options` | Additional options that should be passed to the `configure` script. | No | *configure\_options = '--enable-something'* |
+| `use_system_libs` | Whether to use system provided libs. | No | *use\_system\_libs = True* |
+| `btype` | The build type (see `cerbero/build/build.py:BuildType` for allowed build types). | No | *btype = BuildType.CUSTOM* |
+| `stype` | The source type (see `cerbero/build/source.py:SourceType` for allowed source types). | No | *stype = SourceType.CUSTOM* |
+| `files_category` | A list of files that should be shipped with packages including this recipe *category*. See below for more details. Cerbero comes with some predefined categories that should be used if the files being installed match a category criteria. The predefined categories are: `libs` (for libraries), `bins` (for binaries), `devel` (for development files - header, pkgconfig files, etc), `python` (for python files) and `lang` (for language files). *Note that for the `bins` and `libs` categories there is no need to specify the files extensions as Cerbero will do it for you.* | Yes\* | *files\_bins = \['some-binary'\]* *files\_libs = \['libsomelib'\]* *files\_devel = \['include/something'\] files\_python = \['site-packages/some/pythonfile%(pext)s'\]* *files\_lang = \['foo'\]* |
+| `platform_files_category` | Same as *`files_category`* but for platform specific files. | No | *platform\_files\_some\_category = {Platform.LINUX: \['/some/file'\]}* |
+
+> ![warning] At least one “files” category should be set.
+
+Apart from the attributes listed above, it is also possible to override
+some Recipe methods. For example the `prepare` method can be overridden
+to do anything before the software is built, or the `install` and
+`post_install` methods for overriding what should be done during or
+after installation. Take a look at the existing recipes in
+`cerbero/recipes` for example.
+
+Alternatively, you can pass some options to cerbero-uninstalled so some
+of these attributes are already set for you. For
+example:
+
+```
+./cerbero-uninstalled add-recipe --licenses "LGPL" --deps "glib" --origin "git://git.my-app.com" --commit "git-commit-to-use" my-app 1.0
+```
+
+See `./cerbero-uninstalled add-recipe -h` for help.
+
+As an example, this is the recipe used to build the Pitivi video editor:
+
+```
+class Recipe(recipe.Recipe):
+ name = 'pitivi'
+ version = '0.95'
+ licenses = [License.GPLv2Plus]
+ remotes = {'origin': 'git://git.gnome.org/pitivi'}
+ config_sh = 'sh ./autogen.sh --noconfigure && ./configure'
+ configure_options = "--disable-help"
+ commit = 'origin/master'
+ deps = ['gst-editing-services-1.0',
+ 'gst-python-1.0',
+ 'gst-libav-1.0',
+ 'gst-plugins-bad-1.0',
+ 'gst-plugins-ugly-1.0',
+ 'gst-transcoder',
+ 'numpy',
+ 'matplotlib',
+ 'gnome-icon-theme',
+ 'gnome-icon-theme-symbolic',
+ 'shared-mime-info'] # brings in gtk+
+
+ files_libs = ['libpitivi-1.0']
+ files_typelibs = [
+ 'Pitivi-1.0',
+ ]
+ use_system_libs = True
+ files_bins = ['pitivi']
+ files_lang = ['pitivi']
+ files_pitivi = ['lib/pitivi/python/pitivi',
+ 'share/pitivi/',
+ 'share/applications/pitivi.desktop']
+```
+
+Cerbero gets the software sources to build from a GIT repository, which
+is specified via the `git_root` configuration variable from the Cerbero
+configuration file (see the "Build from software" section in [Installing
+on Linux](sdk-installing-on-linux.md)) and can be overridden by the
+`remotes` attribute inside the recipes (if setting the `origin` remote).
+In this case where no “commit” attribute is specified, Cerbero will use
+the commit named “sdk-0.2+git” from the GIT repository when building
+Snappy.
+
+Once the recipe is ready, instruct Cerbero to build it:
+
+``` bash
+./cerbero-uninstalled build my-app
+```
+
+## Adding a package for you software
+
+To distribute your software with GStreamer it is necessary to put it into
+a package or installer, depending on the target platform. This is done
+by selecting the files that should be included. To add a package you
+have to create a package file in `cerbero/packages`. The package files
+are Python scripts too and there are already many examples of package
+files in `cerbero/packages`.
+
+Now, to create an empty package, do:
+
+``` bash
+./cerbero-uninstalled add-package my-app 1.0
+```
+
+This will create an initial package file in `packages/my-app.package`.
+
+The following Package attributes are used to describe your package:
+
+| Attribute Name | Description | Required | Example |
+|----------------|-------------|----------|---------|
+| `name` | The package name. | Yes | *name = 'my-app'* |
+| `shortdesc` | A short description of the package. | No | *shortdesc = 'some-short-desc'* |
+| `longdesc` | A long description of the package. | No | *longdesc = 'Some Longer Description'* |
+| `codename` | The release codename. | No | *codename = 'MyAppReleaseName'* |
+| `vendor` | Vendor for this package.| No | *vendor = 'MyCompany'* |
+| `url` | The package url | No | *url = 'http://www.my-app.com'* |
+| `version` | The package version. | Yes | *version = '1.0'* |
+| `license` | The package license (see `cerbero/enums.py:License` for allowed licenses). | Yes | *license = License.LGPLv2Plus* |
+| `uuid` | The package unique id | Yes | *uuid = '6cd161c2-4535-411f-8287-e8f6a892f853'* |
+| `deps` | A list of package dependencies as package names. | No | *deps = \['other', 'package', 'names'\]* |
+| `sys_deps` | The system dependencies for this package. | No | *sys\_deps= {Distro.DEBIAN: \['python'\]}* |
+| `files` | A list of files included in the **runtime** package in the form *“recipe\_name:category1:category2:...”* *If the recipe category is omitted, all categories are included.* | Yes\* | *files = \['my-app'\]* *files = \['my-app:category1'\]* |
+| `files_devel` | A list of files included in the **devel** package in the form *“recipe\_name:category1:category2:...”* | Yes\* | *files\_devel = \['my-app:category\_devel'\]* |
+| `platform_files` | Same as *files* but allowing to specify different files for different platforms. | Yes\* | *platform\_files = {Platform.WINDOWS: \['my-app:windows\_only\_category'\]}* |
+| `platform_files_devel` | Same as *files\_devel* but allowing to specify different files for different platforms. | Yes\* | *platform\_files\_devel = {Platform.WINDOWS: \['my-app:windows\_only\_category\_devel'\]}* |
+
+> ![warning] At least one of the “files” attributes should be set.
+
+Alternatively you can also pass some options to `cerbero-uninstalled`,
+for
+example:
+
+``` bash
+./cerbero-uninstalled add-package my-app 1.0 --license "LGPL" --codename MyApp --vendor MyAppVendor --url "http://www.my-app.com" --files=my-app:bins:libs --files-devel=my-app:devel --platform-files=linux:my-app:linux_specific --platform-files-devel=linux:my-app:linux_specific_devel,windows:my-app:windows_specific_devel --deps base-system --includes gstreamer-core
+```
+
+See `./cerbero-uninstalled add-package -h` for help.
+
+As an example, this is the package file that is used for packaging the
+`gstreamer-core` package:
+
+```
+class Package(package.Package):
+
+ name = 'gstreamer-1.0-codecs'
+ shortdesc = 'GStreamer 1.0 codecs'
+ longdesc = 'GStreamer 1.0 codecs'
+ version = '1.9.0.1'
+ url = "http://gstreamer.freedesktop.org"
+ license = License.LGPL
+ vendor = 'GStreamer Project'
+ org = 'org.freedesktop.gstreamer'
+ uuid = 'a2e545d5-7819-4636-9e86-3660542f08e5'
+ deps = ['gstreamer-1.0-core', 'base-crypto']
+
+ files = ['flac:libs', 'libkate:libs', 'libdv:libs',
+ 'libogg:libs', 'schroedinger:libs', 'speex:libs',
+ 'libtheora:libs', 'wavpack:libs', 'libvpx:libs',
+ 'taglib:libs', 'opus:libs', 'libvorbis:libs',
+ 'openjpeg:libs', 'openh264:libs', 'spandsp:libs',
+ 'gst-plugins-base-1.0:plugins_codecs', 'gst-plugins-good-1.0:plugins_codecs',
+ 'gst-plugins-bad-1.0:plugins_codecs', 'gst-plugins-ugly-1.0:plugins_codecs',
+ ]
+ files_devel = ['gst-plugins-base-1.0-static:plugins_codecs_devel',
+ 'gst-plugins-good-1.0-static:plugins_codecs_devel',
+ 'gst-plugins-bad-1.0-static:plugins_codecs_devel',
+ 'gst-plugins-ugly-1.0-static:plugins_codecs_devel',
+ 'gst-plugins-bad-1.0-static:codecs_devel']
+ platform_files = {
+ Platform.ANDROID: ['tremor:libs'],
+ Platform.IOS: ['tremor:libs']
+ }
+```
+
+At this point you have two main options: you could either have a single
+package that contains everything your software needs, or depend on a
+shared version of GStreamer.
+
+### Having a private version of GStreamer
+
+To have a private version of GStreamer included in a single package you
+don't have to add the `deps` variable to the package file but instead
+list all files you need in the `files` variables. If you decide to go
+this road you must make sure that you use a different prefix than
+GStreamer in the Cerbero configuration file, otherwise your package
+will have file conflicts with GStreamer.
+
+### Having a shared version of GStreamer
+
+If you decide to use a shared version of GStreamer you can create a
+package file like the other package files in GStreamer. Just
+list all packages you need in the `deps` variable and put the files your
+software needs inside the `files` variables. When building a package
+this way you must make sure that you use the same prefix and
+packages\_prefix as the ones in your Cerbero configuration file.
+
+Finally, build your package by using:
+
+``` bash
+./cerbero-uninstalled package your-package
+```
+
+Where `your-package` is the name of the `.package` file that you created
+in the `packages` directory. This command will build your software and
+all its dependencies, and then make individual packages for them (both
+the dependencies and your software). The resulting files will be in the
+current working directory.
+
+
+ [warning]: images/icons/emoticons/warning.png
\ No newline at end of file
--- /dev/null
+# Playback tutorial 6: Audio visualization
+
+## Goal
+
+GStreamer comes with a set of elements that turn audio into video. They
+can be used for scientific visualization or to spice up your music
+player, for example. This tutorial shows:
+
+ - How to enable audio visualization
+ - How to select the visualization element
+
+## Introduction
+
+Enabling audio visualization in `playbin` is actually very easy. Just
+set the appropriate `playbin` flag and, when an audio-only stream is
+found, it will instantiate the necessary elements to create and display
+the visualization.
+
+If you want to specify the actual element that you want to use to
+generate the visualization, you instantiate it yourself and then tell
+`playbin` about it through the `vis-plugin` property.
+
+This tutorial searches the GStreamer registry for all the elements of
+the Visualization class, tries to select `goom` (or another one if it is
+not available) and passes it to `playbin`.
+
+## A fancy music player
+
+Copy this code into a text file named `playback-tutorial-6.c`.
+
+**playback-tutorial-6.c**
+
+``` c
+#include <gst/gst.h>
+
+/* playbin flags */
+typedef enum {
+ GST_PLAY_FLAG_VIS = (1 << 3) /* Enable rendering of visualizations when there is no video stream. */
+} GstPlayFlags;
+
+/* Return TRUE if this is a Visualization element */
+static gboolean filter_vis_features (GstPluginFeature *feature, gpointer data) {
+ GstElementFactory *factory;
+
+ if (!GST_IS_ELEMENT_FACTORY (feature))
+ return FALSE;
+ factory = GST_ELEMENT_FACTORY (feature);
+ if (!g_strrstr (gst_element_factory_get_klass (factory), "Visualization"))
+ return FALSE;
+
+ return TRUE;
+}
+
+int main(int argc, char *argv[]) {
+ GstElement *pipeline, *vis_plugin;
+ GstBus *bus;
+ GstMessage *msg;
+ GList *list, *walk;
+ GstElementFactory *selected_factory = NULL;
+ guint flags;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Get a list of all visualization plugins */
+ list = gst_registry_feature_filter (gst_registry_get (), filter_vis_features, FALSE, NULL);
+
+ /* Print their names */
+ g_print("Available visualization plugins:\n");
+ for (walk = list; walk != NULL; walk = g_list_next (walk)) {
+ const gchar *name;
+ GstElementFactory *factory;
+
+ factory = GST_ELEMENT_FACTORY (walk->data);
+ name = gst_element_factory_get_longname (factory);
+ g_print(" %s\n", name);
+
+ if (selected_factory == NULL || g_str_has_prefix (name, "GOOM")) {
+ selected_factory = factory;
+ }
+ }
+
+ /* Don't use the factory if it's still empty */
+ /* e.g. no visualization plugins found */
+ if (!selected_factory) {
+ g_print ("No visualization plugins found!\n");
+ return -1;
+ }
+
+ /* We have now selected a factory for the visualization element */
+ g_print ("Selected '%s'\n", gst_element_factory_get_longname (selected_factory));
+ vis_plugin = gst_element_factory_create (selected_factory, NULL);
+ if (!vis_plugin)
+ return -1;
+
+ /* Build the pipeline */
+ pipeline = gst_parse_launch ("playbin uri=http://radio.hbr1.com:19800/ambient.ogg", NULL);
+
+ /* Set the visualization flag */
+ g_object_get (pipeline, "flags", &flags, NULL);
+ flags |= GST_PLAY_FLAG_VIS;
+ g_object_set (pipeline, "flags", flags, NULL);
+
+ /* set vis plugin for playbin */
+ g_object_set (pipeline, "vis-plugin", vis_plugin, NULL);
+
+ /* Start playing */
+ gst_element_set_state (pipeline, GST_STATE_PLAYING);
+
+ /* Wait until error or EOS */
+ bus = gst_element_get_bus (pipeline);
+ msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
+
+ /* Free resources */
+ if (msg != NULL)
+ gst_message_unref (msg);
+ gst_plugin_feature_list_free (list);
+ gst_object_unref (bus);
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+ return 0;
+}
+```
+
+> ![information] If you need help to compile this code, refer to the
+> **Building the tutorials** section for your platform: [Mac] or
+> [Windows] or use this specific command on Linux:
+>
+> `` gcc playback-tutorial-6.c -o playback-tutorial-6 `pkg-config --cflags --libs gstreamer-1.0` ``
+>
+> If you need help to run this code, refer to the **Running the
+> tutorials** section for your platform: [Mac OS X], [Windows][1], for
+> [iOS] or for [android].
+>
+> This tutorial plays music streamed from the [HBR1](http://www.hbr1.com/) Internet radio station. A window should open displaying somewhat psychedelic color patterns moving with the music. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed.
+>
+> Required libraries: `gstreamer-1.0`
+
+## Walkthrough
+
+First off, we indicate `playbin` that we want an audio visualization by
+setting the `GST_PLAY_FLAG_VIS` flag. If the media already contains
+video, this flag has no effect.
+
+``` c
+/* Set the visualization flag */
+g_object_get (pipeline, "flags", &flags, NULL);
+flags |= GST_PLAY_FLAG_VIS;
+g_object_set (pipeline, "flags", flags, NULL);
+```
+
+If no visualization plugin is enforced by the user, `playbin` will use
+`goom` (audio visualization will be disabled if `goom` is not
+available). The rest of the tutorial shows how to find out the available
+visualization elements and enforce one to `playbin`.
+
+``` c
+/* Get a list of all visualization plugins */
+list = gst_registry_feature_filter (gst_registry_get (), filter_vis_features, FALSE, NULL);
+```
+
+`gst_registry_feature_filter()` examines all elements currently in the
+GStreamer registry and selects those for which
+the `filter_vis_features` function returns TRUE. This function selects
+only the Visualization plugins:
+
+``` c
+/* Return TRUE if this is a Visualization element */
+static gboolean filter_vis_features (GstPluginFeature *feature, gpointer data) {
+ GstElementFactory *factory;
+
+ if (!GST_IS_ELEMENT_FACTORY (feature))
+ return FALSE;
+ factory = GST_ELEMENT_FACTORY (feature);
+ if (!g_strrstr (gst_element_factory_get_klass (factory), "Visualization"))
+ return FALSE;
+
+ return TRUE;
+}
+```
+
+A bit of theory regarding the organization of GStreamer elements is in
+place: Each of the files that GStreamer loads at runtime is known as a
+Plugin (`GstPlugin`). A Plugin can contain many Features
+(`GstPluginFeature`). There are different kinds of Features, among them,
+the Element Factories (`GstElementFactory`) that we have been using to
+build Elements (`GstElement`).
+
+This function simply disregards all Features which are not Factories,
+and then all Factories whose class (obtained with
+`gst_element_factory_get_klass()`) does not include “Visualization”. As
+stated in the documentation for `GstElementFactory`, a Factory’s class
+is a “string describing the type of element, as an unordered list
+separated with slashes (/)”. Examples of classes are “Source/Network”,
+“Codec/Decoder/Video”, “Codec/Encoder/Audio” or “Visualization”.
+
+``` c
+/* Print their names */
+g_print("Available visualization plugins:\n");
+for (walk = list; walk != NULL; walk = g_list_next (walk)) {
+ const gchar *name;
+ GstElementFactory *factory;
+
+ factory = GST_ELEMENT_FACTORY (walk->data);
+ name = gst_element_factory_get_longname (factory);
+ g_print(" %s\n", name);
+
+ if (selected_factory == NULL || g_str_has_prefix (name, "GOOM")) {
+ selected_factory = factory;
+ }
+}
+```
+
+Once we have the list of Visualization plugins, we print their names
+(`gst_element_factory_get_longname()`) and choose one (in this case,
+GOOM).
+
+``` c
+/* We have now selected a factory for the visualization element */
+g_print ("Selected '%s'\n", gst_element_factory_get_longname (selected_factory));
+vis_plugin = gst_element_factory_create (selected_factory, NULL);
+if (!vis_plugin)
+ return -1;
+```
+
+The selected factory is used to instantiate an actual `GstElement` which
+is then passed to `playbin` through the `vis-plugin` property:
+
+``` c
+/* set vis plugin for playbin */
+g_object_set (pipeline, "vis-plugin", vis_plugin, NULL);
+```
+
+And we are done.
+
+## Conclusion
+
+This tutorial has shown:
+
+ - How to enable Audio Visualization in `playbin` with the
+ `GST_PLAY_FLAG_VIS` flag
+ - How to enforce one particular visualization element with the
+ `vis-plugin` `playbin` property
+
+It has been a pleasure having you here, and see you soon\!
+
+ [information]: images/icons/emoticons/information.png
+ [Mac]: sdk-installing-on-mac-osx.md
+ [Windows]: sdk-installing-on-windows.md
+ [Mac OS X]: sdk-installing-on-mac-osx.md#building-the-tutorials
+ [1]: sdk-installing-on-windows.md#running-the-tutorials
+ [iOS]: sdk-installing-for-ios-development.md#building-the-tutorials
+ [android]: sdk-installing-for-android-development.md#building-the-tutorials
+ [warning]: images/icons/emoticons/warning.png
--- /dev/null
+# Playback tutorial 5: Color Balance
+
+## Goal
+
+Brightness, Contrast, Hue and Saturation are common video adjustments,
+which are collectively known as Color Balance settings in GStreamer.
+This tutorial shows:
+
+ - How to find out the available color balance channels
+ - How to change them
+
+## Introduction
+[](sdk-basic-tutorial-toolkit-integration.md) has
+already explained the concept of GObject interfaces: applications use
+them to find out if certain functionality is available, regardless of
+the actual element which implements it.
+
+`playbin` implements the Color Balance interface (`GstColorBalance`),
+which allows access to the color balance settings. If any of the
+elements in the `playbin` pipeline support this interface,
+`playbin` simply forwards it to the application, otherwise, a
+colorbalance element is inserted in the pipeline.
+
+This interface allows querying for the available color balance channels
+(`GstColorBalanceChannel`), along with their name and valid range of
+values, and then modify the current value of any of them.
+
+## Color balance example
+
+Copy this code into a text file named `playback-tutorial-5.c`.
+
+**playback-tutorial-5.c**
+
+``` c
+#include <string.h>
+#include <stdio.h>
+#include <gst/gst.h>
+#include <gst/video/colorbalance.h>
+
+typedef struct _CustomData {
+ GstElement *pipeline;
+ GMainLoop *loop;
+} CustomData;
+
+/* Process a color balance command */
+static void update_color_channel (const gchar *channel_name, gboolean increase, GstColorBalance *cb) {
+ gdouble step;
+ gint value;
+ GstColorBalanceChannel *channel = NULL;
+ const GList *channels, *l;
+
+ /* Retrieve the list of channels and locate the requested one */
+ channels = gst_color_balance_list_channels (cb);
+ for (l = channels; l != NULL; l = l->next) {
+ GstColorBalanceChannel *tmp = (GstColorBalanceChannel *)l->data;
+
+ if (g_strrstr (tmp->label, channel_name)) {
+ channel = tmp;
+ break;
+ }
+ }
+ if (!channel)
+ return;
+
+ /* Change the channel's value */
+ step = 0.1 * (channel->max_value - channel->min_value);
+ value = gst_color_balance_get_value (cb, channel);
+ if (increase) {
+ value = (gint)(value + step);
+ if (value > channel->max_value)
+ value = channel->max_value;
+ } else {
+ value = (gint)(value - step);
+ if (value < channel->min_value)
+ value = channel->min_value;
+ }
+ gst_color_balance_set_value (cb, channel, value);
+}
+
+/* Output the current values of all Color Balance channels */
+static void print_current_values (GstElement *pipeline) {
+ const GList *channels, *l;
+
+ /* Output Color Balance values */
+ channels = gst_color_balance_list_channels (GST_COLOR_BALANCE (pipeline));
+ for (l = channels; l != NULL; l = l->next) {
+ GstColorBalanceChannel *channel = (GstColorBalanceChannel *)l->data;
+ gint value = gst_color_balance_get_value (GST_COLOR_BALANCE (pipeline), channel);
+ g_print ("%s: %3d%% ", channel->label,
+ 100 * (value - channel->min_value) / (channel->max_value - channel->min_value));
+ }
+ g_print ("\n");
+}
+
+/* Process keyboard input */
+static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data) {
+ gchar *str = NULL;
+
+ if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) != G_IO_STATUS_NORMAL) {
+ return TRUE;
+ }
+
+ switch (g_ascii_tolower (str[0])) {
+ case 'c':
+ update_color_channel ("CONTRAST", g_ascii_isupper (str[0]), GST_COLOR_BALANCE (data->pipeline));
+ break;
+ case 'b':
+ update_color_channel ("BRIGHTNESS", g_ascii_isupper (str[0]), GST_COLOR_BALANCE (data->pipeline));
+ break;
+ case 'h':
+ update_color_channel ("HUE", g_ascii_isupper (str[0]), GST_COLOR_BALANCE (data->pipeline));
+ break;
+ case 's':
+ update_color_channel ("SATURATION", g_ascii_isupper (str[0]), GST_COLOR_BALANCE (data->pipeline));
+ break;
+ case 'q':
+ g_main_loop_quit (data->loop);
+ break;
+ default:
+ break;
+ }
+
+ g_free (str);
+
+ print_current_values (data->pipeline);
+
+ return TRUE;
+}
+
+int main(int argc, char *argv[]) {
+ CustomData data;
+ GstStateChangeReturn ret;
+ GIOChannel *io_stdin;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Initialize our data structure */
+ memset (&data, 0, sizeof (data));
+
+ /* Print usage map */
+ g_print (
+ "USAGE: Choose one of the following options, then press enter:\n"
+ " 'C' to increase contrast, 'c' to decrease contrast\n"
+ " 'B' to increase brightness, 'b' to decrease brightness\n"
+ " 'H' to increase hue, 'h' to decrease hue\n"
+ " 'S' to increase saturation, 's' to decrease saturation\n"
+ " 'Q' to quit\n");
+
+ /* Build the pipeline */
+ data.pipeline = gst_parse_launch ("playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
+
+ /* Add a keyboard watch so we get notified of keystrokes */
+#ifdef G_OS_WIN32
+ io_stdin = g_io_channel_win32_new_fd (fileno (stdin));
+#else
+ io_stdin = g_io_channel_unix_new (fileno (stdin));
+#endif
+ g_io_add_watch (io_stdin, G_IO_IN, (GIOFunc)handle_keyboard, &data);
+
+ /* Start playing */
+ ret = gst_element_set_state (data.pipeline, GST_STATE_PLAYING);
+ if (ret == GST_STATE_CHANGE_FAILURE) {
+ g_printerr ("Unable to set the pipeline to the playing state.\n");
+ gst_object_unref (data.pipeline);
+ return -1;
+ }
+ print_current_values (data.pipeline);
+
+ /* Create a GLib Main Loop and set it to run */
+ data.loop = g_main_loop_new (NULL, FALSE);
+ g_main_loop_run (data.loop);
+
+ /* Free resources */
+ g_main_loop_unref (data.loop);
+ g_io_channel_unref (io_stdin);
+ gst_element_set_state (data.pipeline, GST_STATE_NULL);
+ gst_object_unref (data.pipeline);
+ return 0;
+}
+```
+
+> ![information] If you need help to compile this code, refer to the
+> **Building the tutorials** section for your platform: [Mac] or
+> [Windows] or use this specific command on Linux:
+>
+> `` gcc playback-tutorial-5.c -o playback-tutorial-5 `pkg-config --cflags --libs gstreamer-1.0 gstreamer-video-1.0` ``
+>
+> If you need help to run this code, refer to the **Running the
+> tutorials** section for your platform: [Mac OS X], [Windows][1], for
+> [iOS] or for [android].
+>
+> This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed.
+>
+>The console should print all commands (Each command is a single upper-case or lower-case letter) and list all available Color Balance channels, typically, CONTRAST, BRIGHTNESS, HUE and SATURATION. Type each command (letter) followed by the Enter key.
+>
+> Required libraries: `gstreamer-1.0 gstreamer-video-1.0`
+
+## Walkthrough
+
+The `main()` function is fairly simple. A `playbin` pipeline is
+instantiated and set to run, and a keyboard watch is installed so
+keystrokes can be monitored.
+
+``` c
+/* Output the current values of all Color Balance channels */
+static void print_current_values (GstElement *pipeline) {
+ const GList *channels, *l;
+
+ /* Output Color Balance values */
+ channels = gst_color_balance_list_channels (GST_COLOR_BALANCE (pipeline));
+ for (l = channels; l != NULL; l = l->next) {
+ GstColorBalanceChannel *channel = (GstColorBalanceChannel *)l->data;
+ gint value = gst_color_balance_get_value (GST_COLOR_BALANCE (pipeline), channel);
+ g_print ("%s: %3d%% ", channel->label,
+ 100 * (value - channel->min_value) / (channel->max_value - channel->min_value));
+ }
+ g_print ("\n");
+}
+```
+
+This method prints the current value for all channels, and exemplifies
+how to retrieve the list of channels. This is accomplished through the
+`gst_color_balance_list_channels()` method. It returns a `GList` which
+needs to be traversed.
+
+Each element in the list is a `GstColorBalanceChannel` structure,
+informing of the channel’s name, minimum value and maximum value.
+`gst_color_balance_get_value()` can then be called on each channel to
+retrieve the current value.
+
+In this example, the minimum and maximum values are used to output the
+current value as a percentage.
+
+``` c
+/* Process a color balance command */
+static void update_color_channel (const gchar *channel_name, gboolean increase, GstColorBalance *cb) {
+ gdouble step;
+ gint value;
+ GstColorBalanceChannel *channel = NULL;
+ const GList *channels, *l;
+
+ /* Retrieve the list of channels and locate the requested one */
+ channels = gst_color_balance_list_channels (cb);
+ for (l = channels; l != NULL; l = l->next) {
+ GstColorBalanceChannel *tmp = (GstColorBalanceChannel *)l->data;
+
+ if (g_strrstr (tmp->label, channel_name)) {
+ channel = tmp;
+ break;
+ }
+ }
+ if (!channel)
+ return;
+```
+
+This method locates the specified channel by name and increases or
+decreases it as requested. Again, the list of channels is retrieved and
+parsed looking for the channel with the specified name. Obviously, this
+list could be parsed only once and the pointers to the channels be
+stored and indexed by something more efficient than a string.
+
+``` c
+ /* Change the channel's value */
+ step = 0.1 * (channel->max_value - channel->min_value);
+ value = gst_color_balance_get_value (cb, channel);
+ if (increase) {
+ value = (gint)(value + step);
+ if (value > channel->max_value)
+ value = channel->max_value;
+ } else {
+ value = (gint)(value - step);
+ if (value < channel->min_value)
+ value = channel->min_value;
+ }
+ gst_color_balance_set_value (cb, channel, value);
+}
+```
+
+The current value for the channel is then retrieved, changed (the
+increment is proportional to its dynamic range), clamped (to avoid
+out-of-range values) and set using `gst_color_balance_set_value()`.
+
+And there is not much more to it. Run the program and observe the effect
+of changing each of the channels in real time.
+
+## Conclusion
+
+This tutorial has shown how to use the color balance interface.
+Particularly, it has shown:
+
+ - How to retrieve the list of color available balance channels
+ with `gst_color_balance_list_channels()`
+ - How to manipulate the current value of each channel using
+ `gst_color_balance_get_value()` and `gst_color_balance_set_value()`
+
+It has been a pleasure having you here, and see you soon\!
+
+
+ [information]: images/icons/emoticons/information.png
+ [Mac]: sdk-installing-on-mac-osx.md
+ [Windows]: sdk-installing-on-windows.md
+ [Mac OS X]: sdk-installing-on-mac-osx.md#building-the-tutorials
+ [1]: sdk-installing-on-windows.md#running-the-tutorials
+ [iOS]: sdk-installing-for-ios-development.md#building-the-tutorials
+ [android]: sdk-installing-for-android-development.md#building-the-tutorials
+ [warning]: images/icons/emoticons/warning.png
--- /dev/null
+# Playback tutorial 7: Custom playbin sinks
+
+## Goal
+
+`playbin` can be further customized by manually selecting its audio and
+video sinks. This allows applications to rely on `playbin` to retrieve
+and decode the media and then manage the final render/display
+themselves. This tutorial shows:
+
+ - How to replace the sinks selected by `playbin`.
+ - How to use a complex pipeline as a sink.
+
+## Introduction
+
+Two properties of `playbin` allow selecting the desired audio and video
+sinks: `audio-sink` and `video-sink` (respectively). The application
+only needs to instantiate the appropriate `GstElement` and pass it to
+`playbin` through these properties.
+
+This method, though, only allows using a single Element as sink. If a
+more complex pipeline is required, for example, an equalizer plus an
+audio sink, it needs to be wrapped in a Bin, so it looks to
+`playbin` as if it was a single Element.
+
+A Bin (`GstBin`) is a container that encapsulates partial pipelines so
+they can be managed as single elements. As an example, the
+`GstPipeline` we have been using in all tutorials is a type of
+`GstBin`, which does not interact with external Elements. Elements
+inside a Bin connect to external elements through Ghost Pads
+(`GstGhostPad`), this is, Pads on the surface of the Bin which simply
+forward data from an external Pad to a given Pad on an internal Element.
+
+
+
+**Figure 1:** A Bin with two Elements and one Ghost Pad.
+
+`GstBin`s are also a type of `GstElement`, so they can be used wherever
+an Element is required, in particular, as sinks for `playbin` (and they
+are then known as **sink-bins**).
+
+## An equalized player
+
+Copy this code into a text file named `playback-tutorial-7.c`.
+
+**playback-tutorial7.c**
+
+``` c
+#include <gst/gst.h>
+
+int main(int argc, char *argv[]) {
+ GstElement *pipeline, *bin, *equalizer, *convert, *sink;
+ GstPad *pad, *ghost_pad;
+ GstBus *bus;
+ GstMessage *msg;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Build the pipeline */
+ pipeline = gst_parse_launch ("playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
+
+ /* Create the elements inside the sink bin */
+ equalizer = gst_element_factory_make ("equalizer-3bands", "equalizer");
+ convert = gst_element_factory_make ("audioconvert", "convert");
+ sink = gst_element_factory_make ("autoaudiosink", "audio_sink");
+ if (!equalizer || !convert || !sink) {
+ g_printerr ("Not all elements could be created.\n");
+ return -1;
+ }
+
+ /* Create the sink bin, add the elements and link them */
+ bin = gst_bin_new ("audio_sink_bin");
+ gst_bin_add_many (GST_BIN (bin), equalizer, convert, sink, NULL);
+ gst_element_link_many (equalizer, convert, sink, NULL);
+ pad = gst_element_get_static_pad (equalizer, "sink");
+ ghost_pad = gst_ghost_pad_new ("sink", pad);
+ gst_pad_set_active (ghost_pad, TRUE);
+ gst_element_add_pad (bin, ghost_pad);
+ gst_object_unref (pad);
+
+ /* Configure the equalizer */
+ g_object_set (G_OBJECT (equalizer), "band1", (gdouble)-24.0, NULL);
+ g_object_set (G_OBJECT (equalizer), "band2", (gdouble)-24.0, NULL);
+
+ /* Set playbin's audio sink to be our sink bin */
+ g_object_set (GST_OBJECT (pipeline), "audio-sink", bin, NULL);
+
+ /* Start playing */
+ gst_element_set_state (pipeline, GST_STATE_PLAYING);
+
+ /* Wait until error or EOS */
+ bus = gst_element_get_bus (pipeline);
+ msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
+
+ /* Free resources */
+ if (msg != NULL)
+ gst_message_unref (msg);
+ gst_object_unref (bus);
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+ return 0;
+}
+```
+
+> ![information] If you need help to compile this code, refer to the
+> **Building the tutorials** section for your platform: [Mac] or
+> [Windows] or use this specific command on Linux:
+>
+> `` gcc playback-tutorial-7.c -o playback-tutorial-7 `pkg-config --cflags --libs gstreamer-1.0` ``
+>
+> If you need help to run this code, refer to the **Running the
+> tutorials** section for your platform: [Mac OS X], [Windows][1], for
+> [iOS] or for [android].
+>
+> This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed. The higher frequency bands have been attenuated, so the movie sound should have a more powerful bass component.<
+>
+> Required libraries: `gstreamer-1.0`
+
+## Walkthrough
+
+``` c
+/* Create the elements inside the sink bin */
+equalizer = gst_element_factory_make ("equalizer-3bands", "equalizer");
+convert = gst_element_factory_make ("audioconvert", "convert");
+sink = gst_element_factory_make ("autoaudiosink", "audio_sink");
+if (!equalizer || !convert || !sink) {
+ g_printerr ("Not all elements could be created.\n");
+ return -1;
+}
+```
+
+All the Elements that compose our sink-bin are instantiated. We use an
+`equalizer-3bands` and an `autoaudiosink`, with an `audioconvert` in
+between, because we are not sure of the capabilities of the audio sink
+(since they are hardware-dependant).
+
+``` c
+/* Create the sink bin, add the elements and link them */
+bin = gst_bin_new ("audio_sink_bin");
+gst_bin_add_many (GST_BIN (bin), equalizer, convert, sink, NULL);
+gst_element_link_many (equalizer, convert, sink, NULL);
+```
+
+This adds the new Elements to the Bin and links them just as we would do
+if this was a pipeline.
+
+``` c
+pad = gst_element_get_static_pad (equalizer, "sink");
+ghost_pad = gst_ghost_pad_new ("sink", pad);
+gst_pad_set_active (ghost_pad, TRUE);
+gst_element_add_pad (bin, ghost_pad);
+gst_object_unref (pad);
+```
+
+Now we need to create a Ghost Pad so this partial pipeline inside the
+Bin can be connected to the outside. This Ghost Pad will be connected to
+a Pad in one of the internal Elements (the sink pad of the equalizer),
+so we retrieve this Pad with `gst_element_get_static_pad()`. Remember
+from [](sdk-basic-tutorial-multithreading-and-pad-availability.md) that
+if this was a Request Pad instead of an Always Pad, we would need to use
+`gst_element_request_pad()`.
+
+The Ghost Pad is created with `gst_ghost_pad_new()` (pointing to the
+inner Pad we just acquired), and activated with `gst_pad_set_active()`.
+It is then added to the Bin with `gst_element_add_pad()`, transferring
+ownership of the Ghost Pad to the bin, so we do not have to worry about
+releasing it.
+
+Finally, the sink Pad we obtained from the equalizer needs to be release
+with `gst_object_unref()`.
+
+At this point, we have a functional sink-bin, which we can use as the
+audio sink in `playbin`. We just need to instruct `playbin` to use it:
+
+``` c
+/* Set playbin's audio sink to be our sink bin */
+g_object_set (GST_OBJECT (pipeline), "audio-sink", bin, NULL);
+```
+
+It is as simple as setting the `audio-sink` property on `playbin` to
+the newly created sink.
+
+``` c
+/* Configure the equalizer */
+g_object_set (G_OBJECT (equalizer), "band1", (gdouble)-24.0, NULL);
+g_object_set (G_OBJECT (equalizer), "band2", (gdouble)-24.0, NULL);
+```
+
+The only bit remaining is to configure the equalizer. For this example,
+the two higher frequency bands are set to the maximum attenuation so the
+bass is boosted. Play a bit with the values to feel the difference (Look
+at the documentation for the `equalizer-3bands` element for the allowed
+range of values).
+
+## Exercise
+
+Build a video bin instead of an audio bin, using one of the many
+interesting video filters GStreamer offers, like `solarize`,
+`vertigotv` or any of the Elements in the `effectv` plugin. Remember to
+use the color space conversion element `videoconvert` if your
+pipeline fails to link due to incompatible caps.
+
+## Conclusion
+
+This tutorial has shown:
+
+ - How to set your own sinks to `playbin` using the audio-sink and
+ video-sink properties.
+ - How to wrap a piece of pipeline into a `GstBin` so it can be used as
+ a **sink-bin** by `playbin`.
+
+It has been a pleasure having you here, and see you soon\!
+
+ [information]: images/icons/emoticons/information.png
+ [Mac]: sdk-installing-on-mac-osx.md
+ [Windows]: sdk-installing-on-windows.md
+ [Mac OS X]: sdk-installing-on-mac-osx.md#building-the-tutorials
+ [1]: sdk-installing-on-windows.md#running-the-tutorials
+ [iOS]: sdk-installing-for-ios-development.md#building-the-tutorials
+ [android]: sdk-installing-for-android-development.md#building-the-tutorials
+ [warning]: images/icons/emoticons/warning.png
--- /dev/null
+# Playback tutorial 9: Digital audio pass-through
+
+## Goal
+
+This tutorial shows how GStreamer handles digital audio pass-through.
+
+## Introduction
+
+Besides the common analog format, high-end audio systems usually also
+accept data in digital form, either compressed or uncompressed. This is
+convenient because the audio signal then travels from the computer to
+the speakers in a form that is more resilient to interference and noise,
+resulting higher quality.
+
+The connection is typically made through an
+[S/PDIF](http://en.wikipedia.org/wiki/SPDIF) cable which can either be
+optical (with [TOSLINK](http://en.wikipedia.org/wiki/TOSLINK)
+connectors) or coaxial (with [RCA](http://en.wikipedia.org/wiki/RCA)
+connectors). S/PDIF is also known as IEC 60958 type II (IEC 958 before
+1998).
+
+In this scenario, GStreamer does not need to perform audio decoding; it
+can simply output the encoded data, acting in *pass-through* mode, and
+let the external audio system perform the decoding.
+
+## Inner workings of GStreamer audio sinks
+
+First off, digital audio output must be enabled at the system level. The
+method to achieve this depend on the operating system, but it generally
+involves going to the audio control panel and activating a checkbox
+reading “Digital Audio Output” or similar.
+
+The main GStreamer audio sinks for each platform, Pulse Audio
+(`pulsesink`) for Linux, `osxaudiosink` for OS X and Direct Sound
+(`directsoundsink`) for Windows, detect when digital audio output is
+available and change their input caps accordingly to accept encoded
+data. For example, these elements typically accept `audio/x-raw` data:
+when digital audio output is enabled in the system, they may also
+accept `audio/mpeg`, `audio/x-ac3`, `audio/x-eac3` or `audio/x-dts`.
+
+Then, when `playbin` builds the decoding pipeline, it realizes that the
+audio sink can be directly connected to the encoded data (typically
+coming out of a demuxer), so there is no need for a decoder. This
+process is automatic and does not need any action from the application.
+
+On Linux, there exist other audio sinks, like Alsa (`alsasink`) which
+work differently (a “digital device” needs to be manually selected
+through the `device` property of the sink). Pulse Audio, though, is the
+commonly preferred audio sink on Linux.
+
+## Precautions with digital formats
+
+When Digital Audio Output is enabled at the system level, the GStreamer
+audio sinks automatically expose all possible digital audio caps,
+regardless of whether the actual audio decoder at the end of the S/PDIF
+cable is able to decode all those formats. This is so because there is
+no mechanism to query an external audio decoder which formats are
+supported, and, in fact, the cable can even be disconnected during this
+process.
+
+For example, after enabling Digital Audio Output in the system’s Control
+Panel, `directsoundsink` will automatically expose `audio/x-ac3`,
+`audio/x-eac3` and `audio/x-dts` caps in addition to `audio/x-raw`.
+However, one particular external decoder might only understand raw
+integer streams and would try to play the compressed data as such (a
+painful experience for your ears, rest assured).
+
+Solving this issue requires user intervention, since only the user knows
+the formats supported by the external decoder.
+
+On some systems, the simplest solution is to inform the operating system
+of the formats that the external audio decoder can accept. In this way,
+the GStreamer audio sinks will only offer these formats. The acceptable
+audio formats are commonly selected from the operating system’s audio
+configuration panel, from the same place where Digital Audio Output is
+enabled, but, unfortunately, this option is not available in all audio
+drivers.
+
+Another solution involves, using a custom sinkbin (see
+[](sdk-playback-tutorial-custom-playbin-sinks.md)) which includes a
+`capsfilter` element (see [](sdk-basic-tutorial-handy-elements.md))
+and an audio sink. The caps that the external decoder supports are
+then set in the capsfiler so the wrong format is not output. This
+allows the application to enforce the appropriate format instead of
+relying on the user to have the system correctly configured. Still
+requires user intervention, but can be used regardless of the options
+the audio driver offers.
+
+Please do not use `autoaudiosink` as the audio sink, as it currently
+only supports raw audio, and will ignore any compressed format.
+
+## Conclusion
+
+This tutorial has shown a bit of how GStreamer deals with digital audio.
+In particular, it has shown that:
+
+ - Applications using `playbin` do not need to do anything special to
+ enable digital audio output: it is managed from the audio control
+ panel of the operating system.
+
+It has been a pleasure having you here, and see you soon!
--- /dev/null
+# Playback tutorial 8: Hardware-accelerated video decoding
+
+### Goal
+
+Hardware-accelerated video decoding has rapidly become a necessity, as
+low-power devices grow more common. This tutorial (more of a lecture,
+actually) gives some background on hardware acceleration and explains
+how does GStreamer benefit from it.
+
+Sneak peek: if properly setup, you do not need to do anything special to
+activate hardware acceleration; GStreamer automatically takes advantage
+of it.
+
+### Introduction
+
+Video decoding can be an extremely CPU-intensive task, especially for
+higher resolutions like 1080p HDTV. Fortunately, modern graphics cards,
+equipped with programmable GPUs, are able to take care of this job,
+allowing the CPU to concentrate on other duties. Having dedicated
+hardware becomes essential for low-power CPUs which are simply incapable
+of decoding such media fast enough.
+
+In the current state of things (June 2016) each GPU manufacturer offers
+a different method to access their hardware (a different API), and a
+strong industry standard has not emerged yet.
+
+As of June 2016, there exist at least 8 different video decoding
+acceleration APIs:
+
+ - [VAAPI](http://en.wikipedia.org/wiki/Video_Acceleration_API) (*Video
+Acceleration API*): Initially designed by
+[Intel](http://en.wikipedia.org/wiki/Intel) in 2007, targeted at the X
+Window System on Unix-based operating systems, now open-source. It now also
+supports Wayland through dmabuf. It is
+currently not limited to Intel GPUs as other manufacturers are free to
+use this API, for example, [Imagination
+Technologies](http://en.wikipedia.org/wiki/Imagination_Technologies) or
+[S3 Graphics](http://en.wikipedia.org/wiki/S3_Graphics). Accessible to
+GStreamer through the [gstreamer-vaapi](https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/) package.
+
+- [VDPAU](http://en.wikipedia.org/wiki/VDPAU) (*Video Decode and
+Presentation API for UNIX*): Initially designed by
+[NVidia](http://en.wikipedia.org/wiki/NVidia) in 2008, targeted at the X
+Window System on Unix-based operating systems, now open-source. Although
+it is also an open-source library, no manufacturer other than NVidia is
+using it yet. Accessible to GStreamer through
+the [vdpau](http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/sys/vdpau) element in plugins-bad.
+
+ - [OpenMAX](http://en.wikipedia.org/wiki/OpenMAX) (*Open Media
+Acceleration*): Managed by the non-profit technology consortium [Khronos
+Group](http://en.wikipedia.org/wiki/Khronos_Group "Khronos Group"),
+it is a "royalty-free, cross-platform set of C-language programming
+interfaces that provides abstractions for routines especially useful for
+audio, video, and still images". Accessible to GStreamer through
+the [gst-omx](http://git.freedesktop.org/gstreamer/gst-omx) plugin.
+
+ - [OVD](http://developer.amd.com/sdks/AMDAPPSDK/assets/OpenVideo_Decode_API.PDF)
+(*Open Video Decode*): Another API from [AMD
+Graphics](http://en.wikipedia.org/wiki/AMD_Graphics), designed to be a
+platform agnostic method for softrware developers to leverage the
+[Universal Video
+Decode](http://en.wikipedia.org/wiki/Unified_Video_Decoder) (UVD)
+hardware inside AMD Radeon graphics cards. Currently unavailable to
+GStreamer .
+
+ - [DCE](http://en.wikipedia.org/wiki/Distributed_Codec_Engine)
+(*Distributed Codec Engine*): An open source software library ("libdce")
+and API specification by [Texas
+Instruments](http://en.wikipedia.org/wiki/Texas_Instruments), targeted
+at Linux systems and ARM platforms. Accessible to GStreamer through
+the [gstreamer-ducati](https://github.com/robclark/gst-ducati) plugin.
+
+ - [Android
+ MediaCodec](https://developer.android.com/reference/android/media/MediaCodec.html): This is Android's API to access the device's
+ hardware decoder and encoder if available. This is accessible through the
+ `androidmedia` plugin in gst-plugins-bad. This includes both encoding and
+ decoding.
+
+ - Apple VideoTool Box Framework: Apple's API to access h is available
+ through the `applemedia` plugin which includes both encoding through
+ the `vtenc` element and decoding through the `vtdec` element.
+
+ - Video4Linux: Recent Linux kernels have a kernel API to expose
+ hardware codecs in a standard way, this is now supported by the
+ `v4l2` plugin in `gst-plugins-good`. This can support both decoding
+ and encoding depending on the platform.
+
+### Inner workings of hardware-accelerated video decoding plugins
+
+These APIs generally offer a number of functionalities, like video
+decoding, post-processing, or presentation of the decoded
+frames. Correspondingly, plugins generally offer a different GStreamer
+element for each of these functions, so pipelines can be built to
+accommodate any need.
+
+For example, the `gstreamer-vaapi` plugin offers the `vaapidecode`,
+`vaapipostproc` and `vaapisink` elements that allow
+hardware-accelerated decoding through VAAPI, upload of raw video frames
+to GPU memory, download of GPU frames to system memory and presentation
+of GPU frames, respectively.
+
+It is important to distinguish between conventional GStreamer frames,
+which reside in system memory, and frames generated by
+hardware-accelerated APIs. The latter reside in GPU memory and cannot
+be touched by GStreamer. They can usually be downloaded to system
+memory and treated as conventional GStreamer frames when they are
+mapped, but it is far more efficient to leave them in the GPU and
+display them from there.
+
+GStreamer needs to keep track of where these “hardware buffers” are
+though, so conventional buffers still travel from element to
+element. They look like regular buffers, but mapping their content is
+much slower as it has to be retrieved from the special memory used by
+hardware accelerated elements. This special memory types are
+negotiated using the allocation query mechanism.
+
+This all means that, if a particular hardware acceleration API is
+present in the system, and the corresponding GStreamer plugin is also
+available, auto-plugging elements like `playbin` are free to use
+hardware acceleration to build their pipelines; the application does not
+need to do anything special to enable it. Almost:
+
+When `playbin` has to choose among different equally valid elements,
+like conventional software decoding (through `vp8dec`, for example) or
+hardware accelerated decoding (through `vaapidecode`, for example), it
+uses their *rank* to decide. The rank is a property of each element that
+indicates its priority; `playbin` will simply select the element that
+is able to build a complete pipeline and has the highest rank.
+
+So, whether `playbin` will use hardware acceleration or not will depend
+on the relative ranks of all elements capable of dealing with that media
+type. Therefore, the easiest way to make sure hardware acceleration is
+enabled or disabled is by changing the rank of the associated element,
+as shown in this code:
+
+``` c
+static void enable_factory (const gchar *name, gboolean enable) {
+ GstRegistry *registry = NULL;
+ GstElementFactory *factory = NULL;
+
+ registry = gst_registry_get_default ();
+ if (!registry) return;
+
+ factory = gst_element_factory_find (name);
+ if (!factory) return;
+
+ if (enable) {
+ gst_plugin_feature_set_rank (GST_PLUGIN_FEATURE (factory), GST_RANK_PRIMARY + 1);
+ }
+ else {
+ gst_plugin_feature_set_rank (GST_PLUGIN_FEATURE (factory), GST_RANK_NONE);
+ }
+
+ gst_registry_add_feature (registry, GST_PLUGIN_FEATURE (factory));
+ return;
+}
+```
+
+The first parameter passed to this method is the name of the element to
+modify, for example, `vaapidecode` or `fluvadec`.
+
+The key method is `gst_plugin_feature_set_rank()`, which will set the
+rank of the requested element factory to the desired level. For
+convenience, ranks are divided in NONE, MARGINAL, SECONDARY and PRIMARY,
+but any number will do. When enabling an element, we set it to
+PRIMARY+1, so it has a higher rank than the rest of elements which
+commonly have PRIMARY rank. Setting an element’s rank to NONE will make
+the auto-plugging mechanism to never select it.
+
+> ![warning] The GStreamer developers often rank hardware decoders lower than
+> the software ones when they are defective. This should act as a warning.
+
+## Conclusion
+
+This tutorial has shown a bit how GStreamer internally manages hardware
+accelerated video decoding. Particularly,
+
+ - Applications do not need to do anything special to enable hardware
+ acceleration if a suitable API and the corresponding GStreamer
+ plugin are available.
+ - Hardware acceleration can be enabled or disabled by changing the
+ rank of the decoding element with `gst_plugin_feature_set_rank()`.
+
+It has been a pleasure having you here, and see you soon!
+
+ [warning]: images/icons/emoticons/warning.png
--- /dev/null
+# Playback tutorial 1: Playbin usage
+
+## Goal
+
+We have already worked with the `playbin` element, which is capable of
+building a complete playback pipeline without much work on our side.
+This tutorial shows how to further customize `playbin` in case its
+default values do not suit our particular needs.
+
+We will learn:
+
+- How to find out how many streams a file contains, and how to switch
+ among them.
+
+- How to gather information regarding each stream.
+
+As a side note, even though its name is `playbin`, you can pronounce it
+“playbin”, since the original `playbin` element is deprecated and nobody
+should be using it.
+
+## Introduction
+
+More often than not, multiple audio, video and subtitle streams can be
+found embedded in a single file. The most common case are regular
+movies, which contain one video and one audio stream (Stereo or 5.1
+audio tracks are considered a single stream). It is also increasingly
+common to find movies with one video and multiple audio streams, to
+account for different languages. In this case, the user selects one
+audio stream, and the application will only play that one, ignoring the
+others.
+
+To be able to select the appropriate stream, the user needs to know
+certain information about them, for example, their language. This
+information is embedded in the streams in the form of “metadata”
+(annexed data), and this tutorial shows how to retrieve it.
+
+Subtitles can also be embedded in a file, along with audio and video,
+but they are dealt with in more detail in [Playback tutorial 2: Subtitle
+management]. Finally, multiple video streams can also be found in a
+single file, for example, in DVD with multiple angles of the same scene,
+but they are somewhat rare.
+
+> ![information] Embedding multiple streams inside a single file is
+> called “multiplexing” or “muxing”, and such file is then known as a
+> “container”. Common container formats are Matroska (.mkv), Quicktime
+> (.qt, .mov, .mp4), Ogg (.ogg) or Webm (.webm).
+>
+> Retrieving the individual streams from within the container is called
+> “demultiplexing” or “demuxing”.
+
+The following code recovers the amount of streams in the file, their
+associated metadata, and allows switching the audio stream while the
+media is playing.
+
+## The multilingual player
+
+Copy this code into a text file named `playback-tutorial-1.c` (or find
+it in the GStreamer installation).
+
+**playback-tutorial-1.c**
+
+``` c
+#include <gst/gst.h>
+
+/* Structure to contain all our information, so we can pass it around */
+typedef struct _CustomData {
+ GstElement *playbin; /* Our one and only element */
+
+ gint n_video; /* Number of embedded video streams */
+ gint n_audio; /* Number of embedded audio streams */
+ gint n_text; /* Number of embedded subtitle streams */
+
+ gint current_video; /* Currently playing video stream */
+ gint current_audio; /* Currently playing audio stream */
+ gint current_text; /* Currently playing subtitle stream */
+
+ GMainLoop *main_loop; /* GLib's Main Loop */
+} CustomData;
+
+/* playbin flags */
+typedef enum {
+ GST_PLAY_FLAG_VIDEO = (1 << 0), /* We want video output */
+ GST_PLAY_FLAG_AUDIO = (1 << 1), /* We want audio output */
+ GST_PLAY_FLAG_TEXT = (1 << 2) /* We want subtitle output */
+} GstPlayFlags;
+
+/* Forward definition for the message and keyboard processing functions */
+static gboolean handle_message (GstBus *bus, GstMessage *msg, CustomData *data);
+static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data);
+
+int main(int argc, char *argv[]) {
+ CustomData data;
+ GstBus *bus;
+ GstStateChangeReturn ret;
+ gint flags;
+ GIOChannel *io_stdin;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Create the elements */
+ data.playbin = gst_element_factory_make ("playbin", "playbin");
+
+ if (!data.playbin) {
+ g_printerr ("Not all elements could be created.\n");
+ return -1;
+ }
+
+ /* Set the URI to play */
+ g_object_set (data.playbin, "uri", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_cropped_multilingual.webm", NULL);
+
+ /* Set flags to show Audio and Video but ignore Subtitles */
+ g_object_get (data.playbin, "flags", &flags, NULL);
+ flags |= GST_PLAY_FLAG_VIDEO | GST_PLAY_FLAG_AUDIO;
+ flags &= ~GST_PLAY_FLAG_TEXT;
+ g_object_set (data.playbin, "flags", flags, NULL);
+
+ /* Set connection speed. This will affect some internal decisions of playbin */
+ g_object_set (data.playbin, "connection-speed", 56, NULL);
+
+ /* Add a bus watch, so we get notified when a message arrives */
+ bus = gst_element_get_bus (data.playbin);
+ gst_bus_add_watch (bus, (GstBusFunc)handle_message, &data);
+
+ /* Add a keyboard watch so we get notified of keystrokes */
+#ifdef G_OS_WIN32
+ io_stdin = g_io_channel_win32_new_fd (fileno (stdin));
+#else
+ io_stdin = g_io_channel_unix_new (fileno (stdin));
+#endif
+ g_io_add_watch (io_stdin, G_IO_IN, (GIOFunc)handle_keyboard, &data);
+
+ /* Start playing */
+ ret = gst_element_set_state (data.playbin, GST_STATE_PLAYING);
+ if (ret == GST_STATE_CHANGE_FAILURE) {
+ g_printerr ("Unable to set the pipeline to the playing state.\n");
+ gst_object_unref (data.playbin);
+ return -1;
+ }
+
+ /* Create a GLib Main Loop and set it to run */
+ data.main_loop = g_main_loop_new (NULL, FALSE);
+ g_main_loop_run (data.main_loop);
+
+ /* Free resources */
+ g_main_loop_unref (data.main_loop);
+ g_io_channel_unref (io_stdin);
+ gst_object_unref (bus);
+ gst_element_set_state (data.playbin, GST_STATE_NULL);
+ gst_object_unref (data.playbin);
+ return 0;
+}
+
+/* Extract some metadata from the streams and print it on the screen */
+static void analyze_streams (CustomData *data) {
+ gint i;
+ GstTagList *tags;
+ gchar *str;
+ guint rate;
+
+ /* Read some properties */
+ g_object_get (data->playbin, "n-video", &data->n_video, NULL);
+ g_object_get (data->playbin, "n-audio", &data->n_audio, NULL);
+ g_object_get (data->playbin, "n-text", &data->n_text, NULL);
+
+ g_print ("%d video stream(s), %d audio stream(s), %d text stream(s)\n",
+ data->n_video, data->n_audio, data->n_text);
+
+ g_print ("\n");
+ for (i = 0; i < data->n_video; i++) {
+ tags = NULL;
+ /* Retrieve the stream's video tags */
+ g_signal_emit_by_name (data->playbin, "get-video-tags", i, &tags);
+ if (tags) {
+ g_print ("video stream %d:\n", i);
+ gst_tag_list_get_string (tags, GST_TAG_VIDEO_CODEC, &str);
+ g_print (" codec: %s\n", str ? str : "unknown");
+ g_free (str);
+ gst_tag_list_free (tags);
+ }
+ }
+
+ g_print ("\n");
+ for (i = 0; i < data->n_audio; i++) {
+ tags = NULL;
+ /* Retrieve the stream's audio tags */
+ g_signal_emit_by_name (data->playbin, "get-audio-tags", i, &tags);
+ if (tags) {
+ g_print ("audio stream %d:\n", i);
+ if (gst_tag_list_get_string (tags, GST_TAG_AUDIO_CODEC, &str)) {
+ g_print (" codec: %s\n", str);
+ g_free (str);
+ }
+ if (gst_tag_list_get_string (tags, GST_TAG_LANGUAGE_CODE, &str)) {
+ g_print (" language: %s\n", str);
+ g_free (str);
+ }
+ if (gst_tag_list_get_uint (tags, GST_TAG_BITRATE, &rate)) {
+ g_print (" bitrate: %d\n", rate);
+ }
+ gst_tag_list_free (tags);
+ }
+ }
+
+ g_print ("\n");
+ for (i = 0; i < data->n_text; i++) {
+ tags = NULL;
+ /* Retrieve the stream's subtitle tags */
+ g_signal_emit_by_name (data->playbin, "get-text-tags", i, &tags);
+ if (tags) {
+ g_print ("subtitle stream %d:\n", i);
+ if (gst_tag_list_get_string (tags, GST_TAG_LANGUAGE_CODE, &str)) {
+ g_print (" language: %s\n", str);
+ g_free (str);
+ }
+ gst_tag_list_free (tags);
+ }
+ }
+
+ g_object_get (data->playbin, "current-video", &data->current_video, NULL);
+ g_object_get (data->playbin, "current-audio", &data->current_audio, NULL);
+ g_object_get (data->playbin, "current-text", &data->current_text, NULL);
+
+ g_print ("\n");
+ g_print ("Currently playing video stream %d, audio stream %d and text stream %d\n",
+ data->current_video, data->current_audio, data->current_text);
+ g_print ("Type any number and hit ENTER to select a different audio stream\n");
+}
+
+/* Process messages from GStreamer */
+static gboolean handle_message (GstBus *bus, GstMessage *msg, CustomData *data) {
+ GError *err;
+ gchar *debug_info;
+
+ switch (GST_MESSAGE_TYPE (msg)) {
+ case GST_MESSAGE_ERROR:
+ gst_message_parse_error (msg, &err, &debug_info);
+ g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
+ g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
+ g_clear_error (&err);
+ g_free (debug_info);
+ g_main_loop_quit (data->main_loop);
+ break;
+ case GST_MESSAGE_EOS:
+ g_print ("End-Of-Stream reached.\n");
+ g_main_loop_quit (data->main_loop);
+ break;
+ case GST_MESSAGE_STATE_CHANGED: {
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->playbin)) {
+ if (new_state == GST_STATE_PLAYING) {
+ /* Once we are in the playing state, analyze the streams */
+ analyze_streams (data);
+ }
+ }
+ } break;
+ }
+
+ /* We want to keep receiving messages */
+ return TRUE;
+}
+
+/* Process keyboard input */
+static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data) {
+ gchar *str = NULL;
+
+ if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) == G_IO_STATUS_NORMAL) {
+ int index = g_ascii_strtoull (str, NULL, 0);
+ if (index < 0 || index >= data->n_audio) {
+ g_printerr ("Index out of bounds\n");
+ } else {
+ /* If the input was a valid audio stream index, set the current audio stream */
+ g_print ("Setting current audio stream to %d\n", index);
+ g_object_set (data->playbin, "current-audio", index, NULL);
+ }
+ }
+ g_free (str);
+ return TRUE;
+}
+```
+
+> ![information] If you need help to compile this code, refer to the
+> **Building the tutorials** section for your platform: [Mac] or
+> [Windows] or use this specific command on Linux:
+>
+> `` gcc playback-tutorial-1.c -o playback-tutorial-1 `pkg-config --cflags --libs gstreamer-1.0` ``
+>
+> If you need help to run this code, refer to the **Running the
+> tutorials** section for your platform: [Mac OS X], [Windows][1], for
+> [iOS] or for [android].
+>
+> This tutorial opens a window and displays a movie, with accompanying
+> audio. The media is fetched from the Internet, so the window might take
+> a few seconds to appear, depending on your connection speed. The number
+> of audio streams is shown in the terminal, and the user can switch from
+> one to another by entering a number and pressing enter. A small delay is
+> to be expected.
+>
+> Bear in mind that there is no latency management (buffering), so on slow
+> connections, the movie might stop after a few seconds. See how [Tutorial
+> 12: Live streaming] solves this issue.
+>
+> Required libraries: `gstreamer-1.0`
+
+## Walkthrough
+
+``` c
+/* Structure to contain all our information, so we can pass it around */
+typedef struct _CustomData {
+ GstElement *playbin; /* Our one and only element */
+
+ gint n_video; /* Number of embedded video streams */
+ gint n_audio; /* Number of embedded audio streams */
+ gint n_text; /* Number of embedded subtitle streams */
+
+ gint current_video; /* Currently playing video stream */
+ gint current_audio; /* Currently playing audio stream */
+ gint current_text; /* Currently playing subtitle stream */
+
+ GMainLoop *main_loop; /* GLib's Main Loop */
+} CustomData;
+```
+
+We start, as usual, putting all our variables in a structure, so we can
+pass it around to functions. For this tutorial, we need the amount of
+streams of each type, and the currently playing one. Also, we are going
+to use a different mechanism to wait for messages that allows
+interactivity, so we need a GLib's main loop object.
+
+``` c
+/* playbin flags */
+typedef enum {
+ GST_PLAY_FLAG_VIDEO = (1 << 0), /* We want video output */
+ GST_PLAY_FLAG_AUDIO = (1 << 1), /* We want audio output */
+ GST_PLAY_FLAG_TEXT = (1 << 2) /* We want subtitle output */
+} GstPlayFlags;
+```
+
+Later we are going to set some of `playbin`'s flags. We would like to
+have a handy enum that allows manipulating these flags easily, but since
+`playbin` is a plug-in and not a part of the GStreamer core, this enum
+is not available to us. The “trick” is simply to declare this enum in
+our code, as it appears in the `playbin` documentation: `GstPlayFlags`.
+GObject allows introspection, so the possible values for these flags can
+be retrieved at runtime without using this trick, but in a far more
+cumbersome way.
+
+``` c
+/* Forward definition for the message and keyboard processing functions */
+static gboolean handle_message (GstBus *bus, GstMessage *msg, CustomData *data);
+static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data);
+```
+
+Forward declarations for the two callbacks we will be using.
+`handle_message` for the GStreamer messages, as we have already seen,
+and `handle_keyboard` for key strokes, since this tutorial is
+introducing a limited amount of interactivity.
+
+We skip over the creation of the pipeline, the instantiation of
+`playbin` and pointing it to our test media through the `uri`
+property. `playbin` is in itself a pipeline, and in this case it is the
+only element in the pipeline, so we skip completely the creation of the
+pipeline, and use directly the `playbin` element.
+
+We focus on some of the other properties of `playbin`, though:
+
+``` c
+/* Set flags to show Audio and Video but ignore Subtitles */
+g_object_get (data.playbin, "flags", &flags, NULL);
+flags |= GST_PLAY_FLAG_VIDEO | GST_PLAY_FLAG_AUDIO;
+flags &= ~GST_PLAY_FLAG_TEXT;
+g_object_set (data.playbin, "flags", flags, NULL);
+```
+
+`playbin`'s behavior can be changed through its `flags` property, which
+can have any combination of `GstPlayFlags`. The most interesting values
+are:
+
+| Flag | Description |
+|---------------------------|------------------------------------------------------------------------------------------------------------------------------------|
+| GST_PLAY_FLAG_VIDEO | Enable video rendering. If this flag is not set, there will be no video output. |
+| GST_PLAY_FLAG_AUDIO | Enable audio rendering. If this flag is not set, there will be no audio output. |
+| GST_PLAY_FLAG_TEXT | Enable subtitle rendering. If this flag is not set, subtitles will not be shown in the video output. |
+| GST_PLAY_FLAG_VIS | Enable rendering of visualisations when there is no video stream. Playback tutorial 6: Audio visualization goes into more details. |
+| GST_PLAY_FLAG_DOWNLOAD | See Basic tutorial 12: Streaming and Playback tutorial 4: Progressive streaming. |
+| GST_PLAY_FLAG_BUFFERING | See Basic tutorial 12: Streaming and Playback tutorial 4: Progressive streaming. |
+| GST_PLAY_FLAG_DEINTERLACE | If the video content was interlaced, this flag instructs playbin to deinterlace it before displaying it. |
+
+In our case, for demonstration purposes, we are enabling audio and video
+and disabling subtitles, leaving the rest of flags to their default
+values (this is why we read the current value of the flags with
+`g_object_get()` before overwriting it with `g_object_set()`).
+
+``` c
+/* Set connection speed. This will affect some internal decisions of playbin */
+g_object_set (data.playbin, "connection-speed", 56, NULL);
+```
+
+This property is not really useful in this example.
+`connection-speed` informs `playbin` of the maximum speed of our network
+connection, so, in case multiple versions of the requested media are
+available in the server, `playbin` chooses the most appropriate. This is
+mostly used in combination with streaming protocols like `mms` or
+`rtsp`.
+
+We have set all these properties one by one, but we could have all of
+them with a single call to `g_object_set()`:
+
+``` c
+g_object_set (data.playbin, "uri", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_cropped_multilingual.webm", "flags", flags, "connection-speed", 56, NULL);
+```
+
+This is why `g_object_set()` requires a NULL as the last parameter.
+
+``` c
+ /* Add a keyboard watch so we get notified of keystrokes */
+#ifdef _WIN32
+ io_stdin = g_io_channel_win32_new_fd (fileno (stdin));
+#else
+ io_stdin = g_io_channel_unix_new (fileno (stdin));
+#endif
+ g_io_add_watch (io_stdin, G_IO_IN, (GIOFunc)handle_keyboard, &data);
+```
+
+These lines connect a callback function to the standard input (the
+keyboard). The mechanism shown here is specific to GLib, and not really
+related to GStreamer, so there is no point in going into much depth.
+Applications normally have their own way of handling user input, and
+GStreamer has little to do with it besides the Navigation interface
+discussed briefly in [Tutorial 17: DVD playback].
+
+``` c
+/* Create a GLib Main Loop and set it to run */
+data.main_loop = g_main_loop_new (NULL, FALSE);
+g_main_loop_run (data.main_loop);
+```
+
+To allow interactivity, we will no longer poll the GStreamer bus
+manually. Instead, we create a `GMainLoop`(GLib main loop) and set it
+running with `g_main_loop_run()`. This function blocks and will not
+return until `g_main_loop_quit()` is issued. In the meantime, it will
+call the callbacks we have registered at the appropriate
+times: `handle_message` when a message appears on the bus, and
+`handle_keyboard` when the user presses any key.
+
+There is nothing new in handle\_message, except that when the pipeline
+moves to the PLAYING state, it will call the `analyze_streams` function:
+
+``` c
+/* Extract some metadata from the streams and print it on the screen */
+static void analyze_streams (CustomData *data) {
+ gint i;
+ GstTagList *tags;
+ gchar *str;
+ guint rate;
+
+ /* Read some properties */
+ g_object_get (data->playbin, "n-video", &data->n_video, NULL);
+ g_object_get (data->playbin, "n-audio", &data->n_audio, NULL);
+ g_object_get (data->playbin, "n-text", &data->n_text, NULL);
+```
+
+As the comment says, this function just gathers information from the
+media and prints it on the screen. The number of video, audio and
+subtitle streams is directly available through the `n-video`,
+`n-audio` and `n-text` properties.
+
+``` c
+for (i = 0; i < data->n_video; i++) {
+ tags = NULL;
+ /* Retrieve the stream's video tags */
+ g_signal_emit_by_name (data->playbin, "get-video-tags", i, &tags);
+ if (tags) {
+ g_print ("video stream %d:\n", i);
+ gst_tag_list_get_string (tags, GST_TAG_VIDEO_CODEC, &str);
+ g_print (" codec: %s\n", str ? str : "unknown");
+ g_free (str);
+ gst_tag_list_free (tags);
+ }
+}
+```
+
+Now, for each stream, we want to retrieve its metadata. Metadata is
+stored as tags in a `GstTagList` structure, which is a list of data
+pieces identified by a name. The `GstTagList` associated with a stream
+can be recovered with `g_signal_emit_by_name()`, and then individual
+tags are extracted with the `gst_tag_list_get_*` functions
+like `gst_tag_list_get_string()` for example.
+
+> ![information]
+> This rather unintuitive way of retrieving the tag list
+> is called an Action Signal. Action signals are emitted by the
+> application to a specific element, which then performs an action and
+> returns a result. They behave like a dynamic function call, in which
+> methods of a class are identified by their name (the signal's name)
+> instead of their memory address. These signals are listed In the
+> documentation along with the regular signals, and are tagged “Action”.
+> See `playbin`, for example.
+
+`playbin` defines 3 action signals to retrieve metadata:
+`get-video-tags`, `get-audio-tags` and `get-text-tags`. The name if the
+tags is standardized, and the list can be found in the `GstTagList`
+documentation. In this example we are interested in the
+`GST_TAG_LANGUAGE_CODE` of the streams and their `GST_TAG_*_CODEC`
+(audio, video or text).
+
+``` c
+g_object_get (data->playbin, "current-video", &data->current_video, NULL);
+g_object_get (data->playbin, "current-audio", &data->current_audio, NULL);
+g_object_get (data->playbin, "current-text", &data->current_text, NULL);
+```
+
+Once we have extracted all the metadata we want, we get the streams that
+are currently selected through 3 more properties of `playbin`:
+`current-video`, `current-audio` and `current-text`.
+
+It is interesting to always check the currently selected streams and
+never make any assumption. Multiple internal conditions can make
+`playbin` behave differently in different executions. Also, the order in
+which the streams are listed can change from one run to another, so
+checking the metadata to identify one particular stream becomes crucial.
+
+``` c
+/* Process keyboard input */
+static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data) {
+ gchar *str = NULL;
+
+ if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) == G_IO_STATUS_NORMAL) {
+ int index = g_ascii_strtoull (str, NULL, 0);
+ if (index < 0 || index >= data->n_audio) {
+ g_printerr ("Index out of bounds\n");
+ } else {
+ /* If the input was a valid audio stream index, set the current audio stream */
+ g_print ("Setting current audio stream to %d\n", index);
+ g_object_set (data->playbin, "current-audio", index, NULL);
+ }
+ }
+ g_free (str);
+ return TRUE;
+}
+```
+
+Finally, we allow the user to switch the running audio stream. This very
+basic function just reads a string from the standard input (the
+keyboard), interprets it as a number, and tries to set the
+`current-audio` property of `playbin` (which previously we have only
+read).
+
+Bear in mind that the switch is not immediate. Some of the previously
+decoded audio will still be flowing through the pipeline, while the new
+stream becomes active and is decoded. The delay depends on the
+particular multiplexing of the streams in the container, and the length
+`playbin` has selected for its internal queues (which depends on the
+network conditions).
+
+If you execute the tutorial, you will be able to switch from one
+language to another while the movie is running by pressing 0, 1 or 2
+(and ENTER). This concludes this tutorial.
+
+## Conclusion
+
+This tutorial has shown:
+
+- A few more of `playbin`'s properties: `flags`, `connection-speed`,
+ `n-video`, `n-audio`, `n-text`, `current-video`, `current-audio` and
+ `current-text`.
+
+- How to retrieve the list of tags associated with a stream
+ with `g_signal_emit_by_name()`.
+
+- How to retrieve a particular tag from the list with
+ `gst_tag_list_get_string()`or `gst_tag_list_get_uint()`
+
+- How to switch the current audio simply by writing to the
+ `current-audio` property.
+
+The next playback tutorial shows how to handle subtitles, either
+embedded in the container or in an external file.
+
+Remember that attached to this page you should find the complete source
+code of the tutorial and any accessory files needed to build it.
+
+It has been a pleasure having you here, and see you soon!
+
+ [Playback tutorial 2: Subtitle management]: sdk-playback-tutorial-subtitle-management.md
+ [information]: images/icons/emoticons/information.png
+ [Mac]: sdk-installing-on-mac-osx.md
+ [Windows]: sdk-installing-on-windows.md
+ [Mac OS X]: sdk-installing-on-mac-osx.md#building-the-tutorials
+ [1]: sdk-installing-on-windows.md#running-the-tutorials
+ [iOS]: sdk-installing-for-ios-development.md#building-the-tutorials
+ [android]: sdk-installing-for-android-development.md#building-the-tutorials
--- /dev/null
+# Playback tutorial 4: Progressive streaming
+
+## Goal
+
+[](sdk-basic-tutorial-streaming.md) showed how to
+enhance the user experience in poor network conditions, by taking
+buffering into account. This tutorial further expands
+[](sdk-basic-tutorial-streaming.md) by enabling
+the local storage of the streamed media, and describes the advantages of
+this technique. In particular, it shows:
+
+ - How to enable progressive downloading
+ - How to know what has been downloaded
+ - How to know where it has been downloaded
+ - How to limit the amount of downloaded data that is kept
+
+## Introduction
+
+When streaming, data is fetched from the network and a small buffer of
+future-data is kept to ensure smooth playback (see
+[](sdk-basic-tutorial-streaming.md)). However, data
+is discarded as soon as it is displayed or rendered (there is no
+past-data buffer). This means, that if a user wants to jump back and
+continue playback from a point in the past, data needs to be
+re-downloaded.
+
+Media players tailored for streaming, like YouTube, usually keep all
+downloaded data stored locally for this contingency. A graphical widget
+is also normally used to show how much of the file has already been
+downloaded.
+
+`playbin` offers similar functionalities through the `DOWNLOAD` flag
+which stores the media in a local temporary file for faster playback of
+already-downloaded chunks.
+
+This code also shows how to use the Buffering Query, which allows
+knowing what parts of the file are available.
+
+## A network-resilient example with local storage
+
+Copy this code into a text file named `playback-tutorial-4.c`.
+
+**playback-tutorial-4.c**
+
+``` c
+#include <gst/gst.h>
+#include <string.h>
+
+#define GRAPH_LENGTH 78
+
+/* playbin flags */
+typedef enum {
+ GST_PLAY_FLAG_DOWNLOAD = (1 << 7) /* Enable progressive download (on selected formats) */
+} GstPlayFlags;
+
+typedef struct _CustomData {
+ gboolean is_live;
+ GstElement *pipeline;
+ GMainLoop *loop;
+ gint buffering_level;
+} CustomData;
+
+static void got_location (GstObject *gstobject, GstObject *prop_object, GParamSpec *prop, gpointer data) {
+ gchar *location;
+ g_object_get (G_OBJECT (prop_object), "temp-location", &location, NULL);
+ g_print ("Temporary file: %s\n", location);
+ g_free (location);
+ /* Uncomment this line to keep the temporary file after the program exits */
+ /* g_object_set (G_OBJECT (prop_object), "temp-remove", FALSE, NULL); */
+}
+
+static void cb_message (GstBus *bus, GstMessage *msg, CustomData *data) {
+
+ switch (GST_MESSAGE_TYPE (msg)) {
+ case GST_MESSAGE_ERROR: {
+ GError *err;
+ gchar *debug;
+
+ gst_message_parse_error (msg, &err, &debug);
+ g_print ("Error: %s\n", err->message);
+ g_error_free (err);
+ g_free (debug);
+
+ gst_element_set_state (data->pipeline, GST_STATE_READY);
+ g_main_loop_quit (data->loop);
+ break;
+ }
+ case GST_MESSAGE_EOS:
+ /* end-of-stream */
+ gst_element_set_state (data->pipeline, GST_STATE_READY);
+ g_main_loop_quit (data->loop);
+ break;
+ case GST_MESSAGE_BUFFERING:
+ /* If the stream is live, we do not care about buffering. */
+ if (data->is_live) break;
+
+ gst_message_parse_buffering (msg, &data->buffering_level);
+
+ /* Wait until buffering is complete before start/resume playing */
+ if (data->buffering_level < 100)
+ gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
+ else
+ gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
+ break;
+ case GST_MESSAGE_CLOCK_LOST:
+ /* Get a new clock */
+ gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
+ gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
+ break;
+ default:
+ /* Unhandled message */
+ break;
+ }
+}
+
+static gboolean refresh_ui (CustomData *data) {
+ GstQuery *query;
+ gboolean result;
+
+ query = gst_query_new_buffering (GST_FORMAT_PERCENT);
+ result = gst_element_query (data->pipeline, query);
+ if (result) {
+ gint n_ranges, range, i;
+ gchar graph[GRAPH_LENGTH + 1];
+ gint64 position = 0, duration = 0;
+
+ memset (graph, ' ', GRAPH_LENGTH);
+ graph[GRAPH_LENGTH] = '\0';
+
+ n_ranges = gst_query_get_n_buffering_ranges (query);
+ for (range = 0; range < n_ranges; range++) {
+ gint64 start, stop;
+ gst_query_parse_nth_buffering_range (query, range, &start, &stop);
+ start = start * GRAPH_LENGTH / (stop - start);
+ stop = stop * GRAPH_LENGTH / (stop - start);
+ for (i = (gint)start; i < stop; i++)
+ graph [i] = '-';
+ }
+ if (gst_element_query_position (data->pipeline, GST_TIME_FORMAT, &position) &&
+ GST_CLOCK_TIME_IS_VALID (position) &&
+ gst_element_query_duration (data->pipeline, GST_TIME_FORMAT, &duration) &&
+ GST_CLOCK_TIME_IS_VALID (duration)) {
+ i = (gint)(GRAPH_LENGTH * (double)position / (double)(duration + 1));
+ graph [i] = data->buffering_level < 100 ? 'X' : '>';
+ }
+ g_print ("[%s]", graph);
+ if (data->buffering_level < 100) {
+ g_print (" Buffering: %3d%%", data->buffering_level);
+ } else {
+ g_print (" ");
+ }
+ g_print ("\r");
+ }
+
+ return TRUE;
+
+}
+
+int main(int argc, char *argv[]) {
+ GstElement *pipeline;
+ GstBus *bus;
+ GstStateChangeReturn ret;
+ GMainLoop *main_loop;
+ CustomData data;
+ guint flags;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Initialize our data structure */
+ memset (&data, 0, sizeof (data));
+ data.buffering_level = 100;
+
+ /* Build the pipeline */
+ pipeline = gst_parse_launch ("playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
+ bus = gst_element_get_bus (pipeline);
+
+ /* Set the download flag */
+ g_object_get (pipeline, "flags", &flags, NULL);
+ flags |= GST_PLAY_FLAG_DOWNLOAD;
+ g_object_set (pipeline, "flags", flags, NULL);
+
+ /* Uncomment this line to limit the amount of downloaded data */
+ /* g_object_set (pipeline, "ring-buffer-max-size", (guint64)4000000, NULL); */
+
+ /* Start playing */
+ ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
+ if (ret == GST_STATE_CHANGE_FAILURE) {
+ g_printerr ("Unable to set the pipeline to the playing state.\n");
+ gst_object_unref (pipeline);
+ return -1;
+ } else if (ret == GST_STATE_CHANGE_NO_PREROLL) {
+ data.is_live = TRUE;
+ }
+
+ main_loop = g_main_loop_new (NULL, FALSE);
+ data.loop = main_loop;
+ data.pipeline = pipeline;
+
+ gst_bus_add_signal_watch (bus);
+ g_signal_connect (bus, "message", G_CALLBACK (cb_message), &data);
+ g_signal_connect (pipeline, "deep-notify::temp-location", G_CALLBACK (got_location), NULL);
+
+ /* Register a function that GLib will call every second */
+ g_timeout_add_seconds (1, (GSourceFunc)refresh_ui, &data);
+
+ g_main_loop_run (main_loop);
+
+ /* Free resources */
+ g_main_loop_unref (main_loop);
+ gst_object_unref (bus);
+ gst_element_set_state (pipeline, GST_STATE_NULL);
+ gst_object_unref (pipeline);
+ g_print ("\n");
+ return 0;
+}
+```
+
+> ![information] If you need help to compile this code, refer to the
+> **Building the tutorials** section for your platform: [Mac] or
+> [Windows] or use this specific command on Linux:
+>
+> `` gcc playback-tutorial-4.c -o playback-tutorial-4 `pkg-config --cflags --libs gstreamer-1.0` ``
+>
+> If you need help to run this code, refer to the **Running the
+> tutorials** section for your platform: [Mac OS X], [Windows][1], for
+> [iOS] or for [android].
+>
+> This tutorial opens a window and displays a movie, with accompanying
+> audio. The media is fetched from the Internet, so the window might
+> take a few seconds to appear, depending on your connection
+> speed. In the console window, you should see a message indicating
+> where the media is being stored, and a text graph representing the
+> downloaded portions and the current position. A buffering message
+> appears whenever buffering is required, which might never happen is
+> your network connection is fast enough
+>
+> Required libraries: `gstreamer-1.0`
+
+
+## Walkthrough
+
+This code is based on that of [](sdk-basic-tutorial-streaming.md). Let’s review
+only the differences.
+
+### Setup
+
+``` c
+/* Set the download flag */
+g_object_get (pipeline, "flags", &flags, NULL);
+flags |= GST_PLAY_FLAG_DOWNLOAD;
+g_object_set (pipeline, "flags", flags, NULL);
+```
+
+By setting this flag, `playbin` instructs its internal queue (a
+`queue2` element, actually) to store all downloaded
+data.
+
+``` c
+g_signal_connect (pipeline, "deep-notify::temp-location", G_CALLBACK (got_location), NULL);
+```
+
+`deep-notify` signals are emitted by `GstObject` elements (like
+`playbin`) when the properties of any of their children elements
+change. In this case we want to know when the `temp-location` property
+changes, indicating that the `queue2` has decided where to store the
+downloaded
+data.
+
+``` c
+static void got_location (GstObject *gstobject, GstObject *prop_object, GParamSpec *prop, gpointer data) {
+ gchar *location;
+ g_object_get (G_OBJECT (prop_object), "temp-location", &location, NULL);
+ g_print ("Temporary file: %s\n", location);
+ g_free (location);
+ /* Uncomment this line to keep the temporary file after the program exits */
+ /* g_object_set (G_OBJECT (prop_object), "temp-remove", FALSE, NULL); */
+}
+```
+
+The `temp-location` property is read from the element that triggered the
+signal (the `queue2`) and printed on screen.
+
+When the pipeline state changes from `PAUSED` to `READY`, this file is
+removed. As the comment reads, you can keep it by setting the
+`temp-remove` property of the `queue2` to `FALSE`.
+
+> ![warning]
+> On Windows this file is usually created inside the `Temporary Internet Files` folder, which might hide it from Windows Explorer. If you cannot find the downloaded files, try to use the console.
+
+### User Interface
+
+In `main` we also install a timer which we use to refresh the UI every
+second.
+
+``` c
+/* Register a function that GLib will call every second */
+g_timeout_add_seconds (1, (GSourceFunc)refresh_ui, &data);
+```
+
+The `refresh_ui` method queries the pipeline to find out which parts of
+the file have been downloaded and what the currently playing position
+is. It builds a graph to display this information (sort of a text-mode
+user interface) and prints it on screen, overwriting the previous one so
+it looks like it is animated:
+
+ [---->------- ]
+
+The dashes ‘`-`’ indicate the downloaded parts, and the greater-than
+sign ‘`>`’ shows the current position (turning into an ‘`X`’ when the
+pipeline is paused). Keep in mind that if your network is fast enough,
+you will not see the download bar (the dashes) advance at all; it will
+be completely full from the beginning.
+
+``` c
+static gboolean refresh_ui (CustomData *data) {
+ GstQuery *query;
+ gboolean result;
+ query = gst_query_new_buffering (GST_FORMAT_PERCENT);
+ result = gst_element_query (data->pipeline, query);
+```
+
+The first thing we do in `refresh_ui` is construct a new Buffering
+`GstQuery` with `gst_query_new_buffering()` and pass it to the pipeline
+(`playbin`) with `gst_element_query()`. In [](sdk-basic-tutorial-time-management.md) we have
+already seen how to perform simple queries like Position and Duration
+using specific methods. More complex queries, like Buffering, need to
+use the more general `gst_element_query()`.
+
+The Buffering query can be made in different `GstFormat` (TIME, BYTES,
+PERCENTAGE and a few more). Not all elements can answer the query in all
+the formats, so you need to check which ones are supported in your
+particular pipeline. If `gst_element_query()` returns `TRUE`, the query
+succeeded. The answer to the query is contained in the same
+`GstQuery` structure we created, and can be retrieved using multiple
+parse methods:
+
+``` c
+n_ranges = gst_query_get_n_buffering_ranges (query);
+for (range = 0; range < n_ranges; range++) {
+ gint64 start, stop;
+ gst_query_parse_nth_buffering_range (query, range, &start, &stop);
+ start = start * GRAPH_LENGTH / (stop - start);
+ stop = stop * GRAPH_LENGTH / (stop - start);
+ for (i = (gint)start; i < stop; i++)
+ graph [i] = '-';
+}
+```
+
+Data does not need to be downloaded in consecutive pieces from the
+beginning of the file: Seeking, for example, might force to start
+downloading from a new position and leave a downloaded chunk behind.
+Therefore, `gst_query_get_n_buffering_ranges()` returns the number of
+chunks, or *ranges* of downloaded data, and then, the position and size
+of each range is retrieved with `gst_query_parse_nth_buffering_range()`.
+
+The format of the returned values (start and stop position for each
+range) depends on what we requested in the
+`gst_query_new_buffering()` call. In this case, PERCENTAGE. These
+values are used to generate the graph.
+
+``` c
+if (gst_element_query_position (data->pipeline, &format, &position) &&
+ GST_CLOCK_TIME_IS_VALID (position) &&
+ gst_element_query_duration (data->pipeline, &format, &duration) &&
+ GST_CLOCK_TIME_IS_VALID (duration)) {
+ i = (gint)(GRAPH_LENGTH * (double)position / (double)(duration + 1));
+ graph [i] = data->buffering_level < 100 ? 'X' : '>';
+}
+```
+
+Next, the current position is queried. It could be queried in the
+PERCENT format, so code similar to the one used for the ranges is used,
+but currently this format is not well supported for position queries.
+Instead, we use the TIME format and also query the duration to obtain a
+percentage.
+
+The current position is indicated with either a ‘`>`’ or an ‘`X`’
+depending on the buffering level. If it is below 100%, the code in the
+`cb_message` method will have set the pipeline to `PAUSED`, so we print
+an ‘`X`’. If the buffering level is 100% the pipeline is in the
+`PLAYING` state and we print a ‘`>`’.
+
+``` c
+if (data->buffering_level < 100) {
+ g_print (" Buffering: %3d%%", data->buffering_level);
+} else {
+ g_print (" ");
+}
+```
+
+Finally, if the buffering level is below 100%, we report this
+information (and delete it otherwise).
+
+### Limiting the size of the downloaded file
+
+``` c
+/* Uncomment this line to limit the amount of downloaded data */
+/* g_object_set (pipeline, "ring-buffer-max-size", (guint64)4000000, NULL); */
+```
+
+Uncomment line 139 to see how this can be achieved. This reduces the
+size of the temporary file, by overwriting already played regions.
+Observe the download bar to see which regions are kept available in the
+file.
+
+## Conclusion
+
+This tutorial has shown:
+
+ - How to enable progressive downloading with the
+ `GST_PLAY_FLAG_DOWNLOAD` `playbin` flag
+ - How to know what has been downloaded using a Buffering `GstQuery`
+ - How to know where it has been downloaded with the
+ `deep-notify::temp-location` signal
+ - How to limit the size of the temporary file with
+ the `ring-buffer-max-size` property of `playbin`.
+
+It has been a pleasure having you here, and see you soon!
+
+ [information]: images/icons/emoticons/information.png
+ [Mac]: sdk-installing-on-mac-osx.md
+ [Windows]: sdk-installing-on-windows.md
+ [Mac OS X]: sdk-installing-on-mac-osx.md#building-the-tutorials
+ [1]: sdk-installing-on-windows.md#running-the-tutorials
+ [iOS]: sdk-installing-for-ios-development.md#building-the-tutorials
+ [android]: sdk-installing-for-android-development.md#building-the-tutorials
+ [warning]: images/icons/emoticons/warning.png
--- /dev/null
+# Playback tutorial 3: Short-cutting the pipeline
+
+## Goal
+
+[](sdk-basic-tutorial-short-cutting-the-pipeline.md) showed
+how an application can manually extract or inject data into a pipeline
+by using two special elements called `appsrc` and `appsink`.
+`playbin` allows using these elements too, but the method to connect
+them is different. To connect an `appsink` to `playbin` see [](sdk-playback-tutorial-custom-playbin-sinks.md).
+This tutorial shows:
+
+ - How to connect `appsrc` with `playbin`
+ - How to configure the `appsrc`
+
+## A playbin waveform generator
+
+Copy this code into a text file named `playback-tutorial-3.c`.
+
+**playback-tutorial-3.c**
+
+``` c
+#include <gst/gst.h>
+#include <gst/audio/audio.h>
+#include <string.h>
+
+#define CHUNK_SIZE 1024 /* Amount of bytes we are sending in each buffer */
+#define SAMPLE_RATE 44100 /* Samples per second we are sending */
+
+/* Structure to contain all our information, so we can pass it to callbacks */
+typedef struct _CustomData {
+ GstElement *pipeline;
+ GstElement *app_source;
+
+ guint64 num_samples; /* Number of samples generated so far (for timestamp generation) */
+ gfloat a, b, c, d; /* For waveform generation */
+
+ guint sourceid; /* To control the GSource */
+
+ GMainLoop *main_loop; /* GLib's Main Loop */
+} CustomData;
+
+/* This method is called by the idle GSource in the mainloop, to feed CHUNK_SIZE bytes into appsrc.
+ * The ide handler is added to the mainloop when appsrc requests us to start sending data (need-data signal)
+ * and is removed when appsrc has enough data (enough-data signal).
+ */
+static gboolean push_data (CustomData *data) {
+ GstBuffer *buffer;
+ GstFlowReturn ret;
+ int i;
+ GstMapInfo map;
+ gint16 *raw;
+ gint num_samples = CHUNK_SIZE / 2; /* Because each sample is 16 bits */
+ gfloat freq;
+
+ /* Create a new empty buffer */
+ buffer = gst_buffer_new_and_alloc (CHUNK_SIZE);
+
+ /* Set its timestamp and duration */
+ GST_BUFFER_TIMESTAMP (buffer) = gst_util_uint64_scale (data->num_samples, GST_SECOND, SAMPLE_RATE);
+ GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale (CHUNK_SIZE, GST_SECOND, SAMPLE_RATE);
+
+ /* Generate some psychodelic waveforms */
+ gst_buffer_map (buffer, &map, GST_MAP_WRITE);
+ raw = (gint16 *)map.data;
+ data->c += data->d;
+ data->d -= data->c / 1000;
+ freq = 1100 + 1000 * data->d;
+ for (i = 0; i < num_samples; i++) {
+ data->a += data->b;
+ data->b -= data->a / freq;
+ raw[i] = (gint16)(500 * data->a);
+ }
+ gst_buffer_unmap (buffer, &map);
+ data->num_samples += num_samples;
+
+ /* Push the buffer into the appsrc */
+ g_signal_emit_by_name (data->app_source, "push-buffer", buffer, &ret);
+
+ /* Free the buffer now that we are done with it */
+ gst_buffer_unref (buffer);
+
+ if (ret != GST_FLOW_OK) {
+ /* We got some error, stop sending data */
+ return FALSE;
+ }
+
+ return TRUE;
+}
+
+/* This signal callback triggers when appsrc needs data. Here, we add an idle handler
+ * to the mainloop to start pushing data into the appsrc */
+static void start_feed (GstElement *source, guint size, CustomData *data) {
+ if (data->sourceid == 0) {
+ g_print ("Start feeding\n");
+ data->sourceid = g_idle_add ((GSourceFunc) push_data, data);
+ }
+}
+
+/* This callback triggers when appsrc has enough data and we can stop sending.
+ * We remove the idle handler from the mainloop */
+static void stop_feed (GstElement *source, CustomData *data) {
+ if (data->sourceid != 0) {
+ g_print ("Stop feeding\n");
+ g_source_remove (data->sourceid);
+ data->sourceid = 0;
+ }
+}
+
+/* This function is called when an error message is posted on the bus */
+static void error_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
+ GError *err;
+ gchar *debug_info;
+
+ /* Print error details on the screen */
+ gst_message_parse_error (msg, &err, &debug_info);
+ g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
+ g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
+ g_clear_error (&err);
+ g_free (debug_info);
+
+ g_main_loop_quit (data->main_loop);
+}
+
+/* This function is called when playbin has created the appsrc element, so we have
+ * a chance to configure it. */
+static void source_setup (GstElement *pipeline, GstElement *source, CustomData *data) {
+ GstAudioInfo info;
+ GstCaps *audio_caps;
+
+ g_print ("Source has been created. Configuring.\n");
+ data->app_source = source;
+
+ /* Configure appsrc */
+ gst_audio_info_set_format (&info, GST_AUDIO_FORMAT_S16, SAMPLE_RATE, 1, NULL);
+ audio_caps = gst_audio_info_to_caps (&info);
+ g_object_set (source, "caps", audio_caps, "format", GST_FORMAT_TIME, NULL);
+ g_signal_connect (source, "need-data", G_CALLBACK (start_feed), data);
+ g_signal_connect (source, "enough-data", G_CALLBACK (stop_feed), data);
+ gst_caps_unref (audio_caps);
+ g_free (audio_caps_text);
+}
+
+int main(int argc, char *argv[]) {
+ CustomData data;
+ GstBus *bus;
+
+ /* Initialize cumstom data structure */
+ memset (&data, 0, sizeof (data));
+ data.b = 1; /* For waveform generation */
+ data.d = 1;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Create the playbin element */
+ data.pipeline = gst_parse_launch ("playbin uri=appsrc://", NULL);
+ g_signal_connect (data.pipeline, "source-setup", G_CALLBACK (source_setup), &data);
+
+ /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
+ bus = gst_element_get_bus (data.pipeline);
+ gst_bus_add_signal_watch (bus);
+ g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, &data);
+ gst_object_unref (bus);
+
+ /* Start playing the pipeline */
+ gst_element_set_state (data.pipeline, GST_STATE_PLAYING);
+
+ /* Create a GLib Main Loop and set it to run */
+ data.main_loop = g_main_loop_new (NULL, FALSE);
+ g_main_loop_run (data.main_loop);
+
+ /* Free resources */
+ gst_element_set_state (data.pipeline, GST_STATE_NULL);
+ gst_object_unref (data.pipeline);
+ return 0;
+}
+```
+
+To use an `appsrc` as the source for the pipeline, simply instantiate a
+`playbin` and set its URI to `appsrc://`
+
+``` c
+/* Create the playbin element */
+data.pipeline = gst_parse_launch ("playbin uri=appsrc://", NULL);
+```
+
+`playbin` will create an internal `appsrc` element and fire the
+`source-setup` signal to allow the application to configure
+it:
+
+``` c
+g_signal_connect (data.pipeline, "source-setup", G_CALLBACK (source_setup), &data);
+```
+
+In particular, it is important to set the caps property of `appsrc`,
+since, once the signal handler returns, `playbin` will instantiate the
+next element in the pipeline according to these
+caps:
+
+``` c
+/* This function is called when playbin has created the appsrc element, so we have
+ * a chance to configure it. */
+static void source_setup (GstElement *pipeline, GstElement *source, CustomData *data) {
+ GstAudioInfo info;
+ GstCaps *audio_caps;
+
+ g_print ("Source has been created. Configuring.\n");
+ data->app_source = source;
+
+ /* Configure appsrc */
+ gst_audio_info_set_format (&info, GST_AUDIO_FORMAT_S16, SAMPLE_RATE, 1, NULL);
+ audio_caps = gst_audio_info_to_caps (&info);
+ g_object_set (source, "caps", audio_caps, "format", GST_FORMAT_TIME, NULL);
+ g_signal_connect (source, "need-data", G_CALLBACK (start_feed), data);
+ g_signal_connect (source, "enough-data", G_CALLBACK (stop_feed), data);
+ gst_caps_unref (audio_caps);
+ g_free (audio_caps_text);
+}
+```
+
+The configuration of the `appsrc` is exactly the same as in
+[](sdk-basic-tutorial-short-cutting-the-pipeline.md):
+the caps are set to `audio/x-raw`, and two callbacks are registered,
+so the element can tell the application when it needs to start and stop
+pushing data. See [](sdk-basic-tutorial-short-cutting-the-pipeline.md)
+for more details.
+
+From this point onwards, `playbin` takes care of the rest of the
+pipeline, and the application only needs to worry about generating more
+data when told so.
+
+To learn how data can be extracted from `playbin` using the
+`appsink` element, see [](sdk-playback-tutorial-custom-playbin-sinks.md).
+
+## Conclusion
+
+This tutorial applies the concepts shown in
+[](sdk-basic-tutorial-short-cutting-the-pipeline.md) to
+`playbin`. In particular, it has shown:
+
+ - How to connect `appsrc` with `playbin` using the special
+ URI `appsrc://`
+ - How to configure the `appsrc` using the `source-setup` signal
+
+It has been a pleasure having you here, and see you soon!
\ No newline at end of file
--- /dev/null
+# Playback tutorial 2: Subtitle management
+
+## Goal
+
+This tutorial is very similar to the previous one, but instead of
+switching among different audio streams, we will use subtitle streams.
+This will allow us to learn:
+
+ - How to choose the subtitle stream
+
+ - How to add external subtitles
+
+ - How to customize the font used for the subtitles
+
+## Introduction
+
+We already know (from the previous tutorial) that container files can
+hold multiple audio and video streams, and that we can very easily
+choose among them by changing the `current-audio` or
+`current-video` `playbin` property. Switching subtitles is just as
+easy.
+
+It is worth noting that, just like it happens with audio and video,
+`playbin` takes care of choosing the right decoder for the subtitles,
+and that the plugin structure of GStreamer allows adding support for new
+formats as easily as copying a file. Everything is invisible to the
+application developer.
+
+Besides subtitles embedded in the container, `playbin` offers the
+possibility to add an extra subtitle stream from an external URI.
+
+This tutorial opens a file which already contains 5 subtitle streams,
+and adds another one from another file (for the Greek language).
+
+## The multilingual player with subtitles
+
+Copy this code into a text file named `playback-tutorial-2.c` (or find
+it in the GStreamer installation).
+
+**playback-tutorial-2.c**
+
+``` c
+#include <stdio.h>
+#include <gst/gst.h>
+
+/* Structure to contain all our information, so we can pass it around */
+typedef struct _CustomData {
+ GstElement *playbin; /* Our one and only element */
+
+ gint n_video; /* Number of embedded video streams */
+ gint n_audio; /* Number of embedded audio streams */
+ gint n_text; /* Number of embedded subtitle streams */
+
+ gint current_video; /* Currently playing video stream */
+ gint current_audio; /* Currently playing audio stream */
+ gint current_text; /* Currently playing subtitle stream */
+
+ GMainLoop *main_loop; /* GLib's Main Loop */
+} CustomData;
+
+/* playbin flags */
+typedef enum {
+ GST_PLAY_FLAG_VIDEO = (1 << 0), /* We want video output */
+ GST_PLAY_FLAG_AUDIO = (1 << 1), /* We want audio output */
+ GST_PLAY_FLAG_TEXT = (1 << 2) /* We want subtitle output */
+} GstPlayFlags;
+
+/* Forward definition for the message and keyboard processing functions */
+static gboolean handle_message (GstBus *bus, GstMessage *msg, CustomData *data);
+static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data);
+
+int main(int argc, char *argv[]) {
+ CustomData data;
+ GstBus *bus;
+ GstStateChangeReturn ret;
+ gint flags;
+ GIOChannel *io_stdin;
+
+ /* Initialize GStreamer */
+ gst_init (&argc, &argv);
+
+ /* Create the elements */
+ data.playbin = gst_element_factory_make ("playbin", "playbin");
+
+ if (!data.playbin) {
+ g_printerr ("Not all elements could be created.\n");
+ return -1;
+ }
+
+ /* Set the URI to play */
+ g_object_set (data.playbin, "uri", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.ogv", NULL);
+
+ /* Set the subtitle URI to play and some font description */
+ g_object_set (data.playbin, "suburi", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer_gr.srt", NULL);
+ g_object_set (data.playbin, "subtitle-font-desc", "Sans, 18", NULL);
+
+ /* Set flags to show Audio, Video and Subtitles */
+ g_object_get (data.playbin, "flags", &flags, NULL);
+ flags |= GST_PLAY_FLAG_VIDEO | GST_PLAY_FLAG_AUDIO | GST_PLAY_FLAG_TEXT;
+ g_object_set (data.playbin, "flags", flags, NULL);
+
+ /* Add a bus watch, so we get notified when a message arrives */
+ bus = gst_element_get_bus (data.playbin);
+ gst_bus_add_watch (bus, (GstBusFunc)handle_message, &data);
+
+ /* Add a keyboard watch so we get notified of keystrokes */
+#ifdef G_OS_WIN32
+ io_stdin = g_io_channel_win32_new_fd (fileno (stdin));
+#else
+ io_stdin = g_io_channel_unix_new (fileno (stdin));
+#endif
+ g_io_add_watch (io_stdin, G_IO_IN, (GIOFunc)handle_keyboard, &data);
+
+ /* Start playing */
+ ret = gst_element_set_state (data.playbin, GST_STATE_PLAYING);
+ if (ret == GST_STATE_CHANGE_FAILURE) {
+ g_printerr ("Unable to set the pipeline to the playing state.\n");
+ gst_object_unref (data.playbin);
+ return -1;
+ }
+
+ /* Create a GLib Main Loop and set it to run */
+ data.main_loop = g_main_loop_new (NULL, FALSE);
+ g_main_loop_run (data.main_loop);
+
+ /* Free resources */
+ g_main_loop_unref (data.main_loop);
+ g_io_channel_unref (io_stdin);
+ gst_object_unref (bus);
+ gst_element_set_state (data.playbin, GST_STATE_NULL);
+ gst_object_unref (data.playbin);
+ return 0;
+}
+
+/* Extract some metadata from the streams and print it on the screen */
+static void analyze_streams (CustomData *data) {
+ gint i;
+ GstTagList *tags;
+ gchar *str;
+ guint rate;
+
+ /* Read some properties */
+ g_object_get (data->playbin, "n-video", &data->n_video, NULL);
+ g_object_get (data->playbin, "n-audio", &data->n_audio, NULL);
+ g_object_get (data->playbin, "n-text", &data->n_text, NULL);
+
+ g_print ("%d video stream(s), %d audio stream(s), %d text stream(s)\n",
+ data->n_video, data->n_audio, data->n_text);
+
+ g_print ("\n");
+ for (i = 0; i < data->n_video; i++) {
+ tags = NULL;
+ /* Retrieve the stream's video tags */
+ g_signal_emit_by_name (data->playbin, "get-video-tags", i, &tags);
+ if (tags) {
+ g_print ("video stream %d:\n", i);
+ gst_tag_list_get_string (tags, GST_TAG_VIDEO_CODEC, &str);
+ g_print (" codec: %s\n", str ? str : "unknown");
+ g_free (str);
+ gst_tag_list_free (tags);
+ }
+ }
+
+ g_print ("\n");
+ for (i = 0; i < data->n_audio; i++) {
+ tags = NULL;
+ /* Retrieve the stream's audio tags */
+ g_signal_emit_by_name (data->playbin, "get-audio-tags", i, &tags);
+ if (tags) {
+ g_print ("audio stream %d:\n", i);
+ if (gst_tag_list_get_string (tags, GST_TAG_AUDIO_CODEC, &str)) {
+ g_print (" codec: %s\n", str);
+ g_free (str);
+ }
+ if (gst_tag_list_get_string (tags, GST_TAG_LANGUAGE_CODE, &str)) {
+ g_print (" language: %s\n", str);
+ g_free (str);
+ }
+ if (gst_tag_list_get_uint (tags, GST_TAG_BITRATE, &rate)) {
+ g_print (" bitrate: %d\n", rate);
+ }
+ gst_tag_list_free (tags);
+ }
+ }
+
+ g_print ("\n");
+ for (i = 0; i < data->n_text; i++) {
+ tags = NULL;
+ /* Retrieve the stream's subtitle tags */
+ g_print ("subtitle stream %d:\n", i);
+ g_signal_emit_by_name (data->playbin, "get-text-tags", i, &tags);
+ if (tags) {
+ if (gst_tag_list_get_string (tags, GST_TAG_LANGUAGE_CODE, &str)) {
+ g_print (" language: %s\n", str);
+ g_free (str);
+ }
+ gst_tag_list_free (tags);
+ } else {
+ g_print (" no tags found\n");
+ }
+ }
+
+ g_object_get (data->playbin, "current-video", &data->current_video, NULL);
+ g_object_get (data->playbin, "current-audio", &data->current_audio, NULL);
+ g_object_get (data->playbin, "current-text", &data->current_text, NULL);
+
+ g_print ("\n");
+ g_print ("Currently playing video stream %d, audio stream %d and subtitle stream %d\n",
+ data->current_video, data->current_audio, data->current_text);
+ g_print ("Type any number and hit ENTER to select a different subtitle stream\n");
+}
+
+/* Process messages from GStreamer */
+static gboolean handle_message (GstBus *bus, GstMessage *msg, CustomData *data) {
+ GError *err;
+ gchar *debug_info;
+
+ switch (GST_MESSAGE_TYPE (msg)) {
+ case GST_MESSAGE_ERROR:
+ gst_message_parse_error (msg, &err, &debug_info);
+ g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
+ g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
+ g_clear_error (&err);
+ g_free (debug_info);
+ g_main_loop_quit (data->main_loop);
+ break;
+ case GST_MESSAGE_EOS:
+ g_print ("End-Of-Stream reached.\n");
+ g_main_loop_quit (data->main_loop);
+ break;
+ case GST_MESSAGE_STATE_CHANGED: {
+ GstState old_state, new_state, pending_state;
+ gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
+ if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->playbin)) {
+ if (new_state == GST_STATE_PLAYING) {
+ /* Once we are in the playing state, analyze the streams */
+ analyze_streams (data);
+ }
+ }
+ } break;
+ }
+
+ /* We want to keep receiving messages */
+ return TRUE;
+}
+
+/* Process keyboard input */
+static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data) {
+ gchar *str = NULL;
+
+ if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) == G_IO_STATUS_NORMAL) {
+ int index = atoi (str);
+ if (index < 0 || index >= data->n_text) {
+ g_printerr ("Index out of bounds\n");
+ } else {
+ /* If the input was a valid subtitle stream index, set the current subtitle stream */
+ g_print ("Setting current subtitle stream to %d\n", index);
+ g_object_set (data->playbin, "current-text", index, NULL);
+ }
+ }
+ g_free (str);
+ return TRUE;
+}
+```
+
+
+> ![information] Need help?
+>
+> If you need help to compile this code, refer to the **Building the
+> tutorials** section for your platform: [Linux], [Mac OS X] or
+> [Windows], or use this specific command on Linux:
+>
+> `` gcc playback-tutorial-2.c -o playback-tutorial-2 `pkg-config --cflags --libs gstreamer-1.0` ``
+>
+> If you need help to run this code, refer to the **Running the
+> tutorials** section for your platform: [Linux][1], [Mac OS X][2] or
+> [Windows][3].
+>
+> This tutorial opens a window and displays a movie, with accompanying
+> audio. The media is fetched from the Internet, so the window might
+> take a few seconds to appear, depending on your connection
+> speed. The number of subtitle streams is shown in the terminal, and
+> the user can switch from one to another by entering a number and
+> pressing enter. A small delay is to be
+> expected. _Please read the note at the bottom of this
+> page._ Bear in mind that
+> there is no latency management (buffering), so on slow connections,
+> the movie might stop after a few seconds. See how
+> [](sdk-basic-tutorial-streaming.md) solves this issue.
+>
+> Required libraries: `gstreamer-1.0`
+
+## Walkthrough
+
+This tutorial is copied from
+[](sdk-playback-tutorial-playbin-usage.md) with some changes, so let's
+review only the changes.
+
+``` c
+/* Set the subtitle URI to play and some font description */
+g_object_set (data.playbin, "suburi", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer_gr.srt", NULL);
+g_object_set (data.playbin, "subtitle-font-desc", "Sans, 18", NULL);
+```
+
+After setting the media URI, we set the `suburi` property, which points
+`playbin` to a file containing a subtitle stream. In this case, the
+media file already contains multiple subtitle streams, so the one
+provided in the `suburi` is added to the list, and will be the currently
+selected one.
+
+Note that metadata concerning a subtitle stream (like its language)
+resides in the container file, therefore, subtitles not embedded in a
+container will not have metadata. When running this tutorial you will
+find that the first subtitle stream does not have a language tag.
+
+The `subtitle-font-desc` property allows specifying the font to render
+the subtitles. Since [Pango](http://www.pango.org/) is the library used
+to render fonts, you can check its documentation to see how this font
+should be specified, in particular, the
+[pango-font-description-from-string](http://developer.gnome.org/pango/stable/pango-Fonts.html#pango-font-description-from-string) function.
+
+In a nutshell, the format of the string representation is `[FAMILY-LIST]
+[STYLE-OPTIONS] [SIZE]` where `FAMILY-LIST` is a comma separated list of
+families optionally terminated by a comma, `STYLE_OPTIONS` is a
+whitespace separated list of words where each word describes one of
+style, variant, weight, or stretch, and `SIZE` is an decimal number
+(size in points). For example the following are all valid string
+representations:
+
+ - sans bold 12
+ - serif, monospace bold italic condensed 16
+ - normal 10
+
+The commonly available font families are: Normal, Sans, Serif and
+Monospace.
+
+The available styles are: Normal (the font is upright), Oblique (the
+font is slanted, but in a roman style), Italic (the font is slanted in
+an italic style).
+
+The available weights are: Ultra-Light, Light, Normal, Bold, Ultra-Bold,
+Heavy.
+
+The available variants are: Normal, Small\_Caps (A font with the lower
+case characters replaced by smaller variants of the capital characters)
+
+The available stretch styles
+are: Ultra-Condensed, Extra-Condensed, Condensed, Semi-Condensed, Normal, Semi-Expanded, Expanded,
+Extra-Expanded, Ultra-Expanded
+
+
+
+``` c
+/* Set flags to show Audio, Video and Subtitles */
+g_object_get (data.playbin, "flags", &flags, NULL);
+flags |= GST_PLAY_FLAG_VIDEO | GST_PLAY_FLAG_AUDIO | GST_PLAY_FLAG_TEXT;
+g_object_set (data.playbin, "flags", flags, NULL);
+```
+
+We set the `flags` property to allow Audio, Video and Text (Subtitles).
+
+The rest of the tutorial is the same as [](sdk-playback-tutorial-playbin-usage.md), except
+that the keyboard input changes the `current-text` property instead of
+the `current-audio`. As before, keep in mind that stream changes are not
+immediate, since there is a lot of information flowing through the
+pipeline that needs to reach the end of it before the new stream shows
+up.
+
+## Conclusion
+
+This tutorial showed how to handle subtitles from `playbin`, whether
+they are embedded in the container or in a different file:
+
+ - Subtitles are chosen using the `current-tex`t and `n-tex`t
+ properties of `playbin`.
+
+ - External subtitle files can be selected using the `suburi` property.
+
+ - Subtitle appearance can be customized with the
+ `subtitle-font-desc` property.
+
+The next playback tutorial shows how to change the playback speed.
+
+Remember that attached to this page you should find the complete source
+code of the tutorial and any accessory files needed to build it.
+It has been a pleasure having you here, and see you soon\!
+
+ [information]: images/icons/emoticons/information.png
--- /dev/null
+# Playback tutorials
+
+These tutorials explain everything you need to know to produce a media
+playback application using GStreamer.
--- /dev/null
+# QtGStreamer vs C GStreamer
+
+QtGStreamer is designed to mirror the C GStreamer API as closely as
+possible. There are, of course, minor differences. They are documented
+here.
+
+## Common Functions
+
+<table>
+<colgroup>
+<col width="50%" />
+<col width="50%" />
+</colgroup>
+<thead>
+<tr class="header">
+<th>C GStreamer</th>
+<th>QtGStreamer</th>
+</tr>
+</thead>
+<tbody>
+<tr class="odd">
+<td><code>gst_element_factory_make()</code></td>
+<td><code>QGst::ElementFactory::make(const QString &factoryName, const char *elementName=NULL)</code></td>
+</tr>
+<tr class="even">
+<td><code>gst_parse_bin_from_description()</code></td>
+<td><code>QGst::Bin::fromDescription(const QString &description, BinFromDescriptionOption ghostUnlinkedPads=Ghost)</code></td>
+</tr>
+<tr class="odd">
+<td><code>gst_caps_from_string()</code></td>
+<td><p><code>QGst::Caps::fromString(const QString &string)</code></p></td>
+</tr>
+<tr class="even">
+<td><code>g_signal_connect()</code></td>
+<td><code>QGlib::connect(GObject* instance, const char *detailedSignal, T *receiver, R(T::*)(Args...) slot, ConnectFlags flags)</code></td>
+</tr>
+</tbody>
+</table>
+
+## Naming Convention
+
+QtGStreamer follows a strict naming policy to help make cross
+referencing easier:
+
+#### Namespaces
+
+The "G" namespace (`GObject`, `GValue`, etc...) is referred to as
+"QGlib".
+
+The "Gst" namespace (`GstObject`, `GstElement`, etc...) is referred to
+as "QGst".
+
+#### Class Names
+
+Class names should be the same as their G\* equivalents, with the
+namespace prefix removed. For example, "`GstObject`" becomes
+"`QGst::Object`", "`GParamSpec`" becomes "`QGlib::ParamSpec`", etc...
+
+#### Method Names
+
+In general the method names should be the same as the GStreamer ones,
+with the g\[st\]\_\<class\> prefix removed and converted to camel case.
+
+For example,
+
+``` c
+gboolean gst_caps_is_emtpy(const GstCaps *caps);
+```
+
+becomes:
+
+``` c
+namespace QGst {
+ class Caps {
+ bool isEmpty() const;
+ }
+}
+```
+
+There are cases where this may not be followed:
+
+1. **Properties**. Most property getters have a "get" prefix, for
+ example, `gst_object_get_name()`. In QtGStreamer the "get" prefix is
+ omitted, so this becomes just `name()`.
+2. **Overloaded members**. In C there is no possibility to have two
+ methods with the same name, so overloaded members usually have some
+ extra suffix, like "\_full". For example, `g_object_set_data()` and
+ `g_object_set_data_full()`. In C++ we just add a method with the
+ same name, or put optional parameters in the existing method.
+3. **Other cases where the glib/gstreamer method name doesn't make much
+ sense**. For example, `gst_element_is_locked_state()`. That doesn't
+ make sense in english, as "sate" is the subject and should go before
+ the verb "is". So, it becomes `stateIsLocked()`.
+
+## Reference Counting
+
+Reference counting is handled the same way as Qt does. There is no need
+to call `g_object_ref()`` and g_object_unref()`.
+
+## Access to GStreamer Elements
+
+QtGStreamer provides access to the underlying C objects, in case you
+need them. This is accessible with a simple cast:
+
+``` c
+ElementPtr qgstElement = QGst::ElementFactory::make("playbin");
+GstElement* gstElement = GST_ELEMENT(qgstElement);
+```
--- /dev/null
+# Qt tutorials
+
+## Welcome to the GStreamer SDK Qt tutorials
+
+These tutorials describe Qt-specific topics. General GStreamer concepts
+will not be explained in these tutorials, so the [Basic
+tutorials](sdk-basic-tutorials.md) should
+be reviewed first. The reader should also be familiar with basic Qt
+programming techniques.
+
+The Qt tutorials have the same structure as the [Android
+tutorials](sdk-android-tutorials.md): Each one builds on top of the
+previous one and adds progressively more functionality, until a working
+media player application is obtained in \#FIXME\#
--- /dev/null
+# Table of Concepts
+
+This table shows in which tutorial each of the following key GStreamer
+concepts is discussed.
+
+- Action signals: [Playback tutorial 1: Playbin usage]
+- Audio switching: [Playback tutorial 1: Playbin usage]
+- Buffers: [Basic tutorial 8: Short-cutting the pipeline]
+- Bus: [Basic tutorial 2: GStreamer concepts]
+- Capabilities: [Basic tutorial 6: Media formats and Pad Capabilities]
+- Debugging: [Basic tutorial 11: Debugging tools]
+- Discoverer: [Basic tutorial 9: Media information gathering]
+- Elements: [Basic tutorial 2: GStreamer concepts]
+- gst-discoverer-1.0: [Basic tutorial 10: GStreamer tools]
+- gst-launch-1.0: [Basic tutorial 10: GStreamer tools], [gst-launch-1.0]
+- GUI: [Basic tutorial 5: GUI toolkit integration]
+- Links: [Basic tutorial 2: GStreamer concepts]
+- Pads: [Basic tutorial 3: Dynamic pipelines]
+- Pad Availability: [Basic tutorial 7: Multithreading and Pad
+ Availability]
+- Pipelines: [Basic tutorial 2: GStreamer concepts]
+- Queries: [Basic tutorial 4: Time management]
+- Seeks: [Basic tutorial 4: Time management]
+- Signals: [Basic tutorial 3: Dynamic pipelines]
+- States: [Basic tutorial 3: Dynamic pipelines]
+- Subtitles: [Playback tutorial 2: Subtitle management]
+- Tags: [Playback tutorial 1: Playbin usage]
+- Tools: [Basic tutorial 10: GStreamer tools]
+- Threads: [Basic tutorial 7: Multithreading and Pad Availability]
+
+ [Playback tutorial 1: Playbin usage]: sdk-playback-tutorial-playbin-usage.md
+ [Basic tutorial 8: Short-cutting the pipeline]: sdk-basic-tutorial-short-cutting-the-pipeline.md
+ [Basic tutorial 2: GStreamer concepts]: sdk-basic-tutorial-concepts.md
+ [Basic tutorial 6: Media formats and Pad Capabilities]: sdk-basic-tutorial-media-formats-and-pad-capabilities.md
+ [Basic tutorial 11: Debugging tools]: sdk-basic-tutorial-debugging-tools.md
+ [Basic tutorial 9: Media information gathering]: sdk-basic-tutorial-media-information-gathering.md
+ [Basic tutorial 10: GStreamer tools]: sdk-basic-tutorial-gstreamer-tools.md
+ [gst-launch-1.0]: gst-launch.md
+ [Basic tutorial 5: GUI toolkit integration]: sdk-basic-tutorial-toolkit-integration.md
+ [Basic tutorial 3: Dynamic pipelines]: sdk-basic-tutorial-dynamic-pipelines.md
+ [Basic tutorial 7: Multithreading and Pad Availability]: sdk-basic-tutorial-multithreading-and-pad-availability.md
+ [Basic tutorial 4: Time management]: sdk-basic-tutorial-time-management.md
+ [Playback tutorial 2: Subtitle management]: sdk-playback-tutorial-subtitle-management.md
--- /dev/null
+---
+short-description: Learn how to use GStreamer
+...
+
+# Tutorials
+
+## Welcome to the GStreamer Tutorials!
+
+The following sections introduce a series of tutorials designed to help
+you learn how to use GStreamer, the multi-platform, modular,
+open-source, media streaming framework.
+
+### Prerequisites
+
+Before following these tutorials, you need to set up your development
+environment according to your platform. If you have not done so yet, go
+to the [installing GStreamer] page and come back here afterwards.
+
+The tutorials are currently written only in the C programming language,
+so you need to be comfortable with it. Even though C is not an
+Object-Oriented (OO) language per se, the GStreamer framework uses
+`GObject`s, so some knowledge of OO concepts will come in handy.
+Knowledge of the `GObject` and `GLib` libraries is not mandatory, but
+will make the trip easier.
+
+### Source code
+
+Every tutorial represents a self-contained project, with full source
+code in C (and eventually in other languages too). Source code
+snippets are introduced alongside the text, and the full code (with
+any other required files like makefiles or project files) is
+distributed with GStreamer, as explained in the installation
+instructions.
+
+### A short note on GObject and GLib
+
+GStreamer is built on top of the `GObject` (for object orientation) and
+`GLib` (for common algorithms) libraries, which means that every now and
+then you will have to call functions of these libraries. Even though the
+tutorials will make sure that deep knowledge of these libraries is not
+required, familiarity with them will certainly ease the process of
+learning GStreamer.
+
+You can always tell which library you are calling because all GStreamer
+functions, structures and types have the `gst_` prefix, whereas GLib and
+GObject use `g_`.
+
+### Sources of documentation
+
+You have the `GObject` and `GLib` reference guides, and, of course the
+upstream [GStreamer documentation].
+
+### Structure
+
+The tutorials are organized in sections, revolving about a common theme:
+
+- [Basic tutorials]: Describe general topics required to understand
+ the rest of tutorials in GStreamer.
+- [Playback tutorials]: Explain everything you need to know to produce
+ a media playback application using GStreamer.
+- [Android tutorials]: Tutorials dealing with the few Android-specific
+ topics you need to know.
+- [iOS tutorials]: Tutorials dealing with the few iOS-specific topics
+ you need to know.
+
+If you cannot remember in which tutorial a certain GStreamer concept is
+explained, use the following:
+
+- [Table of Concepts]
+
+### Sample media
+
+The audio and video clips used throughout these tutorials are all
+publicly available and the copyright remains with their respective
+authors. In some cases they have been re-encoded for demonstration
+purposes.
+
+- [Sintel, the Durian Open Movie Project]
+
+ [installing GStreamer]: sdk-installing.md
+ [GStreamer documentation]: http://gstreamer.freedesktop.org/documentation/
+ [Basic tutorials]: sdk-basic-tutorials.md
+ [Playback tutorials]: sdk-playback-tutorials.md
+ [Android tutorials]: sdk-android-tutorials.md
+ [iOS tutorials]: sdk-ios-tutorials.md
+ [Table of Concepts]: sdk-table-of-concepts.md
+ [Sintel, the Durian Open Movie Project]: http://www.sintel.org/
--- /dev/null
+# Using appsink/appsrc in Qt
+
+## Goal
+
+For those times when you need to stream data into or out of GStreamer
+through your application, GStreamer includes two helpful elements:
+
+ - `appsink` - Allows applications to easily extract data from a
+ GStreamer pipeline
+ - `appsrc` - Allows applications to easily stream data into a
+ GStreamer pipeline
+
+This tutorial will demonstrate how to use both of them by constructing a
+pipeline to decode an audio file, stream it into an application's code,
+then stream it back into your audio output device. All this, using
+QtGStreamer.
+
+## Steps
+
+First, the files. These are also available in the
+`examples/appsink-src` directory of the QGstreamer SDK.
+
+**CMakeLists.txt**
+
+```
+project(qtgst-example-appsink-src)
+find_package(QtGStreamer REQUIRED)
+find_package(Qt4 REQUIRED)
+include_directories(${QTGSTREAMER_INCLUDES} ${QT_QTCORE_INCLUDE_DIRS})
+add_definitions(${QTGSTREAMER_DEFINITIONS})
+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${QTGSTREAMER_FLAGS}")
+add_executable(appsink-src main.cpp)
+target_link_libraries(appsink-src ${QTGSTREAMER_UTILS_LIBRARIES} ${QT_QTCORE_LIBRARIES})
+```
+
+**main.cpp**
+
+``` c
+#include <iostream>
+#include <QtCore/QCoreApplication>
+#include <QGlib/Error>
+#include <QGlib/Connect>
+#include <QGst/Init>
+#include <QGst/Bus>
+#include <QGst/Pipeline>
+#include <QGst/Parse>
+#include <QGst/Message>
+#include <QGst/Utils/ApplicationSink>
+#include <QGst/Utils/ApplicationSource>
+
+class MySink : public QGst::Utils::ApplicationSink
+{
+public:
+ MySink(QGst::Utils::ApplicationSource *src)
+ : QGst::Utils::ApplicationSink(), m_src(src) {}
+protected:
+ virtual void eos()
+ {
+ m_src->endOfStream();
+ }
+ virtual QGst::FlowReturn newBuffer()
+ {
+ m_src->pushBuffer(pullBuffer());
+ return QGst::FlowOk;
+ }
+private:
+ QGst::Utils::ApplicationSource *m_src;
+};
+
+class Player : public QCoreApplication
+{
+public:
+ Player(int argc, char **argv);
+ ~Player();
+private:
+ void onBusMessage(const QGst::MessagePtr & message);
+private:
+ QGst::Utils::ApplicationSource m_src;
+ MySink m_sink;
+ QGst::PipelinePtr pipeline1;
+ QGst::PipelinePtr pipeline2;
+};
+Player::Player(int argc, char **argv)
+ : QCoreApplication(argc, argv), m_sink(&m_src)
+{
+ QGst::init(&argc, &argv);
+ if (argc <= 1) {
+ std::cerr << "Usage: " << argv[0] << " <audio_file>" << std::endl;
+ std::exit(1);
+ }
+ const char *caps = "audio/x-raw-int,channels=1,rate=8000,"
+ "signed=(boolean)true,width=16,depth=16,endianness=1234";
+ /* source pipeline */
+ QString pipe1Descr = QString("filesrc location=\"%1\" ! "
+ "decodebin2 ! "
+ "audioconvert ! "
+ "audioresample ! "
+ "appsink name=\"mysink\" caps=\"%2\"").arg(argv[1], caps);
+ pipeline1 = QGst::Parse::launch(pipe1Descr).dynamicCast<QGst::Pipeline>();
+ m_sink.setElement(pipeline1->getElementByName("mysink"));
+ QGlib::connect(pipeline1->bus(), "message::error", this, &Player::onBusMessage);
+ pipeline1->bus()->addSignalWatch();
+ /* sink pipeline */
+ QString pipe2Descr = QString("appsrc name=\"mysrc\" caps=\"%1\" ! autoaudiosink").arg(caps);
+ pipeline2 = QGst::Parse::launch(pipe2Descr).dynamicCast<QGst::Pipeline>();
+ m_src.setElement(pipeline2->getElementByName("mysrc"));
+ QGlib::connect(pipeline2->bus(), "message", this, &Player::onBusMessage);
+ pipeline2->bus()->addSignalWatch();
+ /* start playing */
+ pipeline1->setState(QGst::StatePlaying);
+ pipeline2->setState(QGst::StatePlaying);
+}
+Player::~Player()
+{
+ pipeline1->setState(QGst::StateNull);
+ pipeline2->setState(QGst::StateNull);
+}
+void Player::onBusMessage(const QGst::MessagePtr & message)
+{
+ switch (message->type()) {
+ case QGst::MessageEos:
+ quit();
+ break;
+ case QGst::MessageError:
+ qCritical() << message.staticCast<QGst::ErrorMessage>()->error();
+ break;
+ default:
+ break;
+ }
+}
+
+int main(int argc, char **argv)
+{
+ Player p(argc, argv);
+ return p.exec();
+}
+```
+
+### Walkthrough
+
+As this is a very simple example, most of the action happens in the
+`Player`'s constructor. First, GStreamer is initialized through
+`QGst::init()`:
+
+**GStreamer Initialization**
+
+``` c
+ QGst::init(&argc, &argv);
+```
+
+Now we can construct the first half of the pipeline:
+
+**Pipeline Setup**
+
+``` c
+ const char *caps = "audio/x-raw-int,channels=1,rate=8000,"
+ "signed=(boolean)true,width=16,depth=16,endianness=1234";
+
+ /* source pipeline */
+ QString pipe1Descr = QString("filesrc location=\"%1\" ! "
+ "decodebin2 ! "
+ "audioconvert ! "
+ "audioresample ! "
+ "appsink name=\"mysink\" caps=\"%2\"").arg(argv[1], caps);
+ pipeline1 = QGst::Parse::launch(pipe1Descr).dynamicCast<QGst::Pipeline>();
+ m_sink.setElement(pipeline1->getElementByName("mysink"));
+ QGlib::connect(pipeline1->bus(), "message::error", this, &Player::onBusMessage);
+ pipeline1->bus()->addSignalWatch();
+```
+
+`QGst::Parse::launch()` parses the text description of a pipeline and
+returns a `QGst::PipelinePtr`. In this case, the pipeline is composed
+of:
+
+ - A `filesrc` element to read the file
+ - `decodebin2` to automatically examine the stream and pick the right
+ decoder(s)
+ - `audioconvert` and `audioresample` to convert the output of the
+ `decodebin2` into the caps specified for the `appsink`
+ - An `appsink` element with specific caps
+
+Next, we tell our `MySink` class (which is a subclass
+of `QGst::Utils::ApplicationSink`) what `appsink` element to use.
+
+The second half of the pipeline is created similarly:
+
+**Second Pipeline**
+
+``` c
+ /* sink pipeline */
+ QString pipe2Descr = QString("appsrc name=\"mysrc\" caps=\"%1\" ! autoaudiosink").arg(caps);
+ pipeline2 = QGst::Parse::launch(pipe2Descr).dynamicCast<QGst::Pipeline>();
+ m_src.setElement(pipeline2->getElementByName("mysrc"));
+ QGlib::connect(pipeline2->bus(), "message", this, &Player::onBusMessage);
+ pipeline2->bus()->addSignalWatch();
+```
+
+Finally, the pipeline is started:
+
+**Starting the pipeline**
+
+``` c
+ /* start playing */
+ pipeline1->setState(QGst::StatePlaying);
+ pipeline2->setState(QGst::StatePlaying);
+```
+
+Once the pipelines are started, the first one begins pushing buffers
+into the `appsink` element. Our `MySink` class implements the
+`newBuffer()` method, which is called by QGStreamer when a new buffer is
+ready for processing:
+
+**MySink::newBuffer()**
+
+``` c
+ virtual QGst::FlowReturn newBuffer()
+ {
+ m_src->pushBuffer(pullBuffer());
+ return QGst::FlowOk;
+ }
+```
+
+Our implementation takes the new buffer and pushes it into the
+`appsrc` element, which got assigned in the `Player` constructor:
+
+**Player::Player()**
+
+``` c
+Player::Player(int argc, char **argv)
+ : QCoreApplication(argc, argv), m_sink(&m_src)
+```
+
+From there, buffers flow into the `autoaudiosink` element, which
+automatically figures out a way to send it to your speakers.
+
+## Conclusion
+
+You should now have an understanding of how to push and pull arbitrary
+data into and out of a GStreamer pipeline.
+
+It has been a pleasure having you here, and see you soon\!
--- /dev/null
+# Windows deployment
+
+This page explains how to deploy GStreamer along your
+application. There are different mechanisms, which have been reviewed
+in [](sdk-deploying-your-application.md). The details for some of the
+mechanisms are given here, and more options might be added to this
+documentation in the future.
+
+## Shared GStreamer
+
+This is the easiest way to deploy GStreamer, although most of the time
+it installs unnecessary files which grow the size of the installer and
+the target drive free space requirements. Since GStreamer might be shared
+among all applications that use it, though, the extra space requirements
+are somewhat blurred.
+
+Simply pack GStreamer **runtime** installer ([the same one you
+installed in your development machine](sdk-installing-on-windows.md))
+inside your installer (or download it from your installer) and execute
+it silently using `msiexec`. `msiexec` is the tool that wraps most of
+the Windows Installer functionality and offers a number of options to
+suit your needs. You can review these options by
+executing `msiexec` without parameters. For example using 1.8.1:
+
+```
+msiexec /i gstreamer-1.0-x86-1.8.1.msi
+```
+
+This will bring up the installation dialog as if the user had
+double-clicked on the `msi` file. Usually, you will want to let the user
+choose where they want to install GStreamer. An environment variable will
+let your application locate it later on.
+
+## Private deployment of GStreamer
+
+You can use the same method as the shared GStreamer, but instruct its
+installer to deploy to your application’s folder (or a
+subfolder). Again, use the `msiexec` parameters that suit you best. For
+example:
+
+```
+msiexec /passive INSTALLDIR=C:\Desired\Folder /i gstreamer-1.0-x86-1.8.1.msi
+```
+
+This will install GStreamer to `C:\Desired\Folder` showing a progress
+dialog, but not requiring user intervention.
+
+## Deploy only necessary files, by manually picking them
+
+On the other side of the spectrum, if you want to reduce the space
+requirements (and installer size) to the maximum, you can manually
+choose which GStreamer libraries to deploy. Unfortunately, you are on
+your own on this road, besides using the [Dependency
+Walker](http://www.dependencywalker.com/) tool to discover inter-DLL
+dependencies.
+
+Bear in mind that GStreamer is modular in nature. Plug-ins are loaded
+depending on the media that is being played, so, if you do not know in
+advance what files you are going to play, you do not know which DLLs you
+need to deploy.
+
+## Deploy only necessary packages, using provided Merge Modules
+
+If you are building your installer using one of the Professional
+editions of [Visual
+Studio](http://www.microsoft.com/visualstudio/en-us/products/2010-editions/professional/overview)
+or [WiX](http://wix.sf.net) you can take advantage of pre-packaged
+[Merge
+Modules](http://msdn.microsoft.com/en-us/library/windows/desktop/aa369820\(v=vs.85\).aspx).
+GStreamer is divided in packages, which roughly take care of
+different tasks. There is the core package, the playback package, the
+networking package, etc. Each package contains the necessary libraries
+and files to accomplish its task.
+
+The Merge Modules are pieces that can be put together to build a larger
+Windows Installer. In this case, you just need to create a deployment
+project for your application with Visual Studio and then add the Merge
+Modules for the GStreamer packages your application needs.
+
+This will produce a smaller installer than deploying the complete
+GStreamer, without the added burden of having to manually pick each
+library. You just need to know which packages your application requires.
+
+| Package name | Dependencies | Licenses | Description |
+|--------------|--------------|----------|-------------|
+| base-system-1.0 | |JPEG, FreeType, BSD-like, LGPL, LGPL-2+, LGPL-2.1, LibPNG and MIT | Base system dependencies |
+| gstreamer-1.0-capture | gstreamer-1.0-core, gstreamer-1.0-encoding | LGPL and LGPL-2+ | GStreamer plugins for capture |
+| gstreamer-1.0-codecs | base-crypto, gstreamer-1.0-core | BSD, Jasper-2.0, BSD-like, LGPL, LGPL-2, LGPL-2+, LGPL-2.1 and LGPL-2.1+ | GStreamer codecs |
+| gstreamer-1.0-codecs-gpl | gstreamer-1.0-core | BSD-like, LGPL, LGPL-2+ and LGPL-2.1+ | GStreamer codecs under the GPL license and/or with patents issues |
+| gstreamer-1.0-core | base-system-1.0 | LGPL and LGPL-2+ | GStreamer core |
+| gstreamer-1.0-dvd | gstreamer-1.0-core | GPL-2+, LGPL and LGPL-2+ | GStreamer DVD support |
+| gstreamer-1.0-effects | gstreamer-1.0-core | LGPL and LGPL-2+ | GStreamer effects and instrumentation plugins |
+| gstreamer-1.0-net | base-crypto, gstreamer-1.0-core | GPL-3, LGPL, LGPL-2+, LGPL-2.1+ and LGPL-3+ | GStreamer plugins for network protocols |
+| gstreamer-1.0-playback | gstreamer-1.0-core | LGPL and LGPL-2+ | GStreamer plugins for playback |
+| gstreamer-1.0-system | gstreamer-1.0-core | LGPL, LGPL-2+ and LGPL-2.1+ | GStreamer system plugins |
+| gstreamer-1.0-visualizers | gstreamer-1.0-core | LGPL and LGPL-2+ | GStreamer visualization plugins |
+| gstreamer-1.0-encoding | gstreamer-1.0-core, gstreamer-1.0-playback | LGPL and LGPL2+ | GStreamer plugins for encoding |
+| gstreamer-1.0-editing | gstreamer-1.0-core, gstreamer-1.0-devtools | LGPL and LGPL2+ | GStreamer libraries and plugins for non linear editing |
+| gstreamer-1.0-devtools | gstreamer-1.0-core | LGPL and LGPL2+ | GStreamer developers tools |
+| gstreamer-1.0-libav | gstreamer-1.0-core | LGPL and LGPL2+ | GStreamer plugins wrapping ffmpeg |
+| gstreamer-1.0-net-restricted | base-crypto, gstreamer-1.0-core | LGPL and LGPL2+ | GStreamer plugins for network protocols with potential patent issues in some countries |
+| gstreamer-1.0-codecs-restricted | gstreamer-1.0-core | LGPL and LGPL2+ | GStreamer restricted codecs with potential patent issues in some countries |
+| base-crypto | base-system-1.0 | LGPL and LGPL2+ | Cryptographic libraries |
+
+
+If you include a merge module in your deployment project, remember to
+include also its dependencies. Otherwise, the project will build
+correctly and install flawlessly, but, when executing your application,
+it will miss files.
\ No newline at end of file
--- /dev/null
+index.md
+ sdk-installing.md
+ sdk-installing-for-android-development.md
+ sdk-installing-for-ios-development.md
+ sdk-installing-on-mac-osx.md
+ sdk-installing-on-windows.md
+ sdk-installing-on-linux.md
+ sdk-building-from-source-using-cerbero.md
+ sdk-tutorials.md
+ sdk-basic-tutorials.md
+ sdk-basic-tutorial-hello-world.md
+ sdk-basic-tutorial-concepts.md
+ sdk-basic-tutorial-dynamic-pipelines.md
+ sdk-basic-tutorial-time-management.md
+ sdk-basic-tutorial-toolkit-integration.md
+ sdk-basic-tutorial-media-formats-and-pad-capabilities.md
+ sdk-basic-tutorial-multithreading-and-pad-availability.md
+ sdk-basic-tutorial-short-cutting-the-pipeline.md
+ sdk-basic-tutorial-media-information-gathering.md
+ sdk-basic-tutorial-gstreamer-tools.md
+ sdk-basic-tutorial-debugging-tools.md
+ sdk-basic-tutorial-streaming.md
+ sdk-basic-tutorial-playback-speed.md
+ sdk-basic-tutorial-handy-elements.md
+ sdk-basic-tutorial-platform-specific-elements.md
+ sdk-playback-tutorials.md
+ sdk-playback-tutorial-playbin-usage.md
+ sdk-playback-tutorial-subtitle-management.md
+ sdk-playback-tutorial-short-cutting-the-pipeline.md
+ sdk-playback-tutorial-progressive-streaming.md
+ sdk-playback-tutorial-color-balance.md
+ sdk-playback-tutorial-audio-visualization.md
+ sdk-playback-tutorial-custom-playbin-sinks.md
+ sdk-playback-tutorial-hardware-accelerated-video-decoding.md
+ sdk-playback-tutorial-digital-audio-pass-through.md
+ sdk-android-tutorials.md
+ sdk-android-tutorial-link-against-gstreamer.md
+ sdk-android-tutorial-a-running-pipeline.md
+ sdk-android-tutorial-video.md
+ sdk-android-tutorial-media-player.md
+ sdk-android-tutorial-a-complete-media-player.md
+ sdk-ios-tutorials.md
+ sdk-ios-tutorial-link-against-gstreamer.md
+ sdk-ios-tutorial-a-running-pipeline.md
+ sdk-ios-tutorial-video.md
+ sdk-ios-tutorial-a-basic-media-player.md
+ sdk-ios-tutorial-a-complete-media-player.md
+ sdk-table-of-concepts.md
+ sdk-deploying-your-application.md
+ sdk-mac-osx-deployment.md
+ sdk-windows-deployment.md
+ sdk-multiplatform-deployment-using-cerbero.md
+ gstreamer-command-line-tools.md
+ gst-inspect.md
+ gst-launch.md
+ ges-launch.md
+ manual-index.md
+ manual-introduction.md
+ manual-gstreamer.md
+ manual-motivation.md
+ manual-intro-basics.md
+ manual-building.md
+ manual-init.md
+ manual-elements.md
+ manual-bins.md
+ manual-bus.md
+ manual-pads.md
+ manual-data.md
+ manual-helloworld.md
+ manual-advanced.md
+ manual-queryevents.md
+ manual-metadata.md
+ manual-interfaces.md
+ manual-clocks.md
+ manual-buffering.md
+ manual-dparams.md
+ manual-threads.md
+ manual-autoplugging.md
+ manual-dataaccess.md
+ manual-highlevel.md
+ manual-playback-components.md
+ manual-appendices.md
+ manual-programs.md
+ manual-compiling.md
+ manual-checklist-element.md
+ manual-porting.md
+ manual-porting-1.0.md
+ manual-integration.md
+ manual-licensing.md
+ manual-quotes.md
+ pwg-index.md
+ pwg-introduction.md
+ pwg-intro-preface.md
+ pwg-intro-basics.md
+ pwg-building.md
+ pwg-building-boiler.md
+ pwg-building-pads.md
+ pwg-building-chainfn.md
+ pwg-building-eventfn.md
+ pwg-building-queryfn.md
+ pwg-statemanage-states.md
+ pwg-building-args.md
+ pwg-building-signals.md
+ pwg-building-testapp.md
+ pwg-advanced.md
+ pwg-advanced-request.md
+ pwg-scheduling.md
+ pwg-negotiation.md
+ pwg-allocation.md
+ pwg-building-types.md
+ pwg-advanced-events.md
+ pwg-advanced-clock.md
+ pwg-advanced-qos.md
+ pwg-dparams.md
+ pwg-advanced-interfaces.md
+ pwg-advanced-tagging.md
+ pwg-other.md
+ pwg-other-base.md
+ pwg-other-oneton.md
+ pwg-other-ntoone.md
+ pwg-other-manager.md
+ pwg-appendix.md
+ pwg-checklist-element.md
+ pwg-porting.md
+ pwg-porting-1_0.md
+ pwg-licensing-advisory.md
+ sdk-legal-information.md
projects / platform / upstream / gstreamer.git / commitdiff
