1 <chapter id="chapter-dataaccess">
2 <title>Pipeline manipulation</title>
4 This chapter will discuss how you can manipulate your pipeline in several
5 ways from your application on. Parts of this chapter are downright
6 hackish, so be assured that you'll need some programming knowledge
7 before you start reading this.
10 Topics that will be discussed here include how you can insert data into
11 a pipeline from your application, how to read data from a pipeline,
12 how to manipulate the pipeline's speed, length, starting point and how
13 to listen to a pipeline's data processing.
16 <sect1 id="section-data-probe">
17 <title>Data probing</title>
19 Probing is best envisioned as a pad listener. Technically, a probe is
20 nothing more than a signal callback that can be attached to a pad.
21 Those signals are by default not fired at all (since that may have a
22 negative impact on performance), but can be enabled by attaching a
23 probe using <function>gst_pad_add_buffer_probe ()</function>,
24 <function>gst_pad_add_event_probe ()</function>, or
25 <function>gst_pad_add_data_probe ()</function>.
26 Those functions attach the signal handler and
27 enable the actual signal emission. Similarly, one can use the
28 <function>gst_pad_remove_buffer_probe ()</function>,
29 <function>gst_pad_remove_event_probe ()</function>, or
30 <function>gst_pad_remove_data_probe ()</function>
31 to remove the signal handlers again.
34 Probes run in pipeline threading context, so callbacks should try to
35 not block and generally not do any weird stuff, since this could
36 have a negative impact on pipeline performance or, in case of bugs,
37 cause deadlocks or crashes. More precisely, one should usually not
38 call any GUI-related functions from within a probe callback, nor try
39 to change the state of the pipeline. An application may post custom
40 messages on the pipeline's bus though to communicate with the main
41 application thread and have it do things like stop the pipeline.
44 In any case, most common buffer operations
45 that elements can do in <function>_chain ()</function> functions, can
46 be done in probe callbacks as well. The example below gives a short
47 impression on how to use them (even if this usage is not entirely
48 correct, but more on that below):
50 <programlisting><!-- example-begin probe.c -->
51 #include <gst/gst.h>
54 cb_have_data (GstPad *pad,
59 guint16 *data = (guint16 *) GST_BUFFER_DATA (buffer), t;
62 for (y = 0; y < 288; y++) {
63 for (x = 0; x < 384 / 2; x++) {
64 t = data[384 - 1 - x];
65 data[384 - 1 - x] = data[x];
79 GstElement *pipeline, *src, *sink, *filter, *csp;
84 gst_init (&argc, &argv);
85 loop = g_main_loop_new (NULL, FALSE);
88 pipeline = gst_pipeline_new ("my-pipeline");
89 src = gst_element_factory_make ("videotestsrc", "src");
91 g_error ("Could not create 'videotestsrc' element");
93 filter = gst_element_factory_make ("capsfilter", "filter");
94 g_assert (filter != NULL); /* should always exist */
96 csp = gst_element_factory_make ("ffmpegcolorspace", "csp");
98 g_error ("Could not create 'ffmpegcolorspace' element");
100 sink = gst_element_factory_make ("xvimagesink", "sink");
102 sink = gst_element_factory_make ("ximagesink", "sink");
104 g_error ("Could not create neither 'xvimagesink' nor 'ximagesink' element");
107 gst_bin_add_many (GST_BIN (pipeline), src, filter, csp, sink, NULL);
108 gst_element_link_many (src, filter, csp, sink, NULL);
109 filtercaps = gst_caps_new_simple ("video/x-raw-rgb",
110 "width", G_TYPE_INT, 384,
111 "height", G_TYPE_INT, 288,
112 "framerate", GST_TYPE_FRACTION, 25, 1,
113 "bpp", G_TYPE_INT, 16,
114 "depth", G_TYPE_INT, 16,
115 "endianness", G_TYPE_INT, G_BYTE_ORDER,
117 g_object_set (G_OBJECT (filter), "caps", filtercaps, NULL);
118 gst_caps_unref (filtercaps);
120 pad = gst_element_get_pad (src, "src");
121 gst_pad_add_buffer_probe (pad, G_CALLBACK (cb_have_data), NULL);
122 gst_object_unref (pad);
125 gst_element_set_state (pipeline, GST_STATE_PLAYING);
127 /* wait until it's up and running or failed */
128 if (gst_element_get_state (pipeline, NULL, NULL, -1) == GST_STATE_CHANGE_FAILURE) {
129 g_error ("Failed to go into PLAYING state");
132 g_print ("Running ...\n");
133 g_main_loop_run (loop);
136 gst_element_set_state (pipeline, GST_STATE_NULL);
137 gst_object_unref (pipeline);
141 <!-- example-end probe.c --></programlisting>
143 Compare that output with the output of <quote>gst-launch-0.10
144 videotestsrc ! xvimagesink</quote>, just so you know what you're
148 The above example is not really correct though. Strictly speaking, a
149 pad probe callback is only allowed to modify the buffer content if the
150 buffer is writable, and it is only allowed to modify buffer metadata like
151 timestamps, caps, etc. if the buffer metadata is writable. Whether this
152 is the case or not depends a lot on the pipeline and the elements
153 involved. Often enough, this is the case, but sometimes it is not,
154 and if it is not then unexpected modification of the data or metadata
155 can introduce bugs that are very hard to debug and track down. You can
156 check if a buffer and its metadata are writable with
157 <function>gst_buffer_is_writable ()</function> and
158 <function>gst_buffer_is_metadata_writable ()</function>. Since you
159 can't pass back a different buffer than the one passed in, there is no
160 point of making a buffer writable in the callback function.
163 Pad probes are suited best for looking at data as it passes through
164 the pipeline. If you need to modify data, you should write your own
165 GStreamer element. Base classes like GstAudioFilter, GstVideoFilter or
166 GstBaseTransform make this fairly easy.
169 If you just want to inspect buffers as they pass through the pipeline,
170 you don't even need to set up pad probes. You could also just insert
171 an identity element into the pipeline and connect to its "handoff"
172 signal. The identity element also provides a few useful debugging tools
173 like the "dump" property or the "last-message" property (the latter is
174 enabled by passing the '-v' switch to gst-launch).
178 <sect1 id="section-data-spoof">
179 <title>Manually adding or removing data from/to a pipeline</title>
181 Many people have expressed the wish to use their own sources to inject
182 data into a pipeline. Some people have also expressed the wish to grab
183 the output in a pipeline and take care of the actual output inside
184 their application. While either of these methods are stongly
185 discouraged, &GStreamer; offers hacks to do this. <emphasis>However,
186 there is no support for those methods.</emphasis> If it doesn't work,
187 you're on your own. Also, synchronization, thread-safety and other
188 things that you've been able to take for granted so far are no longer
189 guaranteed if you use any of those methods. It's always better to
190 simply write a plugin and have the pipeline schedule and manage it.
191 See the Plugin Writer's Guide for more information on this topic. Also
192 see the next section, which will explain how to embed plugins statically
197 url="http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gstreamer-app.html">New
198 API</ulink> was developed to make data insertion and extraction easy
199 for applications. It can be found as GstAppSrc and GstAppSink in the
201 url="http://gstreamer.freedesktop.org/modules/gst-plugins-base.html">
202 gst-plugins-base</ulink> module.
205 After all those disclaimers, let's start. There's three possible
206 elements that you can use for the above-mentioned purposes. Those are
207 called <quote>fakesrc</quote> (an imaginary source),
208 <quote>fakesink</quote> (an imaginary sink) and <quote>identity</quote>
209 (an imaginary filter). The same method applies to each of those
210 elements. Here, we will discuss how to use those elements to insert
211 (using fakesrc) or grab (using fakesink or identity) data from a
212 pipeline, and how to set negotiation.
215 Those who're paying close attention will notice that the purpose
216 of identity is almost identical to that of probes. Indeed, this is
217 true. Probes allow for the same purpose, and a bunch more, and
218 with less overhead plus dynamic removing/adding of handlers, but
219 apart from those, probes and identity have the same purpose, just
220 in a completely different implementation type.
223 <sect2 id="section-spoof-handoff">
224 <title>Inserting or grabbing data</title>
226 The three before-mentioned elements (fakesrc, fakesink and identity)
227 each have a <quote>handoff</quote> signal that will be called in
228 the <function>_get ()</function>- (fakesrc) or <function>_chain
229 ()</function>-function (identity, fakesink). In the signal handler,
230 you can set (fakesrc) or get (identity, fakesink) data to/from the
231 provided buffer. Note that in the case of fakesrc, you have to set
232 the size of the provided buffer using the <quote>sizemax</quote>
233 property. For both fakesrc and fakesink, you also have to set the
234 <quote>signal-handoffs</quote> property for this method to work.
237 Note that your handoff function should <emphasis>not</emphasis>
238 block, since this will block pipeline iteration. Also, do not try
239 to use all sort of weird hacks in such functions to accomplish
240 something that looks like synchronization or so; it's not the right
241 way and will lead to issues elsewhere. If you're doing any of this,
242 you're basically misunderstanding the &GStreamer; design.
246 <sect2 id="section-spoof-format">
247 <title>Forcing a format</title>
249 Sometimes, when using fakesrc as a source in your pipeline, you'll
250 want to set a specific format, for example a video size and format
251 or an audio bitsize and number of channels. You can do this by
252 forcing a specific <classname>GstCaps</classname> on the pipeline,
253 which is possible by using <emphasis>filtered caps</emphasis>. You
254 can set a filtered caps on a link by using the
255 <quote>capsfilter</quote> element in between the two elements, and
256 specifying a <classname>GstCaps</classname> as
257 <quote>caps</quote> property on this element. It will then
258 only allow types matching that specified capability set for
259 negotiation. See also <xref linkend="section-caps-filter"/>.
263 <sect2 id="section-spoof-example">
264 <title>Example application</title>
266 This example application will generate black/white (it switches
267 every second) video to an X-window output by using fakesrc as a
268 source and using filtered caps to force a format. Since the depth
269 of the image depends on your X-server settings, we use a colorspace
270 conversion element to make sure that the output to your X server
271 will have the correct bitdepth. You can also set timestamps on the
272 provided buffers to override the fixed framerate.
274 <programlisting><!-- example-begin fakesrc.c -->
275 #include <string.h> /* for memset () */
276 #include <gst/gst.h>
279 cb_handoff (GstElement *fakesrc,
284 static gboolean white = FALSE;
286 /* this makes the image black/white */
287 memset (GST_BUFFER_DATA (buffer), white ? 0xff : 0x0,
288 GST_BUFFER_SIZE (buffer));
296 GstElement *pipeline, *fakesrc, *flt, *conv, *videosink;
300 gst_init (&argc, &argv);
301 loop = g_main_loop_new (NULL, FALSE);
304 pipeline = gst_pipeline_new ("pipeline");
305 fakesrc = gst_element_factory_make ("fakesrc", "source");
306 flt = gst_element_factory_make ("capsfilter", "flt");
307 conv = gst_element_factory_make ("ffmpegcolorspace", "conv");
308 videosink = gst_element_factory_make ("xvimagesink", "videosink");
311 g_object_set (G_OBJECT (flt), "caps",
312 gst_caps_new_simple ("video/x-raw-rgb",
313 "width", G_TYPE_INT, 384,
314 "height", G_TYPE_INT, 288,
315 "framerate", GST_TYPE_FRACTION, 1, 1,
316 "bpp", G_TYPE_INT, 16,
317 "depth", G_TYPE_INT, 16,
318 "endianness", G_TYPE_INT, G_BYTE_ORDER,
320 gst_bin_add_many (GST_BIN (pipeline), fakesrc, flt, conv, videosink, NULL);
321 gst_element_link_many (fakesrc, flt, conv, videosink, NULL);
323 /* setup fake source */
324 g_object_set (G_OBJECT (fakesrc),
325 "signal-handoffs", TRUE,
326 "sizemax", 384 * 288 * 2,
327 "sizetype", 2, NULL);
328 g_signal_connect (fakesrc, "handoff", G_CALLBACK (cb_handoff), NULL);
331 gst_element_set_state (pipeline, GST_STATE_PLAYING);
332 g_main_loop_run (loop);
335 gst_element_set_state (pipeline, GST_STATE_NULL);
336 gst_object_unref (GST_OBJECT (pipeline));
340 <!-- example-end fakesrc.c --></programlisting>
344 <sect1 id="section-data-manager">
345 <title>Embedding static elements in your application</title>
347 The <ulink type="http"
348 url="http://gstreamer.freedesktop.org/data/doc/gstreamer/head/pwg/html/index.html">Plugin
349 Writer's Guide</ulink> describes in great detail how to write elements
350 for the &GStreamer; framework. In this section, we will solely discuss
351 how to embed such elements statically in your application. This can be
352 useful for application-specific elements that have no use elsewhere in
356 Dynamically loaded plugins contain a structure that's defined using
357 <function>GST_PLUGIN_DEFINE ()</function>. This structure is loaded
358 when the plugin is loaded by the &GStreamer; core. The structure
359 contains an initialization function (usually called
360 <function>plugin_init</function>) that will be called right after that.
361 It's purpose is to register the elements provided by the plugin with
362 the &GStreamer; framework. If you want to embed elements directly in
363 your application, the only thing you need to do is to replace
364 <function>GST_PLUGIN_DEFINE ()</function> with
365 <function>GST_PLUGIN_DEFINE_STATIC ()</function>. This will cause the
366 elements to be registered when your application loads, and the elements
367 will from then on be available like any other element, without them
368 having to be dynamically loadable libraries. In the example below, you
369 would be able to call <function>gst_element_factory_make
370 ("my-element-name", "some-name")</function> to create an instance of the
376 * Here, you would write the actual plugin code.
382 register_elements (GstPlugin *plugin)
384 return gst_element_register (plugin, "my-element-name",
385 GST_RANK_NONE, MY_PLUGIN_TYPE);
388 GST_PLUGIN_DEFINE_STATIC (
391 "my-private-plugins",
392 "Private elements of my application",
397 "http://www.my-application.net/"