1 # Basic tutorial 7: Multithreading and Pad Availability
5 GStreamer handles multithreading automatically, but, under some
6 circumstances, you might need to decouple threads manually. This
7 tutorial shows how to do this and, in addition, completes the exposition
8 about Pad Availability. More precisely, this document explains:
10 - How to create new threads of execution for some parts of the
13 - What is the Pad Availability
15 - How to replicate streams
21 GStreamer is a multithreaded framework. This means that, internally, it
22 creates and destroys threads as it needs them, for example, to decouple
23 streaming from the application thread. Moreover, plugins are also free
24 to create threads for their own processing, for example, a video decoder
25 could create 4 threads to take full advantage of a CPU with 4 cores.
27 On top of this, when building the pipeline an application can specify
28 explicitly that a *branch* (a part of the pipeline) runs on a different
29 thread (for example, to have the audio and video decoders executing
32 This is accomplished using the `queue` element, which works as follows.
33 The sink pad just enqueues data and returns control. On a different
34 thread, data is dequeued and pushed downstream. This element is also
35 used for buffering, as seen later in the streaming tutorials. The size
36 of the queue can be controlled through properties.
38 ### The example pipeline
40 This example builds the following pipeline:
42 ![](attachments/basic-tutorial-7.png)
44 The source is a synthetic audio signal (a continuous tone) which is
45 split using a `tee` element (it sends through its source pads everything
46 it receives through its sink pad). One branch then sends the signal to
47 the audio card, and the other renders a video of the waveform and sends
50 As seen in the picture, queues create a new thread, so this pipeline
51 runs in 3 threads. Pipelines with more than one sink usually need to be
52 multithreaded, because, to be synchronized, sinks usually block
53 execution until all other sinks are ready, and they cannot get ready if
54 there is only one thread, being blocked by the first sink.
58 In [Basic tutorial 3: Dynamic
59 pipelines](sdk-basic-tutorial-dynamic-pipelines.md) we saw
60 an element (`uridecodebin`) which had no pads to begin with, and they
61 appeared as data started to flow and the element learned about the
62 media. These are called **Sometimes Pads**, and contrast with the
63 regular pads which are always available and are called **Always Pads**.
65 The third kind of pad is the **Request Pad**, which is created on
66 demand. The classical example is the `tee` element, which has one sink
67 pad and no initial source pads: they need to be requested and then
68 `tee` adds them. In this way, an input stream can be replicated any
69 number of times. The disadvantage is that linking elements with Request
70 Pads is not as automatic, as linking Always Pads, as the walkthrough for
71 this example will show.
73 Also, to request (or release) pads in the PLAYING or PAUSED states, you
74 need to take additional cautions (Pad blocking) which are not described
75 in this tutorial. It is safe to request (or release) pads in the NULL or
78 Without further delay, let's see the code.
80 ## Simple multithreaded example
82 Copy this code into a text file named `basic-tutorial-7.c` (or find it
83 in the SDK installation).
85 **basic-tutorial-7.c**
90 int main(int argc, char *argv[]) {
91 GstElement *pipeline, *audio_source, *tee, *audio_queue, *audio_convert, *audio_resample, *audio_sink;
92 GstElement *video_queue, *visual, *video_convert, *video_sink;
95 GstPadTemplate *tee_src_pad_template;
96 GstPad *tee_audio_pad, *tee_video_pad;
97 GstPad *queue_audio_pad, *queue_video_pad;
99 /* Initialize GStreamer */
100 gst_init (&argc, &argv);
102 /* Create the elements */
103 audio_source = gst_element_factory_make ("audiotestsrc", "audio_source");
104 tee = gst_element_factory_make ("tee", "tee");
105 audio_queue = gst_element_factory_make ("queue", "audio_queue");
106 audio_convert = gst_element_factory_make ("audioconvert", "audio_convert");
107 audio_resample = gst_element_factory_make ("audioresample", "audio_resample");
108 audio_sink = gst_element_factory_make ("autoaudiosink", "audio_sink");
109 video_queue = gst_element_factory_make ("queue", "video_queue");
110 visual = gst_element_factory_make ("wavescope", "visual");
111 video_convert = gst_element_factory_make ("videoconvert", "csp");
112 video_sink = gst_element_factory_make ("autovideosink", "video_sink");
114 /* Create the empty pipeline */
115 pipeline = gst_pipeline_new ("test-pipeline");
117 if (!pipeline || !audio_source || !tee || !audio_queue || !audio_convert || !audio_resample || !audio_sink ||
118 !video_queue || !visual || !video_convert || !video_sink) {
119 g_printerr ("Not all elements could be created.\n");
123 /* Configure elements */
124 g_object_set (audio_source, "freq", 215.0f, NULL);
125 g_object_set (visual, "shader", 0, "style", 1, NULL);
127 /* Link all elements that can be automatically linked because they have "Always" pads */
128 gst_bin_add_many (GST_BIN (pipeline), audio_source, tee, audio_queue, audio_convert, audio_resample, audio_sink,
129 video_queue, visual, video_convert, video_sink, NULL);
130 if (gst_element_link_many (audio_source, tee, NULL) != TRUE ||
131 gst_element_link_many (audio_queue, audio_convert, audio_resample, audio_sink, NULL) != TRUE ||
132 gst_element_link_many (video_queue, visual, video_convert, video_sink, NULL) != TRUE) {
133 g_printerr ("Elements could not be linked.\n");
134 gst_object_unref (pipeline);
138 /* Manually link the Tee, which has "Request" pads */
139 tee_src_pad_template = gst_element_class_get_pad_template (GST_ELEMENT_GET_CLASS (tee), "src_%d");
140 tee_audio_pad = gst_element_request_pad (tee, tee_src_pad_template, NULL, NULL);
141 g_print ("Obtained request pad %s for audio branch.\n", gst_pad_get_name (tee_audio_pad));
142 queue_audio_pad = gst_element_get_static_pad (audio_queue, "sink");
143 tee_video_pad = gst_element_request_pad (tee, tee_src_pad_template, NULL, NULL);
144 g_print ("Obtained request pad %s for video branch.\n", gst_pad_get_name (tee_video_pad));
145 queue_video_pad = gst_element_get_static_pad (video_queue, "sink");
146 if (gst_pad_link (tee_audio_pad, queue_audio_pad) != GST_PAD_LINK_OK ||
147 gst_pad_link (tee_video_pad, queue_video_pad) != GST_PAD_LINK_OK) {
148 g_printerr ("Tee could not be linked.\n");
149 gst_object_unref (pipeline);
152 gst_object_unref (queue_audio_pad);
153 gst_object_unref (queue_video_pad);
155 /* Start playing the pipeline */
156 gst_element_set_state (pipeline, GST_STATE_PLAYING);
158 /* Wait until error or EOS */
159 bus = gst_element_get_bus (pipeline);
160 msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
162 /* Release the request pads from the Tee, and unref them */
163 gst_element_release_request_pad (tee, tee_audio_pad);
164 gst_element_release_request_pad (tee, tee_video_pad);
165 gst_object_unref (tee_audio_pad);
166 gst_object_unref (tee_video_pad);
170 gst_message_unref (msg);
171 gst_object_unref (bus);
172 gst_element_set_state (pipeline, GST_STATE_NULL);
174 gst_object_unref (pipeline);
179 > ![Information](images/icons/emoticons/information.png)
182 > If you need help to compile this code, refer to the **Building the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Build), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Build) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Build), or use this specific command on Linux:
184 > ``gcc basic-tutorial-7.c -o basic-tutorial-7 `pkg-config --cflags --libs gstreamer-1.0` ``
186 >If you need help to run this code, refer to the **Running the tutorials** section for your platform: [Linux](sdk-installing-on-linux.md#InstallingonLinux-Run), [Mac OS X](sdk-installing-on-mac-osx.md#InstallingonMacOSX-Run) or [Windows](sdk-installing-on-windows.md#InstallingonWindows-Run).
188 > This tutorial plays an audible tone through the audio card and opens a window with a waveform representation of the tone. The waveform should be a sinusoid, but due to the refreshing of the window might not appear so.
190 > Required libraries: `gstreamer-1.0`
195 /* Create the elements */
196 audio_source = gst_element_factory_make ("audiotestsrc", "audio_source");
197 tee = gst_element_factory_make ("tee", "tee");
198 audio_queue = gst_element_factory_make ("queue", "audio_queue");
199 audio_convert = gst_element_factory_make ("audioconvert", "audio_convert");
200 audio_resample = gst_element_factory_make ("audioresample", "audio_resample");
201 audio_sink = gst_element_factory_make ("autoaudiosink", "audio_sink");
202 video_queue = gst_element_factory_make ("queue", "video_queue");
203 visual = gst_element_factory_make ("wavescope", "visual");
204 video_convert = gst_element_factory_make ("videoconvert", "video_convert");
205 video_sink = gst_element_factory_make ("autovideosink", "video_sink");
208 All the elements in the above picture are instantiated here:
210 `audiotestsrc` produces a synthetic tone. `wavescope` consumes an audio
211 signal and renders a waveform as if it was an (admittedly cheap)
212 oscilloscope. We have already worked with the `autoaudiosink` and
215 The conversion elements (`audioconvert`, `audioresample` and
216 `videoconvert`) are necessary to guarantee that the pipeline can be
217 linked. Indeed, the Capabilities of the audio and video sinks depend on
218 the hardware, and you do not know at design time if they will match the
219 Caps produced by the `audiotestsrc` and `wavescope`. If the Caps
220 matched, though, these elements act in “pass-through” mode and do not
221 modify the signal, having negligible impact on performance.
224 /* Configure elements */
225 g_object_set (audio_source, "freq", 215.0f, NULL);
226 g_object_set (visual, "shader", 0, "style", 1, NULL);
229 Small adjustments for better demonstration: The “freq” property of
230 `audiotestsrc` controls the frequency of the wave (215Hz makes the wave
231 appear almost stationary in the window), and this style and shader for
232 `wavescope` make the wave continuous. Use the `gst-inspect-1.0` tool
233 described in [Basic tutorial 10: GStreamer
234 tools](sdk-basic-tutorial-gstreamer-tools.md) to learn all
235 the properties of these
239 /* Link all elements that can be automatically linked because they have "Always" pads */
240 gst_bin_add_many (GST_BIN (pipeline), audio_source, tee, audio_queue, audio_convert, audio_sink,
241 video_queue, visual, video_convert, video_sink, NULL);
242 if (gst_element_link_many (audio_source, tee, NULL) != TRUE ||
243 gst_element_link_many (audio_queue, audio_convert, audio_sink, NULL) != TRUE ||
244 gst_element_link_many (video_queue, visual, video_convert, video_sink, NULL) != TRUE) {
245 g_printerr ("Elements could not be linked.\n");
246 gst_object_unref (pipeline);
251 This code block adds all elements to the pipeline and then links the
252 ones that can be automatically linked (the ones with Always Pads, as the
255 > ![Warning](images/icons/emoticons/warning.png)
256 > `gst_element_link_many()` can actually link elements with Request Pads. It internally requests the Pads so you do not have worry about the elements being linked having Always or Request Pads. Strange as it might seem, this is actually inconvenient, because you still need to release the requested Pads afterwards, and, if the Pad was requested automatically by `gst_element_link_many()`, it is easy to forget. Stay out of trouble by always requesting Request Pads manually, as shown in the next code block.
259 /* Manually link the Tee, which has "Request" pads */
260 tee_src_pad_template = gst_element_class_get_pad_template (GST_ELEMENT_GET_CLASS (tee), "src_%d");
261 tee_audio_pad = gst_element_request_pad (tee, tee_src_pad_template, NULL, NULL);
262 g_print ("Obtained request pad %s for audio branch.\n", gst_pad_get_name (tee_audio_pad));
263 queue_audio_pad = gst_element_get_static_pad (audio_queue, "sink");
264 tee_video_pad = gst_element_request_pad (tee, tee_src_pad_template, NULL, NULL);
265 g_print ("Obtained request pad %s for video branch.\n", gst_pad_get_name (tee_video_pad));
266 queue_video_pad = gst_element_get_static_pad (video_queue, "sink");
267 if (gst_pad_link (tee_audio_pad, queue_audio_pad) != GST_PAD_LINK_OK ||
268 gst_pad_link (tee_video_pad, queue_video_pad) != GST_PAD_LINK_OK) {
269 g_printerr ("Tee could not be linked.\n");
270 gst_object_unref (pipeline);
273 gst_object_unref (queue_audio_pad);
274 gst_object_unref (queue_video_pad);
277 To link Request Pads, they need to be obtained by “requesting” them to
278 the element. An element might be able to produce different kinds of
279 Request Pads, so, when requesting them, the desired Pad Template must be
280 provided. Pad templates are obtained with
281 `gst_element_class_get_pad_template()` and are identified by their name.
282 In the documentation for the `tee` element we see that it has two pad
283 templates named “sink” (for its sink Pads) and “src_%d” (for the Request
286 Once we have the Pad template, we request two Pads from the tee (for the
287 audio and video branches) with `gst_element_request_pad()`.
289 We then obtain the Pads from the downstream elements to which these
290 Request Pads need to be linked. These are normal Always Pads, so we
291 obtain them with `gst_element_get_static_pad()`.
293 Finally, we link the pads with `gst_pad_link()`. This is the function
294 that `gst_element_link()` and `gst_element_link_many()` use internally.
296 The sink Pads we have obtained need to be released with
297 `gst_object_unref()`. The Request Pads will be released when we no
298 longer need them, at the end of the program.
300 We then set the pipeline to playing as usual, and wait until an error
301 message or an EOS is produced. The only thing left to so is cleanup the
305 /* Release the request pads from the Tee, and unref them */
306 gst_element_release_request_pad (tee, tee_audio_pad);
307 gst_element_release_request_pad (tee, tee_video_pad);
308 gst_object_unref (tee_audio_pad);
309 gst_object_unref (tee_video_pad);
312 `gst_element_release_request_pad()` releases the pad from the `tee`, but
313 it still needs to be unreferenced (freed) with `gst_object_unref()`.
317 This tutorial has shown:
319 - How to make parts of a pipeline run on a different thread by using
322 - What is a Request Pad and how to link elements with request pads,
323 with `gst_element_class_get_pad_template()`, `gst_element_request_pad()`, `gst_pad_link()` and
324 `gst_element_release_request_pad()`.
326 - How to have the same stream available in different branches by using
329 The next tutorial builds on top of this one to show how data can be
330 manually injected into and extracted from a running pipeline.
332 It has been a pleasure having you here, and see you soon!