1 # iOS tutorial 2: A running pipeline
7 As seen in the [Basic](tutorials/basic/index.md) and
8 [Playback](tutorials/playback/index.md) tutorials, GStreamer integrates
9 nicely with GLib’s main loops, so pipeline operation and user interface
10 can be monitored simultaneously in a very simple way. However, platforms
11 like iOS or Android do not use GLib and therefore extra care must be
12 taken to keep track of the pipeline progress without blocking the user
17 - How to move the GStreamer-handling code to a separate Dispatch Queue
18 whereas UI managing still happens from the Main Dispatch Queue
19 - How to communicate between the Objective-C UI code and the C
24 When using a Graphical User Interface (UI), if the application waits for
25 GStreamer calls to complete the user experience will suffer. The usual
26 approach, with the [GTK+ toolkit](http://www.gtk.org/) for example, is
27 to relinquish control to a GLib `GMainLoop` and let it control the
28 events coming from the UI or GStreamer.
30 Other graphical toolkits that are not based on GLib, like the [Cocoa
31 Touch](https://developer.apple.com/library/ios/documentation/General/Conceptual/DevPedia-CocoaCore/Cocoa.html)
32 framework used on iOS devices, cannot use this option, though. The
33 solution used in this tutorial uses a GLib `GMainLoop` for its
34 simplicity, but moves it to a separate thread (a [Dispatch
35 Queue](http://developer.apple.com/library/ios/#documentation/General/Conceptual/ConcurrencyProgrammingGuide/OperationQueues/OperationQueues.html)
36 different than the main one) so it does not block the user interface
39 Additionally, this tutorial shows a few places where caution has to be
40 taken when calling from Objective-C to C and vice versa.
42 The code below builds a pipeline with an `audiotestsrc` and
43 an `autoaudiosink` (it plays an audible tone). Two buttons in the UI
44 allow setting the pipeline to PLAYING or PAUSED. A Label in the UI shows
45 messages sent from the C code (for errors and state changes).
49 A toolbar at the bottom of the screen contains a Play and a Pause
50 button. Over the toolbar there is a Label used to display messages from
51 GStreamer. This tutorial does not require more elements, but the
52 following lessons will build their User Interfaces on top of this one,
53 adding more components.
55 ## The View Controller
57 The `ViewController` class manages the UI, instantiates
58 the `GStreamerBackend` and also performs some UI-related tasks on its
64 #import "ViewController.h"
65 #import "GStreamerBackend.h"
66 #import <UIKit/UIKit.h>
68 @interface ViewController () {
69 GStreamerBackend *gst_backend;
74 @implementation ViewController
77 * Methods from UIViewController
84 play_button.enabled = FALSE;
85 pause_button.enabled = FALSE;
87 gst_backend = [[GStreamerBackend alloc] init:self];
90 - (void)didReceiveMemoryWarning
92 [super didReceiveMemoryWarning];
93 // Dispose of any resources that can be recreated.
96 /* Called when the Play button is pressed */
97 -(IBAction) play:(id)sender
102 /* Called when the Pause button is pressed */
103 -(IBAction) pause:(id)sender
109 * Methods from GstreamerBackendDelegate
112 -(void) gstreamerInitialized
114 dispatch_async(dispatch_get_main_queue(), ^{
115 play_button.enabled = TRUE;
116 pause_button.enabled = TRUE;
117 message_label.text = @"Ready";
121 -(void) gstreamerSetUIMessage:(NSString *)message
123 dispatch_async(dispatch_get_main_queue(), ^{
124 message_label.text = message;
131 An instance of the `GStreamerBackend` in stored inside the class:
134 @interface ViewController () {
135 GStreamerBackend *gst_backend;
139 This instance is created in the `viewDidLoad` function through a custom
140 `init:` method in the `GStreamerBackend`:
147 play_button.enabled = FALSE;
148 pause_button.enabled = FALSE;
150 gst_backend = [[GStreamerBackend alloc] init:self];
154 This custom method is required to pass the object that has to be used as
155 the UI delegate (in this case, ourselves, the `ViewController`).
157 The Play and Pause buttons are also disabled in the
158 `viewDidLoad` function, and they are not re-enabled until the
159 `GStreamerBackend` reports that it is initialized and ready.
162 /* Called when the Play button is pressed */
163 -(IBAction) play:(id)sender
168 /* Called when the Pause button is pressed */
169 -(IBAction) pause:(id)sender
175 These two methods are called when the user presses the Play or Pause
176 buttons, and simply forward the call to the appropriate method in the
180 -(void) gstreamerInitialized
182 dispatch_async(dispatch_get_main_queue(), ^{
183 play_button.enabled = TRUE;
184 pause_button.enabled = TRUE;
185 message_label.text = @"Ready";
190 The `gstreamerInitialized` method is defined in the
191 `GStreamerBackendDelegate` protocol and indicates that the backend is
192 ready to accept commands. In this case, the Play and Pause buttons are
193 re-enabled and the Label text is set to “Ready”. This method is called
194 from a Dispatch Queue other than the Main one; therefore the need for
196 [dispatch_async()](https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man3/dispatch_async.3.html) call
197 wrapping all UI code.
200 -(void) gstreamerSetUIMessage:(NSString *)message
202 dispatch_async(dispatch_get_main_queue(), ^{
203 message_label.text = message;
208 The `gstreamerSetUIMessage:` method also belongs to the
209 `GStreamerBackendDelegate` protocol. It is called when the backend wants
210 to report some message to the user. In this case, the message is copied
211 onto the Label in the UI, again, from within a
212 [dispatch_async()](https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man3/dispatch_async.3.html) call.
214 ## The GStreamer Backend
216 The `GStreamerBackend` class performs all GStreamer-related tasks and
217 offers a simplified interface to the application, which does not need to
218 deal with all the GStreamer details. When it needs to perform any UI
219 action, it does so through a delegate, which is expected to adhere to
220 the `GStreamerBackendDelegate` protocol:
222 **GStreamerBackend.m**
225 #import "GStreamerBackend.h"
229 GST_DEBUG_CATEGORY_STATIC (debug_category);
230 #define GST_CAT_DEFAULT debug_category
232 @interface GStreamerBackend()
233 -(void)setUIMessage:(gchar*) message;
235 -(void)check_initialization_complete;
238 @implementation GStreamerBackend {
239 id ui_delegate; /* Class that we use to interact with the user interface */
240 GstElement *pipeline; /* The running pipeline */
241 GMainContext *context; /* GLib context used to run the main loop */
242 GMainLoop *main_loop; /* GLib main loop */
243 gboolean initialized; /* To avoid informing the UI multiple times about the initialization */
250 -(id) init:(id) uiDelegate
252 if (self = [super init])
254 self->ui_delegate = uiDelegate;
256 GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-2", 0, "iOS tutorial 2");
257 gst_debug_set_threshold_for_name("tutorial-2", GST_LEVEL_DEBUG);
259 /* Start the bus monitoring task */
260 dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
271 GST_DEBUG("Setting the pipeline to NULL");
272 gst_element_set_state(pipeline, GST_STATE_NULL);
273 gst_object_unref(pipeline);
280 if(gst_element_set_state(pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE) {
281 [self setUIMessage:"Failed to set pipeline to playing"];
287 if(gst_element_set_state(pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_FAILURE) {
288 [self setUIMessage:"Failed to set pipeline to paused"];
296 /* Change the message on the UI through the UI delegate */
297 -(void)setUIMessage:(gchar*) message
299 NSString *string = [NSString stringWithUTF8String:message];
300 if(ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerSetUIMessage:)])
302 [ui_delegate gstreamerSetUIMessage:string];
306 /* Retrieve errors from the bus and show them on the UI */
307 static void error_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
311 gchar *message_string;
313 gst_message_parse_error (msg, &err, &debug_info);
314 message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);
315 g_clear_error (&err);
317 [self setUIMessage:message_string];
318 g_free (message_string);
319 gst_element_set_state (self->pipeline, GST_STATE_NULL);
322 /* Notify UI about pipeline state changes */
323 static void state_changed_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
325 GstState old_state, new_state, pending_state;
326 gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
327 /* Only pay attention to messages coming from the pipeline, not its children */
328 if (GST_MESSAGE_SRC (msg) == GST_OBJECT (self->pipeline)) {
329 gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));
330 [self setUIMessage:message];
335 /* Check if all conditions are met to report GStreamer as initialized.
336 * These conditions will change depending on the application */
337 -(void) check_initialization_complete
339 if (!initialized && main_loop) {
340 GST_DEBUG ("Initialization complete, notifying application.");
341 if (ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerInitialized)])
343 [ui_delegate gstreamerInitialized];
349 /* Main method for the bus monitoring code */
354 GError *error = NULL;
356 GST_DEBUG ("Creating pipeline");
358 /* Create our own GLib Main Context and make it the default one */
359 context = g_main_context_new ();
360 g_main_context_push_thread_default(context);
363 pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);
365 gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
366 g_clear_error (&error);
367 [self setUIMessage:message];
372 /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
373 bus = gst_element_get_bus (pipeline);
374 bus_source = gst_bus_create_watch (bus);
375 g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
376 g_source_attach (bus_source, context);
377 g_source_unref (bus_source);
378 g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, (__bridge void *)self);
379 g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, (__bridge void *)self);
380 gst_object_unref (bus);
382 /* Create a GLib Main Loop and set it to run */
383 GST_DEBUG ("Entering main loop...");
384 main_loop = g_main_loop_new (context, FALSE);
385 [self check_initialization_complete];
386 g_main_loop_run (main_loop);
387 GST_DEBUG ("Exited main loop");
388 g_main_loop_unref (main_loop);
392 g_main_context_pop_thread_default(context);
393 g_main_context_unref (context);
394 gst_element_set_state (pipeline, GST_STATE_NULL);
395 gst_object_unref (pipeline);
405 #### Interface methods:
408 -(id) init:(id) uiDelegate
410 if (self = [super init])
412 self->ui_delegate = uiDelegate;
414 GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-2", 0, "iOS tutorial 2");
415 gst_debug_set_threshold_for_name("tutorial-2", GST_LEVEL_DEBUG);
417 /* Start the bus monitoring task */
418 dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
427 The `init:` method creates the instance by calling `[super init]`,
428 stores the delegate object that will handle the UI interaction and
429 launches the `app_function`, from a separate, concurrent, Dispatch
430 Queue. The `app_function` monitors the GStreamer bus for messages and
431 warns the application when interesting things happen.
433 `init:` also registers a new GStreamer debug category and sets its
434 threshold, so we can see the debug output from within Xcode and keep
435 track of our application progress.
441 GST_DEBUG("Setting the pipeline to NULL");
442 gst_element_set_state(pipeline, GST_STATE_NULL);
443 gst_object_unref(pipeline);
449 The `dealloc` method takes care of bringing the pipeline to the NULL
450 state and releasing it.
455 if(gst_element_set_state(pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE) {
456 [self setUIMessage:"Failed to set pipeline to playing"];
462 if(gst_element_set_state(pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_FAILURE) {
463 [self setUIMessage:"Failed to set pipeline to paused"];
468 The `play` and `pause` methods simply try to set the pipeline to the
469 desired state and warn the application if something fails.
471 #### Private methods:
474 /* Change the message on the UI through the UI delegate */
475 -(void)setUIMessage:(gchar*) message
477 NSString *string = [NSString stringWithUTF8String:message];
478 if(ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerSetUIMessage:)])
480 [ui_delegate gstreamerSetUIMessage:string];
485 `setUIMessage:` turns the C strings that GStreamer uses (UTF8 `char *`)
486 into `NSString *` and displays them through the
487 `gstreamerSetUIMessage` method of the `GStreamerBackendDelegate`. The
488 implementation of this method is marked as `@optional`, and hence the
489 check for its existence in the delegate with `respondsToSelector:`
492 /* Retrieve errors from the bus and show them on the UI */
493 static void error_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
497 gchar *message_string;
499 gst_message_parse_error (msg, &err, &debug_info);
500 message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);
501 g_clear_error (&err);
503 [self setUIMessage:message_string];
504 g_free (message_string);
505 gst_element_set_state (self->pipeline, GST_STATE_NULL);
508 /* Notify UI about pipeline state changes */
509 static void state_changed_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
511 GstState old_state, new_state, pending_state;
512 gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
513 /* Only pay attention to messages coming from the pipeline, not its children */
514 if (GST_MESSAGE_SRC (msg) == GST_OBJECT (self->pipeline)) {
515 gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));
516 [self setUIMessage:message];
522 The `error_cb()` and `state_changed_cb()` are callbacks registered to
523 the `error` and `state-changed` events in GStreamer, and their goal is
524 to inform the user about these events. These callbacks have been widely
525 used in the [Basic tutorials](tutorials/basic/index.md) and their
526 implementation is very similar, except for two points:
528 Firstly, the messages are conveyed to the user through the
529 `setUIMessage:` private method discussed above.
531 Secondly, they require an instance of a `GStreamerBackend` object in
532 order to call its instance method `setUIMessage:`, which is passed
533 through the `userdata` pointer of the callbacks (the `self` pointer in
534 these implementations). This is discussed below when registering the
535 callbacks in the `app_function`.
538 /* Check if all conditions are met to report GStreamer as initialized.
539 * These conditions will change depending on the application */
540 -(void) check_initialization_complete
542 if (!initialized && main_loop) {
543 GST_DEBUG ("Initialization complete, notifying application.");
544 if (ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerInitialized)])
546 [ui_delegate gstreamerInitialized];
553 `check_initialization_complete()` verifies that all conditions are met
554 to consider the backend ready to accept commands and tell the
555 application if so. In this simple tutorial the only conditions are that
556 the main loop exists and that we have not already told the application
557 about this fact. Later (more complex) tutorials include additional
560 Finally, most of the GStreamer work is performed in the app_function.
561 It exists with almost identical content in the Android tutorial, which
562 exemplifies how the same code can run on both platforms with little
566 /* Create our own GLib Main Context and make it the default one */
567 context = g_main_context_new ();
568 g_main_context_push_thread_default(context);
571 It first creates a GLib context so all `GSource`s are kept in the same
572 place. This also helps cleaning after GSources created by other
573 libraries which might not have been properly disposed of. A new context
574 is created with `g_main_context_new()` and then it is made the default
575 one for the thread with `g_main_context_push_thread_default()`.
579 pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);
581 gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
582 g_clear_error (&error);
583 [self setUIMessage:message];
589 It then creates a pipeline the easy way, with `gst_parse_launch()`. In
590 this case, it is simply an `audiotestsrc` (which produces a continuous
591 tone) and an `autoaudiosink`, with accompanying adapter
595 /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
596 bus = gst_element_get_bus (pipeline);
597 bus_source = gst_bus_create_watch (bus);
598 g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
599 g_source_attach (bus_source, context);
600 g_source_unref (bus_source);
601 g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, (__bridge void *)self);
602 g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, (__bridge void *)self);
603 gst_object_unref (bus);
606 These lines create a bus signal watch and connect to some interesting
607 signals, just like we have been doing in the [Basic
608 tutorials](tutorials/basic/index.md). The creation of the watch is done
609 step by step instead of using `gst_bus_add_signal_watch()` to exemplify
610 how to use a custom GLib context. The interesting bit here is the usage
612 [__bridge](http://clang.llvm.org/docs/AutomaticReferenceCounting.html#bridged-casts)
613 cast to convert an Objective-C object into a plain C pointer. In this
614 case we do not worry much about transferal of ownership of the object,
615 because it travels through C-land untouched. It re-emerges at the
616 different callbacks through the userdata pointer and cast again to a
617 `GStreamerBackend *`.
620 /* Create a GLib Main Loop and set it to run */
621 GST_DEBUG ("Entering main loop...");
622 main_loop = g_main_loop_new (context, FALSE);
623 [self check_initialization_complete];
624 g_main_loop_run (main_loop);
625 GST_DEBUG ("Exited main loop");
626 g_main_loop_unref (main_loop);
630 Finally, the main loop is created and set to run. Before entering the
631 main loop, though, `check_initialization_complete()` is called. Upon
632 exit, the main loop is disposed of.
634 And this is it! This has been a rather long tutorial, but we covered a
635 lot of territory. Building on top of this one, the following ones are
636 shorter and focus only on the new topics.
640 This tutorial has shown:
642 - How to handle GStreamer code from a separate thread using a
644 Queue](http://developer.apple.com/library/ios/#documentation/General/Conceptual/ConcurrencyProgrammingGuide/OperationQueues/OperationQueues.html) other
646 - How to pass objects between the Objective-C UI code and the C
649 Most of the methods introduced in this tutorial,
650 like `check_initialization_complete()`and `app_function()`, and the
651 interface methods `init:`, `play:`, `pause:`,
652 `gstreamerInitialized:` and `setUIMessage:` will continue to be used in
653 the following tutorials with minimal modifications, so better get used
656 It has been a pleasure having you here, and see you soon!
659 [screenshot]: images/tutorials/ios-a-running-pipeline-screenshot.png