Media Codec: Encoding and Decoding Media Files

- - -

This tutorial demonstrates how you can encode and decode media files.

- -

Warm-up

- -

Become familiar with the Media Codec API basics by learning about:

- -

-Initializing Media Codecs -
Initialize the media codecs for use.
Managing Media Codecs -
Prepare and start the codec.
Releasing Resources -
Reset the codec and destroy the handle.

- - -

Initializing Media Codecs

- -

Before using the Media codec:

To use the functions and data types of the Media Codec API (in mobile and wearable applications), include the <media_codec.h> header file in your application:

-#include <media_codec.h>
+    
+      
+        
+          Media Codec: Encoding and Decoding Media Files
+          This tutorial demonstrates how you can encode and decode media
+            files.
+          Warm-up
+          Become familiar with the Media Codec API basics by learning about:
+          
+            
+              Initializing Media Codecs
+              Initialize the media codecs for use.
+            
+            Managing Media Codecs
+              Prepare and start the codec.
+            
+            Releasing Resources
+              Reset the codec and destroy the handle.
+            
+          
+          Initializing Media Codecs
+          Creating and Configuring MediaCodec
+          Before using the Media codec:
+          
+            
+              To use the functions and data types of the Media Codec API (in
+                mobile
+                and wearable
+                applications), include the <media_codec.h>
+                header file in your application:
+              #include <media_codec.h>
 
-Define a handle for the media codec and pass it to the mediacodec_create() function. The handle must be passed to all other Media Codec APIs.
--mediacodec_h *mediacodec;
+            Define a handle for the media codec and pass it to the mediacodec_create()
+              function. The handle must be passed to all other Media Codec APIs.
+              mediacodec_h *mediacodec;
 ret = mediacodec_create(&mediacodec);
-
-If the handle is created normally, set the codec and configuration using the mediacodec_set_codec() function.
-The enumerations mediacodec_codec_type_e (in mobile and wearable applications) and mediacodec_support_type_e (in mobile and wearable applications) define the media codec type and support type.
-ret = mediacodec_set_codec(mediacodec, (mediacodec_codec_type_e)codecid, flag); 
-
-Set the video decoder using the mediacodec_set_vdec_info() or mediacodec_set_venc_info() function. 
-ret = mediacodec_set_vdec_info(mediacodec, width, height);
-
-// Or
-
-ret = mediacodec_set_venc_info(mediacodec, width, height, fps, target_bits);
-
-
- Managing Media Codecs
- 
-The Media Codec usage follows a basic pattern:
-Create and configure the Media Codec handle.
-Loop until done:
-If an input buffer is ready, read a chunk of input and copy it into the buffer. 
-If an output buffer is ready, copy the output from the buffer.
-Release the Media Codec handle. 
-
-
-To manage the codec process loop:
-
-
-If the prerequisites are set normally, prepare the media codec using the mediacodec_prepare() function.
-
-ret = mediacodec_prepare(mediacodec);
+            
+            If the handle is created normally, set the codec and
+              configuration using the mediacodec_set_codec()
+              function.
+              The enumerations mediacodec_codec_type_e
+                (in mobile
+                and wearable
+                applications) and mediacodec_support_type_e
+                (in mobile
+                and wearable
+                applications) define the media codec type and support type.
+              ret = mediacodec_set_codec(mediacodec, (mediacodec_codec_type_e)codecid, flag); 
+            
+            To configure the video encoder, perform the following function:

+              ret = mediacodec_set_venc_info(mediacodec, width, height, fps, target_bits);
+              To configure the video decoder, perform the following function:

+              ret = mediacodec_set_vdec_info(mediacodec, width, height);
 
-
-
-Start the media codec using the mediacodec_process_input() and mediacodec_get_output() functions.
-media_packet_h in_buf = NULL;
+              To configure the audio encoder, perform the following function:

+              ret = mediacodec_set_aenc_info(mediacodec, samplerate, channel, bit, bitrate);
+
+              To configure the audio decoder, perform the following function:

+              ret = mediacodec_set_adec_info(mediacodec, samplerate, channel, bit);
+


+              

+            
+          
+          Setting Callbacks
+          
+            
+              To receive notifications when the input buffers are used,
+                register a callback function using the mediacodec_set_input_buffer_used_cb()
+                function. The callback is invoked when the input buffers are
+                queued to the codec.
+              ret = mediacodec_set_input_buffer_used_cb(mediacodec, _input_buffer_used_cb, NULL);
+              If the media_packet is used, the media_packet should be
+                destroyed when this callback is invoked.
+              static void _input_buffer_used_cb( media_packet_h pkt, void *user_data)
{
    media_packet_destroy(pkt);
    return;
}
+
+            
+              To receive notifications when the output buffers are dequeued,
+                register a callback function using the mediacodec_set_output_buffer_available_cb()
+                function. The callback is invoked when the output buffers are
+                dequeued.
+              ret = mediacodec_set_output_buffer_availalbe_cb(mediacodec, _output_buffer_available_cb, mediacodec);
+              If the media_packet is dequeued from the codec, this callback
+                is invoked.

+                It is possible to get output packet through
+                mediacodec_get_output() when the callback is invoked.
+              static void _output_buffer_available_cb( media_packet_h pkt, void *user_data)
{
    media_packet_h out_pkt;
    mediacodec_h mediacodec = (mediacodec_h)user_data;

    if ( pkt != NULL ) {
        mediacodec_get_output(mediacodec, &out_pkt, 0);
        media_packet_destroy(out_pkt); 
    }
    return;
}
+
+          
+          Â Â Â  Â  

+          Managing Media Codecs
+          Filling packet with data
+          Once the media_packet is allocated with corresponding mime types of
+            codecs, it is possible to fill media_packet with data.
+          To fill packet with data:
+          
+            Get the data pointer from media_packet and set buffer size on
+              preallocated packet.

+              unsigned char nal[48] = {0x00, 0x00, 0x00, 0x01, 0x67, 0x4D, 0x40, 0x33,
			0x9A, 0x73, 0x80, 0xA0, 0x08, 0xB4, 0x20, 0x00,
			0x32, 0xE9, 0xE0, 0x09, 0x89, 0x68, 0x11, 0xE3,
			0x06, 0x23, 0xC0, 0x00, 0x00, 0x00, 0x01, 0x68,
			0xEE, 0x3C, 0x80, 0x00, 0x00, 0x00, 0x01, 0x65,
			0x88, 0x80, 0x01, 0x48, 0x00, 0x06, 0x57, 0xFD};

media_format_h fmt;
media_packet_h pkt;
void *data;

ret = media_format_set_video_mime (fmt, MEDIA_FORMAT_H264_SP);
ret = media_format_set_video_width(fmt, 1280);
ret = media_format_set_video_height(fmt, 544);
ret = media_packet_create_alloc(fmt, NULL, NULL, &pkt);

ret = media_packet_get_buffer_data_ptr (pkt, &data);
memcpy(data, nal, 48);
ret = media_packet_set_buffer_size (pkt, 48);
+
+            
+            If the memory buffer might contain extra padding bytes after
+              each row of pixels, It is possible to check whether the stride in
+              uncompressed video frame is not same as the video width. If then,
+              strided copy is needed as below.

+              void _fill_buffer(media_packet_h pkt, unsigned char *yuv, int width, int height)
{
    int i;

    /* Y plane */
    media_packet_get_video_stride_width(pkt, 0, &stride);
    media_packet_get_video_plane_data_ptr(pkt, 0, &data);

    for (i = 0 ; i < height; i++) {
    memcpy(data, yuv, width);
    data += stride;
} 

+
+            
+          
+          Managing Mediacodec
+          The Media Codec usage follows a basic pattern:
+          
+            Create and configure the Media Codec handle.
+            Loop until done:
+              
+                If an input buffer is ready, read a chunk of input and copy
+                  it into the buffer.
+                If an output buffer is ready, copy the output from the
+                  buffer.
+              
+            
+            Release the Media Codec handle.
+          
+          To manage the codec process loop:
+          
+            If the prerequisites are set normally, prepare the media codec
+              using the mediacodec_prepare()
+              function.
+              ret = mediacodec_prepare(mediacodec);
+
+            Before calling mediacodec_process_input(), it is needed to set
+              flag in media_packet.

+              

+              If the media_packet contains codec data such as SPS, PPS in case
+              of H.264, it is needed to set codec config flag using
+              media_packet_set_flags().

+              ret = media_packet_set_flags(pkt, MEDIA_PACKET_CODEC_CONFIG);
+If the media_packet contains the end of stream, it is needed to set end of
+              stream flag. The eos callback will be invoked if the eos packet is
+              decoded or encoded and eos callback is set.

+              ret = media_packet_set_flags(pkt, MEDIA_PACKET_END_OF_STREAM);
+
+            Start the media codec using the mediacodec_process_input()
+              and mediacodec_get_output()
+              functions.
+              media_packet_h in_buf = NULL;
 ret = mediacodec_process_input (mediacodec, in_buf, 0);
 
 media_packet_h output_buf = NULL;
 ret = mediacodec_get_output (mediacodec, &output_buf, 0);
 
-
-
-
- Releasing Resources
-
-To reset the codec and destroy the handle using the mediacodec_unprepare() and mediacodec_destroy() functions after you have finished work with the media codec:
-ret = mediacodec_unprepare(mediacodec);
+

After calling mediacodec_get_output(), you can check frame using + media_packet()
+ If the media_packet contains key frame or codec data, you can + check as below.
+
```
bool keyframe;
bool codec_config;

ret = media_packet_is_sync_frame(pkt, &keyframe);
ret = media_packet_is_codec_config(pkt, &codec_config);
+
```
+
+

Releasing Resources

To reset the codec and destroy the handle using the mediacodec_unprepare() + and mediacodec_destroy() + functions after you have finished work with the media codec:

ret = mediacodec_unprepare(mediacodec);
 
 ret = mediacodec_destroy(mediacodec);

Afterwards, the media codec state is changed to MEDIACODEC_STATE_NONE.

- - - - -