GstVideoDecoder

## GstVideoDecoder GstVideoDecoder — Base class for video decoders |

## Synopsis

    #define             GST_VIDEO_DECODER_ERROR             (el,
                                                             w,
                                                             domain,
                                                             code,
                                                             text,
                                                             debug,
                                                             ret)
    #define             GST_VIDEO_DECODER_FLOW_NEED_DATA
    #define             GST_VIDEO_DECODER_MAX_ERRORS
    #define             GST_VIDEO_DECODER_SINK_NAME
    #define             GST_VIDEO_DECODER_SINK_PAD          (obj)
    #define             GST_VIDEO_DECODER_SRC_NAME
    #define             GST_VIDEO_DECODER_SRC_PAD           (obj)
    #define             GST_VIDEO_DECODER_STREAM_LOCK       (decoder)
    #define             GST_VIDEO_DECODER_STREAM_UNLOCK     (decoder)
    #define             GST_VIDEO_DECODER_INPUT_SEGMENT     (obj)
    #define             GST_VIDEO_DECODER_OUTPUT_SEGMENT    (obj)
    struct              GstVideoDecoder;
    struct              GstVideoDecoderClass;
    void                gst_video_decoder_add_to_frame      (GstVideoDecoder *decoder,
                                                             int n_bytes);
    GstBuffer *         gst_video_decoder_alloc_output_buffer
                                                            (GstVideoDecoder *decoder);
    GstFlowReturn       gst_video_decoder_alloc_output_frame
                                                            (GstVideoDecoder *decoder,
                                                             GstVideoCodecFrame *frame);
    GstFlowReturn       gst_video_decoder_drop_frame        (GstVideoDecoder *dec,
                                                             GstVideoCodecFrame *frame);
    GstFlowReturn       gst_video_decoder_finish_frame      (GstVideoDecoder *decoder,
                                                             GstVideoCodecFrame *frame);
    GstVideoCodecFrame * gst_video_decoder_get_frame        (GstVideoDecoder *decoder,
                                                             int frame_number);
    GstClockTimeDiff    gst_video_decoder_get_max_decode_time
                                                            (GstVideoDecoder *decoder,
                                                             GstVideoCodecFrame *frame);
    gint                gst_video_decoder_get_max_errors    (GstVideoDecoder *dec);
    GstVideoCodecFrame * gst_video_decoder_get_oldest_frame (GstVideoDecoder *decoder);
    gboolean            gst_video_decoder_get_packetized    (GstVideoDecoder *decoder);
    GstFlowReturn       gst_video_decoder_have_frame        (GstVideoDecoder *decoder);
    void                gst_video_decoder_get_latency       (GstVideoDecoder *decoder,
                                                             GstClockTime *min_latency,
                                                             GstClockTime *max_latency);
    void                gst_video_decoder_set_latency       (GstVideoDecoder *decoder,
                                                             GstClockTime min_latency,
                                                             GstClockTime max_latency);
    gint                gst_video_decoder_get_estimate_rate (GstVideoDecoder *dec);
    GstVideoCodecState * gst_video_decoder_get_output_state (GstVideoDecoder *decoder);
    void                gst_video_decoder_set_estimate_rate (GstVideoDecoder *dec,
                                                             gboolean enabled);
    GstVideoCodecState * gst_video_decoder_set_output_state (GstVideoDecoder *decoder,
                                                             GstVideoFormat fmt,
                                                             guint width,
                                                             guint height,
                                                             GstVideoCodecState *reference);
    void                gst_video_decoder_set_max_errors    (GstVideoDecoder *dec,
                                                             gint num);
    void                gst_video_decoder_set_packetized    (GstVideoDecoder *decoder,
                                                             gboolean packetized);

## Description

This base class is for video decoders turning encoded data into raw video frames.

GstVideoDecoder and subclass should cooperate as follows.

1. Configuration

   - Initially, GstVideoDecoder calls `start` when the decoder element is activated, which allows subclass to perform any global setup.

   - GstVideoDecoder calls `set_format` to inform subclass of caps describing input video data that it is about to receive, including possibly configuration data. While unlikely, it might be called more than once, if changing input parameters require reconfiguration.

   - GstVideoDecoder calls `stop` at end of all processing.

2. Data processing

   - Base class gathers input data, and optionally allows subclass to parse this into subsequently manageable chunks, typically corresponding to and referred to as 'frames'.

   - Input frame is provided to subclass' `handle_frame`. The ownership of the frame is given to `handle_frame`.

   - If codec processing results in decoded data, subclass should call `gst_video_decoder_finish_frame` to have decoded data pushed. downstream.

3. Shutdown phase

   - GstVideoDecoder class calls `stop` to inform the subclass that data parsing will be stopped.

Subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It also needs to set the fixed caps on srcpad, when the format is ensured. This is typically when base class calls subclass' `set_format` function, though it might be delayed until calling `gst_video_decoder_finish_frame`.

Subclass is also responsible for providing (presentation) timestamps (likely based on corresponding input ones). If that is not applicable or possible, baseclass provides limited framerate based interpolation.

Similarly, the baseclass provides some limited (legacy) seeking support (upon explicit subclass request), as full-fledged support should rather be left to upstream demuxer, parser or alike. This simple approach caters for seeking and duration reporting using estimated input bitrates.

Baseclass provides some support for reverse playback, in particular in case incoming data is not packetized or upstream does not provide fragments on keyframe boundaries. However, subclass should then be prepared for the parsing and frame processing stage to occur separately (rather than otherwise the latter immediately following the former), and should ensure the parsing stage properly marks keyframes or rely on upstream to do so properly for incoming data.

Things that subclass need to take care of:

- Provide pad templates

- Set source pad caps when appropriate

- Configure some baseclass behaviour parameters.

- Optionally parse input data, if it is not considered packetized. Data will be provided to `parse` which should invoke `gst_video_decoder_add_to_frame` and `gst_video_decoder_have_frame` as appropriate.

- Accept data in `handle_frame` and provide decoded results to `gst_video_decoder_finish_frame`.

## Details

### GST_VIDEO_DECODER_ERROR()

    #define             GST_VIDEO_DECODER_ERROR(el, w, domain, code, text, debug, ret)

Utility function that video decoder elements can use in case they encountered a data processing error that may be fatal for the current "data unit" but need not prevent subsequent decoding. Such errors are counted and if there are too many, as configured in the context's max_errors, the pipeline will post an error message and the application will be requested to stop further media processing. Otherwise, it is considered a "glitch" and only a warning is logged. In either case, `ret` is set to the proper value to return to upstream/caller (indicating either GST_FLOW_ERROR or GST_FLOW_OK).

`el` : | the base video decoder element that generates the error
`weight` : | element defined weight of the error, added to error count
`domain` : | like CORE, LIBRARY, RESOURCE or STREAM (see gstreamer-GstGError)
`code` : | error code defined for that domain (see gstreamer-GstGError)
`text` : | the message to display (format string and args enclosed in parentheses)
`debug` : | debugging information for the message (format string and args enclosed in parentheses)
`ret` : | variable to receive return value

Since 0.10.36

### GST_VIDEO_DECODER_FLOW_NEED_DATA

    #define GST_VIDEO_DECODER_FLOW_NEED_DATA GST_FLOW_CUSTOM_SUCCESS

Returned while parsing to indicate more data is needed.

Since 0.10.36

### GST_VIDEO_DECODER_MAX_ERRORS

    #define GST_VIDEO_DECODER_MAX_ERRORS     10

Default maximum number of errors tolerated before signaling error.

Since 0.10.36

### GST_VIDEO_DECODER_SINK_NAME

    #define GST_VIDEO_DECODER_SINK_NAME    "sink"

The name of the templates for the sink pad.

Since 0.10.36

### GST_VIDEO_DECODER_SINK_PAD()

    #define GST_VIDEO_DECODER_SINK_PAD(obj)        (((GstVideoDecoder *) (obj))->sinkpad)

Gives the pointer to the sink GstPad object of the element.

`obj` : | a GstVideoDecoder

Since 0.10.36

### GST_VIDEO_DECODER_SRC_NAME

    #define GST_VIDEO_DECODER_SRC_NAME     "src"

The name of the templates for the source pad.

Since 0.10.36

### GST_VIDEO_DECODER_SRC_PAD()

    #define GST_VIDEO_DECODER_SRC_PAD(obj)         (((GstVideoDecoder *) (obj))->srcpad)

Gives the pointer to the source GstPad object of the element.

`obj` : | a GstVideoDecoder

Since 0.10.36

### GST_VIDEO_DECODER_STREAM_LOCK()

    #define GST_VIDEO_DECODER_STREAM_LOCK(decoder) g_rec_mutex_lock (&GST_VIDEO_DECODER (decoder)->stream_lock)

Obtain a lock to protect the decoder function from concurrent access.

`decoder` : | video decoder instance

Since 0.10.36

### GST_VIDEO_DECODER_STREAM_UNLOCK()

    #define GST_VIDEO_DECODER_STREAM_UNLOCK(decoder) g_rec_mutex_unlock (&GST_VIDEO_DECODER (decoder)->stream_lock)

Release the lock that protects the decoder function from concurrent access.

`decoder` : | video decoder instance

Since 0.10.36

### GST_VIDEO_DECODER_INPUT_SEGMENT()

    #define GST_VIDEO_DECODER_INPUT_SEGMENT(obj)     (GST_VIDEO_DECODER_CAST (obj)->input_segment)

Gives the segment of the element.

`obj` : | base decoder instance

Since 0.10.36

### GST_VIDEO_DECODER_OUTPUT_SEGMENT()

    #define GST_VIDEO_DECODER_OUTPUT_SEGMENT(obj)     (GST_VIDEO_DECODER_CAST (obj)->output_segment)

Gives the segment of the element.

`obj` : | base decoder instance

Since 0.10.36

### struct GstVideoDecoder

    struct GstVideoDecoder {
    };

The opaque GstVideoDecoder data structure.

Since 0.10.36

### struct GstVideoDecoderClass

    struct GstVideoDecoderClass {
      gboolean      (*open)           (GstVideoDecoder *decoder);

      gboolean      (*close)          (GstVideoDecoder *decoder);

      gboolean      (*start)          (GstVideoDecoder *decoder);

      gboolean      (*stop)           (GstVideoDecoder *decoder);

      GstFlowReturn (*parse)          (GstVideoDecoder *decoder,
    				   GstVideoCodecFrame *frame,
    				   GstAdapter *adapter,
    				   gboolean at_eos);

      gboolean      (*set_format)     (GstVideoDecoder *decoder,
    				   GstVideoCodecState * state);

      gboolean      (*reset)          (GstVideoDecoder *decoder,
    				   gboolean hard);

      GstFlowReturn (*finish)         (GstVideoDecoder *decoder);

      GstFlowReturn (*handle_frame)   (GstVideoDecoder *decoder,
    				   GstVideoCodecFrame *frame);

      gboolean      (*sink_event)     (GstVideoDecoder *decoder,
    				   GstEvent *event);

      gboolean      (*src_event)      (GstVideoDecoder *decoder,
    				   GstEvent *event);

      gboolean      (*decide_allocation)  (GstVideoDecoder *decoder, GstQuery *query);
    };

Subclasses can override any of the available virtual methods or not, as needed. At minimum `handle_frame` needs to be overridden, and `set_format` and likely as well. If non-packetized input is supported or expected, `parse` needs to be overridden as well.

`open` () | Optional. Called when the element changes to GST_STATE_READY. Allows opening external resources.
`close` () | Optional. Called when the element changes to GST_STATE_NULL. Allows closing external resources.
`start` () | Optional. Called when the element starts processing. Allows opening external resources.
`stop` () | Optional. Called when the element stops processing. Allows closing external resources.
`parse` () | Required for non-packetized input. Allows chopping incoming data into manageable units (frames) for subsequent decoding.
`set_format` () | Notifies subclass of incoming data format (caps).
`reset` () | Optional. Allows subclass (decoder) to perform post-seek semantics reset.
`finish` () | Optional. Called to request subclass to dispatch any pending remaining data (e.g. at EOS).
`handle_frame` () | Provides input data frame to subclass.
`sink_event` () | Optional. Event handler on the sink pad. This function should return TRUE if the event was handled and should be discarded (i.e. not unref'ed).
`src_event` () | Optional. Event handler on the source pad. This function should return TRUE if the event was handled and should be discarded (i.e. not unref'ed).
`decide_allocation` () | Optional. Setup the allocation parameters for allocating output buffers. The passed in query contains the result of the downstream allocation query.

Since 0.10.36

### gst_video_decoder_add_to_frame ()

    void                gst_video_decoder_add_to_frame      (GstVideoDecoder *decoder,
                                                             int n_bytes);

Removes next `n_bytes` of input data and adds it to currently parsed frame.

`decoder` : | a GstVideoDecoder
`n_bytes` : | an encoded GstVideoCodecFrame

Since 0.10.36

### gst_video_decoder_alloc_output_buffer ()

    GstBuffer *         gst_video_decoder_alloc_output_buffer
                                                            (GstVideoDecoder *decoder);

Helper function that uses ``gst_pad_alloc_buffer_and_set_caps()`` to allocate a buffer to hold a video frame for `decoder`'s current GstVideoCodecState.

`decoder` : | a GstVideoDecoder
Returns : | allocated buffer. [transfer full]

Since 0.10.36

### gst_video_decoder_alloc_output_frame ()

    GstFlowReturn       gst_video_decoder_alloc_output_frame
                                                            (GstVideoDecoder *decoder,
                                                             GstVideoCodecFrame *frame);

Helper function that uses ``gst_pad_alloc_buffer_and_set_caps()`` to allocate a buffer to hold a video frame for `decoder`'s current GstVideoCodecState. Subclass should already have configured video state and set src pad caps.

`decoder` : | a GstVideoDecoder
`frame` : | a GstVideoCodecFrame
Returns : | result from pad alloc call

Since 0.10.36

### gst_video_decoder_drop_frame ()

    GstFlowReturn       gst_video_decoder_drop_frame        (GstVideoDecoder *dec,
                                                             GstVideoCodecFrame *frame);

Similar to `gst_video_decoder_finish_frame()`, but drops `frame` in any case and posts a QoS message with the frame's details on the bus. In any case, the frame is considered finished and released.

`dec` : | a GstVideoDecoder
`frame` : | the GstVideoCodecFrame to drop. [transfer full]
Returns : | a GstFlowReturn, usually GST_FLOW_OK.

Since 0.10.36

### gst_video_decoder_finish_frame ()

    GstFlowReturn       gst_video_decoder_finish_frame      (GstVideoDecoder *decoder,
                                                             GstVideoCodecFrame *frame);

`frame` should have a valid decoded data buffer, whose metadata fields are then appropriately set according to frame data and pushed downstream. If no output data is provided, `frame` is considered skipped. In any case, the frame is considered finished and released.

`decoder` : | a GstVideoDecoder
`frame` : | a decoded GstVideoCodecFrame. [transfer full]
Returns : | a GstFlowReturn resulting from sending data downstream

Since 0.10.36

### gst_video_decoder_get_frame ()

    GstVideoCodecFrame * gst_video_decoder_get_frame        (GstVideoDecoder *decoder,
                                                             int frame_number);

Get a pending unfinished GstVideoCodecFrame

`decoder` : | a GstVideoDecoder
`frame_number` : | system_frame_number of a frame
Returns : | pending unfinished GstVideoCodecFrame identified by `frame_number`. [transfer none]

Since 0.10.36

### gst_video_decoder_get_max_decode_time ()

    GstClockTimeDiff    gst_video_decoder_get_max_decode_time
                                                            (GstVideoDecoder *decoder,
                                                             GstVideoCodecFrame *frame);

Determines maximum possible decoding time for `frame` that will allow it to decode and arrive in time (as determined by QoS events). In particular, a negative result means decoding in time is no longer possible and should therefore occur as soon/skippy as possible.

`decoder` : | a GstVideoDecoder
`frame` : | a GstVideoCodecFrame
Returns : | max decoding time.

Since 0.10.36

### gst_video_decoder_get_max_errors ()

    gint                gst_video_decoder_get_max_errors    (GstVideoDecoder *dec);

`dec` : | a GstVideoDecoder
Returns : | currently configured decoder tolerated error count.

Since 0.10.36

### gst_video_decoder_get_oldest_frame ()

    GstVideoCodecFrame * gst_video_decoder_get_oldest_frame (GstVideoDecoder *decoder);

Get the oldest pending unfinished GstVideoCodecFrame

`decoder` : | a GstVideoDecoder
Returns : | oldest pending unfinished GstVideoCodecFrame. [transfer full]

Since 0.10.36

### gst_video_decoder_get_packetized ()

    gboolean            gst_video_decoder_get_packetized    (GstVideoDecoder *decoder);

Queries whether input data is considered packetized or not by the base class.

`decoder` : | a GstVideoDecoder
Returns : | TRUE if input data is considered packetized.

Since 0.10.36

### gst_video_decoder_have_frame ()

    GstFlowReturn       gst_video_decoder_have_frame        (GstVideoDecoder *decoder);

Gathers all data collected for currently parsed frame, gathers corresponding metadata and passes it along for further processing, i.e. `handle_frame`.

`decoder` : | a GstVideoDecoder
Returns : | a GstFlowReturn

Since 0.10.36

### gst_video_decoder_get_latency ()

    void                gst_video_decoder_get_latency       (GstVideoDecoder *decoder,
                                                             GstClockTime *min_latency,
                                                             GstClockTime *max_latency);

`decoder` : | a GstVideoDecoder
`min_latency` : | the configured minimum latency. [out][allow-none]
`max_latency` : | the configured maximum latency. [out][allow-none]
Returns : | the configured encoding latency.

Since 0.10.36

### gst_video_decoder_set_latency ()

    void                gst_video_decoder_set_latency       (GstVideoDecoder *decoder,
                                                             GstClockTime min_latency,
                                                             GstClockTime max_latency);

Informs baseclass of encoding latency.

`decoder` : | a GstVideoDecoder
`min_latency` : | minimum latency
`max_latency` : | maximum latency

Since 0.10.36

### gst_video_decoder_get_estimate_rate ()

    gint                gst_video_decoder_get_estimate_rate (GstVideoDecoder *dec);

`dec` : | a GstVideoDecoder
Returns : | currently configured byte to time conversion setting

Since 0.10.36

### gst_video_decoder_get_output_state ()

    GstVideoCodecState * gst_video_decoder_get_output_state (GstVideoDecoder *decoder);

Get the GstVideoCodecState currently describing the output stream.

`decoder` : | a GstVideoDecoder
Returns : | GstVideoCodecState describing format of video data. [transfer full]

Since 0.10.36

### gst_video_decoder_set_estimate_rate ()

    void                gst_video_decoder_set_estimate_rate (GstVideoDecoder *dec,
                                                             gboolean enabled);

Allows baseclass to perform byte to time estimated conversion.

`dec` : | a GstVideoDecoder
`enabled` : | whether to enable byte to time conversion

Since 0.10.36

### gst_video_decoder_set_output_state ()

    GstVideoCodecState * gst_video_decoder_set_output_state (GstVideoDecoder *decoder,
                                                             GstVideoFormat fmt,
                                                             guint width,
                                                             guint height,
                                                             GstVideoCodecState *reference);

Creates a new GstVideoCodecState with the specified `fmt`, `width` and `height` as the output state for the decoder. Any previously set output state on `decoder` will be replaced by the newly created one.

If the subclass wishes to copy over existing fields (like pixel aspec ratio, or framerate) from an existing GstVideoCodecState, it can be provided as a `reference`.

If the subclass wishes to override some fields from the output state (like pixel-aspect-ratio or framerate) it can do so on the returned GstVideoCodecState.

The new output state will only take effect (set on pads and buffers) starting from the next call to #`gst_video_decoder_finish_frame()`.

`decoder` : | a GstVideoDecoder
`fmt` : | a GstVideoFormat
`width` : | The width in pixels
`height` : | The height in pixels
`reference` : | An optional reference GstVideoCodecState. [allow-none][transfer none]
Returns : | the newly configured output state. [transfer full]

Since 0.10.36

### gst_video_decoder_set_max_errors ()

    void                gst_video_decoder_set_max_errors    (GstVideoDecoder *dec,
                                                             gint num);

Sets numbers of tolerated decoder errors, where a tolerated one is then only warned about, but more than tolerated will lead to fatal error. Default is set to GST_VIDEO_DECODER_MAX_ERRORS.

`dec` : | a GstVideoDecoder
`num` : | max tolerated errors

Since 0.10.36

### gst_video_decoder_set_packetized ()

    void                gst_video_decoder_set_packetized    (GstVideoDecoder *decoder,
                                                             gboolean packetized);

Allows baseclass to consider input data as packetized or not. If the input is packetized, then the `parse` method will not be called.

`decoder` : | a GstVideoDecoder
`packetized` : | whether the input data should be considered as packetized.

Since 0.10.36