# onnxinference

This element can apply an ONNX model to video buffers. It attaches the tensor output to the buffer as a ref GstTensorMeta.

To install ONNX on your system, follow the instructions in the README.md in with this plugin.

## Example launch line

Test image file, model file (SSD) and label file can be found here: https://gitlab.collabora.com/gstreamer/onnx-models

     GST_DEBUG=ssdobjectdetector:5 \
     gst-launch-1.0 filesrc location=onnx-models/images/bus.jpg ! \
     jpegdec ! videoconvert ! onnxinference execution-provider=cpu model-file=onnx-models/models/ssd_mobilenet_v1_coco.onnx ! \
     ssdobjectdetector label-file=onnx-models/labels/COCO_classes.txt ! videoconvert ! imagefreeze ! autovideosink

Note: in order for downstream tensor decoders to correctly parse the tensor data in the GstTensorMeta, meta data must be attached to tensors. The inference element gets this model metadata from the modelinfo file annexed to the model. The modelinfo-helper tool can be used to create a modelinfo file: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/tree/main/subprojects/gst-devtools/modelinfo-helper

## Modelinfo example for ssd_mobilenet_v1_coco.onnx

     [modelinfo]
     version=1.0
     group-id=ssd-mobilenet-v1-variant-1-out

     [image_tensor:0]
     id=image_tensor_0
     type=uint8
     dims=-1,-1,-1,3
     dir=input
     ranges=0.0,255.0;0.0,255.0;0.0,255.0

     [detection_boxes:0]
     id=ssd-mobilenet-v1-variant-1-out-boxes
     type=float32
     dims=-1,-1,4
     dir=output

     [detection_classes:0]
     id=ssd-mobilenet-v1-variant-1-out-classes
     type=float32
     dims=-1,-1
     dir=output

     [detection_scores:0]
     id=ssd-mobilenet-v1-variant-1-out-scores
     type=float32
     dims=-1,-1
     dir=output

     [num_detections:0]
     id=generic-variant-1-out-count
     type=float32
     dims=-1
     dir=output

The modelinfo file should be placed alongside the model file with a `.modelinfo` suffix appended to the model filename. For example:

     /path/to/model.onnx
     /path/to/model.onnx.modelinfo

As a convenience, sample models with their modelinfo files are available here: https://gitlab.collabora.com/gstreamer/onnx-models/-/tree/master/models

## Hierarchy

    GObject
        ╰──GInitiallyUnowned
            ╰──GstObject
                ╰──GstElement
                    ╰──GstBaseTransform
                        ╰──onnxinference

## Factory details

Authors: – Aaron Boxer

Classification: – `Filter/Effect/Video`

Rank – primary

Plugin – onnx

Package – GStreamer Bad Plug-ins

## Pad Templates

### `sink`

    video/x-raw:
             format: { RGB, RGBA, BGR, BGRA }
              width: [ 1, 2147483647 ]
             height: [ 1, 2147483647 ]
          framerate: [ 0/1, 2147483647/1 ]

Presence – always

Direction – sink

Object type – GstPad

### `src`

    video/x-raw:
             format: { RGB, RGBA, BGR, BGRA }
              width: [ 1, 2147483647 ]
             height: [ 1, 2147483647 ]
          framerate: [ 0/1, 2147483647/1 ]

Presence – always

Direction – src

Object type – GstPad

## Properties

### execution-provider

    “execution-provider” GstOnnxExecutionProvider *

ONNX execution provider

Flags : Read / Write

Default value : cpu (0)

Since : 1.24

### input-image-format

    “input-image-format” GstMlInputImageFormat *

Input image format

Flags : Read / Write

Default value : hwc (0)

### input-tensor-offset

    “input-tensor-offset” gfloat

offset each tensor value by this value

Flags : Read / Write

Default value : 0

### input-tensor-scale

    “input-tensor-scale” gfloat

Divide each tensor value by this value

Flags : Read / Write

Default value : 1

### model-file

    “model-file” gchararray

ONNX model file

Flags : Read / Write

Default value : NULL

Since : 1.24

### optimization-level

    “optimization-level” GstOnnxOptimizationLevel *

ONNX optimization level

Flags : Read / Write

Default value : disable-all (0)

Since : 1.24

## Named constants

### GstMlInputImageFormat

#### Members

`hwc` (0) – Height Width Channel (HWC) a.k.a. interleaved image data format
`chw` (1) – Channel Height Width (CHW) a.k.a. planar image data format

### GstOnnxExecutionProvider

#### Members

`cpu` (0) – CPU execution provider
`cuda` (1) – CUDA execution provider

### GstOnnxOptimizationLevel

#### Members

`disable-all` (0) – Disable all optimization
`enable-basic` (1) – Enable basic optimizations (redundant node removals))
`enable-extended` (2) – Enable extended optimizations (redundant node removals + node fusions)
`enable-all` (3) – Enable all possible optimizations

The results of the search are