# gst-validate-launcher

`gst-validate-launcher` is an application to run unit or integration testsuites providing a set of options and features to help debugging them easier.

## Run the GStreamer unit tests

Running GStreamer unit tests inside `gst-build` is as simple as doing:

    gst-validate-launcher check.gst*

If you only want to run GStreamer core tests:

    gst-validate-launcher check.gstreamer*

Or to run unit tests from gst-plugins-base

    gst-validate-launcher check.gst-plugins-base

## Run the GstValidate default testsuite

GstValidate comes with a default testsuite to be executed on a default set of media samples. Those media samples are stored with `git-lfs` so you will need it to be able to launch the default testsuite.

We recommendusing `gst-build` to setup everything needed to run the testsuite and you can simply do:

    gst-validate-launcher validate

This will only launch the GstValidate tests and not other applications that might be supported (currently `ges-launch` is also supported and has its own default testsuite).

## Example of a testsuite implementation

To implement a testsuite, you will have to write some simple python code that defines the tests to be launched by `gst-validate-launcher`.

In this example, we will assume that you want to write a whole new testsuite based on your own media samples and scenarios. The set of media files and the testsuite implementation file will be structured as follow:

    testsuite_folder/
      |-> testsuite.py
      |-> sample_files/
          |-> file.mp4
          |-> file1.mkv
          |-> file2.ogv
      |-> scenarios
          |-> scenario.scenario
          |-> scenario1.scenario

You should generate the `.media_info` files. To generate them for local files, you can use:

    gst-validate-launcher --medias-paths /path/to/sample_files/ --generate-media-info

For remote streams, you should use `gst-validate-media-check-1.0`. For an http stream you can for example do:

    gst-validate-media-check-GST_API_VERSION http://someonlinestream.com/thestream \
                  --output-file /path/to/testsuite_folder/sample_files/thestream.stream_info

The `gst-validate-launcher` will use the generated `.media_info` and `.stream_info` files to validate the tests as those contain the necessary information.

Then you will need to write the `testsuite.py` file. You can for example implement the following testsuite:

    """
    The GstValidate custom testsuite
    """

    import os
    from launcher.baseclasses import MediaFormatCombination
    from launcher.apps.gstvalidate import *
    TEST_MANAGER = "validate"

    KNOWN_ISSUES = {}

    def setup_tests(test_manager, options):
        print("Setting up the custom testsuite")
        assets_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), ".", "samples_files"))
        options.add_paths(assets_dir)

        # This step will register default data for the test manager:
        # - scenarios such as `play_15s`, `reverse_playback` etc.
        # - encoding formats such as "mp4,h264,mp3" etc.
        # - blacklist such as dash.media_check.*
        # - test generators:
        #   - GstValidatePlaybinTestsGenerator
        #   - GstValidateMediaCheckTestsGenerator
        #   - GstValidateTranscodingTestsGenerator
        # This 'defaults' can be found in 'gst-devtools/validate/launcher/apps/gstvalidate.py#register_defaults'
        # test_manager.register_defaults()

        # Add scenarios
        scenarios = []
        scenarios.append("play_5s")
        scenarios.append("seek_backward")
        test_manager.set_scenarios(scenarios)

        # Add encoding formats used by the transcoding generator
        test_manager.add_encoding_formats([
                MediaFormatCombination("mp4", "mp3", "h264"),])

        # Add generators
        # GstValidatePlaybinTestsGenerator needs at least one media file
        test_manager.add_generators([GstValidateMediaCheckTestsGenerator(test_manager)])
        # GstValidatePlaybinTestsGenerator needs at least one scenario
        test_manager.add_generators([GstValidatePlaybinTestsGenerator(test_manager)])
        # GstValidateTranscodingTestsGenerator needs at least one MediaFormatCombination
        test_manager.add_generators([GstValidateTranscodingTestsGenerator(test_manager)])

        # list of combo to blacklist tests. Here it blacklists all tests with playback.seek_backward
        test_manager.set_default_blacklist([
                ("custom_testsuite.file.playback.seek_backward.*",
                 "Not supported by this testsuite."),])

        # you can even pass known issues to bypass an existing error in your custom testsuite
        test_manager.add_expected_issues(KNOWN_ISSUES)
        return True

Once this is done, you've got a testsuite that will:

- Run playbin pipelines on `file.mp4`, `file1.mkv` and `file2.ogv`> executing `play_5s` and `seek_backward` scenarios

- Transcode `file.mp4,` `file1.mkv` and `file2.ogv` to h264 and mp3 in a MP4 container

The only thing to do to run the testsuite is:

    gst-validate-launcher --testsuites-dir=/path/to/testsuite_folder/ testsuite

# Invocation

You can find detailed information about the launcher by launching it:

    gst-validate-launcher --help

You can list all the tests with:

    gst-validate-launcher --testsuites-dir=/path/to/testsuite_folder/ testsuite -L

The results of the search are