validate: Allow variables to be set with other types than strings

And use value serialization from GStreamer to convert

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

scenario: Add an action to remove a feature/plugin from the registry

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

valiadate: Add a test for setting/checking (timed) properties

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

validate:scenario: Add action to set and check several properties at once

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

validate: Add an action type to set timed value properties

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

debug-viewer: Add screenshot

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/210>

Back to development

Release 1.17.1

validate: flow: record GstRegionOfInterestMeta

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/206>

validate:launcher: Ensure that the main directory exists

Fixes https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/issues/53 part 1

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/209>

validate:tests: Cleanup the way set environment vars

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/209>

validate: Stop trying to support uninstalled autotools

Fix https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/issues/53 part 2

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/209>

validateflow: Plug leak

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/205>

scenario: Fix waiting for signal

The signal callback signature was just wrong and not generic leading to
crash if waiting for any signal that didn't match it. This commit fixes
it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/205>

scenario: Add an option to set properties on all instances

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/205>

validate:flow: Add a way to dump buffer content as hex

Useful in unit tests with very small buffers

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/205>

validate: Cleanup implementation of appsrc-push

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/205>

validate: Remove useless condition

CID 1462652

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/192>

validate:ssim: Avoid dereferencing NULL pointer

CID 1462650

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/192>

validate: Give a proper argv[0] when running test files

Fixes CID 1462613

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/192>

validate:transcoding: Exit after printing the help

Fixes CID 1455575

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/192>

validate: Ensure Scenario isn't NULL when preparing action

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/192>

validate: Update documentation now core plugins are integrated

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/204>

validate: Make extracheck and flow part of core instead plugins

It makes things more complex and doesn't bring anything!

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/204>

launcher: Limit copies of massive debug logs in markdown file

When debugging is activated, we could end up with log files ranging in the
multi-megabyte or even gigabyte range. Copying those is expensive from a cpu/io
point of view in addition to clobbering the storage.

Instead of always copying those files, check if they are smaller than 500kB. If
not, don't copy them and instead provide a link to their location.

Fixes #52

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/203>

validateflow: Allow specifying checksum type

And add an extra mode 'checksum-as-id' which basically numerate
buffers checksums as they are being received so that it is simpler
to compare expectations when you are tracking buffers from both
sinkpads and srcpads.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/202>

validate: Add a parameter to force waiting on the test clock

This allows to wait for a new buffer to reach the sink without
actually cranking that buffer, allowing to quite the test without
waiting for EOS in a 100% reproducible way

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/201>

validate: Error out when a config hasn't been used at all

It probably means a plugin is not available

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/201>

Ship validate as part of a gst-devtools tarball

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/199>

meson: make debug_viewer a feature option

... and disable by default.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/199>

validate: Fix mixup in quarks usages

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/200>

validate: Add missing return func when GLib < 2.50

CID 1463854

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/200>

validate: Update documentation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/200>

validateflow: Add a way to configure when to generate expectations

By default, generate them whenever the file is missing but adding a way
to override that with `validateflow,generate-expectations=true` to force
regenerating them or setting `validateflow,generate-expectations=false`
to disallow generating them (on CI servers for example)

Also update the validateflow documentation to take that into account
and remove references to pipeline.json file which is now gone!

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/200>

scenario: Minor cleanup around pipeline change state management

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/198>

scenario: Mark seek as done only when reaching next state

There is a race where following actions could generate a
flush-start/flush-stop dance but the state change resulting from the
seek hasn't been committed yet, leading to the ASYNC_START being
ignored by GstBin since its pending_state is not VOID when receiving
the ASYNC_START message.

Conceptually it is totally correct to consider an action done when
the state change of the pipeline is stabilized..

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/198>

validate:launcher: Stop wrongly erroring on testsuite loading failure

When the testsuite was actually already loaded as the .py
file was explicitly passed in

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

validate: Fix loading configs from a caps

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

flow: Use bat to color diffs when possible

Adding a function to check if can output colored logs

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

launcher: check: Properly set PLUGIN_PATH and registry when running in gst-build

This makes registry

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

launcher: Ensure that -j tests run in parallel when running forever

So that you can reproduce the issue you want faster!

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

docs: Update gst-validate-launcher documentation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

launcher: Add support for running tests inside rr

Allowing us to easily run the tests forever and then replay the
failures!

https://rr-project.org/

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

validate: add missing gir annotation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/194>

validate-scenario: Initialize variable correctly

ret needs to be always reset to FALSE *before* checking attempting to load the
individual files. Otherwise there's the possibility it would silently accept an
invalid scenario name

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/196>

validate-scenario: Remove unused variable

We only need to check whether the field is present and of a given type

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/196>

validate: Stop using g_file_peek_path

It was introduced in 2.56 so is too recent

Fixes https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/issues/51

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/195>

validate: Add including support in the structure file parser

Adding proper error reporting support

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/191>

validate: utils: Do not try to replace vars in debug info fields

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/190>

validate: Fix parsing validate tests files with vars on windows

And ensure that we escape windows path in variables

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/190>

validate:launcher:check: Add support for gst-tester-1.0

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/189>

validate: Add support for known-issues in the .validatetest

And add some tests about remaining actions failures

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/189>

validate: Ensure a meta structure is found in test files

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/189>

validate:scenario: Check remaining action on stop, not EOS

When ignoring EOS, on addition action could end up not being executed
and no error was reported which was wrong.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/189>

validate: Fix rendering destination directory path creation

We were ending up creating file:/some/path in cwd

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>

validate: Plug some leaks

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>

validate: Add details on all g_log message reports.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>

validate: Add a mechanism to mark tests as skipped

And use it when a plugin is missing and the user didn't ask for
failure when it happens

And use the TAP[0] synthax to report it

[0]: https://testanything.org

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>

validate: Enhance issue reporting from structures

And properly bail out when required
This is preparatory work for gst-test in core which will use the TAP
protocol

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>

validate:flow: Handle some more segment fields filtering

Those slept through when implementing filtering

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/188>

ssim: Minor improvements on the way we attach to pads

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>

validate-scenario: Refactor seek handling

* Store all seek values into a list of pending seeks instead
  of hardcoding some values
* Store all segments that sinks received
* Match segments to seeks when all sinks received segments with
  the same seqnum
* Detect when a seek did *not* result in segments with identical
  matching seqnums

Should allow checking for all types of seek handling, including
flush-less seeks

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>

validate-report: Add new check for top-level seek

This issue is to detect seeks that don't result in segments
with identical seqnums. This check can be done at the top-level
scenario

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>

validate-pad-monitor: Post GstBaseSink SEGMENT on the bus

Allows higher-level bin or app (like validate-scenario) to know
what each sink currently has in terms of SEGMENT.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>

validate-element-monitor: Detect basesink elements

And add a macro

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>

validate-scenario: Add logging for scenario lock taking/releasing

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>

pad-monitor: Reliably track pending seeks

Instead of overriding all values when receiving a seek, store
them as a list of expected values.

This allows handling several seeks in a row, like non-flushing
seeks.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>

validate: Allow overidding issue severity from configs

Refactoring sensibly to allow getting configs outside the `core` namespace
and outside plugin names.

The `GST_VALIDATE_OVERRIDE` env variable should probably be removed
all together at some point.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/185>

validate: Add an option to describe issue types

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/185>

validate: Show the exact file line when error out in structure files

And minor stdout enhancements

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/185>

validate:launcher: Mark some more tests as too long for valgrind

Apply suggestion to debug-viewer/org.freedesktop.GstDebugViewer.appdata.xml.in

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/155>

Metainfo: backport flathub fixes

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/155>

validate: Do not check pulling thread when thread is paused

With decodebin3 we have cases where a task has been started in
the `typefind` element but the demuxer is the one pulling (from
its own thread)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/183>

validate: Add a check-position action type

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/182>

validate:scenario: Enhance failures messages

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/182>

validate: Add a way to run a TestClock in scenarios

A TestClock will be used automatically when a scenario has a
`crank-clock` action.

And make `validate` and `debug-viewer` options features in meson,
no reason they weren't and now we require gst-check to build validate

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/182>

validate: Introduce the concept of "Test files"

This way we can have a single file that wraps scenarios,
`gst-validate-1.0` arguments, as well as a configuration.

It changes the name of `description` of scenarios to use `meta`
The goal is to replace tests describes in python with dictionary
to fully self contained `.validatetest` files which look like:

```
meta,
    handles-states=true,
    ignore-eos=true,
    gst-validate-args = {
         "videotestsrc pattern=blue ! video/x-raw,format=I420,framerate=1/1 ! timeoverlay ! $(videosink) name=videosink allocation-meta-flags=0",
    },
    configs = {
         "$(validateflow), pad=videosink:sink, buffers-checksum=true, ignored-fields={\"buffers=meta\", }",
    }

play
seek, start=0.0, stop=5.0, flags=accurate+flush, rate=1.0
crank-clock, expected-elapsed-time=0.0
crank-clock, repeat=4, expected-elapsed-time=1.0
crank-clock, expected-elapsed-time=1.0
stop, on-message=eos
```

validate:flow: Sensibly improve stdout

validate: Fix criticals around NULL structure usage

validate: Fix 'repeat' parameter on non ASYNC action types

validate:scenario: Do not consider action with 'on-message' as on addition

validate: report: Handle when reporting NULL action

And add information about the action repeat state.

validate: Handle comments in multiline expressions

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/179>

validate: Fix multi variable in a single structure field

We were keeping using the GMatchInfo even after modifying the string
which is explicitly stated as invalid in the GRegex documentation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/179>

validate:scenario: Fix 'on-message' actions execution

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/177>

validate: Add { and [ as line continuation markers

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/177>

validate-launcher: Fix syntax error

This error prevents downloading assets from scratch. This regression was
introduced by MR !145 / commit 2581fef6843bfb53f3fc6f629577c1f013ef84e7

utils: Fix double free in error case

This was detected by Coverity. The content point would have been freed gain in
the done: label.

CID 1461289

validate: Use gst_print for validate report messages

ci: Port from only: to rules

This fixed CI breakage introduced by gst-ci!247

validate: Plug some leaks

And add some valgrind suppression for fontconfig

validate:ssim: Allow specifying file framerate to use frame numbers during comparison

validate:scenario: Make the action->prepare function return a GstValidateExecuteActionReturn

Implementers might want to report the error themselves

validate: Add a way to check last frame number

This introduces a new 'timecode-frame-number' in the 'check-last-sample'
action type se we can verify the number of output frames.

validate: Enhance debug message on invalid expression function call

validate:utils: Allow plain string in `gst_validate_utils_get_strv`

validatessim: Avoid having ':' in file names

This is unsupported on windows

validate:ssim: Flush cairo surface before getting pixels

validate:launcher: Fix reporting on failure when running forever/fatal