scenario: Minor cleanup around pipeline change state management

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/198>

scenario: Mark seek as done only when reaching next state

There is a race where following actions could generate a
flush-start/flush-stop dance but the state change resulting from the
seek hasn't been committed yet, leading to the ASYNC_START being
ignored by GstBin since its pending_state is not VOID when receiving
the ASYNC_START message.

Conceptually it is totally correct to consider an action done when
the state change of the pipeline is stabilized..

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/198>

validate:launcher: Stop wrongly erroring on testsuite loading failure

When the testsuite was actually already loaded as the .py
file was explicitly passed in

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

validate: Fix loading configs from a caps

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

flow: Use bat to color diffs when possible

Adding a function to check if can output colored logs

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

launcher: check: Properly set PLUGIN_PATH and registry when running in gst-build

This makes registry

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

launcher: Ensure that -j tests run in parallel when running forever

So that you can reproduce the issue you want faster!

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

docs: Update gst-validate-launcher documentation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

launcher: Add support for running tests inside rr

Allowing us to easily run the tests forever and then replay the
failures!

https://rr-project.org/

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

validate: add missing gir annotation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/194>

validate-scenario: Initialize variable correctly

ret needs to be always reset to FALSE *before* checking attempting to load the
individual files. Otherwise there's the possibility it would silently accept an
invalid scenario name

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/196>

validate-scenario: Remove unused variable

We only need to check whether the field is present and of a given type

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/196>

validate: Stop using g_file_peek_path

It was introduced in 2.56 so is too recent

Fixes https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/issues/51

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/195>

validate: Add including support in the structure file parser

Adding proper error reporting support

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/191>

validate: utils: Do not try to replace vars in debug info fields

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/190>

validate: Fix parsing validate tests files with vars on windows

And ensure that we escape windows path in variables

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/190>

validate:launcher:check: Add support for gst-tester-1.0

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/189>

validate: Add support for known-issues in the .validatetest

And add some tests about remaining actions failures

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/189>

validate: Ensure a meta structure is found in test files

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/189>

validate:scenario: Check remaining action on stop, not EOS

When ignoring EOS, on addition action could end up not being executed
and no error was reported which was wrong.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/189>

validate: Fix rendering destination directory path creation

We were ending up creating file:/some/path in cwd

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>

validate: Plug some leaks

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>

validate: Add details on all g_log message reports.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>

validate: Add a mechanism to mark tests as skipped

And use it when a plugin is missing and the user didn't ask for
failure when it happens

And use the TAP[0] synthax to report it

[0]: https://testanything.org

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>

validate: Enhance issue reporting from structures

And properly bail out when required
This is preparatory work for gst-test in core which will use the TAP
protocol

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>

validate:flow: Handle some more segment fields filtering

Those slept through when implementing filtering

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/188>

ssim: Minor improvements on the way we attach to pads

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>

validate-scenario: Refactor seek handling

* Store all seek values into a list of pending seeks instead
  of hardcoding some values
* Store all segments that sinks received
* Match segments to seeks when all sinks received segments with
  the same seqnum
* Detect when a seek did *not* result in segments with identical
  matching seqnums

Should allow checking for all types of seek handling, including
flush-less seeks

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>

validate-report: Add new check for top-level seek

This issue is to detect seeks that don't result in segments
with identical seqnums. This check can be done at the top-level
scenario

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>

validate-pad-monitor: Post GstBaseSink SEGMENT on the bus

Allows higher-level bin or app (like validate-scenario) to know
what each sink currently has in terms of SEGMENT.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>

validate-element-monitor: Detect basesink elements

And add a macro

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>

validate-scenario: Add logging for scenario lock taking/releasing

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>

pad-monitor: Reliably track pending seeks

Instead of overriding all values when receiving a seek, store
them as a list of expected values.

This allows handling several seeks in a row, like non-flushing
seeks.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>

validate: Allow overidding issue severity from configs

Refactoring sensibly to allow getting configs outside the `core` namespace
and outside plugin names.

The `GST_VALIDATE_OVERRIDE` env variable should probably be removed
all together at some point.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/185>

validate: Add an option to describe issue types

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/185>

validate: Show the exact file line when error out in structure files

And minor stdout enhancements

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/185>

validate:launcher: Mark some more tests as too long for valgrind

Apply suggestion to debug-viewer/org.freedesktop.GstDebugViewer.appdata.xml.in

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/155>

Metainfo: backport flathub fixes

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/155>

validate: Do not check pulling thread when thread is paused

With decodebin3 we have cases where a task has been started in
the `typefind` element but the demuxer is the one pulling (from
its own thread)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/183>

validate: Add a check-position action type

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/182>

validate:scenario: Enhance failures messages

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/182>

validate: Add a way to run a TestClock in scenarios

A TestClock will be used automatically when a scenario has a
`crank-clock` action.

And make `validate` and `debug-viewer` options features in meson,
no reason they weren't and now we require gst-check to build validate

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/182>

validate: Introduce the concept of "Test files"

This way we can have a single file that wraps scenarios,
`gst-validate-1.0` arguments, as well as a configuration.

It changes the name of `description` of scenarios to use `meta`
The goal is to replace tests describes in python with dictionary
to fully self contained `.validatetest` files which look like:

```
meta,
    handles-states=true,
    ignore-eos=true,
    gst-validate-args = {
         "videotestsrc pattern=blue ! video/x-raw,format=I420,framerate=1/1 ! timeoverlay ! $(videosink) name=videosink allocation-meta-flags=0",
    },
    configs = {
         "$(validateflow), pad=videosink:sink, buffers-checksum=true, ignored-fields={\"buffers=meta\", }",
    }

play
seek, start=0.0, stop=5.0, flags=accurate+flush, rate=1.0
crank-clock, expected-elapsed-time=0.0
crank-clock, repeat=4, expected-elapsed-time=1.0
crank-clock, expected-elapsed-time=1.0
stop, on-message=eos
```

validate:flow: Sensibly improve stdout

validate: Fix criticals around NULL structure usage

validate: Fix 'repeat' parameter on non ASYNC action types

validate:scenario: Do not consider action with 'on-message' as on addition

validate: report: Handle when reporting NULL action

And add information about the action repeat state.

validate: Handle comments in multiline expressions

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/179>

validate: Fix multi variable in a single structure field

We were keeping using the GMatchInfo even after modifying the string
which is explicitly stated as invalid in the GRegex documentation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/179>

validate:scenario: Fix 'on-message' actions execution

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/177>

validate: Add { and [ as line continuation markers

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/177>

validate-launcher: Fix syntax error

This error prevents downloading assets from scratch. This regression was
introduced by MR !145 / commit 2581fef6843bfb53f3fc6f629577c1f013ef84e7

utils: Fix double free in error case

This was detected by Coverity. The content point would have been freed gain in
the done: label.

CID 1461289

validate: Use gst_print for validate report messages

ci: Port from only: to rules

This fixed CI breakage introduced by gst-ci!247

validate: Plug some leaks

And add some valgrind suppression for fontconfig

validate:ssim: Allow specifying file framerate to use frame numbers during comparison

validate:scenario: Make the action->prepare function return a GstValidateExecuteActionReturn

Implementers might want to report the error themselves

validate: Add a way to check last frame number

This introduces a new 'timecode-frame-number' in the 'check-last-sample'
action type se we can verify the number of output frames.

validate: Enhance debug message on invalid expression function call

validate:utils: Allow plain string in `gst_validate_utils_get_strv`

validatessim: Avoid having ':' in file names

This is unsupported on windows

validate:ssim: Flush cairo surface before getting pixels

validate:launcher: Fix reporting on failure when running forever/fatal

validate:ssim: Enhance printing position

Adding a new `gst_validate_print_position` method which also
sends messages to the runner if required.

validate:launcher: Add a generator to generate test for frame accurate seeking

validate:ssim: Use stream time to reference frames

validate:ssim: Don't check neighbor frames when comparing exact same frame

validate:ssim: Enhance debugging message when similarity do not match

Printing out where the diff image file is

validate:launcher: Enhance support for running ssim tests

Using a special 'ssim' variable in pipeline dicts to activate it
Similare to what we do for validateflow

validate:launcher: Add support for skipped media info files

Those are skipped to generate tests by default but are updated when
required, this will allow us to generate specific test on demand for
those

validate:ssim: Avoid segfaults trying to attach pads without a template

validate:scenario:Handle GStreamer serialized timestamps

validate:launcher: Never try to load a testsuite with the same name from different locations

validate:media-check: Fix wrong exit code

There are code paths where the runner doesn't contain any issue but an error has already been reported

validate:launcher: Add support for the imagesequence protocol

validate:launcher: Fix reporting on failure when running forever/fatal

validate:launcher: Cache GstValidateMediaDescriptor to avoid reparsing .media_info

Saving another second at startup

validate:launcher: Generate profiling data even if an exception happens

validate:launcher: Batch inspecting scenarios

Removing almost 1 second to start running tests with the default
testsuite

validate:launcher: Fix printed test number

validate:launcher: Allow passing configs paths when loading from dictionary

The same way we allow it for scenarios

debug-viewer: Display nanoseconds in the timestamp column

validate: Don't use sprintf + glib format modifiers

We do not have a way to know the format modifiers to use with string
functions provided by the system. `G_GUINT64_FORMAT` and other string
modifiers only work for glib string formatting functions. We cannot
use them for string functions provided by the stdlib. See:
https://developer.gnome.org/glib/stable/glib-Basic-Types.html#glib-Basic-Types.description

```
 ../validate/plugins/flow/formatting.c: In function 'format_number':
../validate/plugins/flow/formatting.c:68:22: error: unknown conversion type character 'l' in format [-Werror=format=]
   sprintf (dest_str, "%" G_GUINT64_FORMAT, number);
                      ^~~
In file included from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/glib-2.0/glib/gtypes.h:32,
                 from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/glib-2.0/glib/galloca.h:32,
                 from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/glib-2.0/glib.h:30,
                 from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/gstreamer-1.0/gst/gst.h:27,
                 from ../validate/plugins/flow/formatting.h:26,
                 from ../validate/plugins/flow/formatting.c:30:
/builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/lib/glib-2.0/include/glibconfig.h:69:28: note: format string is defined here
 #define G_GUINT64_FORMAT "llu"
                            ^
../validate/plugins/flow/formatting.c:68:22: error: too many arguments for format [-Werror=format-extra-args]
   sprintf (dest_str, "%" G_GUINT64_FORMAT, number);
                      ^~~
../validate/plugins/flow/formatting.c:68:22: error: unknown conversion type character 'l' in format [-Werror=format=]
In file included from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/glib-2.0/glib/gtypes.h:32,
                 from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/glib-2.0/glib/galloca.h:32,
                 from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/glib-2.0/glib.h:30,
                 from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/gstreamer-1.0/gst/gst.h:27,
                 from ../validate/plugins/flow/formatting.h:26,
                 from ../validate/plugins/flow/formatting.c:30:
/builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/lib/glib-2.0/include/glibconfig.h:69:28: note: format string is defined here
 #define G_GUINT64_FORMAT "llu"
                            ^
../validate/plugins/flow/formatting.c:68:22: error: too many arguments for format [-Werror=format-extra-args]
   sprintf (dest_str, "%" G_GUINT64_FORMAT, number);
                      ^~~
```

Needed for https://gitlab.freedesktop.org/gstreamer/cerbero/merge_requests/419

meson: remove vs_module_defs

The GST_EXPORT should handle it.

launcher: Do not dump output on known issues

And remove dead code

validate: Fix leak parsing structure files

validate: Enhance error reporting for errors in struct files

Get a sense of files and line numbers in the parsed GstStructure
and take that information when reporting GstValidateAction errors
by letting the user know where the action comes from in the messages.

And accept non-literal string in printing formats.

validate: Add a specific error type for check actions

And mark it as `NO_BACKTRACE | FULL_DETAILS`, same as for
other action failure types.

validate: Enhance error reporting when scenario or configs are invalid

validate: Add a flags to issues

Currently those allow registering issue that:
- Won't print backtrace as it is sometimes useless info
- Will repeat the details even in smart mode

validate: Consider ',' as continuing line marker in struct files

Since `,` is the separator between fields of GstStructure we can
safely consider that if a line ends with it, the following line
is the logical continuity of the serialized GstStructure.

This makes writing those files more convenient and reading them
more pleasant as we do not need to add extra `\` at end of lines
anymore

validate:launcher: Handle validate report bigger than allowed by the socket

This almost never happens but I had a case where we had a report
with a GstSample in the caps that were reported leading to an
error printed.

validate:launcher: Reference exception in a var as we use it in the handling

validate:flow: Make field filtering in what is logged more generic

Instead of forcing it on event, allow specifying filters on anything
we log, meaning also buffers

validateflow: Add a logged-event-fields configuration

validate:scenario: Stop rounding up clocktime values

This doesn't make any sense in that context

validate:launcher: Fix defining several scenario with a same config

When generating tests from dictionary the dict format allows passing
several scenario for a same config and pipelines, but this was breaking
the case where expected flow is different with each config, instead we
should generate one config per scenario, fixing the expectation files
generated.