validate: Allow using the new nested structure syntax

And port the deeply nested tests we have

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/233>

validate: Only consider the first pipeline when using test files

And port the deeply nested tests we have

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/233>

validate: Add missing GstValidateAction annotations

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/233>

validate: Add an `expected-values` parameter to `wait, message-type=XX`

Allowing more precise filtering of the message we are waiting for.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/233>

validate: scenario: Add a GstValidateScenario::action-done signal

Allowing application to know when a specific action is done.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/233>

validate: Enhance printing action execution information

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/233>

validate: Add an API to get the bin monitor scenario

This is useful for applications that use Validate directly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/233>

scenario: Ensure that messages are handled from the right thread

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/233>

debug-viewer: Make appdata valid again

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/229>

validate:scenario: Fix the refcount management for actions in structures

Handling the refcounting the same whether the action is blocking or not
as we were leaking a ref for non-blocking waits.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/231>

validate:scenario: Minor cleanup

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/231>

validate: Use gst_validate_action_*ref everywhere.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/231>

validate: Add support to check properties of object properties

And recursively

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/231>

validate: add sync-version

Be able by the command line to change the sync version
which is usually the GST_VALIDATE_TESTSUITE_VERSION
from the test suite

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/227>

scenario: Add a 'non-blocking' flag to the `wait` signal

This way we can execute actions that will lead to the signal
emission later in the execution.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/228>

validate:scenario: Rename 'interlaced' action to 'non-blocking'

It is a better and more understandable naming.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/228>

meson: Check cairo-png dependency

Should check whether libpng dependent methods are available or not

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-build/-/issues/128
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/225>

meson: Enable some MSVC warnings for parity with GCC/Clang

This makes it easier to do development with MSVC by making it warn
on common issues that GCC/Clang error out for in our CI configuration.

Continuation from https://gitlab.freedesktop.org/gstreamer/gst-build/-/merge_requests/223

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/226>

meson: update glib minimum version to 2.56

In order to support the symbol g_enum_to_string in various
project using GStreamer ( gst-validate etc.), the glib minimum
version should be 2.56.0.

Remove compat code as glib requirement
is now > 2.56

Version used by Ubuntu 18.04 LTS

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/224>

validate:launcher: Bump hard timeouts for all transcodin tests

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/172>

launcher: Avoid variable framerate when encoding to theora

It is not supported by theoraenc.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/172>

validate: Scale down even more to speed up encoding in VP9

From 3min to 50secs to execute here.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/172>

validate:transcoding: Port to GstTranscoder

Remove flag to force EOS on sigintr, making it the only choice
Also add support for variable framerate

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/172>

validate: Use pad.last_flowret instead of trying to compute it ourselves

Which makes it more accurate

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/172>

report: Add a way to force backtraces on reports

And stop report simple debug message

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/172>

validate:launcher: Add a way to define test timeout from dicts

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/172>

Meson: Use pkg-config generator

gstvalidate.supp: update location of gst.supp

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/221>

docs: Update gst-validate-launcher documentation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/193>

ci: include template from gst-ci master branch again

Back to development

Release 1.18.0

Revert "Revert "launcher: add webrtcbin datachannel tests to valgrind skip list""

This reverts commit ff79af843bc40a554795ee2a0682cfd12272dd24.

Looks like most of these are still problematic and flaky in valgrind,
so re-add to skiplist for now.

https://gitlab.freedesktop.org/thaytan/gst-plugins-base/-/jobs/4275045

Revert "launcher: add webrtcbin datachannel tests to valgrind skip list"

This reverts commit 369c74941f1607b421bc2f16edcaea0b887926a9.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1383

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/220>

Release 1.17.90

validate:check: Blacklist new ges valgrind test

I can't find what the issue is and the test is very long anyway

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/219>

validate: flow: Plug some leaks

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/219>

validate:scenario: Plug some leaks

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/219>

launcher: add webrtcbin datachannel tests to valgrind skip list

They were previously not run because the sctp plugin wasn't built
but they will be run now that we bundle libusrsctp.

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1465

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/218>

validate: silence g-ir-scanner warnings about GST_IS_VALIDATE_*

Which it complains about because we say our prefix is 'gst_validate'
so it should really be GST_VALIDATE_IS_* instead.

Hide the boilerplate defines from g-ir-scanner, it doesn't need
to process them.

Fixes #46

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/214>

Revert "validate: fix up gir namespace and symbol prefix"

This reverts commit b73e81614021a856fc7d3ff221a9bbf5ebf2ce6e.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/214>

validate: fix up gir namespace and symbol prefix

Change gir namespace and symbol prefix from
GstValidate / gst_validate to Gst / gst, same
as we do for other libs like GstVideo etc.

Helps with warnings about GST_IS_VALIDATE_*

Fixes #46, Closes !214

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/215>

validate: Fix media descriptor mp3 like formats

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/216>

validate-scenario: reflow #ifdef to work around bogus g-ir-scanner warning

Looks like a scanner bug.

The endif comments are the wrong way round too, but that's not it.

gst-validate-scenario.c:126: mismatched #endif  /* G_HAVE_GNUC_VARARGS */

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/213>

Back to development

Release 1.17.2

validate: Always check if scenario is done from the right thread

Action will be set_done from the right thread and we will check if the action is done from there

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/212>

validate: Print errors on action failures

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

validate: Plug some leaks

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

validate: Move action finalization to _set_done where it belongs

gst_validate_action_set_done is the place where we should finalize the
action, not in `execute_next`, this way we better handle printing
interlaced action finalization too.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

validate:scenario: Replace the `sub-action` with a `foreach` action type

Sub-actions were really hard to use and conceptually weird. The
implementation was ugly and made the code complex for nothing.

Instead this commit introduces a `foreach` action type which allows
repeating actions passed in an `actions` array the number of time
specified by any `GstIntRange` value defined in the structure or its
`repeat` field.

This commit also makes sure that all action got through
gst_validate_action_set_done upon finalization.

+ Cleanup surrounding code
+ Add tests

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

validate: scenario: Implement 'repeat' by copying actions

Instead of trying to reuse the same action structure and deal with
that in a complex way, copy the action the required number of times.

And add a simple test

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

validate:scenario: Round results of expressions in a sensible way

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

launcher: Keep running tests forever on KNOWN_ERROR

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

validate: Add private action type to check number of action type calls

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

validate:scenario: Avoid dereferencing NULL structure

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

validate: Fix marking expected issues as criticals

And never mark a repeat expected reports as repeated

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

validate: Add a way to use the expression parser in any field

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

validate: Allow variables to be set with other types than strings

And use value serialization from GStreamer to convert

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

scenario: Add an action to remove a feature/plugin from the registry

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

valiadate: Add a test for setting/checking (timed) properties

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

validate:scenario: Add action to set and check several properties at once

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

validate: Add an action type to set timed value properties

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/207>

debug-viewer: Add screenshot

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/210>

Back to development

Release 1.17.1

validate: flow: record GstRegionOfInterestMeta

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/206>

validate:launcher: Ensure that the main directory exists

Fixes https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/issues/53 part 1

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/209>

validate:tests: Cleanup the way set environment vars

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/209>

validate: Stop trying to support uninstalled autotools

Fix https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/issues/53 part 2

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/209>

validateflow: Plug leak

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/205>

scenario: Fix waiting for signal

The signal callback signature was just wrong and not generic leading to
crash if waiting for any signal that didn't match it. This commit fixes
it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/205>

scenario: Add an option to set properties on all instances

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/205>

validate:flow: Add a way to dump buffer content as hex

Useful in unit tests with very small buffers

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/205>

validate: Cleanup implementation of appsrc-push

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/205>

validate: Remove useless condition

CID 1462652

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/192>

validate:ssim: Avoid dereferencing NULL pointer

CID 1462650

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/192>

validate: Give a proper argv[0] when running test files

Fixes CID 1462613

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/192>

validate:transcoding: Exit after printing the help

Fixes CID 1455575

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/192>

validate: Ensure Scenario isn't NULL when preparing action

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/192>

validate: Update documentation now core plugins are integrated

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/204>

validate: Make extracheck and flow part of core instead plugins

It makes things more complex and doesn't bring anything!

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/204>

launcher: Limit copies of massive debug logs in markdown file

When debugging is activated, we could end up with log files ranging in the
multi-megabyte or even gigabyte range. Copying those is expensive from a cpu/io
point of view in addition to clobbering the storage.

Instead of always copying those files, check if they are smaller than 500kB. If
not, don't copy them and instead provide a link to their location.

Fixes #52

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/203>

validateflow: Allow specifying checksum type

And add an extra mode 'checksum-as-id' which basically numerate
buffers checksums as they are being received so that it is simpler
to compare expectations when you are tracking buffers from both
sinkpads and srcpads.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/202>

validate: Add a parameter to force waiting on the test clock

This allows to wait for a new buffer to reach the sink without
actually cranking that buffer, allowing to quite the test without
waiting for EOS in a 100% reproducible way

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/201>

validate: Error out when a config hasn't been used at all

It probably means a plugin is not available

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/201>

Ship validate as part of a gst-devtools tarball

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/199>

meson: make debug_viewer a feature option

... and disable by default.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/199>

validate: Fix mixup in quarks usages

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/200>

validate: Add missing return func when GLib < 2.50

CID 1463854

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/200>

validate: Update documentation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/200>

validateflow: Add a way to configure when to generate expectations

By default, generate them whenever the file is missing but adding a way
to override that with `validateflow,generate-expectations=true` to force
regenerating them or setting `validateflow,generate-expectations=false`
to disallow generating them (on CI servers for example)

Also update the validateflow documentation to take that into account
and remove references to pipeline.json file which is now gone!

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/200>

scenario: Minor cleanup around pipeline change state management

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/198>

scenario: Mark seek as done only when reaching next state

There is a race where following actions could generate a
flush-start/flush-stop dance but the state change resulting from the
seek hasn't been committed yet, leading to the ASYNC_START being
ignored by GstBin since its pending_state is not VOID when receiving
the ASYNC_START message.

Conceptually it is totally correct to consider an action done when
the state change of the pipeline is stabilized..

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/198>

validate:launcher: Stop wrongly erroring on testsuite loading failure

When the testsuite was actually already loaded as the .py
file was explicitly passed in

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

validate: Fix loading configs from a caps

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

flow: Use bat to color diffs when possible

Adding a function to check if can output colored logs

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

launcher: check: Properly set PLUGIN_PATH and registry when running in gst-build

This makes registry

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

launcher: Ensure that -j tests run in parallel when running forever

So that you can reproduce the issue you want faster!

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>

docs: Update gst-validate-launcher documentation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>