Thibault Saunier [Fri, 8 May 2020 21:35:59 +0000 (17:35 -0400)]
validate: utils: Do not try to replace vars in debug info fields
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/190>
Thibault Saunier [Fri, 8 May 2020 21:35:39 +0000 (17:35 -0400)]
validate: Fix parsing validate tests files with vars on windows
And ensure that we escape windows path in variables
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/190>
Thibault Saunier [Sun, 3 May 2020 05:24:32 +0000 (01:24 -0400)]
validate:launcher:check: Add support for gst-tester-1.0
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/189>
Thibault Saunier [Thu, 7 May 2020 04:23:07 +0000 (00:23 -0400)]
validate: Add support for known-issues in the .validatetest
And add some tests about remaining actions failures
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/189>
Thibault Saunier [Thu, 7 May 2020 02:36:59 +0000 (22:36 -0400)]
validate: Ensure a meta structure is found in test files
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/189>
Thibault Saunier [Thu, 7 May 2020 02:20:58 +0000 (22:20 -0400)]
validate:scenario: Check remaining action on stop, not EOS
When ignoring EOS, on addition action could end up not being executed
and no error was reported which was wrong.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/189>
Thibault Saunier [Mon, 4 May 2020 21:59:28 +0000 (17:59 -0400)]
validate: Fix rendering destination directory path creation
We were ending up creating file:/some/path in cwd
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>
Thibault Saunier [Mon, 4 May 2020 20:59:54 +0000 (16:59 -0400)]
validate: Plug some leaks
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>
Thibault Saunier [Sun, 3 May 2020 05:22:04 +0000 (01:22 -0400)]
validate: Add details on all g_log message reports.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>
Thibault Saunier [Sun, 3 May 2020 05:20:19 +0000 (01:20 -0400)]
validate: Add a mechanism to mark tests as skipped
And use it when a plugin is missing and the user didn't ask for
failure when it happens
And use the TAP[0] synthax to report it
[0]: https://testanything.org
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>
Thibault Saunier [Sun, 3 May 2020 04:54:56 +0000 (00:54 -0400)]
validate: Enhance issue reporting from structures
And properly bail out when required
This is preparatory work for gst-test in core which will use the TAP
protocol
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/187>
Thibault Saunier [Wed, 6 May 2020 02:57:08 +0000 (22:57 -0400)]
validate:flow: Handle some more segment fields filtering
Those slept through when implementing filtering
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/188>
Thibault Saunier [Thu, 9 Apr 2020 20:04:53 +0000 (16:04 -0400)]
ssim: Minor improvements on the way we attach to pads
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>
Edward Hervey [Tue, 5 Jun 2018 15:56:36 +0000 (17:56 +0200)]
validate-scenario: Refactor seek handling
* Store all seek values into a list of pending seeks instead
of hardcoding some values
* Store all segments that sinks received
* Match segments to seeks when all sinks received segments with
the same seqnum
* Detect when a seek did *not* result in segments with identical
matching seqnums
Should allow checking for all types of seek handling, including
flush-less seeks
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>
Edward Hervey [Fri, 15 Jun 2018 08:52:46 +0000 (10:52 +0200)]
validate-report: Add new check for top-level seek
This issue is to detect seeks that don't result in segments
with identical seqnums. This check can be done at the top-level
scenario
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>
Edward Hervey [Tue, 5 Jun 2018 15:55:29 +0000 (17:55 +0200)]
validate-pad-monitor: Post GstBaseSink SEGMENT on the bus
Allows higher-level bin or app (like validate-scenario) to know
what each sink currently has in terms of SEGMENT.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>
Edward Hervey [Tue, 5 Jun 2018 15:53:51 +0000 (17:53 +0200)]
validate-element-monitor: Detect basesink elements
And add a macro
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>
Edward Hervey [Tue, 5 Jun 2018 15:51:44 +0000 (17:51 +0200)]
validate-scenario: Add logging for scenario lock taking/releasing
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>
Thibault Saunier [Thu, 30 Apr 2020 16:39:44 +0000 (12:39 -0400)]
pad-monitor: Reliably track pending seeks
Instead of overriding all values when receiving a seek, store
them as a list of expected values.
This allows handling several seeks in a row, like non-flushing
seeks.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/174>
Thibault Saunier [Wed, 29 Apr 2020 03:06:24 +0000 (23:06 -0400)]
validate: Allow overidding issue severity from configs
Refactoring sensibly to allow getting configs outside the `core` namespace
and outside plugin names.
The `GST_VALIDATE_OVERRIDE` env variable should probably be removed
all together at some point.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/185>
Thibault Saunier [Wed, 29 Apr 2020 03:26:13 +0000 (23:26 -0400)]
validate: Add an option to describe issue types
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/185>
Thibault Saunier [Tue, 28 Apr 2020 16:51:21 +0000 (12:51 -0400)]
validate: Show the exact file line when error out in structure files
And minor stdout enhancements
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/185>
Thibault Saunier [Wed, 29 Apr 2020 01:30:29 +0000 (21:30 -0400)]
validate:launcher: Mark some more tests as too long for valgrind
Bilal Elmoussaoui [Mon, 27 Apr 2020 21:46:02 +0000 (21:46 +0000)]
Apply suggestion to debug-viewer/org.freedesktop.GstDebugViewer.appdata.xml.in
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/155>
Bilal Elmoussaoui [Thu, 23 Jan 2020 15:53:03 +0000 (15:53 +0000)]
Metainfo: backport flathub fixes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/155>
Thibault Saunier [Thu, 19 Mar 2020 21:25:28 +0000 (18:25 -0300)]
validate: Do not check pulling thread when thread is paused
With decodebin3 we have cases where a task has been started in
the `typefind` element but the demuxer is the one pulling (from
its own thread)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/183>
Thibault Saunier [Sat, 25 Apr 2020 03:33:16 +0000 (23:33 -0400)]
validate: Add a check-position action type
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/182>
Thibault Saunier [Sat, 25 Apr 2020 03:32:59 +0000 (23:32 -0400)]
validate:scenario: Enhance failures messages
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/182>
Thibault Saunier [Mon, 13 Apr 2020 20:23:32 +0000 (16:23 -0400)]
validate: Add a way to run a TestClock in scenarios
A TestClock will be used automatically when a scenario has a
`crank-clock` action.
And make `validate` and `debug-viewer` options features in meson,
no reason they weren't and now we require gst-check to build validate
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/182>
Thibault Saunier [Fri, 24 Apr 2020 19:41:10 +0000 (15:41 -0400)]
validate: Introduce the concept of "Test files"
This way we can have a single file that wraps scenarios,
`gst-validate-1.0` arguments, as well as a configuration.
It changes the name of `description` of scenarios to use `meta`
The goal is to replace tests describes in python with dictionary
to fully self contained `.validatetest` files which look like:
```
meta,
handles-states=true,
ignore-eos=true,
gst-validate-args = {
"videotestsrc pattern=blue ! video/x-raw,format=I420,framerate=1/1 ! timeoverlay ! $(videosink) name=videosink allocation-meta-flags=0",
},
configs = {
"$(validateflow), pad=videosink:sink, buffers-checksum=true, ignored-fields={\"buffers=meta\", }",
}
play
seek, start=0.0, stop=5.0, flags=accurate+flush, rate=1.0
crank-clock, expected-elapsed-time=0.0
crank-clock, repeat=4, expected-elapsed-time=1.0
crank-clock, expected-elapsed-time=1.0
stop, on-message=eos
```
Thibault Saunier [Fri, 24 Apr 2020 00:11:14 +0000 (20:11 -0400)]
validate:flow: Sensibly improve stdout
Thibault Saunier [Fri, 24 Apr 2020 00:10:48 +0000 (20:10 -0400)]
validate: Fix criticals around NULL structure usage
Thibault Saunier [Fri, 24 Apr 2020 00:09:53 +0000 (20:09 -0400)]
validate: Fix 'repeat' parameter on non ASYNC action types
Thibault Saunier [Thu, 23 Apr 2020 01:13:06 +0000 (21:13 -0400)]
validate:scenario: Do not consider action with 'on-message' as on addition
Thibault Saunier [Thu, 9 Apr 2020 20:01:25 +0000 (16:01 -0400)]
validate: report: Handle when reporting NULL action
And add information about the action repeat state.
Thibault Saunier [Wed, 22 Apr 2020 17:02:29 +0000 (13:02 -0400)]
validate: Handle comments in multiline expressions
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/179>
Thibault Saunier [Wed, 22 Apr 2020 15:27:16 +0000 (11:27 -0400)]
validate: Fix multi variable in a single structure field
We were keeping using the GMatchInfo even after modifying the string
which is explicitly stated as invalid in the GRegex documentation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/179>
Thibault Saunier [Tue, 21 Apr 2020 19:48:20 +0000 (15:48 -0400)]
validate:scenario: Fix 'on-message' actions execution
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/177>
Thibault Saunier [Tue, 21 Apr 2020 19:28:00 +0000 (15:28 -0400)]
validate: Add { and [ as line continuation markers
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/177>
Nicolas Dufresne [Mon, 13 Apr 2020 19:38:05 +0000 (15:38 -0400)]
validate-launcher: Fix syntax error
This error prevents downloading assets from scratch. This regression was
introduced by MR !145 / commit
2581fef6843bfb53f3fc6f629577c1f013ef84e7
Nicolas Dufresne [Tue, 7 Apr 2020 22:33:08 +0000 (18:33 -0400)]
utils: Fix double free in error case
This was detected by Coverity. The content point would have been freed gain in
the done: label.
CID 1461289
Thibault Saunier [Tue, 24 Mar 2020 00:28:45 +0000 (21:28 -0300)]
validate: Use gst_print for validate report messages
Nicolas Dufresne [Tue, 17 Mar 2020 15:51:32 +0000 (11:51 -0400)]
ci: Port from only: to rules
This fixed CI breakage introduced by gst-ci!247
Thibault Saunier [Tue, 10 Mar 2020 14:50:26 +0000 (11:50 -0300)]
validate: Plug some leaks
And add some valgrind suppression for fontconfig
Thibault Saunier [Wed, 4 Mar 2020 14:07:32 +0000 (11:07 -0300)]
validate:ssim: Allow specifying file framerate to use frame numbers during comparison
Thibault Saunier [Wed, 4 Mar 2020 00:36:21 +0000 (21:36 -0300)]
validate:scenario: Make the action->prepare function return a GstValidateExecuteActionReturn
Implementers might want to report the error themselves
Thibault Saunier [Fri, 28 Feb 2020 16:51:58 +0000 (13:51 -0300)]
validate: Add a way to check last frame number
This introduces a new 'timecode-frame-number' in the 'check-last-sample'
action type se we can verify the number of output frames.
Thibault Saunier [Thu, 20 Feb 2020 11:52:38 +0000 (08:52 -0300)]
validate: Enhance debug message on invalid expression function call
Thibault Saunier [Tue, 4 Feb 2020 21:13:51 +0000 (18:13 -0300)]
validate:utils: Allow plain string in `gst_validate_utils_get_strv`
Thibault Saunier [Mon, 10 Feb 2020 17:55:18 +0000 (14:55 -0300)]
validatessim: Avoid having ':' in file names
This is unsupported on windows
Thibault Saunier [Mon, 3 Feb 2020 19:23:37 +0000 (16:23 -0300)]
validate:ssim: Flush cairo surface before getting pixels
Thibault Saunier [Mon, 3 Feb 2020 14:14:33 +0000 (11:14 -0300)]
validate:launcher: Fix reporting on failure when running forever/fatal
Thibault Saunier [Tue, 14 Jan 2020 13:26:54 +0000 (10:26 -0300)]
validate:ssim: Enhance printing position
Adding a new `gst_validate_print_position` method which also
sends messages to the runner if required.
Thibault Saunier [Wed, 8 Jan 2020 18:26:41 +0000 (15:26 -0300)]
validate:launcher: Add a generator to generate test for frame accurate seeking
Thibault Saunier [Wed, 8 Jan 2020 18:21:11 +0000 (15:21 -0300)]
validate:ssim: Use stream time to reference frames
Thibault Saunier [Wed, 8 Jan 2020 18:18:15 +0000 (15:18 -0300)]
validate:ssim: Don't check neighbor frames when comparing exact same frame
Thibault Saunier [Wed, 8 Jan 2020 18:14:34 +0000 (15:14 -0300)]
validate:ssim: Enhance debugging message when similarity do not match
Printing out where the diff image file is
Thibault Saunier [Tue, 7 Jan 2020 18:48:51 +0000 (15:48 -0300)]
validate:launcher: Enhance support for running ssim tests
Using a special 'ssim' variable in pipeline dicts to activate it
Similare to what we do for validateflow
Thibault Saunier [Tue, 7 Jan 2020 18:46:21 +0000 (15:46 -0300)]
validate:launcher: Add support for skipped media info files
Those are skipped to generate tests by default but are updated when
required, this will allow us to generate specific test on demand for
those
Thibault Saunier [Mon, 6 Jan 2020 19:29:33 +0000 (16:29 -0300)]
validate:ssim: Avoid segfaults trying to attach pads without a template
Thibault Saunier [Mon, 6 Jan 2020 19:26:12 +0000 (16:26 -0300)]
validate:scenario:Handle GStreamer serialized timestamps
Thibault Saunier [Mon, 17 Feb 2020 13:32:48 +0000 (10:32 -0300)]
validate:launcher: Never try to load a testsuite with the same name from different locations
Thibault Saunier [Thu, 13 Feb 2020 13:41:07 +0000 (10:41 -0300)]
validate:media-check: Fix wrong exit code
There are code paths where the runner doesn't contain any issue but an error has already been reported
Thibault Saunier [Tue, 11 Feb 2020 19:01:07 +0000 (16:01 -0300)]
validate:launcher: Add support for the imagesequence protocol
Thibault Saunier [Mon, 3 Feb 2020 14:14:33 +0000 (11:14 -0300)]
validate:launcher: Fix reporting on failure when running forever/fatal
Thibault Saunier [Wed, 8 Jan 2020 12:54:15 +0000 (09:54 -0300)]
validate:launcher: Cache GstValidateMediaDescriptor to avoid reparsing .media_info
Saving another second at startup
Thibault Saunier [Wed, 8 Jan 2020 12:23:19 +0000 (09:23 -0300)]
validate:launcher: Generate profiling data even if an exception happens
Thibault Saunier [Tue, 7 Jan 2020 22:29:05 +0000 (19:29 -0300)]
validate:launcher: Batch inspecting scenarios
Removing almost 1 second to start running tests with the default
testsuite
Thibault Saunier [Mon, 6 Jan 2020 19:27:59 +0000 (16:27 -0300)]
validate:launcher: Fix printed test number
Thibault Saunier [Mon, 6 Jan 2020 19:27:19 +0000 (16:27 -0300)]
validate:launcher: Allow passing configs paths when loading from dictionary
The same way we allow it for scenarios
Philippe Normand [Mon, 2 Mar 2020 12:55:18 +0000 (12:55 +0000)]
debug-viewer: Display nanoseconds in the timestamp column
Nirbheek Chauhan [Thu, 27 Feb 2020 11:03:36 +0000 (16:33 +0530)]
validate: Don't use sprintf + glib format modifiers
We do not have a way to know the format modifiers to use with string
functions provided by the system. `G_GUINT64_FORMAT` and other string
modifiers only work for glib string formatting functions. We cannot
use them for string functions provided by the stdlib. See:
https://developer.gnome.org/glib/stable/glib-Basic-Types.html#glib-Basic-Types.description
```
../validate/plugins/flow/formatting.c: In function 'format_number':
../validate/plugins/flow/formatting.c:68:22: error: unknown conversion type character 'l' in format [-Werror=format=]
sprintf (dest_str, "%" G_GUINT64_FORMAT, number);
^~~
In file included from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/glib-2.0/glib/gtypes.h:32,
from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/glib-2.0/glib/galloca.h:32,
from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/glib-2.0/glib.h:30,
from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/gstreamer-1.0/gst/gst.h:27,
from ../validate/plugins/flow/formatting.h:26,
from ../validate/plugins/flow/formatting.c:30:
/builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/lib/glib-2.0/include/glibconfig.h:69:28: note: format string is defined here
#define G_GUINT64_FORMAT "llu"
^
../validate/plugins/flow/formatting.c:68:22: error: too many arguments for format [-Werror=format-extra-args]
sprintf (dest_str, "%" G_GUINT64_FORMAT, number);
^~~
../validate/plugins/flow/formatting.c:68:22: error: unknown conversion type character 'l' in format [-Werror=format=]
In file included from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/glib-2.0/glib/gtypes.h:32,
from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/glib-2.0/glib/galloca.h:32,
from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/glib-2.0/glib.h:30,
from /builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/include/gstreamer-1.0/gst/gst.h:27,
from ../validate/plugins/flow/formatting.h:26,
from ../validate/plugins/flow/formatting.c:30:
/builds/nirbheek/cerbero/cerbero-build/dist/windows_x86_64/lib/glib-2.0/include/glibconfig.h:69:28: note: format string is defined here
#define G_GUINT64_FORMAT "llu"
^
../validate/plugins/flow/formatting.c:68:22: error: too many arguments for format [-Werror=format-extra-args]
sprintf (dest_str, "%" G_GUINT64_FORMAT, number);
^~~
```
Needed for https://gitlab.freedesktop.org/gstreamer/cerbero/merge_requests/419
Thibault Saunier [Wed, 26 Feb 2020 17:21:52 +0000 (14:21 -0300)]
meson: remove vs_module_defs
The GST_EXPORT should handle it.
Thibault Saunier [Wed, 26 Feb 2020 15:05:39 +0000 (12:05 -0300)]
launcher: Do not dump output on known issues
And remove dead code
Thibault Saunier [Wed, 26 Feb 2020 13:52:12 +0000 (10:52 -0300)]
validate: Fix leak parsing structure files
Thibault Saunier [Tue, 25 Feb 2020 14:00:57 +0000 (11:00 -0300)]
validate: Enhance error reporting for errors in struct files
Get a sense of files and line numbers in the parsed GstStructure
and take that information when reporting GstValidateAction errors
by letting the user know where the action comes from in the messages.
And accept non-literal string in printing formats.
Thibault Saunier [Tue, 25 Feb 2020 13:54:00 +0000 (10:54 -0300)]
validate: Add a specific error type for check actions
And mark it as `NO_BACKTRACE | FULL_DETAILS`, same as for
other action failure types.
Thibault Saunier [Thu, 16 Jan 2020 20:36:54 +0000 (17:36 -0300)]
validate: Enhance error reporting when scenario or configs are invalid
Thibault Saunier [Tue, 14 Jan 2020 13:23:39 +0000 (10:23 -0300)]
validate: Add a flags to issues
Currently those allow registering issue that:
- Won't print backtrace as it is sometimes useless info
- Will repeat the details even in smart mode
Thibault Saunier [Thu, 20 Feb 2020 01:16:44 +0000 (22:16 -0300)]
validate: Consider ',' as continuing line marker in struct files
Since `,` is the separator between fields of GstStructure we can
safely consider that if a line ends with it, the following line
is the logical continuity of the serialized GstStructure.
This makes writing those files more convenient and reading them
more pleasant as we do not need to add extra `\` at end of lines
anymore
Thibault Saunier [Tue, 11 Feb 2020 12:18:23 +0000 (09:18 -0300)]
validate:launcher: Handle validate report bigger than allowed by the socket
This almost never happens but I had a case where we had a report
with a GstSample in the caps that were reported leading to an
error printed.
Thibault Saunier [Thu, 6 Feb 2020 13:34:40 +0000 (10:34 -0300)]
validate:launcher: Reference exception in a var as we use it in the handling
Thibault Saunier [Tue, 4 Feb 2020 21:14:25 +0000 (18:14 -0300)]
validate:flow: Make field filtering in what is logged more generic
Instead of forcing it on event, allow specifying filters on anything
we log, meaning also buffers
Thibault Saunier [Tue, 4 Feb 2020 19:59:39 +0000 (16:59 -0300)]
validateflow: Add a logged-event-fields configuration
Thibault Saunier [Tue, 11 Feb 2020 18:57:56 +0000 (15:57 -0300)]
validate:scenario: Stop rounding up clocktime values
This doesn't make any sense in that context
Thibault Saunier [Thu, 6 Feb 2020 17:33:56 +0000 (14:33 -0300)]
validate:launcher: Fix defining several scenario with a same config
When generating tests from dictionary the dict format allows passing
several scenario for a same config and pipelines, but this was breaking
the case where expected flow is different with each config, instead we
should generate one config per scenario, fixing the expectation files
generated.
Thibault Saunier [Thu, 16 Jan 2020 00:22:49 +0000 (21:22 -0300)]
validate:launcher: Print the duration of the test run in the logs
Thibault Saunier [Thu, 16 Jan 2020 00:15:30 +0000 (21:15 -0300)]
validate:flow: Add a way to set the types of events to log/ignore
Added two properties to the plugin:
* ignored-event-types: A list of event types to be ignored when logging events
* logged-event-types: A list of event types to be logged when logging events
This commits also moves the "ignored-event-fields" property to using a proper
GstValueList for the list of event fields to be taken into account, instead
of the home grown separated by comas list of string, making the API more
uniform.
This also adds a simple helper method: `gst_validate_utils_get_strv`
Stéphane Cerveau [Tue, 3 Dec 2019 17:26:18 +0000 (18:26 +0100)]
gst-validate-launcher: separate known error from passed tests
Introduce known_error in statistics to keep in mind the expected
error result.
Stéphane Cerveau [Mon, 2 Dec 2019 13:46:59 +0000 (14:46 +0100)]
gst-validate-launcher: update documentation
Use the new api to create your custom testsuite.
Fix some broken links and enhance the logging system.
Nicolas Dufresne [Sun, 12 Jan 2020 04:00:06 +0000 (23:00 -0500)]
launcher: Allow partionning the tests
This introduce new command line options, --parts and --part-index. When
--parts is set to a value larger then 1, the tests will be split in the
same number of group. The group number identified by --part-index will
be executed.
This is being added in orther to support gliblab CI parallel feature.
Thibault Saunier [Mon, 12 Aug 2019 20:56:41 +0000 (16:56 -0400)]
validate:launcher: Try to dump logs with bat if avalaible
We got to many issues with mdv, it seems not to be maintained
bat is a very good replacement.
Brady J. Garvin [Sun, 5 Jan 2020 20:09:07 +0000 (14:09 -0600)]
validate:launcher: Support mixed str/bytes control sequences.
It is not safe for `_preformat_levels` to assume that all of the fields in a
`TerminalController` have the same type; at least in my environment, some of
these fields are populated with `bytes` while others remain strings.
This change conditionally applies decoding to each control sequence separately
using a helper function `_as_string`. As a side-effect, it also eliminates some
code repetition in `_preformat_levels`.
Closes #50.
Thibault Saunier [Mon, 30 Dec 2019 15:57:57 +0000 (12:57 -0300)]
validate:launcher: Enhance progress reporting using a progress bar
This also allows us to properly report progress on the CI
Thibault Saunier [Mon, 30 Dec 2019 13:27:06 +0000 (10:27 -0300)]
validate:launcher: Take our timeout factor into account for gstcheck
Thibault Saunier [Thu, 7 Nov 2019 18:57:41 +0000 (15:57 -0300)]
validate:scenario: fix some typos
Thibault Saunier [Thu, 7 Nov 2019 18:55:17 +0000 (15:55 -0300)]
validate:scenario: Implement an action to check property value
Alexandru Băluț [Sat, 28 Dec 2019 21:39:23 +0000 (22:39 +0100)]
meson: Remove unused gtk_doc option
The "doc" option is available to disable the documentation.
Mathieu Duponchelle [Mon, 23 Dec 2019 09:27:11 +0000 (10:27 +0100)]
check: unblacklist removed systemclock tests
See https://gitlab.freedesktop.org/gstreamer/gstreamer/merge_requests/348
Mathieu Duponchelle [Sat, 14 Dec 2019 09:45:30 +0000 (10:45 +0100)]
check: unblacklist gstreamer.pipelines_parse_launch.delayed_link
It should not be flaky anymore after
https://gitlab.freedesktop.org/gstreamer/gstreamer/merge_requests/343