ZeroDream/gstreamer
2
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Update

Browse Source
This commit is contained in:
Akkariin Meiko
2022-03-12 03:16:09 +08:00
Unverified
parent 12b76e0c7a
commit 27c4ec74a1
10075 changed files with 5122287 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
subprojects/gst-devtools/docs/api.md
+6
View File
@@ -0,0 +1,6 @@
# GstValidate API reference
This is GstValidate API reference but note that the GstValidate is not
totally stable and might very well change even between minor versions.
The override API should be mostly stable still.
subprojects/gst-devtools/docs/fakesrc.simple.validatetest
+8
View File
@@ -0,0 +1,8 @@
meta,
args = {
"fakesrc num-buffers=1 ! fakesink name=sink",
},
configs = {
"$(validateflow), pad=sink:sink, buffers-checksum=true",
}
# The validate tool will simply play the pipeline until EOS is reached.
subprojects/gst-devtools/docs/fakesrc.simple.validatetest.ini
Symlink
+1
View File
@@ -0,0 +1 @@
fakesrc.simple.validatetest
subprojects/gst-devtools/docs/fakesrc.simple/flow-expectations/log-sink-sink-expected
+4
View File
@@ -0,0 +1,4 @@
event stream-start: GstEventStreamStart, flags=(GstStreamFlags)GST_STREAM_FLAG_NONE, group-id=(uint)1;
event segment: format=BYTES, start=0, offset=0, stop=18446744073709551615, time=0, base=0, position=0
buffer: checksum=da39a3ee5e6b4b0d3255bfef95601890afd80709, dts=0:00:00.000000000, flags=discont
event eos: (no structure)
subprojects/gst-devtools/docs/ges-validate-action-types.md
+1040
View File
File diff suppressed because it is too large Load Diff
subprojects/gst-devtools/docs/gi-index.md
+1
View File
@@ -0,0 +1 @@
# GstValidate API reference
subprojects/gst-devtools/docs/gst-validate-action-types.md
+1531
View File
File diff suppressed because it is too large Load Diff
subprojects/gst-devtools/docs/gst-validate-config.md
+110
View File
@@ -0,0 +1,110 @@
---
title: Configuration
short-description: GstValidate configuration
...
# GstValidate Configuration
GstValidate comes with some possible configuration files
to setup its plugins and core behaviour. The config format is very similar
to the [scenario](gst-validate-scenarios.md) file format.
You can check the [ssim plugin](plugins/ssim.md)
and the [validate flow plugin](gst-validate-flow.md)
for examples.
## Core settings parameters
Config name should be `core`.
### `verbosity`
Default: `position`
See [GstValidateVerbosityFlags](GstValidateVerbosityFlags) for possible values.
### `action`
The [action type](gst-validate-action-types.md) to execute, the action type
must be a CONFIG action or the action type must have a `as-config` argument. When the `action`
is specified in a parameter, a validate action is executed using the other parameters of the
config as configuration for the validate scenario action.
#### Example:
```
GST_VALIDATE_CONFIG="core, action=set-property, target-element-name="videotestsrc0", property-name=pattern, property-value=blue" gst-validate-1.0 videotestsrc ! autovideosink
```
This will execute the `set-property, target-element-name="videotestsrc0",
property-name=pattern, property-value=blue` validate action directly from the
config file
### `scenario-action-execution-interval`
Default: `0` meaning that action are executed in `idle` callbacks.
Set the interval between [GstValidateScenario](gst-validate-scenarios.md) actions execution.
### `max-latency`
Default: `GST_CLOCK_TIME_NONE` - disabled
Set the maximum latency reported by the pipeline, over that defined latency the scenario will report
an `config::latency-too-high` issue.
### `max-dropped`
Default: `GST_CLOCK_TIME_NONE` - disabled
The maximum number of dropped buffers, a `config::too-many-buffers-dropped` issue will be reported
if that limit is reached.
### `fail-on-missing-plugin`
Default: `false` meaning that tests are marked as skipped when a GStreamer plugin is missing.
## Variables
You can use variables in the configs the same way you can set them in
[gst-validate-scenarios](gst-validate-scenarios.md).
Defaults variables are:
- `$(TMPDIR)`: The default temporary directory as returned by `g_get_tmp_dir`.
- `$(CONFIG_PATH)`: The path of the running scenario.
- `$(CONFIG_DIR)`: The directory the running scenario is in.
- `$(CONFIG_NAME)`: The name of the config file
- `$(LOGSDIR)`: The directory where to place log files. This uses the
`GST_VALIDATE_LOGSDIR` environment variable if available or `$(TMPDIR)` if
the variables hasn't been set. (Note that the
[gst-validate-launcher](gst-validate-launcher.md) set the environment
variables).
You can also set you own variables by using the `set-vars=true` argument:
``` yaml
core, set-vars=true, log-path=$(CONFIG_DIR/../log)
```
It is also possible to set global variables (also usable from
[scenarios](gst-validate-scenarios.md)) with:
``` yaml
set-globals, TESTSUITE_ROOT_DIR=$(CONFIG_DIR)
```
## `change-issue-severity` settings parameters
You can change issues severity with the `change-issue-severity` configuration
with the following parameters:
* `issue-id`: The GQuark name of the issue, for example: `event::segment-has-wrong-start`,
You can use `gst-validate-1.0 --print-issue-types` to list all issue types.
* `new-severity`: The new [`severity`](GstValidateReportLevel) of the issue
* `element-name` (*optional*): The name of the element the severity
change applies to
* `element-factory-name` (*optional*): The element factory name of the elements the
severity change applies to
* `element-classification` (*optional*): The classification of the elements the
severity change applies to
subprojects/gst-devtools/docs/gst-validate-environment-variables.md
+171
View File
@@ -0,0 +1,171 @@
---
title: Environment variables
short-description: Environment variables influencing runtime behaviour
...
# GstValidate Environment Variables
The runtime behaviour of GstValidate applications can be influenced by a
number of environment variables.
**GST_VALIDATE.**
This environment variable can be set to a list of debug options, which
cause GstValidate to print out different types of test result
information and consider differently the level of the reported issues.
* `fatal-criticals`: Causes GstValidate to consider only critical issues as import enough
to consider the test failed (default behaviour)
* `fatal-warnings`: Causes GstValidate to consider warning, and critical issues as
import enough to consider the test failed
* `fatal-issues`: Causes GstValidate to consider issue, warning, and critical issues
as import enough to consider the test failed
* `print-issues`: Causes GstValidate to print issue, warning and critical issues in
the final reports (default behaviour)
* `print-warnings`: Causes GstValidate to only print warning and critical issues in the
final reports
* `print-criticals`: Causes GstValidate to only print critical issues in the final
reports
**GST_VALIDATE_FILE.**
Set this variable to a colon-separated list of paths to redirect all
GstValidate messages to this file. If left unset, debug messages are
output to standard error.
You can use the special names `stdout` and `stderr` to use those output.
**GST_VALIDATE_APPS_DIR.**
Set this variable to a colon separated list of paths. The validate test
runner will execute all `.py` scripts found within the directories.
By default GstValidate will look for test applications in the folders:
* subprojects/gst-examples/webrtc/check/validate/apps
* subprojects/gst-editing-services/tests/validate
**GST_VALIDATE_PLUGIN_PATH.**
Set this variable to a colon-separated list of paths. GstValidate will
scan these paths for GstPlugin files and add them to the GstRegistry.
By default GstValidate will look for plugins in the user data directory
specified in the [XDG standard]:
`.local/share/gstreamer-GST_API_VERSION/plugins` and the
system wide user data directory:
`/usr/lib/gstreamer-GST_API_VERSION/validate`
**GST_VALIDATE_SCENARIOS_PATH.**
Set this variable to a colon-separated list of paths. GstValidate will
scan these paths for GstValidate scenario files. By default GstValidate
will look for scenarios in the user data directory as specified in the
[XDG standard]:
`.local/share/gstreamer-GST_API_VERSION/validate/scenarios` and the
system wide user data directory:
`/usr/lib/gstreamer-GST_API_VERSION/validate/scenarios`
**GST_VALIDATE_CONFIG.**
Set this variable to a colon-separated list of paths to GstValidate
config files or directly as a string in the GstCaps serialization
format. The config file has a format similar to the scenario file. The
name of the configuration corresponds to the name of the plugin the
configuration applies to.
The special name "core" is used to configure GstValidate core
functionalities (monitors, scenarios, etc...).
If you want to make sure to set a property on a element of a type (for
example to disable QoS on all sinks) you can do:
```
core, action=set-property, target-element-klass=Sink
```
If you want the GstPipeline to get dumped when an issue of a certain
level (and higher) happens, you can do:
```
core, action=dot-pipeline, report-level=issue
```
Note that you will still need to set GST_DEBUG_DUMP_DOT_DIR.
For more examples you can look at the ssim GstValidate plugin
documentation to see how to configure that plugin.
You can also check that a src pad is pushing buffers at a minimum
frequency. For example to check if v4l2src is producing at least 60 frames
per second you can do:
``` yaml
core,min-buffer-frequency=60,target-element-factory-name=v4l2src
```
This config accepts the following fields:
- `min-buffer-frequency`: the expected minimum rate, in buffers per
second, at which buffers are pushed on the pad
- `target-element-{factory-name,name,klass}`: the factory-name, object
name or class of the element to check
- `name`: (optional) only check the frequency if the src pad has this
name
- `buffer-frequency-start`: (optional) if defined, validate will
ignore the frequency of the pad during the time specified in this
field, in ns. This can be useful when testing live pipelines where
configuring and setting up elements can take some time slowing down
the first buffers until the pipeline reaches its cruising speed.
**GST_VALIDATE_OVERRIDE.**
Set this variable to a colon-separated list of dynamically linkable
files that GstValidate will scan looking for overrides. By default
GstValidate will look for scenarios in the user data directory as
specified in the [XDG standard]:
`.local/share/gstreamer-GST_API_VERSION/validate/scenarios` and the
system wide user data directory:
`/usr/lib/gstreamer-GST_API_VERSION/validate/scenarios`
**GST_VALIDATE_SCENARIO_WAIT_MULITPLIER.**
A decimal number to set as a multiplier for the wait actions. For
example if you set `GST_VALIDATE_SCENARIO_WAIT_MULITPLIER=0.5`, for a
wait action that has a duration of 2.0 the waiting time will only be of
1.0 second. If set to 0, wait action will be ignored.
**GST_VALIDATE_REPORTING_DETAILS.**
The reporting level can be set through the
GST_VALIDATE_REPORTING_DETAILS environment variable, as a
comma-separated list of (optional) object categories / names and levels.
Omit the object category / name to set the global level.
Examples:
```
GST_VALIDATE_REPORTING_DETAILS=synthetic,h264parse:all
GST_VALIDATE_REPORTING_DETAILS=none,h264parse::sink_0:synthetic
```
Levels being:
* `none`: No debugging level specified or desired. Used to deactivate
debugging output.
* `synthetic`: Summary of the issues found, with no details.
* `subchain`: If set as the default level, similar issues can be reported multiple
times for different subchains. If set as the level for a particular
object (`my_object:subchain`), validate will report the issues where
the object is the first to report an issue for a subchain.
* `monitor`: If set as the default level, all the distinct issues for all the
monitors will be reported. If set as the level for a particular
object, all the distinct issues for this object will be reported.
Note that if the same issue happens twice on the same object, up
until this level that issue is only reported once.
* `all`: All the issues will be reported, even those that repeat themselves
inside the same object. This can be **very** verbose if set
globally.
Setting the reporting level allows to control the way issues are
reported when calling [gst_validate_runner_printf()](gst_validate_runner_printf).
[XDG standard]: http://www.freedesktop.org/wiki/Software/xdg-user-dirs/
subprojects/gst-devtools/docs/gst-validate-flow.md
+196
View File
@@ -0,0 +1,196 @@
---
title: Validate Flow
short-description: Validate a pad data flow
...
# Validate Flow
Validate Flow — GStreamer validate component to record a log of buffers and
events flowing in a specified pad and compare it with an expectation file.
## Description
This component exists for the purpose of testing non-regular-playback use cases
where the test author specifies the full pipeline, a series of actions and needs
to check whether the generated buffers and events make sense.
The testing procedure goes like this:
1. The test author writes a [.validatetest](gst-validate-test-file.md) test
where validateflow is used. A pad where monitoring will occur is specified
and possibly a list of [actions](gst-validate-action-types.md) to run can
also be specified.
2. The test author runs the test with the desired pipeline, the configuration
and the actions. Since an expectation file does not exist at
this point, validateflow will create one. The author should check its
contents for any missing or unwanted events. No actual checking is done by
validateflow in this step, since there is nothing to compare to yet.
3. Further executions of the test will also record the produced buffers and
events, but now they will be compared to the previous log (expectation file).
Any difference will be reported as a test failure. The original expectation
file is never modified by validateflow. Any desired changes can be made by
editing the file manually or deleting it and running the test again.
## Example
### Simplest example
The following is an example of a `fakesrc.simple.validatetest` file using
validateflow.
{{ fakesrc.simple.validatetest.ini }}
Then generate the expectation file with:
``` bash
gst-validate-1.0 --set-test-file /path/to/fakesrc.simple.validatetest
```
This will generate the
`/path/to/fakesrc.simple/flow-expectations/log-sink-sink-expected` file
containing:
{{ plugins/fakesrc.simple/flow-expectations/log-sink-sink-expected.log }}
Note that the test will be marked as "SKIPPED" when we generate expectation
files.
The test can now be run with:
```
gst-validate-1.0 --set-test-file /path/to/fakesrc.simple.validatetest
```
### Example controlling the source
The following is an example of the `qtdemux_change_edit_list.validatetest` file using validateflow.
``` ini
set-globals, media_dir="$(test_dir)/../../../medias/"
meta,
seek=false,
handles-states=false,
args = {
"appsrc ! qtdemux ! fakesink async=false",
},
configs = {
"$(validateflow), pad=fakesink0:sink, record-buffers=false",
}
# Scenario action types
appsrc-push, target-element-name=appsrc0, file-name="$(media_dir)/fragments/car-20120827-85.mp4/init.mp4"
appsrc-push, target-element-name=appsrc0, file-name="$(media_dir)/fragments/car-20120827-85.mp4/media1.mp4"
checkpoint, text="A moov with a different edit list is now pushed"
appsrc-push, target-element-name=appsrc0, file-name="$(media_dir)/fragments/car-20120827-86.mp4/init.mp4"
appsrc-push, target-element-name=appsrc0, file-name="$(media_dir)/fragments/car-20120827-86.mp4/media2.mp4"
stop
```
This example shows the elements of a typical validate flow test (a pipeline, a
config and a scenario). Some actions typically used together with validateflow
can also be seen. Notice variable interpolation is used to fill absolute paths
for media files in the scenario (`$(test_dir)`). In the configuration,
`$(validateflow)` is expanded to something like this, containing proper paths
for expectations and actual results (these values are interpolated from the
`.validatetest` file location):
``` ini
validateflow, expectations-dir="/validate/test/file/path/validateqtdemux_change_edit_list/flow-expectations/", actual-results-dir="$(GST_VALIDATE_LOGSDIR)/logs/validate/launch_pipeline/qtdemux_change_edit_list"
```
The resulting log looks like this:
``` ini
event stream-start: GstEventStreamStart, flags=(GstStreamFlags)GST_STREAM_FLAG_NONE, group-id=(uint)1;
event caps: video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)2.1, profile=(string)main, codec_data=(buffer)014d4015ffe10016674d4015d901b1fe4e1000003e90000bb800f162e48001000468eb8f20, width=(int)426, height=(int)240, pixel-aspect-ratio=(fraction)1/1;
event segment: format=TIME, start=0:00:00.000000000, offset=0:00:00.000000000, stop=none, time=0:00:00.000000000, base=0:00:00.000000000, position=0:00:00.000000000
event tag: GstTagList-stream, taglist=(taglist)"taglist\,\ video-codec\=\(string\)\"H.264\\\ /\\\ AVC\"\;";
event tag: GstTagList-global, taglist=(taglist)"taglist\,\ datetime\=\(datetime\)2012-08-27T01:00:50Z\,\ container-format\=\(string\)\"ISO\\\ fMP4\"\;";
event tag: GstTagList-stream, taglist=(taglist)"taglist\,\ video-codec\=\(string\)\"H.264\\\ /\\\ AVC\"\;";
event caps: video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)2.1, profile=(string)main, codec_data=(buffer)014d4015ffe10016674d4015d901b1fe4e1000003e90000bb800f162e48001000468eb8f20, width=(int)426, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)24000/1001;
CHECKPOINT: A moov with a different edit list is now pushed
event caps: video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3, profile=(string)main, codec_data=(buffer)014d401effe10016674d401ee8805017fcb0800001f480005dc0078b168901000468ebaf20, width=(int)640, height=(int)360, pixel-aspect-ratio=(fraction)1/1;
event segment: format=TIME, start=0:00:00.041711111, offset=0:00:00.000000000, stop=none, time=0:00:00.000000000, base=0:00:00.000000000, position=0:00:00.041711111
event tag: GstTagList-stream, taglist=(taglist)"taglist\,\ video-codec\=\(string\)\"H.264\\\ /\\\ AVC\"\;";
event tag: GstTagList-stream, taglist=(taglist)"taglist\,\ video-codec\=\(string\)\"H.264\\\ /\\\ AVC\"\;";
event caps: video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3, profile=(string)main, codec_data=(buffer)014d401effe10016674d401ee8805017fcb0800001f480005dc0078b168901000468ebaf20, width=(int)640, height=(int)360, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)24000/1001;
```
## Configuration
In order to use the plugin a validate configuration must be provided,
containing a line starting by `validateflow` followed by a number of settings.
Every `validateflow` line creates a `ValidateFlowOverride`, which listens to a
given pad. A test may have several `validateflow` lines, therefore having
several overrides and listening to different pads with different settings.
* `pad`: Required. Name of the pad that will be monitored.
* `record-buffers`: Default: false. Whether buffers will be logged. By default
only events are logged.
* `buffers-checksum`: Default: 'none'. Define the type of checksums to be used
valid values are:
* `none`: No checksum recorded
* `as-id`: Record checksum as 'ids' where the IDs are incremented on each new
checksum passed in
* `md5`: md5 checksum
* `sha1`: sha1 checksum
* `sha256`: sha256 checksum
* `sha512`: sha512 checksum
* *Note*: for backward compatibility reasons, this can be passed as a
boolean and it will default to 'sha1' if true, 'none' if false.
* `ignored-fields`: Default: `"stream-start={ stream-id }"` (as they are often
non reproducible). Key with a serialized GstValueList(str) of fields to not
record.
* `logged-fields`: Default: `NULL` Key with a serialized GstValueList(str) of
fields to record, eg. `logged-event-fields="stream-start={flags},
caps={width, height, framerate}, buffer={pts}"`. Overrides
`ignored-event-fields` for specified event types.
* `ignored-event-types`: Default: `{ }`. List of event type names to not record
* `logged-event-types`: Default: `NULL`. List of event type names to not record,
if noone provided, all events are logged, except the ones defined in the
`ignored-event-types`.
* `expectations-dir`: Path to the directory where the expectations will be
written if they don't exist, relative to the current working directory. By
default the current working directory is used, but this setting is usually
set automatically as part of the `%(validateflow)s` expansion to a correct
path like `~/gst-validate/gst-integration-testsuites/flow-expectations/<test
name>`.
* `actual-results-dir`: Path to the directory where the events will be recorded.
The expectation file will be compared to this. By default the current working
directory is used, but this setting is usually set automatically as part of
the `%(validateflow)s` expansion to the test log directory, i.e.
`~/gst-validate/logs/validate/launch_pipeline/<test name>`.
* `generate-expectations`: Default: unset. When set to `true` the expectation
file will be written and no testing will be done and if set to `false`, the
expectation file will be required. If a validateflow config is used without
specifying any other parametters, the validateflow plugin will consider that
all validateflow overrides will use that value.
## Scenario actions
Scenarios with validateflow work in the same way as other tests. Often
validatetests will use appsrc in order to control the flow of data precisely,
possibly interleaving events in between. The following is a list of useful
actions.
* `appsrc-push`: Pushes a buffer from an appsrc element and waits for the chain
operation to finish. A path to a file is provided, optionally with an offset
and/or size.
* `appsrc-eos`: Queues an EOS event from the appsrc. The action finishes
immediately at this point.
* `stop`: Tears down the pipeline and stops the test.
* `checkpoint`: Records a "checkpoint" message in all validateflow overrides,
with an optional explanation message. This is useful to check certain events
or buffers are sent at a specific moment in the scenario, and can also help
to the comprehension of the scenario.
More details on these actions can be queried from the command line, like this:
``` bash
gst-validate-1.0 --inspect-action-type appsrc-push
```
subprojects/gst-devtools/docs/gst-validate-launcher.md
+167
View File
@@ -0,0 +1,167 @@
---
short-description: Integration testsuite builder and launcher
...
# gst-validate-launcher
`gst-validate-launcher` is an application to run unit or integration testsuites
providing a set of options and features to help debugging them.
## Run the GStreamer unit tests
Running GStreamer unit tests is as simple as doing:
```
gst-validate-launcher check.gst*
```
If you only want to run GStreamer core tests:
```
gst-validate-launcher check.gstreamer*
```
Or to run unit tests from gst-plugins-base
```
gst-validate-launcher check.gst-plugins-base
```
You can also run them inside valgrind with the `-vg` option or inside gdb with
`--gdb` for example.
## Run the GstValidate default testsuite
GstValidate comes with a default testsuite to be executed on a default
set of media samples. Those media samples are stored with `git-lfs` so
you will need it to be able to launch the default testsuite.
Then you can run:
```
gst-validate-launcher validate
```
This will only launch the GstValidate tests and not other applications
that might be supported (currently `ges-launch` is also supported and
has its own default testsuite).
## Example of a testsuite implementation
To implement a testsuite, you will have to write some simple python code
that defines the tests to be launched by `gst-validate-launcher`.
In this example, we will assume that you want to write a whole new
testsuite based on your own media samples and [scenarios](GstValidateScenario). The
set of media files and the testsuite implementation file will be
structured as follow:
testsuite_folder/
|-> testsuite.py
|-> sample_files/
|-> file.mp4
|-> file1.mkv
|-> file2.ogv
|-> scenarios
|-> scenario.scenario
|-> scenario1.scenario
You should generate the `.media_info` files. To generate them for local
files, you can use:
gst-validate-launcher --medias-paths /path/to/sample_files/ --generate-media-info
For remote streams, you should use
`gst-validate-media-check-1.0`. For an http stream you can
for example do:
gst-validate-media-check-GST_API_VERSION http://someonlinestream.com/thestream \
--output-file /path/to/testsuite_folder/sample_files/thestream.stream_info
The `gst-validate-launcher` will use the generated `.media_info` and
`.stream_info` files to validate the tests as those contain the
necessary information.
Then you will need to write the `testsuite.py` file. You can for example
implement the following testsuite:
``` python
"""
The GstValidate custom testsuite
"""
import os
from launcher.baseclasses import MediaFormatCombination
from launcher.apps.gstvalidate import *
TEST_MANAGER = "validate"
KNOWN_ISSUES = {}
def setup_tests(test_manager, options):
print("Setting up the custom testsuite")
assets_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), ".", "samples_files"))
options.add_paths(assets_dir)
# This step will register default data for the test manager:
# - scenarios such as `play_15s`, `reverse_playback` etc.
# - encoding formats such as "mp4,h264,mp3" etc.
# - blacklist such as dash.media_check.*
# - test generators:
# - GstValidatePlaybinTestsGenerator
# - GstValidateMediaCheckTestsGenerator
# - GstValidateTranscodingTestsGenerator
# This 'defaults' can be found in 'gst-devtools/validate/launcher/apps/gstvalidate.py#register_defaults'
# test_manager.register_defaults()
# Add scenarios
scenarios = []
scenarios.append("play_5s")
scenarios.append("seek_backward")
test_manager.set_scenarios(scenarios)
# Add encoding formats used by the transcoding generator
test_manager.add_encoding_formats([
MediaFormatCombination("mp4", "mp3", "h264"),])
# Add generators
# GstValidatePlaybinTestsGenerator needs at least one media file
test_manager.add_generators([GstValidateMediaCheckTestsGenerator(test_manager)])
# GstValidatePlaybinTestsGenerator needs at least one scenario
test_manager.add_generators([GstValidatePlaybinTestsGenerator(test_manager)])
# GstValidateTranscodingTestsGenerator needs at least one MediaFormatCombination
test_manager.add_generators([GstValidateTranscodingTestsGenerator(test_manager)])
# list of combo to blacklist tests. Here it blacklists all tests with playback.seek_backward
test_manager.set_default_blacklist([
("custom_testsuite.file.playback.seek_backward.*",
"Not supported by this testsuite."),])
# you can even pass known issues to bypass an existing error in your custom testsuite
test_manager.add_expected_issues(KNOWN_ISSUES)
return True
```
Once this is done, you've got a testsuite that will:
- Run playbin pipelines on `file.mp4`, `file1.mkv` and `file2.ogv`&gt;
executing `play_5s` and `seek_backward` scenarios
- Transcode `file.mp4,` `file1.mkv` and `file2.ogv` to h264 and
mp3 in a MP4 container
The only thing to do to run the testsuite is:
gst-validate-launcher --testsuites-dir=/path/to/testsuite_folder/ testsuite
# Invocation
You can find detailed information about the launcher by launching it:
gst-validate-launcher --help
You can list all the tests with:
gst-validate-launcher --testsuites-dir=/path/to/testsuite_folder/ testsuite -L
subprojects/gst-devtools/docs/gst-validate-media-check.md
+32
View File
@@ -0,0 +1,32 @@
---
short-description: Tool to test GStreamer media types discovery
...
# gst-validate-media-check
`gst-validate-media-check` is command line tool checking that media
files discovering works properly with `gst-discoverer` over multiple
runs. It needs a reference text file containing valid information about
a media file (which can be generated with the same tool) and then it
will be able to check that the reference matches what will be reported
by `gst-discoverer` in the following runs.
For example, given that we have a valid `reference.media_info` file, we
can run:
gst-validate-media-check-GST_API_VERSION file:///./file.ogv --expected-results reference.media_info
It will then output any error encountered and return an exit code
different from 0 if any error is found.
# Invocation
`gst-validate-media-check` takes an URI to analyze and some extra
options to control the output.
## Options
* `-o`, `--output-file`: The output file to store the results.
* `-f`, `--full`: Fully analize the file frame by frame.
* `-e`, `--expected-results`: Path to file containing the expected results (or the last results
found) for comparison with new results.
subprojects/gst-devtools/docs/gst-validate-scenarios.md
+110
View File
@@ -0,0 +1,110 @@
---
title: Scenarios
short-description: The GstValidate Scenario format
...
# GstValidate Scenario File Format
To be able to define a list of actions to execute on a [`GstPipeline`],
a dedicated file format is used. The name of the scenario is the name of
the file without its `.scenario` extension. The scenario file format is
based on the [`GstStructure`] serialized format which is a basic, type
aware, key value format. It takes the type of the action in the first
comma separated field, and then some key value pairs in the form
`parameter=value` separated by commas. The values type will be guessed
if not casted as in `parameter=(string)value`. You can force the type
guessing system to actually know what type you want by giving it the
right hints. For example to make sure the value is a double, you should
add a decimal (ie. `1` will be considered as a `int`, but `1.0` will be
considered as a `double` and `"1.0"` will be considered as a `string`).
For example to represent a seek action, you should add the following
line in the `.scenario` file.
seek, playback-time=10.0, start=0.0, flags=accurate+flush
The files to be used as scenario should have a `.scenario` extension and
should be placed either in
`$USER_DATA_DIR/gstreamer-1.0/validate/scenarios` ,
`$GST_DATADIR/gstreamer-1.0/validate/scenarios` or in a path defined in
the \$GST\_VALIDATE\_SCENARIOS\_PATH environment variable.
Each line in the `.scenario` file represent an action (you can also use
`\ ` at the end of a line write a single action on multiple lines).
Usually you should start you scenario with a `meta` structure
in order for the user to have more information about the
scenario. It can contain a `summary` field which is a string explaining
what the scenario does and then several info fields about the scenario.
You can find more info about it running:
gst-validate-1.0 --inspect-action-type action_type_name
So a basic scenario file that will seek three times and stop would look
like:
```
meta, summary="Seeks at 1.0 to 2.0 then at \
3.0 to 0.0 and then seeks at \
1.0 to 2.0 for 1.0 second (between 2.0 and 3.0).", \
seek=true, duration=5.0, min-media-duration=4.0
seek, playback-time=1.0, rate=1.0, start=2.0, flags=accurate+flush
seek, playback-time=3.0, rate=1.0, start=0.0, flags=accurate+flush
seek, playback-time=1.0, rate=1.0, start=2.0, stop=3.0, flags=accurate+flush
```
Many action types have been implemented to help users define their own
scenarios. For example there are:
- `seek`: Seeks into the stream.
- `play`: Set the pipeline state to `GST_STATE_PLAYING`.
- `pause`: Set the pipeline state to `GST_STATE_PAUSED`.
- `stop`: Stop the execution of the pipeline.
> **NOTE**: This action actually posts a [`GST_MESSAGE_REQUEST_STATE`]
> message requesting [`GST_STATE_NULL`] on the bus and the application
> should quit.
To get all the details about the registered action types, you can list
them all with:
```
gst-validate-1.0 --inspect-action-type
```
and to include transcoding specific action types:
```
gst-validate-transcoding-1.0 --inspect-action-type
```
Many scenarios are distributed with `gst-validate`, you can list them
all using:
```
gst-validate-1.0 --list-scenarios
```
You can find more information about the scenario implementation and
action types in the [`GstValidateScenario` section].
[`GstPipeline`]: GstPipeline
[`GstStructure`]: GstStructure
[`GST_MESSAGE_REQUEST_STATE`]: GST_MESSAGE_REQUEST_STATE
[`GST_STATE_NULL`]: GST_STATE_NULL
[`GstValidateScenario` section]: GstValidateScenario
## Default variables
Any action can use the default variables:
- `$(position)`: The current position in the pipeline as reported by
[gst_element_query_position()](gst_element_query_position)
- `$(duration)`: The current duration of the pipeline as reported by
[gst_element_query_duration()](gst_element_query_duration)
- `$(TMPDIR)`: The default temporary directory as returned by `g_get_tmp_dir`.
- `$(SCENARIO_PATH)`: The path of the running scenario.
- `$(SCENARIO_DIR)`: The directory the running scenario is in.
- `$(SCENARIO_NAME)`: The name the running scenario
It is also possible to set variables in scenario with the `set-vars` action.
subprojects/gst-devtools/docs/gst-validate-test-file.md
+118
View File
@@ -0,0 +1,118 @@
---
title: Test file
short-description: GstValidate test file
...
# GstValidate Test file
A `.validatetest` file describes a fully contained validate test case. It
includes the arguments of the tool supposed to be used to run the test as well
as possibly a [configuration](gst-validate-config.md) and a set of action to
describe the validate [scenario](gst-validate-scenarios.md).
# The file format
A validate test file requires a `meta` structure which contains the same
information as the [scenario](gst-validate-scenarios.md) `meta` with some
additional fields described below. The `meta` structure should be either the
first or the one following the `set-globals` structure. The `set-globals`
structures allows you to set global variables for the rest of the
`.validatetest` file and is a free form variables setter. For example you can
do:
``` yaml
set-globals, media_dir=$(test_dir)/../../media
```
## Tool arguments
In the case of [`gst-validate`](gst-validate.md) it **has to** contain an
`args` field with `gst-validate` argv arguments like:
``` yaml
# This is the default tool so it is not mandatory for the `gst-validate` tool
tool = "gst-validate-$(gst_api_version)",
args = {
# pipeline description
videotestrc num-buffers=2 ! $(videosink),
# Random extra argument
--set-media-info $(test-dir)/some.media_info
}
```
## configs
The `configs` field is an array of structures containing the same content as
usual [configs](gst-validate-config.md) files.
For example:
``` yaml
configs = {
# Set videotestsrc0 pattern value to `blue`
"core, action=set-property, target-element-name=videotestsrc0, property-name=pattern, property-value=blue",
"$(validateflow), pad=sink1:sink, caps-properties={ width, height };",
}
```
Note: Since this is GstStructure synthax, we need to have the structures in the
array as strings/within quotes.
## expected-issues
The `expected-issues` field is an array of `expected-issue` structures containing
information about issues to expect (which can be known bugs or not).
Use `gst-validate-1.0 --print-issue-types` to print information about all issue types.
For example:
``` yaml
expected-issues = {
"expected-issue, issue-id=scenario::not-ended",
}
```
Note: Since this is GstStructure synthax, we need to have the structures in the
array as strings/within quotes.
### Fields:
* `issue-id`: (string): Issue ID - Mandatory if `summary` is not provided.
* `summary`: (string): Summary - Mandatory if `issue-id` is not provided.
* `details`: Regex string to match the issue details `detected-on`: (string):
The name of the element the issue happened on `level`: (string):
Issue level
* `sometimes`: (boolean): Default: `false` - Wheteher the issue happens only
sometimes if `false` and the issue doesn't happen, an error will
be issued.
* `issue-url`: (string): The url of the issue in the bug tracker if the issue is
a bug.
### Variables
The same way
Validate testfile will define some variables to make those files relocable:
* `$(test_dir)`: The directory where the `.validatetest` file is in.
* `$(test_name)`: The name of the test file (without extension).
* `$(test_name_dir)`: The name of the test directory (test_name with folder
separator instead of `.`).
* `$(validateflow)`: The validateflow structure name with the default/right
values for the `expectations-dir` and `actual-results-dir`
fields. See [validateflow](gst-validate-flow.md) for more
information.
* `$(videosink)`: The GStreamer videosink to use if the test can work with
different sinks for the video. It allows the tool to use
fakesinks when the user doesn't want to have visual feedback
for example.
* `$(audiosink)`: The GStreamer audiosink to use if the test can work with
different sinks for the audio. It allows the tool to use
fakesinks when the user doesn't want to have audio feedback
for example.
subprojects/gst-devtools/docs/gst-validate-transcoding.md
+126
View File
@@ -0,0 +1,126 @@
---
short-description: Tool to test GStreamer components
...
# gst-validate-transcoding
`gst-validate-transcoding` is tool to create media files transcoding
pipelines running inside the GstValidate monitoring infrastructure.
You can for example transcode any media file to Vorbis audio + VP8 video
in a WebM container by doing:
gst-validate-transcoding-GST_API_VERSION file:///./file.ogg file:///.../transcoded.webm -o 'video/webm:video/x-vp8:audio/x-vorbis'
`gst-validate-transcoding` will list every issue encountered during the
execution of the transcoding operation in a human readable report like
the one below:
issue : buffer is out of the segment range Detected on theoradec0.srcpad
at 0:00:00.096556426 Details : buffer is out of segment and shouldn't be
pushed. Timestamp: 0:00:25.000 - duration: 0:00:00.040 Range:
0:00:00.000 - 0:00:04.520 Description : buffer being pushed is out of
the current segment's start-stop range. Meaning it is going to be
discarded downstream without any use
The return code of the process will be 18 in case a `CRITICAL` issue has
been found.
## The encoding profile serialization format
This is the serialization format of a [GstEncodingProfile](GstEncodingProfile).
Internally the transcoding application uses [GstEncodeBin](encodebin).
`gst-validate-transcoding-GST_API_VERSION` uses its own serialization
format to describe the [`GstEncodeBin.profile`](encodebin:profile) property of the
encodebin.
The simplest serialized profile looks like:
muxer_source_caps:videoencoder_source_caps:audioencoder_source_caps
For example to encode a stream into a WebM container, with an OGG audio
stream and a VP8 video stream, the serialized [GstEncodingProfile](GstEncodingProfile)
will look like:
video/webm:video/x-vp8:audio/x-vorbis
You can also set the preset name of the encoding profile using the
caps+preset\_name syntax as in:
video/webm:video/x-vp8+youtube-preset:audio/x-vorbis
Moreover, you can set the [presence](gst_encoding_profile_set_presence) property
of an encoding profile using the `|presence` syntax as in:
video/webm:video/x-vp8|1:audio/x-vorbis
This field allows you to specify how many times maximum a
[GstEncodingProfile](GstEncodingProfile) can be used inside an encodebin.
You can also use the `restriction_caps->encoded_format_caps` syntax to
specify the [restriction caps](GstEncodingProfile:restriction-caps)
to be set on a [GstEncodingProfile](GstEncodingProfile). It
corresponds to the restriction [GstCaps](GstCaps) to apply before the encoder
that will be used in the profile. The fields present in restriction caps
are properties of the raw stream (that is, before encoding), such as
height and width for video and depth and sampling rate for audio. This
property does not make sense for muxers.
To force a video stream to be encoded with a Full HD resolution (using
WebM as the container format, VP8 as the video codec and Vorbis as the
audio codec), you should use:
video/webm:video/x-raw,width=1920,height=1080->video/x-vp8:audio/x-vorbis
### Some serialized encoding formats examples:
MP3 audio and H264 in MP4:
<div class="informalexample">
video/quicktime,variant=iso:video/x-h264:audio/mpeg,mpegversion=1,layer=3
</div>
Vorbis and theora in OGG:
<div class="informalexample">
application/ogg:video/x-theora:audio/x-vorbis
</div>
AC3 and H264 in MPEG-TS:
<div class="informalexample">
video/mpegts:video/x-h264:audio/x-ac3
</div>
# Invocation
`gst-validate-transcoding` takes and input URI and an output URI, plus a
few options to control how transcoding should be tested.
## Options
* `--set-scenario`: Let you set a scenario, it can be a full path to a scenario file or
the name of the scenario (name of the file without the `.scenario`
extension).
* `-l`, `--list-scenarios`: List the avalaible scenarios that can be run.
* `--scenarios-defs-output-file`: The output file to store scenarios details. Implies
`--list-scenario`.
* `-t`, `--inspect-action-type`: Inspect the avalaible action types with which to write scenarios if
no parameter passed, it will list all avalaible action types
otherwize will print the full description of the wanted types.
* `--set-configs`: Let you set a config scenario. The scenario needs to be set as
`config`. You can specify a list of scenarios separated by `:`. It
will override the GST\_VALIDATE\_SCENARIO environment variable.
* `-e`, `--eos-on-shutdown`: If an EOS event should be sent to the pipeline if an interrupt is
received, instead of forcing the pipeline to stop. Sending an EOS
will allow the transcoding to finish the files properly before
exiting.
* `-r`, `--force-reencoding`: Whether to try to force reencoding, meaning trying to only remux if
possible, defaults to `TRUE`.
subprojects/gst-devtools/docs/gst-validate.md
+58
View File
@@ -0,0 +1,58 @@
---
short-description: Tool to test GStreamer components
...
# gst-validate
`gst-validate` is the simplest `gst-launch`-like pipeline launcher
running inside GstValidate monitoring infrastructure. Monitors are added
to it to identify issues in the used elements. At the end it will print
a report with some information about all the issues encountered during
its run. To view issues as they are detected, set the environment
variable `GST_DEBUG=validate:2`{.shell} and they will get printed in the
GStreamer debug log. You can basically run any [GstPipeline](GstPipeline) pipeline
using this tool. If you are not familiar with `gst-launch` syntax,
please refer to `gst-launch`'s documentation.
Simple playback pipeline:
gst-validate-1.0 playbin uri=file:///path/to/some/media/file
Transcoding pipeline:
gst-validate-1.0 filesrc location=/media/file/location ! qtdemux name=d ! queue \
! x264enc ! h264parse ! mpegtsmux name=m ! progressreport \
! filesink location=/root/test.ts d. ! queue ! faac ! m.
It will list each issue that has been encountered during the execution
of the specified pipeline in a human readable report like:
issue : buffer is out of the segment range Detected on theoradec0.srcpad at 0:00:00.096556426
Details : buffer is out of segment and shouldn't be pushed. Timestamp: 0:00:25.000 - duration: 0:00:00.040 Range: 0:00:00.000 - 0:00:04.520
Description : buffer being pushed is out of the current segment's start-stop range. Meaning it is going to be discarded downstream without any use
The return code of the process will be 18 in case a `CRITICAL` issue has
been found.
# Invocation
`gst-validate` takes a mandatory description of the pipeline to launch,
similar to `gst-launch`, and some extra options.
## Options
* `--set-scenario`: Let you set a scenario, it can be a full path to a scenario file or
the name of the scenario (name of the file without the `.scenario`
extension).
* `-l`, `--list-scenarios`: List the avalaible scenarios that can be run.
* `--scenarios-defs-output-file`: The output file to store scenarios details. Implies
`--list-scenario`.
* `-t`, `--inspect-action-type`: Inspect the avalaible action types with which to write scenarios if
no parameter passed, it will list all avalaible action types
otherwize will print the full description of the wanted types.
* `--set-media-info`: Set a media\_info XML file descriptor to share information about the
media file that will be reproduced.
* `--set-configs`: Let you set a config scenario. The scenario needs to be set as
`config`. You can specify a list of scenarios separated by "`:`". It
will override the GST\_VALIDATE\_SCENARIO environment variable.
subprojects/gst-devtools/docs/index.md
+26
View File
@@ -0,0 +1,26 @@
# GStreamer Validate
GstValidate is a tool that allows GStreamer developers to check that the
GstElements they write behave the way they are supposed to. It was first
started to provide plug-ins developers with a tool to check that they
use the framework the proper way.
GstValidate implements a monitoring logic that allows the system to
check that the elements of a GstPipeline respect some rules GStreamer
components have to follow to make them properly interact together. For
example, a GstValidatePadMonitor will make sure that if we receive a
GstSegment from upstream, an equivalent segment is sent downstream
before any buffer gets out.
Then GstValidate implements a reporting system that allows users to get
detailed informations about what was not properly handled by the
elements. The generated reports are ordered by level of importance from
"issue" to "critical".
Some tools have been implemented to help developers validate and test
their GstElement, see [gst-validate](gst-validate.md) for example.
On top of that, the notion of a [validation scenario](gst-validate-scenarios.md)
has been implemented so that developers can easily execute a set of actions on
pipelines to test real world interactive cases and reproduce existing
issues in a convenient way.
subprojects/gst-devtools/docs/meson.build
+75
View File
@@ -0,0 +1,75 @@
build_hotdoc = false
if meson.is_cross_build()
if get_option('doc').enabled()
error('Documentation enabled but building the doc while cross building is not supported yet.')
endif
message('Documentation not built as building it while cross building is not supported yet.')
subdir_done()
endif
if static_build
if get_option('doc').enabled()
error('Documentation enabled but not supported when building statically.')
endif
message('Building statically, can\'t build the documentation')
subdir_done()
endif
if not build_gir
if get_option('doc').enabled()
error('Documentation enabled but introspection not built.')
endif
message('Introspection not built, won\'t build documentation requiring it')
subdir_done()
endif
hotdoc_p = find_program('hotdoc', required: get_option('doc'))
if not hotdoc_p.found()
message('Hotdoc not found, not building the documentation')
subdir_done()
endif
required_hotdoc_extensions = ['gi-extension']
hotdoc = import('hotdoc')
foreach extension: required_hotdoc_extensions
if not hotdoc.has_extensions(extension)
if get_option('doc').enabled()
error('Documentation enabled but @0@ missing'.format(extension))
endif
message('@0@ extension not found, not building documentation'.format(extension))
subdir_done()
endif
endforeach
excludes = ['gettext.h',
'gst-validate-internal.h',
'gst-validate-i18n-lib.c'
]
build_hotdoc = true
validate_excludes = []
foreach f: excludes
validate_excludes += [join_paths(meson.current_source_dir(), '..',
'validate', 'gst', 'validate', f)]
endforeach
validate_sources = gstvalidate_headers + gstvalidate_sources
hotdoc = import('hotdoc')
plugins_doc = []
libs_doc = [hotdoc.generate_doc('gst-devtools',
project_version: apiversion,
sitemap: 'sitemap.txt',
index: 'index.md',
gi_c_sources: validate_sources,
gi_c_source_filters: validate_excludes,
gi_index: 'gi-index.md',
gi_smart_index: true,
gi_sources: [validate_gir[0].full_path()],
disable_incremental_build: true,
dependencies : [validate_dep],
)]
subprojects/gst-devtools/docs/plugins/index.md
+3
View File
@@ -0,0 +1,3 @@
# GstValidate plugins
GstValidate offers a plugin system to extend the checks and add new functionnality
subprojects/gst-devtools/docs/plugins/ssim.md
+87
View File
@@ -0,0 +1,87 @@
---
title: SSIM plugin
short_description: GstValidate plugin to detect frame corruptions
...
# SSIM plugin
GstValidate plugin to run the ssim algorithm on the buffers flowing in the
pipeline to find regressions and detect frame corruptions.
It allows you to generate image files from the buffers flowing in the pipeline
(either as raw in the many formats supported by GStreamer or as png) and then
check them against pre generated, reference images.
The ssim algorithm will set a value of 1.0 when images are perfectly identical,
and -1.0 if they have nothing in common. By default we consider images as similar
if they have at least a ssim value of 0.95 but you can override it defining the value
under which the test will be considered as failed.
Errors are reported on the GstValidate reporting system. You can also ask
the plugin to generate grey scale output images. Those will be named in a way
that should lets you precisely see where and how the test failed.
# Configuration
The configuration of the plugin is done through a validate configuration file,
specified with the %GST_VALIDATE_CONFIG environment variable. Each line starting
with 'ssim,' will configure the ssim plugin. In practice each configuration statement
will lead to the creation of a #GstValidateOverride object which will then dump
image files and if wanted compare those with a set of reference images.
The following parameters can be passed in the configuration file:
- element-classification: The target element classification as define in
gst_element_class_set_metadata
- output-dir: The directory in which the image files will be saved
- min-avg-priority: (default 0.95): The minimum average similarity
under which we consider the test as failing
- min-lowest-priority: (default 1): The minimum 'lowest' similarity
under which we consider the test as failing
- reference-images-dir: Define the directory in which the files to be
compared can be found
- result-output-dir: The folder in which to store resulting grey scale
images when the test failed. In that folder you will find images
with the structural difference between the expected result and the actual
result.
- output-video-format: The format in which you want the images to be saved
- reference-video-format: The format in which the reference images are stored
- check-recurrence: The recurrence in seconds (as float) the frames should
be dumped and checked.By default it is GST_CLOCK_TIME_NONE, meaning each
and every frame is checked. Not that in any case, after a discontinuity
in the stream (after a seek or a change in the video format for example)
a check is done. And if recurrence == 0, images will be checked only after
such discontinuity
- is-config: Property letting the plugin know that the config line is exclusively
used to configure the following configuration expressions. In practice this
means that it will change the default values for the other configuration
expressions.
- framerate: (GstFraction): The framerate to use to compute frame number from
timestamp, allowing to compare frames by 'frame number' instead of trying to
match timestamp between reference images and output images.
# Example #
Let's take a special configuration where we want to compare frames that are
outputted by a video decoder with the ones after a agingtv element we would
call my_agingtv. We force to check one frame every 5.0 seconds only (with
check-recurrence=5.0) so the test is fast.
The configuration file:
``` shell
core, action=set-property, target-element-klass=Sink, property-name=sync, property-value=false
validatessim, is-config=true, output-video-format="I420", reference-video-format="I420"
validatessim, element-classification="Video/Decoder", output-dir=/tmp/test/before-agingtv/
validatessim, element-name=my_agingtv, output-dir=/tmp/test/after-agingtv/, \
reference-images-dir=/tmp/test/before-agingtv/, \
result-output-dir=/tmp/test/failures, check-recurrence=5.0
```
Save that content in a file called check_agingtv_ssim.config
## Launch the pipeline
``` shell
GST_VALIDATE_CONFIG=check_agingtv_ssim.config gst-validate-1.0-debug uridecodebin uri=file://a/file ! videoconvert ! agingtv name=my_agingtv ! videoconvert ! autovideosink
```
subprojects/gst-devtools/docs/plugins/validateflow.md
+3
View File
@@ -0,0 +1,3 @@
---
redirect: gst-validate-flow.md
...
subprojects/gst-devtools/docs/sitemap.txt
+16
View File
@@ -0,0 +1,16 @@
index.md
gst-validate.md
gst-validate-transcoding.md
gst-validate-media-check.md
gst-validate-launcher.md
gst-validate-scenarios.md
gst-validate-test-file.md
gst-validate-config.md
gst-validate-environment-variables.md
gst-validate-action-types.md
ges-validate-action-types.md
gst-validate-flow.md
gi-index
plugins/index.md
plugins/ssim.md
plugins/validateflow.md