ZeroDream/gstreamer
2
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Update

Browse Source
This commit is contained in:
Akkariin Meiko
2022-03-12 03:16:09 +08:00
Unverified
parent 12b76e0c7a
commit 27c4ec74a1
10075 changed files with 5122287 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
subprojects/gst-docs/markdown/tutorials/playback/audio-visualization.md
+259
View File
@@ -0,0 +1,259 @@
# Playback tutorial 6: Audio visualization
{{ ALERT_PY.md }}
{{ ALERT_JS.md }}
## Goal
GStreamer comes with a set of elements that turn audio into video. They
can be used for scientific visualization or to spice up your music
player, for example. This tutorial shows:
- How to enable audio visualization
- How to select the visualization element
## Introduction
Enabling audio visualization in `playbin` is actually very easy. Just
set the appropriate `playbin` flag and, when an audio-only stream is
found, it will instantiate the necessary elements to create and display
the visualization.
If you want to specify the actual element that you want to use to
generate the visualization, you instantiate it yourself and then tell
`playbin` about it through the `vis-plugin` property.
This tutorial searches the GStreamer registry for all the elements of
the Visualization class, tries to select `goom` (or another one if it is
not available) and passes it to `playbin`.
## A fancy music player
Copy this code into a text file named `playback-tutorial-6.c`.
**playback-tutorial-6.c**
``` c
#include <gst/gst.h>
/* playbin flags */
typedef enum {
GST_PLAY_FLAG_VIS = (1 << 3) /* Enable rendering of visualizations when there is no video stream. */
} GstPlayFlags;
/* Return TRUE if this is a Visualization element */
static gboolean filter_vis_features (GstPluginFeature *feature, gpointer data) {
GstElementFactory *factory;
if (!GST_IS_ELEMENT_FACTORY (feature))
return FALSE;
factory = GST_ELEMENT_FACTORY (feature);
if (!g_strrstr (gst_element_factory_get_klass (factory), "Visualization"))
return FALSE;
return TRUE;
}
int main(int argc, char *argv[]) {
GstElement *pipeline, *vis_plugin;
GstBus *bus;
GstMessage *msg;
GList *list, *walk;
GstElementFactory *selected_factory = NULL;
guint flags;
/* Initialize GStreamer */
gst_init (&argc, &argv);
/* Get a list of all visualization plugins */
list = gst_registry_feature_filter (gst_registry_get (), filter_vis_features, FALSE, NULL);
/* Print their names */
g_print("Available visualization plugins:\n");
for (walk = list; walk != NULL; walk = g_list_next (walk)) {
const gchar *name;
GstElementFactory *factory;
factory = GST_ELEMENT_FACTORY (walk->data);
name = gst_element_factory_get_longname (factory);
g_print(" %s\n", name);
if (selected_factory == NULL || g_str_has_prefix (name, "GOOM")) {
selected_factory = factory;
}
}
/* Don't use the factory if it's still empty */
/* e.g. no visualization plugins found */
if (!selected_factory) {
g_print ("No visualization plugins found!\n");
return -1;
}
/* We have now selected a factory for the visualization element */
g_print ("Selected '%s'\n", gst_element_factory_get_longname (selected_factory));
vis_plugin = gst_element_factory_create (selected_factory, NULL);
if (!vis_plugin)
return -1;
/* Build the pipeline */
pipeline = gst_parse_launch ("playbin uri=http://radio.hbr1.com:19800/ambient.ogg", NULL);
/* Set the visualization flag */
g_object_get (pipeline, "flags", &flags, NULL);
flags |= GST_PLAY_FLAG_VIS;
g_object_set (pipeline, "flags", flags, NULL);
/* set vis plugin for playbin */
g_object_set (pipeline, "vis-plugin", vis_plugin, NULL);
/* Start playing */
gst_element_set_state (pipeline, GST_STATE_PLAYING);
/* Wait until error or EOS */
bus = gst_element_get_bus (pipeline);
msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
/* Free resources */
if (msg != NULL)
gst_message_unref (msg);
gst_plugin_feature_list_free (list);
gst_object_unref (bus);
gst_element_set_state (pipeline, GST_STATE_NULL);
gst_object_unref (pipeline);
return 0;
}
```
> ![information] If you need help to compile this code, refer to the
> **Building the tutorials** section for your platform: [Mac] or
> [Windows] or use this specific command on Linux:
>
> `` gcc playback-tutorial-6.c -o playback-tutorial-6 `pkg-config --cflags --libs gstreamer-1.0` ``
>
> If you need help to run this code, refer to the **Running the
> tutorials** section for your platform: [Mac OS X], [Windows][1], for
> [iOS] or for [android].
>
> This tutorial plays music streamed from the [HBR1](http://www.hbr1.com/) Internet radio station. A window should open displaying somewhat psychedelic color patterns moving with the music. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed.
>
> Required libraries: `gstreamer-1.0`
## Walkthrough
First off, we indicate `playbin` that we want an audio visualization by
setting the `GST_PLAY_FLAG_VIS` flag. If the media already contains
video, this flag has no effect.
``` c
/* Set the visualization flag */
g_object_get (pipeline, "flags", &flags, NULL);
flags |= GST_PLAY_FLAG_VIS;
g_object_set (pipeline, "flags", flags, NULL);
```
If no visualization plugin is enforced by the user, `playbin` will use
`goom` (audio visualization will be disabled if `goom` is not
available). The rest of the tutorial shows how to find out the available
visualization elements and enforce one to `playbin`.
``` c
/* Get a list of all visualization plugins */
list = gst_registry_feature_filter (gst_registry_get (), filter_vis_features, FALSE, NULL);
```
`gst_registry_feature_filter()` examines all elements currently in the
GStreamer registry and selects those for which
the `filter_vis_features` function returns TRUE. This function selects
only the Visualization plugins:
``` c
/* Return TRUE if this is a Visualization element */
static gboolean filter_vis_features (GstPluginFeature *feature, gpointer data) {
GstElementFactory *factory;
if (!GST_IS_ELEMENT_FACTORY (feature))
return FALSE;
factory = GST_ELEMENT_FACTORY (feature);
if (!g_strrstr (gst_element_factory_get_klass (factory), "Visualization"))
return FALSE;
return TRUE;
}
```
A bit of theory regarding the organization of GStreamer elements is in
place: Each of the files that GStreamer loads at runtime is known as a
Plugin (`GstPlugin`). A Plugin can contain many Features
(`GstPluginFeature`). There are different kinds of Features, among them,
the Element Factories (`GstElementFactory`) that we have been using to
build Elements (`GstElement`).
This function simply disregards all Features which are not Factories,
and then all Factories whose class (obtained with
`gst_element_factory_get_klass()`) does not include “Visualization”. As
stated in the documentation for `GstElementFactory`, a Factory’s class
is a “string describing the type of element, as an unordered list
separated with slashes (/)”. Examples of classes are “Source/Network”,
“Codec/Decoder/Video”, “Codec/Encoder/Audio” or “Visualization”.
``` c
/* Print their names */
g_print("Available visualization plugins:\n");
for (walk = list; walk != NULL; walk = g_list_next (walk)) {
const gchar *name;
GstElementFactory *factory;
factory = GST_ELEMENT_FACTORY (walk->data);
name = gst_element_factory_get_longname (factory);
g_print(" %s\n", name);
if (selected_factory == NULL || g_str_has_prefix (name, "GOOM")) {
selected_factory = factory;
}
}
```
Once we have the list of Visualization plugins, we print their names
(`gst_element_factory_get_longname()`) and choose one (in this case,
GOOM).
``` c
/* We have now selected a factory for the visualization element */
g_print ("Selected '%s'\n", gst_element_factory_get_longname (selected_factory));
vis_plugin = gst_element_factory_create (selected_factory, NULL);
if (!vis_plugin)
return -1;
```
The selected factory is used to instantiate an actual `GstElement` which
is then passed to `playbin` through the `vis-plugin` property:
``` c
/* set vis plugin for playbin */
g_object_set (pipeline, "vis-plugin", vis_plugin, NULL);
```
And we are done.
## Conclusion
This tutorial has shown:
- How to enable Audio Visualization in `playbin` with the
`GST_PLAY_FLAG_VIS` flag
- How to enforce one particular visualization element with the
`vis-plugin` `playbin` property
It has been a pleasure having you here, and see you soon\!
[information]: images/icons/emoticons/information.svg
[Mac]: installing/on-mac-osx.md
[Windows]: installing/on-windows.md
[Mac OS X]: installing/on-mac-osx.md#building-the-tutorials
[1]: installing/on-windows.md#running-the-tutorials
[iOS]: installing/for-ios-development.md#building-the-tutorials
[android]: installing/for-android-development.md#building-the-tutorials
[warning]: images/icons/emoticons/warning.svg
subprojects/gst-docs/markdown/tutorials/playback/color-balance.md
+312
View File
@@ -0,0 +1,312 @@
# Playback tutorial 5: Color Balance
{{ ALERT_PY.md }}
{{ ALERT_JS.md }}
## Goal
Brightness, Contrast, Hue and Saturation are common video adjustments,
which are collectively known as Color Balance settings in GStreamer.
This tutorial shows:
- How to find out the available color balance channels
- How to change them
## Introduction
[](tutorials/basic/toolkit-integration.md) has
already explained the concept of GObject interfaces: applications use
them to find out if certain functionality is available, regardless of
the actual element which implements it.
`playbin` implements the Color Balance interface (`GstColorBalance`),
which allows access to the color balance settings. If any of the
elements in the `playbin` pipeline support this interface,
`playbin` simply forwards it to the application, otherwise, a
colorbalance element is inserted in the pipeline.
This interface allows querying for the available color balance channels
(`GstColorBalanceChannel`), along with their name and valid range of
values, and then modify the current value of any of them.
## Color balance example
Copy this code into a text file named `playback-tutorial-5.c`.
**playback-tutorial-5.c**
``` c
#include <string.h>
#include <stdio.h>
#include <gst/gst.h>
#include <gst/video/colorbalance.h>
typedef struct _CustomData {
GstElement *pipeline;
GMainLoop *loop;
} CustomData;
/* Process a color balance command */
static void update_color_channel (const gchar *channel_name, gboolean increase, GstColorBalance *cb) {
gdouble step;
gint value;
GstColorBalanceChannel *channel = NULL;
const GList *channels, *l;
/* Retrieve the list of channels and locate the requested one */
channels = gst_color_balance_list_channels (cb);
for (l = channels; l != NULL; l = l->next) {
GstColorBalanceChannel *tmp = (GstColorBalanceChannel *)l->data;
if (g_strrstr (tmp->label, channel_name)) {
channel = tmp;
break;
}
}
if (!channel)
return;
/* Change the channel's value */
step = 0.1 * (channel->max_value - channel->min_value);
value = gst_color_balance_get_value (cb, channel);
if (increase) {
value = (gint)(value + step);
if (value > channel->max_value)
value = channel->max_value;
} else {
value = (gint)(value - step);
if (value < channel->min_value)
value = channel->min_value;
}
gst_color_balance_set_value (cb, channel, value);
}
/* Output the current values of all Color Balance channels */
static void print_current_values (GstElement *pipeline) {
const GList *channels, *l;
/* Output Color Balance values */
channels = gst_color_balance_list_channels (GST_COLOR_BALANCE (pipeline));
for (l = channels; l != NULL; l = l->next) {
GstColorBalanceChannel *channel = (GstColorBalanceChannel *)l->data;
gint value = gst_color_balance_get_value (GST_COLOR_BALANCE (pipeline), channel);
g_print ("%s: %3d%% ", channel->label,
100 * (value - channel->min_value) / (channel->max_value - channel->min_value));
}
g_print ("\n");
}
/* Process keyboard input */
static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data) {
gchar *str = NULL;
if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) != G_IO_STATUS_NORMAL) {
return TRUE;
}
switch (g_ascii_tolower (str[0])) {
case 'c':
update_color_channel ("CONTRAST", g_ascii_isupper (str[0]), GST_COLOR_BALANCE (data->pipeline));
break;
case 'b':
update_color_channel ("BRIGHTNESS", g_ascii_isupper (str[0]), GST_COLOR_BALANCE (data->pipeline));
break;
case 'h':
update_color_channel ("HUE", g_ascii_isupper (str[0]), GST_COLOR_BALANCE (data->pipeline));
break;
case 's':
update_color_channel ("SATURATION", g_ascii_isupper (str[0]), GST_COLOR_BALANCE (data->pipeline));
break;
case 'q':
g_main_loop_quit (data->loop);
break;
default:
break;
}
g_free (str);
print_current_values (data->pipeline);
return TRUE;
}
int main(int argc, char *argv[]) {
CustomData data;
GstStateChangeReturn ret;
GIOChannel *io_stdin;
/* Initialize GStreamer */
gst_init (&argc, &argv);
/* Initialize our data structure */
memset (&data, 0, sizeof (data));
/* Print usage map */
g_print (
"USAGE: Choose one of the following options, then press enter:\n"
" 'C' to increase contrast, 'c' to decrease contrast\n"
" 'B' to increase brightness, 'b' to decrease brightness\n"
" 'H' to increase hue, 'h' to decrease hue\n"
" 'S' to increase saturation, 's' to decrease saturation\n"
" 'Q' to quit\n");
/* Build the pipeline */
data.pipeline = gst_parse_launch ("playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
/* Add a keyboard watch so we get notified of keystrokes */
#ifdef G_OS_WIN32
io_stdin = g_io_channel_win32_new_fd (fileno (stdin));
#else
io_stdin = g_io_channel_unix_new (fileno (stdin));
#endif
g_io_add_watch (io_stdin, G_IO_IN, (GIOFunc)handle_keyboard, &data);
/* Start playing */
ret = gst_element_set_state (data.pipeline, GST_STATE_PLAYING);
if (ret == GST_STATE_CHANGE_FAILURE) {
g_printerr ("Unable to set the pipeline to the playing state.\n");
gst_object_unref (data.pipeline);
return -1;
}
print_current_values (data.pipeline);
/* Create a GLib Main Loop and set it to run */
data.loop = g_main_loop_new (NULL, FALSE);
g_main_loop_run (data.loop);
/* Free resources */
g_main_loop_unref (data.loop);
g_io_channel_unref (io_stdin);
gst_element_set_state (data.pipeline, GST_STATE_NULL);
gst_object_unref (data.pipeline);
return 0;
}
```
> ![information] If you need help to compile this code, refer to the
> **Building the tutorials** section for your platform: [Mac] or
> [Windows] or use this specific command on Linux:
>
> `` gcc playback-tutorial-5.c -o playback-tutorial-5 `pkg-config --cflags --libs gstreamer-1.0 gstreamer-video-1.0` ``
>
> If you need help to run this code, refer to the **Running the
> tutorials** section for your platform: [Mac OS X], [Windows][1], for
> [iOS] or for [android].
>
> This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed.
>
>The console should print all commands (Each command is a single upper-case or lower-case letter) and list all available Color Balance channels, typically, CONTRAST, BRIGHTNESS, HUE and SATURATION. Type each command (letter) followed by the Enter key.
>
> Required libraries: `gstreamer-1.0 gstreamer-video-1.0`
## Walkthrough
The `main()` function is fairly simple. A `playbin` pipeline is
instantiated and set to run, and a keyboard watch is installed so
keystrokes can be monitored.
``` c
/* Output the current values of all Color Balance channels */
static void print_current_values (GstElement *pipeline) {
const GList *channels, *l;
/* Output Color Balance values */
channels = gst_color_balance_list_channels (GST_COLOR_BALANCE (pipeline));
for (l = channels; l != NULL; l = l->next) {
GstColorBalanceChannel *channel = (GstColorBalanceChannel *)l->data;
gint value = gst_color_balance_get_value (GST_COLOR_BALANCE (pipeline), channel);
g_print ("%s: %3d%% ", channel->label,
100 * (value - channel->min_value) / (channel->max_value - channel->min_value));
}
g_print ("\n");
}
```
This method prints the current value for all channels, and exemplifies
how to retrieve the list of channels. This is accomplished through the
`gst_color_balance_list_channels()` method. It returns a `GList` which
needs to be traversed.
Each element in the list is a `GstColorBalanceChannel` structure,
informing of the channel’s name, minimum value and maximum value.
`gst_color_balance_get_value()` can then be called on each channel to
retrieve the current value.
In this example, the minimum and maximum values are used to output the
current value as a percentage.
``` c
/* Process a color balance command */
static void update_color_channel (const gchar *channel_name, gboolean increase, GstColorBalance *cb) {
gdouble step;
gint value;
GstColorBalanceChannel *channel = NULL;
const GList *channels, *l;
/* Retrieve the list of channels and locate the requested one */
channels = gst_color_balance_list_channels (cb);
for (l = channels; l != NULL; l = l->next) {
GstColorBalanceChannel *tmp = (GstColorBalanceChannel *)l->data;
if (g_strrstr (tmp->label, channel_name)) {
channel = tmp;
break;
}
}
if (!channel)
return;
```
This method locates the specified channel by name and increases or
decreases it as requested. Again, the list of channels is retrieved and
parsed looking for the channel with the specified name. Obviously, this
list could be parsed only once and the pointers to the channels be
stored and indexed by something more efficient than a string.
``` c
/* Change the channel's value */
step = 0.1 * (channel->max_value - channel->min_value);
value = gst_color_balance_get_value (cb, channel);
if (increase) {
value = (gint)(value + step);
if (value > channel->max_value)
value = channel->max_value;
} else {
value = (gint)(value - step);
if (value < channel->min_value)
value = channel->min_value;
}
gst_color_balance_set_value (cb, channel, value);
}
```
The current value for the channel is then retrieved, changed (the
increment is proportional to its dynamic range), clamped (to avoid
out-of-range values) and set using `gst_color_balance_set_value()`.
And there is not much more to it. Run the program and observe the effect
of changing each of the channels in real time.
## Conclusion
This tutorial has shown how to use the color balance interface.
Particularly, it has shown:
- How to retrieve the list of color available balance channels
with `gst_color_balance_list_channels()`
- How to manipulate the current value of each channel using
`gst_color_balance_get_value()` and `gst_color_balance_set_value()`
It has been a pleasure having you here, and see you soon\!
[information]: images/icons/emoticons/information.svg
[Mac]: installing/on-mac-osx.md
[Windows]: installing/on-windows.md
[Mac OS X]: installing/on-mac-osx.md#building-the-tutorials
[1]: installing/on-windows.md#running-the-tutorials
[iOS]: installing/for-ios-development.md#building-the-tutorials
[android]: installing/for-android-development.md#building-the-tutorials
[warning]: images/icons/emoticons/warning.svg
subprojects/gst-docs/markdown/tutorials/playback/custom-playbin-sinks.md
+226
View File
@@ -0,0 +1,226 @@
# Playback tutorial 7: Custom playbin sinks
{{ ALERT_PY.md }}
{{ ALERT_JS.md }}
## Goal
`playbin` can be further customized by manually selecting its audio and
video sinks. This allows applications to rely on `playbin` to retrieve
and decode the media and then manage the final render/display
themselves. This tutorial shows:
- How to replace the sinks selected by `playbin`.
- How to use a complex pipeline as a sink.
## Introduction
Two properties of `playbin` allow selecting the desired audio and video
sinks: `audio-sink` and `video-sink` (respectively). The application
only needs to instantiate the appropriate `GstElement` and pass it to
`playbin` through these properties.
This method, though, only allows using a single Element as sink. If a
more complex pipeline is required, for example, an equalizer plus an
audio sink, it needs to be wrapped in a Bin, so it looks to
`playbin` as if it was a single Element.
A Bin (`GstBin`) is a container that encapsulates partial pipelines so
they can be managed as single elements. As an example, the
`GstPipeline` we have been using in all tutorials is a type of
`GstBin`, which does not interact with external Elements. Elements
inside a Bin connect to external elements through Ghost Pads
(`GstGhostPad`), this is, Pads on the surface of the Bin which simply
forward data from an external Pad to a given Pad on an internal Element.
![](images/bin-element-ghost.png)
**Figure 1:** A Bin with two Elements and one Ghost Pad.
`GstBin`s are also a type of `GstElement`, so they can be used wherever
an Element is required, in particular, as sinks for `playbin` (and they
are then known as **sink-bins**).
## An equalized player
Copy this code into a text file named `playback-tutorial-7.c`.
**playback-tutorial7.c**
``` c
#include <gst/gst.h>
int main(int argc, char *argv[]) {
GstElement *pipeline, *bin, *equalizer, *convert, *sink;
GstPad *pad, *ghost_pad;
GstBus *bus;
GstMessage *msg;
/* Initialize GStreamer */
gst_init (&argc, &argv);
/* Build the pipeline */
pipeline = gst_parse_launch ("playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
/* Create the elements inside the sink bin */
equalizer = gst_element_factory_make ("equalizer-3bands", "equalizer");
convert = gst_element_factory_make ("audioconvert", "convert");
sink = gst_element_factory_make ("autoaudiosink", "audio_sink");
if (!equalizer || !convert || !sink) {
g_printerr ("Not all elements could be created.\n");
return -1;
}
/* Create the sink bin, add the elements and link them */
bin = gst_bin_new ("audio_sink_bin");
gst_bin_add_many (GST_BIN (bin), equalizer, convert, sink, NULL);
gst_element_link_many (equalizer, convert, sink, NULL);
pad = gst_element_get_static_pad (equalizer, "sink");
ghost_pad = gst_ghost_pad_new ("sink", pad);
gst_pad_set_active (ghost_pad, TRUE);
gst_element_add_pad (bin, ghost_pad);
gst_object_unref (pad);
/* Configure the equalizer */
g_object_set (G_OBJECT (equalizer), "band1", (gdouble)-24.0, NULL);
g_object_set (G_OBJECT (equalizer), "band2", (gdouble)-24.0, NULL);
/* Set playbin's audio sink to be our sink bin */
g_object_set (GST_OBJECT (pipeline), "audio-sink", bin, NULL);
/* Start playing */
gst_element_set_state (pipeline, GST_STATE_PLAYING);
/* Wait until error or EOS */
bus = gst_element_get_bus (pipeline);
msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
/* Free resources */
if (msg != NULL)
gst_message_unref (msg);
gst_object_unref (bus);
gst_element_set_state (pipeline, GST_STATE_NULL);
gst_object_unref (pipeline);
return 0;
}
```
> ![information] If you need help to compile this code, refer to the
> **Building the tutorials** section for your platform: [Mac] or
> [Windows] or use this specific command on Linux:
>
> `` gcc playback-tutorial-7.c -o playback-tutorial-7 `pkg-config --cflags --libs gstreamer-1.0` ``
>
> If you need help to run this code, refer to the **Running the
> tutorials** section for your platform: [Mac OS X], [Windows][1], for
> [iOS] or for [android].
>
> This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed. The higher frequency bands have been attenuated, so the movie sound should have a more powerful bass component.<
>
> Required libraries: `gstreamer-1.0`
## Walkthrough
``` c
/* Create the elements inside the sink bin */
equalizer = gst_element_factory_make ("equalizer-3bands", "equalizer");
convert = gst_element_factory_make ("audioconvert", "convert");
sink = gst_element_factory_make ("autoaudiosink", "audio_sink");
if (!equalizer || !convert || !sink) {
g_printerr ("Not all elements could be created.\n");
return -1;
}
```
All the Elements that compose our sink-bin are instantiated. We use an
`equalizer-3bands` and an `autoaudiosink`, with an `audioconvert` in
between, because we are not sure of the capabilities of the audio sink
(since they are hardware-dependant).
``` c
/* Create the sink bin, add the elements and link them */
bin = gst_bin_new ("audio_sink_bin");
gst_bin_add_many (GST_BIN (bin), equalizer, convert, sink, NULL);
gst_element_link_many (equalizer, convert, sink, NULL);
```
This adds the new Elements to the Bin and links them just as we would do
if this was a pipeline.
``` c
pad = gst_element_get_static_pad (equalizer, "sink");
ghost_pad = gst_ghost_pad_new ("sink", pad);
gst_pad_set_active (ghost_pad, TRUE);
gst_element_add_pad (bin, ghost_pad);
gst_object_unref (pad);
```
Now we need to create a Ghost Pad so this partial pipeline inside the
Bin can be connected to the outside. This Ghost Pad will be connected to
a Pad in one of the internal Elements (the sink pad of the equalizer),
so we retrieve this Pad with `gst_element_get_static_pad()`. Remember
from [](tutorials/basic/multithreading-and-pad-availability.md) that
if this was a Request Pad instead of an Always Pad, we would need to use
`gst_element_request_pad()`.
The Ghost Pad is created with `gst_ghost_pad_new()` (pointing to the
inner Pad we just acquired), and activated with `gst_pad_set_active()`.
It is then added to the Bin with `gst_element_add_pad()`, transferring
ownership of the Ghost Pad to the bin, so we do not have to worry about
releasing it.
Finally, the sink Pad we obtained from the equalizer needs to be release
with `gst_object_unref()`.
At this point, we have a functional sink-bin, which we can use as the
audio sink in `playbin`. We just need to instruct `playbin` to use it:
``` c
/* Set playbin's audio sink to be our sink bin */
g_object_set (GST_OBJECT (pipeline), "audio-sink", bin, NULL);
```
It is as simple as setting the `audio-sink` property on `playbin` to
the newly created sink.
``` c
/* Configure the equalizer */
g_object_set (G_OBJECT (equalizer), "band1", (gdouble)-24.0, NULL);
g_object_set (G_OBJECT (equalizer), "band2", (gdouble)-24.0, NULL);
```
The only bit remaining is to configure the equalizer. For this example,
the two higher frequency bands are set to the maximum attenuation so the
bass is boosted. Play a bit with the values to feel the difference (Look
at the documentation for the `equalizer-3bands` element for the allowed
range of values).
## Exercise
Build a video bin instead of an audio bin, using one of the many
interesting video filters GStreamer offers, like `solarize`,
`vertigotv` or any of the Elements in the `effectv` plugin. Remember to
use the color space conversion element `videoconvert` if your
pipeline fails to link due to incompatible caps.
## Conclusion
This tutorial has shown:
- How to set your own sinks to `playbin` using the audio-sink and
video-sink properties.
- How to wrap a piece of pipeline into a `GstBin` so it can be used as
a **sink-bin** by `playbin`.
It has been a pleasure having you here, and see you soon\!
[information]: images/icons/emoticons/information.svg
[Mac]: installing/on-mac-osx.md
[Windows]: installing/on-windows.md
[Mac OS X]: installing/on-mac-osx.md#building-the-tutorials
[1]: installing/on-windows.md#running-the-tutorials
[iOS]: installing/for-ios-development.md#building-the-tutorials
[android]: installing/for-android-development.md#building-the-tutorials
[warning]: images/icons/emoticons/warning.svg
subprojects/gst-docs/markdown/tutorials/playback/digital-audio-pass-through.md
+101
View File
@@ -0,0 +1,101 @@
# Playback tutorial 9: Digital audio pass-through
## Goal
This tutorial shows how GStreamer handles digital audio pass-through.
## Introduction
Besides the common analog format, high-end audio systems usually also
accept data in digital form, either compressed or uncompressed. This is
convenient because the audio signal then travels from the computer to
the speakers in a form that is more resilient to interference and noise,
resulting higher quality.
The connection is typically made through an
[S/PDIF](http://en.wikipedia.org/wiki/SPDIF) cable which can either be
optical (with [TOSLINK](http://en.wikipedia.org/wiki/TOSLINK)
connectors) or coaxial (with [RCA](http://en.wikipedia.org/wiki/RCA)
connectors). S/PDIF is also known as IEC 60958 type II (IEC 958 before
1998).
In this scenario, GStreamer does not need to perform audio decoding; it
can simply output the encoded data, acting in *pass-through* mode, and
let the external audio system perform the decoding.
## Inner workings of GStreamer audio sinks
First off, digital audio output must be enabled at the system level. The
method to achieve this depend on the operating system, but it generally
involves going to the audio control panel and activating a checkbox
reading “Digital Audio Output” or similar.
The main GStreamer audio sinks for each platform, Pulse Audio
(`pulsesink`) for Linux, `osxaudiosink` for OS X and Direct Sound
(`directsoundsink`) for Windows, detect when digital audio output is
available and change their input caps accordingly to accept encoded
data. For example, these elements typically accept `audio/x-raw` data:
when digital audio output is enabled in the system, they may also
accept `audio/mpeg`, `audio/x-ac3`, `audio/x-eac3` or `audio/x-dts`.
Then, when `playbin` builds the decoding pipeline, it realizes that the
audio sink can be directly connected to the encoded data (typically
coming out of a demuxer), so there is no need for a decoder. This
process is automatic and does not need any action from the application.
On Linux, there exist other audio sinks, like Alsa (`alsasink`) which
work differently (a “digital device” needs to be manually selected
through the `device` property of the sink). Pulse Audio, though, is the
commonly preferred audio sink on Linux.
## Precautions with digital formats
When Digital Audio Output is enabled at the system level, the GStreamer
audio sinks automatically expose all possible digital audio caps,
regardless of whether the actual audio decoder at the end of the S/PDIF
cable is able to decode all those formats. This is so because there is
no mechanism to query an external audio decoder which formats are
supported, and, in fact, the cable can even be disconnected during this
process.
For example, after enabling Digital Audio Output in the system’s Control
Panel, `directsoundsink` will automatically expose `audio/x-ac3`,
`audio/x-eac3` and `audio/x-dts` caps in addition to `audio/x-raw`.
However, one particular external decoder might only understand raw
integer streams and would try to play the compressed data as such (a
painful experience for your ears, rest assured).
Solving this issue requires user intervention, since only the user knows
the formats supported by the external decoder.
On some systems, the simplest solution is to inform the operating system
of the formats that the external audio decoder can accept. In this way,
the GStreamer audio sinks will only offer these formats. The acceptable
audio formats are commonly selected from the operating system’s audio
configuration panel, from the same place where Digital Audio Output is
enabled, but, unfortunately, this option is not available in all audio
drivers.
Another solution involves, using a custom sinkbin (see
[](tutorials/playback/custom-playbin-sinks.md)) which includes a
`capsfilter` element (see [](tutorials/basic/handy-elements.md))
and an audio sink. The caps that the external decoder supports are
then set in the capsfiler so the wrong format is not output. This
allows the application to enforce the appropriate format instead of
relying on the user to have the system correctly configured. Still
requires user intervention, but can be used regardless of the options
the audio driver offers.
Please do not use `autoaudiosink` as the audio sink, as it currently
only supports raw audio, and will ignore any compressed format.
## Conclusion
This tutorial has shown a bit of how GStreamer deals with digital audio.
In particular, it has shown that:
- Applications using `playbin` do not need to do anything special to
enable digital audio output: it is managed from the audio control
panel of the operating system.
It has been a pleasure having you here, and see you soon!
subprojects/gst-docs/markdown/tutorials/playback/hardware-accelerated-video-decoding.md
+183
View File
@@ -0,0 +1,183 @@
# Playback tutorial 8: Hardware-accelerated video decoding
{{ ALERT_PY.md }}
{{ ALERT_JS.md }}
### Goal
Hardware-accelerated video decoding has rapidly become a necessity, as
low-power devices grow more common. This tutorial (more of a lecture,
actually) gives some background on hardware acceleration and explains
how does GStreamer benefit from it.
Sneak peek: if properly setup, you do not need to do anything special to
activate hardware acceleration; GStreamer automatically takes advantage
of it.
### Introduction
Video decoding can be an extremely CPU-intensive task, especially for
higher resolutions like 1080p HDTV. Fortunately, modern graphics cards,
equipped with programmable GPUs, are able to take care of this job,
allowing the CPU to concentrate on other duties. Having dedicated
hardware becomes essential for low-power CPUs which are simply incapable
of decoding such media fast enough.
In the current state of things (June 2016) each GPU manufacturer offers
a different method to access their hardware (a different API), and a
strong industry standard has not emerged yet.
As of June 2016, there exist at least 8 different video decoding
acceleration APIs:
- [VAAPI](http://en.wikipedia.org/wiki/Video_Acceleration_API) (*Video
Acceleration API*): Initially designed by
[Intel](http://en.wikipedia.org/wiki/Intel) in 2007, targeted at the X
Window System on Unix-based operating systems, now open-source. It now also
supports Wayland through dmabuf. It is
currently not limited to Intel GPUs as other manufacturers are free to
use this API, for example, [Imagination
Technologies](http://en.wikipedia.org/wiki/Imagination_Technologies) or
[S3 Graphics](http://en.wikipedia.org/wiki/S3_Graphics). Accessible to
GStreamer through the [gstreamer-vaapi](https://gitlab.freedesktop.org/gstreamer/gstreamer/-/tree/main/subprojects/gstreamer-vaapi/) package.
- [OpenMAX](http://en.wikipedia.org/wiki/OpenMAX) (*Open Media
Acceleration*): Managed by the non-profit technology consortium [Khronos
Group](http://en.wikipedia.org/wiki/Khronos_Group "Khronos Group"),
it is a "royalty-free, cross-platform set of C-language programming
interfaces that provides abstractions for routines especially useful for
audio, video, and still images". Accessible to GStreamer through
the [gst-omx](http://git.freedesktop.org/gstreamer/gst-omx) plugin.
- [OVD](http://developer.amd.com/sdks/AMDAPPSDK/assets/OpenVideo_Decode_API.PDF)
(*Open Video Decode*): Another API from [AMD
Graphics](http://en.wikipedia.org/wiki/AMD_Graphics), designed to be a
platform agnostic method for softrware developers to leverage the
[Universal Video
Decode](http://en.wikipedia.org/wiki/Unified_Video_Decoder) (UVD)
hardware inside AMD Radeon graphics cards. Currently unavailable to
GStreamer .
- [DCE](http://en.wikipedia.org/wiki/Distributed_Codec_Engine)
(*Distributed Codec Engine*): An open source software library ("libdce")
and API specification by [Texas
Instruments](http://en.wikipedia.org/wiki/Texas_Instruments), targeted
at Linux systems and ARM platforms. Accessible to GStreamer through
the [gstreamer-ducati](https://github.com/robclark/gst-ducati) plugin.
- [Android
MediaCodec](https://developer.android.com/reference/android/media/MediaCodec.html): This is Android's API to access the device's
hardware decoder and encoder if available. This is accessible through the
`androidmedia` plugin in gst-plugins-bad. This includes both encoding and
decoding.
- Apple VideoTool Box Framework: Apple's API to access hardware is available
through the `applemedia` plugin which includes both encoding through
the `vtenc` element and decoding through the `vtdec` element.
- Video4Linux: Recent Linux kernels have a kernel API to expose
hardware codecs in a standard way, this is now supported by the
`v4l2` plugin in `gst-plugins-good`. This can support both decoding
and encoding depending on the platform.
### Inner workings of hardware-accelerated video decoding plugins
These APIs generally offer a number of functionalities, like video
decoding, post-processing, or presentation of the decoded
frames. Correspondingly, plugins generally offer a different GStreamer
element for each of these functions, so pipelines can be built to
accommodate any need.
For example, the `gstreamer-vaapi` plugin offers the `vaapidecode`,
`vaapipostproc` and `vaapisink` elements that allow
hardware-accelerated decoding through VAAPI, upload of raw video frames
to GPU memory, download of GPU frames to system memory and presentation
of GPU frames, respectively.
It is important to distinguish between conventional GStreamer frames,
which reside in system memory, and frames generated by
hardware-accelerated APIs. The latter reside in GPU memory and cannot
be touched by GStreamer. They can usually be downloaded to system
memory and treated as conventional GStreamer frames when they are
mapped, but it is far more efficient to leave them in the GPU and
display them from there.
GStreamer needs to keep track of where these “hardware buffers” are
though, so conventional buffers still travel from element to
element. They look like regular buffers, but mapping their content is
much slower as it has to be retrieved from the special memory used by
hardware accelerated elements. This special memory types are
negotiated using the allocation query mechanism.
This all means that, if a particular hardware acceleration API is
present in the system, and the corresponding GStreamer plugin is also
available, auto-plugging elements like `playbin` are free to use
hardware acceleration to build their pipelines; the application does not
need to do anything special to enable it. Almost:
When `playbin` has to choose among different equally valid elements,
like conventional software decoding (through `vp8dec`, for example) or
hardware accelerated decoding (through `vaapidecode`, for example), it
uses their *rank* to decide. The rank is a property of each element that
indicates its priority; `playbin` will simply select the element that
is able to build a complete pipeline and has the highest rank.
So, whether `playbin` will use hardware acceleration or not will depend
on the relative ranks of all elements capable of dealing with that media
type. Therefore, the easiest way to make sure hardware acceleration is
enabled or disabled is by changing the rank of the associated element,
as shown in this code:
``` c
static void enable_factory (const gchar *name, gboolean enable) {
GstRegistry *registry = NULL;
GstElementFactory *factory = NULL;
registry = gst_registry_get_default ();
if (!registry) return;
factory = gst_element_factory_find (name);
if (!factory) return;
if (enable) {
gst_plugin_feature_set_rank (GST_PLUGIN_FEATURE (factory), GST_RANK_PRIMARY + 1);
}
else {
gst_plugin_feature_set_rank (GST_PLUGIN_FEATURE (factory), GST_RANK_NONE);
}
gst_registry_add_feature (registry, GST_PLUGIN_FEATURE (factory));
return;
}
```
The first parameter passed to this method is the name of the element to
modify, for example, `vaapidecode` or `fluvadec`.
The key method is `gst_plugin_feature_set_rank()`, which will set the
rank of the requested element factory to the desired level. For
convenience, ranks are divided in NONE, MARGINAL, SECONDARY and PRIMARY,
but any number will do. When enabling an element, we set it to
PRIMARY+1, so it has a higher rank than the rest of elements which
commonly have PRIMARY rank. Setting an element’s rank to NONE will make
the auto-plugging mechanism to never select it.
> ![warning] The GStreamer developers often rank hardware decoders lower than
> the software ones when they are defective. This should act as a warning.
## Conclusion
This tutorial has shown a bit how GStreamer internally manages hardware
accelerated video decoding. Particularly,
- Applications do not need to do anything special to enable hardware
acceleration if a suitable API and the corresponding GStreamer
plugin are available.
- Hardware acceleration can be enabled or disabled by changing the
rank of the decoding element with `gst_plugin_feature_set_rank()`.
It has been a pleasure having you here, and see you soon!
[warning]: images/icons/emoticons/warning.svg
subprojects/gst-docs/markdown/tutorials/playback/index.md
+4
View File
@@ -0,0 +1,4 @@
# Playback tutorials
These tutorials explain everything you need to know to produce a media
playback application using GStreamer.
subprojects/gst-docs/markdown/tutorials/playback/playbin-usage.md
+594
View File
@@ -0,0 +1,594 @@
# Playback tutorial 1: Playbin usage
{{ ALERT_PY.md }}
{{ ALERT_JS.md }}
## Goal
We have already worked with the `playbin` element, which is capable of
building a complete playback pipeline without much work on our side.
This tutorial shows how to further customize `playbin` in case its
default values do not suit our particular needs.
We will learn:
- How to find out how many streams a file contains, and how to switch
among them.
- How to gather information regarding each stream.
## Introduction
More often than not, multiple audio, video and subtitle streams can be
found embedded in a single file. The most common case are regular
movies, which contain one video and one audio stream (Stereo or 5.1
audio tracks are considered a single stream). It is also increasingly
common to find movies with one video and multiple audio streams, to
account for different languages. In this case, the user selects one
audio stream, and the application will only play that one, ignoring the
others.
To be able to select the appropriate stream, the user needs to know
certain information about them, for example, their language. This
information is embedded in the streams in the form of “metadata”
(annexed data), and this tutorial shows how to retrieve it.
Subtitles can also be embedded in a file, along with audio and video,
but they are dealt with in more detail in [Playback tutorial 2: Subtitle
management]. Finally, multiple video streams can also be found in a
single file, for example, in DVD with multiple angles of the same scene,
but they are somewhat rare.
> ![information] Embedding multiple streams inside a single file is
> called “multiplexing” or “muxing”, and such file is then known as a
> “container”. Common container formats are Matroska (.mkv), Quicktime
> (.qt, .mov, .mp4), Ogg (.ogg) or Webm (.webm).
>
> Retrieving the individual streams from within the container is called
> “demultiplexing” or “demuxing”.
The following code recovers the amount of streams in the file, their
associated metadata, and allows switching the audio stream while the
media is playing.
## The multilingual player
Copy this code into a text file named `playback-tutorial-1.c` (or find
it in the GStreamer installation).
**playback-tutorial-1.c**
``` c
#include <gst/gst.h>
/* Structure to contain all our information, so we can pass it around */
typedef struct _CustomData {
GstElement *playbin; /* Our one and only element */
gint n_video; /* Number of embedded video streams */
gint n_audio; /* Number of embedded audio streams */
gint n_text; /* Number of embedded subtitle streams */
gint current_video; /* Currently playing video stream */
gint current_audio; /* Currently playing audio stream */
gint current_text; /* Currently playing subtitle stream */
GMainLoop *main_loop; /* GLib's Main Loop */
} CustomData;
/* playbin flags */
typedef enum {
GST_PLAY_FLAG_VIDEO = (1 << 0), /* We want video output */
GST_PLAY_FLAG_AUDIO = (1 << 1), /* We want audio output */
GST_PLAY_FLAG_TEXT = (1 << 2) /* We want subtitle output */
} GstPlayFlags;
/* Forward definition for the message and keyboard processing functions */
static gboolean handle_message (GstBus *bus, GstMessage *msg, CustomData *data);
static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data);
int main(int argc, char *argv[]) {
CustomData data;
GstBus *bus;
GstStateChangeReturn ret;
gint flags;
GIOChannel *io_stdin;
/* Initialize GStreamer */
gst_init (&argc, &argv);
/* Create the elements */
data.playbin = gst_element_factory_make ("playbin", "playbin");
if (!data.playbin) {
g_printerr ("Not all elements could be created.\n");
return -1;
}
/* Set the URI to play */
g_object_set (data.playbin, "uri", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_cropped_multilingual.webm", NULL);
/* Set flags to show Audio and Video but ignore Subtitles */
g_object_get (data.playbin, "flags", &flags, NULL);
flags |= GST_PLAY_FLAG_VIDEO | GST_PLAY_FLAG_AUDIO;
flags &= ~GST_PLAY_FLAG_TEXT;
g_object_set (data.playbin, "flags", flags, NULL);
/* Set connection speed. This will affect some internal decisions of playbin */
g_object_set (data.playbin, "connection-speed", 56, NULL);
/* Add a bus watch, so we get notified when a message arrives */
bus = gst_element_get_bus (data.playbin);
gst_bus_add_watch (bus, (GstBusFunc)handle_message, &data);
/* Add a keyboard watch so we get notified of keystrokes */
#ifdef G_OS_WIN32
io_stdin = g_io_channel_win32_new_fd (fileno (stdin));
#else
io_stdin = g_io_channel_unix_new (fileno (stdin));
#endif
g_io_add_watch (io_stdin, G_IO_IN, (GIOFunc)handle_keyboard, &data);
/* Start playing */
ret = gst_element_set_state (data.playbin, GST_STATE_PLAYING);
if (ret == GST_STATE_CHANGE_FAILURE) {
g_printerr ("Unable to set the pipeline to the playing state.\n");
gst_object_unref (data.playbin);
return -1;
}
/* Create a GLib Main Loop and set it to run */
data.main_loop = g_main_loop_new (NULL, FALSE);
g_main_loop_run (data.main_loop);
/* Free resources */
g_main_loop_unref (data.main_loop);
g_io_channel_unref (io_stdin);
gst_object_unref (bus);
gst_element_set_state (data.playbin, GST_STATE_NULL);
gst_object_unref (data.playbin);
return 0;
}
/* Extract some metadata from the streams and print it on the screen */
static void analyze_streams (CustomData *data) {
gint i;
GstTagList *tags;
gchar *str;
guint rate;
/* Read some properties */
g_object_get (data->playbin, "n-video", &data->n_video, NULL);
g_object_get (data->playbin, "n-audio", &data->n_audio, NULL);
g_object_get (data->playbin, "n-text", &data->n_text, NULL);
g_print ("%d video stream(s), %d audio stream(s), %d text stream(s)\n",
data->n_video, data->n_audio, data->n_text);
g_print ("\n");
for (i = 0; i < data->n_video; i++) {
tags = NULL;
/* Retrieve the stream's video tags */
g_signal_emit_by_name (data->playbin, "get-video-tags", i, &tags);
if (tags) {
g_print ("video stream %d:\n", i);
gst_tag_list_get_string (tags, GST_TAG_VIDEO_CODEC, &str);
g_print (" codec: %s\n", str ? str : "unknown");
g_free (str);
gst_tag_list_free (tags);
}
}
g_print ("\n");
for (i = 0; i < data->n_audio; i++) {
tags = NULL;
/* Retrieve the stream's audio tags */
g_signal_emit_by_name (data->playbin, "get-audio-tags", i, &tags);
if (tags) {
g_print ("audio stream %d:\n", i);
if (gst_tag_list_get_string (tags, GST_TAG_AUDIO_CODEC, &str)) {
g_print (" codec: %s\n", str);
g_free (str);
}
if (gst_tag_list_get_string (tags, GST_TAG_LANGUAGE_CODE, &str)) {
g_print (" language: %s\n", str);
g_free (str);
}
if (gst_tag_list_get_uint (tags, GST_TAG_BITRATE, &rate)) {
g_print (" bitrate: %d\n", rate);
}
gst_tag_list_free (tags);
}
}
g_print ("\n");
for (i = 0; i < data->n_text; i++) {
tags = NULL;
/* Retrieve the stream's subtitle tags */
g_signal_emit_by_name (data->playbin, "get-text-tags", i, &tags);
if (tags) {
g_print ("subtitle stream %d:\n", i);
if (gst_tag_list_get_string (tags, GST_TAG_LANGUAGE_CODE, &str)) {
g_print (" language: %s\n", str);
g_free (str);
}
gst_tag_list_free (tags);
}
}
g_object_get (data->playbin, "current-video", &data->current_video, NULL);
g_object_get (data->playbin, "current-audio", &data->current_audio, NULL);
g_object_get (data->playbin, "current-text", &data->current_text, NULL);
g_print ("\n");
g_print ("Currently playing video stream %d, audio stream %d and text stream %d\n",
data->current_video, data->current_audio, data->current_text);
g_print ("Type any number and hit ENTER to select a different audio stream\n");
}
/* Process messages from GStreamer */
static gboolean handle_message (GstBus *bus, GstMessage *msg, CustomData *data) {
GError *err;
gchar *debug_info;
switch (GST_MESSAGE_TYPE (msg)) {
case GST_MESSAGE_ERROR:
gst_message_parse_error (msg, &err, &debug_info);
g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
g_clear_error (&err);
g_free (debug_info);
g_main_loop_quit (data->main_loop);
break;
case GST_MESSAGE_EOS:
g_print ("End-Of-Stream reached.\n");
g_main_loop_quit (data->main_loop);
break;
case GST_MESSAGE_STATE_CHANGED: {
GstState old_state, new_state, pending_state;
gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->playbin)) {
if (new_state == GST_STATE_PLAYING) {
/* Once we are in the playing state, analyze the streams */
analyze_streams (data);
}
}
} break;
}
/* We want to keep receiving messages */
return TRUE;
}
/* Process keyboard input */
static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data) {
gchar *str = NULL;
if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) == G_IO_STATUS_NORMAL) {
int index = g_ascii_strtoull (str, NULL, 0);
if (index < 0 || index >= data->n_audio) {
g_printerr ("Index out of bounds\n");
} else {
/* If the input was a valid audio stream index, set the current audio stream */
g_print ("Setting current audio stream to %d\n", index);
g_object_set (data->playbin, "current-audio", index, NULL);
}
}
g_free (str);
return TRUE;
}
```
> ![information] If you need help to compile this code, refer to the
> **Building the tutorials** section for your platform: [Mac] or
> [Windows] or use this specific command on Linux:
>
> `` gcc playback-tutorial-1.c -o playback-tutorial-1 `pkg-config --cflags --libs gstreamer-1.0` ``
>
> If you need help to run this code, refer to the **Running the
> tutorials** section for your platform: [Mac OS X], [Windows][1], for
> [iOS] or for [android].
>
> This tutorial opens a window and displays a movie, with accompanying
> audio. The media is fetched from the Internet, so the window might take
> a few seconds to appear, depending on your connection speed. The number
> of audio streams is shown in the terminal, and the user can switch from
> one to another by entering a number and pressing enter. A small delay is
> to be expected.
>
> Bear in mind that there is no latency management (buffering), so on slow
> connections, the movie might stop after a few seconds. See how [Tutorial
> 12: Live streaming] solves this issue.
>
> Required libraries: `gstreamer-1.0`
## Walkthrough
``` c
/* Structure to contain all our information, so we can pass it around */
typedef struct _CustomData {
GstElement *playbin; /* Our one and only element */
gint n_video; /* Number of embedded video streams */
gint n_audio; /* Number of embedded audio streams */
gint n_text; /* Number of embedded subtitle streams */
gint current_video; /* Currently playing video stream */
gint current_audio; /* Currently playing audio stream */
gint current_text; /* Currently playing subtitle stream */
GMainLoop *main_loop; /* GLib's Main Loop */
} CustomData;
```
We start, as usual, putting all our variables in a structure, so we can
pass it around to functions. For this tutorial, we need the amount of
streams of each type, and the currently playing one. Also, we are going
to use a different mechanism to wait for messages that allows
interactivity, so we need a GLib's main loop object.
``` c
/* playbin flags */
typedef enum {
GST_PLAY_FLAG_VIDEO = (1 << 0), /* We want video output */
GST_PLAY_FLAG_AUDIO = (1 << 1), /* We want audio output */
GST_PLAY_FLAG_TEXT = (1 << 2) /* We want subtitle output */
} GstPlayFlags;
```
Later we are going to set some of `playbin`'s flags. We would like to
have a handy enum that allows manipulating these flags easily, but since
`playbin` is a plug-in and not a part of the GStreamer core, this enum
is not available to us. The “trick” is simply to declare this enum in
our code, as it appears in the `playbin` documentation: `GstPlayFlags`.
GObject allows introspection, so the possible values for these flags can
be retrieved at runtime without using this trick, but in a far more
cumbersome way.
``` c
/* Forward definition for the message and keyboard processing functions */
static gboolean handle_message (GstBus *bus, GstMessage *msg, CustomData *data);
static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data);
```
Forward declarations for the two callbacks we will be using.
`handle_message` for the GStreamer messages, as we have already seen,
and `handle_keyboard` for key strokes, since this tutorial is
introducing a limited amount of interactivity.
We skip over the creation of the pipeline, the instantiation of
`playbin` and pointing it to our test media through the `uri`
property. `playbin` is in itself a pipeline, and in this case it is the
only element in the pipeline, so we skip completely the creation of the
pipeline, and use directly the `playbin` element.
We focus on some of the other properties of `playbin`, though:
``` c
/* Set flags to show Audio and Video but ignore Subtitles */
g_object_get (data.playbin, "flags", &flags, NULL);
flags |= GST_PLAY_FLAG_VIDEO | GST_PLAY_FLAG_AUDIO;
flags &= ~GST_PLAY_FLAG_TEXT;
g_object_set (data.playbin, "flags", flags, NULL);
```
`playbin`'s behavior can be changed through its `flags` property, which
can have any combination of `GstPlayFlags`. The most interesting values
are:
| Flag | Description |
|---------------------------|------------------------------------------------------------------------------------------------------------------------------------|
| GST_PLAY_FLAG_VIDEO | Enable video rendering. If this flag is not set, there will be no video output. |
| GST_PLAY_FLAG_AUDIO | Enable audio rendering. If this flag is not set, there will be no audio output. |
| GST_PLAY_FLAG_TEXT | Enable subtitle rendering. If this flag is not set, subtitles will not be shown in the video output. |
| GST_PLAY_FLAG_VIS | Enable rendering of visualisations when there is no video stream. Playback tutorial 6: Audio visualization goes into more details. |
| GST_PLAY_FLAG_DOWNLOAD | See Basic tutorial 12: Streaming and Playback tutorial 4: Progressive streaming. |
| GST_PLAY_FLAG_BUFFERING | See Basic tutorial 12: Streaming and Playback tutorial 4: Progressive streaming. |
| GST_PLAY_FLAG_DEINTERLACE | If the video content was interlaced, this flag instructs playbin to deinterlace it before displaying it. |
In our case, for demonstration purposes, we are enabling audio and video
and disabling subtitles, leaving the rest of flags to their default
values (this is why we read the current value of the flags with
`g_object_get()` before overwriting it with `g_object_set()`).
``` c
/* Set connection speed. This will affect some internal decisions of playbin */
g_object_set (data.playbin, "connection-speed", 56, NULL);
```
This property is not really useful in this example.
`connection-speed` informs `playbin` of the maximum speed of our network
connection, so, in case multiple versions of the requested media are
available in the server, `playbin` chooses the most appropriate. This is
mostly used in combination with streaming protocols like `hls` or
`rtsp`.
We have set all these properties one by one, but we could have all of
them with a single call to `g_object_set()`:
``` c
g_object_set (data.playbin, "uri", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_cropped_multilingual.webm", "flags", flags, "connection-speed", 56, NULL);
```
This is why `g_object_set()` requires a NULL as the last parameter.
``` c
/* Add a keyboard watch so we get notified of keystrokes */
#ifdef _WIN32
io_stdin = g_io_channel_win32_new_fd (fileno (stdin));
#else
io_stdin = g_io_channel_unix_new (fileno (stdin));
#endif
g_io_add_watch (io_stdin, G_IO_IN, (GIOFunc)handle_keyboard, &data);
```
These lines connect a callback function to the standard input (the
keyboard). The mechanism shown here is specific to GLib, and not really
related to GStreamer, so there is no point in going into much depth.
Applications normally have their own way of handling user input, and
GStreamer has little to do with it besides the Navigation interface
discussed briefly in [Tutorial 17: DVD playback].
``` c
/* Create a GLib Main Loop and set it to run */
data.main_loop = g_main_loop_new (NULL, FALSE);
g_main_loop_run (data.main_loop);
```
To allow interactivity, we will no longer poll the GStreamer bus
manually. Instead, we create a `GMainLoop`(GLib main loop) and set it
running with `g_main_loop_run()`. This function blocks and will not
return until `g_main_loop_quit()` is issued. In the meantime, it will
call the callbacks we have registered at the appropriate
times: `handle_message` when a message appears on the bus, and
`handle_keyboard` when the user presses any key.
There is nothing new in handle\_message, except that when the pipeline
moves to the PLAYING state, it will call the `analyze_streams` function:
``` c
/* Extract some metadata from the streams and print it on the screen */
static void analyze_streams (CustomData *data) {
gint i;
GstTagList *tags;
gchar *str;
guint rate;
/* Read some properties */
g_object_get (data->playbin, "n-video", &data->n_video, NULL);
g_object_get (data->playbin, "n-audio", &data->n_audio, NULL);
g_object_get (data->playbin, "n-text", &data->n_text, NULL);
```
As the comment says, this function just gathers information from the
media and prints it on the screen. The number of video, audio and
subtitle streams is directly available through the `n-video`,
`n-audio` and `n-text` properties.
``` c
for (i = 0; i < data->n_video; i++) {
tags = NULL;
/* Retrieve the stream's video tags */
g_signal_emit_by_name (data->playbin, "get-video-tags", i, &tags);
if (tags) {
g_print ("video stream %d:\n", i);
gst_tag_list_get_string (tags, GST_TAG_VIDEO_CODEC, &str);
g_print (" codec: %s\n", str ? str : "unknown");
g_free (str);
gst_tag_list_free (tags);
}
}
```
Now, for each stream, we want to retrieve its metadata. Metadata is
stored as tags in a `GstTagList` structure, which is a list of data
pieces identified by a name. The `GstTagList` associated with a stream
can be recovered with `g_signal_emit_by_name()`, and then individual
tags are extracted with the `gst_tag_list_get_*` functions
like `gst_tag_list_get_string()` for example.
> ![information]
> This rather unintuitive way of retrieving the tag list
> is called an Action Signal. Action signals are emitted by the
> application to a specific element, which then performs an action and
> returns a result. They behave like a dynamic function call, in which
> methods of a class are identified by their name (the signal's name)
> instead of their memory address. These signals are listed In the
> documentation along with the regular signals, and are tagged “Action”.
> See `playbin`, for example.
`playbin` defines 3 action signals to retrieve metadata:
`get-video-tags`, `get-audio-tags` and `get-text-tags`. The name if the
tags is standardized, and the list can be found in the `GstTagList`
documentation. In this example we are interested in the
`GST_TAG_LANGUAGE_CODE` of the streams and their `GST_TAG_*_CODEC`
(audio, video or text).
``` c
g_object_get (data->playbin, "current-video", &data->current_video, NULL);
g_object_get (data->playbin, "current-audio", &data->current_audio, NULL);
g_object_get (data->playbin, "current-text", &data->current_text, NULL);
```
Once we have extracted all the metadata we want, we get the streams that
are currently selected through 3 more properties of `playbin`:
`current-video`, `current-audio` and `current-text`.
It is interesting to always check the currently selected streams and
never make any assumption. Multiple internal conditions can make
`playbin` behave differently in different executions. Also, the order in
which the streams are listed can change from one run to another, so
checking the metadata to identify one particular stream becomes crucial.
``` c
/* Process keyboard input */
static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data) {
gchar *str = NULL;
if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) == G_IO_STATUS_NORMAL) {
int index = g_ascii_strtoull (str, NULL, 0);
if (index < 0 || index >= data->n_audio) {
g_printerr ("Index out of bounds\n");
} else {
/* If the input was a valid audio stream index, set the current audio stream */
g_print ("Setting current audio stream to %d\n", index);
g_object_set (data->playbin, "current-audio", index, NULL);
}
}
g_free (str);
return TRUE;
}
```
Finally, we allow the user to switch the running audio stream. This very
basic function just reads a string from the standard input (the
keyboard), interprets it as a number, and tries to set the
`current-audio` property of `playbin` (which previously we have only
read).
Bear in mind that the switch is not immediate. Some of the previously
decoded audio will still be flowing through the pipeline, while the new
stream becomes active and is decoded. The delay depends on the
particular multiplexing of the streams in the container, and the length
`playbin` has selected for its internal queues (which depends on the
network conditions).
If you execute the tutorial, you will be able to switch from one
language to another while the movie is running by pressing 0, 1 or 2
(and ENTER). This concludes this tutorial.
## Conclusion
This tutorial has shown:
- A few more of `playbin`'s properties: `flags`, `connection-speed`,
`n-video`, `n-audio`, `n-text`, `current-video`, `current-audio` and
`current-text`.
- How to retrieve the list of tags associated with a stream
with `g_signal_emit_by_name()`.
- How to retrieve a particular tag from the list with
`gst_tag_list_get_string()`or `gst_tag_list_get_uint()`
- How to switch the current audio simply by writing to the
`current-audio` property.
The next playback tutorial shows how to handle subtitles, either
embedded in the container or in an external file.
Remember that attached to this page you should find the complete source
code of the tutorial and any accessory files needed to build it.
It has been a pleasure having you here, and see you soon!
[Playback tutorial 2: Subtitle management]: tutorials/playback/subtitle-management.md
[information]: images/icons/emoticons/information.svg
[Mac]: installing/on-mac-osx.md
[Windows]: installing/on-windows.md
[Mac OS X]: installing/on-mac-osx.md#building-the-tutorials
[1]: installing/on-windows.md#running-the-tutorials
[iOS]: installing/for-ios-development.md#building-the-tutorials
[android]: installing/for-android-development.md#building-the-tutorials
subprojects/gst-docs/markdown/tutorials/playback/progressive-streaming.md
+433
View File
@@ -0,0 +1,433 @@
# Playback tutorial 4: Progressive streaming
{{ ALERT_PY.md }}
{{ ALERT_JS.md }}
## Goal
[](tutorials/basic/streaming.md) showed how to
enhance the user experience in poor network conditions, by taking
buffering into account. This tutorial further expands
[](tutorials/basic/streaming.md) by enabling
the local storage of the streamed media, and describes the advantages of
this technique. In particular, it shows:
- How to enable progressive downloading
- How to know what has been downloaded
- How to know where it has been downloaded
- How to limit the amount of downloaded data that is kept
## Introduction
When streaming, data is fetched from the network and a small buffer of
future-data is kept to ensure smooth playback (see
[](tutorials/basic/streaming.md)). However, data
is discarded as soon as it is displayed or rendered (there is no
past-data buffer). This means, that if a user wants to jump back and
continue playback from a point in the past, data needs to be
re-downloaded.
Media players tailored for streaming, like YouTube, usually keep all
downloaded data stored locally for this contingency. A graphical widget
is also normally used to show how much of the file has already been
downloaded.
`playbin` offers similar functionalities through the `DOWNLOAD` flag
which stores the media in a local temporary file for faster playback of
already-downloaded chunks.
This code also shows how to use the Buffering Query, which allows
knowing what parts of the file are available.
## A network-resilient example with local storage
Copy this code into a text file named `playback-tutorial-4.c`.
**playback-tutorial-4.c**
``` c
#include <gst/gst.h>
#include <string.h>
#define GRAPH_LENGTH 78
/* playbin flags */
typedef enum {
GST_PLAY_FLAG_DOWNLOAD = (1 << 7) /* Enable progressive download (on selected formats) */
} GstPlayFlags;
typedef struct _CustomData {
gboolean is_live;
GstElement *pipeline;
GMainLoop *loop;
gint buffering_level;
} CustomData;
static void got_location (GstObject *gstobject, GstObject *prop_object, GParamSpec *prop, gpointer data) {
gchar *location;
g_object_get (G_OBJECT (prop_object), "temp-location", &location, NULL);
g_print ("Temporary file: %s\n", location);
g_free (location);
/* Uncomment this line to keep the temporary file after the program exits */
/* g_object_set (G_OBJECT (prop_object), "temp-remove", FALSE, NULL); */
}
static void cb_message (GstBus *bus, GstMessage *msg, CustomData *data) {
switch (GST_MESSAGE_TYPE (msg)) {
case GST_MESSAGE_ERROR: {
GError *err;
gchar *debug;
gst_message_parse_error (msg, &err, &debug);
g_print ("Error: %s\n", err->message);
g_error_free (err);
g_free (debug);
gst_element_set_state (data->pipeline, GST_STATE_READY);
g_main_loop_quit (data->loop);
break;
}
case GST_MESSAGE_EOS:
/* end-of-stream */
gst_element_set_state (data->pipeline, GST_STATE_READY);
g_main_loop_quit (data->loop);
break;
case GST_MESSAGE_BUFFERING:
/* If the stream is live, we do not care about buffering. */
if (data->is_live) break;
gst_message_parse_buffering (msg, &data->buffering_level);
/* Wait until buffering is complete before start/resume playing */
if (data->buffering_level < 100)
gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
else
gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
break;
case GST_MESSAGE_CLOCK_LOST:
/* Get a new clock */
gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
break;
default:
/* Unhandled message */
break;
}
}
static gboolean refresh_ui (CustomData *data) {
GstQuery *query;
gboolean result;
query = gst_query_new_buffering (GST_FORMAT_PERCENT);
result = gst_element_query (data->pipeline, query);
if (result) {
gint n_ranges, range, i;
gchar graph[GRAPH_LENGTH + 1];
gint64 position = 0, duration = 0;
memset (graph, ' ', GRAPH_LENGTH);
graph[GRAPH_LENGTH] = '\0';
n_ranges = gst_query_get_n_buffering_ranges (query);
for (range = 0; range < n_ranges; range++) {
gint64 start, stop;
gst_query_parse_nth_buffering_range (query, range, &start, &stop);
start = start * GRAPH_LENGTH / (stop - start);
stop = stop * GRAPH_LENGTH / (stop - start);
for (i = (gint)start; i < stop; i++)
graph [i] = '-';
}
if (gst_element_query_position (data->pipeline, GST_FORMAT_TIME, &position) &&
GST_CLOCK_TIME_IS_VALID (position) &&
gst_element_query_duration (data->pipeline, GST_FORMAT_TIME, &duration) &&
GST_CLOCK_TIME_IS_VALID (duration)) {
i = (gint)(GRAPH_LENGTH * (double)position / (double)(duration + 1));
graph [i] = data->buffering_level < 100 ? 'X' : '>';
}
g_print ("[%s]", graph);
if (data->buffering_level < 100) {
g_print (" Buffering: %3d%%", data->buffering_level);
} else {
g_print (" ");
}
g_print ("\r");
}
return TRUE;
}
int main(int argc, char *argv[]) {
GstElement *pipeline;
GstBus *bus;
GstStateChangeReturn ret;
GMainLoop *main_loop;
CustomData data;
guint flags;
/* Initialize GStreamer */
gst_init (&argc, &argv);
/* Initialize our data structure */
memset (&data, 0, sizeof (data));
data.buffering_level = 100;
/* Build the pipeline */
pipeline = gst_parse_launch ("playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL);
bus = gst_element_get_bus (pipeline);
/* Set the download flag */
g_object_get (pipeline, "flags", &flags, NULL);
flags |= GST_PLAY_FLAG_DOWNLOAD;
g_object_set (pipeline, "flags", flags, NULL);
/* Uncomment this line to limit the amount of downloaded data */
/* g_object_set (pipeline, "ring-buffer-max-size", (guint64)4000000, NULL); */
/* Start playing */
ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
if (ret == GST_STATE_CHANGE_FAILURE) {
g_printerr ("Unable to set the pipeline to the playing state.\n");
gst_object_unref (pipeline);
return -1;
} else if (ret == GST_STATE_CHANGE_NO_PREROLL) {
data.is_live = TRUE;
}
main_loop = g_main_loop_new (NULL, FALSE);
data.loop = main_loop;
data.pipeline = pipeline;
gst_bus_add_signal_watch (bus);
g_signal_connect (bus, "message", G_CALLBACK (cb_message), &data);
g_signal_connect (pipeline, "deep-notify::temp-location", G_CALLBACK (got_location), NULL);
/* Register a function that GLib will call every second */
g_timeout_add_seconds (1, (GSourceFunc)refresh_ui, &data);
g_main_loop_run (main_loop);
/* Free resources */
g_main_loop_unref (main_loop);
gst_object_unref (bus);
gst_element_set_state (pipeline, GST_STATE_NULL);
gst_object_unref (pipeline);
g_print ("\n");
return 0;
}
```
> ![information] If you need help to compile this code, refer to the
> **Building the tutorials** section for your platform: [Mac] or
> [Windows] or use this specific command on Linux:
>
> `` gcc playback-tutorial-4.c -o playback-tutorial-4 `pkg-config --cflags --libs gstreamer-1.0` ``
>
> If you need help to run this code, refer to the **Running the
> tutorials** section for your platform: [Mac OS X], [Windows][1], for
> [iOS] or for [android].
>
> This tutorial opens a window and displays a movie, with accompanying
> audio. The media is fetched from the Internet, so the window might
> take a few seconds to appear, depending on your connection
> speed. In the console window, you should see a message indicating
> where the media is being stored, and a text graph representing the
> downloaded portions and the current position. A buffering message
> appears whenever buffering is required, which might never happen is
> your network connection is fast enough
>
> Required libraries: `gstreamer-1.0`
## Walkthrough
This code is based on that of [](tutorials/basic/streaming.md). Let’s review
only the differences.
### Setup
``` c
/* Set the download flag */
g_object_get (pipeline, "flags", &flags, NULL);
flags |= GST_PLAY_FLAG_DOWNLOAD;
g_object_set (pipeline, "flags", flags, NULL);
```
By setting this flag, `playbin` instructs its internal queue (a
`queue2` element, actually) to store all downloaded
data.
``` c
g_signal_connect (pipeline, "deep-notify::temp-location", G_CALLBACK (got_location), NULL);
```
`deep-notify` signals are emitted by `GstObject` elements (like
`playbin`) when the properties of any of their children elements
change. In this case we want to know when the `temp-location` property
changes, indicating that the `queue2` has decided where to store the
downloaded
data.
``` c
static void got_location (GstObject *gstobject, GstObject *prop_object, GParamSpec *prop, gpointer data) {
gchar *location;
g_object_get (G_OBJECT (prop_object), "temp-location", &location, NULL);
g_print ("Temporary file: %s\n", location);
g_free (location);
/* Uncomment this line to keep the temporary file after the program exits */
/* g_object_set (G_OBJECT (prop_object), "temp-remove", FALSE, NULL); */
}
```
The `temp-location` property is read from the element that triggered the
signal (the `queue2`) and printed on screen.
When the pipeline state changes from `PAUSED` to `READY`, this file is
removed. As the comment reads, you can keep it by setting the
`temp-remove` property of the `queue2` to `FALSE`.
> ![warning]
> On Windows this file is usually created inside the `Temporary Internet Files` folder, which might hide it from Windows Explorer. If you cannot find the downloaded files, try to use the console.
### User Interface
In `main` we also install a timer which we use to refresh the UI every
second.
``` c
/* Register a function that GLib will call every second */
g_timeout_add_seconds (1, (GSourceFunc)refresh_ui, &data);
```
The `refresh_ui` method queries the pipeline to find out which parts of
the file have been downloaded and what the currently playing position
is. It builds a graph to display this information (sort of a text-mode
user interface) and prints it on screen, overwriting the previous one so
it looks like it is animated:
[---->------- ]
The dashes ‘`-`’ indicate the downloaded parts, and the greater-than
sign ‘`>`’ shows the current position (turning into an ‘`X`’ when the
pipeline is paused). Keep in mind that if your network is fast enough,
you will not see the download bar (the dashes) advance at all; it will
be completely full from the beginning.
``` c
static gboolean refresh_ui (CustomData *data) {
GstQuery *query;
gboolean result;
query = gst_query_new_buffering (GST_FORMAT_PERCENT);
result = gst_element_query (data->pipeline, query);
```
The first thing we do in `refresh_ui` is construct a new Buffering
`GstQuery` with `gst_query_new_buffering()` and pass it to the pipeline
(`playbin`) with `gst_element_query()`. In [](tutorials/basic/time-management.md) we have
already seen how to perform simple queries like Position and Duration
using specific methods. More complex queries, like Buffering, need to
use the more general `gst_element_query()`.
The Buffering query can be made in different `GstFormat` (TIME, BYTES,
PERCENTAGE and a few more). Not all elements can answer the query in all
the formats, so you need to check which ones are supported in your
particular pipeline. If `gst_element_query()` returns `TRUE`, the query
succeeded. The answer to the query is contained in the same
`GstQuery` structure we created, and can be retrieved using multiple
parse methods:
``` c
n_ranges = gst_query_get_n_buffering_ranges (query);
for (range = 0; range < n_ranges; range++) {
gint64 start, stop;
gst_query_parse_nth_buffering_range (query, range, &start, &stop);
start = start * GRAPH_LENGTH / (stop - start);
stop = stop * GRAPH_LENGTH / (stop - start);
for (i = (gint)start; i < stop; i++)
graph [i] = '-';
}
```
Data does not need to be downloaded in consecutive pieces from the
beginning of the file: Seeking, for example, might force to start
downloading from a new position and leave a downloaded chunk behind.
Therefore, `gst_query_get_n_buffering_ranges()` returns the number of
chunks, or *ranges* of downloaded data, and then, the position and size
of each range is retrieved with `gst_query_parse_nth_buffering_range()`.
The format of the returned values (start and stop position for each
range) depends on what we requested in the
`gst_query_new_buffering()` call. In this case, PERCENTAGE. These
values are used to generate the graph.
``` c
if (gst_element_query_position (data->pipeline, GST_FORMAT_TIME, &position) &&
GST_CLOCK_TIME_IS_VALID (position) &&
gst_element_query_duration (data->pipeline, GST_FORMAT_TIME, &duration) &&
GST_CLOCK_TIME_IS_VALID (duration)) {
i = (gint)(GRAPH_LENGTH * (double)position / (double)(duration + 1));
graph [i] = data->buffering_level < 100 ? 'X' : '>';
}
```
Next, the current position is queried. It could be queried in the
PERCENT format, so code similar to the one used for the ranges is used,
but currently this format is not well supported for position queries.
Instead, we use the TIME format and also query the duration to obtain a
percentage.
The current position is indicated with either a ‘`>`’ or an ‘`X`’
depending on the buffering level. If it is below 100%, the code in the
`cb_message` method will have set the pipeline to `PAUSED`, so we print
an ‘`X`’. If the buffering level is 100% the pipeline is in the
`PLAYING` state and we print a ‘`>`’.
``` c
if (data->buffering_level < 100) {
g_print (" Buffering: %3d%%", data->buffering_level);
} else {
g_print (" ");
}
```
Finally, if the buffering level is below 100%, we report this
information (and delete it otherwise).
### Limiting the size of the downloaded file
``` c
/* Uncomment this line to limit the amount of downloaded data */
/* g_object_set (pipeline, "ring-buffer-max-size", (guint64)4000000, NULL); */
```
Uncomment line 139 to see how this can be achieved. This reduces the
size of the temporary file, by overwriting already played regions.
Observe the download bar to see which regions are kept available in the
file.
## Conclusion
This tutorial has shown:
- How to enable progressive downloading with the
`GST_PLAY_FLAG_DOWNLOAD` `playbin` flag
- How to know what has been downloaded using a Buffering `GstQuery`
- How to know where it has been downloaded with the
`deep-notify::temp-location` signal
- How to limit the size of the temporary file with
the `ring-buffer-max-size` property of `playbin`.
It has been a pleasure having you here, and see you soon!
[information]: images/icons/emoticons/information.svg
[Mac]: installing/on-mac-osx.md
[Windows]: installing/on-windows.md
[Mac OS X]: installing/on-mac-osx.md#building-the-tutorials
[1]: installing/on-windows.md#running-the-tutorials
[iOS]: installing/for-ios-development.md#building-the-tutorials
[android]: installing/for-android-development.md#building-the-tutorials
[warning]: images/icons/emoticons/warning.svg
subprojects/gst-docs/markdown/tutorials/playback/short-cutting-the-pipeline.md
+271
View File
@@ -0,0 +1,271 @@
# Playback tutorial 3: Short-cutting the pipeline
{{ ALERT_PY.md }}
{{ ALERT_JS.md }}
## Goal
[](tutorials/basic/short-cutting-the-pipeline.md) showed
how an application can manually extract or inject data into a pipeline
by using two special elements called `appsrc` and `appsink`.
`playbin` allows using these elements too, but the method to connect
them is different. To connect an `appsink` to `playbin` see [](tutorials/playback/custom-playbin-sinks.md).
This tutorial shows:
- How to connect `appsrc` with `playbin`
- How to configure the `appsrc`
## A playbin waveform generator
Copy this code into a text file named `playback-tutorial-3.c`.
**playback-tutorial-3.c**
``` c
#include <gst/gst.h>
#include <gst/audio/audio.h>
#include <string.h>
#define CHUNK_SIZE 1024 /* Amount of bytes we are sending in each buffer */
#define SAMPLE_RATE 44100 /* Samples per second we are sending */
/* Structure to contain all our information, so we can pass it to callbacks */
typedef struct _CustomData {
GstElement *pipeline;
GstElement *app_source;
guint64 num_samples; /* Number of samples generated so far (for timestamp generation) */
gfloat a, b, c, d; /* For waveform generation */
guint sourceid; /* To control the GSource */
GMainLoop *main_loop; /* GLib's Main Loop */
} CustomData;
/* This method is called by the idle GSource in the mainloop, to feed CHUNK_SIZE bytes into appsrc.
* The ide handler is added to the mainloop when appsrc requests us to start sending data (need-data signal)
* and is removed when appsrc has enough data (enough-data signal).
*/
static gboolean push_data (CustomData *data) {
GstBuffer *buffer;
GstFlowReturn ret;
int i;
GstMapInfo map;
gint16 *raw;
gint num_samples = CHUNK_SIZE / 2; /* Because each sample is 16 bits */
gfloat freq;
/* Create a new empty buffer */
buffer = gst_buffer_new_and_alloc (CHUNK_SIZE);
/* Set its timestamp and duration */
GST_BUFFER_TIMESTAMP (buffer) = gst_util_uint64_scale (data->num_samples, GST_SECOND, SAMPLE_RATE);
GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale (num_samples, GST_SECOND, SAMPLE_RATE);
/* Generate some psychodelic waveforms */
gst_buffer_map (buffer, &map, GST_MAP_WRITE);
raw = (gint16 *)map.data;
data->c += data->d;
data->d -= data->c / 1000;
freq = 1100 + 1000 * data->d;
for (i = 0; i < num_samples; i++) {
data->a += data->b;
data->b -= data->a / freq;
raw[i] = (gint16)(500 * data->a);
}
gst_buffer_unmap (buffer, &map);
data->num_samples += num_samples;
/* Push the buffer into the appsrc */
g_signal_emit_by_name (data->app_source, "push-buffer", buffer, &ret);
/* Free the buffer now that we are done with it */
gst_buffer_unref (buffer);
if (ret != GST_FLOW_OK) {
/* We got some error, stop sending data */
return FALSE;
}
return TRUE;
}
/* This signal callback triggers when appsrc needs data. Here, we add an idle handler
* to the mainloop to start pushing data into the appsrc */
static void start_feed (GstElement *source, guint size, CustomData *data) {
if (data->sourceid == 0) {
g_print ("Start feeding\n");
data->sourceid = g_idle_add ((GSourceFunc) push_data, data);
}
}
/* This callback triggers when appsrc has enough data and we can stop sending.
* We remove the idle handler from the mainloop */
static void stop_feed (GstElement *source, CustomData *data) {
if (data->sourceid != 0) {
g_print ("Stop feeding\n");
g_source_remove (data->sourceid);
data->sourceid = 0;
}
}
/* This function is called when an error message is posted on the bus */
static void error_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
GError *err;
gchar *debug_info;
/* Print error details on the screen */
gst_message_parse_error (msg, &err, &debug_info);
g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
g_clear_error (&err);
g_free (debug_info);
g_main_loop_quit (data->main_loop);
}
/* This function is called when playbin has created the appsrc element, so we have
* a chance to configure it. */
static void source_setup (GstElement *pipeline, GstElement *source, CustomData *data) {
GstAudioInfo info;
GstCaps *audio_caps;
g_print ("Source has been created. Configuring.\n");
data->app_source = source;
/* Configure appsrc */
gst_audio_info_set_format (&info, GST_AUDIO_FORMAT_S16, SAMPLE_RATE, 1, NULL);
audio_caps = gst_audio_info_to_caps (&info);
g_object_set (source, "caps", audio_caps, "format", GST_FORMAT_TIME, NULL);
g_signal_connect (source, "need-data", G_CALLBACK (start_feed), data);
g_signal_connect (source, "enough-data", G_CALLBACK (stop_feed), data);
gst_caps_unref (audio_caps);
}
int main(int argc, char *argv[]) {
CustomData data;
GstBus *bus;
/* Initialize custom data structure */
memset (&data, 0, sizeof (data));
data.b = 1; /* For waveform generation */
data.d = 1;
/* Initialize GStreamer */
gst_init (&argc, &argv);
/* Create the playbin element */
data.pipeline = gst_parse_launch ("playbin uri=appsrc://", NULL);
g_signal_connect (data.pipeline, "source-setup", G_CALLBACK (source_setup), &data);
/* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
bus = gst_element_get_bus (data.pipeline);
gst_bus_add_signal_watch (bus);
g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, &data);
gst_object_unref (bus);
/* Start playing the pipeline */
gst_element_set_state (data.pipeline, GST_STATE_PLAYING);
/* Create a GLib Main Loop and set it to run */
data.main_loop = g_main_loop_new (NULL, FALSE);
g_main_loop_run (data.main_loop);
/* Free resources */
gst_element_set_state (data.pipeline, GST_STATE_NULL);
gst_object_unref (data.pipeline);
return 0;
}
```
> ![information] If you need help to compile this code, refer to the
> **Building the tutorials** section for your platform: [Mac] or
> [Windows] or use this specific command on Linux:
>
> `` gcc playback-tutorial-3.c -o playback-tutorial-3 `pkg-config --cflags --libs gstreamer-1.0 gstreamer-audio-1.0` ``
>
> If you need help to run this code, refer to the **Running the
> tutorials** section for your platform: [Mac OS X], [Windows][1], for
> [iOS] or for [android].
>
> This tutorial opens a window and displays a movie, with accompanying
> audio. The media is fetched from the Internet, so the window might
> take a few seconds to appear, depending on your connection
> speed. In the console window, you should see a message indicating
> where the media is being stored, and a text graph representing the
> downloaded portions and the current position. A buffering message
> appears whenever buffering is required, which might never happen is
> your network connection is fast enough
>
> Required libraries: `gstreamer-1.0` `gstreamer-audio-1.0`
To use an `appsrc` as the source for the pipeline, simply instantiate a
`playbin` and set its URI to `appsrc://`
``` c
/* Create the playbin element */
data.pipeline = gst_parse_launch ("playbin uri=appsrc://", NULL);
```
`playbin` will create an internal `appsrc` element and fire the
`source-setup` signal to allow the application to configure
it:
``` c
g_signal_connect (data.pipeline, "source-setup", G_CALLBACK (source_setup), &data);
```
In particular, it is important to set the caps property of `appsrc`,
since, once the signal handler returns, `playbin` will instantiate the
next element in the pipeline according to these
caps:
``` c
/* This function is called when playbin has created the appsrc element, so we have
* a chance to configure it. */
static void source_setup (GstElement *pipeline, GstElement *source, CustomData *data) {
GstAudioInfo info;
GstCaps *audio_caps;
g_print ("Source has been created. Configuring.\n");
data->app_source = source;
/* Configure appsrc */
gst_audio_info_set_format (&info, GST_AUDIO_FORMAT_S16, SAMPLE_RATE, 1, NULL);
audio_caps = gst_audio_info_to_caps (&info);
g_object_set (source, "caps", audio_caps, "format", GST_FORMAT_TIME, NULL);
g_signal_connect (source, "need-data", G_CALLBACK (start_feed), data);
g_signal_connect (source, "enough-data", G_CALLBACK (stop_feed), data);
gst_caps_unref (audio_caps);
}
```
The configuration of the `appsrc` is exactly the same as in
[](tutorials/basic/short-cutting-the-pipeline.md):
the caps are set to `audio/x-raw`, and two callbacks are registered,
so the element can tell the application when it needs to start and stop
pushing data. See [](tutorials/basic/short-cutting-the-pipeline.md)
for more details.
From this point onwards, `playbin` takes care of the rest of the
pipeline, and the application only needs to worry about generating more
data when told so.
To learn how data can be extracted from `playbin` using the
`appsink` element, see [](tutorials/playback/custom-playbin-sinks.md).
## Conclusion
This tutorial applies the concepts shown in
[](tutorials/basic/short-cutting-the-pipeline.md) to
`playbin`. In particular, it has shown:
- How to connect `appsrc` with `playbin` using the special
URI `appsrc://`
- How to configure the `appsrc` using the `source-setup` signal
It has been a pleasure having you here, and see you soon!
subprojects/gst-docs/markdown/tutorials/playback/subtitle-management.md
+393
View File
@@ -0,0 +1,393 @@
# Playback tutorial 2: Subtitle management
{{ ALERT_PY.md }}
{{ ALERT_JS.md }}
## Goal
This tutorial is very similar to the previous one, but instead of
switching among different audio streams, we will use subtitle streams.
This will allow us to learn:
- How to choose the subtitle stream
- How to add external subtitles
- How to customize the font used for the subtitles
## Introduction
We already know (from the previous tutorial) that container files can
hold multiple audio and video streams, and that we can very easily
choose among them by changing the `current-audio` or
`current-video` `playbin` property. Switching subtitles is just as
easy.
It is worth noting that, just like it happens with audio and video,
`playbin` takes care of choosing the right decoder for the subtitles,
and that the plugin structure of GStreamer allows adding support for new
formats as easily as copying a file. Everything is invisible to the
application developer.
Besides subtitles embedded in the container, `playbin` offers the
possibility to add an extra subtitle stream from an external URI.
This tutorial opens a file which already contains 5 subtitle streams,
and adds another one from another file (for the Greek language).
## The multilingual player with subtitles
Copy this code into a text file named `playback-tutorial-2.c` (or find
it in the GStreamer installation).
**playback-tutorial-2.c**
``` c
#include <stdio.h>
#include <gst/gst.h>
/* Structure to contain all our information, so we can pass it around */
typedef struct _CustomData {
GstElement *playbin; /* Our one and only element */
gint n_video; /* Number of embedded video streams */
gint n_audio; /* Number of embedded audio streams */
gint n_text; /* Number of embedded subtitle streams */
gint current_video; /* Currently playing video stream */
gint current_audio; /* Currently playing audio stream */
gint current_text; /* Currently playing subtitle stream */
GMainLoop *main_loop; /* GLib's Main Loop */
} CustomData;
/* playbin flags */
typedef enum {
GST_PLAY_FLAG_VIDEO = (1 << 0), /* We want video output */
GST_PLAY_FLAG_AUDIO = (1 << 1), /* We want audio output */
GST_PLAY_FLAG_TEXT = (1 << 2) /* We want subtitle output */
} GstPlayFlags;
/* Forward definition for the message and keyboard processing functions */
static gboolean handle_message (GstBus *bus, GstMessage *msg, CustomData *data);
static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data);
int main(int argc, char *argv[]) {
CustomData data;
GstBus *bus;
GstStateChangeReturn ret;
gint flags;
GIOChannel *io_stdin;
/* Initialize GStreamer */
gst_init (&argc, &argv);
/* Create the elements */
data.playbin = gst_element_factory_make ("playbin", "playbin");
if (!data.playbin) {
g_printerr ("Not all elements could be created.\n");
return -1;
}
/* Set the URI to play */
g_object_set (data.playbin, "uri", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.ogv", NULL);
/* Set the subtitle URI to play and some font description */
g_object_set (data.playbin, "suburi", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer_gr.srt", NULL);
g_object_set (data.playbin, "subtitle-font-desc", "Sans, 18", NULL);
/* Set flags to show Audio, Video and Subtitles */
g_object_get (data.playbin, "flags", &flags, NULL);
flags |= GST_PLAY_FLAG_VIDEO | GST_PLAY_FLAG_AUDIO | GST_PLAY_FLAG_TEXT;
g_object_set (data.playbin, "flags", flags, NULL);
/* Add a bus watch, so we get notified when a message arrives */
bus = gst_element_get_bus (data.playbin);
gst_bus_add_watch (bus, (GstBusFunc)handle_message, &data);
/* Add a keyboard watch so we get notified of keystrokes */
#ifdef G_OS_WIN32
io_stdin = g_io_channel_win32_new_fd (fileno (stdin));
#else
io_stdin = g_io_channel_unix_new (fileno (stdin));
#endif
g_io_add_watch (io_stdin, G_IO_IN, (GIOFunc)handle_keyboard, &data);
/* Start playing */
ret = gst_element_set_state (data.playbin, GST_STATE_PLAYING);
if (ret == GST_STATE_CHANGE_FAILURE) {
g_printerr ("Unable to set the pipeline to the playing state.\n");
gst_object_unref (data.playbin);
return -1;
}
/* Create a GLib Main Loop and set it to run */
data.main_loop = g_main_loop_new (NULL, FALSE);
g_main_loop_run (data.main_loop);
/* Free resources */
g_main_loop_unref (data.main_loop);
g_io_channel_unref (io_stdin);
gst_object_unref (bus);
gst_element_set_state (data.playbin, GST_STATE_NULL);
gst_object_unref (data.playbin);
return 0;
}
/* Extract some metadata from the streams and print it on the screen */
static void analyze_streams (CustomData *data) {
gint i;
GstTagList *tags;
gchar *str;
guint rate;
/* Read some properties */
g_object_get (data->playbin, "n-video", &data->n_video, NULL);
g_object_get (data->playbin, "n-audio", &data->n_audio, NULL);
g_object_get (data->playbin, "n-text", &data->n_text, NULL);
g_print ("%d video stream(s), %d audio stream(s), %d text stream(s)\n",
data->n_video, data->n_audio, data->n_text);
g_print ("\n");
for (i = 0; i < data->n_video; i++) {
tags = NULL;
/* Retrieve the stream's video tags */
g_signal_emit_by_name (data->playbin, "get-video-tags", i, &tags);
if (tags) {
g_print ("video stream %d:\n", i);
gst_tag_list_get_string (tags, GST_TAG_VIDEO_CODEC, &str);
g_print (" codec: %s\n", str ? str : "unknown");
g_free (str);
gst_tag_list_free (tags);
}
}
g_print ("\n");
for (i = 0; i < data->n_audio; i++) {
tags = NULL;
/* Retrieve the stream's audio tags */
g_signal_emit_by_name (data->playbin, "get-audio-tags", i, &tags);
if (tags) {
g_print ("audio stream %d:\n", i);
if (gst_tag_list_get_string (tags, GST_TAG_AUDIO_CODEC, &str)) {
g_print (" codec: %s\n", str);
g_free (str);
}
if (gst_tag_list_get_string (tags, GST_TAG_LANGUAGE_CODE, &str)) {
g_print (" language: %s\n", str);
g_free (str);
}
if (gst_tag_list_get_uint (tags, GST_TAG_BITRATE, &rate)) {
g_print (" bitrate: %d\n", rate);
}
gst_tag_list_free (tags);
}
}
g_print ("\n");
for (i = 0; i < data->n_text; i++) {
tags = NULL;
/* Retrieve the stream's subtitle tags */
g_print ("subtitle stream %d:\n", i);
g_signal_emit_by_name (data->playbin, "get-text-tags", i, &tags);
if (tags) {
if (gst_tag_list_get_string (tags, GST_TAG_LANGUAGE_CODE, &str)) {
g_print (" language: %s\n", str);
g_free (str);
}
gst_tag_list_free (tags);
} else {
g_print (" no tags found\n");
}
}
g_object_get (data->playbin, "current-video", &data->current_video, NULL);
g_object_get (data->playbin, "current-audio", &data->current_audio, NULL);
g_object_get (data->playbin, "current-text", &data->current_text, NULL);
g_print ("\n");
g_print ("Currently playing video stream %d, audio stream %d and subtitle stream %d\n",
data->current_video, data->current_audio, data->current_text);
g_print ("Type any number and hit ENTER to select a different subtitle stream\n");
}
/* Process messages from GStreamer */
static gboolean handle_message (GstBus *bus, GstMessage *msg, CustomData *data) {
GError *err;
gchar *debug_info;
switch (GST_MESSAGE_TYPE (msg)) {
case GST_MESSAGE_ERROR:
gst_message_parse_error (msg, &err, &debug_info);
g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
g_clear_error (&err);
g_free (debug_info);
g_main_loop_quit (data->main_loop);
break;
case GST_MESSAGE_EOS:
g_print ("End-Of-Stream reached.\n");
g_main_loop_quit (data->main_loop);
break;
case GST_MESSAGE_STATE_CHANGED: {
GstState old_state, new_state, pending_state;
gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->playbin)) {
if (new_state == GST_STATE_PLAYING) {
/* Once we are in the playing state, analyze the streams */
analyze_streams (data);
}
}
} break;
}
/* We want to keep receiving messages */
return TRUE;
}
/* Process keyboard input */
static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data) {
gchar *str = NULL;
if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) == G_IO_STATUS_NORMAL) {
int index = atoi (str);
if (index < 0 || index >= data->n_text) {
g_printerr ("Index out of bounds\n");
} else {
/* If the input was a valid subtitle stream index, set the current subtitle stream */
g_print ("Setting current subtitle stream to %d\n", index);
g_object_set (data->playbin, "current-text", index, NULL);
}
}
g_free (str);
return TRUE;
}
```
> ![information] Need help?
>
> If you need help to compile this code, refer to the **Building the
> tutorials** section for your platform: [Linux], [Mac OS X] or
> [Windows], or use this specific command on Linux:
>
> `` gcc playback-tutorial-2.c -o playback-tutorial-2 `pkg-config --cflags --libs gstreamer-1.0` ``
>
> If you need help to run this code, refer to the **Running the
> tutorials** section for your platform: [Linux][1], [Mac OS X][2] or
> [Windows][3].
>
> This tutorial opens a window and displays a movie, with accompanying
> audio. The media is fetched from the Internet, so the window might
> take a few seconds to appear, depending on your connection
> speed. The number of subtitle streams is shown in the terminal, and
> the user can switch from one to another by entering a number and
> pressing enter. A small delay is to be
> expected. _Please read the note at the bottom of this
> page._ Bear in mind that
> there is no latency management (buffering), so on slow connections,
> the movie might stop after a few seconds. See how
> [](tutorials/basic/streaming.md) solves this issue.
>
> Required libraries: `gstreamer-1.0`
## Walkthrough
This tutorial is copied from
[](tutorials/playback/playbin-usage.md) with some changes, so let's
review only the changes.
``` c
/* Set the subtitle URI to play and some font description */
g_object_set (data.playbin, "suburi", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer_gr.srt", NULL);
g_object_set (data.playbin, "subtitle-font-desc", "Sans, 18", NULL);
```
After setting the media URI, we set the `suburi` property, which points
`playbin` to a file containing a subtitle stream. In this case, the
media file already contains multiple subtitle streams, so the one
provided in the `suburi` is added to the list, and will be the currently
selected one.
Note that metadata concerning a subtitle stream (like its language)
resides in the container file, therefore, subtitles not embedded in a
container will not have metadata. When running this tutorial you will
find that the first subtitle stream does not have a language tag.
The `subtitle-font-desc` property allows specifying the font to render
the subtitles. Since [Pango](http://www.pango.org/) is the library used
to render fonts, you can check its documentation to see how this font
should be specified, in particular, the
[pango-font-description-from-string](http://developer.gnome.org/pango/stable/pango-Fonts.html#pango-font-description-from-string) function.
In a nutshell, the format of the string representation is `[FAMILY-LIST]
[STYLE-OPTIONS] [SIZE]` where `FAMILY-LIST` is a comma separated list of
families optionally terminated by a comma, `STYLE_OPTIONS` is a
whitespace separated list of words where each word describes one of
style, variant, weight, or stretch, and `SIZE` is an decimal number
(size in points). For example the following are all valid string
representations:
- sans bold 12
- serif, monospace bold italic condensed 16
- normal 10
The commonly available font families are: Normal, Sans, Serif and
Monospace.
The available styles are: Normal (the font is upright), Oblique (the
font is slanted, but in a roman style), Italic (the font is slanted in
an italic style).
The available weights are: Ultra-Light, Light, Normal, Bold, Ultra-Bold,
Heavy.
The available variants are: Normal, Small\_Caps (A font with the lower
case characters replaced by smaller variants of the capital characters)
The available stretch styles
are: Ultra-Condensed, Extra-Condensed, Condensed, Semi-Condensed, Normal, Semi-Expanded, Expanded,
Extra-Expanded, Ultra-Expanded
``` c
/* Set flags to show Audio, Video and Subtitles */
g_object_get (data.playbin, "flags", &flags, NULL);
flags |= GST_PLAY_FLAG_VIDEO | GST_PLAY_FLAG_AUDIO | GST_PLAY_FLAG_TEXT;
g_object_set (data.playbin, "flags", flags, NULL);
```
We set the `flags` property to allow Audio, Video and Text (Subtitles).
The rest of the tutorial is the same as [](tutorials/playback/playbin-usage.md), except
that the keyboard input changes the `current-text` property instead of
the `current-audio`. As before, keep in mind that stream changes are not
immediate, since there is a lot of information flowing through the
pipeline that needs to reach the end of it before the new stream shows
up.
## Conclusion
This tutorial showed how to handle subtitles from `playbin`, whether
they are embedded in the container or in a different file:
- Subtitles are chosen using the `current-tex`t and `n-tex`t
properties of `playbin`.
- External subtitle files can be selected using the `suburi` property.
- Subtitle appearance can be customized with the
`subtitle-font-desc` property.
The next playback tutorial shows how to change the playback speed.
Remember that attached to this page you should find the complete source
code of the tutorial and any accessory files needed to build it.
It has been a pleasure having you here, and see you soon\!
[information]: images/icons/emoticons/information.svg