GES Basic Beginner Tutorial

Intro

GStreamer Editing Services or GES is a library built to streamline the creation of multimedia editing. 

Based on the GStreamer multimedia framework and the GNonLin set of plugins, its goals are to suit all types of

editing-related applications. GStreamer and GNonLin give the programmer all the building blocks to create Non-Linear

editors, but requires them to write a substantial amount of code to do so. GES aims to close the gap between 

GStreamer and GNonLin and the application developer by offering a series of classes to simplify the creation of many 

kinds of editing-related applications.

Figure 1. The GES Architecture

Goal

Introduce new users to the inner workings of starting a new project in GES using Python. After this tutorial, you will be

able to create a video out of a set of pictures. This includes:

• Setting up the boilerplate and the main function

• Initializing GStreamer and GES

• Setting up the timeline and layer

• Setting up the pipeline and profiles

• Riding the bus

Before we begin

This tutorial assumes that you have the following Debian packages installed:

• gir1.2-ges-1.0

• gstreamer1.0-plugins-base

• gstreamer1.0-plugins-good

• gstreamer1.0-plugins-bad

• gstreamer1.0-plugins-ugly

All documentation relating to GI library functions can be found here.

All documentation relating to GLib library functions can be found here.

All documentation relating to Gst library functions can be found here.

All documentation relating to GES library functions can be found here.

All documentation relating to GstPbutils library functions can be found here.

The overall documentation of GStreamer can be found here. This was used as a reference to explain the functionality 

of the functions used in this code.

Setting Up the Boilerplate

Every new GES project requires you to set up some boilerplate. This includes:

• Importing the required libraries

• import gi

• from gi.repository import GLib

• gi.require_version('Gst', '1.0')

• from gi.repository import Gst

• gi.require_version('GES', '1.0')

• from gi.repository import GES

• gi.require_version('GstPbutils', '1.0')

• from gi.repository import GstPbutils

• These imports are explained as follows

• GObject Introspection (gi)  - Allows easy access to the entirety of the GNOME software platform. 

• GLib - A low-level core library that forms the basis of GStreamer.

• Gst - The GStreamer core.

• GES - GStreamer Editing Services, a library based on GStreamer for creating multimedia editing

  applications.

• GstPbutils - A collection of plugins created for GStreamer. 

• We use gi.require_version() to choose the version of some of the libraries we import, in case the

  user’s system has multiple versions installed or a newer version has been found online. For this example,

  we will use version 1.0 for Gst, GES, and GstPbutils.

• Choosing the framerate for the video

• FRAME_RATE = # desired frames per second

• Set your desired frame rate to your specific rate. The most common setting for frame rates in movies is 

  24 frames per second (FPS) (as in 24 photos are taken per second by the camera). With a higher FPS,

  the video may appear to be moving faster than intended. With a lower FPS, the video may appear to be

  in slow motion. It is important to update the FPS depending on what FPS the images being used were

  intended to be shown in.

• Creating the to_nanoseconds function

• def to_nanoseconds(frame_number):

•     return Gst.util_uint64_scale_ceil(frame_number, Gst.SECOND,       FRAME_RATE)

• This function returns the time in nanoseconds that has passed before the frame that was passed in as a

  parameter shows up in the video.

• It is worth mentioning that an equivalent function is built into GES as Timeline.get_frame_time in

  versions 1.18 or newer.

• Since we are using version 1.0 of GES, we do not have access to that function and therefore create 

  to_nanoseconds for ease of use.

Initializing GStreamer and GES

Setting up the main function

def main(image_filenames, output_filename):

• image_filenames - The name of the pictures on your device

• output_filename - The name of the video after creation

All GES video projects require the initialization of Gst and GES. In order to initialize these, call:

Gst.init(None)

      GES.init()

inside the main function.

• Gst.init(None)- Initializes the GStreamer library, setting up internal path lists, registering built-in elements,

  and loading standard plugins.

• GES.init() - Initialize the GStreamer Editing Service. It is important to initialize the GStreamer library before

  attempting to initialize GES.

Setting Up the Timeline and Layer

The timeline in GES is where the pictures will be stored and converted into a video. A timeline is composed as a set of

tracks and layers. A track is essentially an output source for the timeline. A layer is responsible for collecting and 

ordering clips in the timeline. Any layer within the timeline has a priority, corresponding to their index within the timeline. 

The layer with the highest priority is 0. As the indices increase, the priority becomes lesser. The function  

ges_timeline_move_layer() shall be used if you wish to change how layers are prioritized in a timeline.  For this 

tutorial, we will be working with one (1) layer. 

To set up your timeline with a single layer, simply call:

     timeline = GES.Timeline.new_audio_video()

     layer = timeline.append_layer()

• new_audio_video()- creates a new timeline containing a single audio track and a single video track.

• append_layer()- adds the layer to the timeline at the lowest priority by default.

Adding Assets to the Layer

Now it is time to add the pictures into the layer. To start, we must loop over every image filename, along with their 

respective frame number, and add it to our layer at the correct start and end time.

We will use a for loop in conjunction with enumerate()in order to capture each file name with its associated frame 

number. 

• As a reminder, enumerate()returns the “count” of the element in the list that you are at, as well as the element 

  itself. By default, enumerate()starts its “count” at 0.

for frame_number, image_filename in enumerate(image_filenames):

In order to add an asset to the layer, we must first create the asset. We are creating a URI clip asset, which will allow us 

to use the media file in GES. However, before creating a URI clip asset, we must first use  

Gst.filename_to_uri(filename)in order to convert the filename to a valid URI string.

• On a Windows system, filename should be in UTF-8 encoding

• It may also be possible to use Glib.filename_to_uri(filename); however, the function we are using 

  attempts to handle relative file paths as well.

Now, to actually create the URI clip asset, we will use the function request_sync()in conjunction with  

filename_to_uri(). We will pass the return value from filename_to_uri() to request_sync()which will 

synchronously create a URI clip asset for the URI we passed as a parameter.

• In larger applications, it could potentially be dangerous to create URI clips synchronously. It is recommended as 

  best practice to use asset_new()in those cases.

asset = GES.UriClipAsset.request_sync(

           Gst.filename_to_uri(image_filename),

        )

The URI clip asset is now created and stored in the variable called asset.

Remember to_nanoseconds()from earlier? Now is the time where it comes in handy. We would like to get the time, 

in nanoseconds, between the current frame and the next frame. We will store the current frame’s time in the variable 

start_time and the next frame’s time in the variable end_time 

      start_time = to_nanoseconds(frame_number)

      end_time = to_nanoseconds(frame_number + 1)

We now have all of the necessary parameters to add our asset to our layer. The function that we will use to add the 

asset to the layer is GES.Layer.add_asset(). When using add_asset()it requires 6 parameters.

• The asset to extract the new clip from; In our case, this is the variable asset.

• The start; In our case, this is the variable start_time.

• The inpoint; In our case, we want all of our inpoints to be 0.

• The duration; In our case, this is the start_time subtracted from the end_time.

• The track types; In our case, we will want to get all of the supported formats for our asset.

• We will use the function get_supported_formats() on the asset variable in order to get the 

  supported formats.

• You could also use GES.TrackType.UNKNOWN for only the default formats.

layer.add_asset(

        asset,

        start_time,

        0,

        end_time - start_time,

        asset.get_supported_formats(),

    )

After all of the pictures have been converted to URI clip assets and added to our layer in the correct order, we are ready 

to move on to setting up the pipeline.

Creating the Pipeline

Before creating the pipeline, it is important to understand the job of the pipeline. The pipeline is in charge of managing 

the global clock, as well as providing a “bus” to the application. It is important to know that the pipeline has a variety of 

states that it can be in at any given time. The state can be NULL, READY, PAUSED, and PLAYING. Usually, the clock 

does not need to be manually changed. The clock selection algorithm by default will select a clock by an element that 

is closest to the source. If no clock is provided by an element, Gst.SystemClock is used.

Now onto creating the pipeline. GES makes creating the pipeline itself extremely straightforward, the function 

GES.Pipeline.new() creates a new pipeline and returns it.

pipeline = GES.Pipeline.new()

In order to place our timeline into the pipeline, we will use the function set_timeline(), which has 1 parameter of the 

timeline you want to add to the pipeline. In our case, it is the variable timeline.

• You should only call this function 1 time on any given pipeline, as a pipeline can only have its timeline set once.

       pipeline.set_timeline(timeline)

Creating the Encoding Profile

Encoding profiles describe the media types and settings that you want to use for a certain encoding process. Typically, 

the top-level encoding profiles are GstPbutils.EncodingContainerProfile which have a readable name as well 

as a description of the container format to use. These profiles reference one or more GstPbutils.EncodingProfile 

which describe what encoding format should be used for each stream. 

It is now time to create our container profile. We will be utilizing the function EncodingContainerProfile.new()to 

create our Encoding Container Profile. 

EncodingContainerProfile.new()has 4 parameters:

• Name; The name of the profile, in our case we will have it be None.

• Description; The description of the profile, in our case we will have it be None.

• Format; The format to use for this profile, in our case we will want to get the capabilities (caps) from the string 

  ‘video/ogg’ using the function from_string().

• Preset; The preset to use for this profile, in our case we will have it be None.

We are only working on a single video source, therefore we do not need to worry about profile naming.

container_profile = GstPbutils.EncodingContainerProfile.new(

        None,

        None,

        Gst.Caps.from_string('video/ogg'),

        None,

    )

Now that the container profile is created, it is time to add our video profile to the container. We will use the function 

add_profile() to add the new profile to the container profile. We will create the new profile using  

GstPbutils.EncodingVideoProfile.new() which has 4 parameters:

• Format: The capabilities (caps) of the profile.

• We will use caps.from_string()to define our caps for the profile. The caps we will define are 

  “video/x-theora”, a width of 1920 (pixels), a height of 1080 (pixels), and a framerate as a function of 

  the FRAME_RATE variable we created earlier.

• Preset: We do not have a preset, in our case None.

• Restriction: We do not need to restrict any capabilities, in our case None.

• Presence: How many times this stream must be used, 0 is any number of times so we will use that in our case.

     container_profile.add_profile(GstPbutils.EncodingVideoProfile.new(

        Gst.Caps.from_string(', '.join([

            'video/x-theora',

            'width=(int)1920',

            'height=(int)1080',

            f'framerate=(fraction){FRAME_RATE}/1',

        ])),

        None,

        None,

        0,

    ))

Now that the capabilities are defined, added to the new profile, and then the profile is added to our container profile, it 

is time to set our render settings for our pipeline. In order to set our render settings, we will use the function 

set_render_settings()which has 2 parameters:

• Output URI: The URI to save the pipeline’s rendering result to. In our case, we will use the user’s  

  output_filename in conjunction with the function filename_to_uri() to convert the filename to a URI.

• Profile: The encoding profile to use for rendering the pipeline’s timeline. In our case we will use our container 

  profile that we created during the previous steps.

pipeline.set_render_settings(

        Gst.filename_to_uri(output_filename),

        container_profile,

    )

Riding the Bus

Now we will work on initializing and connecting the bus for the pipeline. The bus is an object that delivers the 

information (messages) from the streaming threads to the application in a first-in first-out (FIFO) method. Prior to 

initializing the bus for the pipeline, we will use the function GLib.MainLoop()to create the main event loop for 

our application.

    loop = GLib.MainLoop()

Next, we will use the function get_bus()to get the bus of the pipeline and save it into a variable named bus. We will 

use the function add_signal_watch() on our bus in order to make the bus emit the signal “message” every time 

a message is posted on the bus.

    bus = pipeline.get_bus()

    bus.add_signal_watch()

We will then use the function connect()to connect the “message” signal to a function, in this case it is a lambda 

function that will quit the main loop if the message is of type Gst.MessageType.EOS which means “end of stream”. 

This connection will allow for our application to terminate, instead of looping forever, when our bus runs out of things 

to do, A.K.A. “end of stream”.

    bus.connect(

        'message',

        lambda unused_bus, message:

            loop.quit() if message.type == Gst.MessageType.EOS else None,

    )

Setting Pipeline Mode and State

Next we will set the mode for our pipeline. In order to set the mode for our pipeline, we will use the function  

set_mode(). It has 1 parameter, which is the mode for the pipeline, in our case we will use the mode 

GES.PipelineFlags.RENDER. This mode will render the pipeline with forced decoding.

    pipeline.set_mode(GES.PipelineFlags.RENDER)

It is also important to set the state of the pipeline prior to starting the main loop of the application. We will use the 

function set_state() to set the state of our pipeline to Gst.State.PLAYING. This state means that the pipeline 

clock is running and data is flowing.

    pipeline.set_state(Gst.State.PLAYING)

Rendering the Video

We will now invoke the function run()on the loop variable we created earlier. This loop will run until the 

“end of stream” type message is received in the loop we created earlier. While this loop is running, the video is being 

rendered by the application.

    loop.run()

Once the loop has terminated, we will want to remove the signal watching from the pipeline’s bus. It is important to 

remove any signal watching, as the functions can be called many times.

    bus.remove_signal_watch()

Finally, we will set the pipeline’s state to Gst.State.NULL using the same function set_state() from before. It is 

important to set the pipeline to the Gst.State.NULL state, because the pipeline then automatically flushes the bus 

to ensure no circular references exist.

    pipeline.set_state(Gst.State.NULL)

Source Code

A github gist of the entire program can be found below. The code comes courtesy of our professor Brady James Garvin.

https://gist.github.com/wblazer/c4e931418f24133ab5a10977084af74d