Implementing Multiple Audio Tracks

Introduction

The ability to associate multiple audio tracks with a single title can be useful in a couple of key scenarios:

• Playing back the same video in different languages for a broader reach globally 
• Providing audio with descriptions for the visually challenged

Note that adding audio tracks can also be done in Studio - see Adding Audio Tracks to Videos using the Media Module for more details.

Video formats supported

Multiple audio tracks are supported for HLS V4+ and DASH - with and without DRM protection.

Sample

Below is a simple example of a video with multiple audio tracks.

Setup

The setup for Dynamic Ingest requests is the same, whether you are ingesting a video, images, audio tracks, WebVTT files, or all of these:

Request URL

      https://ingest.api.brightcove.com/v1/accounts/{{account_id}}/videos/{{video_id}}/ingest-requests

Authentication

Authentication requires an access token passed as a Bearer token in an Authorization header:

      Authorization: Bearer {access_token}

To get access tokens, you will need client credentials (see below). For the process of obtaining an access token, see Get Access Tokens.

Note on S3

If your source files will be pulled from a protected S3 bucket, you will need to set a bucket policy to allow Video Cloud to access the files. See Using Dynamic Ingest with S3 for details.

Workflow

There are two primary use cases:

• Ingest a new video with multiple audio tracks
• Add multiple audio tracks to an existing video

We will look at the details of the API requests in a section below.

Audio track metadata

There are some metadata fields attached to audio tracks - some of these are set when you ingest the tracks, while others are created by Video Cloud. Some of these fields are read-only, and others can be updated by you. The metadata fields will be detailed below in the context of the relevant API requests, but two particular ones require some explanation here, as they are crucial in determining how the Brightcove Player will handle the multiple audio tracks.

language

The language field, set for each track during ingestion, specifies the language for the track. This is important if the track is a dub of spoken words in the video. You can use basic codes such as fr or codes with a locale identifier, such as fr-CA. See the ISO Language Code Table.

variant

The variant field describes the kind of audio track. The possible values, with standard meanings are:

• main - the main track, usually the one muxed into the video file
• alternate - an alternate audio track
• commentary - an audio track that provides commentary on the video track
• dub - a track containing a dubbed version of spoken words in a different language
• descriptive - track is descriptive of the video content in some way

Account defaults

You can set account defaults for language and variant to determine what audio track will be treated as the default by the Brightcove Player (the default can also be overridden by updating the track metadata, as we will see in a section below). To set the defaults for your account, Contact Brightcove Support.

Ingest audio tracks

Now we will look at the API calls to ingest audio tracks for the two use cases described earlier.

New video with multiple audio tracks

Create the video object (CMS API)

1. You can add several items of video metadata when you create the video object, but here we will just add the minimum: a name for the video:

         {
             "name": "YOUR_VIDEO_NAME"
         }

2. Submit the JSON above (with the placeholder text replaced by your video name) as the request body for POST request to https://cms.api.brightcove.com/v1/accounts/YOUR_ACCOUNT_ID/videos
3. You will get a lot of video metadata back in the response, but the important piece here is the id (the video id), which you need for the next step.

Ingest the video and audio tracks

Next we will ingest the video and audio tracks (we could add other assets like images and text tracks, but we will keep it simple here). The one thing you may find a little confusing is that audio_tracks appears twice in the JSON for the request body:

• An audio_tracks object within the master object contains metadata for the audio track included in the video file (if any - this is also referred to as the muxed in audio) - this will include metadata only, without a URL for the audio file, since the audio track is already included in the video file. Remember that the encoded audio track can be compressed as Stereo, 5.1, 7.1 as long as it complies with the rules for Audio Formats and, as indicated in the Known Issues, beyond setting a Default Audio, there is no other way to order audio tracks
• A top-level audio_tracks object describing the additional audio tracks you are ingesting - these will include a URL for audio file, as well as other metadata

1. The JSON data to be sent in the request body is as follows:

         {
           "master": {
               "url": "https://support.brightcove.com/test-assets/videos/Great_Blue_Heron.mp4",
               "audio_tracks": [
                   {
                       "language": "en",
                       "variant": "main"
                   }
               ]
           },
           "audio_tracks": {
               "merge_with_existing": true,
               "masters": [
                   {
                       "url": "https://support.brightcove.com/test-assets//audio/celtic_lullaby.m4a",
                       "language": "en",
                       "variant": "alternate"
                   },
                   {
                       "url": "https://support.brightcove.com/test-assets//audio/audio1.m4a",
                       "language": "en",
                       "variant": "commentary"
                   }
               ]
           },
             "profile": "BoltIngestProfile",
             "capture-images": true,
             "callbacks": [
                 "https://solutions.brightcove.com/bcls/di-api/di-callbacks.php"
             ]
         }

2. Send the JSON above, substituting your own URLs for the placeholders, and adjusting the language and variant values, in a POST request to https://ingest.api.brightcove.com/v1/accounts/ACCOUNT_ID/videos/ID/ingest-requests (the ID here is the video id returned from request to create the video object)

Add audio tracks to an existing video

To add additional audio tracks to an existing video, the procedure is the same, except that you do not need to make the request to the CMS API to create the video, because it already exists. And in the request to the Dynamic Ingest API, you will not need to include the URL for the video file or the audio_tracks under master providing the metadata for the muxed in audio. You do, however, want to include the metadata for the muxed in audio track in the existing video. It is also important to remember that the audio track has to be compressed following the rules in Audio Formats. So the JSON for the ingest request will look like this:

      {
        "audio_tracks": {
            "merge_with_existing": true,
            "masters": [
                {
                    "url": "https://support.brightcove.com/test-assets/audio/celtic_lullaby.m4a",
                    "language": "en",
                    "variant": "alternate"
                },
                {
                    "url": "https://support.brightcove.com/test-assets/audio/audio1.m4a",
                    "language": "en",
                    "variant": "commentary"
                }
            ]
        },
          "profile": "BoltIngestProfile",
          "capture-images": true,
          "callbacks": [
              "https://solutions.brightcove.com/bcls/di-api/di-callbacks.php"
          ]
      }

Audio track fields for ingest

Notifications

If you specify one or more callback URLs (as in the sample JSON for the ingest request above), Video Cloud will send a notification for each of the audio tracks you ingest. Notifications will look like this:

      {
        "entity": "default/audio128",
        "entityType": "DYNAMIC_RENDITION",
        "version": "1",
        "action": "CREATE",
        "jobId": "0f703adb-0f17-4a35-8395-21c7fcdd2649",
        "videoId": "5298468208001",
        "dynamicRenditionId": "default/audio128",
        "accountId": "1910141565001",
        "status": "SUCCESS",
        "language" : "en",
        "variant" : "alternate"
      }

To identify notifications for audio tracks, look for the language and variant fields in the notification. The "action": "CREATE" and "status": "SUCCESS" fields indicate that the track was successfully ingested.

Managing audio tracks

Once you have ingested the audio tracks, you can manage them via the CMS API.

Get all audio track metadata for a video

To retrieve the metadata for all audio tracks associated with a video, make a GET request to:

      https://cms.api.brightcove.com/v1/accounts/account_id/videos/video id/audio_tracks

The response will be an array of objects containing metadata for each audio track. See the table of response fields below for details.

      [
        {
          "id": "en_alternate",
          "language": "en",
          "variant": "alternate",
          "duration": 86100,
          "encoding_rates": [
            64000,
            96000,
            127000
          ]
        },
        {
          "id": "en_commentary",
          "language": "en",
          "variant": "commentary",
          "duration": 34203,
          "encoding_rates": [
            10000,
            13000,
            15000
          ]
        },
        {
          "id": "en_main",
          "language": "en",
          "variant": "main",
          "duration": 31488,
          "encoding_rates": [
            62000,
            94000,
            125000
          ]
        }
      ]

You can also view this information in Studio by viewing the video in the Media module:

Get metadata for one audio track

To retrieve the metadata for one audio track associated with a video, make a GET request to:

      https://cms.api.brightcove.com/v1/accounts/account_id/videos/video id/audio_tracks/audio_track_id

The response will be an object containing metadata for each audio track. See the table of response fields below for details.

Updating audio track metadata

You can update any writable metadata field for an audio track by making a PATCH request to:

      https://cms.api.brightcove.com/v1/accounts/account_id/videos/video id/audio_tracks/audio_track_id

In the request body, include the field(s) you want to modify - for example:

      {
          "language": "es",
          "is_default": true
      }

Delete an audio track

To delete an audio track, send a DELETE request to:

      https://cms.api.brightcove.com/v1/accounts/account_id/videos/video id/audio_tracks/audio_track_id

Note that the success response code may be 202 (Accepted) rather than 204 (No content) because the deletion process is asynchronous and may not be completed immediately.

Playback

For information on how the Brightcove web and SDK players handle multiple audio tracks see:

• Multiple Audio Tracks (web player)
• Multiple Audio Tracks with the Native SDKs

Known issues

Audio masters not stored

• Video Cloud will not store audio masters

Audio-only files must be used

Audio tracks must be audio-only files without video tracks

HLSv3, HLS with audio and video in the same segment

• The playback API will not return HLSv3 manifests
• All HLS manifests will include de-muxed video/audio

Smooth Streaming

Smooth Streaming URLs will not be available.

Social Distribution

It is not possible to select which audio track will be used for distribution. The track included in the video source (the muxed in audio) will be used, always.

Studio

• Studio will display information about audio tracks
• To add audio tracks using Studio, see Multiple Audio Tracks

Ordering of audio tracks beyond the one "default" track

• You can pick a default audio track with CMS API per title by setting the is_default field to true
• There is also an account default, which can be set by Support
• This will only affect the "default" track in HLS manifests
• No other ordering is possible

Ingesting more than one audio track from a single source

We only support one audio track per source. Each audio track must be ingested separately.

DRM protection on videos that include only audio

As soon as a video track is added, DRM protection will be enabled.

End-user friendly labels

We do not support custom labels for audio tracks. If you require that, you will need to make the change on the client side via the Player API.

In some instances, switching tracks may cause the Brightcove Player to become unstable

• Track switching before all audio segments have been downloaded
• When the video is played back using the Silverlight plugin (in versions of IE less than 10, or any version of IE in versions of Window less than 8) - multiple audio tracks are not supported in Silverlight.
• If the audio and video are different durations, the player will may stop whenever the shorter one runs out.

Video "duration"

The video duration reported by the catalog/Playback API may be incorrect if the audio tracks have different durations.