Ingesting Videos with Multi-Language Audio Tracks

Introduction

Language audio tracks in your source videos can be preserved in your Video Cloud video renditions. Ingestion of multi-language audio tracks is supported through the Dynamic Ingest API only,

For general information on how to ingest videos using the API, please see the Overview of the Dynamic Ingest API.

Notes and Limitations

You must input only ISO 639-2 three letter codes, using only Terminology and not Bibliography codes.
When no supplied language codes are matched with the source audio, we will process the default language and tag it with supplied language code
If you do specify a Bibliography language code and it matches a source track, we will attach the language code to that track, but you should always use Terminology language codes.
When multiple tracks are specified as default we will use the first one found in the source (if any) or the first one ingested as the default
If you specify a non-English track as default, but no language is supplied for the track, we will tag it as "eng".
All the audio tracks must have the same number of renditions.
There is limit of 100 jobs that can be in the processing queue (unless you specify Low Priority processing). If you are adding a large number of audio tracks, you may hit this limit, and requests will fail until the processing queue falls below 100. See Fallback/Retry Strategy for more details.
Not all audio service types are supported: only the following types may be used:
- main
- alternate
- commentary
- dub
- descriptive
When you add eng > main and eng > commentary tracks during ingestion and the source contains eng with no audio service type, the source audio will be missing in the output renditions.
If you include two tracks with the same language code, and specify one as variant and provide no service type for the other, the ingest will fail.
When a video with a single audio track is replaced by a video with multiple language audio tracks, the track for the original single language will also be included in the renditions, with its original language identifier.
Both multiple audio tracks within the source video and additional separate audio files can be used for multiple audio tracks. Video Cloud does not store audio masters, however - separate audio files used as alternate audio tracks must be hosted on public URLs.
By default, external audio tracks will be merged with those contained in the video. If instead you want to replace the audio tracks contained in the video, you can set the merge_with_existing flag to false.
Ingest will not fail the job for any master track that is not part of the media info and that track will be skipped.
For the master audio tracks input, any one of the languages or the variants can be passed in the master.audio_tracks object as well. In that case we will look for that in media info and skip it if not present.
Where the language or variant for a track is not specified in the request, we will default to eng for the language, and main for the variant.

Using the Dynamic Ingest API

There are two steps to ingesting a video using the Dynamic Ingest API

Create a video object in Video Cloud using the CMS API
Ingest the video using the Dynamic Ingest API

Create the video object

To create a new video object in your Video Cloud video library, make POST request to:

https://cms.api.brightcove.com/v1/accounts/{account_id}/videos

You must include a request body that contains at least the video name:

{
"name": "My Video Title"
}

You can also include additional video metadata (see the CMS API Reference), but these can also be added later in Studio or via the CMS API.

The response will be a JSON object containing many video properties. The first one will be the video id, and you will need this value to make the ingest request.

Ingest the video

After you create the video object and retrieve its id, you can ingest the video to Video Cloud by sending a POST request to:

https://ingest.api.brightcove.com/v1/accounts/{account_id}/videos/{video_id}/ingest-requests

Request body

First let's look at a simple example of a video that contains three audio tracks. The request body will look something like this:

{
  "master": {
    "url": "{source_url}",
    "audio_tracks": [
      {
        "language": "eng",
        "variant": "main"
      },
      {
        "language": "spa",
        "variant": "main"
      }
    ]
  },
  "profile": "multi-platform-extended-static-with-mp4"
}

Now let's look at a more complex example, where the video contains multiple audio tracks, and we are adding some additional audio tracks that are in separate files:

{
  "master": {
    "url": "{source_url}",
    "audio_tracks": [
      {
        "language": "eng",
        "variant": "main"
      },
      {
        "language": "fra",
        "variant": "main"
      },
      {
        "language": "spa",
        "variant": "main"
      }
    ]
  },
  "profile": "Multiplatform Extended with MP4",
  "audio_tracks": 
    {
      "merge_with_existing": true,
      "masters": [
        {
          "url": "https://some.site.com/audio/alternate-en.m4a",
          "language": "eng",
          "variant": "alternate"
        },
        {
          "url": "https://some.site.com/audio/commentary-es.m4a",
          "language": "spa",
          "variant": "commentary"
        }
    }
  ]
}

In this example:

The master > url is the location of the source video file
The master > audio_tracks provide information about the audio tracks embedded in the video
Note that the language codes should come from https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry
The profile is the name of the ingest profile to use - if you omit this, the default profile for the account will be used
The additional audio_tracks array contains information about separate, externally hosted audio files to be used as additional audio tracks
merge_with_existing means "use the remote audio tracks in addition to the embedded ones" if true (or replace the embedded tracks with the remote ones if false)
The variant field describes the kind of audio track. The possible values, with standard meanings are:
- main - the main track, usually the one muxed into the video file
- alternate - an alternate audio track
- commentary - an audio track that provides commentary on the video track
- dub - a track containing a dubbed version of spoken words in a different language
- descriptive - track is descriptive of the video content in some way
Note that the variant field is optional.
Known limitation:

If the actual audience tracks don't have variant in their own metadata, do not use it in the request. If it is present in the request, but not in the audio track metadata, the track will be skipped. When you are not sure, first try to ingest the file without variant as the system might be able to identify the tracks with the language-code you provide.

Additional notes:

There are additional (all optional) fields for the ingest request, such as adding images and text tracks. See the Dynamic Ingest API Reference for full details.
You do not have to ingest all video assets at once. For instance, in the example above, you could just ingest the video with its embedded audio tracks first, and then submit a separate request to add the additional remote audio tracks.

Multiple audio tracks in the Brightcove Player

When you embed a video with multiple audio tracks in a Brightcove Player,the tracks will be available from a selector in the player control bar:

Setting the default audio track

You cannot set the default audio track during ingestion (the first audio track encountered in processing will be set as the default). You can change the default track either in Studio or using the CMS API, however.

In Studio

For instructions on setting the default audio track in Studio, see Setting the default audio track.

Using the CMS API

For instructions on setting the default audio track using the CMS API, see Updating audio tracks.

Audio service type

Dolby audio codecs (ac-3 and eac-3) support including audio service type metadata. (AAC audio cannot contain service type metadata.)

The following service types can be specified for supported Dolby tracks:

main
music_and_effects
visually_impaired
hearing_impaired
dialogue
commentary
emergency
voice_over