Multi Language Audio with the Live API

This topic explains how create a live job with multiple audio tracks in different languages.

Introduction

This feature allows ingesting multiple audio languages, descriptive or ambient audio experience tracks to be available for live playback. You can follow the steps here using the Live API. You can also use Video Cloud Studio. For details, see the Multi Language Audio with the Live Module document.

Requirements

There are a few requirements for multiple language audio tracks.

Enable your account

  • Contact customer support to enable the Live API with the multi-language audio feature
  • Make sure that you have an API key for the Live API (provided to you when the account is set up)

Setup your encoder

  • Use an encoder that supports RTP protocol. Multi-language audio support is limited to rtp or srt protocol.
  • Get the following values from your encoder: For details, refer to your encoder documentation.
    • Define the PID (Packet Identifier) for each audio track in your encoder
    • Define the Video PID asscociated with the video in your encoder

    If you don't know the PID values from your encoder, see the Configure your encoder section.

Create your Live Job

In order to add multiple audio tracks to your job, a list of audio tracks must be provided in the request body when creating a Live Job:

{
  "live_stream": true,
  "region": "us-west-2",
  "protocol": "rtp",
  "cidr_whitelist": [ /* omitted… */ ],
  "outputs": [ /* omitted… */ ],
  "alternate_audio": {
    "tracks": [ { Track Details } ]
  }
}

Supported fields for the Track object

The table below contains a full description of supported fields in the track object.

Track Fields
Field Type Required Description
language string Yes

The code for the language to be used; at present this is flexible and could be in any of the various language formats required/supported by the specs, specifically RFC5456/BCP47 which also covers ISO-639 as per the requirements

HLS > LANGUAGE field

DASH > AdaptationSet:lang field
video_pid integer Yes The Packet Identifier (PID) from the MPEG-TS input stream for the video track
pid integer Yes The Packet Identifier (PID) from the MPEG-TS input stream for a specific audio track
default boolean Yes (for the default track) The audio track marked as DEFAULT in the EXT-X-MEDIA as well as the track to be muxed into video stream

If not present, the first track is the default.

If multiple playlists are defined with different defaults, the top level default track is the muxed one.
label string A text description to be used for the track, preferred to be used by the player where possible; used in: HLS > NAME field

Default: the language with an ID for each track e.g. en-0 if not specified
variant enum Maps to the DASH Role Schema values:
  • main
  • alternate
  • commentary
  • supplementary
  • dub

HLS > CHARACTERISTICS

DASH > Role urn:mpeg:dash:role:2011.

Default: main
streams [] array

This mimics the streams field of the playlist output type; example: { “source”: “720p” } - matches the label of the output source

If not present, uses the default audio settings.

HLS > GROUP-ID. (Object)

Example

This example body will create a Job with 2 audio tracks.

Request

POST https://api.bcovlive.io/v1/jobs
  Content-Type: application/json
  X-API-KEY: your_API_key

Request body

{
  "live_stream": true,
  "region": "us-west-2",
  "protocol": "rtp",
  "cidr_whitelist": [
    "0.0.0.0/0"
  ],
  "outputs": [
    {
      "label": "hls720p",
      "live_stream": true,
      "height": 720,
      "video_bitrate": 2400,
      "segment_seconds": 6,
      "keyframe_interval": 90
    },
    {
      "label": "hls540p",
      "live_stream": true,
      "height": 540,
      "video_bitrate": 1200,
      "segment_seconds": 6,
      "keyframe_interval": 90
    }
  ],
  "alternate_audio": {
    "tracks": [
      {
        "label": "English",
        "language": "en",
        "variant": "main",
        "video_pid": 256,
        "pid": 257,
        "default": true
      },
      {
        "label": "Spanish",
        "language": "es",
        "variant": "main",
        "video_pid": 256,
        "pid": 258
      }
    ]
  }
}

Sample response

 
{
  "id": "76f814fbcd7840e99ebf0e335c933730",
  "outputs": [
      {
          "id": "0-76f814fbcd7840e99ebf0e335c933730",
          "playback_url": "https://playback-qa.a-live.io/76f814fbcd7840e99ebf0e335c933730/us-west-2/NA/profile_0/chunklist.m3u8",
          "playback_url_dvr": "https://playback-qa.a-live.io/76f814fbcd7840e99ebf0e335c933730/us-west-2/NA/profile_0/chunklist_dvr.m3u8",
          "playback_url_vod": "https://playback-qa.a-live.io/76f814fbcd7840e99ebf0e335c933730/us-west-2/NA/profile_0/chunklist_vod.m3u8",
          "playback_added_cdns": [],
          "label": "hls720p"
      },
      {
          "id": "1-76f814fbcd7840e99ebf0e335c933730",
          "playback_url": "https://playback-qa.a-live.io/76f814fbcd7840e99ebf0e335c933730/us-west-2/NA/profile_1/chunklist.m3u8",
          "playback_url_dvr": "https://playback-qa.a-live.io/76f814fbcd7840e99ebf0e335c933730/us-west-2/NA/profile_1/chunklist_dvr.m3u8",
          "playback_url_vod": "https://playback-qa.a-live.io/76f814fbcd7840e99ebf0e335c933730/us-west-2/NA/profile_1/chunklist_vod.m3u8",
          "playback_added_cdns": [],
          "label": "hls540p"
      },
      {
          "id": "2-76f814fbcd7840e99ebf0e335c933730",
          "playlist_type": "defaultS3",
          "type": "playlist",
          "alternate_audio": {
              "tracks": [
                  {
                      "label": "English",
                      "language": "en",
                      "name": "Alt0",
                      
                      "pid": 257,
                      "playlistDefault": true,
                      "default": true,
                      "variant": "main",
                      "profile_sources": [
                          "profile_0"
                      ]
                  },
                  {
                      "label": "Spanish",
                      "language": "es",
                      "name": "Alt1",
                      
                      "pid": 258,
                      "playlistDefault": false,
                      "default": false,
                      "variant": "main",
                      "profile_sources": [
                          "profile_0"
                      ]
                  }
              ]
          },
          "filename": "playlist.m3u8",
          "dvr_filename": "playlist_dvr.m3u8",
          "playback_url": "https://playback-qa.a-live.io/76f814fbcd7840e99ebf0e335c933730/us-west-2/NA/playlist.m3u8",
          "playback_url_dvr": "https://playback-qa.a-live.io/76f814fbcd7840e99ebf0e335c933730/us-west-2/NA/playlist_dvr.m3u8",
          "playback_added_cdns": []
      }
  ],
  "stream_url": "rtp://ep3-usw2.a-live.io:11780",
  "stream_name": "76f814fbcd7840e99ebf0e335c933730.stream",
  "static": false,
  "alternate_audio": {
      "tracks": [
          {
              "label": "English",
              "language": "en",
              "pid": 257,
              "default": true,
              "variant": "main",
              "name": "Alt0"
          },
          {
              "label": "Spanish",
              "language": "es",
              "pid": 258,
              "default": false,
              "variant": "main",
              "name": "Alt1"
          }
      ]
  },
  "event_length": 93600,
  "encryption": {},
  "playback_url": "https://playback-qa.a-live.io/76f814fbcd7840e99ebf0e335c933730/us-west-2/NA/playlist.m3u8",
  "playback_url_dvr": "https://playback-qa.a-live.io/76f814fbcd7840e99ebf0e335c933730/us-west-2/NA/playlist_dvr.m3u8",
  "playback_added_cdns": []
}

Configure your encoder

Configure your encoder to deliver MPEG-TS over RTP to the stream_url from the create response. Make sure that the audio tracks provided in the request body are configured with the correct PID in your encoder.

Example with ffprobe

If you don't know the PIDs from your encoder, you can use a tool such as ffprobe (included with ffmpeg) to inspect the PIDs in the stream.

ffprobe /tmp/ma.ts
ffprobe version 5.0.1 Copyright (c) 2007-2022 the FFmpeg developers
  built with Apple clang version 13.0.0 (clang-1300.0.29.3)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/5.0.1-with-options_1 --enable-shared --cc=clang --host-cflags= --host-ldflags= --enable-gpl --enable-libaom --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-libsnappy --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-demuxer=dash --enable-opencl --enable-audiotoolbox --enable-videotoolbox --disable-htmlpages --enable-libvmaf --enable-version3
  libavutil      57. 17.100 / 57. 17.100
  libavcodec     59. 18.100 / 59. 18.100
  libavformat    59. 16.100 / 59. 16.100
  libavdevice    59.  4.100 / 59.  4.100
  libavfilter     8. 24.100 /  8. 24.100
  libswscale      6.  4.100 /  6.  4.100
  libswresample   4.  3.100 /  4.  3.100
  libpostproc    56.  3.100 / 56.  3.100
Input #0, mpegts, from '/tmp/ma.ts':
  Duration: 00:00:04.86, start: 1.400000, bitrate: 502 kb/s
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0xff]: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p(progressive), 360x240 [SAR 1:1 DAR 3:2], 30 fps, 30 tbr, 90k tbn
  Stream #0:1[0x100]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, mono, fltp, 98 kb/s
  Stream #0:2[0x102]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, mono, fltp, 98 kb/s

The hex number after the stream will indicate the PID. For example:

  • 0xff == 255
  • 0x100 == 256
  • 0x101 == 257

Test Playback

Load the playback_url into the Brightcove Player or Safari to confirm playback.

DRM Playback

Currently, it is not possible to create DRM live jobs with multi-language audio tracks from Live Module UI.

Limitations

  • Audio tracks can only be defined for a new live job. Currently, it is not possible to update the audio tracks for existing live events.