Automated Captions using the Live API

Introduction

The automated captions feature allows you to enable automatic audio transcribing to create captions for your live stream.

If you are new to the Live API, review the Create a Live Stream Using the Brightcove Live API document.

Setup

Live automated captions are available to Video Cloud customers.

1. Contact Brightcove customer support to enable your account.
2. Once you have enabled your account, you can turn on automated captions for each live stream that you create.

Turn on automated captions

To create a job with transcribe support for automated captions, add the following attributes to the regular Job creation payload:

{
"transcribe_state": "ON"
}

Example

This example body will create a live job with automated captions.

Request

POST https://api.bcovlive.io/v1/jobs
Content-Type: application/json
X-API-KEY: your_API_key

Request body

{
  "live_stream": true,  
  "region": "us-west-2",
  "reconnect_time": 1800,
  "live_sliding_window_duration": 1,
  "transcribe_state": "ON",
  "outputs": [
      {
          "label": "hls1080p",
          "live_stream": true,
          "width": 1920,
          "height": 1080,
          "video_codec": "h264",
          "h264_profile": "main",
          "video_bitrate": 2400,
          "segment_seconds": 6,
          "keyframe_interval": 60
      },
      {
          "label": "hls720p",
          "live_stream": true,
          "width": 1280,
          "height": 720,
          "video_codec": "h264",
          "h264_profile": "main",
          "video_bitrate": 1843,
          "segment_seconds": 6,
          "keyframe_interval": 60
      },
      {
          "label": "hls480p",
          "live_stream": true,
          "width": 640,
          "height": 360,
          "video_codec": "h264",
          "h264_profile": "main",
          "video_bitrate": 819,
          "segment_seconds": 6,
          "keyframe_interval": 60
      }
  ]
}

Set the language

You can set the language for the generated captions. In this example, the language is set to English, US:

{
  "transcribe_state": "ON",
  "transcribe_options": {
      "language_code": "en-US"
  }
}

Turn on captions mid stream

Enabling automated captions is supported after the job has been created any time independent of the job state.

The endpoint would be:

PATCH https://alive-auth.api.brightcove.com/v2/accounts/{{account_id}}/{{job_region}}/jobs/{{job_id}}/transcribe

  Payload:
  {
    "transcribe_state": "ON"/"OFF"
  }

Limitations

Here is a list of limitations with the live automated captions feature:

• Transcription is only supported for non low latency HLS Streams. Low Latency HLS and DASH streams don't have transcription support at this time.
• You cannot enable Transcribe support for Jobs already created in your account.
• Currently, we provide captions only in one language. We don't support multi language captions at the moment. The Default language for captions is set to en-US
• Automated captions work best for jobs less than 4 hours in duration. If your live stream exceeds 4 hours, captions may drop for a brief interval.
• The accuracy threshold for automated captions is around a 30% word error rate, which can fluctuate.
• Currently, automated captions will not be saved when creating a clip or VOD from the live event.
• The clippable window for live jobs enabled with the Live Manifest API (for auto-captioning) is reduced from 7 days to 24 hours.
• Live Manifest API playlists are always relative to now so a 1HR DVR playlist will only ever show segments from now - 1HR. This means that SEP inactive jobs might display an error on the player depending on the DVR window. The live jobs won't play the last segments for seven days, as seen on live jobs that do not use the Live Manifest API.
• Delays in the underlying transcription service can result in empty WebVTT segments being delivered to players. If this occurs, the recommended workaround is to turn the transcription processing off and back on using the Turn on captions midstream endpoint.