Automated Captions using the Live API

In this topic you will learn how to use the Live API to enable automated captions in your live stream.

Introduction

The automated captions feature allows you to enable automatic audio transcribing to create captions for your live stream.

If you are new to the Live API, review the Create a Live Stream Using the Brightcove Live API document.

Setup

Live automated captions are available to Video Cloud customers.

  1. Contact Brightcove customer support to enable your account.
  2. Once you have enabled your account, you can turn on automated captions for each live stream that you create.

Turn on automated captions

To create a job with transcribe support for automated captions, add the following attributes to the regular Job creation payload:

{
"transcribe_state": "ON"
}

Example

This example body will create a live job with automated captions.

Request

POST https://api.bcovlive.io/v1/jobs
Content-Type: application/json
X-API-KEY: your_API_key

Request body

{
  "live_stream": true,  
  "region": "us-west-2",
  "reconnect_time": 1800,
  "live_sliding_window_duration": 1,
  "transcribe_state": "ON",
  "outputs": [
      {
          "label": "hls1080p",
          "live_stream": true,
          "width": 1920,
          "height": 1080,
          "video_codec": "h264",
          "h264_profile": "main",
          "video_bitrate": 2400,
          "segment_seconds": 6,
          "keyframe_interval": 60
      },
      {
          "label": "hls720p",
          "live_stream": true,
          "width": 1280,
          "height": 720,
          "video_codec": "h264",
          "h264_profile": "main",
          "video_bitrate": 1843,
          "segment_seconds": 6,
          "keyframe_interval": 60
      },
      {
          "label": "hls480p",
          "live_stream": true,
          "width": 640,
          "height": 360,
          "video_codec": "h264",
          "h264_profile": "main",
          "video_bitrate": 819,
          "segment_seconds": 6,
          "keyframe_interval": 60
      }
  ]
}

Set the language

You can set the language for the generated captions. In this example, the language is set to English, US:

{
  "transcribe_state": "ON",
  "transcribe_options": {
      "language_code": "en-US"
  }
}

Turn on captions mid stream

Enabling automated captions is supported after the job has been created any time independent of the job state. New Endpoints have been defined and shall be shared shortly to accomplish the sam.

The endpoint would be:

PATCH https://alive-auth.api.brightcove.com/v2/accounts/{{account_id}}/{{job_region}}/jobs/{{job_id}}/transcribe

  Payload:
  {
    "transcribe_state": "ON"/"OFF"
  }

Limitations

Here is a list of limitations with the live automated captions feature:

  • Transcription is only supported for non low latency HLS Streams. Low Latency HLS and DASH streams don't have transcription support at this time.
  • You cannot enable Transcribe support for Jobs already created in your account.
  • Currently, we provide captions only in one language. We don't support multi language captions at the moment. The Default language for captions is set to en-US
  • Automated captions work best for jobs less than 4 hours in duration. If your live stream exceeds 4 hours, captions may drop for a brief interval.
  • The accuracy threshold for automated captions is around a 30% word error rate, which can fluctuate.
  • Currently, automated captions will not be saved when creating a clip or VOD from the live event.