Category: Media

Search inside videos using AWS Media and AI/ML Services

Search inside videos using AWS Media and AI/ML Services

In today’s content-driven world, it’s difficult to search for relevant content without losing productivity. Content always requires metadata to give it context and make it searchable. Tagging is a means to classify content to make it structured, indexed, and—ultimately—useful. Much time is spent on managing content and tagging it.

Manually assigning content tags is possible when you produce little new content or are interested in making content less searchable. However, manual tagging becomes impractical when you are creating lots of new content.

Content gets generated in various formats, such as text, audio, and video. A text-based search engine ranks relevancy based on tags or text inside the content. However, searches on audiovisual (AV) files are based only on the tags on the associated text, and not by what is being said during the playback.

To unfold the true power of AV content, we must use audio dialogue to provide a more granular level of metadata for the content—and make it searchable.

With an AV search solution, you should be able to perform the following tasks:

  1. Tag the dialogue.
  2. Make your AV contents searchable based on the content tagging.
  3. Jump directly to the position in the content where the searched keyword was used.

In this post, I describe how to search within AV files using the following AWS media and AI/ML services:

  • Amazon Transcribe: An automatic speech recognition (ASR) service that makes it easy for you to add speech-to-text capability to your applications.
  • Amazon Elastic Transcoder: Media transcoding in the cloud. It is designed to be a highly scalable, easy-to-use, and cost-effective way for you to convert (or “transcode”) media files from a source format. You can create versions that play back on devices like smartphones, tablets, and PCs.
  • Amazon CloudSearch: A managed service in the AWS Cloud that makes it simple and cost-effective to set up, manage, and scale a search solution for your website or application.

Other AWS Services:

  • AWS Lambda: Lets you run code without provisioning or managing servers. You pay only for the compute time that you consume.
  • Amazon API Gateway: A fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale.
  • Amazon S3: An object storage service that offers industry-leading scalability, data availability, security, and performance.

Solution overview

This solution gets created in two parts:

  • Ingesting the AV file
    • Uploading the MP4 file to an S3 bucket.
    • Transcoding the media file into audio format.
    • Transcribe the audio file into text.
    • Indexing the contents into Amazon CloudSearch.
    • Testing the index in CloudSearch.
  • Searching for content
    • Creating a simple HTML user interface for querying content.
    • Listing the results and rendering them from an S3 bucket using CloudFront.

Prerequisites

To walk through this solution, you need the following AWS resources:

{
    "Version": "2008-10-17",
    "Id": "PolicyForCloudFrontPrivateContent",
    "Statement": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity xxxxxxxxxxx"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::<<bucketname>>/*"
        }
    ]
}

Step 1: Uploading a video to the S3 bucket

Upload a video file in MP4 format into the /inputvideo folder.

Step 2: Transcoding the video into audio format

  1. Create a Lambda function (VideoSearchDemo-Transcoder) with the runtime of your choice.
  2. Associate the Lambda execution role with the function to access the S3 bucket and Amazon CloudWatch Logs.
  3. Add an S3 trigger to the Lambda function on all ObjectCreated events in the /inputvideo folder. ­­For more information, see Event Source Mapping for AWS Services.

Use the following Node.js 6.10 Lambda code to initiate the Elastic Transcoder job and put the transcoded audio file to the /inputaudio folder.

'use strict';
var aws = require("aws-sdk");
var s3 = new aws.S3();

var eltr = new aws.ElasticTranscoder({
 region: "us-east-1"
});

exports.handler = (event, context, callback) => {
    console.log('Received event:', JSON.stringify(event, null, 2));
    
var docClient = new aws.DynamoDB.DocumentClient();
    var bucket = event.Records[0].s3.bucket.name;
    var key = event.Records[0].s3.object.key;

    var pipelineId = "ElasticTranscoder Pipeline ID";
    var audioLocation = "inputaudio";
        
    var newKey = key.split('.')[0];
    var str = newKey.lastIndexOf("/");
    newKey = newKey.substring(str+1);

     var params = {
      PipelineId: pipelineId,
      Input: {
       Key: key,
       FrameRate: "auto",
       Resolution: "auto",
       AspectRatio: "auto",
       Interlaced: "auto",
       Container: "auto"
      },
      Outputs: [
       {
        Key:  audioLocation+'/'+ newKey +".mp3",
        PresetId: "1351620000001-300010"  //mp3 320
       }
      ]
     };
    
     eltr.createJob(params, function(err, data){
      if (err){
       console.log('Received event:Error = ',err);
      } else {
       console.log('Received event:Success =',data);
      }
    });
};

Step 3: Transcribing the audio file into JSON format

Add another S3 trigger to the Lambda function on all ObjectCreated events for the /inputaudio folder, which invokes the transcription job. Fill in values for the inputaudiolocation and outputbucket.

'use strict';
var aws = require('aws-sdk');
var s3 = new aws.S3();
var transcribeservice = new aws.TranscribeService({apiVersion: '2017-10-26'});

exports.handler = (event, context, callback) => {
    console.log('Received event:', JSON.stringify(event, null, 2));
    var bucket = event.Records[0].s3.bucket.name;
    var key = event.Records[0].s3.object.key;    
    var newKey = key.split('.')[0];
    var str = newKey.lastIndexOf("/");
    newKey = newKey.substring(str+1);
    
    var inputaudiolocation = "https://s3.amazonaws.com/<<bucket name>>/<<input file location>>/";
    var mp3URL = inputaudiolocation+newKey+".mp3";
    var outputbucket = "<<bucket name>>";
    var params = {
        LanguageCode: "en-US", /* required */
        Media: { /* required */
          MediaFileUri: mp3URL
        },
        MediaFormat: "mp3", /* required */
        TranscriptionJobName: newKey, /* required */
        MediaSampleRateHertz: 44100,
        OutputBucketName: outputbucket
      };
      transcribeservice.startTranscriptionJob(params, function(err, data){
      if (err){
       console.log('Received event:Error = ',err);
      } else {
       console.log('Received event:Success = ',data);
      }
     });
};

The following is a transcribed JSON file example:

{
   "jobName":"JOB ID",
   "accountId":" AWS account Id",
   "results":{
      "transcripts":[
         {
            "transcript":"In nineteen ninety four, young Wall Street hotshot named Jeff Bezos was at a crossroads in his career"
         }
      ],
      "items":[
         {
            "start_time":"0.04",
            "end_time":"0.17",
            "alternatives":[
               {
                  "confidence":"1.0000",
                  "content":"In"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"0.17",
            "end_time":"0.5",
            "alternatives":[
               {
                  "confidence":"1.0000",
                  "content":"nineteen"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"0.5",
            "end_time":"0.71",
            "alternatives":[
               {
                  "confidence":"1.0000",
                  "content":"ninety"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"0.71",
            "end_time":"1.11",
            "alternatives":[
               {
                  "confidence":"0.7977",
                  "content":"four"
               }
            ],
            "type":"pronunciation"
         },
         {
            "alternatives":[
               {
                  "confidence":null,
                  "content":","
               }
            ],
            "type":"punctuation"
         },
         {
            "start_time":"1.18",
            "end_time":"1.59",
            "alternatives":[
               {
                  "confidence":"0.9891",
                  "content":"young"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"1.59",
            "end_time":"1.78",
            "alternatives":[
               {
                  "confidence":"0.8882",
                  "content":"Wall"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"1.78",
            "end_time":"2.01",
            "alternatives":[
               {
                  "confidence":"0.8725",
                  "content":"Street"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"2.01",
            "end_time":"2.51",
            "alternatives":[
               {
                  "confidence":"0.9756",
                  "content":"hotshot"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"2.51",
            "end_time":"2.75",
            "alternatives":[
               {
                  "confidence":"0.9972",
                  "content":"named"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"2.76",
            "end_time":"3.07",
            "alternatives":[
               {
                  "confidence":"1.0000",
                  "content":"Jeff"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"3.08",
            "end_time":"3.56",
            "alternatives":[
               {
                  "confidence":"1.0000",
                  "content":"Bezos"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"3.57",
            "end_time":"3.75",
            "alternatives":[
               {
                  "confidence":"1.0000",
                  "content":"was"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"3.75",
            "end_time":"3.83",
            "alternatives":[
               {
                  "confidence":"0.9926",
                  "content":"at"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"3.83",
            "end_time":"3.88",
            "alternatives":[
               {
                  "confidence":"1.0000",
                  "content":"a"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"3.88",
            "end_time":"4.53",
            "alternatives":[
               {
                  "confidence":"1.0000",
                  "content":"crossroads"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"4.53",
            "end_time":"4.6",
            "alternatives":[
               {
                  "confidence":"1.0000",
                  "content":"in"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"4.6",
            "end_time":"4.75",
            "alternatives":[
               {
                  "confidence":"1.0000",
                  "content":"his"
               }
            ],
            "type":"pronunciation"
         },
         {
            "start_time":"4.75",
            "end_time":"5.35",
            "alternatives":[
               {
                  "confidence":"1.0000",
                  "content":"career"
               }
            ],
            "type":"pronunciation"
         }
         "status":"COMPLETED"
      }

Step 4: Indexing the contents into Amazon CloudSearch

The outputbucket value from the previous step is the S3 bucket parent folder that parses the JSON output of the transcription job. Use that value to ingests the JSON output into the Amazon ES search collection. Add another S3 trigger for the Lambda function on all ObjectCreated events for that folder.

'use strict';
var aws = require("aws-sdk");
const https = require("https");
var s3 = new aws.S3();

exports.handler = (event, context, callback) => {
    console.log('Received event:', JSON.stringify(event, null, 2));
    var cloudsearchdomain = new aws.CloudSearchDomain({endpoint: '<<Mention CloudSearch End Point>>'});

    var bucket = event.Records[0].s3.bucket.name;
    var key = event.Records[0].s3.object.key;
    var newKey = key.split('.')[0];
    var str = newKey.lastIndexOf("/");
    newKey = newKey.substring(str+1);

    var params = {
        Bucket: bucket,
        Key: key
    }
    
    s3.getObject(params, function (err, data) {
    if (err) {
        console.log(err);
    } else {
        console.log(data.Body.toString()); //this will log data to console
        var body = JSON.parse(data.Body.toString());
        
        var indexerDataStart = '[';
        var indexerData = '';
        var indexerDataEnd = ']';
        var undef;
        var fileEndPoint = "http://<<CloudFront End Point URL>>/inputvideo/"+newKey+".mp4";

        for(var i = 0; i < body.results.items.length; i++) {

            if (body.results.items[i].start_time != undef &&
                body.results.items[i].end_time != undef &&
                body.results.items[i].alternatives[0].confidence != undef &&
                body.results.items[i].alternatives[0].content != undef &&
                body.results.items[i].type != undef &&
                fileEndPoint != undef
            ) {
                if (i !=0){
                    indexerData = indexerData + ',';
                }
                indexerData = indexerData + '{\"type\": \"add\",';
                indexerData = indexerData + '\"id\":\"'+i+'\",';
                indexerData = indexerData + '\"fields\": {';
                    indexerData = indexerData + '\"start_time\":'+'\"'+body.results.items[i].start_time+'\"'+',';
                    indexerData = indexerData + '\"end_time\":'+'\"'+body.results.items[i].end_time+'\"'+',';
                    indexerData = indexerData + '\"confidence\":'+'\"'+body.results.items[i].alternatives[0].confidence+'\"'+',';
                    indexerData = indexerData + '\"content\":'+'\"'+body.results.items[i].alternatives[0].content+'\"'+',';
                    indexerData = indexerData + '\"type\":'+'\"'+body.results.items[i].type+'\"'+',';
                    indexerData = indexerData + '\"url\":'+'\"'+fileEndPoint+'\"';
                indexerData = indexerData + '}}';
            }
          }
        var csparams = {contentType: 'application/json', documents : (indexerDataStart+indexerData+indexerDataEnd) };

        cloudsearchdomain.uploadDocuments(csparams, function(err, data) {
            if(err) {
                console.log('Error uploading documents to cloudsearch', err, err.stack);
            } else {
                console.log("Uploaded Documents to cloud search successfully!");
            }
        });
    }
})
};

Step 5: Testing the index in CloudSearch

In the CloudSearch console, on your domain dashboard, validate that the contents are indexed in the CloudSearch domain. The Searchable documents field should have a finite number.

Run a test search on the CloudSearch domain, using a keyword that you know is in the video. You should see the intended results, as shown in the following test search screenshot:

Step 6: Querying contents with a simple HTML user interface

Now it’s time to search for the content keywords. In API Gateway, create an API to query on the CloudSearch domain using the following Lambda code:

var aws = require('aws-sdk');

var csd = new aws.CloudSearchDomain({
  endpoint: '<<CloudSearch Endpoint>>',
  apiVersion: '2013-01-01'
});

exports.handler = (event, context, callback) => {
    var query;
    if (event.queryStringParameters !== null && event.queryStringParameters !== undefined) {
        if (event.queryStringParameters.query !== undefined &&
            event.queryStringParameters.query !== null &&
            event.queryStringParameters.query !== "") {
            console.log("Received name: " + event.queryStringParameters.query);
            query = event.queryStringParameters.query;
        }
    }
    console.log('Received event: EventName= ', query);

    var params = {
          query: query /* required */
    };

     csd.search(params, function (err, data) {
        if (err) {console.log(err, err.stack);callback(null, null);} // an error occurred
        else     {
            var response = {
                "statusCode": 200,
                "body": JSON.stringify(data['hits']['hit']),
                "headers": { "Access-Control-Allow-Origin": "*", "Content-Type": "application/json" }
            };
            callback(null, response);
        }          // successful response
    });
};

Deploy and test the API in API Gateway. The following screenshot shows an example execution diagram.

Host a static HTML user interface component in S3 with public access under the /static folder. Use the following code to build the HTML file, replacing the value for api_gateway_url with the CloudFront URL.

When you test the page, enter a keyword in the search box and choose Search. The CloudSearch API is called.

<!DOCTYPE html>
<html lang='en'>
  <head>
    <meta charset='utf-8'>
    <meta http-equiv='X-UA-Compatible' content='IE=edge'>
    <meta name='viewport' content='width=device-width, initial-scale=1'>
    <title>CloudSearch - Contents</title>
    <link rel='stylesheet' href='https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css' integrity='sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u' crossorigin='anonymous'>
  </head>
  <body>
    <div class='container'>
        <h1>Video Search - Contents</h1>
        <p>This is video search capability with CloudSearch, Lambda, API Gateway and static web hosting </p>

        <div class="form-group">
          <label for="usr">Search String:</label>
          <input type="text" class="form-control" id="query">
          <button id="search">Search</button>

        </div>

        <div class='table-responsive'>
            <table class='table table-striped' style='display: none'>
                <tr>
                    <th>Content</th>
                    <th>Confidence</th>
                    <th>Start Time</th>
                    <th>Video</th>
                </tr>
            </table>
        </div>
    </div>
    <script src='https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js'></script>
    <script>

      $(search).click(function() {
      $('table.table').empty();
        var query = $( "#query" ).val();
        if (!query) {
          alert('Please enter search string');
          return false;
        }
        var api_gateway_url = 'https://<<Mention API Endpoint>>/prod/?query='+query;
        var rows = [];
        $.get( api_gateway_url, function( data ) {
          if (data.length>0){
            rows.push(`                <tr> \
                                  <th>Content</th> \
                                  <th>Confidence</th> \
                                  <th>Start Time</th> \
                                  <th>Video</th> \
                              </tr> \
                              <hr> `);

            data.forEach(function(item) {
                console.log('your message'+item);
                var start = item['fields']['start_time'];
                var source = item['fields']['url']+"#t="+start;
                rows.push(`<tr> \
                    <td>${item['fields']['content']}</td> \
                    <td>${item['fields']['confidence']}</td> \
                    <td>${item['fields']['start_time']}</td> \
                    <td><video controls id="myVideo" width="320" height="176"><source src=${source} type="video/mp4"></video> </td> \

                </tr>`);
            });
            // show the now filled table and hide the "loading" message
            $('table.table').append(rows.join()).show();
         }

        });

    });
    </script>
  </body>
</html>

Step 6: Listing the resulting videos and rendering them from the S3 location using CloudFront

The results page should look something like the following:

The query response JSON has a link to the CloudFront video and the position in video where the keyword was mentioned. The following HTML video control tag, already included in the static HTML code earlier, lets you invoke the video from the point where the keyword was mentioned.

var source = item['fields']['url']+"#t="+start;
<td><video controls id="myVideo" width="320" height="176"><source src=${source} type="video/mp4"></video> </td> \

Cleanup

To avoid incurring future costs after you’re done with this walkthrough, delete the resources that you created:

For AWS services and resources that invoke your Lambda function directly, first delete the trigger in the service where you originally configured it. For this example, delete the Lambda function trigger in API Gateway. Then, delete the Lambda function in the Lambda console.

Conclusion

This post showed you how to build an AV search solution using various AWS media and AI/ML services such as Transcribe, Elastic Transcoder, CloudSearch, Lambda, S3, and CloudFront.

You can easily integrate the solution with any existing search functionality. Give your viewers the ability to search within AV files and find relevant audio content without losing productivity.

Some representative use cases for this solution could include the following:

  • EdTech content search. The education tech industry publishes the majority of its content in video format for schools, colleges, and competitive exams. Tagging each file is a time-consuming task and may not even be comprehensive. This video search functionality could improve the productivity of both content authors and end users.
  • Customer service and sentiment analysis. The most common use of media analytics is to mine customer sentiment to support marketing and customer service activities. Customer service call recording gives great insights to customer sentiments. This solution can improve the productivity of call center agents by giving them ability to search historic audio contents and learn how to provide a better customer experience.

If you have any comments or questions about this post, please let us know in the comments.

from AWS Media Blog

Broadcasting TV channels using the Veset Nimbus playout platform on AWS

Broadcasting TV channels using the Veset Nimbus playout platform on AWS

Co-Authored by Zavisa Bjelorgrlic, AWS M&E Partner SA, Igor Kroll, CEO and Gatis Gailis, CTO & Founder from Veset. The content and opinions in this post are those of the third-party author and AWS is not responsible for the content or accuracy of this post.

Many video content owners use the Veset Nimbus playout platform to originate linear TV channels from AWS. This solution requires no upfront investment nor long-term commitment. It is a fast and economically viable option to create and manage linear TV channels.

With Veset Nimbus, a channel can be launched, modified, or shut down more quickly than with a traditional broadcasting platform. It can also be delivered to different distribution platforms—both OTT and traditional. This post describes the benefits of using this broadcasting platform.

Overview

You can access Veset Nimbus through a monthly subscription, self-service SaaS platform managed through a web interface. To sign up, you must have an AWS account. You pay a fixed monthly fee per channel to Veset and are charged per actual use for the AWS resources used.

The platform automatically provisions AWS resources, such as Amazon S3, Amazon S3 Glacier, and Amazon EC2 instances, and does not require any additional engineering involvement on your part.

Over 65 linear TV channels operate on the platform. The majority of the channels are in high definition (HD), some are in standard definition (SD), and a few have 4K definition or Ultra HD (UHD). There are 21 customers in 15 countries, including Fashion TV, Ginx TV, XITE, In The Box, ProSiebenSat.1 Puls 4’s Zappn TV, A1 Telekom Austria, Siminn, GospelTruth TV, Levira, and YuppTV.

Channels are distributed through the following methods:

  • Traditional broadcast distribution using a multichannel video programming distributor (MPVD) such as satellite, terrestrial, cable, and IPTV headends.
  • Over the top (OTT) directly to end users, including YouTube, PlayStation Vue, Samsung, FuboTV, and ZAPPN TV.

Solution architecture

Veset Nimbus is a multi-tenant SaaS platform, which is highly available and automatically scaling to sustain the processing load. The system uses microservices running in Docker containers deployed in the Spotinst Ocean cluster in AWS.

The Spotinst platform allows the seamless use of EC2 Spot Instances, On-Demand Instances, and Reserved Instances, leading to significant saving in costs. You can also use the platform to manage automatic scaling for different ingestion loads and to provide resilience by spreading the containers over different Availability Zones.

Veset Nimbus allows you to manage channels on two tiers:

  • An initial “broadcaster-level” tier with one content library stored in an Amazon S3 bucket and multiple channels created from this library.
  • A higher “service provider-level” tier, which can group multiple broadcaster-level organizations.

Media assets (both source files and transcoded versions) can be moved optionally from S3 to Amazon S3 Glacier for long-term archiving using lifecycle policies. The following diagram shows the Veset Nimbus solution architecture in the 1+1 Hot Standby configuration.

The steps to create a channel are as follows:

  • Ingest content: Content is ingested from S3 buckets pre-loaded by content owners or fetched from customer-designated repositories (external S3 bucket, FTP, SFTP, and public clouds). Aspera and Signiant are used for accelerated delivery or large video contribution. The ingest process extracts metadata and makes copies to the library S3 bucket. Lifecycle policies can be activated to save copies of the original and processed inputs to Amazon S3 Glacier vaults as a long-term
  • Input live content: Live stream inputs are connected directly to playout using different modes: stream in UDP/RTP arriving through AWS Elemental MediaConnect or Real Time Messaging Protocol (RTMP) or HTTP live streaming (HLS) streams directly connected to playout. If the feeds contain SCTE-35 markers, then those can be used for live switching or ad insertion.
  • Create a playlist: A playlist is a frame-precise schedule that defines primary (video assets from S3 or live streams) and secondary events (broadcast graphics, SCTE35 markers, EPG data) to be played out in the channel. Veset Nimbus incorporates an add-on module (Veset Altos) for long-term planning and scheduling. Veset Altos manages customer workflows to create daily playlists. You can also use playlists imported from third-party software.
  • Create a channel: The channel is created using content library/live events and the playlist created in the previous steps. Different options can be configured with SD, HD, or UHD definition and different output encoded in MPEG2, AVC, or HEVC.
  • Add redundancy: The channel configuration offers different redundancy levels to match the availability and cost requirements:
    • Basic option with a single playout configuration.
    • 1+1 Warm Backup configuration with a backup playout in standby mode in a different AWS Availability Zone. It’s activated automatically by a monitoring alert, or switched manually during maintenance or software-update periods.
    • 1+1 Hot Backup with two synchronized active playouts running on separate EC2 instances in different Availability Zone
    • (Optional) Cross-Region backup channel configured in a different AWS Region.

Veset Nimbus playout has two deployment options:

  • Full cloud—All components of the system reside in AWS.
  • Hybrid playout—Offers the flexibility to deploy main, backup, or both of the playout channels outside AWS.

Veset recommends using the full cloud deployment whenever possible, due to a higher level of redundancy and agility of cloud architecture. In some cases, however, you might opt for the hybrid playout deployment as a backup to cloud playout.

The channel’s playout components are deployed on EC2 instances or on-premises servers when you use hybrid playout deployment. The configuration depends on launch options for redundancy, resolution, and output profiles.

The content to play out is cached from S3 on an Amazon EBS disk or local storage (in the case of the hybrid playout deployment). Caching improves reliability and prevents delays in playout in case of longer time required to obtain contents from S3. Cached contents are encoded and multiplexed with incoming live streams.

The playout enables the live application of graphic overlays, subtitling and management of up to 8 multiple audio tracks. The module inserts ad breaks with SCTE-35 triggers, and it generates EPG data. For more traditional and regulated linear distribution, playout creates as-run logs and runs a compliance recording for each playout.

For increased redundancy, the channel’s primary and backup playouts can be distributed between different Availability Zones of the AWS Region. Every instance is monitored by Amazon CloudWatch and restarted automatically in case of failure. EC2 instances are selected on the basis of the video quality needed for optimizing the costs of operation. A typical setting uses different instance types with the Windows operating system:

  • SD video: c5.xlarge
  • HD video: c5.2xlarge
  • UHD Video (25 fps), HEVC output: g3.4xlarge
  • UHD (50 fps) AVC output: p3.2xlarge

Instance types are regularly reviewed for performance using CloudWatch to gather AWS and application logs. Instances are also verified against new instances and families made available on AWS to improve, whenever possible, the operating costs.

You deploy the playout automatically using SaltStack. The automatic deployment allows easy reconfiguration of the EC2 instance types to match your resolution and quality needs. It also allows deployment in multiple Availability Zones, simple activation on a different AWS Region, and configuration of all other details like subnets, security groups, network ACL, and CloudWatch monitoring.

In addition to CloudWatch, Veset Nimbus uses internally built monitoring systems to ensure that EC2 instances are available, and that the outputs from each of the playouts are aligned with predefined parameters. You can configure alerts of variable level of sensitivity for incidents, such as black or frozen video, jitter, or no audio.

Distribution

Veset Nimbus has multiple ways to deliver feeds originated by its playout to various designated distribution points:

  • The video stream created is sent in real-time transport protocol (RTP) to AWS Elemental MediaLive for real-time encoding. MediaLive is a broadcast-grade live video-processing service with built-in self-healing and high availability. The output of MediaLive can be sent to a CDN, such as Amazon CloudFront for OTT distribution.If you require delivery to multichannel video programming distributor (MPVD) head-ends, then use AWS Elemental MediaConnect.
  • For just-in-time packaging and where you require MPEG dynamic adaptive streaming over HTTP (DASH) output, insert AWS Elemental MediaPackage in the flow immediately after the output from MediaLive.
  • All Veset Nimbus playouts have Zixi Broadcaster preinstalled and integrated into the management portal. Use Zixi Broadcaster to manage multiple incoming live feeds, create multiple outputs, do live recording, and transport outputs to one or several distribution destinations.

Conclusion

Running the Veset Nimbus playout platform on AWS provides you with an extremely fast and economically viable tool to originate linear TV channels. A channel can be launched, modified, or shut down in minutes—compared to months to set up a traditional broadcasting platform. It can be delivered in various flavors to different distribution platforms—both OTT and traditional. The combination of Veset Nimbus and AWS Managed Services like MediaLive, MediaPackage, and MediaConnect lets you solve distribution and monetization challenges in one seamlessly managed and transparently priced environment.

High uptimes and reliability demand of linear television, combined with increased cost efficiency and flexibility, are addressed through a robust solution architecture. The use of different EC2 instances for different types of video quality allows for optimization of operational costs. Using tools such as Spotinst to leverage a mix of EC2 instances brings further efficiencies and resilience. Furthermore, by running playouts on instances in multiple Availability Zones, or even separate AWS Regions, Veset Nimbus offers a solution with high availability at the levels required in broadcasting.

from AWS Media Blog

AWS Media Services Demo: Live Streaming with Automated Multi-Language Subtitling

AWS Media Services Demo: Live Streaming with Automated Multi-Language Subtitling

Until now, to create captions for a live event, stenographers had to be listening closely to capture the speech to convert it to text. To make it more complex, for live content, this has to be done in as close to real-time as possible. If the captions were needed in additional languages, the text was then sent to a translator, which makes it almost impossible to do for live content. These steps are labour-intensive and expensive. To streamline this process, AWS introduced a Live Streaming with Automated Multi-Language Subtitling solution which simplifies the labor-intensive process for real-time transcription for caption creation, and translation for multi-language subtitling.

Live Streaming with Automated Multi-Language Subtitling allows you to capture, transcribe your content to make it more accessible at a lower cost, and translate those caption to easily regionalize content in languages you wouldn’t have before to make it available across the globe.

Watch the five minute video below to see the Live Streaming With Automated Multi-Language Subtitling solution in action. Once you’ve done that, head over to the Solutions page use the solution out-of-the-box, customize the solution to meet your specific use case, or work with AWS Partner Network (APN) partners to implement an end-to-end subtitling workflow.

 

from AWS Media Blog

In the News: Fox Sports Uses AWS S3 to Enable Remote Production for FIFA Women’s World Cup

In the News: Fox Sports Uses AWS S3 to Enable Remote Production for FIFA Women’s World Cup

From Sports Video Group News, read how Fox Sports is using AWS along with APN partners to power their coverage of the 2019 FIFA Women’s World Cup in Paris. With Amazon S3 at the core, Fox Sports’ Paris-based studio-show works on the live production in conjunction with operations, graphics, and playout based out of Los Angeles.

You can learn more about how they used a similar workflow for the 2018 Russia World Cup in the video below:

from AWS Media Blog

AWS Thinkbox Partners Help take Your Studio to the Cloud

AWS Thinkbox Partners Help take Your Studio to the Cloud

Written by Chris Bober, Sr Partner Manager – Thinkbox


AWS Thinkbox Partners: Autodesk, Chaos Group, Houdini, Isotropix, OTOY, ftrack, NIM, BeBop, Teradici, Sohonet, WekaIO, Qumulo

Today we’re announcing the new AWS Thinkbox partner site which highlights AWS Thinkbox and APN partners that can help accelerate your journey to a hybrid or even a full content production pipeline in the cloud. The partners on this page are uniquely positioned to help your studio meet the demands of rapid and efficient content production by leveraging the power of AWS.

These partners are dedicated to help creative studios focused on computer graphics (CG) or editing content to work in a secure, scalable, and repeatable way that not only helps scale studio infrastructure, but creativity as well. Each partner offers solutions for elastic rendering, virtual workstations, studio management, and asset management to power all aspects of your cloud studio, enabling you to quickly adapt to changing requirements and to scale your studio based on increased or decreased demands. With AWS Thinkbox and our partner solutions you can optimize your costs in ways not possible with dated, monolithic, on-premises infrastructure.

To learn more about our partner solutions begin by exploring https://www.thinkboxsoftware.com/partners.

To learn how AWS Thinkbox is enabling the Studio in the Cloud and how that fits into other Media & Entertainment services at AWS, check out the Content Production section of the Media Value Chain eBook.

from AWS Media Blog

Vroom! NASCAR Puts Fans in the Race with AWS Media Services

Vroom! NASCAR Puts Fans in the Race with AWS Media Services

Just about every weekend between February and November, NASCAR thrills U.S. motorsports enthusiasts with some of the highest-horsepower racing action around. The sights, sounds, smells, and experiences of U.S. stock car racing are unlike any other: race cars reaching nearly 200 miles per hour (322 km/h); engines revving past 9,000 rpm; exceeding three G-forces in the turns. Elite drivers’ skills and emotions—and the crew chief geniuses making late-pit-stop strategy decisions to keep the professional drivers on course to win. Pit crews who move with the speed and precision of professional athletes. And the millions of passionate race fans cheering on their favorite drivers and crews at the track and on an ever-increasing variety of smart devices and mobile phones.

Over the past 70-plus years, NASCAR has become the most popular motorsport in the U.S. Also counting racing enthusiasts abroad among its fan base, NASCAR events are broadcast in more than 185 countries and territories. Before 1979, fans watched race day action on television and had to content themselves with 15- to 30-minute video highlight packages on sports programs from the big U.S. networks. In 1979, the entire Daytona 500 was broadcast live for the first time ever – and post-race fisticuffs attracted passionate fans across the country. Continually finding new ways for an increasingly large audience of fans to engage NASCAR action online and on mobile devices, the racing association introduced a new digital platform in 2015. Once again, NASCAR pulled ahead: in 2018, the racing circuit reported race-day digital content consumption up 30% year-over-year and digital video views increased nearly 50% in 2018. With fans demanding more immersive race experiences than ever, NASCAR is putting viewers inside the race cars. The NASCAR Drive mobile app lets subscribers slip in the race car alongside the driver to experience every curve of the track, easily access real-time race stats and predictions, and even gain deeper insight into NASCAR history.

DAYTONA BEACH, FL – FEBRUARY 21: Denny Hamlin, driver of the #11 FedEx Express Toyota, takes the checkered flag ahead of Martin Truex Jr., driver of the #78 Bass Pro Shops/Tracker Boats Toyota, to win the NASCAR Sprint Cup Series DAYTONA 500 at Daytona International Speedway on February 21, 2016 in Daytona Beach, Florida. (Photo by Chris Trotman/NASCAR via Getty Images)

Second-screen experiences are a critical component of consuming live NASCAR content for race fans. For example, the number of unique viewers live streaming FOX coverage of the Daytona 500 to their laptops, tablets, phones, and connected devices increased by more than 50% from 2018, while both FOX and NBC’s streaming numbers for NASCAR coverage have gone up four years in a row. “We give our fans a lot of ways to consume live NASCAR content, such as NASCAR Drive, a 360-camera feed, and communications between drivers and their crew chiefs. We’re also providing on-demand content 24/7 so that NASCAR is at fans’ fingertips when and where they want it,” said Steve Stum, Vice President of Operations and Technical Production, NASCAR.

NASCAR Digital uses AWS Media Services to power its NASCAR Drive mobile app, and to deliver broadcast-quality content for more than 80 million fans worldwide across a wide range of delivery formats and device platforms. AWS Media Services, including AWS Elemental MediaLive and AWS Elemental MediaStore, help NASCAR provide fans instant access to the driver’s view of the race track during races, augmented by audio and a continually updated leaderboard. To ensure the highest-quality video stream and lowest latency for fans who choose to experience race day on a second screen, NASCAR Digital uses these tools to package, process and store the video for delivery via Amazon CloudFront. NASCAR will also leverage AWS for cloud-based machine learning and artificial intelligence services to help streamline formerly labor-intensive processes, including Amazon Rekognition, Amazon Transcribe, and Amazon SageMaker.  This combination of AWS services helps NASCAR deliver more content, innovate new services more efficiently, and scale without compromising time and capital.

“Moving into the cloud has been our best decision yet, eliminating a lot of the friction and overhead associated with traditional production and delivery,” said Stum. “We look forward to continue using the technology to push the limits of what’s possible for our second-screen viewing experience and beyond.”

——-

Feature Image: DAYTONA BEACH, FL – FEBRUARY 17: William Byron, driver of the #24 Axalta Chevrolet, and Alex Bowman, driver of the #88 Nationwide Chevrolet, lead the field to the green flag to start the Monster Energy NASCAR Cup Series 61st Annual Daytona 500 at Daytona International Speedway on February 17, 2019 in Daytona Beach, Florida. (Photo by Jared C. Tilton/Getty Images)

from AWS Media Blog

Textron Systems Displays New Geospatial Analytic Tool with AWS at the 2019 GEOINT Symposium

Textron Systems Displays New Geospatial Analytic Tool with AWS at the 2019 GEOINT Symposium

Video is often thought of in terms of entertainment, but it also informs decisions across a range of industries, from defense to oil and gas, agriculture, and beyond. Intelligence agencies, for instance, analyze live footage from aircraft to gather and act on timely intel, or in agriculture, footage from an unmanned aerial system (UAS) can be used to evaluate crop yield or fight fires. To support quick and seamless video delivery for various defense, agricultural and industrial applications, solutions provider Textron Systems has released SeeGEO™, which can be configured for integration with cloud-based services from Amazon Web Services’ (AWS). The SeeGEO geospatial analytic platform creates, packages, and delivers live and on-demand aerial footage that gives analysts a bird’s eye view of particular regions of interest.

The variant of SeeGEO on-display at GEOINT is configured for integration with AWS cloud-based services, demonstrating a powerful video management system. After SeeGEO ingests and captures live contribution feeds from aerial camera sensor systems, which are then run through AWS Elemental Live to generate Adaptive Bitrate (ABR) streaming profiles of the content. These ABR-formatted streams are then sent to AWS Elemental Delta as MPEG-DASH, where they are packaged and saved as live-to-video-on-demand assets and stored in Amazon Simple Storage Service (Amazon S3), or local storage. All of the AWS tools can be packaged under the SeeGEO interface, which can run in a browser on desktop, laptop, or mobile devices. Once a video asset is created and formatted as an HEVC file to reduce file storage requirements, users can quickly access the video for review and editing. Further, the solution allows users to overlay video and imagery against an integrated mapping component, extract KLV metadata, and generate advanced geospatial products and reports. A platform-agnostic tool, it can be used with AWS Elemental Live, AWS Elemental Server, and AWS Elemental Delta when on-premises or with cloud services like AWS Elemental MediaLive, AWS Elemental MediaConvert, and AWS Elemental MediaPackage.

“We’re helping people use video in a unique way. Rather than distributing it widely to a massive audience like that of a sporting event, our core audience comprises one analyst or a small team that depends on the video to make mission-critical decisions during or after flight,” shared Daryl Madden, Senior Vice President of Geospatial Systems, Textron Systems. “The applications are endless, and with the ability to integrate with AWS, we can rest assured that whatever the use case, our customers will receive the high-quality video they need to make decisions in the field.”

SeeGEO is a trademark of Overwatch Systems Ltd.

from AWS Media Blog

Kayo Sports Delivers Rich User Experiences with AWS

Kayo Sports Delivers Rich User Experiences with AWS

Kayo Sports offers Australia’s sports fans a massive selection of more than 50 live sports powered by FOX SPORTS Australia, ESPN, and beIN SPORTS. Watch to learn how Kayo Sports uses AWS services for media, compute, storage, and analysis to deliver 30,000 hours of live content and rich interactive user experiences.

from AWS Media Blog

A Conversation with Colleen Camuccio of Dow Jones

A Conversation with Colleen Camuccio of Dow Jones

In our latest “Ahead in the Cloud” post featuring Colleen Camuccio, VP of Program Management at Dow Jones, she explains how the AWS Cloud has helped transform and optimize their customer data platform program, which she directs. Now, she’s looking ahead to more innovation and business value for Dow Jones, leveraging the cloud. As if that wasn’t a big enough job, Colleen starts her day at 5am to make time for health, and her family.

Read the full post here

from AWS Media Blog

Glo Brings High-Quality Yoga Instruction to the Masses with AWS

Glo Brings High-Quality Yoga Instruction to the Masses with AWS

Glo challenges people to pause amidst life’s daily distractions to find a quiet place for personal progress. The subscription-based yoga studio hosts community classes free-of-charge at its Santa Monica-based facility, but what makes Glo unique is its extensive library of on-demand yoga, meditation, and Pilates courses taught by renowned instructors from around the world. The Glo content library exceeds 5,000 classes, all accessible for streaming download and viewing from the Glo website and from iOS or Android apps.

Glo creates on-demand video assets for all its instructional courses, ranging from single classes to four-week programs. This requires scaling video processing from tens to hundreds of hours of content a month. In addition, every video in the Glo content library includes branded, four-to-six second opening bumpers. Ahead of a recent corporate rebrand, Glo recognized the need to re-encode and re-edit each of its video courses to bring all on-demand content current with its new brand.

Given the scope and size of the Glo video archive and the need to scale video processing over time, the company initially considered outsourcing its video encoding needs, but chose instead to overhaul its entire video production pipeline under the guidance of Director of Engineering Jeff Neil and Cloud Architect Nery Orellana. The Glo team reached out to its trusted Amazon Web Services (AWS) partner, Onica, which introduced them to the suite of media services offered by AWS. Glo leaned heavily on cloud services from AWS to streamline its content production processes and support its expanding video library and growing audience.

“We looked at all of the options on the table, but ultimately decided that the project provided the perfect opportunity to retool our entire infrastructure, and AWS played a huge role in that,” said Jeff Neil, Director of Engineering at Glo.

“There’s no way we could have programmatically re-encoded our library in such a quick timeframe without AWS Elemental Media Services,” added Nery Orellana, Cloud Architect at Glo.

To retrofit existing video assets with new bumpers and branding, Glo crafted a workflow in which the team splices content into an L cut, cuts out the old bumper and then splices in new branding using a custom-built framework—a process that makes it easy to dynamically re-encode content when future changes are required. The video encoding and distribution workflow then begins with content residing in Amazon Simple Storage Service (Amazon S3).

Once content is located to Amazon S3, it runs through an AWS Step Functions serverless orchestration service for approval and is transcoded using AWS Elemental MediaConvert, a video processing service for file-based content. The MediaConvert service allows Glo to easily and reliably transcode on-demand content for multiscreen delivery. The service functions independently or as part of AWS Elemental Media Services, a family of cloud services that form the foundation of cloud-based workflows and offer the capabilities needed to create, package, and deliver video.

Amazon Route53 Domain Name System (DNS) service performs geospatial lookup to identify the regions from which members are trying to access content, allowing Glo to more intelligently service geographically dispersed customers by storing content in relevant regional S3 buckets.

“Having AWS power our video workflow has helped us dramatically streamline and operationalize costs,” Orellana noted. “By using reserved transcoding slots, we’re able to scale to accommodate large volumes of content. Looking at a per-minute cost analysis, it’s much more cost efficient for us to use MediaConvert versus other tier-based solutions, which tend to result in unnecessary overage charges.”

AWS Elemental MediaConvert incorporates Quality-Defined Variable Bitrate (QVBR) encoding, another draw for Glo. Since most of its classes are made available for offline viewing, the company recognized the advantages of QVBR for its video processing. “When you’re taking any piece of content and don’t have a great internet connection, great deliverability is a must. QVBR helps us dramatically reduce the payload size of offline content to mobile apps,” Orellana explained. “We can easily shrink the file size so that when users are streaming classes, they’re getting a consistent, high-quality viewing experience without using unnecessary bandwidth.”

As Glo’s business matures, the company has set its sights on augmenting its offering with new capabilities, from live streaming and captioning to support for popular streaming devices. It expects that AWS tools will play a role in accommodating these demands, helping make the Glo video workflow even more intelligent. “MediaConvert and the AWS platform have given us greater flexibility and building blocks for future opportunities. When we want to add on to our service, we can easily make changes, so we don’t have to rebuild the entire workflow from scratch,” Neil concluded.

For more information about Glo, visit: https://www.glo.com/?utm_source=Feed&utm_medium=PR&utm_campaign=Amazon_AWS

from AWS Media Blog