Harmonic Inc. has been demonstrating how increased workflow automation combined with discrete applications for AI are delivering tangible opportunities to create more streaming TV channels, improve the user experience around sport, and better monetize viewer engagement.
The story begins with FAST channel creation, where one route to success is to ‘recycle’ library content into a linear schedule but do so with minimal human touch. Jean Macher, Senior Director, Global SaaS Solutions at Harmonic, explains that AI tools can be used to create a channel automatically, assuming content in the asset management system is correctly tagged.
“You can decide the theme for a channel and an AI scheduling algorithm can select the content assets that make the schedule.”
Using the case of a sports channel, the editorial team could specify that 20% of the programme time should be about football and 20% related to tennis, and give percentages for other sports, and other parameters to help content selection, and then let the AI make the choices. “You still need some human supervision today, but that mainly relates to analyzing and tagging content [when it is first ingested as an asset]. After that you can create the channel without human touch,” says Macher.
This all assumes a schedule containing only ‘canned’ linear content. Macher stresses that the introduction of live content to a schedule demands more sophistication and oversight, partly to ensure the seamless transition between recorded and live segments. “Any live portion would involve the same cost and effort as any other live channel, although that is very efficient with our cloud playout,” he declares.
Harmonic provides the technology to do the scheduling, transcoding, packaging, ad insertion and playout for FAST and other linear channels and the company has a raft of innovations to help improve the user experience when watching, especially when there is live content involved, with sport the obvious use-case.
One example is the creation of a highlights timeline so viewers joining a sports event late (or indeed those already watching) can easily jump to past action that is now presented as an on-demand asset, while the match is still playing. Macher recently demonstrated a proof-of-concept using the Harmonic VOS 360 SaaS platform (which supports the creation, monetization and delivery of linear TV, live streaming and VOD from public clouds).
In this PoC, the commentary speech from a live soccer match was translated into a text transcript, which in turn was analyzed in real-time using ChatGPT to make decisions on which segments of a game were a highlight. This video was then turned into a highlight clip together with a summary of what the viewer would see. These highlights are then presented on a timeline in the streaming player.
When the viewer returns to a highlight clip, a non-interruptive ad is shown alongside or around the clip, in the form of an L-bar or a ‘double-box’ where the clip appears in a box on one side of the screen and the ad is contained in a box on the other side. These ads are inserted using server-side ad insertion and can be addressable, and they can be sold programmatically.
Macher points out that the same non-interruptive ad formats can be used for the livestream, too, which might avoid the need for a standard interruptive ad break or provides additional advertising inventory. The double-box for example, where the main feed reduces in size to provide the room for the advertising box alongside it, does not unduly influence the viewing experience if the match has been interrupted because of an injury.
“There are low and high action moments in a game, as with baseball. We are using this with baseball [with a customer] in the U.S.,” Macher reveals.
In this scenario, a human makes the decision about when the action is sufficiently dull to allow a non-interruptive ad to appear, whether that is a producer in the gallery or a game-watcher that works for a sports data provider and is present at the venue, ready to press a button (with the latter option resulting in a data feed that the ad scheduling system uses).
Another powerful use of AI, also relating to sport, is commentary cloning, which allows a channel owner to localize content for additional markets, assuming they have the rights to stream there. The main use-case is for clipping. In demonstrations this year, Harmonic has shown a soccer match with the original commentary and then the same piece of action where the commentator’s voice has been cloned and now gives the commentary in a second language.
The original audio is processed and translated before the new voice is created. When showing this, Macher pointed out how the cloned commentary mimics the original voice and preserves the style and tone, including the excited raised voice as a chance [to score, during a football match] is created and then missed.
Voice cloning for multi-market on-demand commentaries is an example of something that was “absolutely not possible before AI”, in the words of Macher.
He notes that AI is not completely new to television, with Harmonic using Machine Learning to boost video compression, in the form of content-aware encoding (CAE) (with its EyeQ product) for ten years. The new applications for AI can be split into two main types, he observes: cognitive services that analyze content and which are well suited for playout workflows like the self-driving FAST channels mentioned above, and GenAI for the creation of additional content to complement the main video feed , as with voice cloning.
Macher notes that running AI processes does carry a cost and the need for a business case. In the case of voice cloning, the compute resource needed for real-time processing is not cheap. “The hope is that when you run calculations involving human labour costs and AI and compute costs, it will show that AI is more efficient and economic. We are getting there, but the use of AI is not a no-brainer today.”
Photo credit: joshua hanson on Unsplash
