Artificial Intelligence (AI) is the powering force which is taking video compression to a whole new level. Since the introduction of video encoding for broadcast, two decades ago, statistical multiplexing, dual pass encoding and more latterly, migration to the cloud have all been technological contributors to pushing the limits of compression to new heights.
The next generation of developments will leverage one of AI’s major strong points - its capacity for visual detection in high detail. This visual capacity of AI is already leading medical advances, particularly in the areas of early cancer recognition.
Harmonic was the first company to develop these technologies for the use of content-aware encoding (CAE), by mimicking the way that the human eye looks at the world. This optimizes encoding according to the content type - basically the areas in a picture where the human eye would focus. An example of this could be a soccer player with the ball, generally, the viewer focuses on the action in the picture.
Successful video compression also needs to achieve the ideal of minimizing bit rate, while maximizing picture quality. The challenge is that compression algorithms impact computing resources and latency during live processing. Machine-learning enhanced algorithms can help to address these problems, by using intelligent motion estimation, to anticipate the amount of capacity needed in the subsequent video frames.
Machine learning algorithms
There are three main categories of machine learning algorithms – supervised, unsupervised and reinforcement learning. Of these three, supervised learning algorithms are the most used in the field of video encoding and compression. Moreover, its predictive capabilities, based on AI’s capacity to process and compare huge amounts of big data, make it particularly suitable for video processing.
Unsupervised learning algorithms bring their contribution to the encoding evolution, with their ability to find clusters of video. This means that it can find similarities in video content, in order to leverage bandwidth economies.
Finally, reinforcement learning algorithms can be used to improve the overall compression algorithm over time. The algorithm can be deployed and then continues to improve, ‘autonomously’, over time, as more and more data (or in this case video) is fed into it.
Looking forward, the use of machine learning algorithms is set to accelerate the pace of evolutions in video compression exponentially - as they reduce both the time and the cost of developments. This means that customized compression algorithms, rather than the generic ones of today, will also become possible. All these aspects will become particularly important as broadcasters and other content providers migrate their business models to the cloud.
--
During NAB 2019, Harmonic will have multiple demonstrations that focus on artificial intelligence with real use cases. Jean-Louis Diascorn will also be giving a presentation on artificial intelligence, “AI Technology Is Changing the Future of Video Compression,” on Monday, April 8, 2019.