How AI stops mobile videos from flopping when shoppers don’t flip

FacebooktwitterredditpinterestlinkedinmailFacebooktwitterredditpinterestlinkedinmail

By Amit Sharon, VP customer success, Cloudinary

If a picture is worth a thousand words, a video is worth even more. In fact, nearly 90 percent of consumers rely on videos to help them decide what to buy. Landing pages that include video get high conversion rates of at least 80 percent. It’s little wonder that retailers are increasingly using video content to engage customers with their brand and boost sales. 

Nowadays, most of these videos are consumed on mobiles. According to Cisco’s Visual Networking Index, mobile video traffic accounted for 59 percent of total mobile data traffic in 2017. Estimates are that by 2022 nearly four-fifths (79 percent) of the world’s mobile data traffic will be video. 

Retailers are increasingly using video content to engage customers with their brand and boost sales

Video for the ‘horizontally-challenged’

Video, however, turns out to be a lot easier to look at than it is to publish. There are many seemingly minor, but critical details to get right in order to keep viewers engaged. One such detail is adjusting image aspect ratios for different phones, social channels and viewing orientations. Although vertical videos are on the rise, many professionally produced videos still use the horizontal 16:9 aspect ratio. This means that in order for the mobile viewer to get the best user experience, they would need to flip their device. This rarely ever happens.

Studies have shown that smartphone users hold their phones vertically about 94 percent of the time, even when watching 16:9 videos that aren’t mobile-optimised. One simple reason is that today many people use their mobile cases as wallets and don’t want their cards to fall out. Others just aren’t bothered to flip, especially when on the move, even if the user experience is lousy. Even if this poor experience isn’t your fault and outside your influence, it’s certainly not ideal. 

To compound the problem, social media requires publishers to adapt video aspect ratios for many different devices, channels and formats. Instagram and Facebook feed videos are in 1:1 square and 4:5 vertical formats; Instagram and Facebook ‘Stories’ are in 2:3 vertical and 9:16 full portrait/vertical formats, respectively.    

Given all these variables, the seemingly minor detail of optimising video aspect ratios can escalate into quite an overwhelming job for your developers. Wouldn’t it be much easier if videos could automatically detect and play with the correct aspect ratio, regardless of the mobile device, social channel or user behaviour?

AI to the rescue

Retailers are able to deliver their content-aware videos in any aspect ratio they need, whether it’s Landscape, Square, Facebook Vertical, Instagram Story Vertical

Fortunately AI has been successfully applied to a new generation of video management and Digital Asset Management (DAM) solutions to tackle this problem. These use AI and deep learning to analyse video images and automatically detect the ‘must keep’ and subject elements. When the image needs to be resized, it intelligently crops out less important image elements and places key image elements prominently in the frame. As the cropped video plays out, it continues to position the key elements prominently for the duration. Once the initial image analysis has been performed in the DAM or Video Management tool, retailers are able to deliver their content-aware videos in any aspect ratio they need, whether it’s Landscape, Square, Facebook Vertical, Instagram Story Vertical.

No matter how your users decide to hold their devices, you are guaranteed to deliver the best video experience

This functionality can even be extended and serve up a horizontal or vertical video depending on a device’s orientation. If someone lands on your page while browsing in vertical orientation, the AI-augmented tool can set the HTML5 video source to the vertically cropped video. If the user decides to switch to horizontal, source changes to the original landscape video – offering full screen video for both orientations.

This all happens automatically without your developers needing to undertake any time-consuming manual editing. In other words, advanced AI and deep learning algorithms enable retailers to deliver videos in the right size and aspect ratio, for every platform or device and even automate the whole process. 

It also means that no matter how your users decide to hold their devices today or in the future, you are guaranteed to deliver the best video experience. And for retailers this is absolutely imperative in today’s video-saturated world.