Streaming a 360 Degree Videos Effectively with Small Size: Facebook takes up the challenge to do it

Hit Brother

9 years ago

Facebook is preparing for a world in which video is no longer flat.

The social network’s push into virtual reality with its Oculus Rift along with similar initiatives by Microsoft and Google is expected to create a flood of video for the devices. Meanwhile, companies like GoPro are developing cameras that people can use to film themselves in 360 degrees while skydiving and snowboarding.

By Maher Saba, Engineering Director, Video

Starting today, 360-degree videos will begin rolling out in News Feed.

Our goal at Facebook is to connect you with the people and things that matter, every single day. Over time, we’ve seen that people enjoy more immersive content in their News Feeds. This is one reason video has been so successful on Facebook, and we’re excited to take it a step further with 360 video.

To create 360 videos, a special set of cameras is used to record all 360 degrees of a scene simultaneously. This means that when the video is playing on Facebook, you can choose what angle you want to see it from. On the web you can do this by dragging around the video with your cursor, and on mobile devices you do it by dragging with your finger — or even just by turning your device. You’ll be able to hold up your phone and the 360 video will follow you as you turn, looking around, to experience things from all over the world like never before.

360 Video on FacebookPosted by 360 Videos on Facebook on Tuesday, September 22, 2015

Facebook executives spent a significant amount of time discussing 360 video, which lets viewers put themselves into the scene and gives them a view in any direction they choose. This 3-D view is considered to be an improvement over the two-dimensional video widely used today.

Three-D videos are closely related to virtual reality, albeit with slight differences, explained Jay Parikh in an interview with Fortune. Virtual reality is more immersive than 360 video in that people can interact with the content itself, he said. For example, in a virtual reality game like the upcoming Minecraft VR game, users can pick up objects and build towers in a virtual environment that seems to surround them.

Part of the reason the files are so big is because of the way the 360 videos are formatted to accommodate the screens of computers and mobile devices. Think of 360 video as being a traditional rectangular image that is stretched and wrapped around an imaginary sphere, so that people can view it at different angles, Pio explained.

However, this creates some problems with how the video looks because some parts appear distorted as the image stretches to accommodate the sphere. Pio likened this to how land near the North and South Poles appears stretched on a globe and doesn’t accurately reflect its true size.

Additionally, stretching makes file sizes larger than they need to be. An excess of pixels end up having to be streamed.

'); var s = document.createElement('script'); s.type = 'text/javascript'; s.async = true; s.src = 'https://ad.admitad.com/shuffle/289c251618/'+subid_block+'?inject_to='+injectTo; var x = document.getElementsByTagName('script')[0]; x.parentNode.insertBefore(s, x); })();

To shave them off, Facebook instead decided to wrap the video around an imaginary square rather than a sphere. Essentially, each side of the square displays a portion of the 360 video and represents a different viewing perspective. Facebook’s video system then stitches the chunks back together and streams it without the distortion and excess imagery.

Facebook has worked on similar techniques to reduce the size of 360 videos when displayed through virtual reality devices like the Oculus. In this case, the 360 video files must be even bigger because the devices require files to be displayed at higher resolutions than traditional flat screens. This bulk results in file sizes that can lead to buffering, or long loading times, for people with slow Internet connections.

To solve this problem, Facebook developed a way to stream the 360 videos to VR devices that involves cutting the video into chunks. Only images in the viewer’s line of sight are streamed.

While gawking at a 360 video of the Eiffel Tower, for example, you’d want the monument to appear clearly while you may not care about how the buildings to the left look because they only appear in your peripheral vision. However, when you move your head, those buildings would appear sharper.

Facebook essentially cuts the video into 30 sections that represent different viewing perspectives of the scene. Additionally, each section has five different resolutions that Facebook’s servers can stream to viewers depending on their Internet connection.

In effect, Facebook’s system automatically streams the right viewing perspective of the video at its highest quality each time a person turns his or her head with the rest of the video streaming in chunks at lower qualities.