Inspired by the popular Pokemon Go and Snapchat filters, the Web giant is pivoting toward the far horizon of augmented reality.
Social networking services giant Facebook has taken its first steps toward augmented reality with a new platform focused on smartphone cameras. At its annual developer conference, the company debuted Facebook Spaces, its first social networking experience for virtual reality.
Inspired by the popular Pokemon Go and Snapchat filters, the Web giant is pivoting toward the far horizon of augmented reality. Its move comes two years after it spent $2 billion on start-up Oculus, which launched its VR headset last year. Both initiatives are part of a deepening commitment to machine learning and computer vision as core technologies driving new products and features.
“I used to think glasses would be first mainstream product for AR and we’d get the form factor we wanted in 5-10 years,” said Facebook chief executive Mark Zuckerberg in an opening keynote. “But we are starting to see on the smartphone…the beginning of a new platform…today we’ll start building together the camera as the first mainstream AR platform,” he said.
Specifically, Facebook announced a closed beta for an AR platform using smartphone cameras called AR studio. It supports 3D imaging, real-time face tracking and provides a scripting API for developers. Beta users include games giant Electronic Arts and the Manchester United soccer team.
Facebook will use AR to create more vivid ways to share experiences, including virtual games and post-it notes. It relies on machine learning for services such as object recognition and 3D mapping.
“It will take a while for this to develop…[but] over time this will be a real important technology that changes how we use phones and eventually all technology,” Zuckerberg said.
Figure 1: Zuckerberg pitched AR as a way to connect people. (Source: EE Times)
Qualcomm and Nvidia announced support for Caffe2, Facebook’s open source deep learning framework. Qualcomm also is working with the Web giant on the framework it uses for its Snapdragon neural network engine. The mobile chip vendor has long supported AR as the ultimate goal of virtual reality, although it sold off its Vuforia AR software platform in 2015. Separately, Schroepfer showed Santa Cruz, a self-contained Oculus VR headset, not linked to a PC or smartphone. It uses four cameras to do 3D mapping as a user moves through a scene.
Facebook’s CTO, Mike Schroepfer, said the web giant aims to lead the revolution in computer vision that started in 2012 with milestone accuracy results in the ImageNet competition. It released Mask R-CNN, its latest contribution to the field last month, an algorithm that helps run style transfers at 30 frames/second at 720-progressive resolution n smartphones today.
Separately, Schroepfer showed Santa Cruz, a self-contained Oculus VR headset not linked to a PC or smartphone. It uses four cameras to do 3D mapping as a user moves through a scene.
Facebook Spaces, the company’s first VR environment hosted on its website, lets users customise their avatar and upload their own 360-degree images and videos. The beta creates environments tailored for the Oculus headset that can also be viewed on OCs and smartphones. It uses Facebook Messenger to make VR video calls.
Messenger now hosts 1.2 billion users a month. It also integrates with music services and has businesses including American Express, Hyatt, PayPal and Subway building services with it. A new machine-learning service will help small business users automate replies to queries based on data on their websites.
First published by EE Times U.S.