VEProf

At the annual F8 conference, Mark Zuckerberg revealed the coming up of the new platform that supports augmented reality technology. Facebook will bring your camera to a new level, showing you new effects and possibilities. Soon, lots of developers will be able to create special digital effects that will be layered on the real image that your camera is depicting. It will provide a huge range of possibilities for both developers and users, previously available only in digital sphere. As Mark Zuckerberg said, the technology is up to be used and developed, so we will see it progress.

At the beginning, Facebook will present how to utilize these AR effects in videos, images, and live streams from smartphones. During his presentation, he showed how it is possible to add a digital coffee cup to your working table and to add digital sharks that aggressively flow around your morning cereal. So, this way, you can add digital items to brighten your surroundings when you want it. However, Facebook team also works on making it possible to stick certain digital figures in a particular place, so every time you turn on your camera, you will see the digital image you’ve set for this place. It means you can leave a digital post-it note on your fridge, so if the person who lives with you uses a camera, one can see it. It reminds us the same scheme as used in PokemonGo, but Zuckerberg makes it global and more usable.

It is quite an ambitious goal. Basically, it is a complicated task to achieve. The tech behind it is hard, and we haven’t even mentioned the mapping, logistics, and data storage sides of this concept. Facebook must find the real answers on lots of questions. The most important one is, if people really need to turn on their camera to see another reality, do they want it. However, Facebook is great in making us believe we need the thing we see. And as we know, they are already working on the technical aspects of AR implementation. We could see it by Mehanna’s artist filter demo presented.

You have seen the apps by Facebook and others that can transform your selfie into the work of art in Picasso style. Such app works with deep neural networks that are also a type of artificial intelligence that is revolutionary for the world of technology. These networks are supported by the smartphone, so they don’t work in a special data center or somewhere on the Internet. It is an important moment when it comes to the idea of Zuckerberg, as for his AR to work, it should work on the smartphone. If it is powered by the data center somewhere on the web, it takes too long and cannot be called a ‘reality’. There is no sense in it, if the smartphone cannot work with this technology.

Maybe you think about augmented reality through the camera of your phone as something childish or useless, especially in terms of the efforts it takes. However, it has more practical usage and possibilities than you might think.

To apply a digital effect to your smile, for example, the system must identify the smile on your face from the picture the camera provides. That’s why it involves neural networks. Neural networks can be set and taught for certain patterns. For example, neural networks can also track movements, so they apply to our world. Facebook decided to go further and to make effects appear, not only according to the things we see, but also according to the things we say. It is also a job for neural networks. They are working to make all these effects in AR possible.

Most of the technologies need more than a few months to be done, especially those that let you leave a digital object in a certain location so everybody sees it. To make it work, you must do an endless amount of things. To stick a digital object to a physical one, Facebook developers must create their own map of locations and make it possible to share it with other users.

If somebody wants to leave a note in a certain location, there will be a need to record an accurate location, using GPS and to record the geometrical location, making it possible for somebody else with their camera turned on to see this digital object at this place, even if they are there for the first time. Here is a need for a digital map made especially for such an app.

If we look closer, we will face another problem. The Internet has wider limits than the physical possibilities and performance of our smartphones. If Facebook’s AR app is powered by huge data centers and servers, our smartphones’ processing power is way less. Facebook has worked on this problem, and they made corrections that allowed them to make their technology more operable and effective for mobile devices, but this AR system is very complicated, so it will probably need more power and processing from our smartphones. Anyway, the professionals work to optimize it for their users.

Lots of chip manufacturers, including Intel and Qualcomm, are looking for ways to build more powerful mobile processors that can be used for more purposes in the future, as people are spending more time with their smartphones and not with computers. More processing power will enable machine learning systems to operate on our smartphones and increase the learning possibilities of neural networks in two or even three times. If these chip producers find a way to build such processors, the doubtful idea of an AR app will become something usual and another level of fun.

Though the idea might seem raw, the technical side of it is almost planned, so we can only wait and look at what the future brings.

Continue reading about Technologies