The new Facebook project, called 'Ego4D', is an AI that can see, hear, and remember everything a person does in order to show results that can help with daily tasks . This system can be applied in different scenarios and devices, but it makes great sense in augmented reality glasses. Mainly, because they have cameras with a more human point of view and microphones to be able to obtain all the information.
Until now, artificial intelligence systems have been able to learn day-to-day routines to, for example, show us the fastest route to work. What Facebook intends to accomplish, however, goes much further. The project, still in development, is being carried out together with a group of 13 universities around the world, where more than 700 participants have helped capture 2,200 hours of first-person video while performing daily tasks . This, coupled with a “five benchmark challenges” system, will allow an AI to understand much more complex situations. The challenges shown by Facebook are a clear example of what an AI can do with a more human point of view.
– Episodic memory: What happened when? (for example, “Where have I left my keys?”)
– Forecast: What am I likely to do next? (for example, “Wait, you've already added salt to this recipe”)
– Hand and object manipulation: What am I doing? (for example, “Teach me to play the drums”)
– Audiovisual dailyization: Who said when what? (for example, “What was the main topic during the class?”)
– Social interaction: Who is interacting with whom? (for example, “Help me better hear the person speaking to me in this noisy restaurant”)
Keeping the example of the episodic memory, the Facebook AI will be able to remember the last place where it has seen the keys and notify the owner. This can also be applied to other objects.
An advanced Facebook AI that has little to offer in terms of privacy
Kristen Grauman, AI researcher at Facebook, has highlighted to The Verge that the objective of this artificial intelligence is that it can be used in devices that go beyond portable cameras , such as in domestic auxiliary robots . Of course, privacy is concerned, an important point in which Facebook is not exactly an expert company.
Grauman, in fact, has confirmed to the aforementioned medium that they “expect” that companies that use this technology for commercial devices include privacy measures in their applications. Therefore, the protection by Facebook does not appear to be guaranteed.
