Meta CEO envisions a virtual world where language, creation are effortless

Meta, the newly formed parent company of Facebook, is banking on AI and new virtual world Metaverse to fuel its next leg of growth as the social network’s business matures.

This week, the company gave users a glimpse of projects it is building in the Metaverse.

Speak and it shall appear

In a video, CEO Mark Zuckerberg revealed plans for new ‘generative AI’ models that would let users describe a world and then generate aspects of it.

 He showcased Builder Bot, where, as a legless 3D avatar, he and a friend created a beach and blue skies just by speaking commands. “Hey builder bot,” Zuckerberg said. “Let’s go to the beach.” A virtual beach appears that is all generated by AI, he added.

Digital assistants that actually understand you

Another area of AI research is Project CAIRaoke, which aims to make digital assistants better when it comes to conversational AI. Digital assistants such as Siri and Alexa, though vastly improved over the years, still can get flummoxed when people speak with some complexity.

Currently, digital assistants rely on four separate components – natural language understanding, dialog state tracking, dialog policy management and natural language generation. They must be linked together to work, which can lead to sub-optimal output. Meta’s version unifies these components, which it said improves on performance.

Describing it as a “fully end-to-end neural model for building on-device assistants,” Zuckerberg said it was using the model within its video-calling Portal devices with plans to eventually integrate it into AR and VR devices.

Figure 1: Source: Meta

Instant translation service -- even if it's Asturian to Urdu

Zuckerberg also announced that his company was working on a universal speech translator in a bid to offer instant speech-to-speech translation across all languages.

While there are several digital tools for translating the most widely spoken languages, not much exist for lesser known tongues that do not have a large training data set for AI. Meta is aiming to change that so Amharic can be translated to Kazakh, for example.

Most machine translation systems work this way: speech is converted to text, then converted to text in the target language, and finally into speech. This is not efficient and excludes languages that are mainly oral, according to Meta. The company is working on direct speech-to-speech translation.

Life after Facebook

Last year, the company rocked the tech world with plans to refocus its business on futuristic concepts such as augmented reality (AR) and virtual reality (VR), resulting in the Metaverse. Meta has opted to spend big on infrastructure to achieve its dream – including plans to build one of the world’s fastest AI supercomputers, the AI Research SuperCluster (RSC).

The pivot comes after Meta reported a quarter-over-quarter decline in daily active users for the first time, tanking shares by 26% and wiping out more than $230 billion in market valuation in one day. Meanwhile, its longest serving director, billionaire Peter Thiel, also said he was leaving the board.