An opinion piece by the chief product officer of DeepPavlov, which is building a tech stack to give computers human-like comprehension of language

The big idea behind modern Conversational AI is about AI helping humans do various tasks primarily through voice or text, which can range from ordering a product online to planning an entire honeymoon. The tech giants are making vast investments in Conversational AI, but plenty of challenges remain. Even significant investments cannot guarantee a breakthrough.

Open source is the key to faster progress in Conversational AI. We cannot only rely on tech giants to advance the field. While they are capable of creating complex products in-house – deploying more resources into R&D than any lab or startup can muster – their efforts are not enough to unlock the full promise of Conversational AI. Here are reasons why.

1. We need an environment that fosters innovation

Big tech companies stand on the shoulders of other tech giants when it comes to ideas, people, and expertise. In 1988, for example, Apple sued Microsoft for stealing ideas from the Macintosh, even though both used ones from Xerox PARC. Or take the case of Dave Cutler, whom Microsoft hired from DEC to build a new version of OS/2 for IBM PCs. In the end, the product was re-built as Windows NT and has evolved to Windows 11. Innovation demands an environment that encourages the free-flowing of ideas between organizations and experts.

2. Corporations lock innovations within their ecosystems

In the late 2000s, Apple, Google, Amazon, and other corporations started to develop closed ecosystems to maximize revenue from millions of their customers. While continuing to create new products, improving existing products, and seeking to innovate, they have locked up their breakthroughs. This should not be a surprise − tech giants want to control the whole life cycle and future iterations of their products as well as profit from them. But this demand for control flies in the face of a decentralized internet, where protocols like HTTP still power its growth even today.

3. Conversational AI demands mistakes

Conversational AI is one of the most complex areas for both research and product development. It is also one of the fastest. It requires rapid development which, of course, necessitates making mistakes throughout the journey. But publicly held corporations are risk averse and tend to avoid mistakes especially with new products. As a result, we are constantly reinventing the wheel – the same ideas over and over again.

Imagine a company developing an idea behind closed doors and something goes wrong. But no one outside the company knows about it or has access to what went wrong. An example is Microsoft’s Cortana, which enjoyed a massive amount of invested resources. There was talk that indicated Cortana could have had an amazing trajectory. Instead, it failed. The point: Did the larger AI community learn from its failure?

4. Cooperation at the core

If we say corporations do not cooperate in Conversational AI, this will not be true. There are dozens of great and truly massive projects by Amazon, Google, and others. Although we can name lots of joint projects, they are all on the surface level. For example, we know several examples of cooperation in integrating multiple assistants into one device.

However, the real innovation should happen on a much deeper layer, at the core. At the moment, all core technologies are more or less on the same level, and all of them are locked within the big tech companies, so we cannot fully unlock the potential progress in Conversatıonal AI.

But if they cooperate more freely, they can create truly amazing products together. For example, there were many haters when Linux was released as a free, open-source operating system. The former CEO of Microsoft, Steve Ballmer, saw it as a threat to intellectual property and called the technology a cancer in 2001. It took Microsoft more than 10 years to change its attitude; Linux turned out to be a unique phenomenon. The greater the variety of industries that invested in it, the better it became. Now Linux is everywhere except the desktop − from your Android phone and Smart TV to the equipment of NASA.

We believe it is possible to build a ‘Linux' moment in Conversational AI. Open source is the only way to significantly accelerate innovation in and focus on the different prospective application areas. Having a core that is open to everybody and ensuring simple access to it can boost development and bring in new products in different application areas, from personal AI assistants in our home devices to the complex systems in industry.

Unless we cooperate at the real core, we are slowing down the progress in the most promising and fast-developing fields.