Google’s New Meena Chatbot: The Dawn of AI-Assistants is Here

Feb 06,2020 by Anushka Agarwal
Inner banner

Do you remember how amazing you felt when you had your first conversation with Apple’s Siri? Well, Siri has been a treat for quite some years, and people all around the world went giddy with the amazing application. For some time, we also enjoyed Amazon’s Alexa and HP’s Cortana. With all the current AI-bots ravishing customers with their machine yet human-alike skills, chatbot service became more prominent. It was almost time for Google to step into the race, with its Meena chatbot.

Just in time, Google came forth with its human-like chatbot, Meena, on a highly competitive note such that it can speak with you in a near-human manner. Google claims, Meena is the best chatbot in the market with users having the possibility of talking practically anything.

Google’s Chatbot Meena will be the Best Chatbot till Date 

According to the experts at Google, Meena will be the best chatbot in the market for many reasons. Meena has been trained on a whopping 341 gigabytes of public social media chatter, higher than any other AI-platform. Meena chatbot consists of a neural network of about 2.6 billion parameters. As per Google, Meena can talk about almost anything and can even make bad jokes.

As Google said in its post, “We present Meena, a multi-turn open-domain chatbot trained end-to-end on data mined and filtered from public domain social media conversations. This 2.6B parameter neural network is trained to minimize perplexity, an automatic metric that we compare against the human judgment of multi-turn conversation quality. “

Meena is a special chatbot in many ways. It is smart, talks just like humans, is trained with 40 billion words, uses a different model of seq2seq transformer architecture. It is better than any other voice assistant right now.

The Tech World is a Playground for AI-voice Assistants 

AI-powered chatbots have been increasingly adopted by those enterprises, in recent years, looking forward to interactive, productive, and effective conversations, and customer service. On the other hand, chatbots find their usage almost everywhere, in enhancing workplace performance, boosting revenue, and improving the coherence of the employees.

Meena, one of the newest breeds of open-domain chatbots, can engage humans in any conversation, in a manner, that they will not feel that they are talking to a bot. It is, therefore, referred to as a medium to close the gap between chatbots and human performance, by Google experts.

The chatbot Meena was released in Google AI’s recent paper, ‘Towards a human-like open-domain Chatbot’ and has been the talk of the tech town since then. Google talked about several aspects of Meena, one of the major ones being the Sensibleness and Specificity Average. The SSA test scores for human agents us 86% while Meena scored an impressive 76% in the test. This metric was studied by the researches in Google to judge the conversational abilities of the AI-based assistants. 

Sensibleness and Specificity Average

Figure: The following graph shows the Sensibleness and Specificity Average (SSA) of Humans, Meena, Mitsuku, Cleverbot, XiaoIce, and DialoGPT. The Sensibleness and Specificity Average (SSA) states how sensible a specific conversation or response is. The sensibility index of Meena chatbot is the most closet to humans which means that it can have near-human conversations. However, this is not the first time Google has experimented with chatbots & language models. Their first instance was in 2015.

Meena can indulge in human-like conversations 

In the paper, Google elaborated on the fact that most of the AI-based assistants lack the sense and the basic understanding of the conversational tactics followed around the globe. For example, if you ask specific questions, the AI assistants will reply to those questions like ‘I don’t know’ or ‘I don’t know what you are referring to’. On the other hand, Meena chatbot can indulge in such conversations, making them feel more related and generic in comparison with the other assistants.

conversation between meena chatbot and human

Figure 1: The following is an extract of a conversation between Meena and a human. Meena can respond to questions like movie suggestions, crack jokes, and indulge in humorous questions as well. According to Google, this versatility of Meena is what brings human behavior into a chatbot. With Meena being so smart, it is quite evident that it will create a hype in the tech world.

Meena Chatbot is Trained on 341 Gigabytes of Public Social Media Conversations

Although Meena is not available for public demo as of now. Once the model is studied for safety and bias, it will overshadow the other AI-assistants, according to Google. Meena is an open-domain chatbot based on the Transformer seq2seq dataset. It consists of 2.6 billion parameters with a highly multi-turn fashion on 300GB+ text data and social media chatter.

  1. The Training dataset: Meena is special because of its public domain social media conversations. For each conversation it calculates total response data making a pair of the 7 messages. The total data fed to Meena is 40 billion words and 341 GB of text data. 
  2.  Architecture: Evolved Transformer makes Meena a highly advance model architecture. ET is a progressive version of the NAS model or the original Transformer data structure. The ET model carries 1 ET encoder block and 13 ET decoder block. decoder used in Meena
    Figure: The evolved transformer decoder used in Meena. In this, a set of 7 responses or conversations are put into the process through 1 encoder block and 13 Decoder blocks. This is how Meena generates her response to customer queries.
  3. The training:  for 30 days on TPU v3 pod. 
  4. Evaluation metrics: Meena chatbot passes all the new evaluation metrics, like the Sensibleness and Specificity Average (SSA) by Google. The index reviews a conversation and tells if it is good or not based on human judgment.
  5. It’s forte: The training of Meena on a bigger dataset and model size makes it more practical. It also has better architecture and decoding parameters. 
  6. The purpose: Defining the performance of any bot doesn’t relate with one parameter. As per the Google experts, Meena succeeds in achieving the required ‘perplexity’. Perplexity is a commonly used term in NLP, and is highly related to the SSA metric that mirrors the human judgment. Furthermore, decreasing the perplexity of Meena will make it indulge in more human-like conversations. 

Sensibleness and Specificity Average of Humans

Figure: The following graph shows the Sensibleness and Specificity Average (SSA) of Humans, Meena, Mitsuku, Cleverbot, XiaoIce, and DialoGPT along with their perplexity score. Humans and Meena lie in a close range with each other as compared to the other bots.

Google First-ever AI Chatbot, Meena and its Impact on the AI-chatbot World

Google always gets on our nerves when it comes to impressive tech announcements. One of the biggest obstacles faced by any chatbot is to handle the infinite user queries during a conversation. So, when Google said that Meena can talk about anything, it got the tech world intrigued.  

The primary aim of AI-assistants is to get them to complete our tasks, provide accurate information, or provide emotional support for a challenge that we are facing. Meena, amongst the other chatbots, places human-likeliness above all, and this is what makes it a special chatbot. Its open domain makes it a perfect chatbot to talk about anything.

The Chatbot Market Dynamics (2019-2024)

As per the chatbot market, the expectations are it will grow from USD 2.9 billion in 2019 to USD 9.4 billion by 2024, with a CAGR of 29.7%. The major driving forces for this market include 24*7 customer support, instant responses, answers to simple and complex questions, fruitful conversations, complaints registered quickly, amiable customer experience, and friendliness and approachability.

chatbot market expectation growth by 2024

As far as the market growth of chatbots is concerned, the Asia-Pacific region will achieve tremendous upsurge, with more and more companies investing in chatbots and machine learning technologies.

global chatbot market

Figure: There are Five Major regions of the global chatbot market, North America, Asia-Pacific, Middle-East and Africa, and Latin America. North America will hold the largest market share; however, Asia Pacific will witness the maximum growth in the market as it holds more than 50% of the world population.

The end-user vertical of the chatbot market will constitute, BFSI, Healthcare, IT and telecommunications, retail, utilities, government operations, etc.

How does the Future for the Chatbots Look like, now with Meena in it?

Google is not releasing Meena for a public demo since they need to lower the perplexity of neural conversational models of the chatbot through improvements in their algorithms, architectures, data, and computation. Google now wants to concentrate on other key factors like personality and the factuality of the chatbot.  Also, Google wants to review the risks and benefits associated with the launch of the model checkpoint. It might be available for public research in the coming months.

With Meena, one thing that can be postulated is the fact that chatbots are about to get very smart. Aiding in fulfilling business operations, handling major business verticals, indulge in emotional conversations, and detangle humans from loneliness.

Leave a Reply

Notify of