A gentle introduction to MACAW: A superior QA model to GPT-3
Hey Siri! How do you have answers to all my questions!?
Hey Google! I keep hearing about this GPT-3…What is it!?
Introduction
How do our AI assistants know the answer to all possible questions we bombard them with? Can we know what the mechanisms behind having answers to our questions are? Can we build something on our own or borrow something built that answers our questions? Can we extend the scope of these models beyond just answering our questions? If you are searching for answers to any of these questions or simply have a thirst for AI knowledge, you’re at the right place!
Question-Answering models is a branch of Natural Language Processing that has gained a lot of attention in the past few years. As the name suggests, QA models extract information from an input corpus to ‘answer’ the ‘questions’ we provide the model. Basically, it is analogous to an automated Reading Comprehension exercise.
One of the very first QA models was implemented by BASEBALL, Standford, which used a ‘rule-based language model’ that used a provided spreadsheet to answer questions about scores, and statistics for baseball leagues. Now, I’d like you to consider whether data today can be organized, labelled or contained in a structured or tabular manner. Business insights, images, audio, videos, academic material, finances, weather etc., are all ‘unstructured data’, and an advanced AI is required to understand the information in order to obtain our answers. This is why you would have heard of new QA technologies such as BERT, GPT3, ELECTRA etc.
Types of QA Models
There are different types of QA models which serve different purposes.
- Domain-based:
- Single-domain: Fine-tuned for answering questions from one single domain.
- Open-domain: More generic questions.
2. Answer based:
- Yes/No: Can verify facts and statements and responds in a yes/no manner.
- Extractive: Extracts answers from the text to respond.
- Generative: Can Independently answer questions. (Complex)
In this article, we will be highlighting Allen AI’s MACAW Model, which single-handedly can perform tasks beyond simple question-answering.
Allen AI
Allen Institute for Artificial Intelligence is a renowned, prestigious research institute for performing groundbreaking research work in several domains of the AI industry- NLP, Computer Vision, AI for Environment and many more in incubation.
What is MACAW?
MACAW stands for Multi-Angle Cquestion AnsWering, a multi-functionality QA model with a scope to extend beyond its training for flexible inputs and outputs. It consists of the following features-
- Generating Questions
- Generating Answers
- MCQ answer options
- Generating potentially relevant context
- Generating Explanation
MACAW was built upon T5 (Text-To-Text Transfer Transformer), am encoder decoder model which transforms the NLP problem statements into a text-to-text format. MACAW comes in different sizes- macaw-11b, macaw-3b, macaw-large and an answer-focused version i.e. macaw-answer-11b
Now, with the overwhelming popularity of GPT-3, Chat GPT and other various intelligent conversational agents, one might overlook MACAW. However, MACAW(11 billion parameters) has outperformed GPT-3(175 billion parameters) by 10% due to it’s high question-answering capabilities.
Comparative examples of MACAW
The comparative results between MACAW and GPT-3 can really spark your interest in learning about each of them further.
For instance, both the models were given the following questions-
- Human Behavior:
“Why is it hard to find good building contractors?”
MACAW: “Many bad ones”
GPT-3: “It’s hard to find good building contractors because they are busy. They are busy because they are good.” - Meta-reasoning:
Here, matters get more interesting; take a look-
“What is an implication of something being colored green?”
MACAW: “it contains chlorophyll”
GPT-3: “ it is a plant”
ii. “What is an implication of something being green?”
MACAW: “it is good for the environment”
GPT-3: “It is a leaf.”
If we want further comparison for MACAW with other models, these snippets from the MACAW’s official documentation will be helpful-
I want to use this!
Since you’re reading this article, I hope it’s safe to assume that you might be acquainted with this blessed platform Huggingface. If not, then a quick summary- Huggingface is a data-science platform that helps us use pre-trained models for our purposes. It is the ultimate open-source for Machine Learning, a transfer-learning concept in its own sense. They have excellent documentation, which I would strongly recommend you to read.
Well, the great news is- You can easily use MACAW by using the Huggingface transformers library!
You can refer to the official documentation link, this great YouTube video I found, or my own simple implementation of Macaw. It is an amazing tool!
Imagine how incredible and convenient it would be for teachers to make multiple question types for tests, without any hassle! Oops, sorry kids.
Conclusion:
While the world is swept off its feet with the successful launch of Chat GPT, many other tools can serve horizontal needs with vertical depth. One such tool is Allen AI’s MACAW, which offers not just QA, but question framing, providing MCQs from the input text, explaining its answers, and even giving context. It is like an advanced question-paper pattern (if you know what I mean). MACAW has superior performance than GPT-3 in the QA domain despite far lesser parameters, leaving us with the thought that GPT-3 has the scope to evolve more, and so can MACAW. The QA models can spread their wings wider and fly higher!
Thank you for reading!
Well, you have reached the end of this article! I hope you found this article informative and gained an insight into QA models and Allen AI’s MACAW. If you liked this article, kindly give it a clap and a comment. I appreciate your feed backs and response.
Thank you for your continued support on my ML journey!
Kindly contact me at hiraninetra22@gmail.com or https://www.linkedin.com/in/netrahirani/ for any queries or opportunities.
