Alexa Prize TaskBot Challenge

Three university teams have emerged as top performers in the inaugural Alexa Prize TaskBot Challenge.

Launched in March 2021, the TaskBot Challenge involved ten university teams who competed to develop bots that could assist customers in completing cooking or do-it-yourself home improvement tasks that required multiple steps and decisions.

The teams’ goal: build taskbots that assist customers in multi-step tasks, such as baking a birthday cake or fixing a scratch on a car — and adapt those instructions based on the resources and tools available to the customer. If, for example, a customer ran out of an ingredient halfway through a recipe or didn’t have a specific tool for a DIY project, the taskbot had to adjust the plan and suggest possible solutions.

Customers interacted with the taskbots by saying, “Alexa, let’s work together”. That prompt initiated an interaction with one of ten taskbots. After the interaction ended, customers were asked to rate — on a scale from 1 to 5 — how helpful that taskbot was with the task.

“This challenge was motivated by our north star that Alexa will keep inventing next-generation conversational AI experiences that address our customers’ changing needs,” said Yoelle Maarek, vice president of research and science, Alexa Shopping. “Congratulations to our top-performing teams, and to all of the teams who participated in this inaugural challenge. We were delighted with the high level of engagement from the academic community, and by the advances the teams made against the research directions established for the challenge.”

The TaskBot Challenge is the first conversational AI challenge to incorporate multimodal customer experiences; in addition to receiving verbal instructions, customers were also presented with images or diagrams to guide them through the task.

Success required the teams to address many difficult AI challenges, from knowledge representation and inference, and commonsense and causal reasoning, to language understanding and generation, requiring fusion of multiple AI techniques.

Each of the ten teams participating in the challenge received $250,000 research grants, Alexa-enabled devices, free Amazon Web Services cloud computing services to support their research and development efforts, access to the TaskBot Toolkit, other data resources, and Alexa team support.

Five teams were selected for the finals earlier this year.

GRILLBot, a “multi-modal task-oriented digital assistant to guide users through complex real-world tasks” developed by a team of graduate students at the University of Glasgow, emerged as the winner of the challenge.

A team from the NOVA School of Science and Technology (FCT NOVA)  in Portugal earned second place with their bot, Twiz, and a team from Ohio State University (OSU) earned third-place honors with its TacoBot. The finalists also included QuakerBot from the University of Pennsylvania (UPenn) and GauchoBot from the University of California, Santa Barbara (UCSB).

The top performers earned prize money to be divided among the team members: $500,000 for first place, $100,000 for second, and $50,000 for third.

“The experience of taking the ambitious Alexa Prize TaskBot goals from ideas and research to prototype and launching on a tight timeline was thrilling and humbling,” said Federico Rossetto, a PhD student and one of the GRILL team members.

Another GRILLBot team member said the team realized it was essential to provide a wide variety of images and videos that help illustrate the techniques involved in each task.

“We learned that users learn and enjoy a task more when there are rich image and video elements,” said Iain Mackie, a PhD student. To do that, the students relied on image and video augmentations, applying multimedia elements that could be reusable across different tasks.

“Ultimately, the goal of taskbots is one shared by the research vision from our research lab: to enable people to understand the world and make people’s lives better,” said Carlos Gemmell, GRILLBot’s project leader.

TWIZ, the taskbot developed by the FCT NOVA team, introduced an element of curiosity to the challenge. The bot would mention curious facts related to the tasks being performed, which increased customer engagement.

One of the lessons learned by the team was the limits of some algorithms when faced with human unpredictability.

“Transformer-based algorithms work extremely well in controlled settings, but in conversational settings it is natural for users to ramble until they find what they really want,” noted Rafael Ferreira, a PhD student and FCT NOVA‘s team leader. “This creates many new exploratory behaviors which can lead to conversations that must be fun and engaging.”

OSU TacoBot team members said they learned about the importance of user experience in developing real-life dialogue systems.

“User engagement is a key aspect to consider in developing taskbots. Instead of only focusing on the task, users seem to have an interest in having a chit-chat with the bot from time to time. How to naturally transition between chit-chat and task-oriented turns requires prolonged efforts,” said Huan Sun, an associate computer science and engineering professor at OSU, who served as the team’s faculty advisor.

All 10 teams participating in the competition have published research papers detailing their work.

“The TaskBot Challenge, launched last year, is among three Alexa Prize challenges,” explained Reza Ghanadan, a senior principal research scientist who oversees Amazon’s Alexa Prize competitions. “Our original Alexa Prize competition, SocialBot Grand Challenge, just completed its fourth challenge and we plan to announce the next competition, SGC5, in late June 2022, and earlier this year we introduced the SimBot Challenge, which will soon enter its live interactions phase with ten university teams competing. We are excited about how each of these challenges is contributing to the advancement of human-computer interactions through conversations.”

The inaugural TaskBot Challenge was the first of three that will occur over a three-year period. The second challenge will launch in September 2022. Alexa customers can still interact with the winning taskbots by saying, “Alexa, let’s work together”.