Amazon at ACL: Old standards and new forays

The annual meeting of the Association for Computational Linguistics (ACL) is, according to Google Scholar’s rankings, the preeminent conference on computational linguistics, or the scientific study of language from a computational perspective.

As such, it’s long been a showcase for advances in computational systems that process language, which are vital to several of Amazon’s product offerings. Amazon researchers have 17 papers at this year’s ACL, which will be held virtually, beginning next week.

Many of those papers address topics familiar to readers of this blog:

• Language models, which calculate the probability of particular strings of words and have a wide range of applications in natural-language processing (“Masked language model scoring”, “schuBERT: Optimizing elements of BERT”);
• Natural-language understanding, or determining how to respond to natural-language utterances (“GAN-BERT: Generative adversarial learning for robust text classification with a bunch of labeled examples”, “SeqVAT: Virtual Adversarial Training for Semi-Supervised Sequence Labeling”;
• Dialogue management, or enabling computers to engage in multiple rounds of directed conversation with humans, whether on-screen or through a voice interface (“Recursive template-based frame generation for task-oriented dialog”);
• Question answering, or finding and retrieving answers to questions posed in natural language (“Fluent response generation for conversational question answering”, “Template-based question generation from retrieved sentences for improved unsupervised question answering”, “The cascade Transformer: An application for efficient answer sentence selection”);
• Product discovery, or matching product names to customer queries (“Learning Robust Models for e-Commerce Product Search”, “TXtract: Taxonomy-Aware Knowledge Extraction for Thousands of Product Categories”);
• Machine translation (“Evaluating robustness to input perturbations for Neural Machine Translation”).

But some ACL papers with Amazon coauthors take the road less traveled. Two, for instance, deal with multimodal interactions. One of these is about using natural-language instructions to navigate visual images. The other is about using visual data to aid automatic speech recognition: on a basketball court, for instance, the word “layup” is more common than the word “layoff”, while the opposite might be true in an office.

In a similar vein, applied scientist Colin Lockard and his colleagues show how to use visual relationships between text fields on web pages and visual attributes of the text — such as font size and color — to infer relationships between mentioned entities.

In another paper, Faisal Ladhak, who was a senior applied scientist at Amazon for nearly six years and is now at Columbia University completing his PhD; his advisor, Kathleen McKeown, a professor of computer science at Columbia and an Amazon Scholar; and colleagues propose not just a new solution to a problem but a new problem.

Building on recent advances in summarizing the content of news articles, Ladhak and his colleagues propose a system, trained on synopses from online study guides, that summarizes the content of novel chapters.

In the paper, they demonstrate several modifications of the procedure for summarizing news stories that improve performance on the chapter summary task — a more difficult challenge, since unlike news stories, novel chapters rarely include passages whose express purpose is content summary.

McKeown is also one of the conference’s two keynote speakers. Like other artificial-intelligence disciplines, natural-language processing has been revolutionized by deep learning. Where researchers once spent time trying to discern linguistic patterns that rule-based systems could exploit, they now spend their time developing new neural-network architectures that are suited to linguistic problems.

In her talk, McKeown will examine those cases in which deep learning excels, those in which older methods still have some life in them, and new paradigms that could possibly succeed deep learning.

There are also 11 Amazon researchers on the ACL organizing committee. Senior principal scientist Dilek Hakkani-Tür is the workshops chair, and principal scientist Alessandro Moschitti is a senior area chair, as is Eugene Agichtein, an Amazon Scholar. The eight others are area chairs.

Seven workshops and tutorials at ACL either have Amazon scientists on their organizing committees or will feature work by Amazon scientists.

One of these is the Third Workshop on Fact Extraction and Verification, or FEVER. FEVER is the name of both the workshop and a group of data sets created by researchers at Amazon and the University of Cambridge, which consist of factual assertions and sentences drawn from online resources that either support or refute them.

Associated with past FEVER workshops were two challenges to use the data sets to train machine learning systems on fact verification tasks. Those challenges remain open, and most of the nine papers being presented at the workshop describe work that uses the FEVER data set, which has emerged as a benchmark for fact verification systems. In addition to the papers, the workshop features six invited talks.

Amazon also has close ties to the ACL Workshop on NLP for Conversational AI. Amazon applied scientist Mihail Eric is on the organizing committee, and four other Amazon researchers are on the program committee. Four Amazon papers are on the workshop program — three from Alexa — and Hakkani-Tür is one of the invited speakers.