Tensorflow DevSummit 2018; Annotated Transformer; TF-Hub; CS224n reports; Awesome-NLP; Meta-learning; World Models; Neural Baby Talk; Semantic Plausibility

Welcome to the 20th edition of NLP News! Thanks for reading! 🎉

As it's often the visualizations and figures that best describe ideas and stick in our minds, the Paper Picks this time are more visual. Let me know what you think.

Highlights in this edition are: a Youtube Playlist of the Tensorflow DevSummit 2018; tutorials on the Transformer, Pandas Dataframes, text preprocessing, and TF-Hub; CS224n project reports and a curated list of NLP resources; interactive blog posts about meta-learning and World Models; the latest in AI news; and papers about skin-colored emoji, Neural Baby Talk, and semantic plausibility.

---

Tools and implementations

Efficient Neural Architecture Search (ENAS) via Parameter Sharing — github.com

The official implementation of ENAS, a more inexpensive model search approach in Tensorflow. A Pytorch implementation is available here.

Colorless green recurrent networks dream hierarchically — github.com

Code for the corresponding NAACL 2018 paper for probing whether RNNs can learn hierarchical syntactic relations (tl;dr: they can). 

Competitions

The Conversational Intelligence Challenge 2 (ConvAI2) — convai.io

The second edition of the Conversational Intelligence (ConvAI) Challenge at NIPS 2018 provides a dataset for training with more engaging conversations and a simpler evaluation process.

Retro Contest — blog.openai.com

OpenAI announces a contest that tests algorithms on previously unseen video game levels. The contest uses Gym Retro, a new platform integrating classic games into Gym, starting with 30 SEGA Genesis games.

Tutorials

The Annotated Transformer — nlp.seas.harvard.edu

A line-by-line annotated implementation of the Transformer by Alexander Rush (Harvard) using Pytorch. 

A bird's eye view of optimization algorithms

An overview of optimization algorithms such as gradient descent, Newton's method, and Quasi-Newton with some nice, interactive visualizations.

Python Pandas Dataframe Basics

A Jupyter notebook with all of the most important functionality of Dataframes all in one place.

Text Data Preprocessing: A Walkthrough in Python

Data preprocessing is unsexy but can make a big difference. This post is a useful walkthrough of common ways to preprocess text data in Python.

How to build a simple text classifier with TF-Hub — www.tensorflow.org

A short tutorial on how to build a simple text classifier with the new TF-Hub, a platform to reusable resources as pre-trained modules.

Resources

CS224n NLP with Deep Learning 2018 Project Reports

The project reports of this year's Stanford CS224n Natural Language Processing with Deep Learning course. A great resource to get some inspiration about interesting research topics. Most of the reports are on Question Answering with the SQuAD dataset, but applications range from speech synthesis, to semantic parsing, to visual question answering.

Awesome-NLP Curated List of NLP Resources

This GitHub page aggregates many useful resources for NLP. I particularly like the resources for NLP in non-English languages, e.g. Korean, Arabic, or Thai.

Blog posts and articles

April Fool’s pranks written by neural network

Do you need some inspiration for next April 1? Here's a list of April Fool's pranks written by a neural network.

From zero to research : An introduction to Meta-learning

A visual introduction to meta-learning and how it can be applied to Natural Language Processing. 

World Models — worldmodels.github.io

The interactive version of World Models with interactive demos of agents hallucinating dreams of reinforcement learning environments.

Microsoft’s neo-Nazi sexbot was a great lesson for makers of AI assistants — www.technologyreview.com

Yandex’s head of machine intelligence says Microsoft’s Tay showed how important it is to fix AI problems fast.

Industry insights

Earbud Translators: Not Perfect, Still Handy — cacm.acm.org

Even though earbud translators are far from perfect, they are still fairly reliable for travelers looking for very short answers to very short questions.

Emmanuel Macron Q&A: France's President Discusses Artificial Intelligence Strategy — www.wired.com In an interview with WIRED, French President Emmanuel Macron describes his plans to enhance the country's AI efforts—and differentiate them from those in the US.

Facebook is beefing up AI research team in Seattle — www.geekwire.com

Facebook has hired Luke Zettlemoyer, a research scientist who led AI2's NLP efforts to expand its AI research team in Seattle — adding to an already-fierce competition for talent.

Investing in France’s AI Ecosystem — research.googleblog.com

Google DeepMind announces a new AI research presence in Paris as part of Google's effort to expand its research presence in Europe.

Google veteran Jeff Dean takes over as company's AI chief — www.theverge.com

Jeff Dean, seasoned engineer and head of Google Brain, takes over as head of Google's AI.

Aiming to fill skill gaps in AI, Microsoft makes training courses available to the public — blogs.microsoft.com

Microsoft announces the Microsoft Professional Program in AI, a new publicly available learning track teaching artificial intelligence skills.

Textio expands its AI to help humans craft better recruiting messages — venturebeat.com

Textio, maker of AI-powered tools to augment business writing, announces a new product to help recruiters reach out to job candidates by crafting better recruiting messages.

MIT's AlterEgo headset reads your face to see words you're only saying in your mind — www.zdnet.com

Speech is the interface of the future; in many situations, such as on loud factory floors, speech is unfeasible, however. MIT's AlterEgo interfaces with devices through 'silent speech' and sheds light on how interaction might look like in situations where we would prefer not to speak aloud.

Paper picks

Self-Representation on Twitter Using Emoji Skin Color Modifiers (ICSW 2018): Robertson et al. show that Twitter users use skin-colored emojis mainly for self-representation, i.e. people with darker skin-colored profile photos use darker skin-colored emojis.

Neural Baby Talk (CVPR 2018): Caption generation is one of the most compelling combinations of computer vision and NLP, but existing methods generally don't ground captions in elements of the image. The proposed approach first generates a sentence template with slots tied to specific image locations and then fills in those slots using object detectors. This also allows the use of domain-specific object detectors, which can produce more appropriate captions such as in the image above.

Modeling Semantic Plausibility by Injecting World Knowledge (NAACL-HLT 2018): Research has studied selectional preference, i.e. what are typical events that often occur together (e.g. man-swallow-candy is typical, while man-swallow-paintball is not). What has received less attention is semantic plausibility, i.e. what events are plausible given the properties of the involved entities. man-swallow-candy and man-swallow-paintball are both plausible, while man-swallow-desk is not. Wang et al. create a dataset for this task and show how world knowledge with regard to the relative properties of objects can be incorporated into a model to improve performance.