NLP News - Poincaré embeddings, trolling trolls, A2C comic, General AI Challenge, heuristics for writing, year of PyTorch, BlazingText, MaskGAN, Moments in Time
Highlights in this edition include: Poincaré embeddings implementation; designing a Google Assistant app and an offensive speech detector to troll trolls; a comic intro to Advantage-Actor-Critic (A2C); the General AI Challenge; heuristics for scientific writing; a Year of PyTorch; BlazingText, a fast word2vec; MaskGAN, a new text generation model; and Moments in Time, a new dataset for video understanding.
Tools and implementations
A PyTorch implementation of the NIPS 2017 paper Poincaré Embeddings for Learning Hierarchical Representations.
Detectron is Facebook AI Research's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.
Vecmap implements a general framework to learn bilingual word embeddings (Artexte et al., 2016) and a self-learning extension that enables training with as little as 25 word pairs (Artexte et al., 2017).
Training very deep neural networks requires a lot of memory. Using the tools in this package by OpenAI, you can trade off some of this memory usage with computation to make your model fit into memory more easily.
This guide will teach you how to develop an app for the Google Assistant with the example of an app for learning Spanish.
This blog post teaches you how to troll trolls in your Intercom system by leveraging an offensive speech detector and the power of gifs.
This tutorial introduces normalizing flows, a technique that transforms densities into rich distributions that can be used with generative models, and shows how to use them in Tensorflow.
A great comic that intuitively explains the Advantage-Actor-Critic (A2C) Reinforcement Learning model with the example of a fox looking for food.
A nice tutorial that covers some essentials of using NLP in practice, but overclaims its results---classification is not 90% of NLP problems.
The Future of AI
A comprehensive and nuanced look at AI and its role in society through the lenses of Brad Smith and Harry Shum, two senior executives for legal and research respectively at Microsoft.
The General AI Challenge hosted by GoodAI and supported by Microsoft, NVIDIA, and others invites proposals for solutions to mitigate the risks associated with the AI race (for a $15k prize).
Generalization in Deep Learning
Ali Rahimi draws parallels between Deep Learning and Optics, which has clearly organised mental models. Analogously, ML researchers should develop mental models for Deep Learning at multiple layers of abstraction.
Ferenc Huszár shares some thoughts on whether flat minima generalize better than sharp minima and gives a method for analyzing generalization.
More articles and blog posts
Radek shares some tips on doing ML efficiently, e.g. "Do not ever allow calculations to exceed 10 seconds while you work on a problem".
Zachary Lipton shares some great tips for good scientific writing in a memorable, snappy one-liner format.
Jose Camacho Collados describes how to integrate word embeddings in text-based applications and describes some of the main benefits os using word embeddings in NLP.
Lars Hulstaert provides a brief introduction to transfer learning, introduces some of its applications, and describes why it is a critical skill as a data scientist.
The PyTorch team looks back on its first year, including research papers, packages, key metrics, and funny user tweets.
This Wired article seeks to debunk some of the recent claims around models from Microsoft and Alibaba reading like a human. Also refer to Yoav Goldberg's slides for more details on SQuAD vs. humans.
Wei Xu shares 2016's and 2017's list of NLP's best dressed researchers.
OpenAI shares the lessons they learned when scaling Kubernetes to 2,500 nodes.
A blog post on creating a dataset of metaphors in New York Times headlines and building a metaphor detector.
A New York Times article on Google's recent AutoML efforts.
Amazon launches Amazon SageMaker BlazingText, a fast implementation of word2vec that is both faster and cheaper than fastText.
Zalando Research discusses how it implemented its cross-lingual end-to-end product search in Tensorflow.
The Allen Institute for Artificial Intelligence launches a new CTO residency program to connect top-notch engineers with business mentors.
Kaggle gets in the data science education with Kaggle Learn, a platform providing free courses that emphasize practical data skills instead of abstract theory.
Google announces that it will open four local Google Hubs across France, run by a network of local partners. In addition, it will set up a new AI research team in Google France.
This paper describes a new GAN-based approach for generating high-quality text examples. In contrast to existing models, the approach does not do language modelling but fills in masked words in a sentence.
A comprehensive survey of Deep Learning for sentiment analysis that enumerates many relevant approaches.
This paper introduces Machines Talking to Machines, a new approach to reduce the cost and effort to build dialogue datasets. The approach consists of the following steps: 1) sampling a scenario from a task specification; 2) generating an outline using self-play between a user and a system bot; and 3) using crowdworkers to paraphrase the outline utterances.
MIT CSAIL's Moments in Time Dataset is a large-scale dataset for recognizing and understanding action in videos that consists of one million three second events capturing an ecosystem of changes in the world.
For everyone interested in pre-modern Chinese, the Chinese Text Project is the world's largest digital open-access library of pre-modern Chinese. The site attempts to make use of the digital medium to explore new ways of interacting with these texts that are not possible in print.