NLP News - Coursera Deep Learning course notes; VAE explainer; Metalearning Symposium videos; ML crash crouse; DeepPavlov; universal decoder of linguistic meaning; meta-learning; annotation artefacts
Hey all, we've almost cracked 2,000 subscribers! Thanks for all the support!
This newsletter is a bit shorter than usual, but I hope you'll nevertheless enjoy the content. Highlights include: Visual Coursera Deep Learning course notes; Variational Autoencoder explainer; NIPS 2017 Metalearning Symposium videos; Google's ML crash course; DeepPavlov, a library for training dialogue models; a universal decoder of linguistic meaning from brain activations; meta-learning; and annotation artefacts in NLI data.
Slides, videos, and presentations
Tess Fernandez shares her super detailed and colourful notes about the Coursera Deep Learning specialization course by Andrew Ng.
Slides and videos from the Metalearning Symposium at NIPS 2017 are now available. Speakers include Quoc Le, Oriol Vinyals, Pieter Abbeel, Jürgen Schmidhuber, and many more.
Google open-sources the Machine Learning Crash Course it's previously used internally. The course includes 15 hours of content, 25 lessons, and 40+ exercises.
Tools and implementations
DeepPavlov is an open-source library for building end-to-end dialog systems and training chatbots. It allows you to do slot filling, intent classification, automatic spelling correction, and comes even with a component that enables goal-oriented dialogue.
A demo and code for an abstract meaning representation (AMR) parser that works by processing a given sentence left-to-right, similarly to transition-based dependency parsers.
Who hasn't dreamt of discovering a new planet? Google open-sources the code and models it used to discover two new exoplanets in the Kepler telescope data, so everyone can do the same.
Reptile, a simple meta-learning algorithm developed by OpenAI, which repeatedly samples a task, performs SGD on it, and then updates the initial parameters towards the final parameters learned on that task.
Blog posts and articles
Stephen Merity makes the case for using Cython to speed up Python routines by writing and integrating C code. Cython is also used pervasively in the popular NLP library spacy.
A new Distill article on combining interfaces for visualizing various interpretability techniques. Slightly disappointing that interpretability is only studied for images.
Jeremy Jordan writes about recent advances in learning rate schedules for neural networks and best practices used in the fast.ai courses.
Nikola Mrkšić, CEO of PolyAI advocates for using ML to train conversational agents instead of hand-crafting dialogue trajectories.
The Allen Institute for Artificial Intelligence introduces their new research initiative, Project Alexandria, which will create a new unified and extensive common sense knowledge base.
Pereira et al. show concepts that are sampled from a word vector space (GloVe) to participants. They then train a system to generate the vector representation from the imaging data of the concepts and show that the model can also generate representations of sentences from imaging data.
The authors extend a recently proposed meta-learning approach, Prototypical Networks to semi-supervised learning by allowing it to incorporate unlabelled examples. They look at both scenarios where unlabelled examples are assumed to belong to existing classes as well as the setup where unlabelled examples may belong to unseen classes.
The authors show that the way natural language inference data is gathered via crowd-sourcing leaves clues that allow to guess the correct label only based on the hypothesis. A simple classifier that exploits the clues is correct for about 67% and 53% of examples for the SNLI and MultiNLI datasets respectively.