NLP News - Poincaré embeddings, trolling trolls, A2C comic, General AI Challenge, heuristics for writing, year of PyTorch, BlazingText, MaskGAN, Moments in Time
NLP News - Poincaré embeddings, trolling trolls, A2C comic, General AI Challenge, heuristics for writing, year of PyTorch, BlazingText, MaskGAN, Moments in Time
Highlights in this edition include: Poincaré embeddings implementation; designing a Google Assistant app and an offensive speech detector to troll trolls; a comic intro to Advantage-Actor-Critic (A2C); the General AI Challenge; heuristics for scientific writing; a Year of PyTorch; BlazingText, a fast word2vec; MaskGAN, a new text generation model; and Moments in Time, a new dataset for video understanding.
Tools and implementations
Poincaré Embeddings for Learning Hierarchical Representations — github.com
A PyTorch implementation of the NIPS 2017 paper Poincaré Embeddings for Learning Hierarchical Representations.
Facebook's research platform for object detection research — github.com
Detectron is Facebook AI Research's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.
Vecmap - A framework to learn bilingual word embedding mappings — github.com
Vecmap implements a general framework to learn bilingual word embeddings (Artexte et al., 2016) and a self-learning extension that enables training with as little as 25 word pairs (Artexte et al., 2017).
Saving memory using gradient-checkpointing — github.com
Training very deep neural networks requires a lot of memory. Using the tools in this package by OpenAI, you can trade off some of this memory usage with computation to make your model fit into memory more easily.
Tutorials
The Full-Stack Guide to Actions for Google Assistant — medium.com
This guide will teach you how to develop an app for the Google Assistant with the example of an app for learning Spanish.
NLP Hacks: Trolling the trolls using NLP, Intercom and AWS Lambda — blog.aylien.com
This blog post teaches you how to troll trolls in your Intercom system by leveraging an offensive speech detector and the power of gifs.
Normalizing Flows Tutorial, Part 1: Distributions and Determinants — blog.evjang.com
This tutorial introduces normalizing flows, a technique that transforms densities into rich distributions that can be used with generative models, and shows how to use them in Tensorflow.
Intuitive RL: Intro to Advantage-Actor-Critic (A2C) — hackernoon.com
A great comic that intuitively explains the Advantage-Actor-Critic (A2C) Reinforcement Learning model with the example of a fox looking for food.
How to solve 90% of NLP problems: a step-by-step guide — blog.insightdatascience.com
A nice tutorial that covers some essentials of using NLP in practice, but overclaims its results---classification is not 90% of NLP problems.
The Future of AI
A comprehensive and nuanced look at AI and its role in society through the lenses of Brad Smith and Harry Shum, two senior executives for legal and research respectively at Microsoft.
General AI Challenge — www.general-ai-challenge.org
The General AI Challenge hosted by GoodAI and supported by Microsoft, NVIDIA, and others invites proposals for solutions to mitigate the risks associated with the AI race (for a $15k prize).
Generalization in Deep Learning
Lessons from Optics, The Other Deep Learning — www.argmin.net
Ali Rahimi draws parallels between Deep Learning and Optics, which has clearly organised mental models. Analogously, ML researchers should develop mental models for Deep Learning at multiple layers of abstraction.
The Generalization Mystery: Sharp vs Flat Minima — www.inference.vc
Ferenc Huszár shares some thoughts on whether flat minima generalize better than sharp minima and gives a method for analyzing generalization.
More articles and blog posts
How to do machine learning efficiently — hackernoon.com
Radek shares some tips on doing ML efficiently, e.g. "Do not ever allow calculations to exceed 10 seconds while you work on a problem".
Heuristics for Scientific Writing (a Machine Learning Perspective)
Zachary Lipton shares some great tips for good scientific writing in a memorable, snappy one-liner format.
On the contribution of neural networks and word embeddings in Natural Language Processing — medium.com
Jose Camacho Collados describes how to integrate word embeddings in text-based applications and describes some of the main benefits os using word embeddings in NLP.
Transfer learning: leveraging insights from large data sets — towardsdatascience.com
Lars Hulstaert provides a brief introduction to transfer learning, introduces some of its applications, and describes why it is a critical skill as a data scientist.
PyTorch, a year in.... — pytorch.org
The PyTorch team looks back on its first year, including research papers, packages, key metrics, and funny user tweets.
AI Beat Humans at Reading! Maybe Not — www.wired.com
This Wired article seeks to debunk some of the recent claims around models from Microsoft and Alibaba reading like a human. Also refer to Yoav Goldberg's slides for more details on SQuAD vs. humans.
Best Dressed NLP Researchers 2016 and 2017 — cocoxu.github.io
Wei Xu shares 2016's and 2017's list of NLP's best dressed researchers.
Scaling Kubernetes to 2,500 Nodes — blog.openai.com
OpenAI shares the lessons they learned when scaling Kubernetes to 2,500 nodes.
Using NLP to Spot Metaphors — tech.winton.com
A blog post on creating a dataset of metaphors in New York Times headlines and building a metaphor detector.
Industry insights
Google Sells A.I. for Building A.I. (Novices Welcome) — www.nytimes.com
A New York Times article on Google's recent AutoML efforts.
Amazon SageMaker BlazingText: Parallelizing Word2Vec on Multiple CPUs or GPUs
Amazon launches Amazon SageMaker BlazingText, a fast implementation of word2vec that is both faster and cheaper than fastText.
Building Cross-Lingual End-to-End Product Search with Tensorflow — hanxiao.github.io
Zalando Research discusses how it implemented its cross-lingual end-to-end product search in Tensorflow.
AI2 sets up CTO residency program to link engineers with mentors — www.geekwire.com
The Allen Institute for Artificial Intelligence launches a new CTO residency program to connect top-notch engineers with business mentors.
Kaggle gets in the data science education with Kaggle Learn, a platform providing free courses that emphasize practical data skills instead of abstract theory.
Making France’s digital potential work for everyone — www.blog.google
Google announces that it will open four local Google Hubs across France, run by a network of local partners. In addition, it will set up a new AI research team in Google France.
Paper picks
MaskGAN: Better Text Generation via Filling in the ______
This paper describes a new GAN-based approach for generating high-quality text examples. In contrast to existing models, the approach does not do language modelling but fills in masked words in a sentence.
Deep learning for sentiment analysis
A comprehensive survey of Deep Learning for sentiment analysis that enumerates many relevant approaches.
Building a Conversational Agent Overnight with Dialogue Self-Play
This paper introduces Machines Talking to Machines, a new approach to reduce the cost and effort to build dialogue datasets. The approach consists of the following steps: 1) sampling a scenario from a task specification; 2) generating an outline using self-play between a user and a system bot; and 3) using crowdworkers to paraphrase the outline utterances.
Dataset spotlight
Moments in Time Dataset — moments.csail.mit.edu
MIT CSAIL's Moments in Time Dataset is a large-scale dataset for recognizing and understanding action in videos that consists of one million three second events capturing an ecosystem of changes in the world.
Chinese Text Project — ctext.org
For everyone interested in pre-modern Chinese, the Chinese Text Project is the world's largest digital open-access library of pre-modern Chinese. The site attempts to make use of the digital medium to explore new ways of interacting with these texts that are not possible in print.