NLP News - Yann LeCun vs. Chris Manning; UMAP; Soft Cosine Measure; Convolution Visualizer; NNs on iOS Tutorial; Variational Inference Explainer; RL doesn't work; Interpretability; Neural Voice Cloning
Highlights in this edition include: Yann LeCun vs. Chris Manning; UMAP, a faster t-SNE alternative; Soft Cosine Measure, an alternative to cosine similarity; a Convolution Visualizer; a tutorial on shipping NNs for iOS; Variational Inference explainer; RL doesn't work yet (and why); many resources on Interpretability and Fairness; DL frameworks for NLP; Neural Voice Cloning; ad-hominem attacks; and content-based citation recommendation.
Talks and discussions
Here is a post about the main themes of the discussion.
Implementation of UMAP, a faster alternative to t-SNE. You can find the arXiv paper here.
A notebook demonstrating Soft Cosine Measure (SCM), an alternative to cosine similarity that can assess similarity between two documents even if they have no words in common.
It takes some time to wrap your head around the convolution operation. This interactive visualization can provide you with more intuition. It demonstrates how various convolution parameters affect shapes and data dependencies between the input, weight and output matrices.
Tensor Comprehensions is a C++ library and language by Facebook AI Research that allows researchers and engineers to generate CUDA kernels automatically from layer descriptions.
A detailed tutorial on how to train a neural network with Pytorch, convert it using CoreML, and deploy it on the iOS App Store using React Native.
A high-level explanation of Variational Inference (VI) by Jason Eisner that provides background walks through examples for the different variants, such as Variational Bayes, Variational EM, Mean Field, etc (thanks to Ryan Cotterell for the pointer).
An insightful and comprehensive blog post on the challenges that current Deep Reinforcement Learning faces and why it may often not be the best choice for your problem.
The above blog post outlined why RL doesn't work. This six-part series of blog posts by Ben Recht will provide some intuition for the why. In particular, it provides an introduction to RL and aims to debunk unfounded claims.
A competition on playing Pommerman (a variant of Bomberman) with RL, either one-vs-all or team 2v2. Trained agents are converted to Docker containers and are run against other players' agents.
Interpretability and fairness
A blog post by Lars Hulstaert on the importance of interpretability. The blog post discusses some simple actions and frameworks that you can experiment with yourself.
The Future of Life Institute interviews Percy Liang on interpretable ML systems and the role of predictability in ML.
A comparison (with accompanying paper) of common fairness metrics, their fairness-accuracy tradeoffs, stability, and sensitivity to preprocessing.
A new method by OpenAI that encourages agents to teach each other with examples that also make sense to humans. The approach automatically selects the most informative examples to teach a concept, e.g. the best images to describe the concept of dogs.
The First Conference on Fairness, Accountability, and Transparency (FAT*) took place last week. The livestream is no longer available, but the tutorials can still be watched here.
Absurd and offensive dialogue agents
Confessions of a writer and poet on the absurdity of scripting lines for AI agents.
Researchers believe they can improve conversational A.I. systems by letting them talk to people on the internet. But sometimes, these systems can say things that reflect the worst of human nature.
Other blog posts and articles
A blog post by Sanjeev Arora about his recent paper that provides a new compression-based perspective into the generalization mystery for deep nets.
Twitter deleted 200,000 Russian troll tweets. Read them here. — www.nbcnews.com NBC News published its database of more than 200,000 Russian troll tweets Twitter had deleted, despite their importance to understanding the 2016 election.
Collective Debate is a tool by the MIT Media Lab that tries to engage users in constructive debate. You're not debating against AI chatbots (we're not there yet). Instead, it confronts you with counter-arguments to your positions on controversial issues.
A list of resources compiled by a redditor for interviewing at an AI research company.
A discussion of the 13 most popular Deep Learning frameworks for NLP in Python.
This Nature feature provides an introduction to Deep Learning in biology.
Baidu Research demonstrates in this blog post how they extended their Deep Voice model to learn speaker characteristics from only a few utterances (commonly known as "voice cloning").
Everyone is creating chatbots these days. UK Supermarket, Lidl, worked with Aspect Software to build a chatbot that provides wine recommendations.
Uncommon.co launches and raises $18m to bring objectivity and efficiency to hiring — techcrunch.com Uncommon.co uses ML and NLP to evaluate resumes and compare their information to job descriptions in order to match applications with jobs.
A New York Times article on what the new seating charts at top tech companies says about the company's priorities.
A report from a large number of leading AI researchers on the potential malicious uses of AI in the digital, physical, and political domains. The researchers expect AI to expand existing threats (by making them more scalable), introduce new threats, and change the character of steps (by making them more targeted or optimized).
This paper performs several large-scale annotation studies of ad hominem attacks in reddit and uses neural networks architectures to validate working hypotheses and to provide linguistic insight into triggers of ad hominem attacks.
This paper proposes a new method for recommending citations in an academic paper draft that only uses the content of the document (rather than additional metadata). The authors also release a new dataset consisting of 7 million research articles to facilitate future citation recommendation research.