NLP News - Yann LeCun vs. Chris Manning; UMAP; Soft Cosine Measure; Convolution Visualizer; NNs on iOS Tutorial; Variational Inference Explainer; RL doesn't work; Interpretability; Neural Voice Cloning
NLP News - Yann LeCun vs. Chris Manning; UMAP; Soft Cosine Measure; Convolution Visualizer; NNs on iOS Tutorial; Variational Inference Explainer; RL doesn't work; Interpretability; Neural Voice Cloning
Highlights in this edition include: Yann LeCun vs. Chris Manning; UMAP, a faster t-SNE alternative; Soft Cosine Measure, an alternative to cosine similarity; a Convolution Visualizer; a tutorial on shipping NNs for iOS; Variational Inference explainer; RL doesn't work yet (and why); many resources on Interpretability and Fairness; DL frameworks for NLP; Neural Voice Cloning; ad-hominem attacks; and content-based citation recommendation.
Talks and discussions
Here is a post about the main themes of the discussion.
Implementations
Uniform Manifold Approximation and Projection — github.com
Implementation of UMAP, a faster alternative to t-SNE. You can find the arXiv paper here.
Soft Cosine Measure — github.com
A notebook demonstrating Soft Cosine Measure (SCM), an alternative to cosine similarity that can assess similarity between two documents even if they have no words in common.
Tools
It takes some time to wrap your head around the convolution operation. This interactive visualization can provide you with more intuition. It demonstrates how various convolution parameters affect shapes and data dependencies between the input, weight and output matrices.
Tensor Comprehensions — research.fb.com
Tensor Comprehensions is a C++ library and language by Facebook AI Research that allows researchers and engineers to generate CUDA kernels automatically from layer descriptions.
Tutorials
How I Shipped a Neural Network on iOS with CoreML, PyTorch, and React Native — attardi.org
A detailed tutorial on how to train a neural network with Pytorch, convert it using CoreML, and deploy it on the iOS App Store using React Native.
High-Level Explanation of Variational Inference
A high-level explanation of Variational Inference (VI) by Jason Eisner that provides background walks through examples for the different variants, such as Variational Bayes, Variational EM, Mean Field, etc (thanks to Ryan Cotterell for the pointer).
Reinforcement learning
Deep Reinforcement Learning Doesn't Work Yet — www.alexirpan.com
An insightful and comprehensive blog post on the challenges that current Deep Reinforcement Learning faces and why it may often not be the best choice for your problem.
An Outsider's Tour of Reinforcement Learning
The above blog post outlined why RL doesn't work. This six-part series of blog posts by Ben Recht will provide some intuition for the why. In particular, it provides an introduction to RL and aims to debunk unfounded claims.
A competition on playing Pommerman (a variant of Bomberman) with RL, either one-vs-all or team 2v2. Trained agents are converted to Docker containers and are run against other players' agents.
Interpretability and fairness
Interpreting machine learning models — towardsdatascience.com
A blog post by Lars Hulstaert on the importance of interpretability. The blog post discusses some simple actions and frameworks that you can experiment with yourself.
Transparent and Interpretable AI: An interview with Percy Liang — futureoflife.org
The Future of Life Institute interviews Percy Liang on interpretable ML systems and the role of predictability in ML.
Benchmarks and reproducibility in fair ML — algorithmicfairness.wordpress.com
A comparison (with accompanying paper) of common fairness metrics, their fairness-accuracy tradeoffs, stability, and sensitivity to preprocessing.
Interpretable Machine Learning through Teaching
A new method by OpenAI that encourages agents to teach each other with examples that also make sense to humans. The approach automatically selects the most informative examples to teach a concept, e.g. the best images to describe the concept of dogs.
The First Conference on Fairness, Accountability, and Transparency (FAT*) took place last week. The livestream is no longer available, but the tutorials can still be watched here.
Absurd and offensive dialogue agents
Absurdist Dialogues with Siri — www.theparisreview.org
Confessions of a writer and poet on the absurdity of scripting lines for AI agents.
To Give A.I. the Gift of Gab, Silicon Valley Needs to Offend You — www.nytimes.com
Researchers believe they can improve conversational A.I. systems by letting them talk to people on the internet. But sometimes, these systems can say things that reflect the worst of human nature.
Other blog posts and articles
Proving generalization of deep nets via compression — www.offconvex.org
A blog post by Sanjeev Arora about his recent paper that provides a new compression-based perspective into the generalization mystery for deep nets.
Twitter deleted 200,000 Russian troll tweets. Read them here. — www.nbcnews.com NBC News published its database of more than 200,000 Russian troll tweets Twitter had deleted, despite their importance to understanding the 2016 election.
Collective Debate is a tool by the MIT Media Lab that tries to engage users in constructive debate. You're not debating against AI chatbots (we're not there yet). Instead, it confronts you with counter-arguments to your positions on controversial issues.
Study Guides for Interview at AI Research Company — www.reddit.com
A list of resources compiled by a redditor for interviewing at an AI research company.
13 Deep Learning Frameworks for NLP in Python — medium.com
A discussion of the 13 most popular Deep Learning frameworks for NLP in Python.
Deep learning for biology — www.nature.com
This Nature feature provides an introduction to Deep Learning in biology.
Industry insights
Neural Voice Cloning with a Few Samples — research.baidu.com
Baidu Research demonstrates in this blog post how they extended their Deep Voice model to learn speaker characteristics from only a few utterances (commonly known as "voice cloning").
How We Built the Wine Bot Margot for Lidl using Aspect CXP — blogs.aspect.com
Everyone is creating chatbots these days. UK Supermarket, Lidl, worked with Aspect Software to build a chatbot that provides wine recommendations.
Uncommon.co launches and raises $18m to bring objectivity and efficiency to hiring — techcrunch.com Uncommon.co uses ML and NLP to evaluate resumes and compare their information to job descriptions in order to match applications with jobs.
Why A.I. Researchers at Google Got Desks Next to the Boss — www.nytimes.com
A New York Times article on what the new seating charts at top tech companies says about the company's priorities.
Paper picks
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation (arXiv)
A report from a large number of leading AI researchers on the potential malicious uses of AI in the digital, physical, and political domains. The researchers expect AI to expand existing threats (by making them more scalable), introduce new threats, and change the character of steps (by making them more targeted or optimized).
Before Name-calling: Dynamics and Triggers of Ad Hominem Fallacies in Web Argumentation (NAACL 2018)
This paper performs several large-scale annotation studies of ad hominem attacks in reddit and uses neural networks architectures to validate working hypotheses and to provide linguistic insight into triggers of ad hominem attacks.
Content-Based Citation Recommendation (NAACL 2018)
This paper proposes a new method for recommending citations in an academic paper draft that only uses the content of the document (rather than additional metadata). The authors also release a new dataset consisting of 7 million research articles to facilitate future citation recommendation research.