Hi all, I hope you're all doing well and enjoying your Monday morning. This newsletter features discu
|
September 3 · Issue #31 · View online |
|
Hi all, I hope you’re all doing well and enjoying your Monday morning. This newsletter features discussions of some cool topics such as photo-realistic video generation, the state of ML with Python, advice for people pursuing a MSc or a PhD, some cool resources to learn about ML and NLP, and lots of other content. I really appreciate your feedback, so let me know what you love ❤️ and hate 💔 about this edition. Simply hit reply on the issue. If you were referred by a friend, click here to subscribe. If you enjoyed this issue, give it a tweet 🐦.
|
|
|
In the past two weeks, not one but two new approaches for generating increasingly photo-realistic videos have been proposed. The first one from NVIDIA generates a video based on semantic labels. The second one from UC Berkeley transfers the motion of a person in a source video to a target person. Both have video demonstrations available that are fun to watch ( here and here).
Tell me more The nice (and somewhat contrarian) thing about the Berkeley approach is that it not done end-to-end, but uses detected poses as an intermediate representation to anchor the final output.
Why is this important? Fake videos are nothing new, but so far, they were still expensive to produce or easy to spot (see last year’s Deepfakes). These new approaches indicate that we may soon cross the threshold where it may be hard for non-experts to tell the difference between what is generated and what is fake. In addition to systems for detecting and flagging fake news, we will thus need better mechanisms that identify and flag fake video content as part of our social media platforms.
|
|
Code Python is the de facto lingua franca of Machine Learning. There are few people more suitable to comment on its current state than Travis Oliphant, creator of numpy and scipy and founder of Anaconda. TL;DR: Discussing the current ML frameworks, he highlights that most libraries chose to reimplement numpy and scipy as needed. Pre 2016, the Python scientific ecosystem was mainly shaped by a few people. Numpy was created to unify array objects in Python and to unify the PyData community. While it has now evolved into a large community effort, the large number of DL frameworks essentially “divide” the community again.
Notebooks IPython notebooks are a popular tool and the main development environment for many data scientists. AI2’s Joel Grus gives a contrarian and meme-filled presentation on what not to like about notebooks, focusing mainly on their hidden state and arbitrary execution order, which makes reasoning about state difficult. Come for the memes, stay for the engineering rigour.
Hardware Tim Dettmers has updated his widely read post on which GPUs to get for Deep Learning. TL;DR: NVIDIA is still the best bet. If you want to run RNNs on GPUs, bandwith is the most important performance metric, while for convolutional neural networks, it is FLOPS. For the typical Deep Learning practicioner and researcher, an RTX or a cheap GTX 1080/1070/1060 is the best choice.
|
|
Designing a good poster for a conference, similar to designing a good presentation, takes practice. Xanda Schofield has some advice on how to design a poster faster and better—by focusing on the presentation of the poster first and foremost and bootstrapping the rest of the poster from the presentation.
|
|
While the discourse around AI generally revolves around future capabilities of AI, it is often productive to look back and take inspiration from the humble beginnings. It is valuable to look back not only on the beginnings of AI, but also the beginnings of programming in general. Ada Lovelace is generally thought to be the first programmer, but the technical details of her program are typically not given much attention. This article superbly contextualizes the program by discussing number theory, the Bernoulli numbers, and Charles’ Babbage’s Analytical Engine. Referred to as “Lady Lovelace’s Objection” by Alan Turing, Ada Lovelace argued that despite the Analytical Machine’s capabilities, it cannot really be said to “think”, marking perhaps the first consideration of Artificial Intelligence. Looking at the more immediate past, last week OpenAI lost 0-2 against two top Dota 2 teams at The International, the biggest annual Dota 2 tournament. This Verge article gives a nuanced take of what this loss means and eloquently ties it to the broader conversation of what human-AI competitions allow us to measure and whether human-AI competitions are a good way to benchmark future progress in AI.
|
|
Deep Learning to Learn 🤖 Pieter Abbeel discusses recent meta-learning approaches to enable RL models to learn faster, bridging the gap to human learning.
Planting the Seeds of Probabilistic Thinking 🌱 Shakir Mohamed gives an overview of valuable foundations and tricks for probabilistic thinking and then delves into some of the algorithms, focusing on deep generative models.
|
|
TransmogrifAI 🕴 An end-to-end automated ML library for structured data used in production by Salesforce’s Einstein AI. See this Salesforce blog post for more information.
|
|
NLP’s generalization problem, and how researchers are tackling it 🔬 Ana Marasović highlights ways in which state-of-the-art systems for many NLP tasks are still brittle and spurious and discusses the three main directions (more inductive biases, common sense, and evaluation on unseen tasks and distributions) in which researchers are tackling the problem.
|
|
Adversarial Removal of Demographic Attributes from Text Data (EMNLP 2018) Elazar and Goldberg show that representations of text classifiers can be used to recover demographic information. The decisions of classifiers are thus not agnostic to demographic attributes. Even if the models are trained with an adversarial component, a post-hoc classifier trained on the encoded sentences can still predict demographic attributes with reasonable accuracy. We should thus not rely on adversarial training to obfuscate sensitive information.
- In CoQA, both the teacher and the student can see the text passage, whereas in QuAC only the teacher has access to the passage, which makes for more SQuAD-like interactions with shorter answers.
- In QuAC, the answer is a span from the passage, while in SQuAD the answer is free-form and a span is additionally provided as rationale.
- QuAC focuses on sections from Wikipedia, while CoQA covers six additional domains, e.g. children’s stories, literature, news, etc.
Coreference and pragmatic reasoning is relevant for both and state-of-the-art reading comprehension systems trained on the datasets fall short of human performance by about 20%, indicating a lot of potential for future work.
|
Did you enjoy this issue?
|
|
|
|
If you don't want these updates anymore, please unsubscribe here.
If you were forwarded this newsletter and you like it, you can subscribe here.
|
|
|
|