A Recipe for Training Neural Networks 👩🍳 A new blog post by Andrej Karpathy that gives a plethora of tips for successfully training neural networks. Even if you’re training neural networks professionally, you’re bound to discover something new in this wisdom-filled selection of useful guidelines.
Advice for Better Blog Posts ✍️ Writing blog posts is a great investment, but getting started can be challenging. Rachel Thomas gives some great pieces of advice on how to write better blog posts. In particular, she calls for more people to write blog posts that accompany their papers. I whole-heartedly agree. One aspect to emphasize is that a blog post that accompanies a paper should not just be a summary—it should provide additional context, intuition, examples, and ideally a discussion of applications and future directions, all things that do not usually fit in the paper. One recent blog post that does this nicely is the following:
Universal Transformers 🌍 Mostafa Dehghani provides additional context regarding the short-comings of Transformers and relates their Universal Transformer to other models.
Domain Randomization for Sim2Real Transfer 🎲 I am excited every time Lilian Weng publishes a new blog post. This one, which discusses a technique to close the simulation-to-real world gap in robotics, does not disappoint.