I have mixed feelings on this article, as do many of the people in this comment section. However I do agree that the best neural networks for a problem is the smallest one that can effectively solve that problem. On the contrary, I do disagree that AGI will be able to be created on a small network. We need leaps in other technologies before we can move on from DL - it's the best algorithm we have yet for generalisation problems.