I think continual learning (and incidentally curriculum learning) are going to be some of the more important subtopics within deep learning pretty soon. These things seem under-researched to me right now, despite the fact it's a space where a small academic lab could still compete.

I think part of the problem is most papers are supposed to show an engineering feat or a proven mathematical result. Not enough room for developing knowledge on how networks learn scientifically IMO.