And AlphaGo. I especially liked "Decoupled Neural Interfaces using Synthetic Gradients" because it seemed to open up training of complex models on multiple machines, but also because it provided an unexpected insight into how local learning can be done.
I'm still waiting to see synthetic gradients get applied anywhere else. When it came out, I thought it would at the very least revolutionize training of RNNs by giving them better BPTT, but it seems to've sunk without a trace.
