Thinking about it, and looking at the graphs of learning curves etc, this paper is Google saying to everyone else: our hardware is so much better than yours & we have so much more of it that we can beat anything you do in AI without breaking sweat. You can’t touch us in other words.
Quantity has a quality all of it’s own & quantity counts more than anything else in current RNN training.
Wow. 5000 TPUs
Thinking about it, and looking at the graphs of learning curves etc, this paper is Google saying to everyone else: our hardware is so much better than yours & we have so much more of it that we can beat anything you do in AI without breaking sweat. You can’t touch us in other words.
Quantity has a quality all of it’s own & quantity counts more than anything else in current RNN training.