The novelty of GPT3 is its few shot learning capabilities. GPT3 shows a new, previously-unknown, and, most importantly, extremely useful property of very large transformers trained on text -- that they can learn to do new things quickly. There isn't any ML researcher on record who predicted it.
> There isn't any ML researcher on record who predicted it.
That's just absurd - this was an obvious end-result for LM. NLP researchers knew that something like this was absolutely possible, my professor predicted it like 3 years ago.
> Also, I don't really get the societal penalty for disaster preparation.
It makes a lot of sense: if you're preparing for a disaster, you won't go down with everyone else when disaster strikes. This creates some feeling of resentment which caused people to lash out at you.
you won't go down with everyone else when disaster strikes. This creates some feeling of resentment which caused people to lash out at you.
A well known psychologist was castigated a few years back for his likening people to lobsters. Yet here we have a perfect example of people acting like the proverbial lobsters who drag the escaping lobster back into the pot.
TPUv3 pods have a stated performance of 100 peta "flops," which is half precision operations on hardware designed specifically for machine learning.
The 200 petaflops number of Summit is referring to general purpose double precision computation. This is totally incomparable to anything the TPU can do at any performance.
The numbers I've seen for a TPU v3 pod are ~100 PFLOPs, whereas this article claims over 3 EFLOPs, so that's at least 30 TPU v3 pods. Arguments about how useful FLOPs as a measure actually are aside, that's still quite a lot of computing power.
> Built for the U.S. Department of Energy, this is a machine designed to tackle the grand challenges of our time. It will accelerate the work of the world’s best scientists in high-energy physics, materials discovery, healthcare and more, with the ability to crank out 200 petaflops of computing power to high-precision scientific simulations.
EDIT: OK, it's 200 PFLOPs of high precision math, 3 EFLOPs of lower precision math. I take my comment back.
Statistics [edit: and especially ML] is different from math. It is important to let go of one's preconceived notions and to try to absorb the new field as is. Its jargon exists for a reason --- the normal math jargon doesn't quite have the right concepts --- so it is worthwhile spending the effort to understand it.
Yeah, but I did my PhD in theoretical computer science and have published research papers in ML. So I shouldn't have such a hard time. (It could just be me, but maybe it's a weak signal)
The winning entry used slow and handcrafted controllers, which is why the videos are sped up. A few months ago, http://rll.berkeley.edu/deeplearningrobotics/ was announced. Notice that in the videos here the robot's movement is very fast and humanlike (real time, as opposed to 15x of the KAIST video). This technique will be used in all of the robotics challenges in a year or two, which will look like a discontinuous improvement in robotics performance.
You must see a good therapist, even if you pay out of pocket for your sessions. I am also very introverted and have also gone through a few years of loneliness, though not as intense as yours. It's fine to try and fix your problems on your own, but when you fail for several years, there is no shame in getting help from a professional. Psychotherapy has made great progress in the past 50 years with the discovery of Cognitive Behavioral Therapy. It is no longer the wishy-washy pseudoscience that it used to be. Of course, a therapist is not a magician, but it may really help. You must explore this option.
Maybe, like me, you think that seeing a therapist is shameful because it is akin to "admitting defeat". However, it sounds like you've been in an extremely difficult and painful situation for a number of years, which makes seeing a therapist not a defeat; on the contrary, it is the correct thing to do.
