As a system administrator, it always been a nightmare scenario for me. I was wondering what to do if attacker was powerful enough to have wide range of IPs and random timing. Is it possible to detect such attacks? Are there any good resources to study prevention techniques? Thank you all.
If they've got more bandwidth than your connection you're going to need to go upstream to the network provider and work with them. Alternately, find a hosting provider that includes DDoS protection, or use something like Cloudflare in front of your site/API.
OVH includes DDOS protection for a lot of their services and it works quite well.
They've had to weather a lot of huge attacks because their service is popular with Minecraft server hosting companies, and apparently that industry is rife with DDOS sabotage of competitors.
If you have a fat enough pipe, get a Peakflow-like device. They can usually support a few terabit of traffic (depending on if you're doing simple white/blacklists, or layer 7 mitigation). Might not work against the IoT bots or a sufficiently sophisticated attack as we all saw with the Dyn DDoS, but if you have a good peering agreement they should allow you to announce /32 (IPv4) nullroutes over BGP, which will take care of anything that saturates your uplink, and you can scrub the rest.
If you don't have your own datacenter space, your options are limited. You can use Cloudflare, or serve your content over CDN.
But any DDoS mitigation appliance should be similar.
Layer 7 is difficult because it's expensive to do on a scrubbing device, but also because a sufficiently sophisticated DDoS can look like normal traffic.
Cloudflare can stop HTTP layer 7 stuff, but things like DNS protocol? You can't easily tell what's malicious and what's not (However, I've seen some dumb DDoS's where it's things like querying for XYZXYZXYZ...(lots of characters).org, or DNS reflection attacks, those are easy to filter). In those cases, it's really just a matter of overprovisioning your service, or suffer until the attackers run out of money or get bored.
"run out of money or get bored" - made my day, this should also apply to the site owner. I bet there are even companies who are not aware that they are under attack since years (as loss is trivial yet)
Only thing you can do is have a bigger network than your attacker and/or insure there is equipment in front of your routers that can clean traffic entering or exiting your network.
As time goes on the attacks will get more sophisticated and harder to stop.
Resources for learning: https://news.ycombinator.com/item?id=17063924