Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Depends on scale. At higher end, it was near impossible to scale when you're e.g. inserting a MySQL row per packet. But maybe good enough for a viable business. I would probably try to take it as far as possible on Elastic if I were to write it today.

Same thing if you read the Dremel paper. Worrying about bits helps when scaling.



Because Lucene wasn't good at near-realtime in 2009 or so, Twitter's original (acquired via summize) search was written in mysql. It might have even been a row for every token, not quite sure.

IIRC, when we moved to a highly-customized-Lucene-based system in 2011, we dropped the server count on the cluster from around 400 nodes to around 15.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: