We are currently crawling all domains (in the tlds we cover) once a month, and s...

katz · on March 4, 2009

Naive questions:

How does your data center look like? What is your web crawler written in?

I assume that your startup is fairly small - how difficult is/was it to get enough resources (funding) to scale up? (esp. datacenter wise)

I see that you do not just cover English and other (co.?? org.??) domains are covered fairly well.

epi0Bauqu · on March 5, 2009

The crawler is written in Perl, and has been optimized such that the amount of resources we need to produce our list is actually not that much.

ejs · on March 4, 2009

Ok great, I was hoping so but did not see it anywhere spelled out.

Good to hear!

epi0Bauqu · on March 4, 2009

It's actually on the report a false positive page (http://www.ivegotafang.com/fp.html), but I agree it should be made more clear on the front page as well. Thanks for the feedback!