Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Twitter Polling in the Cloud in 30s using PiCloud (picloud.com)
21 points by rwaliany on Aug 13, 2010 | hide | past | favorite | 7 comments


As cool as PiCloud looks, I haven't found a use for it in days. I am itching to try it out, but it can't (as far as I know) run web apps, and the uses I have for a computationally intensive, stateless (or with little state) program are nonexistent, really...

On top of that, to make the API call worth it, the computations need to take orders of magnitude more time than the network latency... Does anyone have any good ideas of what one could do with it? This Twitter bot wasn't really a good example, as I can run this sort of thing on any of my home computers. It seems to me as if even the PiCloud guys are struggling to find uses for their service...


PiCloud dev here.

Currently, our users are mainly scientists and researchers that leverage PiCloud's parallelism to expedite their computation. A lot of the use cases are simulation and equation solving related - in areas such as finance, geology, hydrology, and biology. Many of the algorithms(especially monte-carlo methods) used in these fields are computationally intensive and embarrassingly parallel (http://bit.ly/dj3qQm) and benefit from PiCloud's compute power.

On web apps, wile we cannot host web apps, we can benefit them. Some of our customers are using us as a web scraper for their backend. Encode videos (http://bit.ly/9JkLph) also works. Ultimately, any background task can use PiCloud; we for instance used it for an automatic Facebook photo tagger app. If your app is already on the cloud (esp. on AWS), you will no latency difference between your own machines and PiCloud's ; PiCloud will save you though from the headache of managing servers for background processing.

With crons, we are aiming to offer the community an easy and realiable way to run periodic work. While you can run a twitter bot on your own computer, it stops working when you power-off your computer or disconnect it from the internet. PiCloud's cron scheduler will always be running 24/7. And if you have multiple compute intensive crons, each will run much quicker by being spread horizontally across our cluster.


Hmm yes, that's basically exactly my point above. It's well suited for scientific work or MC methods, but I would instantly have an order of magnitude more uses for your service if there was a data store I could write to, even if that were just a queue (so I could upload a bunch of data and not have to go back and forth through the entire internet all the time).

All in all, it sounds like a very useful service, and I'm really looking forward to you guys adding more and more features. Hopefully one of these days I'll find something to use it for, because my invitation code has been gathering dust (it's been that long)!


We do offer cloud.files (http://docs.picloud.com/moduledoc.html#module-cloud.files), which allows you to store data. It is an S3 wrapper rather than a full-blown relational db, but you may find that it fits your needs.

You can also connect to any service with us. Amazon SQS works well if you need a highly scalable queueing system.


Hmm, a simple bot to feature a cloud platform. People might get the wrong idea about your service.


Twitterbot isn't working.


Whoops, looks like twitter's real-time search is less than real-time; turns out there can be 10+ second delays. The bot's first search (after you posted) failed to return your tweet - and when your tweet showed up in the next search a minute later, the more than one minute old tweet was ignored. Updated code is on our blog.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: