It's hard to know all the details but it seems like most other video providers* scale by using http caching [max age + etags] with a CDN. Language performance isn't really the bottleneck remote calls are much slower :/
I have never hosted/written a django app, do people typically run a request in a thread in django? I thought python had a GIL just like Ruby (1.9+).
I wouldn't recommend using something like rails to stream video, however there are tons of great asynchronous ways to do this and it seems like the justin.tv guys are familiar with this. I would be curious to hear why the rewrite?
* youtube: http://highscalability.com/youtube-architecture
* itunes: I have a friend on the itunes team and from what i can garnish from him their java servers are single threaded and their boxes hum around 10% cpu usage however they are bound on memory (similar to rails as they are not using java threads for each request). Akamai serves most of the content as I understood it.
> I have never hosted/written a django app, do people typically run a request in a thread in django?
Depends on the handler, mod_wsgi allows for m:n (mixing threads and processes) for instance.
> I thought python had a GIL just like Ruby (1.9+).
That's correct, but any C IO (such as DB queries) will generally release the GIL, which allows other threads to run.
(technically, CPython has a GIL, Python-the-language does not mandate it and neither Jython nor IronPython use a GIL. Pypy also has a gil currently, but Armin has started exploring STM to see if it would be a good replacement)
I have never hosted/written a django app, do people typically run a request in a thread in django? I thought python had a GIL just like Ruby (1.9+).
I wouldn't recommend using something like rails to stream video, however there are tons of great asynchronous ways to do this and it seems like the justin.tv guys are familiar with this. I would be curious to hear why the rewrite?
* youtube: http://highscalability.com/youtube-architecture * itunes: I have a friend on the itunes team and from what i can garnish from him their java servers are single threaded and their boxes hum around 10% cpu usage however they are bound on memory (similar to rails as they are not using java threads for each request). Akamai serves most of the content as I understood it.