Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I am often contacted by people who ask me to scrape a dynamic/JS rendered websites. You might be surprised to know that many of such dynamic websites are actually depending on some API end-point which is being accessed via some AJAX like functionality which you can access directly and get the required data. I often faced the situation where data was not fetched via some external source was already available either in data-field or some JSON like structure hence no need to use Selenium with headless browser.


Sure. This one happens not be Selenium.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: