Google Search Starts Requiring JavaScript Rendering
Web scraping services scramble for a workaround.
- Published:
Google, the largest search engine, has started requiring JavaScript rendering to display search results.
Without it, Google refuses to deliver the query and rather redirects to instructions for enabling JavaScript.
The change took place sometime between January 15 and 16. It was noticed and discussed on Hacker News.
For now, the workaround is to run headless browsers or change the user agent to one of JavaScript-free web browsers, such as Lynx.
Google’s move has affected most commercial scraping APIs, either disrupting their services or notably increasing the latency due to the forced switch to browser-based crawlers.
In addition, it has probably made scraping the website much more costly, at least until a better workaround is found.
Get proxy news and updates directly to your inbox.
