Good news! You can now use Chrome browser without display… Wait… What? While most users want to actually see the page they load in a web browser, some may want to run their browser in a server without display. Headless mode in web browsers can be used for automated testing requiring loading web pages, extracting metadata, generating bitmaps from page contents, etc…
It’s been possible to do so in the past using Xvfb to emulate a framebuffer tricking the browsers to believe there was a display in Linux. But now headless mode is build-in right into browsers, so you don’t need X server running at all, and beside Linux, it will also work in Windows, and Mac OS.
You can launch headless mode in Chrome 59 (currently in dev channel) as follows:
|
1 |
google-chrome-unstable --headless --remote-debugging-port=9222 http://www.cnx-software.com |
You may want to add --disable gpu option to remove an error message. Then go to http://localhost:9222, or http://IP:9222 is you using an actual server in order to open the DevTools interface.
Google also mentions Selenium tool for automated testing, as well as use Node.js or a C++ API to extract data.
Google Chrome won’t be the only browser to support headless mode, as Firefox developers have also been working on it, and it might already work (or not) with nightly builds. To enable it, run the command below:
|
1 |
MOZ_HEADLESS=1 /path/firefox |
Via Edibel
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress
I made some simple expect script to scrape webpages with the Firefox Scrapbook plugin, which can be called from a Telnet interface to Firefox, can´t remember the name.
Also this selenium stuff worked like a charm to take screenshots:
https://github.com/pimuzzo/selenium-python-xvfb
I had a hard time to find out how to save in HTML and render the javascript with the following code in python:
# render the javascript here
html = self.browser.execute_script(“return document.getElementsByTagName(‘html’)[0].innerHTML”)
Or something like wget or similar?