You, chromedockerpythonselenium

dockerized headfull Chrome and Selenium and Python using xvbf

I have a personal productivity project named Benson (a version of which is shared online) that helps me stay organized. I can email myself the URL of an article I want to read, and Benson will convert it to an mp3 for me. Future goals for Benson include:

My goals for Benson are getting more ambitious, so the way it interacts with URLs has to become more sophisticated. Hence my interest in dockerizing Benson along with Selenium and Chrome.

As some of you will know, headless Chrome sometimes can't do the job. But dockerized Chrome doesn't easily run headfull. After an evening of reading tutorials and StackOverflow comments that didn't do the trick, I was able to finally put together something that did work.

My specific use case is a Python app that invokes Selenium and Chrome to interact and scrape some web pages. The fact that I'm using Python is merely incidental to the headfull Chrome solution I finally landed on. In the source files I'll try to distinguish between what's required and what's incidental.

☞ In a nutshell, the X Windows Virtual Frame Buffer, xvbf, package is installed in a selenium/standalone-chrome Docker image to create a virtual display (or "framebuffer") that satisfies headfull Chrome's requirements for a display. Using the xvfb-run helper command to invoke wahtever program it is that starts headfull Chrome will make xvbf's virtual display available to Chrome.

All the files are in this repo: