Official Docker image

[cascandaliato]

Any plans to have a Docker image built for each tag and made available through the GitHub Container Registry?

[mdmintz]

Sounds like a fun project, and maybe a logical next step. There's already a Dockerfile here: SeleniumBase/Dockerfile, which people can use as a boilerplate for setting up SeleniumBase environments. As for the GitHub Container Registry, I haven't used that yet, so I'm not familiar with all the steps.

[mdmintz]

It may be easier if you post steps and I integrate it in the best way from that on my own. However, depending on the steps, I may then defer the work to you, depending on what's involved.

[cascandaliato]

Please have a look at docker-container-tag.yml.

It's a workflow that creates a Docker image whenever a tag is pushed. The image is tagged with ghcr.io/<repo_owner>/seleniumbase:<tag> and ghcr.io/<repo_owner>/seleniumbase:latest.

You might want to create a similar workflow on a cron schedule to push a nightly version.

According to this post, in order to make the image visible in the Packages section of GitHub, I had to add a LABEL to the Dockerfile. This seems confirmed by the official documentation on Connecting a repository to a package. I see that the Docker build action supports a labels parameter so it might be possible to inject the label at build time without having to change the Dockerfile.

Let me know if you need additional help.

[mdmintz]

Sounds like I might be able to start trying out the steps next week. This week there's https://github.com/SeleniumHQ/selenium/milestone/25 for the next version of Selenium, if everything gets done in time.

Side question: How useful will this Docker Image for the GitHub Container Registry be? So far, up until now, the Dockerfile (SeleniumBase/Dockerfile) has been more than enough for people. I may need some more convincing before I put an official Docker Image up like that, because that's a lot of extra code and work if you're the only one who may use it, and looks like you already have that built for your fork of SeleniumBase.

Maybe gather a few more people first and have them comment on or upvote this thread? If there's actually some real demand for this Docker Image, I can make it happen. I just want some more confirmation of that first.

[cascandaliato]

I guess we can leave this discussion open and wait for more people to comment.

I use SeleniumBase to periodically scrape some websites that require authentication and CloudFlare Turnstile verification. All components of my project run as Docker containers. Without an officially versioned image I would have to do the following as part of my project:

• keep an updated copy of this repo locally;
• build a new image whenever a tag is created.

I'll try to to build the image from my fork.

I'm pasting a copy of the files I referred above because I'm going to recreate the fork from scratch.

---

.github/workflows/docker-container-tag.yml

name: Docker image for tag

on:
  push:
    tags:        
      - 'v*'

jobs:
  build-push-image:
    runs-on: ubuntu-latest
    steps:
      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3
      
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      
      - name: Login to GitHub Container Registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.repository_owner }}
          password: ${{ secrets.GITHUB_TOKEN }}
      
      - name: Build and push image
        uses: docker/build-push-action@v6
        with:
          push: true
          tags: ghcr.io/${{ github.repository_owner }}/seleniumbase:${{github.ref_name}},ghcr.io/${{ github.repository_owner }}/seleniumbase:latest

---

Dockerfile

# SeleniumBase Docker Image
FROM ubuntu:22.04
LABEL org.opencontainers.image.source="https://github.com/cascandaliato/SeleniumBase"
SHELL ["/bin/bash", "-o", "pipefail", "-c"]

#======================
# Locale Configuration
#======================
RUN apt-get update
RUN apt-get install -y --no-install-recommends tzdata locales
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
ENV TZ="America/New_York"

#======================
# Install Common Fonts
#======================
RUN apt-get update
RUN apt-get install -y \
    fonts-liberation \
    fonts-open-sans \
    fonts-mononoki \
    fonts-roboto \
    fonts-lato

#============================
# Install Linux Dependencies
#============================
RUN apt-get update
RUN apt-get install -y \
    libasound2 \
    libatk-bridge2.0-0 \
    libatk1.0-0 \
    libatspi2.0-0 \
    libcups2 \
    libdbus-1-3 \
    libdrm2 \
    libgbm1 \
    libgtk-3-0 \
    libnspr4 \
    libnss3 \
    libu2f-udev \
    libvulkan1 \
    libwayland-client0 \
    libxcomposite1 \
    libxdamage1 \
    libxfixes3 \
    libxkbcommon0 \
    libxrandr2

#==========================
# Install useful utilities
#==========================
RUN apt-get update
RUN apt-get install -y xdg-utils

#=================================
# Install Bash Command Line Tools
#=================================
RUN apt-get update
RUN apt-get -qy --no-install-recommends install \
    curl \
    sudo \
    unzip \
    vim \
    wget \
    xvfb

#================
# Install Chrome
#================
RUN apt-get update
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN dpkg -i google-chrome-stable_current_amd64.deb
RUN apt-get -fy --no-install-recommends install
RUN rm google-chrome-stable_current_amd64.deb

#================
# Install Python
#================
RUN apt-get update
RUN apt-get install -y python3 python3-pip python3-setuptools python3-dev python3-tk
RUN alias python=python3
RUN echo "alias python=python3" >> ~/.bashrc
RUN apt-get -qy --no-install-recommends install python3.10
RUN rm /usr/bin/python3
RUN ln -s python3.10 /usr/bin/python3

#===============
# Cleanup Lists
#===============
RUN rm -rf /var/lib/apt/lists/*

#=====================
# Set up SeleniumBase
#=====================
COPY sbase /SeleniumBase/sbase/
COPY seleniumbase /SeleniumBase/seleniumbase/
COPY examples /SeleniumBase/examples/
COPY integrations /SeleniumBase/integrations/
COPY requirements.txt /SeleniumBase/requirements.txt
COPY setup.py /SeleniumBase/setup.py
COPY MANIFEST.in /SeleniumBase/MANIFEST.in
COPY pytest.ini /SeleniumBase/pytest.ini
COPY setup.cfg /SeleniumBase/setup.cfg
COPY virtualenv_install.sh /SeleniumBase/virtualenv_install.sh
RUN find . -name '*.pyc' -delete
RUN pip install --upgrade pip setuptools wheel
RUN cd /SeleniumBase && ls && pip install -r requirements.txt --upgrade
RUN cd /SeleniumBase && pip install .
RUN pip install pyautogui

#=======================
# Download chromedriver
#=======================
RUN seleniumbase get chromedriver --path

#==========================================
# Create entrypoint and grab example tests
#==========================================
COPY integrations/docker/docker-entrypoint.sh /
COPY integrations/docker/run_docker_test_in_chrome.sh /
RUN chmod +x *.sh
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["/bin/bash"]

[ihtaff]

Sounds like I might be able to start trying out the steps next week. This week there's https://github.com/SeleniumHQ/selenium/milestone/25 for the next version of Selenium, if everything gets done in time.

Side question: How useful will this Docker Image for the GitHub Container Registry be? So far, up until now, the Dockerfile (SeleniumBase/Dockerfile) has been more than enough for people. I may need some more convincing before I put an official Docker Image up like that, because that's a lot of extra code and work if you're the only one who may use it, and looks like you already have that built for your fork of SeleniumBase.

Maybe gather a few more people first and have them comment on or upvote this thread? If there's actually some real demand for this Docker Image, I can make it happen. I just want some more confirmation of that first.

I guess we can leave this discussion open and wait for more people to comment.

I use SeleniumBase to periodically scrape some websites that require authentication and CloudFlare Turnstile verification. All components of my project run as Docker containers. Without an officially versioned image I would have to do the following as part of my project:

• keep an updated copy of this repo locally;
• build a new image whenever a tag is created.

I'll try to to build the image from my fork.

I'm pasting a copy of the files I referred above because I'm going to recreate the fork from scratch.

.github/workflows/docker-container-tag.yml

name: Docker image for tag

on:
  push:
    tags:        
      - 'v*'

jobs:
  build-push-image:
    runs-on: ubuntu-latest
    steps:
      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3
      
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      
      - name: Login to GitHub Container Registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.repository_owner }}
          password: ${{ secrets.GITHUB_TOKEN }}
      
      - name: Build and push image
        uses: docker/build-push-action@v6
        with:
          push: true
          tags: ghcr.io/${{ github.repository_owner }}/seleniumbase:${{github.ref_name}},ghcr.io/${{ github.repository_owner }}/seleniumbase:latest

Dockerfile

# SeleniumBase Docker Image
FROM ubuntu:22.04
LABEL org.opencontainers.image.source="https://github.com/cascandaliato/SeleniumBase"
SHELL ["/bin/bash", "-o", "pipefail", "-c"]

#======================
# Locale Configuration
#======================
RUN apt-get update
RUN apt-get install -y --no-install-recommends tzdata locales
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
ENV TZ="America/New_York"

#======================
# Install Common Fonts
#======================
RUN apt-get update
RUN apt-get install -y \
    fonts-liberation \
    fonts-open-sans \
    fonts-mononoki \
    fonts-roboto \
    fonts-lato

#============================
# Install Linux Dependencies
#============================
RUN apt-get update
RUN apt-get install -y \
    libasound2 \
    libatk-bridge2.0-0 \
    libatk1.0-0 \
    libatspi2.0-0 \
    libcups2 \
    libdbus-1-3 \
    libdrm2 \
    libgbm1 \
    libgtk-3-0 \
    libnspr4 \
    libnss3 \
    libu2f-udev \
    libvulkan1 \
    libwayland-client0 \
    libxcomposite1 \
    libxdamage1 \
    libxfixes3 \
    libxkbcommon0 \
    libxrandr2

#==========================
# Install useful utilities
#==========================
RUN apt-get update
RUN apt-get install -y xdg-utils

#=================================
# Install Bash Command Line Tools
#=================================
RUN apt-get update
RUN apt-get -qy --no-install-recommends install \
    curl \
    sudo \
    unzip \
    vim \
    wget \
    xvfb

#================
# Install Chrome
#================
RUN apt-get update
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN dpkg -i google-chrome-stable_current_amd64.deb
RUN apt-get -fy --no-install-recommends install
RUN rm google-chrome-stable_current_amd64.deb

#================
# Install Python
#================
RUN apt-get update
RUN apt-get install -y python3 python3-pip python3-setuptools python3-dev python3-tk
RUN alias python=python3
RUN echo "alias python=python3" >> ~/.bashrc
RUN apt-get -qy --no-install-recommends install python3.10
RUN rm /usr/bin/python3
RUN ln -s python3.10 /usr/bin/python3

#===============
# Cleanup Lists
#===============
RUN rm -rf /var/lib/apt/lists/*

#=====================
# Set up SeleniumBase
#=====================
COPY sbase /SeleniumBase/sbase/
COPY seleniumbase /SeleniumBase/seleniumbase/
COPY examples /SeleniumBase/examples/
COPY integrations /SeleniumBase/integrations/
COPY requirements.txt /SeleniumBase/requirements.txt
COPY setup.py /SeleniumBase/setup.py
COPY MANIFEST.in /SeleniumBase/MANIFEST.in
COPY pytest.ini /SeleniumBase/pytest.ini
COPY setup.cfg /SeleniumBase/setup.cfg
COPY virtualenv_install.sh /SeleniumBase/virtualenv_install.sh
RUN find . -name '*.pyc' -delete
RUN pip install --upgrade pip setuptools wheel
RUN cd /SeleniumBase && ls && pip install -r requirements.txt --upgrade
RUN cd /SeleniumBase && pip install .
RUN pip install pyautogui

#=======================
# Download chromedriver
#=======================
RUN seleniumbase get chromedriver --path

#==========================================
# Create entrypoint and grab example tests
#==========================================
COPY integrations/docker/docker-entrypoint.sh /
COPY integrations/docker/run_docker_test_in_chrome.sh /
RUN chmod +x *.sh
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["/bin/bash"]

Hello , can you give me help please , with docker i get the error docker run --rm -it seleniumbase python3 /SeleniumBase/examples/cdp_mode/raw_gitlab.py
***** SeleniumBase Docker Machine *****
====================================================== {raw_gitlab.py:3:SB} starts ======================================================
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/seleniumbase/core/sb_cdp.py", line 1772, in wait_for_text
element = self.find_element(selector, timeout=timeout)
File "/usr/local/lib/python3.10/dist-packages/seleniumbase/core/sb_cdp.py", line 176, in find_element
raise Exception(message)
Exception:
Element {[for="user_login"]} was not found after 7 seconds!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/seleniumbase/plugins/sb_manager.py", line 1261, in SB
yield sb
File "/SeleniumBase/examples/cdp_mode/raw_gitlab.py", line 7, in
sb.assert_text("Username", '[for="user_login"]', timeout=3)
File "/usr/local/lib/python3.10/dist-packages/seleniumbase/fixtures/base_case.py", line 9958, in assert_text
self.cdp.assert_text(text, selector)
File "/usr/local/lib/python3.10/dist-packages/seleniumbase/core/sb_cdp.py", line 1973, in assert_text
self.wait_for_text(text, selector=selector, timeout=timeout)
File "/usr/local/lib/python3.10/dist-packages/seleniumbase/core/sb_cdp.py", line 1774, in wait_for_text
raise Exception("Element {%s} not found!" % selector)
Exception: Element {[for="user_login"]} not found!
================================================= {raw_gitlab.py:3:SB} failed in 13.41s =================================================

[cascandaliato]

After some testing, the image is now available as ghcr.io/cascandaliato/seleniumbase:version where version is either master or the specific SeleniumBase version (e.g. 4.29.4). The master image gets rebuilt (at least) every day. There will be a delay of approx. 10 minutes between a new version being released and the corresponding image being created.

For reference, the repo is cascandaliato/SeleniumBase, and the workflows are sync-fork.yml and build-docker-image.yml.

[cascandaliato]

@ttraxxrepo, in the example I pasted, the Python code takes care of starting and stopping noVNC:

vnc = subprocess.Popen(["x11vnc", "-forever", "-shared", "-display", display], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT)
...
vnc.kill()

The display being used by Xvfb is different every time you start a SeleniumBase browser so I had to extract it at runtime. Also, keep in mind that Xvfb, noVNC and the browser are stopped as soon as you exit the SeleniumBase context so you might want to add a sleep to have enough time to open noVNC and check the browser status:

import time

with seleniumbase.SB() as browser:
  ...
  time.sleep(600)

@mdmintz, is there a way to tell SeleniumBase to use a specific display number when launching Xvfb?

[andrechalella]

@cascandaliato thanks for the image, great automation!

@ttraxxrepo this below got it working for me. In short:

• x11vnc does not start noVNC, only VNC. I needed to start noVNC manually.
• Cloning websockify isn't needed, the noVNC command takes care of that.
• All that got me a black screen, which I solved with passing xvfb=True to SB().

Dockerfile:

FROM ghcr.io/cascandaliato/seleniumbase:master

RUN apt-get update && \
    apt-get install --yes git x11vnc && \
    git clone https://github.com/novnc/noVNC.git

COPY main.py .

CMD [ "python3", "main.py" ]

main.py:

import os
import re
import subprocess

import seleniumbase

import time

VNC_PORT = 5900

with seleniumbase.SB(xvfb=True) as browser:
    for f in os.listdir("/tmp"):
        if m := re.search(r"^\.X(\d+)-lock$", f):
            display = f":{m.group(1)}"
            break
    vnc = subprocess.Popen(
        (
            [
                "x11vnc",
                "-forever",
                "-shared",
                "-display",
                display,
            ]
        ),
        #stdout=subprocess.DEVNULL,
        #stderr=subprocess.STDOUT,
    )

    novnc = subprocess.Popen(
        (
            [
                '/noVNC/utils/novnc_proxy',
                '--vnc',
                f'localhost:{VNC_PORT}',
            ]
        ),
        #stdout=subprocess.DEVNULL,
        #stderr=subprocess.STDOUT,
    )

    # e.g.
    browser.get("https://www.google.com/")
    time.sleep(30)

    novnc.kill()
    vnc.kill()

[mdmintz]

The SeleniumBase Dockerfile has been updated: SeleniumBase/Dockerfile.
It's stealthy now. (Info: https://www.youtube.com/watch?v=DMKlh_-gdGs)

[Hielobro]

The SeleniumBase Dockerfile has been updated: SeleniumBase/Dockerfile. It's stealthy now. (Info: https://www.youtube.com/watch?v=DMKlh_-gdGs)

Great work! Could you in your spare time create a dockerfile that runs without github repository? So i can use it with requirements with seleniumbase listed? Or could you tell what i can take from Dockerfile for stealth?

[mdmintz]

The included Dockerfile is mainly a boilerplate for people to create their own Dockerfiles from. You could replace the SeleniumBase section with pip install seleniumbase and then run your own examples instead of using the ones provided.