Selenium

What is Selenium?

Selenium is an umbrella project for a range of tools and libraries that enable and support the automation of web browsers.

Selenium is many things but at its core, it is a toolset for web browser automation that uses the best techniques available to remotely control browser instances and emulate a user’s interaction with the browser.

Here's how to integrate Thordata with Selenium.

1

Install Selenium

Install Selenium Wire to extend Selenium's Python bindings because using the default Selenium module for implementing proxies that require authentication makes the whole process complicated. You can do it using the pip command: pip install selenium-wire

Another recommended package for this integration is webdriver-manager. It's a package that simplifies the management of binary drivers for different browsers. In this case, there's no need to manually download a new version of a web driver after each update.

You can install the webdriver-manager using the pip command as well: pip install webdriver-manager

2

Specify your user credentials for proxies to work

USERNAME = "td-customer-USERNAME"

PASSWORD = "PASSWORD"

ENDPOINT = "t.pr.thordata.net:9999"

3

Check proxy status

Check if the proxy is working by visiting ipinfo.thordata.com. If everything is working correctly - it will return an IP address of a proxy you're using.

try:
    driver.get("https://ipinfo.thordata.com/")
    return f'\nYour IP is: {driver.find_element(By.CSS_SELECTOR, "pre").text}'
finally:
    driver.quit()

Full Code Example:

from selenium.webdriver.common.by import By
from seleniumwire import webdriver
# A package to have a chromedriver always up-to-date.
from webdriver_manager.chrome import ChromeDriverManager

USERNAME = "td-customer-USERNAME"
PASSWORD = "PASSWORD"
ENDPOINT = "t.pr.thordata.net:9999"

def chrome_proxy(user: str, password: str, endpoint: str) -> dict:
    wire_options = {
        "proxy": {
            "http": f"http://{user}:{password}@{endpoint}",
            "https": f"https://{user}:{password}@{endpoint}",
        }
    }

    return wire_options

def execute_driver():
    options = webdriver.ChromeOptions()
    options.add_argument('--headless')
    proxies = chrome_proxy(USERNAME, PASSWORD, ENDPOINT)
    driver = webdriver.Chrome(
    ChromeDriverManager(driver_version='<VERSION>').install(), options=options, seleniumwire_options=proxies
    )
    try:
        driver.get("https://ipinfo.thordata.com/")
        return f'\nYour IP is: {driver.find_element(By.CSS_SELECTOR, "pre").text}'
    finally:
        driver.quit()


if __name__ == '__main__':
    print(execute_driver())