Learn Selenium with Python: A Complete Tutorial for Automation Testers

1. What is Selenium WebDriver?

Selenium Webdriver is what the world is talking about. Selenium Webdriver is a web automation tool that is used for web functional testing. It supports different languages like Java, Python, C#, PHP, Ruby, and Perl. WebDriver is now an official W3C Specification, and in short, it is a method of interacting with a web browser. 

Previously, with Selenium RC, Selenium would operate with the browser by injecting JavaScript to interact with elements. With the adoption of WebDriver, browser manufacturers like Google, Mozilla, and Microsoft release their browsers with the ability to be controlled by a hook, that Selenium can tap into. This hook enables Selenium to interact with the web browser in the same way that humans do.

Selenium with Python Video Tutorial (YouTube)

2. Features of Selenium WebDriver

1. Opensource/Freeware: Selenium is free of cost. It means you do not have to buy any license like other software or do not have to pay anyone for using Selenium for automation testing.

2. Web Functional Framework: Selenium can only be used for automation testing of web applications/websites. It is a functional testing tool so we can only automate functional test cases.

3. Multi-Language Support: It is one of the major benefits of Webdriver. Selenium Webdriver supports many languages like Java, Python, PHP, C#, Ruby, Perl, etc. So, it provides the flexibility to automation test engineers to automate test cases using any one language that they are comfortable.

4. Cross OS Support: Selenium supports multiple OS like, Windows, Linux, Unix, and Mac. So if you have written web driver automation code using the Windows platform, then the same automation code could be used on other OS platforms like Unix, Linux, and Mac.

5. Cross-Browser Compatibility Testing: Selenium Webdriver supports almost every browser like Chrome, FF, Safari, Opera, IE, and their versions. You can execute the same automation scripts on different browsers for compatibility testing.

6. Cross-Device Testing: Appium, which is a mobile automation tool is actually built on the top of Selenium Webdriver APIs. In simple words, Appium is another form of Webdriver to e used for the automation testing of mobile applications.

7. Huge Community Support: There is huge community support for Selenium. A lot of tech professionals are contributing to building Selenium as the best automation tool.

8. Integrated with Multiple Frameworks/Tools: Selenium Webdriver can be integrated with various frameworks and tools like Maven, and Ant for compiling source code, with unit testing frameworks like TestNG for application testing and reporting, and with Jenkins for Continuous Integration or Continuous Delivery, automated build and deployment.

9. Integration Support with Other Automation Tools: It also gives the flexibility to integrate Java or Python modules, and other automation tools like Sikuli, and AutoIT for your automation needs.

3. Selenium Variants

There are 4 different variants of Selenium, which are:

1. Selenium IDE 

It’s a record-and-play tool, that comes as a plugin with Firefox only. No coding is required if you want to automate using this tool. But it can only be used for some very basic automation. As you cannot write your own logic or you won’t be able to put conditions or loops in your automation test script.

2. Selenium RC

RC stands for remote control. We have to write code using RC and it supports a number of languages. But it’s outdated now and there is no support available for RC.

3. Selenium Webdriver

Selenium Webdriver is a web automation tool that can be used with different languages like Java, Python, C#, PHP, Ruby, Perl, etc. It’s the successor of RC. Different architecture from what RC had (Not going into details).

4. Selenium Grid

Used for parallel execution of test cases (not an automation tool)

4. How does Selenium work?

 

5. Installation and Configuration

1. Download and Install the latest version of Python from the below link -

• https://www.python.org/downloads/​

2. Download and Install PyCharm (Community Edition) - ​PyCharm is a Python IDE that is used to write Python code.

• https://www.jetbrains.com/pycharm/download/#section=windows​
• After installing PyCharm, create a new Python project in it and create a new Python file.

3. Install the Selenium​ module in Python -

• Open the command prompt and type the following command in it and press enter
• pip3 install –U selenium​
• You can get all the information regarding Selenium's latest PyPI package here - https://pypi.org/project/selenium/

4. Download ChromeDriver and GeckoDriver - Browser drivers are required to execute your Selenium code, if you want to execute your code on Chrome then you would have to download Chromedriver and if you want to execute your Selenium script on Firefox then you would need Geckodriver. You can download drivers from the following links -

• https://chromedriver.chromium.org/downloads​
• https://github.com/mozilla/geckodriver/releases​

• After downloading the drivers, place them alongside your Selenium scripts.

6. Selenium Locators

For the Selenium Locators tutorial please refer to this link - https://www.techlistic.com/2019/07/selenium-webdriver-locators-and-find.html

7. Selenium with Python Commands

7.1. Basic Selenium Operations with Python

Let's take a look at the main Selenium Python commands which are used to automate any website/ web application.

1. Import Webdriver

The first thing to do in any Selenium script is to import the web driver package. Only after importing Webdriver, you can access all the functionality and commands of the Webdriver.

Syntax:

from selenium import webdriver

2. Browser Command

We use browser command to launch any browser which we want to use for automation testing. And along with the browser class we have to give the path of the browser driver. In the following example, we have created an object of web driver and initialized it with the Chrome browser class and we have passed the path of the Chromedriver to the constructor of the Chrome class. The following line of code will launch the Chrome browser session.

The syntax for initializing web driver object with Chrome browser class:

browser = webdriver.Chrome("chromedriver.exe")

​

The syntax for launching Firefox browser:

browser = webdriver.Firefox("geckodriver.exe")

3. Open the URL command

This command is used to open the website URL which we want to test, get() command is used for this purpose. And get() command is called with the web driver object which browser and . (dot) operator.

Syntax to Get URL command:

browser.get('http://selenium.dev/')​

4. Maximize Browser Window Command

This command is used to maximize the browser window which is opened by Selenium Webdriver.

The syntax for maximizing browser window:

self.driver.maximize_window()

5. Implicit Wait Command

It is used to wait for a particular time interval generally seconds, before throwing the "Element not found" exception. So, if the user has set an implicit wait of 30 secs then the Selenium web driver will wait for a maximum of 30 seconds for the element under the action, if that element does not appear on the web page in 30 seconds, then Selenium will fail the script and throw the "Element not found" exception.

Syntax for the implicit wait:

browser.implicitly_wait(30)

6. Find Element Command

This command is used to locate the element on the webpage so that Selenium Webdriver can perform the desired action on that element like., enter text inside the text box, click on the radio button or checkbox or click on the button etc. Find Element command is used along with the locator of the element like., id, class, name, xpath or css selector etc. For e.g., in the sample code we have located a text box with it's id value "firstName".

Syntax for Find Element Command:

browser.find_element(By.ID,'firstName')

Syntax for other Find Element Commands are:

browser.find_element(By.CLASS_NAME, 'search')

browser.find_element(By.NAME, 'lastName')

browser.find_element(By.LINK_TEXT, 'Selenium Tutorial')

browser.find_element(By.PARTIAL_LINK_TEXT, 'Tutorial')

browser.find_element(By.TAG_NAME, 'select')

browser.find_element(By.XPATH, "//*[contains(text(),'Music')]")

browser.find_element(By.CSS_SELECTOR, '#code2 > textarea')

7. Send Keys Command (Enter Text)

This command is used to enter text in any text box, text area, date picker etc. It is used along with find element command. In the example code, we are entering text "Jassi" in a text box using send_keys() command.

Syntax for Enter Text command is send_keys():

browser.find_element(By.ID,'firstName').send_keys("Jassi")

8. Click Command

Click command is used for various purposes, like for checking radio button, for selecting check box and for clicking a button or clicking on a link. This command is also used with find element command.

Syntax for clicking on a link "Selenium Tutorial":

browser.find_element(By.LINK_TEXT, 'Selenium Tutorial').click()

9. Handling Alerts, Frames, and Windows:
During the execution of your script, you may encounter alerts, frames, and multiple windows or tabs. Selenium provides methods to handle these situations effectively.
Use the switch_to.alert methods to accept, dismiss, or retrieve text from an alert. For example, driver.switch_to.alert.accept() accepts an alert.
To switch focus to a specific frame, use the switch_to.frame() method, passing the frame's ID or name as a parameter. For example, driver.switch_to.frame("frame-id").
To handle multiple windows, use the window_handles property to get a list of window handles. Then, use the switch_to.window() method to switch between windows by passing the window handle as a parameter. For example, driver.switch_to.window("window-handle"). Execute your script and ensure that alerts, frames, and windows are handled correctly.
Here's an example code that demonstrates handling alerts, frames, and windows using Selenium with Python:
python
from selenium import webdriver
from selenium.webdriver.common.alert import Alert
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# Launch the Chrome browser
driver = webdriver.Chrome()

# Navigate to the website with alert, frame, and window elements
driver.get("https://www.example.com")

# Handling Alerts
# Clicking a button that triggers an alert
alert_button = driver.find_element_by_id("alert-button")
alert_button.click()

# Switch to the alert and accept it
alert = Alert(driver)
alert.accept()

# Handling Frames
# Switch to a frame by its ID or name
driver.switch_to.frame("frame-id")

# Perform actions within the frame
frame_element = driver.find_element_by_id("frame-element")
frame_element.click()

# Switch back to the default content
driver.switch_to.default_content()

# Handling Windows
# Open a new window by clicking a link that opens in a new window
new_window_link = driver.find_element_by_link_text("Open New Window")
new_window_link.click()

# Switch to the new window
driver.switch_to.window(driver.window_handles[1])

# Perform actions in the new window
new_window_element = driver.find_element_by_id("new-window-element")
new_window_element.click()

# Switch back to the original window
driver.switch_to.window(driver.window_handles[0])

# Close the browser
driver.quit()
In the above code, we first import the necessary modules from the selenium package. We initialize the Chrome driver by creating an instance of webdriver.Chrome(), which launches the Chrome browser.
Next, we navigate to a website that contains elements with alerts, frames, and windows.
To handle alerts, we locate and click a button that triggers an alert. We then switch to the alert using Alert(driver) and use accept() to accept the alert.
For handling frames, we switch to a frame using its ID or name with switch_to.frame("frame-id"). We can then perform actions within the frame, such as finding and interacting with elements.
To handle windows, we open a new window by clicking a link that opens in a new window. We switch to the new window using switch_to.window(driver.window_handles[1]), where driver.window_handles[1] refers to the second window handle. We can then perform actions within the new window. To switch back to the original window, we use switch_to.window(driver.window_handles[0]).

10. Capturing Screenshots:
Capturing screenshots during script execution is useful for documentation, debugging, and error analysis. Selenium provides the save_screenshot() method to capture screenshots. You can specify the file path and name to save the screenshot. 

For example, driver.save_screenshot("screenshot.png"). Experiment with capturing screenshots at different stages of your script, such as before and after interacting with elements. Execute your script and verify that the screenshots are saved successfully.

Here's an example code that demonstrates capturing screenshots using Selenium with Python:
python
from selenium import webdriver

# Launch the Chrome browser
driver = webdriver.Chrome()

# Navigate to the website
driver.get("https://www.example.com")

# Capture a screenshot of the entire page
driver.save_screenshot("full_page_screenshot.png")

# Locate a specific element on the page
element = driver.find_element_by_id("element-id")

# Capture a screenshot of the specific element
element.screenshot("element_screenshot.png")

# Close the browser
driver.quit()
In the above code, we first import the webdriver module from the selenium package. We then initialize the Chrome driver by creating an instance of webdriver.Chrome(), which launches the Chrome browser.
Next, we navigate to the desired website by using the get() method and providing the URL as a parameter.
To capture a screenshot of the entire page, we use the save_screenshot() method of the driver object. We specify the file name with the .png extension. In this example, we save it as "full_page_screenshot.png". The screenshot will be saved in the current working directory.
If you want to capture a screenshot of a specific element on the page, you first need to locate the element using any of the available locating techniques. In this example, we find an element with the ID "element-id". Once the element is located, we can use the screenshot() method of the element itself to capture a screenshot of just that element. Similar to the previous example, we specify the file name as "element_screenshot.png".

11. Quit Command

This command is used close all the browsers opened by the Selenium Webdriver. It is generally used in the end of the Selenium script to close the browser.

Syntax to Close Browser Command:

browser.quit()

7.2. Waits and Synchronization

We will delve into the concept of waits and synchronization in Selenium. Waits play a crucial role in automating web applications, ensuring that the test script waits for certain conditions to be met before proceeding.

1. Understanding the Importance of Waits in Selenium:
When automating web applications, it's essential to account for varying page load times, AJAX requests, and dynamic elements that might not be immediately available. Without proper waits, your test script may fail due to timing issues. Waits help in synchronizing the test script with the web application, ensuring that actions are performed only when the desired conditions are met. They help improve the stability and reliability of your automation tests.

2. Implicit vs. Explicit Waits:

Selenium provides two types of waits: implicit and explicit waits.

i. Implicit Waits: Implicit waits are set globally and apply to all elements. They instruct Selenium to wait for a specified amount of time before throwing a NoSuchElementException if an element is not immediately found. Use the implicitly_wait() method to set an implicit wait timeout.

ii. Explicit Waits: Explicit waits are more flexible and allow you to wait for specific conditions to be met before proceeding. You can wait for elements to be present, visible, clickable, etc. Use the WebDriverWait class along with expected conditions to implement explicit waits.

3. Waiting for Web Elements to be Present, Visible, or Clickable:

With explicit waits, you can wait for various conditions related to web elements:
Presence of Element: Wait until an element is present in the DOM using the presence_of_element_located expected condition.
Visibility of Element: Wait until an element becomes visible using the visibility_of_element_located expected condition.
Clickability of Element: Wait until an element becomes clickable using the element_to_be_clickable expected condition.
Example code for waiting until an element is clickable:
python
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
driver.get("https://www.example.com")

wait = WebDriverWait(driver, 10)
element = wait.until(EC.element_to_be_clickable((By.ID, "element-id")))

# Perform actions on the element once it is clickable
element.click()


4. Handling AJAX Calls and Dynamic Web Elements:

AJAX calls and dynamically changing elements require additional synchronization in Selenium automation. You can use explicit waits along with appropriate expected conditions to handle such scenarios. For example, you can wait for the presence of a specific element that is loaded or modified dynamically using the presence_of_element_located expected condition.
Example code for waiting for an element to be present:
python
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
driver.get("https://www.example.com")

wait = WebDriverWait(driver, 10)
element = wait.until(EC.presence_of_element_located((By.ID, "element-id")))

# Perform actions on the element once it is present
element.click()


5. Customizing Wait Conditions and Timeouts:

Selenium allows you to customize wait conditions and timeouts based on your specific needs. You can create your own expected conditions by extending the ExpectedCondition class and implementing the __call__ method. This enables you to wait for custom conditions to be met before proceeding with your test script.
Example code for creating a custom wait condition:
python
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

class CustomCondition:
    def __call__(self, driver):
        # Custom condition logic
        return driver.title == "Expected Title"

driver = webdriver.Chrome()
driver.get("https://www.example.com")

wait = WebDriverWait(driver, 10)
condition = CustomCondition()
element = wait.until(condition)

# Perform actions once the custom condition is met

7.3. Advanced Selenium Operations with Selenium Python:

You will be equipped with the knowledge to tackle complex automation scenarios using Selenium and Python.

1. Working with Dropdowns, Checkboxes, and Radio Buttons:

Dropdowns, checkboxes, and radio buttons are common elements in web forms. Selenium provides methods to interact with these elements easily.

• Dropdowns: Use the Select class to work with dropdowns. You can select options by value, index, or visible text.

• Checkboxes and Radio Buttons: Use the click() method to toggle checkboxes and radio buttons.

Example code for working with a dropdown:

python
from selenium import webdriver
from selenium.webdriver.support.ui import Select

driver = webdriver.Chrome()
driver.get("https://www.example.com")

dropdown = Select(driver.find_element_by_id("dropdown-id"))
dropdown.select_by_value("option-value")

# or
dropdown.select_by_visible_text("Option Text")

# or
dropdown.select_by_index(2)

2. Performing Mouse Actions:

Selenium allows you to perform various mouse actions, including drag and drop, double-click, and right-click.

• Drag and Drop: Use the drag_and_drop() method to drag an element from one location to another.

• Double-Click: Use the double_click() method to perform a double-click action on an element.

• Right-Click: Use the context_click() method to perform a right-click action on an element.

Example code for performing a drag-and-drop action:

python
from selenium import webdriver
from selenium.webdriver import ActionChains

driver = webdriver.Chrome()
driver.get("https://www.example.com")

source_element = driver.find_element_by_id("source-id")
target_element = driver.find_element_by_id("target-id")

actions = ActionChains(driver)
actions.drag_and_drop(source_element, target_element).perform()

3. Handling File Uploads and Downloads:

Selenium provides methods to handle file uploads and simulate file downloads.

• File Uploads: Use the send_keys() method to set the file path for a file input element.

• File Downloads: By default, Selenium does not directly handle file downloads. However, you can configure the browser to automatically save files to a specific location.

Example code for handling file uploads:

python
from selenium import webdriver

driver = webdriver.Chrome()
driver.get("https://www.example.com")

file_input = driver.find_element_by_id("file-input")
file_input.send_keys("path/to/file.txt")

4. Executing JavaScript Code:

Selenium allows you to execute JavaScript code within the browser. This feature can be useful in scenarios where direct interaction with the DOM is required.

• Use the execute_script() method to execute JavaScript code.

Example code for executing JavaScript:

python
from selenium import webdriver

driver = webdriver.Chrome()
driver.get("https://www.example.com")

driver.execute_script("window.scrollTo(0, document.body.scrollHeight)")

5. Handling Browser Cookies:

Selenium enables you to manage browser cookies, including adding, deleting, and retrieving cookies.

• Use the get_cookies() method to retrieve all cookies.

• Use the add_cookie() method to add a cookie.

• Use the delete_cookie() method to delete a specific cookie.

• Use the delete_all_cookies() method to delete all cookies.

Example code for handling browser cookies:

python
from selenium import webdriver

driver = webdriver.Chrome()
driver.get("https://www.example.com")

# Get all cookies
cookies = driver.get_cookies()
print(cookies)

# Add a cookie
cookie = {'name': 'myCookie', 'value': '12345'}
driver.add_cookie(cookie)

# Delete a specific cookie
driver.delete_cookie('myCookie')

# Delete all cookies
driver.delete_all_cookies()

8. Best Practices and Tips for Selenium Automation with Python:

8.1. Writing Clean and Maintainable Selenium Code in Python:

Writing clean and maintainable code is crucial for the long-term success of any Selenium automation project. We will explore some best practices to achieve this.

• Follow PEP 8 Guidelines: Adhere to the Python coding standards outlined in PEP 8 for consistent code formatting and style.

• Use Meaningful Variable and Function Names: Use descriptive names that convey the purpose and functionality of variables and functions.

• Modularize Code: Break down complex code into smaller, reusable functions and classes for improved readability and maintainability.

8.2. Page Object Model (POM) Design Pattern:

The Page Object Model (POM) design pattern is widely used to improve the structure and maintainability of Selenium automation code. We will explore how to implement POM in Python.

• Create Page Objects: Define separate classes for each web page or component, encapsulating the related elements and actions within the class.

• Use Page Object Methods: Implement methods within page objects to perform actions on the elements and retrieve information from the page.

Example code for implementing POM:

python
class LoginPage:
    def __init__(self, driver):
        self.driver = driver
        self.username_field = driver.find_element_by_id("username")
        self.password_field = driver.find_element_by_id("password")
        self.login_button = driver.find_element_by_id("login-button")

    def enter_username(self, username):
        self.username_field.clear()
        self.username_field.send_keys(username)

    def enter_password(self, password):
        self.password_field.clear()
        self.password_field.send_keys(password)

    def click_login(self):
        self.login_button.click()

8.3. Handling Exceptions and Error Handling:

Handling exceptions and errors is essential for robust Selenium automation. We will explore techniques to handle common exceptions and improve error handling.

• Use Try-Except Blocks: Wrap Selenium operations in try-except blocks to catch and handle exceptions gracefully.

• Implement Explicit Waits: Use explicit waits to wait for specific conditions before performing actions on web elements.

Example code for handling exceptions:

python
from selenium.common.exceptions import NoSuchElementException, TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

try:
    element = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.ID, "element-id"))
    )
except NoSuchElementException:
    # Handle element not found exception
except TimeoutException:
    # Handle timeout exception

8.4. Debugging and Troubleshooting Common Issues:

Debugging and troubleshooting are essential skills for any automation engineer. We will explore techniques to identify and resolve common issues encountered during Selenium automation.

• Logging: Utilize the built-in logging module to log relevant information during test execution for easier debugging.

• Inspect Web Elements: Use browser developer tools to inspect web elements and their properties for accurate element identification.

8.5. Performance Optimization Techniques:

Optimizing the performance of Selenium automation scripts is crucial for faster test execution. We will explore techniques to improve script performance.

• Use Implicit Waits: Set implicit waits to a reasonable timeout to wait for elements globally.

• Reduce Unnecessary Interactions: Minimize unnecessary interactions with the browser, such as maximizing and resizing the window.

• Use Headless Browsing: Utilize headless browsers ChromeOptions to run tests without the browser GUI, improving performance.

Conclusion

In this tutorial, we explored Selenium with Python, a powerful tool for automating web browser interactions. We covered various aspects of Selenium automation, starting from the basics to more advanced concepts.

Next >> Real-World Examples and Demo Scripts in Selenium Python Automation

Author

Vaneesh Behl

Passionately working as an Automation Developer for more than a decade.