Home Developer Programming Language Framework Articles Contact Me

HTML CSS JAVASCRIPT JAVA PYTHON JQUERY REACTJS VUEJS ANGULARJS EMBERJS SASS LESS STYLUS GITHUB GITLAB BIGBUCKET JAVA KOTLIN HADOOP JDBC SERVLETS JAVA BEANS(EJB) JAVA SERVER PAGES(JSP) UNIX LINUX POSTGRESQL MSSQLSERVER SQLITE MONGODB XML FLASK PYTHON MYSQL JAVA C++ JUNIT SELENIUM

Scraping Dropdown option value using BeautifulSoup in Python

Beautiful Soup is a Python library that is used for parsing the data from HTML and XML documents. It creates a parser that assists to parse the specified information.

What is Dropdown Menu?

A dropdown menu is a toggleable menu that allows us to choose the value from the predefined list.

Step 1 (Import Required Library)

# Import Module

from bs4 import BeautifulSoup 
import requests

Step 2 (Get HTML Content)

Here we will use the requests library for getting the HTML Content from the Web URL.

# Get HTML Content

r = requests.get("Enter Web URL:- ")

Step 3 (Parse HTML Content)

# Parse HTML Content

soup = BeautifulSoup(r.content, 'html.parser')

Step 4 (Fetch Dropdown option value)

For creating a Dropdown menu, we use the select tag in HTML, and inside the select tag, we use the option tag for defining the dropdown value.

In BeautifulSoup we will use the find() method for finding the select tag

Inside select tag, we will find all option tag using the find_all() method

Then we will Iterate through all option tags and get option value using text property.

Below is the Implementation:-

# Import Module
from bs4 import BeautifulSoup
import requests

# Get HTML Content
r = requests.get("Enter Web URL:- ")

# Parse HTML Content
soup = BeautifulSoup(r.content, 'html.parser')

# Find select tag
select_tag = soup.find("select")

# find all option tag inside select tag
options = select_tag.find_all("option")

# Iterate through all option tags and get inside text
for option in options:
    print(option.text)

Example:-

Input:-

URL:- https://www.w3schools.com/tags/tryit.asp?filename=tryhtml_select

Output:-

Volvo
Saab
Opel
Audi

Let's Verify our results:-