How to find element by class using Beautifulsoup
Important: we will use real-life example in this tutorial, so you will need requests and Beautifulsoup libraries installed.
Step 1. Let’s start by importing the Beautifulsoup library.
from bs4 import BeautifulSoup
Step 2. Then, import requests library.
import requests
Step 3. Get a source code of your target landing page. We will be looking for guide titles on our homepage in this example.
r=requests.get("https://proxyway.com/")
Universally applicable code would look like this:
r=requests.get("Your URL")
Step 4. Convert HTML code into a Beautifulsoup object named soup.
soup=BeautifulSoup(r.content,"html.parser")
Step 5. Inspect the page to find a class you would like to extract.
The code of this class looks like this:
elements_by_class = soup.find_all(class_ = "archive-list__title")
NOTE: Since we want to find all the titles instead of one, we use soup.find_all() instead of soup.find().
Step 6. Let’s check if the script works by printing its output into the terminal window.
print(elements_by_class)
NOTE: If you want to display only the titles, you can get the string attribute of each scraped element.
for element in elements_by_class: print (element.string)
Results:
Congratulations, you’ve found and extracted the content of class using Beautifulsoup. Here’s the full script:
from bs4 import BeautifulSoup
import requests
r = requests.get("https://proxyway.com/")
soup = BeautifulSoup(r.content, "html.parser")
elements_by_class = soup.find_all(class_ = "archive-list__title")
print(elements_by_class)
for element in elements_by_class:
print (element.string)
