Important: we will use real-life example in this tutorial, so you will need requests and Beautifulsoup libraries installed.
Step 1. Let’s start by importing the Beautifulsoup library.
from bs4 import BeautifulSoup
Step 2. Then, import requests library.
Step 3. Get a source code of your target landing page. We will be looking for guide titles on our homepage in this example.
A universal code looks like this:
Step 4. Convert HTML code into a Beautifulsoup object named soup.
Step 5. Inspect the page to find a class you would like to extract.
The code of this class looks like this:
elements_by_class = soup.find_all(class_ = "archive-list__title")
NOTE: Since we want to find all the titles instead of one, we use soup.find_all() instead of soup.find().
Step 6. Let’s check if the script works by printing its output into the terminal window.
NOTE: If you want to display only the titles, you can get the string attribute of each scraped element.
for element in elements_by_class: print (element.string)
Congratulations, you’ve found and extracted the content of class using Beautifulsoup. Here’s the full script:
from bs4 import BeautifulSoup import requests r = requests.get("https://proxyway.com/") soup = BeautifulSoup(r.content, "html.parser") elements_by_class = soup.find_all(class_ = "archive-list__title") print(elements_by_class) for element in elements_by_class: print (element.string)