How to get ‘href’ attribute of ‘a’ element using Beautiful Soup

A step-by-step guide on how to extract an URL from an a element.

Important: we will use a real-life example in this tutorial, so you will need requests and Beautifulsoup libraries installed.

Step 1. Let’s start by importing the Beautifulsoup library.

from bs4 import BeautifulSoup

Step 2. Then, import requests library.

import requests

Step 3. Get a source code of your target landing page. We will use In-Depth Smartproxy Review & Performance Tests page in this example.

r=requests.get("https://proxyway.com/reviews/smartproxy-proxies")

An universal code might look like this:

r=requests.get("Your URL")

Step 4. Convert HTML code into a Beautifulsoup object named soup.

soup=BeautifulSoup(r.content,"html.parser")

Step 5. Then, find the href attribute you want to extract. We will be using this tag as an example:

a_href=soup.find("a",{"class":"single-intro__cta"}).get("href")

An universal code might look like this:

a_href=soup.find("a",{"class":" class of your target a element"}).get("href")

Step 6. Let’s check if our code works by printing it out.

print(a_href)

Results:

Congratulations, you’ve extracted an URL from an a element. Here’s  the full script:

from bs4 import BeautifulSoup
import requests
r=requests.get("https://proxyway.com/reviews/smartproxy-proxies")
soup=BeautifulSoup(r.content,"html.parser")
a_href=soup.find("a",{"class":"single-intro__cta"}).get("href")
print(a_href)
best-scraping-apis

6 comments

  1. Jamie on February 25, 2022 at 9:43 pm

    Doesnt work

  2. hakan on April 7, 2022 at 11:02 pm

    hey my cheff “get” is not working

    • Chris Becker on April 8, 2022 at 1:43 pm

      What error are you getting? It’s working for us.

  3. mbento on December 23, 2022 at 2:16 pm

    Doesnt work me too

  4. pablo on March 1, 2023 at 9:43 am

    works for me!!! how can i make this into a loop tho

  5. Vijay on April 20, 2023 at 5:39 am

    the find should be replaced by find_all

Submit a comment

Your email address will not be published.

Rate this post