Hands on with Scraping and Plotting Data in Python

Introduction to BeautifulSoup

import requests                       
from bs4 import BeautifulSoup
import re
import pandas as pd
URL = 'https://en.wikipedia.org/wiki/World_Happiness_Report#2019_report'
material = requests.get(URL)
soup = BeautifulSoup(material.content, 'html.parser')
tables = soup.find_all('table', class_='wikitable sortable')
Screen-shot showing the class of table present on the Wikipedia page
Name of class of table present on Wikipedia page.
for table in tables:
ths = table.find_all('th')
headings = [th.text.strip() for th in ths]
if headings[:2] == ['Country or region','Score']:
break
for tr in table.find_all('tr'):
tds = tr.find_all('td')
if not tds:
continue
country_name, score = [td.text.strip() for td in tds[:2]]
df = df.append({
'Country or region': country_name,
'Score': score,
}, ignore_index=True)
df['Score']=df['Score'].astype(float)
df.plot.bar(x='Country or region',y='Score', rot=90, title="Distribution of Happiness score per Country",legend=False);
plot.ylabel("Happiness Score")
plot.xlabel("Countries")
plot.show();
Plot of Happiness Score vs Countries
URL = 'https://en.wikipedia.org/wiki/World_Happiness_Report#2019_report'
material = requests.get(URL)
soup = BeautifulSoup(material.content, 'html.parser')
tables = soup.find_all('table', class_='wikitable sortable')for table in tables:
ths = table.find_all('th')
headings = [th.text.strip() for th in ths]
if headings[:2] == ['Country or region','Score']:
break
df = pd.DataFrame(columns=['Country or region','Score'])for tr in table.find_all('tr'):
tds = tr.find_all('td')
if not tds:
continue
country_name, score = [td.text.strip() for td in tds[:2]]
df = df.append({
'Country or region': country_name,
'Score': score,
}, ignore_index=True)
df['Score']=df['Score'].astype(float)
df.plot.bar(x='Country or region',y='Score', rot=90, title="Distribution of Happiness score per Country",legend=False);
plot.ylabel("Happiness Score")
plot.xlabel("Countries")
plot.show();

--

--

--

Python, DevOps, Cryptography, Infra-Structure Automation. https://syedsaadahmed.com/

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

.NET Core Web API Custom Health Check

API Health Check Image

Is GraphQL going to take over the world?

What is Speech-to-Text?

Scaling teams and managing dependencies

Infrastructure as Code: The New Normal

How to Export Firestore Data Completely Free

BEPRO Network Partners with Kylin Network

[LeetCode] (Easy) 67. Add Binary

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Syed Saad Ahmed

Syed Saad Ahmed

Python, DevOps, Cryptography, Infra-Structure Automation. https://syedsaadahmed.com/

More from Medium

Python Project — Predicting Insurance Premiums

Scraping Web Table Using Python

How to generate fake data using the Faker python package?

ENEM Data Analysis Using Python