Hands on with Scraping and Plotting Data in Python

Introduction to BeautifulSoup

import requests                       
from bs4 import BeautifulSoup
import re
import pandas as pd
URL = 'https://en.wikipedia.org/wiki/World_Happiness_Report#2019_report'
material = requests.get(URL)
soup = BeautifulSoup(material.content, 'html.parser')
tables = soup.find_all('table', class_='wikitable sortable')
Screen-shot showing the class of table present on the Wikipedia page
Name of class of table present on Wikipedia page.
for table in tables:
ths = table.find_all('th')
headings = [th.text.strip() for th in ths]
if headings[:2] == ['Country or region','Score']:
break
for tr in table.find_all('tr'):
tds = tr.find_all('td')
if not tds:
continue
country_name, score = [td.text.strip() for td in tds[:2]]
df = df.append({
'Country or region': country_name,
'Score': score,
}, ignore_index=True)
df['Score']=df['Score'].astype(float)
df.plot.bar(x='Country or region',y='Score', rot=90, title="Distribution of Happiness score per Country",legend=False);
plot.ylabel("Happiness Score")
plot.xlabel("Countries")
plot.show();
Plot of Happiness Score vs Countries
URL = 'https://en.wikipedia.org/wiki/World_Happiness_Report#2019_report'
material = requests.get(URL)
soup = BeautifulSoup(material.content, 'html.parser')
tables = soup.find_all('table', class_='wikitable sortable')for table in tables:
ths = table.find_all('th')
headings = [th.text.strip() for th in ths]
if headings[:2] == ['Country or region','Score']:
break
df = pd.DataFrame(columns=['Country or region','Score'])for tr in table.find_all('tr'):
tds = tr.find_all('td')
if not tds:
continue
country_name, score = [td.text.strip() for td in tds[:2]]
df = df.append({
'Country or region': country_name,
'Score': score,
}, ignore_index=True)
df['Score']=df['Score'].astype(float)
df.plot.bar(x='Country or region',y='Score', rot=90, title="Distribution of Happiness score per Country",legend=False);
plot.ylabel("Happiness Score")
plot.xlabel("Countries")
plot.show();

--

--

--

Python, DevOps, Cryptography, Infra-Structure Automation. https://syedsaadahmed.com/

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Product Roadmap Report Card, 9/2021

How we made our meetings more productive and fun with Cuckoo

Final Year Projects For ECE

Quick formula hacks with SUMIF and COUNTIF

ExcelQuicker.com Tip of the week: Quick formula hacks with SUMIF and COUNTIF

Class Content Order

Top 5 Patterns and Practices of Service Testing

State of Python in 2018

Guide of Create App Signature

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Syed Saad Ahmed

Syed Saad Ahmed

Python, DevOps, Cryptography, Infra-Structure Automation. https://syedsaadahmed.com/

More from Medium

Compare identical data files from different sources

[TUTORIAL] A BASIC DATA ANALYSIS WITH POSITIVES COVID 19 PERU USING PYTHON

Recode data in Python

Stock Market Analysis Using Python Pandas