Web Scrapping and Data Science

I’m trying to learn data scrapping and have some doubts. So I want to work on a project where I need real and live data. To achieve this I want to web scrapping and my doubt is that is there a way to change the scrapped data to a data set compatible to a data science lib like pandas if yes then how we can do it?

You can use the Requests library & Beautiful Soup:
https://beautiful-soup-4.readthedocs.io/en/latest/

And then use pandas’ pandas.read_html to create a df from the beautiful soup object:

https://pandas.pydata.org/pandas-docs/version/0.19.2/generated/pandas.read_html.html

But, more importantly, before you scrape any data, make sure the site allows it. Read up on the ethical rules of scraping data as well.

2 Likes

Alright I will keep this in mind and thanks a lot @lisalisaj for the help !

This topic was automatically closed 41 days after the last reply. New replies are no longer allowed.