I’m trying to learn data scrapping and have some doubts. So I want to work on a project where I need real and live data. To achieve this I want to web scrapping and my doubt is that is there a way to change the scrapped data to a data set compatible to a data science lib like pandas if yes then how we can do it?
You can use the Requests library & Beautiful Soup:
https://beautiful-soup-4.readthedocs.io/en/latest/
And then use pandas’ pandas.read_html
to create a df from the beautiful soup object:
https://pandas.pydata.org/pandas-docs/version/0.19.2/generated/pandas.read_html.html
But, more importantly, before you scrape any data, make sure the site allows it. Read up on the ethical rules of scraping data as well.
2 Likes
This topic was automatically closed 41 days after the last reply. New replies are no longer allowed.