SUMMARY: The purpose of this project is to practice web scraping by extracting specific pieces of information from a website. The Python web scraping code leverages the Selenium module.

INTRODUCTION: The Registry of Open Data on AWS makes datasets publicly available through AWS services. When data is shared on AWS, anyone can analyze it and build services on top of it using a broad range of compute and data analytics products. Sharing data in the cloud also lets data users spend more time on data analysis rather than data acquisition. The script automatically traverses the dataset listing and capture the descriptive data by storing them in a CSV output file.

Starting URLs: https://registry.opendata.aws/

The source code and HTML output can be found here on GitHub.

SUMMARY: The purpose of this project is to practice web scraping by extracting specific pieces of information from a website. The Python web scraping code leverages the Selenium module.

INTRODUCTION: Books to Scarpe is a fictional bookstore that desperately wants to be scraped according to its site owner. It is a safe place for beginners learning web scraping and for developers validating their scraping technologies as well. This iteration of the script automatically traverses the book listing and detail web pages to capture all the descriptive data about the books and store them in a CSV output file.

Starting URLs: http://books.toscrape.com/

The source code and HTML output can be found here on GitHub.

SUMMARY: The purpose of this project is to practice web scraping by extracting specific pieces of information from a website. The Python web scraping code leverages the Selenium module.

INTRODUCTION: Books to Scarpe is a fictional bookstore that desperately wants to be scraped according to the site owner. It is a safe place for beginners learning web scraping and for developers validating their scraping technologies as well. This iteration of the script automatically traverses the book listing web pages (about 50 pages and 1000 items) to capture all the basic data about the books and store them in a CSV output file.

Starting URLs: http://books.toscrape.com/

The source code and HTML output can be found here on GitHub.

SUMMARY: The purpose of this project is to practice web scraping by extracting specific pieces of information from a website. The Python web scraping code leverages the Selenium module.

INTRODUCTION: Metro is a transportation planner and coordinator, designer, builder and operator for one of the country’s largest, most populous counties, Los Angeles. More than 9.6 million people, nearly one-third of California’s residents, live, work and play within its 1,433-square-mile service area. The purpose of this exercise is to practice web scraping by gathering the bus ridership statistics from the agency’s web pages. This iteration of the script automatically traverses the monthly web pages (from January 2009 to June 2020) to capture all bus ridership entries and store the information in a CSV output file.

Starting URLs: http://isotp.metro.net/MetroRidership/Index.aspx

The source code and HTML output can be found here on GitHub.