Creating a Data Frame for Book Prices and Titles

Create a DataFrame from collected data and explore methods to scrape multiple result pages.

Efficiently structure scraped data into data frames using Python, and learn strategies for scaling your web scraping efforts.

Key Insights

  • Create structured data frames effectively in Python by transforming scraped data into dictionaries, making it easier to manage and analyze information.
  • After structuring your data, leverage powerful operations such as sorting by price or identifying the lowest-priced items for more insightful analysis.
  • Plan for larger-scale scraping tasks, considering that scraping all data across multiple pages—such as the total of 1,000 results mentioned—requires additional automation and iteration techniques.

Note: These materials offer prospective students a preview of how our classes are structured. Students enrolled in this course will receive access to the full set of materials, including video lectures, project-based assignments, and instructor feedback.

Now that we've got all that data, let's put it into a data frame. And that's not really pretty easy. We can say books is pd.dataframe and we'll make it from a little dictionary where we'll say the titles column is our titles up there and our prices is our prices up there.

And we'll say let's take a look at that books data frame. Here it is. It's looking pretty good.

Okay. Now we do all kinds of work like finding the cheapest one, sort them by price, all kinds of things we could do. Now what we want to do next is right now we only have the first page but there are 50 pages.

Data Science Certificate: Live & Hands-on, In NYC or Online, 0% Financing, 1-on-1 Mentoring, Free Retake, Job Prep. Named a Top Bootcamp by Forbes, Fortune, & Time Out. Noble Desktop. Learn More.

There are overall a thousand results and we're only looking right now at pages at results 41 to 60. How can we scrape all of them? We'll start exploring that in the next video.

Colin Jaffe

Colin Jaffe is a programmer, writer, and teacher with a passion for creative code, customizable computing environments, and simple puns. He loves teaching code, from the fundamentals of algorithmic thinking to the business logic and user flow of application building—he particularly enjoys teaching JavaScript, Python, API design, and front-end frameworks.

Colin has taught code to a diverse group of students since learning to code himself, including young men of color at All-Star Code, elementary school kids at The Coding Space, and marginalized groups at Pursuit. He also works as an instructor for Noble Desktop, where he teaches classes in the Full-Stack Web Development Certificate and the Data Science & AI Certificate.

Colin lives in Brooklyn with his wife, two kids, and many intricate board games.

More articles by Colin Jaffe

How to Learn Data Science

Master data science with hands-on training. Data science is a field that focuses on creating and improving tools to clean and analyze large amounts of raw data.

Yelp Facebook LinkedIn YouTube Twitter Instagram