Yelp Facebook LinkedIn YouTube Twitter Instagram

Python for Automation

Learn how to gather, store, and analyze web data

In this class, you’ll learn how to automate time-consuming tasks using Python, such as gathering data from the internet. Whether your interests lie in social media, e-commerce, real estate, travel or government affairs, the ability to instantly collect up-to-date data across the web can be a big advantage.

Use Python code to grab precisely the data you want from a website. Along the way, you’ll learn how to write loops so that your web scraping code can process a large number of pages for you.

These fundamentals will be coupled with real-life examples of programs that can scrape valuable data from major websites. The class will also cover common pitfalls, how to run your programs on a regular schedule, and different formats for storing your scraped data.

This course is perfect for any programming beginner to become introduced to the powerful world of coding, and is a great fit for students with a clearly defined learning goal of using Python for web or software development. This course is included in our Data Science Certificate. There is no math or coding requirement necessary, but prior knowledge of HTML & CSS equivalent to our HTML & CSS workshop is required. 

Prerequisites: Knowledge equivalent to our Intro to Python Programming or Python for Data Science Bootcamp courses.

  • Small Classes
  • Computer Provided
  • Top Instructors
  • Free Retake

Take this class as part of a certificate program and save:

Automate Repetitive Tasks in Python

This class will teach students how to scrap and mine the web for any type of data. Whether you are in academics, business, journalism or anything in between  - this class will be extremely beneficial. Students will learn two major skills in this course: Python and Webscraping. Students will first be introduced to the most powerful programming language in the world and later learn one of its many libraries. If you are interested in pursuing this skill further, check out our Python for Data Science Bootcamp.  

  • “Excellent class. By the end of the day, we understood how to scrape a website for information and send results by email, text, or saved to the local computer. Highly useful.”

    Rahmaan Mwongozi

  • “Noble Desktop is definitely one of the best courses I've taken. Class was so informative and interactive, I will definitely come back. ”

    Joyce Li

Learn How to Write Code to Collect Data

In this class, students will be introduced to Python and the Beautiful Soup library. Beautiful Soup is a Python library that makes it easy for programmers to collect information from web pages. 

Learn to Use Python to Compare Price Data on Amazon

Using Python, you will learn how to write code that can scrape price data off of Amazon. This will allow programmers to research the market and find the best possible deal! 

Find Jobs using Webscraping with Python

Job searching is a tedious process but Python can help with that! In this class students will learn how to write programs that automatically strip job information off the internet! 

Practical Automation with Python

Although it is less talked about, automation might be the single biggest payoff for learning Python. By “automation,” I mean code that directly replaces human labor. A computer program, in other words, that finishes tasks you normally do by hand.

The Upside of Automation

The personal benefits of automation are two-fold: your work is less tedious and your day is more productive. Let’s say that, as an example, you need to build a report every month. Being able to automate that process means less typing for you, but also a faster delivery to everyone who reads your report. That’s a win-win scenario.

On a larger scale, there are people who have managed to automate their entire job. Journalists have called this strategy “self-automation” or “auto-automation.” Success means that you can go into work, run an automation script, and spend the rest of the day enjoying Reddit and YouTube. This raises new questions, of course. Are you obligated to tell your boss? Does goofing off all day really make you happy? One thing should be clear though: if code can do your work, you’re wasting time doing it by hand.

Automation can scale to the size of an entire business as well. The websites NomadList and RemoteOK cater to tech workers living abroad, and together bring in a million dollars a year in revenue. Despite supporting thousands of users, both websites are run by a single programmer named Pieter Levels. How is such a thing possible? Levels uses over 200 automated scripts (also known as “cron jobs”) to handle most of the maintenance. 

Why Automation is Good for Beginners

Perhaps most surprising is that automation is relatively easy compared to other uses for Python. Data science and web development, the most popular uses for Python, involve lots of user data and abstract concepts. Automation is easier for beginners to pick up because these sources of difficulty are typically not necessary.

User data causes problems because it is unpredictable. A data scientist, for example, can easily spend a hundred hours per year just correcting typos in their data. When data is entered by hand, there are no guarantees. Human error is also an issue for web developers, but even worse is input that is intentional and malicious. In a SQL injection attack, a user enters code into a normal form field to gain control of the website’s database. An attack like this can bring down your website and expose sensitive data. Because automation involves little to no human input, there are fewer risks like these to consider.

Since automation scripts mimic human behavior, it is also more concrete and easy to understand.You are typically simulating interactions you’ve already done by hand, rather than diving into new technical concepts. Your code might move files around, enter text into a field, or send an email. These are all things you, as a human, have done before. By contrast, there is no human version of what servers and databases do in web development. These are entirely new concepts! This is even more true in data science, which involves higher-level math and statistics. To get started with automation, on the other hand, you don’t need as many abstract concepts because you are just simulating things you have already done.

Automation Projects

If you are excited about getting started with automation, there are several types of projects you could look into. Here are a few ideas:

1. Webscraping: This term refers to pulling data from websites through code rather than by hand. Do you search the same sites repeatedly, looking for deals? Or do you grab quarterly earnings from the same site, over and over? These are the kinds of tasks you can automate with webscraping.

The easiest way to get started with webscraping is through Selenium. Selenium uses an internet browser the same way you do, so it is easy to translate your usual actions, like clicking the mouse or typing in text, into your Python code.

2. Emails and Text Messages: It’s often useful to send an email or text message automatically, using your Python code. If you find a great deal online through webscraping, for instance, how do you alert yourself in time? Automated text messages are also useful for sending scheduled reminders.

The easiest package for sending e-mail is Yagmail, which is specifically designed to work with Gmail accounts. For text messages, you can either use the email-to-text gateway provided by your phone carrier or a service called Twilio.

3. Testing: If you have a website, automation can give you peace of mind from worrying whether your site is up and functioning properly. At the most basic level, you can have a Selenium script running every few minutes to validate that your home page loads. More advanced testing might make sure that other features, such as important forms and videos, are also working as designed.

Learn Python Automation at Noble Desktop

If you’d like to learn how to automate tasks with Python, sign up for our 6-hour Python for Automation class in NYC. This project-based class will focus on building an automated script to check prices and send alerts using Selenium and Yagmail.

If you’re interested in diving deeper with data, this course is featured as part of the Data Science Certificate. The certificate includes an in-depth class on using Python for data science as well as a SQL Bootcamp to make you confident working with databases.