In this class, you’ll learn how to automate time-consuming tasks using Python, such as gathering data from the internet. Whether your interests lie in social media, e-commerce, real estate, travel or government affairs, the ability to instantly collect up-to-date data across the web can be a big advantage.
Use Python code to grab precisely the data you want from a website. Along the way, you’ll learn how to write loops so that your web scraping code can process a large number of pages for you.
These fundamentals will be coupled with real-life examples of programs that can scrape valuable data from major websites. The class will also cover common pitfalls, how to run your programs on a regular schedule, and different formats for storing your scraped data.
This course is perfect for any programming beginner to become introduced to the powerful world of coding, and is a great fit for students with a clearly defined learning goal of using Python for web or software development. This course is included in our Data Science Certificate.
Participants should have knowledge equivalent to our Intro to Python Programming Bootcamp or Python for Data Science Bootcamp courses.
From our hands-on training style to world-class instructors and custom-crafted curriculum, we deliver results our alumni are proud of.
Learn the concepts and skills covered in this course or your tuition is on us. See details and terms & conditions.
Work on projects proven to boost retention
Refined over many cohorts for an optimal learning experience
The teaching method at Noble Desktop is perfect and the classes provide you with infinite knowledge that makes you eager to take everything they offer. I love Noble!
—Ivonne Ackerman
Experienced educators who are driven to help you succeed
Refresh the materials for free within one year
Attend this course in-person in NYC or Live Online via Zoom. Engage with expert instructors, ask questions, and get feedback on your exercises and projects.
Unlike other providers, these are not mass open-enrollment classes. At Noble Desktop, you’ll learn in small groups (typically 8-15 students) and receive personalized attention.
Get face-to-face interaction with an instructor and other students when you learn at our NYC campus. Courses are hands-on with a computer and software provided.
Get the same interactivity and access to the instructor as in-person students. There are no extra fees and we’ll work with you to ensure your remote setup is perfect.
Our instructors are verterans of their field and experienced educators, driven to help you succeed.
Upon completion of this course, you’ll receive an official certificate testifying to your mastery of the curriculum. We’ll send you a link where you can download your certificate, share it online with your friends, post it to your professional network on LinkedIn, and view all your earned certificates. Congratulations on your achievement!
Shareable on
We offer a single free retake of the the class within a year.
You can also access recordings of each session in your student portal within one business day of the session’s end. Recordings are available for one month after the session.
Participants should have knowledge equivalent to our Intro to Python Programming Bootcamp or Python for Data Science Bootcamp courses.
This course does not qualify for payments plans or student financing. See our Payment Plan FAQ to find related programs that qualify.
You may attend this training virtually (online) at the scheduled time the course is offered (New York, Eastern Time).
You have some options:
Although it is less talked about, automation might be the single biggest payoff for learning Python. By “automation,” I mean code that directly replaces human labor. A computer program, in other words, that finishes tasks you normally do by hand.
The personal benefits of automation are two-fold: your work is less tedious and your day is more productive. Let’s say that, as an example, you need to build a report every month. Being able to automate that process means less typing for you, but also a faster delivery to everyone who reads your report. That’s a win-win scenario.
On a larger scale, there are people who have managed to automate their entire job. Journalists have called this strategy “self-automation” or “auto-automation.” Success means that you can go into work, run an automation script, and spend the rest of the day enjoying Reddit and YouTube. This raises new questions, of course. Are you obligated to tell your boss? Does goofing off all day really make you happy? One thing should be clear though: if code can do your work, you’re wasting time doing it by hand.
Automation can scale to the size of an entire business as well. The websites NomadList and RemoteOK cater to tech workers living abroad, and together bring in a million dollars a year in revenue. Despite supporting thousands of users, both websites are run by a single programmer named Pieter Levels. How is such a thing possible? Levels uses over 200 automated scripts (also known as “cron jobs”) to handle most of the maintenance.
Perhaps most surprising is that automation is relatively easy compared to other uses for Python. Data science and web development, the most popular uses for Python, involve lots of user data and abstract concepts. Automation is easier for beginners to pick up because these sources of difficulty are typically not necessary.
User data causes problems because it is unpredictable. A data scientist, for example, can easily spend a hundred hours per year just correcting typos in their data. When data is entered by hand, there are no guarantees. Human error is also an issue for web developers, but even worse is input that is intentional and malicious. In a SQL injection attack, a user enters code into a normal form field to gain control of the website’s database. An attack like this can bring down your website and expose sensitive data. Because automation involves little to no human input, there are fewer risks like these to consider.
Since automation scripts mimic human behavior, it is also more concrete and easy to understand.You are typically simulating interactions you’ve already done by hand, rather than diving into new technical concepts. Your code might move files around, enter text into a field, or send an email. These are all things you, as a human, have done before. By contrast, there is no human version of what servers and databases do in web development. These are entirely new concepts! This is even more true in data science, which involves higher-level math and statistics. To get started with automation, on the other hand, you don’t need as many abstract concepts because you are just simulating things you have already done.
If you are excited about getting started with automation, there are several types of projects you could look into. Here are a few ideas:
1. Webscraping: This term refers to pulling data from websites through code rather than by hand. Do you search the same sites repeatedly, looking for deals? Or do you grab quarterly earnings from the same site, over and over? These are the kinds of tasks you can automate with webscraping.
The easiest way to get started with webscraping is through Selenium. Selenium uses an internet browser the same way you do, so it is easy to translate your usual actions, like clicking the mouse or typing in text, into your Python code.
2. Emails and Text Messages: It’s often useful to send an email or text message automatically, using your Python code. If you find a great deal online through webscraping, for instance, how do you alert yourself in time? Automated text messages are also useful for sending scheduled reminders.
The easiest package for sending e-mail is Yagmail, which is specifically designed to work with Gmail accounts. For text messages, you can either use the email-to-text gateway provided by your phone carrier or a service called Twilio.
3. Testing: If you have a website, automation can give you peace of mind from worrying whether your site is up and functioning properly. At the most basic level, you can have a Selenium script running every few minutes to validate that your home page loads. More advanced testing might make sure that other features, such as important forms and videos, are also working as designed.
If you’d like to learn how to automate tasks with Python, sign up for our 6-hour Python for Automation class in NYC. This project-based class will focus on building an automated script to check prices and send alerts using Selenium and Yagmail.
If you’re interested in diving deeper with data, this course is featured as part of the Data Science Certificate. The certificate includes an in-depth class on using Python for data science as well as a SQL Bootcamp to make you confident working with databases.
