Data Analysis: Online Class and Coursera by the Numbers

Posted on May 18, 2020
The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.
LinkedIn
Github

Introduction and Motivation

Since the early 2010s, the online learning industry has continually grown in size and is projected to surpass over $240 billion U.S. dollars in revenue by 2022.  What started as a hodgepodge of popular courses from a handful of elite universities has developed into a wide span of topics, ranging from beginner to graduate-level. 
While there are now many competitors vying for a portion of the online learning market, Coursera, founded in 2012 by Stanford computer scientist Andrew Ng, has maintained a dominant position, providing the largest number of online courses by platform and generating the most revenue of any online learning company.  In selecting Coursera as the focus of my web scraping project, one major motivation was the fact that, since the outbreak of Covid-19 this year, many universities and institutions are grappling with how to move in-person courses to an online format. 
While I was contemplating this issue recently, I wondered what types of courses in which subjects have already established themselves in an online format and whether or not there might be any particular types of classes that achieve an higher level of satisfaction in the online format.   

Collecting the Data

I used Selenium to scrape data from https://www.coursera.org.  Coursera offers several categories of products aside from courses and offers courses in numerous languages besides English.  My scraped data was filtered by selecting courses conducted in English.  The full dataset contained 2,728 individual course listings across 11 different subjects.

Overview of Courses

In order to understand what sort of courses are poorly-rated, I first looked at what the distribution of courses were by partner and by subject.  Overall, there were more than 190 partners and Google Cloud represented the highest number of courses offered by a single partner with over 175, while the vast majority of partners offered less than 20. 

Business courses represented nearly one-fourth of the total by subject, with a large number of computer science, health, and data science courses.  Partners consisted primarily of major universities across the world, with a large number of private corporations mixed in.  

Popularity of Courses by Subject and Institution

I next analyzed how the poorly-rated courses varied by subject and institution.  I filtered the courses into two categories: those rated above and below four stars.  In my opinion, using a star rating or a similar rating system with a visual representation is a highly non-trivial business decision, particularly when millions of transactions are conducted daily. 

As a large purchaser of books, when I am browsing on Amazon, I have an almost knee-jerk reaction when I see a rating below four stars over a large enough number of reviews that leads me to believe there may be some serious common dissatisfaction with the product.  Though I don't have any particular evidence to support the idea that a rating below four stars leads to a large decline in purchases vs. competitors, I decided to use this as a threshold for the rest of my analysis.  

Data of Ratings

Overwhelmingly, I found that around 97% of all courses listed on Coursera achieved a rating above four stars.  When looking at boxplots of ratings by subject, we see that 4.5-4.7 stars was roughly where the median for each category stood.  Data science had the lowest rating by subject, with a median of about 4.5, but overall, there doesn't appear to be any obvious discrepancy in any particular subject.  

Looking at how the courses faired by subject and partner, it appears that data science had the largest number of poorly-rated courses at 22, with computer science close behind it.  Interestingly, while business courses represented the largest subject with over 600 offered, only 10 of those were poorly-rated.  I'm not certain why this is the case, and unfortunately with limited data, I was not able to draw any other conclusions. 

If I had access to better data, an immediate analysis I would do is to check what the proportion of classes by difficulty level was in each subject and see whether a larger number of business courses had lower difficulty ratings than some of the other subjects and check how this matches with ratings.  One observation I made when looking at this group was that course titles in the low-rated data science and computer science subjects were highly diverse, with numerous special topic courses. 

It appears that a large number of courses rated poorly are extremely unique, with titles such as "Reinforcement Learning in Finance" or "Introduction to Android Graphics", as opposed to something highly general such as "Introduction to Accounting".  

Poorly Rated courses

Finally, I looked at how the number of poorly-rated courses varied by partner.  University of Colorado Boulder had the most poorly-rated courses by any one institution with six.  I checked to see what these courses were and five of them were associated with a master's degree program in electrical engineering offered entirely online. 

I also checked whether or not the courses were taught by the same instructor, as it is often the case that courses at a university taught by the same professor are poorly-rated due to ineffective teaching and not the content of the course itself.  All of the courses were taught by different instructors, so I was unable to note any definitive reason for the ratings 

Conclusion

While my data was limited due to a large number of missing values for many columns after scraping Coursera, the information I gained was interesting.  Business classes represent the majority of courses by subject, though data science, followed closely by computer science, represent the subjects with the most number of ratings below four stars, with data science at 22 and computer science at 19. 

The vast majority of courses were rated four out of five stars or higher.  I'm not sure how Coursera handles courses with poor ratings, as far as what they will allow if certain courses rack up continuously bad reviews, but I am certain that they have some form of removal policy, since from their perspective, they want to allow students the freedom to review courses without restriction, while at the same time providing a large enough amount of highly-rated courses to succeed as a business.

Data Trends

The only trend I was able to see which intuitively makes sense is that the overwhelming number of the 69 courses rated below four stars were intermediate or advanced level, with only a handful at beginning level.  From my own limited experience taking courses online, as the difficulty of a subject grows, being able to ask questions in-person or perhaps have the instructor explain something from homework or out of a book played a large role for me in making the content easier to understand. 

This is more challenging in an online setting due to the fact that students may be more reluctant to ask questions online and screen-sharing can sometimes be an issue. 

In a future project, I plan to rescrape the website to obtain fewer missing values and do analysis on other interesting questions.  In retrospect, I think a more interesting question is determining how many unique classes there are hosted on Coursera which teach a similar topic and seeing whether there are any large discrepancies in the ratings between these. 

While some students at universities with courses hosted on Coursera are required to only take the ones affiliated with their university, an ordinary user can select whichever course he or she pleases and many introductory courses may compete against each other since these tend to be the largest by enrollment and most common. 

About Author

Chase Rendall

Chase is a data scientist with a background in physics, engineering, and math. He previously was a graduate student at Stony Brook University and worked at Brookhaven National Laboratory in materials science. With a strong background in quantitative...
View all posts by Chase Rendall >

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI