Coursera by the Numbers

Avatar
Posted on May 18, 2020

LinkedIn Github

Introduction and Motivation

Since the early 2010s, the online learning industry has continually grown in size and is projected to surpass over $240 billion U.S. dollars in revenue by 2022.  What started as a hodgepodge of popular courses from a handful of elite universities has developed into a wide span of topics, ranging from beginner to graduate-level.  While there are now many competitors vying for a portion of the online learning market, Coursera, founded in 2012 by Stanford computer scientist Andrew Ng, has maintained a dominant position, providing the largest number of online courses by platform and generating the most revenue of any online learning company.  In selecting Coursera as the focus of my web scraping project, one major motivation was the fact that, since the outbreak of Covid-19 this year, many universities and institutions are grappling with how to move in-person courses to an online format.  While I was contemplating this issue recently, I wondered what types of courses in which subjects have already established themselves in an online format and whether or not there might be any particular types of classes that achieve an higher level of satisfaction in the online format.   

Collecting the Data

I used Selenium to scrape data from https://www.coursera.org.  Coursera offers several categories of products aside from courses and offers courses in numerous languages besides English.  My scraped data was filtered by selecting courses conducted in English.  The full dataset contained 2,728 individual course listings across 11 different subjects.

Overview of Courses

In order to understand what sort of courses are poorly-rated, I first looked at what the distribution of courses were by partner and by subject.  Overall, there were more than 190 partners and Google Cloud represented the highest number of courses offered by a single partner with over 175, while the vast majority of partners offered less than 20.  Business courses represented nearly one-fourth of the total by subject, with a large number of computer science, health, and data science courses.  Partners consisted primarily of major universities across the world, with a large number of private corporations mixed in.  

Popularity of Courses by Subject and Institution

I next analyzed how the poorly-rated courses varied by subject and institution.  I filtered the courses into two categories: those rated above and below four stars.  In my opinion, using a star rating or a similar rating system with a visual representation is a highly non-trivial business decision, particularly when millions of transactions are conducted daily.  As a large purchaser of books, when I am browsing on Amazon, I have an almost knee-jerk reaction when I see a rating below four stars over a large enough number of reviews that leads me to believe there may be some serious common dissatisfaction with the product.  Though I don't have any particular evidence to support the idea that a rating below four stars leads to a large decline in purchases vs. competitors, I decided to use this as a threshold for the rest of my analysis.  

Overwhelmingly, I found that around 97% of all courses listed on Coursera achieved a rating above four stars.  When looking at boxplots of ratings by subject, we see that 4.5-4.7 stars was roughly where the median for each category stood.  Data science had the lowest rating by subject, with a median of about 4.5, but overall, there doesn't appear to be any obvious discrepancy in any particular subject.  

Looking at how the courses faired by subject and partner, it appears that data science had the largest number of poorly-rated courses at 22, with computer science close behind it.  Interestingly, while business courses represented the largest subject with over 600 offered, only 10 of those were poorly-rated.  I'm not certain why this is the case, and unfortunately with limited data, I was not able to draw any other conclusions.  If I had access to better data, an immediate analysis I would do is to check what the proportion of classes by difficulty level was in each subject and see whether a larger number of business courses had lower difficulty ratings than some of the other subjects and check how this matches with ratings.  One observation I made when looking at this group was that course titles in the low-rated data science and computer science subjects were highly diverse, with numerous special topic courses.  It appears that a large number of courses rated poorly are extremely unique, with titles such as "Reinforcement Learning in Finance" or "Introduction to Android Graphics", as opposed to something highly general such as "Introduction to Accounting".  

Finally, I looked at how the number of poorly-rated courses varied by partner.  University of Colorado Boulder had the most poorly-rated courses by any one institution with six.  I checked to see what these courses were and five of them were associated with a master's degree program in electrical engineering offered entirely online.  I also checked whether or not the courses were taught by the same instructor, as it is often the case that courses at a university taught by the same professor are poorly-rated due to ineffective teaching and not the content of the course itself.  All of the courses were taught by different instructors, so I was unable to note any definitive reason for the ratings 

Conclusion

While my data was limited due to a large number of missing values for many columns after scraping Coursera, the information I gained was interesting.  Business classes represent the majority of courses by subject, though data science, followed closely by computer science, represent the subjects with the most number of ratings below four stars, with data science at 22 and computer science at 19. 

The vast majority of courses were rated four out of five stars or higher.  I'm not sure how Coursera handles courses with poor ratings, as far as what they will allow if certain courses rack up continuously bad reviews, but I am certain that they have some form of removal policy, since from their perspective, they want to allow students the freedom to review courses without restriction, while at the same time providing a large enough amount of highly-rated courses to succeed as a business.

The only trend I was able to see which intuitively makes sense is that the overwhelming number of the 69 courses rated below four stars were intermediate or advanced level, with only a handful at beginning level.  From my own limited experience taking courses online, as the difficulty of a subject grows, being able to ask questions in-person or perhaps have the instructor explain something from homework or out of a book played a large role for me in making the content easier to understand.  This is more challenging in an online setting due to the fact that students may be more reluctant to ask questions online and screen-sharing can sometimes be an issue. 

In a future project, I plan to rescrape the website to obtain fewer missing values and do analysis on other interesting questions.  In retrospect, I think a more interesting question is determining how many unique classes there are hosted on Coursera which teach a similar topic and seeing whether there are any large discrepancies in the ratings between these.  While some students at universities with courses hosted on Coursera are required to only take the ones affiliated with their university, an ordinary user can select whichever course he or she pleases and many introductory courses may compete against each other since these tend to be the largest by enrollment and most common. 

About Author

Avatar

Chase Rendall

Chase is a data scientist with a background in physics, engineering, and math. He previously was a graduate student at Stony Brook University and worked at Brookhaven National Laboratory in materials science. With a strong background in quantitative...
View all posts by Chase Rendall >

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 airbnb Alex Baransky alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus API Application artist aws beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep Bundles California Cancer Research capstone Career Career Day citibike clustering Coding Course Demo Course Report D3.js data Data Analyst data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization Deep Learning Demo Day Discount dplyr employer networking feature engineering Finance Financial Data Science Flask gbm Get Hired ggplot2 googleVis Hadoop higgs boson Hiring hiring partner events Hiring Partners Industry Experts Instructor Blog Instructor Interview Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter lasso regression Lead Data Scienctist Lead Data Scientist leaflet linear regression Logistic Regression machine learning Maps matplotlib Medical Research Meet the team meetup Networking neural network Neural networks New Courses nlp NYC NYC Data Science nyc data science academy NYC Open Data NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time Portfolio Development prediction Prework Programming PwC python python machine learning python scrapy python web scraping python webscraping Python Workshop R R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn Selenium sentiment analysis Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau team TensorFlow Testimonial tf-idf Top Data Science Bootcamp twitter visualization web scraping Weekend Course What to expect word cloud word2vec XGBoost yelp