Value-based PC Part Selection

Posted on May 14, 2017

The skills the authors demonstrated here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.

The final PC build application can be accessed via this link.

For people looking for more power and upgradability than a laptop can provide, desktop computersΒ offer a reasonable alternative. Unfortunately, the pre-built computers that major manufacturers sell are considerably overpriced considering the quality of their components. For example, gaming computers will come with high end processors and graphics cards packaged with the cheapest available power unit and motherboard. Cutting cornersΒ does not make much sense when it can lead to your computer bursting into flames.

What's the alternative? BuildΒ your own PC!

Without prior experience, entering the world of desktop computers can be daunting without guidance. To begin, how do we know that this collection of parts willΒ function properly when assembled? There are eight key components that comprise the core of a PC build, as shown in the basic chart below.

Example compatibility between various PC parts

Basic compatibility chart for PC components

In the past,Β consumers would have to carefully read product pages and spec sheets toΒ be roughly 90% certain that everything would work together. Enter PCPartPicker.com. In mid-2014, PCPartPicker becameΒ the go to site for checking compatibility between all parts of a build. As of January 2017, this site was valued at $1.9 million with roughly 860,000 daily page views.

The other obstacle to PC building is finding "value", i.e. parts that are reliable and produce the promised specifications (specs). As of now, the only way to find value was to use a build guide that gives suggestions about trusted brands. These can be biased heavily by popularity and do not inform the consumer whereΒ generics are just as good as name-brands.

Motivation:Β makeΒ value-based PC part selectionΒ easier viaΒ an automated recommendation engine

Scraping PCPartPicker with PhantomJS and BeautifulSoup

To build such a recommendation engine, we need dataΒ onΒ eachΒ available product. Since PCPartPicker contains specifications and ratings on all relevant components, as well as updated prices from major retailers, it is the logical choice to scrape for information. Combining simpleΒ PhantomJS, bash, and BeautifulSoup scripts made this process relatively quick and easy.

First, gather all of the product URLs from each subcategory page (here for the CPU subcategory):

Next, parse each productΒ URL and appendΒ it to a compilation file:

Finally, use the scraped URLs to scrape each individual product page:

After all of this scraping the end product is aΒ collection of pages such as the one below (in raw HTML format). Of specific interest for this project:

  1. Name: the product name, sometimes containing information not found elsewhere on the page
  2. Ratings: average value and number of ratings on PCPartPicker
  3. Specifications: a ragged list of features for each component,Β varying wildly in content
  4. Price: the lowest value of the Prices table minus one cent and not accounting for shipping

Data preparation and value analysis with Pandas

The data produced from the scraping process was relatively unstructured and extraction of specifications required RegEx and unit conversion. Since this is a standard and well-covered process, I won't delve into the details here.

More interesting is the determination of the most valuable quantities for each component. A link to the slidesΒ containing analysis can be found in the sidebar of the final application.

Example analysis: Memory

Random access memory (RAM, memory) isΒ on-motherboard storage used for quick access of frequently used data. In simple terms, having more RAM means that your computer can run more processes/applications at the same time without slowing down.

For desktop computers, RAM is sold in sticks with capacity between 2 and 32GB. Typically 2 sticks are used for dual-channel mode which increases the overall performance, i.e. 2 sticks of 4GB run faster than 1 stick of 8GB RAM. While other components such as CPU and GPU have only two or three competing companies, RAM seems to have more options. Can the scraped data tell us what to look for in terms of RAM value?

Below is the correlation heatmap for the memory dataframe. The most significant take away is the extremely strong correlation (0.94) between the product price ($) and the total memory (GB). The presumption from this chart is that there is limitedΒ advantage to buying higher priced RAM, especially considering the marginal correlation (0.19) between price and rating value.

Correlation heatmap for memory dataframe

Our suspicions are confirmed when plotting total memory v. price in a scatterplot. We see that this strong correlation is reinforced by a strong linear relationship. Highlighted in red is the effect of branding: sellingΒ a product with similar specifications at a higher price than other manufacturers. In essence, there is limited advantage to paying for a name-brand RAM supplier: it is better to get the cheapest possible RAM.

Scatterplot: total memory v. price

As mentioned there is no "trusted name" in RAM production. Logically the above relationship makes sense since more competitors should lead to lower prices for the consumer assuming that production is relatively trivial.

Database formation and querying with SQL

Instead of reading the entirety of our dataΒ into memory every time we want to return a build recommendation, we can save our cleaned datasets into a SQL database. This way elementsΒ can be queried as needed, i.e. returning one compatible component instead of the entire collection. The SQLAlchemy (create_engine) and Pandas (to_sql) packages makes this process straight-forward in Python:

Here we have created 8 distinct tables in the pcpart.db which we can query in R using the RSQLite package. Specifically, we can formulate a general SQL query as shown below.

Notes:

  1. Spec_vec contains theΒ column names we want to get from the table of interest
  2. Filter_str contains any conditions that need to be placed on the component of interest (e.g. price > 0 by default)

Building a recommendation engine with Shiny

TheΒ problem of selecting compatible parts for a PC build can be described theoretically via graph theory.Β The eight core components and their compatibility form an undirected graph where the motherboard (6 edges) and the CPU (4 edges) are the most significant nodes. The PSU isΒ linked to all other parts by the relation that the PSU must output more wattage than the sum of the other components. In practice, the majority of PSUs can handle all possible combinations of other components with ease and thus the links between PSU and other components are weak.

There are now two possible components that can be the starting point of our build selection: the motherboard or CPU. Since the motherboard has few numerical specifications that can be easily translated to value, I selected the CPU as the "pivot point": the component that is selected first which influences the choice of every other component.

Once the CPU is selected, the motherboard is determined via socket compatibility and the remaining components are selected in parallel, assuring compatibility of the final build. In practice, this means that our parameter space has been greatly reduced (for better or worse) by selecting CPU from a smaller pool of high value/high rating/high spec components. If we want to map the entire parameter space of builds to find overall build value, we would need to construct an algorithm to loop through all CPUs and record all possible build combinations. Alternatively, we could pivot on a different component (such as the motherboard) and compare build value to see if one is more comprehensive or preferable than the other.

There are currently three operational modes for the recommendation engine withΒ various levels of implementation.

  1. Rating-first
  2. Spec-first
  3. Price-first

Rating-first

This is the simplest mode:Β after compatibility is ensured, the highest rated component is selected. In the default state of the app, this mode can be accessed by sliding each component's 'rating weight' slider to the 1 position.

This option is most useful for components where value is harder to determine: motherboards and cases. By putting price limits on these components, an appropriate choice can be selected.

Spec-first

The default value of the application is spec-first: select the highest value component after compatibility is ensured.

For the majority of components we can assign a numeric variable which indicates the objective value. In this early-stage of development, value is determined by the most significant numeric factors of each component and implemented by SQL "order by" commands. Given more time, a rigorous algorithm for determining value of each component would be developed, ideally with an overall price filter to generate the best build per dollar value.

Price-first

As is implied, price-first selects each component first by compatibility, then price, and finally by rating or spec value. This can be enabled in the app at the bottom of the first filter tab via the "price priority" checkbox.

While most restrictive, this option has the most value to a potential user: rating values and price ranges can be adjustedΒ to generate more realistic builds than can be produced with the first two modes.

Example Usage

  • Enable "price priority"
  • Note that 2GB of memory is unacceptable for a modern operating system
    • Increase the price minimum to ~$25 for 4GB
    • Increase the price minimum to ~$50 for 8GB
  • Change the PSU to fully modular with Bronze efficiency
    • ProducesΒ a good quality, cheap unit for only $35
  • Increase theΒ minimum rating on all components slider to get a well reviewed CPU and motherboard
  • Remove the overclockable filter on GPUs to reduce the price on that component

Final result: $283 build with an average customer rating of 4.41

In practice, it would be useful to have a couple of dropdown choices for parts of interest such as CPU and GPU so that the user could generate compatible value-based builds with those selections in mind. For now, these users could use this tool to assess the value of other components.

Future development will focus on improving the algorithm for value-based selection and implementing user interaction as mentioned above. Another possible development route would include setting use-based filters such as high-end gaming, web-browsing-only, etc.

About Author

Sam O'Mullane

Sam O'Mullane is a multidisciplinary scientist who is passionate about making data science accessible to a wide audience. Sam received his MSc in Quantum Fields and Fundamental Forces from Imperial College London (Th. Physics) in 2013 and PhD...
View all posts by Sam O'Mullane >

Related Articles

Leave a Comment

www.season7gameofthronesonline.com August 18, 2017
I think Arya now have her eyes set on Littlefinger though Littlefinger has somehow mislead Arya onto Sansa but I don't think there will be a fight between the two ladies. Game of Thrones Season 7 Stream

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI