I Know Which Xbox 360 Games Are Coming to Xbox One
Background
I am a huge fan of the Xbox and Microsoft has made a lot of money off of me since they made the Xbox One backwards compatible with Xbox 360 games.My back catalog of games bought through Microsoft’s weekly sales now puts my Steam catalog to shame. My problem is that during these sales, sometimes it’s hard to guess which ones may become backwards compatible in the future. Last Friday was the one year anniversary of the backwards compatibility update and I felt compelled to use my skills as a Data Scientist to make something fun, and hopefully useful.
For those that are unaware, Microsoft has made many Xbox 360 games playable on the Xbox One. They have been making a few games a week compatible because it takes time to test that their software layer works properly with the game. They must also get publishers and rights holders to agree to allow their games to be made compatible. Publishers may not want certain games to be available for many reasons including complicated rights, timing, and remakes or remasters that may lose sales. Microsoft has also decided to not make Xbox 360 games that require the system’s Kinect or other special peripherals (microphones, guitars, skateboards) compatible.
Please also visit the website I created from this project to find the next game you'll buy: Backwards Compatibility Predictor
Scraping Eight Websites
Microsoft surely takes many factors into account when prioritizing their games, but I needed data to analyze so I could think more like them. I decided to collect data from websites using Scrapy, a free Python framework designed to allow a programmer to use various methods to extract data from a web page.
Getting Microsoft's Information on Xbox 360 Games
I grabbed the list of all Xbox 360 games and extracted every single piece of information I could find on their pages including publishers, developers, user review scores and their counts, release dates, Smartglass features, demo availability, multiplayer types, sound features, and the number of add-ons, themes, gamerpics and the like.
Microsoft's Xbox 360 Store
Below is a sample of the Scrapy script I designed to scrape only Microsoft's Xbox 360 store. This is script navigates page by page to find each game's store page and dives into it to gather a variety of information. While in the game's store page, the script employs safety measures to prevent mistakes when a page is out of the ordinary.
Website: Microsoft's Xbox 360 Store
Larry Hryb's Major Nelson Backwards Compatability List
Larry Hryb, the Xbox LIVE's "Major Nelson," Director of Xbox Programming at Microsoft, keeps an up to date list of backwards compatible games on his blog. I notice his list is updated rapidly and is written in a convenient table.
Website: Backwards Compatibility List
Metacritic Xbox 360 Reviews
I then scraped game names, users, and professional scores from Metacritic along with several other factoids like release dates in case another site is lacking.
Website: Metacritic Xbox 360 Reviews
Microsoft's UserVoice Feedback System
I know Microsoft has said in the past that they look at UserVoice votes to help make choices so I scraped the votes and forum comment counts.
Website: UserVoice Feedback (Completed) (Closed)
Exclusive Games from Wikipedia
Microsoft has prioritized their first party titles and I believe exclusivity should increase the likelihood of becoming backwards compatible. I retrieved the differentiations of Xbox 360 exclusives (console and regular) from Wikipedia.
Website: Wikipedia's List of Exclusive Games
Making a List of Kinect Support and Peripherals
I grabbed the names and requirements of Xbox 360 Kinect games (required or supported) from Wikipedia. Remember that this is important because Microsoft says they won’t support Kinect required games, but games with optional Kinect functionality left me wondering.
Kinect Games from Wikipedia
Website: Wikipedia's List of Kinect Games
Required Peripherals
I manually compiled a list of games that required peripherals. This was greatly helped by UserVoice, which includes games that will not be made backwards compatible for various reason.
Making a List of Remasters
It is important to find how much an Xbox One version of the game in the form of remasters, remakes or cross-platform releases had any correlation. I needed to make a manual list of remasters, but I decided to scrape certain sites for references to keep me up to date. I started with the list of Xbox One games off Microsoft’s site and I supplemented it by scraping a GameInformer article that gets regularly updated with the latest remasters. Based off these and further research, I manually made a new list of games that could be found on both platforms.
Game Informer
Website: GameInformer Article on Remakes
Microsoft's Xbox One Store
Website: Microsoft's Xbox One Store
Making the Ultimate Dataset
After scraping these sites, I felt ready to start processing the data into one master dataset. I used R for its ability to easily manipulate and organize data. I imported all the files into data frames from .csv files I had made from the scraping scripts. In R, I recommend using dplyr for grouping and selecting rows and columns, data.table for fast file reading and large data frame access, stringr for fast string manipulation and DataCombine for combining like columns. For parallelization, I recommend doParallel because of its broad compatibility with operating systems and foreach to parallelize your for loops.
Finding more info in the data
I looked for ways I could split the data even further. Examples include splitting console exclusive and regular exclusives into unique columns, separating multiplayer capabilities and filtering UserVoice labels into “In progress” and “Closed” categories for analysis.
Merging the data
One of the most time consuming parts of this project involved merging the data because of the game’s names. The names of games on Microsoft’s sites were abbreviated or shortened due to character limits on their own website or are only available as “XYZ Edition”. Users often used abbreviations or modified names when submitting to UserVoice. Wikipedia articles sometimes used the international name of a game. Punctuation, trademark/copyright marks, and differences in spelling prevent one to one matching.
The solution was to make a new column of names where I would remove as many obstacles as I could so that when datasets are merged, there are the fewest differences possible. I began by removing words including “the,” “edition,” “special,” “full game,” “free to play” and many more. I then removed pluralization and possession from words and converted numbers to roman numerals (“Assassin’s Creed 2” to “Assassin Creed II”). I also removed all non alphanumeric letters as well as spacing and capitalization. For many games, the differences were too great and had to be renamed before any of these processes were done (”COD:MW2” to “Call of Duty: Modern Warfare 2”). To solve this last problem, I wrote a dictionary of 276 games to translate to their proper names.
Wrangling Publishers
Microsoft has said they need publishers and rights owners to agree to make their games backwards compatible, so it is important to ensure publishers are up to date. Publishers are also written inconsistently, but luckily many use unique names that make it easy, like “Konami” or “Namco Bandai.” This was mostly a matter of finding matching words in the name to update them to be consistent. During the decade since some of these games were released, we have seen publishers rise, fall and get acquired. I researched and updated publishers and game to be closer to their modern day states as Microsoft would need to do. This is difficult to do because some of his information is private or complicated by private agreements, so there are probably a few inaccuracies or polishing that could still be done.
Missing values
As I got toward the end of polishing, I realized I was still missing quite a few data points. I went back through my scrapers and grabbed even more information so I could fill in missing spots. However, that still left many missing values. There are 3331 missing values out of 66312 values (5%) though some are less important than others because I found later that I would not be using them (developers). This results in missingness, which occurs when at least some of an observation’s values are not present within the dataset. This weakens or completely stops some statistical and machine techniques because the algorithm won’t understand how to crunch the numbers. Deleting games with a missing value doesn’t help because it severely limits the amount of data we have for analysis, especially if the missingness is not completely at random. In this problem, we would lose 1316 out of 1842 lines of data.
Imputation
To solve this, we use imputation to fill in the missing values. The data is missing at random because certain points are missing from the dataset because games weren’t reviewed, submitted to uservoice, or the point was simply missing. There isn’t a pattern to fill in the continuous or categorical values and it would be inappropriate to assign an average score where review scores are missing. I used K nearest neighbors (KNN) to fill in missing values. KNN makes a tessellated boundary using existing data points and judges where in the boundary the missing point is. The formula uses the K closest observations to the missing data point, and predicts the majority class as the outcome. I used the square root of the number of rows (≈ 43) as a K value to strike a balance of being robust to outliers, stable boundaries and local variations.
Prediction
I separated the data into observations that will either never be backwards compatible or are already backwards compatible. To predict backwards compatibility, this is now a supervised learning problem. I used logistic regression using the relations between the portion of the data that says whether or not a game is backwards compatible. I needed to use a generalized linear model in this regression because I am using both categorical and continuous data to predict the categorical outcome of whether a game would be made backwards compatible. I used forward AIC on a scope from an empty dataset except the game name and a few irrelevant predictors up to all the data minus those useless predictors to find the most effective combination of the data which finds the most significant predictors for analysis.
Conclusions
The most significant influences are having a higher Metacritic score, having a version on Xbox One, a lower price, positive user reviews on Xbox’s website, and having a demo. Uservoice votes are also very significant, but lag slightly behind the others. There is also correlation for having a newer release date, more avatar items, fewer game add ons, and more reviews. Retail only games are also negatively related and has a somewhat significant relation. Also, having a Kinect requirement or required peripherals brought the probability to zero.
These results pique my interest. I wonder if it is easier to test and approve games by using their demos. I am also glad that Uservoice votes positively significantly influenced their priorities. I am also not surprised they try to get higher reviewed games approved first to get more classics onto their system. I imagine having a lower price and being available on their site makes them want to prioritize games people are likeliest to buy digitally on impulse.
Please check out the results of my work on the Backwards Compatibility Predictor website.
Future Work
- Implementing a method of machine learning called clustering to examine the data
- Implementing a neural network if given the time
- Automatically perform update scrape data and machine learning predictions
- Scraping sales to give consumers advice on which Xbox 360 games to buy that week
- Game covers are scraped but not implemented, which I feel would make the website prettier
About Author
Related Articles
Leave a Comment
