What does it take to predict MLB Starting Pitcher Performance accurately?

Contributed by Joseph Wang. He is currently in the NYC Data Science Academy 12 week full time Data Science Bootcamp program taking place between April 11th to July 1st, 2016. This post is based on his third class project - Web Scraping (due on the 6th week of the program).

Introduction: As quoted from Wikipedia: "The history of baseball  can be traced to the 19th century, when amateurs played a baseball-like game by their own informal rules using home made equipment. The popularity of the sport inspired the semipro national baseball clubs in the 1860s. During the Civil War a rumor began that Abner Doubleday started baseball ..."  The sport we know as baseball today has become data intensive regarding answering questions such as which players to recruit based on the needs of  different professional baseball clubs. Due to the long history of Major League Baseball (MLB) games in the United States, and the convention of good record keeping for each official game and even for each position, there is plenty of data to answer data science questions. For instance, is it possible to decide the likelihood that a player is going to be able to maintain a high performance for several years, that a player in a minor league would be successful in the MLB, and what a player can bring into the performance of the ball clubs as a whole, especially with the data revealing the performance of each individual player as well as the chemistry of the team as a whole.  Here, I intend to answer the questions on what factors decide the winning percentage of a starting pitcher, and if it is possible to predict the winning team with statistical significance.

Data sampling and filtering: To start the investigation, I found the structured data at the MLB websites which recorded all the typical and derived performance indices for each pitchers who appeared in MLB games from 1876 to 2016. We would concentrate on the records from 2008 to 2015. To scrape the data from the website, I use the Scrapy + Selenium packages based on Python language to crawl through each data entry which was found many layers below the main website page. The response variable WR, which characterizes the winning ability of a staring pitcher, is defined by the expression WR=(W-L)/(W+L), where W were the games the pitcher was responsible for a win, and L were the games the pitcher was responsible for a loss. A pitcher would get a negative score -1 when there were no games won under his duty. The index always varies between 1 and -1. I consider this index is more informative than the typical index WPCT=(W/W+L) where the pitchers without wins will score equally no matter how many lost games were committed.

The predictor/input variables are the chosen ones that cannot be directly related to each other with certainty by known mathematical equations to avoid redundancy. To sample the data only for starting pitchers, we select the row data labeled with starting records. In addition, only the staring pitchers who have pitched for more than 100 innings per year are chosen to be learned. The 14 baseball predictor variables defined here includes ERA, R , ER , HR, WHIP, GO_AO, OBP , SLG, OPS , K_9 , BB_9 , H_9 , K_BB , and P_IP as explained in Appendix I.

                                                                            FIG. 1: Color Map for Correlation Matrix

Exploratory Data Analysis: Let us first investigate the linear correlation between the response WR and other predictors. As shown in FIG. 1, the majority of predictors are negatively correlated with WR, except the indices K_9 and K_BB, indicating the pitcher’s dominance and ability to throw strikes. Unfortunately, it appears that all the negative predictors are strongly correlated with each other. This may impede the predictive power of linear models based on all these predictor variables. To build a multiple linear regression model, strongly correlated predictors need to be treated properly to avoid large variance inflation which makes the regression coefficients statistically unreliable to have descriptive values.  In addition, we observe some weak nonlinear correlation between predictor variables, which may require nonlinear regression models or machine learning models, such as Random Forest, which can handle the nonlinear correlation. Are these predictors good enough for accurate WR prediction? This is what we will explore in the next section by using machine learning algorithms

                                                                                           FIG. 2: Feature Selection by PCA

Models: In order to evaluate how well the models we construct are effective, we split our observations into 70 percent training set and 30 percent test set. The optimization is done through the training data, and the prediction of errors is done through the test set. The first supervised machine learning model we should try is multiple linear regression. Due to the large number of predictors, it is not wise to search all the possible combinations and try to find one that minimize the errors for the test data. We will be guided by the feature subset selection from principle component analysis (PCA). For predictors which cluster together, we try to minimize the number of the selection in that cluster to minimize the effect of variance inflation. In FIG 2, for example, I only consider one or two predictors in the lower right quadrant. To actually select the best combination, we can refer to FIG 1 to find the predictor which is more weakly correlated with the other cluster members. As we can see, HR is more weakly correlated with other cluster members in the quadrant as shown by the farthest distance in FIG. 2 and the correlation matrix in FIG. 1. This will be the minimal predictor to include in the quadrant. The next predictor to select could be H9 or SLG since it keeps the distance from other members better. We can confirm it through the correlation matrix in FIG 1 by counting the number of dark blue balls in the off-diagonal elements. The rest of the cluster members will not be used for predictive modeling for multiple linear regressions. In the upper-right quadrant, we would select OBP or WHIP along with BB_9 and P_IP. In addition, we will consider all the predictors at the left quadrants since these predictors are very far from each other in FIG. 2 with weak linear correlation. Under the scenario, we are left with four regression models as WR~K_9+K_BB+GO_AO+BB_9+P_IP+HR+OBP/WHIP+H_9/SLG in R language. We find the overall prediction for these four models are not satisfactory suffering from intermediate variance inflation. To make progress, one can reduce one more predictor variable with models as WR~ K_9+K_BB+GO_AO+BB_9+P_IP+HR+OBP/R/ERA/OPS/WHIP. We end up with the best predictive model as  WR~ -0.008339*K_9+0.019755*K_BB+0.045393 *GO_AO+0.011299*BB_9+0.002066*P_IP+0.322154*HR-1.596730*R+0.678461 with the mean squared error(MSE) given as 0.03853194 and the correlation between predicted values and observed value in the test set given by 0.6270265.  If we further reduce one more variable, we cannot find any models with better prediction. As we mentioned earlier, there are some weak nonlinear correlation between predictor variables especially the variable K_BB with other ones. We suspect if the machine learning with random forest will get us a better prediction. We select the predictor variables: K_9, K_BB, GO_AO, BB_9, P_IP, HR, R, the test MSE(out of bag) is 0.04544826 which is not better than multiple linear regression for our data. This may be due to the curse of dimensionality where the number of observations N=1418 we have in our training set after filtering is scarce in the large dimension of the phase space spanned by WR and 7 predictor variables where the tree regression becomes too rough.

Conclusions and Discussions: We have scraped the baseball indices representative of the performance of a starting pitcher at the MLB website, and intend to predict the winning ability of a starting pitcher based on predictor baseball indices. To my best, we can predict WR with 7 predictor indices by a multiple linear regression model with around 60 percent accuracy of statistical significance. To achieve this result, we used PCA to reduce the number of predictor variables by half (from 14 to 7).  To further improve the prediction of the performance of the starting pitcher, one would need to go beyond the predictor variables included in the discussion, including the team winning percentage for each starting pitcher in the observations. Enlargement of the sampling data while fixing the number of predictor variables may be helpful for other machine learning algorithms such as random forest to shine so that the "curse of dimensionality" can be minimized.

Appendix I:

IP: The total number of innings pitched by the pitcher. The pitcher is credited for one-third inning for each out he records.

SLG: Slugging Percentage against a pitcher

ERA: The average number of earned runs allowed by a pitcher; total number of earned runs allowed multiplied by 9 divided by the number of innings pitched

OPS: On-base Percentage plus Slugging Percentage against a pitcher (OBP+SLG)

R: The average number of hits allowed by a pitcher  per inning

K_9: The average number of strikeouts per 9 innings by a pitcher

ER: The average number of earned runs allowed by the pitcher per inning

BB_9: The average number of walks issued per 9 innings by a pitcher 

HR: The average number of home runs allowed by a pitcher per inning

H_9: The average number of hits allowed per 9 innings by a pitcher

WHIP:   The average number of walks and hits by a pitcher, Hits plus walks allowed divided by innings pitched

K_BB: K_9/BB_9

GO_AO: Ratio of outs made on the ground v. the air by a pitcher

P_IP: Number of pitches thrown per a pitcher divided by the number of innings pitched (P/IP)

Appendix II(Python codes for web scraping):

items.py:

# Define here the models for your scraped items

# See documentation in:  http://doc.scrapy.org/en/latest/topics/items.html

from scrapy import Item, Field

class MoviesItem(Item):

# define the fields for your item here like:

# name = scrapy.Field()

Player = Field()

GS = Field()

W = Field()

L = Field()

IP = Field()

GS = Field()

ERA = Field()

R = Field()

ER = Field()

HR = Field()

WHIP = Field()

WPCT = Field()

GO_AO = Field()

OBP = Field()

SLG = Field()

OPS = Field()

K_9 = Field()

BB_9 = Field()

H_9 = Field()

K_BB = Field()

P_IP = Field()

pipelines.py:

from scrapy.exceptions import DropItem

class ValidateItemPipeline(object):

def process_item(self, item, spider):

if not all(item.values()):

raise DropItem("Missing values!")

else:

return item

class WriteItemPipeline(object):

def __init__(self):

self.file = open('MLB2015.txt', 'w')

def process_item(self, item, spider):

index=0

for index in range(49):

line = str(item["Player"][index])+' '+'\t'\

+ '\t' + str(item['W'][index])\

+ '\t' + str(item['L'][index]) \

+ '\t' + str(item['IP'][index]) \

+ '\t' + str(item['GS'][index]) \

+ '\t' + str(item['ERA'][index]) \

+ '\t' + str(item['R'][index]) \

+ '\t' + str(item['ER'][index]) \

+ '\t' + str(item['HR'][index]) \

+ '\t' + str(item['WHIP'][index]) \

+ '\t' + str(item['WPCT'][index]) \

+ '\t' + str(item['GO_AO'][index]) \

+ '\t' + str(item['OBP'][index]) \

+ '\t' + str(item['SLG'][index]) \

+ '\t' + str(item['OPS'][index]) \

+ '\t' + str(item['K_9'][index]) \

+ '\t' + str(item['BB_9'][index]) \

+ '\t' + str(item['H_9'][index]) \

+ '\t' + str(item['K_BB'][index]) \

+ '\t' + str(item['P_IP'][index]) \

+ '\n'

self.file.write(line)

return item

settings.py:

BOT_NAME = 'selenium_demo'

SPIDER_MODULES = ['selenium_demo.spiders']

NEWSPIDER_MODULE = 'selenium_demo.spiders'

ITEM_PIPELINES = {'selenium_demo.pipelines.ValidateItemPipeline': 100,

'selenium_demo.pipelines.WriteItemPipeline': 200}

DOWNLOAD_DELAY = 3

selenium_demo_spider.py:

class DemoSpider(Spider):

name = "selenium_demo"

allowed_urls = ['http://mlb.mlb.com']

start_urls = ["http://mlb.mlb.com/stats/sortable.jsp?c_id=hou#game_type='R'&season=2006&league_code='MLB'&split=&playerType=ALL&sectionType=sp&statType=pitching&elem=%5Bobject+Object%5D&tab_level=child&click_text=Sortable+Player+pitching&season_type=ANY&page=1&ts=1464232459333&sportCode='mlb'&team_id=&active_sw=&position='1'&page_type=SortablePlayer&sortOrder='desc'&sortColumn=w&results=&perPage=50&timeframe=&last_x_days=&extended=0"]

def __init__(self):

self.driver = webdriver.Chrome()

def parse(self, response):

self.driver.get("http://mlb.mlb.com/stats/sortable.jsp?c_id=hou#game_type='R'&season=2006&league_code='MLB'&split=&playerType=ALL&sectionType=sp&statType=pitching&elem=%5Bobject+Object%5D&tab_level=child&click_text=Sortable+Player+pitching&season_type=ANY&page=1&ts=1464232459333&sportCode='mlb'&team_id=&active_sw=&position='1'&page_type=SortablePlayer&sortOrder='desc'&sortColumn=w&results=&perPage=50&timeframe=&last_x_days=&extended=0")

# response.url="http://mlb.mlb.com/stats/sortable.jsp?c_id=hou#game_type='R'&season=2014&league_code='MLB'&split=&playerType=ALL&sectionType=sp&statType=pitching&elem=%5Bobject+Object%5D&tab_level=child&click_text=Sortable+Player+pitching&season_type=ANY&page=2&ts=1463597259434&sportCode='mlb'&team_id=&active_sw=&position='1'&page_type=SortablePlayer&sortOrder='desc'&sortColumn=w&results=&perPage=50&timeframe=&last_x_days=&extended=1"

while True:

try:

response = TextResponse(url=response.url, body=self.driver.page_source, encoding='utf-8')

rows = response.xpath('//tr').extract()

next = self.driver.find_element(By.CLASS_NAME, "paginationWidget-next")

for row in rows:

Player = Selector(text=row).xpath('//td[2]/a/text()').extract()

W = Selector(text=row).xpath('//td[6]/text()').extract()

L = Selector(text=row).xpath('//td[7]/text()').extract()

IP = Selector(text=row).xpath('//td[13]/text()').extract()

GS = Selector(text=row).xpath('//td[10]/text()').extract()

ERA = Selector(text=row).xpath('//td[8]/text()').extract()

R = Selector(text=row).xpath('//td[15]/text()').extract()

ER = Selector(text=row).xpath('//td[16]/text()').extract()

HR = Selector(text=row).xpath('//td[17]/text()').extract()

WHIP = Selector(text=row).xpath('//td[21]/text()').extract()

WPCT = Selector(text=row).xpath('//td[38]/text()').extract()

GO_AO = Selector(text=row).xpath('//td[39]/text()').extract()

OBP = Selector(text=row).xpath('//td[40]/text()').extract()

OPS = Selector(text=row).xpath('//td[42]/text()').extract()

K_9 = Selector(text=row).xpath('//td[43]/text()').extract()

BB_9 = Selector(text=row).xpath('//td[44]/text()').extract()

H_9 = Selector(text=row).xpath('//td[45]/text()').extract()

K_BB = Selector(text=row).xpath('//td[46]/text()').extract()

P_IP = Selector(text=row).xpath('//td[47]/text()').extract()

item = MoviesItem()

item['Player'] = Player

item['W'] = W

item['L'] = L

item['IP'] = IP

item['GS'] = GS

item['ERA']= ERA

item['R'] = R

item['ER'] = ER

item['HR'] = HR

item['WHIP'] = WHIP

item['WPCT'] = WPCT

item['GO_AO'] = GO_AO

item['OBP'] = OBP

item['SLG'] = SLG

item['OPS'] = OPS

item['K_9'] = K_9

item['BB_9'] = BB_9

item['H_9'] = H_9

item['K_BB'] = K_BB

item['P_IP'] = P_IP

yield item

next.click()

time.sleep(5)

except:

break