Data Study on Weather vs. the Stock Market

The skills the author demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.

Weathering the Stock Storm

I love the gentle rain. I like the way it feels when it just whispers quietly on my face and its wisps touch the leaves of grass. The dew is exhilarating. But in the same vein that it refreshes, revives, and renews, it can rear its ugly head quickly. When it insists on summoning its torrential mood, it can destroy, decimate, and devastate life and property quite as easily. Fortunately, data shows such extreme weather conditions are not a frequent occurrence. Well, not yet at least. That being said, we're here to investigate how daily weather affects us in ways that could be very nuanced. The stock market is potentially one of them.

Climate and weather are interrelated. They just differ in the context of time and space. Climate is the average of our day to day weather conditions through the lens of time and space. As they always say, weather is what we "get", climate is what we "expect". But that equation is starting to change and flip on its head.

We still don't know enough about our earth's climate and weather despite all the research done. Although the advances in weather research and technology have progressed significantly over several decades, the specifics of the causes of climate change still elude us. Something that we take for granted everyday is our weather - it affects us all in different ways, big, small or indifferent. It is partly in this context that furthering our understanding of the weather is so important.

Looking past this, our local weather patterns also have an impact to our daily lives. There is no question that even normal weather conditions affect the social, economic, and financial aspects of humanity.

Unleash the Kraken

Let's consider a controversial topic: Weather and stocks. Does weather affect the financial markets? Is  there a correlation between the two? Do stock prices and equities trading transaction volumes follow a pattern in relation to different weather conditions? If there is a correlation, what could be driving it? This research seeks to gain some insight into this area. It beyond the obvious impact of extreme weather and asks if the normal day to day, season to season changes and fluctuations have some type of influence on how the stock market behaves.

Discovery consists of seeing what everyone has seen, and thinking what no one has thought.

     --- Albert Szent Gyorgi, 1937 Nobel Laureate Physiologist

In Greek mythology, Perseus was a demigod who was known to be the ultimate slayer. In all his quests and journeys, he had to face a slew of adversaries that required different approaches and combination of weapons to defeat. He was fortunate that his amazing arsenal was bestowed to him by the gods, like a shield, a sword, and winged sandals. But there was this one special gift that Athena gave him. It was an all-guiding owl that became Perseus' eyes and ears, it goes far and wide to give Perseus all sorts of information for him to be successful in all his quest.

Creating the Data and Stock Analyzer Tool

There are a lot of tools out there that can do a numerical and trend analysis on stocks. But I have yet to see one made for analyzing stocks information against various weather conditions. Thus, I created an app to examine the trends and correlation of stock information with weather conditions. You can access this application, the Bubo Stock Analyzer,  by clicking the image below.

The Quest and Question

This should start our quest for insight and, hopefully, a better understanding of the relationship between weather conditions and the stock market. The question remains: is there a correlation between the different types of weather conditions against the movement of the equities market? Does weather affects us in ways that we don't even know about. I want to start the  quest by focusing the analysis based around weather in New York City.

The research community’s findings on the topic of behavioral finance seem to be divided. Some say that there is absolutely no correlation between weather and some say there is. This is an opportunity for us to investigate and find out for ourselves. It’s pertinent to exclude major storms and hurricanes since these phenomena affect the market drastically due to their impact. So let's use this stock analyzer tool to find out.

Dataset

The data I used for the historical stock information is sourced from Yahoo Finance. For this analysis, we can put in a ticker symbol for a stock index like NASDAQ or S&P 500 (preceded with a ^ caret symbol) or stock, and it will dynamically pull the daily adjusted closing price and transaction volumes for that day. I limit the historical daily data to the years from 2013-2015, as a time series XTS stream.

The R code to perform this data pull and filter based on a date range is shown here:

Weather Data

The weather data is harder to pull in dynamically, so I just sourced it manually from Weather Underground and created a comma-delimited text file to to read into memory and filter on the date ranges as necessary. I used the weather conditions based on the JFK Airport as a baseline for NYC. Why NYC city weather? Research communities in behavioral finance have surmised that the bulk of local market makers and trading agents are represented in NYC. This covers also the years from 2013-2015 and uses the following weather conditions on a daily time series:

1. Maximum Temperature (Fahrenheit)
2. Highest Humidity (%)
3. Maximum Wind Velocity (mph)
4. Maximum Visibility (miles)
5. Cloud Cover (oktas)
6. Rain or Precipitation (inches)

Links

Here is the text file loading procedure and conversion to a time series XTS format for further analysis. Notice that though we are pulling a lot more weather information than the conditions, we are only using the above conditions for the analyzer:

The stock analyzer application is written using R and utilizing the Shiny Dashboard framework. Here are the libraries used to extend its functionality:

The stock analyzer R UI dashboard code creates all the front facing user experience that includes all the widgets to allow you to set the parameters, and main output panel to show you the time series trending charts and correlation information. The R code is as follows:

The server side code takes all the input from the UI side and performs all the statistical calculations and plot rendering and then sends them all back to the front end. This is all done in real time, so any changes on the parameters done by the user are reflected instantaneously on the screen.

Finally, these are global function calls used in the stock analyzer. It might be interesting to look at how the correlation ranges are translated to more descriptive terms to make it easier and more intuitive to interpret the strength of the correlation between the various weather conditions and the index/stock prices and trading volumes.

Using the Stock Analyzer

On a high level, the stock analyzer application is very straightforward in its use. From the left side panel, just enter the index or stock symbol, select if you want to analyze the adjusted closing stock prices or stock transaction volumes, then select the type of the six weather conditions you want to correlate it against. Remember to choose the date range to determine the scope of your analysis.

You will quickly notice as you review the plots, when you look at the plots over an expanse of time from 1 year or more, almost all of them will not show any type of correlation between stocks and weather conditions. The reason for this is that the weather has a cyclical pattern based on annual seasons, so using anything more than 1 year will not reveal any hidden correlations.

I suggest that you start your analysis with an index stock basket first, like S&P 500 (^GSPC), or NASDAQ (^IXIC). Set the date range for 1 year from Jan through end of December for that 1 year only. Once you see the annual plot, you will start to notice some nuance to the pattern. You could probably make out that more often than not there is a positive correlation during the winter/spring weather conditions with the stock information. This is more easily seen by setting the moving average rate of 10 days or so at the small bottom left box located at the corner of the scatter plot.

When the date is set from January to around the summer months - usually May  - the correlation starts to shift to being negative. You can experiment with your own parameters, but you will quickly discover that transactions tracking volume do not have any correlation with the weather, but adjusted closing stock prices do. As far as weather conditions go, there's only one that presents the strongest possibility of a correlation with the stock prices: the maximum temperature. Rain and humidity will at times show some correlation strength but not by a whole lot.

A Note on Interpreting Correlation

The strength of a correlation between a weather condition type against a stock or index price or transaction volume data is measured using Pearson, Kendall, and Spearman. Every correlation has two qualities: strength and direction. The direction of a correlation is either positive or negative. In a negative correlation, the weather condition vs the stock data move in inverse, or opposite, directions. When stock and weather data have a positive correlation, it means they move in the same direction. This means that as one side increases, so does the other one. Here is an example of the S&P 500 index adjusted closing price with a strong positive correlation with maximum temperature from Jan 1, 2015 through Jun 30, 2015:

For data exploration, it would be interesting to see all three correlations from Pearson, Kendall, and Spearman. As a simple guide, Pearson is normally used when there seems to be more a linear relationship (interval data) or better normal distribution in the data points. Kendall or Spearman is used more on ranks (ordinal type data, e.g. cloud cover comes to mind here) and depicts a monotonic relationship. Kendall might also be better over Spearman in some nonlinear cases. But the three are all presented for an interesting comparison. When the Spearman number is greater than Pearson, that would indicate that the correlation mostly likely is not linear and perhaps also an indication of the sample size being too small.

Correlation Guidelines

Here are the correlation guidelines used by this application for your reference:

Summary

After extensive analysis on the various indices and stock prices, the researchers at large on both sides of the fence are partially right. It depends on how one did the analysis. Some weather patterns do affect the stock market, but some don't. Weather definitely has some effects on the equities market, however its correlation needs to be examined carefully based on seasonal shifts, industry that the stock represents, and type of weather conditions.

The below are findings garnered from the discovery process and statistical analysis performed. Moving averages were used to smooth out the time series curve for stocks and weather condition fluctuations. Seasonal shifts were determined by looking at the intersect point when the correlation between weather conditions and stocks information reverse direction. The synopsis of the findings below will demonstrate the patterns and trends are documented in more detail as follows:

Patterns and Trends

• Correlations must be analyzed base on seasonal shifts within the year.
• There tends to be respective winter/spring and summer/autumn trend and correlation.
• The industry that the stock belongs to makes a difference when the seasonal shifts occur.
• The Maximum Temperature shows the strongest effect on stock and index prices, the rest of weather conditions have no or negligible effects.
  • On a whole, Maximum Temperature shows a positive correlation on the stock market, more often during winter months leading usually up to the end of summer (around Memorial Day).
  • A possible theory would be that during the colder months, as the temperature rises, the market optimism might have likely risen with it, feeling more comfortable as the months get warmer.
• During the fall and start of winter, the Maximum Temperature seems to show a negative correlation to the stock market.
  • A possible theory here is that during summers when temperatures are high and a bit uncomfortable, any more rise in temperature during the summer months are frowned upon, creating a more depressed mood for the market.
  • This could imply some effects on lowering the stock price based on that.
• Anomalies do exist in some years, when stocks could be affected by other major factors.
• Stock market transaction volumes have almost no or negligible correlations no matter what weather types and patterns are involved, this shows that transactions small or large are done without any regard to the weather conditions.
• Humidity and sometimes rain, seem to play a small part how it affects stock prices. In theory, when Maximum Temperatures are rising, it could aggravate prices especially during summer months. This still needs to be proven out statistically.
  • Humidity shows a weak positive correlation based on the Pearson and Spearman, and a weak negative during the autumn months (similar trends as the Maximum Temperature).
• Wind velocity seems to show a very weak negative correlation to the stock market during summer months, but almost negligible correlation during winter months.
• Cloud cover and visibility have zero to negligible effect to the stock market (first of all low visibility is very rare in NYC).
• Cloud cover is an interesting discovery. There are researchers out there reporting that high cloud cover impacts trading because of the feeling of gloom and less daylight, but data doesn't seem to support that finding.
• The only exception I found was on a month level in cases where rain volume is high, the cloud cover seems to strengthen that correlation on stock market impact.
  • Rain correlation against stocks is still unclear. It's harder to draw conclusions on rain effects since NYC doesn't rain that much compared to like Seattle.
  • When it does rain continuously for a few days, the stock prices seem to start to dip a little bit (storms and hurricanes are of course more significant due to possible loss or impact to business).
  • For the most part, normal rain volume doesn't seem to affect stock market too much, but when I plot a month where it shows extraordinary rain volume, there is definitely a very strong positive correlation on stock price impact.

Future Analysis

If there is a need to increase the scope of date ranges, we can expand the data set for the weather conditions easily. Future enhancement to the stock analyzer application can also include more predictors for what stocks or indices prices could be, based on the weather patterns for the past few weeks. A multiple linear regression analysis would also be appropriate to check for how combinations of weather conditions drive the equities market.

by Bernard Ong, Data Argonaut
Email: bernard.ong@entense.com
Cell: 201-916-5241