Patterns in Stock Indices over Time

Robin Gane-McCalla
Posted on Apr 30, 2017

The traditional advice given to investors is 'buy and hold'.  I decided to investigate this claim and see what patterns I could find within the major stock indices. The Dow Jones, S&P 500 and the NASDAQ Composite were the indices that I examined. They are all combinations of stocks which are frequently used as an indicator of the US economy. The Dow Jones consists of thirty of the biggest publicly traded stocks in the US, the S&P 500 consists of 500 average to large sized companies and the NASDAQ composite is mainly technology companies. I acquired 30 years of daily financial data for each of the indices at

In my Shiny Application I collected data by year, daily and month. In addition to the High price of an index, I also collected the volume traded and derived the daily change in closing value and the daily percentage change in closing value. Lastly I allowed the user to select a subset of the 30 years of available data. I displayed my data using both a boxplot and a scatterplot. In addition I showed the results of an ANOVA test to show whether the differences observed were statistically significant. You can experiment with my Application here

Many of the relationships I observed were not statistically significant. One interesting results that was significant was the daily percent change of the NASDAQ on a daily basis. The NASDAQ consistently fell slightly on Mondays and increased in value every other day. A similar pattern was observed within the Dow Jones and the S&P but the p-values were higher and the price changes were smaller. These patterns can be explained by markets being closed on weekends and bad news lowering stock prices. This conclusion implies that an investor could consistently make money simply by buying on Monday and selling on Friday. However, when I looked at the NASDAQ data for the last 20 years I found that the p-value went up to 0.58, presumably because traders caught on to this strategy. I also observed that trading volumes were significantly different (p-value 0.00) for all indices based on day of the week, giving a plausible explanation for how markets have adapted to differences in prices by day of the week.

Though a month is longer than a day, I found less significant results by month than by day. Despite several popular phrases such as "the January effect" and "Sell in May and go away", I found no significant prices related differences by month based on the ANOVA analysis. I did find significant differences in volume (p-value = 0.00 for both the NASDAQ and Dow Jones but p-value 0.07 for S&P).

The yearly plots showed some of the most dramatic changes, but many results were not statistically significant. The High price for all indices increased significantly.

yearly high change

Despite visible booms and busts the market increases significantly year by year. Additionally volume trade is significantly different and generally increases year by year but not as dramatically as High price. No significant differences were found for the Close Change or the Percent Close Change.


Despite the unpredictability of markets, there are some patterns in it.  Since market data is publicly available and many people trade on the market, it can be difficult to turn any of these patterns into a profitable trading strategy over the short term.  However, on a yearly basis indices tend to rise which supports the 'buy and hold' strategy.  Code for the Application I wrote is below:

About Author

Robin Gane-McCalla

Robin Gane-McCalla

Robin Gane-McCalla graduated from Hamilton College with a degree in Mathematics and Computer Science. He's worked as a computer programmer, data analyst, researcher and web developer. After working for several companies (including a few start ups) he's decided...
View all posts by Robin Gane-McCalla >

Related Articles

Leave a Comment

No comments found.

View Posts by Categories

Our Recent Popular Posts

View Posts by Tags

#python #trainwithnycdsa 2019 airbnb Alex Baransky alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus API Application artist aws beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep Bundles California Cancer Research capstone Career Career Day citibike clustering Coding Course Demo Course Report D3.js data Data Analyst data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization Deep Learning Demo Day Discount dplyr employer networking feature engineering Finance Financial Data Science Flask gbm Get Hired ggplot2 googleVis Hadoop higgs boson Hiring hiring partner events Hiring Partners Industry Experts Instructor Blog Instructor Interview Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter lasso regression Lead Data Scienctist Lead Data Scientist leaflet linear regression Logistic Regression machine learning Maps matplotlib Medical Research Meet the team meetup Networking neural network Neural networks New Courses nlp NYC NYC Data Science nyc data science academy NYC Open Data NYCDSA NYCDSA Alumni Online Online Bootcamp Open Data painter pandas Part-time Portfolio Development prediction Prework Programming PwC python python machine learning python scrapy python web scraping python webscraping Python Workshop R R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn Selenium sentiment analysis Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau team TensorFlow Testimonial tf-idf Top Data Science Bootcamp twitter visualization web scraping Weekend Course What to expect word cloud word2vec XGBoost yelp