Data Analysis of Performance of Technology Stocks

Posted on Apr 29, 2019

The skills the author demonstrated here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.


Being from a finance background, I have analyzed the performance of financial assets multiple times, but mostly on excel, and paper. So after having learnt web-scraping and python an obvious choice for me was to apply these new technology tools in analyzing stock markets performance, and compare and understand the effectiveness and scalability of these tools. In this text we will use data to analyze the performance of technology stocks.

I decided to pick-up a sector which has grown the most in the recent period and to start with scraping a day's performance, which can later be extended to a longer period, to other sectors and to the overall market.


Webscraping to extract information

I used Yahoo! Finance as the source of information, and scraped the information at the day-end of 18th April, 2019 using Scrapy. The start url was '' and from here the spider developed by me went to 5 pages to collect information on closing price, price change, percent change over previous day, trading volume for the day, 3-mth average for trading volume, market capitalization and PE ratio for 467 stocks. The spider also went to summary page of each stock, through individual urls, and collected information on 3 Y monthly Beta -- Yahoo Finance calculates beta from monthly prices over a time of three years. The S&P500 is used as the benchmark -- and 1 year target estimate of price.


Importing into Python and Data Wrangling

While importing into Jupyter Notebook I used symbol of the stocks as index.


Data Analysis of Performance of Technology Stocks


As per shape of the data frame there were 467 stocks and 9 variables, excluding name.

An issue identified was that all the numeric values were converted into strings, and the there were different units within same column -- e.g. M(illion), B(illion) and regular numbers for market capitalization.

To make data amenable to further analysis I converted the strings into floats and also standardized over columns by converting millions and billions into regular numbers.

For sanity check, the information after transformation was checked for a few stocks against the information available.



Performance Summary

Data Analysis of Performance of Technology Stocks

It was not a good day for the technology stocks.

  • 242 stocks out of 467 had negative returns.
  • Mean percentage change over previous day was -0.24% (with a standard deviation of 2.17%).

Top Gainers of the Day:

Data Analysis of Performance of Technology Stocks


Top Losers of the Day:


Distribution of percentage change for all the stocks, even after excluding the outliers, also showed a negative skew confirming that more number of stocks were in red for the day. 

Data Analysis of Performance of Technology Stocks
Data Analysis of Performance of Technology Stocks



9 stocks were more than 3 standard deviations away from mean for technology stocks.

Data Analysis of Performance of Technology Stocks

Checking outliers based on today's percent change and estimated percent change in 1 year:

Data Analysis of Performance of Technology Stocks

Significant jump is expected in DAVA (Endava plc). On doing further research on web:

Endava plc (DAVA) Analyst Opinion (4/22/2019)
'Endava plc is currently rated as outperform by 5 stock analysts, with the company still having around 2.87% shares outstanding that can be sold short legally. The value of their shares went up by 12.57% last month. Over the course of a year, the stock has grown by 24.22%. Financial analysts are becoming more bullish than ever, with the 2 analysts who cover the activities of DAVA advising investors to buy. Experts on Wall Street have set a price target of $2426.27 for the stock. The decision was arrived after looking at their 7987.57% gain from current levels.'


Effect of Market Capitalization

I decided to see how was the loss and variation across different market capitalization categories. I even tried taking a log of market cap, to account for a few stocks having very high capitalization.

Data Analysis of Performance of Technology Stocks
Data Analysis of Performance of Technology Stocks
Data Analysis of Performance of Technology Stocks


There was no strong link between market capitalization and percentage change in price. Though, as expected, variability of price change was higher in low market capitalization stocks.


To dig deeper into the link between price change for the day and market capitalization, I divided the stocks into three groups as per mkt cap -- low (bottom 25%), medium (middle 50%) and high (top 25%), and plotted a boxplot for the three groups.

Data Analysis of Performance of Technology Stocks
Data Analysis of Performance of Technology Stocks

Percent loss was higher for some of the high market capitalization stocks, as compared to low market capitalization stocks, which is not a good sign, as this shows the loss might not be just because of volatility. High market capitalization stocks are supposed to be more stable. There is a need to observe market closely, as the loss might be sustained.


Effect of Estimated Price in One Year

I also tried to see, through a scatter-plot, if there was a link between the price change today and the price change expected in one year, based on analyst's estimate. 

Data Analysis of Performance of Technology Stocks


An outlier was making it difficult to observe any relationship, so I limited the range of x-axis. 

Data Analysis of Performance of Technology Stocks


Contrary to the expectations the day's price change were not related to percent change estimated in one year. This might be an indicator that there is an uncertainty about long term estimates.


Digging further into this, I calculated today's average price change for stocks that are estimated to gain more than 50%, 0-50% and less than 0%. This provided an interesting observation:

Data Analysis of Performance of Technology Stocks

The shares that are projected to gain most in 1-year are the shares that lost today. It might be because people are feeling that analysts' expectations have become too irrational in the sector.

Investment opportunities can be identified, through going long or short on stocks, by analyzing the reasoning behind the 1-year estimates.


Price Variation Today vs Beta (systematic risk)

I plotted percent price changes today and the beta for stocks, to see if the variation was driven by high-beta stocks, or was spread across technology stocks.

Data Analysis of Performance of Technology Stocks

From the graph it is clear that day's price change were not related to beta.  In fact the variance seems to be higher in the beta 1.0-1.5 range. This also shows that the loss might be more sustained.


How are Current Prices vs PE Ratios

I thought of looking at the PE ratio in the sector. A significant variation will deserve further analysis of fundamentals to identify investment opportunities.

I divide the stocks in 4-quartiles on the basis of PE ratio. 

Data Analysis of Performance of Technology Stocks

The min, max and mean of these quartiles shows significant variation.


I also plotted the current price of the stocks in different quartiles.

Data Analysis of Performance of Technology Stocks

Low prices for low PE ratio quartile definitely deserves a deeper look into, as there might be significant opportunities.



It was a bad day for the technology stocks with overall negative change and also majority of the stocks losing money. Further the downturn is not limited to low-cap and high-beta stocks, so it might be sustained. Market needs to be followed closely for any investment opportunities.


Future Work

  • Exploration over a longer period
  • Comparison to other sectors
  • Comparison to macroeconomic factors
  • Research through other sectors in categories where there might be investment opportunities


External Links

For code, visualizations, and other supporting material, please visit

About Author

Shailendra Dhondiyal, CFA, FRM

I have been a business manager who used data extensively to perform root-cause analysis, build operating models, conduct profitability analysis, develop forecasting models and design pricing strategies. I have successful track record of setting & scaling up businesses,...
View all posts by Shailendra Dhondiyal, CFA, FRM >

Leave a Comment

No comments found.

View Posts by Categories

Our Recent Popular Posts

View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI