In this data web scraping project, I was focusing on the web scraping of the 10-K filings of all the listing companies on NASDAQ from the official website of US securities and exchange commission. The goal of this project is trying to find some clues between the financial performance of the companies and their stock return.
Areas:
All the listing companies in NASDAQ (4715 listings) in 2016
11 industry sectors
Question want to answer:
By evaluating what financial features can we get substantial information about the annual stock return?
Web scraping data on Yahoo finance, all listings
I scraped all the stock price history in 2016 of the 4715 listings from Yahoo Finance. They are separated to 11 sectors, according to companies by industries. The annual stock return can be calculated from the scraped figures, and they are box plotted in the figure below. It seems a majority of companies in the basic industries sector in 2016 had the positive annual stock return, while the healthcare sector had the lowest median annual stock return, which was negative. The rest of the sectors seems to have no big difference with each other if only judging from the box plot.
10-K filing scraping
For the investors, it’s crucial to know as much as possible about the companies before making the decisions of an investment. There are many ways for them to get knowledge of this information. The financial websites, twitter, Bloomberg, etc, everywhere. However, there’s one place that can provide all the needed information, and they are 100% authentic, kosher, and ready to use. It’s the official website of US securities and exchange commission. There are plenty of annual, quarterly report of different companies you would like to find. In this project, I scraped some needed information from the 10-K filings of all the listing companies.
A. What is the 10-K filing?
A 10-K is a comprehensive summary report of a company's performance that must be submitted annually to the Securities and Exchange Commission. Typically, the 10-K contains much more detail than the annual report. It includes information such as company history, organizational structure, equity, holdings, earnings per share, subsidiaries, etc. The 10-K must be filed within 60 days (it used to be 90 days) after the end of the fiscal year.
B. What to scrape?
Return on assets: Net income / Total assets
Return on assets (ROA) is an indicator of how profitable a company is relative to its total assets. ROA gives an idea as to how efficient management is at using its assets to generate earnings.
Return on equity: Net Income / Shareholder's Equity
Return on equity (ROE) measures a corporation's profitability by revealing how much profit a company generates with the money shareholders have invested.
Current ratio: Current assets / Current liabilities
The current ratio is mainly used to give an idea of the company's ability to pay back its liabilities with its assets, it can be used to take a rough measurement of a company’s financial health.
So, all in all, five key figures to be scraped from the 10-K filings:
Net income (loss);
Total assets;
Shareholder’s equity;
Current assets;
Current liabilities.
Data Results
I scraped all the five figures from the 10-K filings on sec.gov, they are plotted in the figures below. We can see that the ROE and ROA of finance sector are very centralized comparing with other sectors. And for the healthcare sector from the analysis of ROA and ROE, the median profitability of the companies in this sector seems to be lower than the companies in other sectors. However, the median current ratio of this sector is among the highest in all the industry sectors.
In order to answer this question:
By evaluating what financial features can we get substantial information about the annual stock return?
I did some simple linear regression to try to find the answer, here’s the result:
Annual stock return vs. Return on equity
Sectors
Intercept
Slope
p-value for Intercept
p-value for slope
Basic Industries
0.355231621
0.18247217
1.58E-15
0.028546516
Capital Goods
0.336702101
0.126801624
4.02E-28
0.106128202
Consumer non Durables
0.225510762
0.018302826
1.49E-12
0.466077375
Consumer services
0.165893504
0.00813999
4.33E-30
0.494754911
Energy
0.368675068
-0.002923266
8.07E-10
0.855902605
Finance
0.358726929
0.347776918
2.06E-60
0.006366251
Healthcare
0.05743789
0.200523609
0.055878211
7.04E-10
Miscellaneous
0.215358904
0.07464239
1.89E-05
0.231190295
Public Utilities
0.240638978
0.263049961
2.72E-12
0.000595729
Technology
0.261536342
0.069303433
1.04E-22
0.095897413
Transportation
0.373388909
-0.009137745
1.88E-06
0.79221854
Annual stock return vs. Return on assets
Sectors
Intercept
Slope
p-value for Intercept
p-value for slope
Basic Industries
0.330594989
0.596238709
1.66E-15
0.004437525
Capital Goods
0.303064458
0.361889481
3.36E-30
1.58E-05
Consumer non Durables
0.170056788
0.765273808
9.45E-07
0.012500255
Consumer services
0.142080656
0.69679516
7.88E-26
1.58E-07
Energy
0.366266572
0.232100176
2.70E-13
0.281701288
Finance
0.352035813
-0.125288529
1.03E-71
0.72050919
Healthcare
0.025691108
0.299685086
0.457359828
2.92E-08
Miscellaneous
0.198963377
0.210699384
2.89E-05
0.156874055
Public Utilities
0.208522685
2.405875682
1.58E-14
1.31E-06
Technology
0.252271318
0.281154548
3.06E-23
0.000146544
Transportation
0.36413647
-0.229605123
4.19E-06
0.743215475
Annual stock return vs. Current ratio
Sectors
Intercept
Slope
p-value for Intercept
p-value for slope
Basic Industries
0.370656669
-1.13E-12
6.87E-19
0.910464037
Capital Goods
0.305918824
-2.42E-12
7.05E-30
0.309531081
Consumer non Durables
0.215105954
-1.47E-12
2.43E-15
0.770846634
Consumer services
0.167368009
-2.18E-12
2.43E-21
0.512133029
Energy
0.267368348
7.92E-13
1.51E-10
0.940114503
Finance
0.132306628
2.75E-12
0.001364733
0.497072028
Healthcare
-0.134979771
6.94E-12
2.72E-07
0.096389706
Miscellaneous
0.238054911
-6.02E-12
1.27E-07
0.506012428
Public Utilities
0.20567902
1.01E-11
5.27E-13
0.245790194
Technology
0.158163786
1.02E-12
4.46E-13
0.514683949
Transportation
0.277715154
3.14E-13
2.53E-06
0.985370257
The bolded ones in here indicate there is a relationship between a specific financial feature and the annual stock return for companies in a specific industry sector. It’s very interesting here that in no industry sector, the current ratio seems to have the relationship with the annual stock return, while for many of the sectors, the annual stock return can be reflected by their return on assets and return on equity.
Conclusion
Two financial features can be used to be evaluated before potential investments:
Return on assets:
Basic Industries / Capital Goods / Consumer on Durables / Consumer Services / Healthcare / Public Utilities / Technology;
Return on equity:
Basic Industries / Finance / Healthcare / Public Utilities.
Warren Buffett used to say: Return on assets is the best way to evaluate a bank. Well, I'm open to this. I may need to take out the banks from the finance sector, and analyze solely on this group, to get a better understanding and more interesting result.
About Author
Yiming Wu
Yiming is a data science fellow in NYC data science academy. He is always passionate about machine learning, data visualization, and coding reusable scripts. He received his Ph.D. degree in Electrical Engineering from the Université de Technologie Belfort-Montbéliard,...