Data Study on The U.S. Congressional Elections
The skills the author demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.
Background:
Campaign finance practices have attracted a great amount of attention in the 21st century U.S. congressional elections. This blog’s primary objective is to present a data-based insight regarding the involvement of money in elections, mainly focusing on the (1) winning factor and the (2) incumbency factor in elections. The blog presents summaries of the money spent by winners in different states and how these are compared to their opponents. In this article, one can find how the electorate have kindly treated incumbents across parties. An associate shiny web app for this blog can be found here.
Obtaining and Cleaning the Data:
The data was collected by web scraping the website http://www.opensecrets.org using Python and Beautifulsoup. The data is comprised of money (in USD) associated with all the candidates of all the house and senate races from the year 2000 to 2016. A note regarding the interpretation of data: for the 2000 and 20002 elections, the website lists the “total amount spent” by candidates; whereas from 2004 and onwards, the website lists the “total amount raised” by candidates.
Therefore, to produce an inclusive analysis, the blog treats all amounts for all years as “the money associated with candidates” or “the money involved in races.” Furthermore, no winners are listed for the year 2004 on the website; therefore, the year 2004 is excluded from the relevant data analysis.
Completeness of Data:
The analysis focuses solely on the data obtained from the website http://www.opensecrets.org and does not take into account money spent by super PACs and other potential sources.
Figure 1: total sums associated with all races for both the Senate and the House
Winners Surpassing their Opponents:
Figure 1 shows the staggering numbers for you to interpret the role of money in American elections. The total amount of money associated with congressional elections, especially with winning candidates, have typically increased consistently since 2000.
Ratios of the total amounts associated with winners and their opponents were lowest in probably the most fiercely contested 2010 elections, where these ratios were 2.13 and 1.43 for the House and the Senate, respectively. Otherwise, winners significantly outraise and probably outspend their opponents in all elections.
Key Observation:
- The gap between winners and their opponents has been consistently increasing since 2010 for both house and senate elections.
Incumbency:
Only 2 candidates have won a US Congressional district as a third party/independent candidate in the 21st century, (1) Bernie Sanders from Vermont in 2000, 2002 and (2) Virgil Goode from Virginia in 2000. Only 3 candidates have won a US Senate election as a third party/independent candidate in the 21st century, (1) Joe Lieberman from Connecticut in 2006, (2) Lisa Murkowski from Alaska in 2010, and (3) Bernie Sanders from Vermont in 2012. However, some of the above have switched to one of the two mainstream parties at various times in their career. There have been few non-incumbent one time winners from third parties. But the dearth of such candidates shows that the politics in the U.S. is heavily dominated by the two major parties: the Democratic Party and the Republican Party.
We earlier noticed how the money has sided well with the winners. As it turns out, the incumbency tends to help the candidate, meaning, once someone is part of the establishment, the person stays in the establishment as seen in Figure 2 from the very low fractions of incumbents being defeated in subsequent elections.
Figure 2: in all elections, the majority of incumbents retain their seats.
Observations:
- In 2006 and 2008, Democrats had nationwide gains in both assemblies. Similarly, in 2010 and 2014, Republicans had nationwide gains in both assemblies.
- No incumbent Democratic Senator lost in 2006, 2008, 2012, and 2016. Similarly, no incumbent Republican lost in 2010 and 2014.
- However, despite spending more money than their opponents, 2 and 4 Democratic incumbent senators lost in 2010 and 2014 elections, respectively. Similarly, despite spending more money than their opponents, 5 Republican incumbent senators lost in 2006 and 2008 elections.
- Only 11.3% of the incumbent senators having more money than their opponents have lost.
- Out of the 271 Senate races, in only 50 cases, candidates associated with more money than their opponents lost (18.45% cases). Candidates were incumbent senators in only 22 of the 50 cases. Furthermore, in 221 cases, candidates associated with more money won, and 173 were incumbents.
The figures below show top 15 and bottom 5 states with winners associated with most money in the Senate and the House, respectively.
Figure 3: Senate winners
Figure 4: House winners
The figure below shows money associated with winners in Senate races divided by the number of congressional districts of respective states. The number of congressional districts of states are typically proportional to the population of the state. You can see that Senate races in states with some of the lowest population numbers involve a lot more money per congressional district of the state. Most such states have only one statewide congressional seat.
Figure 5: money associated with a Senate seat per congressional district of the state
Tables below show the top-10 candidates with most money associated with the Senate and House elections in the 21st century.
Table 1: Top Senate candidates. The amounts are in Million USD.
Table 2: Top House candidates. The amounts are in Million USD.
Summary:
Tremendous sums of money are involved in U.S. elections. Winners are able to raise twice or thrice or even more sums of money compared to their competitors. Data supports the commonly held belief that people tend to like their own representative/senator. It would be interesting to contrast this with people’s approval for the entire Congress.
Importantly, the graphics shown here are intended to stimulate the viewer's curiosity about the subject of money in politics. Why are a significant majority of candidates funded so heavily especially when their probability of winning is already very high? Role of money in electioneering must be probed and communicated to the public. By reading this article, I hope the reader can be an informed participant in our democracy and can engage in a broader conversation about money in politics.
About Author
