« Vendor news: TIBCO's proprietary R runtime; Teradata's appliance integrates R | Main | Gartner: IT to spend $232B on Big Data over 5 years »

October 18, 2012


Feed You can follow this conversation by subscribing to the comment feed for this post.

An complimentary approach to visualization would be to fit a generalized additive model to evaluate the trustworthiness of the visual trends. The code below achieves this (restricting the years to those with data for both democrats and republicans, and eliminating the other category for low data counts).


#get the data
dwNominate <- read.dta("ftp://voteview.com/junkord/HL01111E21_PRES.DTA")

# Make a re-coded party variable
dwNominate$majorParty <- "Other"
dwNominate$majorParty[dwNominate$party == 100] <- "Democrat"
dwNominate$majorParty[dwNominate$party == 200] <- "Republican"

#toss the "Other" data and factorize majorParty with a sum contrast
dwNominate = dwNominate[dwNominate$majorParty!='Other',]
dwNominate$majorParty = factor(dwNominate$majorParty,levels=c('Democrat','Republican'))
contrasts(dwNominate$majorParty) = 'contr.sum'

#narrow data to cong>35
dwNominate = dwNominate[dwNominate$cong>35,]

#fit the gam
fit = gam(
data = dwNominate
, formula = dwnom1 ~ majorParty + s(cong,by=majorParty,k=76,bs='ts')
, weights = I(1/bootse1)

#obtain predictions
preds = ezPredict(fit)

#data for each party separate, with a posteriori bootstrapped 95% CIs
preds = preds
, x = cong
, split = majorParty
, ribbon = T

#data for the party difference, with a posteriori bootstrapped 95% CI
preds = preds
, x = cong
, diff = majorParty
, reverse_diff = T
, ribbon = T

While it would require delving into the source data, what we call Democrats today evolved from the Republican Party of Lincoln. The original Democrats evolved into the Tea Party Republicans (or Goldwaterites). If the data relies on just party names, the plot pre-1900 doesn't really map to today.

I don't think Nominate it's designed to be comparable across time.

The main problem with it is that we're recovering a latent variable and there are known issues with identification. (rotation or translation of the graphic is one of the identification problems). In order to identify the model, it's necessary to impose some constraints, like setting one extreme Rep legislator to +1 and one extreme Dem legislator to -1. This helps to identify the model (in one dimension at least, If I'm not mistaken), but the cost is that one extreme legislator from the 50's doesn't have the same ideology content of an extreme legislator from the 90's.

Another problem is that since Nominate scores are estimated based on actual roll call votes, and pieces of legislation varies along time, we can't be sure whether legislations from the past we're proposed again current legislators would vote different than they voted in the past.

All in all, I don't think we can say that scores are comparable over time.


@Paul, thanks for the correction. I've updated the post above.

@Mike, thanks for the suggestion! I've included a version of your chart in a new blog post.

The comments to this entry are closed.

Search Revolutions Blog

Got comments or suggestions for the blog editor?
Email David Smith.
Follow revodavid on Twitter Follow David on Twitter: @revodavid
Get this blog via email with Blogtrottr