Because it's Friday: Religion and reading level

The dating site OK Cupid often publishes on their blog interesting analyses of based on the self-reported data from their users, and the latest post is no exception. Most commentary has focused on race/gender preference analysis: according to the "Likes" sections of OK Cupid profiles, white males like "sweaty guitar rock, bro-on-bro comedies, things with engines, and dystopias" whereas black women like soul food and the color purple. (Remember: this is based on self-reported race and "likes". It's also notable that the post opens with the rhetorical question, "Is there any way to make fun of other races in public and get away with it?".)

I was most interested, though, in what appeared to be a throwaway analysis at the end of the post. If you use a standardized score to rank the writing proficiency of the users based on the essays they use to describe themselves in their profiles, and then break it down by self-described religion, this is what you find:

Offered with no further comment.

OKTrends: The REAL 'Stuff White People Like'

Comments

[Dan Bowen]

You know what I want to see:

10-commandment infringements, by religion

... prolly not via okcupid data, but there's gotta be some data sets out there that allow for those sorts of estimates to be produced. I'm sure the analysis would have a big astricks tied to it also, but still... that would be an interesting perspective.

[Rama Ramakrishnan]

The rightmost numbers look very close. Error bars would have been nice, to get a sense for statistical distinguishability between the scores.

[Jeremy Miles]

Not sure how useful error bars would be, given the N of over half a million (although probably not so many buddhists). Also, what's the population you're generalizing to?

[Kevin]

Unless you're fascinated in the population of OK Cupid users this data is pretty useless. There is no evidence that the self-selecting users of a free dating website represent any larger group other than the users of that dating website.

The chart should clearly state that the data only applies to OK Cupid users. It is misleading.

The writing proficiency chart shown here only demonstrates that users WRITE their site bios to be comprehended by certain grade level. It says nothing about what level they read at, what level they CAN write to, or what level of education they've achieved. It's interesting that they left out educational data because they have that information. I could read the data and say that Protestants tend to write their bios more simply to appear more fun-loving and less rigid (perhaps a religious stereotype they're trying to avoid) and that atheists tend to be overly verbose in order to compensate for their lack of formal education.

Three non-exclusive possibilities for why these results are reported so sloppily:

1) Confirmation bias is at play.
2) Christian Rudder (the OK Cupid post author/editor) doesn't understand how to accurately report the meaning of the results.
3) OK Cupid gets free publicity by reporting controversial results.

[jef]

Another interpretation can be that all the intelligent Protestants and Catholics are already "taken", while the intelligent atheists are struggling to find a life partner.

Simple. :-)

[Matt]

The index that David Smith is writing about is used to "gauge the understandability of a text". Therefore this analysis isn't about the education level of the subjects but the ability of others to read what the subjects have written. The index used determines that longer sentences and larger words have a heavier weight which corresponds to a higher level of education required by the reader to understand the sentence.

While I am not defending the education letter of the religious I think it is important to realize that if you are writing about your likes and dislikes on a dating site that you are more likely to use words with fewer letters which would garner you negative points as well as use short sentences which can be devastating to your score (assuming that a lower score is bad).

The following example would reduce your grade level by one which means that a reader would need a lower education level to understand it: I like to golf. I like the beach. I like to cook.

The title of the graph is mis-leading as it mentions proficiency level rather than understandability which makes it sound like the religious are stupid. If that was the point of the David Smith’s article then he is mis-used the data when trying to make that point.

[Scott Smith]

While I greatly enjoy the rev blog, this post smacks of prejudice toward religious groups. Why is this acceptable behavior? Haven't we seen enough of this kind of bigotry? I would have been fine had there not been a clear comment that a sample of OK CUPID cannot be inferred to mean anything about the general public. Without clearly mentioning this, the post writer implies to readers that this applies to general public. Otherwise this post is simply a bigoted religious slur. C'mon you can do better than this. Get a backbone.

PS- If you wanted to do a valid study of religious affiliation and writing ability using a scientific sample and rigorous techniques, I'd have no problem with that, provided that you don't move beyond what your data could support.

[alt com]

I always take these OKCupid stat posts with a grain of sand. Anyone trying their best to sound appealing would have to make little lies here and there. Plus there's no way to verify whether or not they were the ones who truly wrote their bios. I wonder if their data would match other dating sites like alt com or plentyoffish.com.