The Glade 4.0

You westerners sure are a tolerant bunch. Relatedly, wtf is wrong with the eastern half of the country!?



Interactive version is here. Racist tweets follow roughly the same east/west split.



*Edit: Added link to article/blog in the quote.

They sure are selective in what they search for to find "hate".

Want hate? Read the tweets after Andrew Breitbart died.

_________________
"It is true that democracy undermines freedom when voters believe they can live off of others' productivity, when they modify the commandment: 'Thou shalt not steal, except by majority vote.' The politics of plunder is no doubt destructive of both morality and the division of labor."

Hmm, really, LA has none whatsoever?

Weird.


RD: based on methodology, this could be erroneous data given the complete dearth of data from LA, LV, ABQ, or a half dozen other western states.

It could also just indicate that western staters don't geotag, don't use twitter as much, or that the "negative/neutral/positive" algorithm they're running was flawed.

Very interesting, but without a lot more insight into methodology, I'll be having a truckload of salt to go with this.

_________________
“The duty of a patriot is to protect his country from its government.” - Thomas Paine

Wait, I also just noticed in the bottom right it says the numbers were aggregated and then normalized. So that's another confounding variable: this is essentially saying these terms, run through the algorithm, show up a higher proportion of times than in areas that aren't bright red. So LA, LV, ABQ and others could have lots more "hateful" tweets than the rest of the country put together, but they have a lower percentage. This could then be disproportionately representative because the tweets were not looking in the southwest for Spanish-language hate speech, for higher populations of individuals identifying as homosexual, or maybe just because racists are more likely to be twitterers in the eastern US than the western (not that any of these are concrete, I'm just speculating). Any of these would confound the results enough to be useless.

In other words: lies, damn lies, and statistics, once again.

_________________
“The duty of a patriot is to protect his country from its government.” - Thomas Paine

Hmmms, confirmation bias results in unusable data applications once again ...

_________________

They address that a bit in part of the post that I didn't quote before:


Yeah, their FAQ acknowledges that potential source of bias:


Totally agreed. Just thought it was an interesting east/west division that makes me go, "Hm. I wonder what's going on here?"

Note the lack of hate at Disney World.

I'm sure that means something. Somehow. To someone.

_________________
Well Ali Baba had them forty thieves, Scheherezade had a thousand tales
But master you in luck 'cause up your sleeves you got a brand of magic never fails...
...Mister Aladdin, sir, What will your pleasure be?
Let me take your order, Jot it down -You ain't never had a friend like me
█ ♣ █

Besides the fact that a lot of the west is near-uninhabited compared to the east? Also, how about people using those terms to refer to themselves, or to the derogatory use of the term. If some black guy tweets the n-word in quotes while talking about imaginary predjudice, does that count as hate speech?

_________________
"Hysterical children shrieking about right-wing anything need to go sit in the corner and be quiet while the adults are talking."

They don't have that important piece of data; Twitter isn't legally allowed to collect and store it. Consequently, the entire study is rather meaningless and guilty of the self-same reductivism it's decrying.

_________________

Would homo include homosexual? Homogeneous?

Also a lot of people of africian decent use th n word in a non hateful way. The red spots do corrospond with areas were people of such decent live

_________________
I prefer to think of them as "Fighting evil in another dimension"

Nevermind reread

_________________
I prefer to think of them as "Fighting evil in another dimension"

See my second post about bias.



Actually, that is what it means: that they should automatically be discarded. Unless one is able to first demonstrate dispersal of geotagging opt-in, and then correct the data for that variable, it means the data is fundamentally unusable. The fact that they knew this issue and failed to correct for it actually makes me more dubious, not less.



Well, based on my sentence above, at this point it should make you instead state "that was an interesting opportunity to learn why statistics are easy to lie with," and then to discard the study entirely. That is, rather than wondering what's going on.


"Scientists" who accidentally overlook inherent biases are called: flawed, mistaken, or "lacking rigor."

"Scientists" who purposely disregard inherent biases while presenting the data as factual are called: liars, frauds, or quacks.



If a pharma company were to present the same type of erroneous findings, they would be (and have been in the past) liable for huge amounts of money.

_________________
“The duty of a patriot is to protect his country from its government.” - Thomas Paine

Almost all of the problems you guys are finding with this data are addressed if you simply read the two paragraphs that go along with it. There's no algorithm to mess things up, each tweet was looked at individually to determine if the word was used in a positive, negative, or neutral manner. It's normalized by overall the overall number of tweets in each county so that population density (or the popularity of twitter varying between regions) is not a factor. If you actually look at the links, this is all explained in detail (including all the words that are used, their variations, which ones had to be left out, etc.)

The only thing I couldn't find is how they deal with non-english tweets. If they normalize by the overall volume of tweets regardless of language, and then look at only english hate speech, then areas with higher proportions of non-english twitter users would be skewed towards being "non-hateful." This would be pretty trivial to correct by using only the volume of english language tweets, which I'm sure there is data for. I suspect they've already thought of this, but it wouldn't hurt to suggest it to them.

Apologies. I thought I misread the OP. That said, what they've instead introduced (and the very reason you use algorithms in the first place) is confirmation bias, selection bias, and perception bias.

Additionally, normalization by density doesn't really help if you don't correct for other factors, as I mentioned above.

_________________
“The duty of a patriot is to protect his country from its government.” - Thomas Paine

What confirmation bias? I don't see how that could be a factor unless the students who were classifying the tweets were given the location of the tweets they were classifying, which would just be retarded.

Selection bias? Of course there's a **** selection bias. They're looking at tweets. Most of the country doesn't even use twitter. But guess what, there's a selection bias in every study like this. That doesn't make the data unusable. Take election polling for example. There are selection biases against people who don't use landlines, don't answer calls from strangers, don't like answering questions from strangers, etc. But we still get very usable data to the point where we can predict the results of an election before it takes place with surprising accuracy.

Not sure what you mean by perception bias in this case.



I guess these are your problems with their normalization technique? Most of your speculations are retarded (except the spanish language one that I responded to above). How would the number of homosexuals in an area be a factor? How would you "correct" for the population of homosexuals? Why would racists be more likely to use twitter (compared to non-racists in their area) in one area of the US vs another? Of course it could be a factor, but you need to at least provide some evidence or an explanation of why you think it would be a significant one.

You seem to be upset that their data and their methodology isn't perfect, but this kind of data never is. That doesn't mean it's not useful.

Again, because they can't access demographic data about the Twits (people who tweet), they can't provide useful information.

_________________

Khross, I'm genuinely confused as to what you mean. What demographic data do they need to make their information useful?

All sorts of things: ethnicity, income, disabled status, gender, sex, etc.

_________________

Edit: Deleted due to coming to civil terms.

_________________
“The duty of a patriot is to protect his country from its government.” - Thomas Paine

I'm sorry if my tone was a little harsh DFK, I'm just in a weird mood right now. I really didn't mean to be so grating, but rereading my post I can see how it turned out that way. I shouldn't have called your reasoning retarded.

Khross, yeah, that would make it a lot more useful for making statements about the population as a whole, as opposed to just twitter users. I still think it's interesting given that limitation.

I'd still say it is pretty useless, given we're trying to track intent without even getting ALL of the stereotypes we're using to track this on the page ...

_________________

Read this a little better. My problem is that a lot of the country probably doesn't tweet about every little thing the way more media driven culture centers (New York, CA, LV) do. These means, as the pollster explained to his credit, that the ratio of "hate tweets":All tweets would be larger. I'm not saying it's utter garbage, but its something that needs to be considered.

There's lots of hate tweets in CA, NY, etc, but they are being lost in the load of "garbage tweets" that certain demographic areas probably have a lot more of them.

_________________
I prefer to think of them as "Fighting evil in another dimension"

Flyover tweets?

Perhaps. I'm just saying where do think people live who tweet about what their cat had for lunch and other pointless stuff like that.

_________________
I prefer to think of them as "Fighting evil in another dimension"

I'm sure you guys are right and this data doesn't actually correlate to anything. It's just a statistical illusion and the nation is actually more homogenous than it seems.