Deducing US presidential candidates' core demography with Facebook activists first names

In this blog post we will first gather data from Facebook and use that data to perform a simple comparison wordcloud. The end goal is to make simple inference on the demography of American presidential candidates support base. Names are not just words but they also carry information. Most obviously you can deduce the person's gender with reasonable confidence and his/her ethnicity with tolerable accuracy. It also carries a highly unreliable indication of it's carrier's age (some names are more common in certain era than others).

First, we begin by loading the required R libraries.

library(Rfacebook)
library(dplyr)
library(wordcloud)
library(tm)
library(RColorBrewer)

Rfacebook is used to access Facebook API and you can read a tutorial on setting it up for example here. Dplyr is the must-have package for data wrangling, making the horrendously unintuitive subsetting tasks of base R much more bearable and also introducing the neat pipe operator %>% which connects numerous operations called on data together, making your R code cleaner and more efficient.

Wordcloud is used for visualizing the, well, final wordcloud and RColorBrewer to give the plot a bit nicer visual look (as you can see from this blog I may not the be the most visually competent person). Tm is a package used for text mining and is used to create the term matrix to feed the comparison.cloud-function. Ok, let's begin!

load("fb_oauth") 

pages <- c("DonaldTrump","hillaryclinton") 

df <- data.frame()
for (i in 1:length(pages)) {
  temp <- getPage(pages[i], token=fb_oauth, n=30) 
  temp2 <- temp$id 
  helpframe <- data.frame()
 
  for (j in 1:length(temp2))  {
    post <- getPost(temp2[j],token=fb_oauth)
    id <- post$likes$from_id
    name <- post$likes$from_name
    likesdf <- data.frame(name,id,stringsAsFactors=FALSE)
    likes <- likesdf$likesname
    name <- post$comments$from_name
    id <- post$comments$from_id
    comsdf <- data.frame(name, id,stringsAsFactors=FALSE)
    alldf <- rbind(likesdf, comsdf)
    helpframe <- rbind(helpframe, alldf,stringsAsFactors=FALSE)
  }
  helpframe <- unique(helpframe)
  fullname <- helpframe$name
  candidate <- pages[i]
  candidate <- rep(candidate,length(fullname))
  helpdf <- data.frame(fullname, candidate,stringsAsFactors=FALSE)
  df <- rbind(df,helpdf,stringsAsFactors=FALSE)
}

Let's look a little closer what that code does. For preparations, first we load up the fb-authentication token I have saved earlier, as in the RFacebook-tutorial. Then we save the public Facebook pages I am interested in a vector called pages. I limited this blogpost to just the Hillary Clinton (who's deadset to win the democratic primary) and Donald Trump (who despite his strong showing in the polls remains a long-shot to actually win the Republican race) Finally we initialize the empty dataframe df for storing the data.

What follows is a nested loop. We use getPage-function from RFacebook to get page data for ith member of earlier initialized pages vector using the fb_oauth. N limits the amount of posts we collect from the page so in this example we get 30 latest posts from each page. We save this all in a dataframe called temp.

However, what we are really interested in is the details of the likes and comments found under the posts published on the page. For that we collect post id's from results of getPage and store them in temp2, also initializing helpframe for storing data in the phase. The second loop uses temp2 as input for getPost-function, much like pages was for getPage. What follows is getting the userID and username for likes and comments respectively. The rest is all about creating a succession of data frames to eventually collect all the userIDs and usernames from all the posts stored in temp2 to helpframe. After all likers and commenters under the page posts are collected, we leave the inner loop and go back to outer loop.

Here we use unique-function to select distinct username-userID pairs (the reason we collected id in the first place, calling unique on username alone might have dropped few unintentional doubles). After unique we no longer have any use for the id-variable so we select only the variable username to a vector called fullname. To accompany name information in our final dataframe, we create the variable candidate to give us a clue which presidential nominee's post said person had liked/commented on Facebook. We do this by using the page name from our pages vector and repeating it for the number of unique persons we collected in the inner loop. Finally, we store these two vectors in helpdf which we combine with our initially empty df to collect the data from all the iterations of the loop in a single data frame.

df$firstname <- word(df$fullname)


vectorofvectors <- list()
for (i in 1:length(pages)) {
 vector  <- df  %>% filter(grepl(pages[i],candidate))  %>% select(firstname)
 vector <- vector$firstname
  vectorofvectors[[i]] <- vector
  }


mycorpus <- Corpus(VectorSource(vectorofvectors))
mytdm <- TermDocumentMatrix(mycorpus)
mymatrix <- as.matrix(mytdm)
colnames(mymatrix) <- pages
comparison.cloud(mymatrix,colors=brewer.pal(6,"Dark2"),title.size=1,random.order = FALSE)

Now we have finally gathered the data from Facebook using a little nested for-loop. Next we use word-function to split the first name from the full name (we make the assumption that people usually put their first name, well, first in Facebook) and save it as third column of the dataframe df.

What we do next felt somewhat unintuitive and there must be a cleaner way to accomplish the task. Nevertheless, the idea behind the code was following:

First we initialize an empty list vectorofvectors to store name-data in vector format. This is because I want to use corpus-related functions from the tm package and want as many documents in my corpus as there are presidential nominees in pages. We use a little for-loop filter the rows where value of candidate equals the i:th value in pages. %>% - pipe operator is used to connect two operations, filter and select, from dplyr-package together in a nice and clean way, used on dataframe df. We turn the resulting one-column dataframe to more appropriate vector and store that in the list at i:th slot.

Finally we read the list into a corpus, turn the corpus to a term-document-matrix, and the term-document-matrix to a normal matrix. Before finally calling the comparison.cloud-function to visualize differences in the first names of the selected presidential nominees, we name the matrix columns to know which portion of the cloud belongs to which nominee. In the comparison.cloud-call, note the colors=brewer.pal-portion in which use colour palette from RColorBrewer to give some nicer visuals to the resulting plot.

And finally, we can enjoy the fruits of our labour:

Two things spring immediately to mind from the comparison cloud. First we notice that the Trump portion of the cloud is more crowded than the Clinton part. Trump did lead in the number of unique likers/commenters in our sample by almost 4000 people, 20 990 to Hillary's 17 151. But we could also deduce that the variability in the names of Trump supporters is smaller and thus they get a proportionally bigger share of the resulting pie, which holds only so many names (cloud collects names from the entire matrix, not for each column individually).

Secondly we see that Trump supporters seem to possess mostly male, Anglo-sounding names (John, James, Michael) whereas Hillary's supporters names are more feminine and ethnically diverse (Sarah, Jose, Mohammed).

I'd reckon most surveys would find the demographics of two nominee's supporters broadly in line with the above, simplistic deduction.