r-primer.Rmd

# An R primer {#primer}

What follows is a highly condensed guide to the basic features of the R programming language and some of its most useful general purpose packages. Use this primer as a quick introduction to the language, or as a reference when reading the rest of the book. You should also read at a minimum the chapters on "[Data Structures](http://adv-r.had.co.nz/Data-structures.html)" and "[Subsetting](http://adv-r.had.co.nz/Subsetting.html)" from *[Advanced R](http://adv-r.had.co.nz/)*, since understanding those concepts are crucial to working effectively in R.^[@wickham_advanced_2014, chs. 2--3.]

## Values

A **value** is the most basic piece of data. There are several kinds of values.

In R, you can generally work with numbers without thinking about the distinction between floating point numbers and integers. **Doubles** (for double-precision floating point numbers) are positive or negative numbers that can have a decimal point. **Integers** are positive or negative whole numbers. Even if a value does not have a decimal point, any number that you write is a double unless it is suffixed with `L` to mark it as an integer. 

```{r}
10.42
-8.67
10L
typeof(-8)
typeof(-8L)
```

Text is stored in **character** vectors (usually called strings in other programming langauges). Character vectors are marked by surrounding single or double quotation marks.

```{r}
"Everything is awesome"
"1" # not the same as 1
identical(1, "1")
```

**Logical** values, or booleans, can be either true or false.

```{r}
TRUE
FALSE
```

**Factors** are a special kind of value, where the values must be part of a predefined set of options. Factors are usually used for categorical data.

```{r}
factor(c("correct", "correct", "incorrect"), levels = c("correct", "incorrect"))
```

Each of these kinds of values has a corresponding missing value, `NA`. These are used for missing or unrecorded data.

```{r}
NA
NA_character_
NA_integer_
NA_real_
```

## Variables and assignment

Values can be stored in **variables**. A value is placed into a variable using the assignment operator, `<-`.^[I'm calling certain parts of R syntax operators, but technically they are special functions with syntactic sugar.]

```{r}
x <- 1
x
y <- -3.14
y
result <- "The results are compelling"
result
```

## Comparison

Values can be compared to one another. The result is always a vector of `TRUE` and `FALSE` values indicating whether the condition was met or not. 

```{r}
x <- 1:6 # a vector of numbers 1 to 6
x > 3
x >= 3
x < 3
x == 3
x != 3
```

Comparison also works with character vectors.

```{r}
"Is this the same?" == "as this?"
"Is this the same?" == "Is this the same?"
```

Notice that equality is tested with the `==` operater, not with `=`.

Often it is helpful to test whether elements of one vector are contained in another vector with the `%in%` function. Notice that the resulting vector is the same length as the vector on the left-hand side.

```{r}
c(1, 4, 2, 8, 0, 10) %in% c(1, 2, 3)

# Are these states a part of New England?
states <- c("VA", "CT", "MA", "SD", "GA", "AL", "ND", "SD", "VT") 
new_england <-  c("MA", "ME", "CT", "RI", "NH", "VT")
states %in% new_england
```

## Functions

R is a functional programming language, so knowing how to apply functions to problems, and to write them yourself, is essential to doing your work. If variables and values are the nouns in R, functions are the verbs.

### Using functions

**Functions** can be called on many different kinds of R vectors and objects. A function takes an input and produces an output. In this case we will use `sum()` to add up a vector of numbers.

```{r}
x <- c(1, 5, 2, 4, 2, 5)
sum(x)
```

Notice that calls to functions are always followed by parentheses, containing the function **arguments**. In the example above, `x` was the only argument to the function; it was the means by which we passed data into the function. Functions can often take many arguments that specify options to function. For instance, if we have a vector of numbers that contains an missing value (`NA`), then the function `sum()` will return `NA`, because `NA` has no value. But by using the `na.rm` argument to `sum()`, we can instruct it to ignore `NA` values.

```{r}
y <- c(1, NA, 2, 4, 2, 5)
sum(y)
sum(y, na.rm = TRUE)
```

R has many built-in functions. For instance, the `sort()` function can sort many different kinds of values.

```{r}
x
sort(x)
sort(x, decreasing = TRUE)

states
sort(states)
sort(states, decreasing = TRUE)
```

It is important to note that most functions you use to work with data are **pure functions**. The results of pure functions depend only on their inputs: the same input will always produce the same output. In general, R functions do not modify their inputs directly, but instead return a copy of the data if they modify the input. Notice that in this example the original `states` vector remains unsorted, even after it has been passed through the `sort()` function, and a new copy of the data, now sorted, is stored in the variable `states_sorted`. This makes it easier to reason about what a function will do and what has changed, but it is a different choice than other langauges make.

```{r}
states
states_sorted <- sort(states)

states
states_sorted
```

### Writing your own functions

You will often have to write your own functions. You can think of a function as encapsulating some action that you take on your data. Consider the following few sentences, each of which includes a year. How can we turn the year in each of these sentences into a number?

```{r}
dates <- c("The Louisiana Purchase happened in 1803.",
           "The Mexican-American war began in 1846.",
           "The Compromise of 1850 was drafted by Henry Clay.")
```

We can write a function that does this work for us. The function will be named `extract_year`. It will have a single argument, `sent`, which will be a sentence containing a year. The body of the function, between `{` and `}`, does the work of finding the 4-character string of digits then turning that into an integer. The last value in the function will be returned as its output.

```{r}
extract_year <- function(sent) {
  require(stringr) # Make sure that the stringr package is loaded
  year_char <- str_extract(sent, "\\d{4}") # Pull out the 4 digit year
  year_int <- as.integer(year_char) # Turn the year (a character) into an integer
  year_int # This is the value that will be returned
}
```

Now we can call the function on our data and get the desired result:

```{r}
extract_year(dates)
```

## Data structures and subsetting

Values in R can be stored in many different kinds of **data structures**, including vectors, lists, data frames, and matrices. These kinds of data structures share a set of operators for subsetting them, that is, for pulling out pieces of the data.

### Vectors

The most basic data structure is a vector. Consider this vector of 10 names.

```{r}
people <- c("Adam", "Betsy", "Charles", "Dana", "Edward", "Felicity", "George",
            "Hannah", "Ian", "Julia")
```

In R, elements of a vector are numbered, starting from `1` to the length of the vector. (Most programming languages have a 0-based indexing.) We can get individual elements of the vector using a numeric vector and the `[` subsetting operator. For example, we can get the fifth element, or the first, eighth, tenth, and sixth elements together.

```{r}
people[5]
people[c(1, 8, 10, 6)]
```

We can also subset a vector by using a logical vector. When we use a logical vector, we get back the elements that correspond to the `TRUE` values. It is usually a good idea to use a logical vector which has same length as the original vector. In this example, we use a function to test whether the names in our `people` vector have a lowercase `e` in them. Then we use that logical vector and the `[` subset operator to return only the names that have that letter in them.

```{r}
library(stringr)
# The result of this function call is a logical vector
str_detect(people, "e") 

people[str_detect(people, "e")]
```

It is possible for a vector to have names, which we can also use for subsetting. Suppose we have a set of numbers from `1` to `10` that correspond to a rank for each person. At first our vector does not have any names.

```{r}
rank <- c(9, 8, 4, 10, 7, 3, 2, 5, 1, 6)
names(rank)
```

But we can use our `people` vector to give it names.

```{r}
names(rank) <- people
```

Now we can get the fifth rank and see the corresponding name.

```{r}
rank[5]
```

We can use a character vector to subset `rank` to using those names.

```{r}
rank[c("Adam", "Hannah")]
```

And we can sort `rank` and see that the names are also sorted, or get only the elements of rank which are above a certain value (another example of logical subsetting).

```{r}
sort(rank)
rank[rank <= 3]
```

### Lists

Lists are conceptually an extension of vectors. They are a vector which can contain other vectors, each of which can contain a different kind of information. Lists can thus have an arbitrary structure. Let's create a list that describes a historical event, recording certain key pieces of information. Notice that some of the information is numeric and some is textual; some of the fields have a single entry, and some are a vector of values.

```{r}
event <- list(
  name = "Louisiana Purchase",
  date = 1803,
  price = 15000000,
  nations = c("France", "United States"),
  negotiators = c("Napoleon", "Thomas Jefferson")
)
```

Because lists can have an arbitrary structure, you can use the `str()` function to see what a list contains. (This function works on any R object.) Notice that this is a list that has five elements, and that each of those elements has a name (`date`, `price`, and so on).

```{r}
str(event)
```

Just like a regular vector, you can subset a list with a numerical index.

```{r}
event[2]
```

But notice that the resulting value is a list containing the date. We have selected an element of the list, but that element still comes wrapped in the list. We can get what is contained inside each element by using the double bracket subset operator, `[[`. Notice that the value returned is now the number that corresponds to the date.

```{r}
event[[2]]
```

Just like with regular vectors, you can also subset a list by its names if it has them. 

```{r}
event["date"]
```

The single bracket subset operator `[` gives us a list, but we can get a named element itself with the double bracked subset operator `[[`.

```{r}
event[["date"]]
```

There is a special syntax for accessing a named element of a list using the `$` operator. This is the equivalent of `event[["date"]]`.

```{r}
event$date
```

### Data frames

A data frame is like a spreadsheet, since it is a table with columns and rows. It is the data structure that we will use most in this book, though we will mostly use the [dplyr](https://cran.r-project.org/package=dplyr) package instead of base R to manipulate tabular data. Technically a data frame is a list where the constituent vectors are all the same length, and so you can use the same subset operators on data frames. Even if you mostly use [dplyr](https://cran.r-project.org/package=dplyr), it is still important to understand how data frames work in base R.^[The [tibble](https://cran.r-project.org/package=tibble) package, part of the [tidyverse](https://cran.r-project.org/package=tidyverse), changes a few things about how data frames function in R. It makes them print in a more friendly way, but it also changes the `[` and `[[` operators so that they do not use the argument `drop = TRUE` by default. See that package's documentation for more detail.]

We can load a data frame of historical state populations from the [historydata](https://cran.r-project.org/package=historydata) package. In an interactive session, you can get a nice view of the data frame by running `View(us_state_populations)`.

```{r, warning=FALSE, message=FALSE}
library(historydata)
library(tidyverse)
us_state_populations
``` 

You can get the vector for any column by using the column name with `$`. 

```{r}
head(us_state_populations$state)
```

You can also subset a data frame using `[`. But because data frames are two dimensional, you should specify a subset index for both rows and columns. Leave either one blank to specify all columns or all rows. In the examples below there are extra spaces to make it clear where we are subsetting by rows and where by columns.

```{r}
us_state_populations[5:8,    ]    # rows 5-8, all columns
us_state_populations[   , 2:3]    # all rows, columns 2-3
us_state_populations[5:8, 2:3] # rows 5-8, colums 2-3
```

You can subset a data frame using a logical operator. Here we use a common idiom to find create a logical vector where we check whether each row in the `state` column is equal to `"Massachusetts"`, and then use that vector to subset the data frame's rows. (Notice the space after the `,`.) The result is that we have filtered the data frame to just the Massachusetts population. This is much less verbose in [dplyr](https://cran.r-project.org/package=dplyr), but there are times when you will need to manipulate a data frame in base R.

```{r}
us_state_populations[us_state_populations$state == "Massachusetts", ]
```

### Matrices

A matrix is a very common data structure in computational history, useful for holding counts of words in documents, similarities between texts, distances between points, and the like. A matrix is a vector where all of the elements are of the same type, but it is arranged in two dimensions. Consider this matrix, which could represent how similar documents are to one another.

```{r}
m <- matrix(c(NA, NA, NA, NA, NA, 0.31, NA, NA, NA, NA, 0.29, 0.95, NA,
              NA, NA, 0.66, 0.7, 0.18, NA, NA, 0.41, 0.3, 0.49, 0.7, NA),
            nrow = 5, ncol = 5)
rownames(m) <- letters[1:5]
colnames(m) <- letters[1:5]
m
```

Notice that we only need the upper triangle of the matrix because presumably the similarities of two documents are the same in either direction, and the similarity of a document to itself is not a very meaningful question. You can subset a matrix with `[` by rows and columns. Here we get the similarity of `a` to `c`, and then the similarity of `a` to every text.

```{r}
m["a", "c"] 
m["a", ]
```

Notice that the `[` operator simplifies the results to a one-dimensional vector or single number if possible, rather than giving us a matrix back. This is sometimes undesirable, and we can avoid it by using the `drop = FALSE` argument to `[`.

```{r}
m["a", "c", drop = FALSE]
m["a", , drop = FALSE]
```

## Iteration

One of  the reasons to use a computer is that computers don't mind doing the same thing over and over again. We call that iteration. 

Note that because in R everything is a vector and most functions are vectorized, you won't need to think about iteration explicity nearly as often as you might in other language. Take a simple example:

```{r}
letters[1:10] %in% c("a", "e", "i", "o", "u")
```

To check whether an individual letter is in a vector of vowels, we didn't have to loop through each letter and each vowel. R handles the iteration for us.

Or take a slightly more complicated example, in which we filter the rows of a data frame.

```{r}
dijon_prices %>% 
  filter(commodity == "best wheat")
```

Again, we didn't need to explicitly loop through each row and check its commodity. 

In general, R saves us a lot of time and lets us write very clean and readable code because we don't have to handle obvious cases of iteration. That said, there are times when you need to handle iteration yourself. This can be done by using functional programming, or by using `for` loops.

### Iteration with functional programming

In functional programming, the idea is that you take write a function that does some work on each individual item of a vector or list, and then apply that function to each item in the vector using another function. R has a number of built-in functions for doing so, including `lapply()`, `vapply()`, `sapply()`, `Map()`, `Reduce()` and so on. These functions aren't entirely consistent, and for the most part you are better off using the functions in the [purrr](https://cran.r-project.org/package=purrr) package, which is part of the tidyverse.

Let's write a fairly-straightforward function which checks whether a website is working correctly or not. If you ask a web server for a web page, it will return a number, called a [status code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status), indicating whether that URL is available or not. This function takes a URL as its input and returns the status code as its output. So that we can see what is happening, each time it checks a website, it will print a message with the URL.

```{r}
check_status <- function(url) {
  message("Checking ", url)
  result <- httr::HEAD(url)
  result$status_code
}
```

We can run that function to check a single URL.

```{r}
check_status("https://dh-r.lincolnmullen.com")
```

The result that we get back, status code `200`, means that the website is working correctly and that page is available.

This works fine for a single website, but what if we have a list of website we want to check?

```{r}
websites <- c("https://dh-r.lincolnmullen.com/",
              "https://lincolnmullen.com/projects/slavery/",
              "https://lincolnmullen.com/this-does-not-exist/")
names(websites) <- c("DH-R", "Map of slavery", "Does not exist")
```

We don't want to call the function over and over again. Instead we can use the function `map_int()` from the purrr package. That function will take two inputs. The first is a vector of inputs. The second is a function which will be called one time for each of the inputs. The output will be a vector of integers.

```{r}
library(purrr)
results <- map_int(websites, check_status)
```

We can see that the function was called three times, once for each of the websites in our vector. We can also see what the results were:

```{r}
results
```

Two of the websites are working fine (status code `200`). One of the websites was not found, because it does not exist (status code `404`.)

As you might guess, we used the `map_int()` function because we were expecting a vector of integers. The purrr package contains other type-safe functions, such as `map_chr()` for characters, `map_dbl()` for numbers, and `map()` for lists. 

Once we have bought into the functional programming approach, we can use other functions to further manipulate our data. For instance, once we have our vector of results, we can write a function to check whether a website is okay or not. Then we can keep or reject the results based on that test.

```{r}
is_ok <- function(code) { code == 200}
keep(results, is_ok)
discard(results, is_ok)
```

The full scope of functional programming is well beyond the scope of this primer. For more details, see the chapters in Hadley Wickham's [*Advanced R*](http://adv-r.had.co.nz) and the [purrr package documentation](https://purrr.tidyverse.org).