StatOracle–05 Working with Data in R
To work effectively with R, one must master a range of tools that can be used for creating and transforming data.
Working with Data in R
5.1. Logical and arithmetic operations in R
For an unprepared reader, the text of this chapter of our textbook may appear unreadable: it will consist primarily of R dialogues. True, at first, before gaining experience, reading such dialogues is difficult. But the very purpose of our course is to teach students to work with R. An experienced R user, when working with R, quickly picks out commands and the system’s responses to them. That is exactly what is shown in the R dialogues in our textbook. Dear readers, when your eyes (and in reality — your brain) learn to read R dialogues with ease, you will be able to conclude that you have made significant progress in mastering this language.
By the way, what is the best way to work through the R dialogues presented here? One approach (when working with an electronic textbook) is to select and copy the text from the box. Discard the system’s responses to the entered commands (lines that do not begin with the > sign) — only the commands themselves will remain. Remove the > signs at the beginning of each line — these were inserted by RStudio when it displayed in the console that it had received those expressions as commands. Paste the remainder into the RStudio script editor, select what you need, and press Ctrl+Enter or the Run button (the green arrow). Did you get the same result as in our example? What can you change? How can you apply this to the problem you are solving?
We will examine several more approaches to working with the PelophylaxExample data frame, and then move on to a more systematic overview of R’s capabilities. In a certain sense, we are using a “flipped” logic for introducing R. We began with the example in Figure 3.2.1, which showed how R reads a data file, performs calculations, creates and saves plots — all by executing a fairly simple script. After a simple example with data entered directly in the R console, we created the PelophylaxExample frame and examined some tools for working with it. This provided an occasion to discuss the types of objects and data in R. It will now be logical to proceed more systematically through R’s capabilities and examine logical and arithmetic operations, as well as certain features of vectors, and then apply these to working with our data frame. The authors hope that this “flipped” approach to introducing R will make it clearer why the various features of this powerful data-analysis tool can be useful. This approach is not suited to a reference manual, but we hope it will ease the introduction to R for students who work through each point and topic of this textbook in sequence.
For working with data files, it is important to learn how to apply logical constructs. Examples of these are shown in R dialogue 5.1.1. Looking ahead, we note that logical constructs help, for example, to select data that satisfy certain conditions.
R dialogue 5.1.1. Examples of logical constructs in R
> a <- 2; b <- 5 # Create two objects > a < b # Less-than / greater-than comparison [1] TRUE > a > b [1] FALSE > a <= b [1] TRUE > > # If you write a = b, R will redefine a by b (treating it as a <- b) > a == b # Testing equality ==, inequality != [1] FALSE > a != b [1] TRUE > > # The & sign means "simultaneously", and | means "or", "at least one of": > a & b < 4 [1] FALSE > a & b <= 5 # 'a and b are greater than or equal to 5' [1] TRUE > a | b == 2 # 'at least one of a and b equals 2' [1] TRUE > a<3 & b>3 # Combining two conditions [1] TRUE > a<3 & b>5 [1] FALSE > |
Both when using logical conditions and, of course, in countless other situations, arithmetic operations are performed. Some of these are trivial; others warrant individual mention (R dialogue 5.1.2).
R dialogue 5.1.2. Some arithmetic operations: remainders and integer parts of division, exponentiation, logarithms
> a <- 2; b <- 5 # Create two objects > b %% a # Remainder of division [1] 1 > b %/% a # Integer part of the division [1] 2 > a ** b # Exponentiation (^ also works) [1] 32 > sqrt(4) # Square root [1] 2 > 1000 ** (1/3) [1] 10 > round(a/b) [1] 0 > round(a/b, 2) # Rounding to a specified number of digits [1] 0.4 > abs(a - b) # Absolute value [1] 3 > log(2) > log(2) # By default, natural logarithms are computed (base e, Euler’s number) [1] 0.6931472 > log10(1000) # Base-10 logarithm [1] 3 > log(4, base = 2) # Logarithm to a specified base [1] 2 > |
5.2. Missing values
The R language uses the special notation NA for missing values. Recall the columns with frog body-part measurements in the file PelophylaxExamples. Table 2.2.1 has no unfilled cells. But suppose a particular value is unknown to us. What should we do? Under no circumstances should the value 0 be used: that would mean we measured a particular body part and recorded its length as zero. We must indicate that this is precisely an absence of information. The simplest solution is to leave the corresponding cell of the data table empty. In R, this corresponds to placing the notation NA — missing data — in that “cell”. Incidentally, NA is not a text value; its notation need not be enclosed in quotation marks. It represents the genuine absence of a value.
We can read the file PelophylaxExamples.csv in such a way as to create a file with missing values. We will use the na.strings argument to the function read.csv(). We will create a different object from our main frame PE for this purpose. For example, if we use the command PE1 <- read.csv('PelophylaxExamples.csv', sep = ";", dec = ",", na.strings = '"Dnipro'), the file-reading function will discard any mention of the Dnipro basin. However, it will be more interesting if we lose some numerical values. For example, let us discard the value 42, which appears in the file twice: it is the length of the secondary tibia in two frogs, one of which is the first (R dialogue 5.2.1).
Incidentally, if you are reading a *.csv file that contains empty cells, you must tell R to interpret those cells as NA: DATAFRAMENAME <- read.csv('FILENAME', sep = "SIGN", dec = "SIGN", na.strings = '"").
R dialogue 5.2.1. Missing values and some operations with them
> > # An artificial frame containing missing values is created:
> PE1 <- read.csv('PelophylaxExamples.csv', sep = ";", dec = ",", na.strings = 42)
> str(PE1$Ci)
int [1:57] NA 41 37 38 45 37 37 34 31 33 ...
> complete.cases(PE1) # A function that selects the rows that have no missing values:
[1] FALSE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
[15] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
[29] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE FALSE TRUE TRUE
[33] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
[47] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
> sum(complete.cases(PE1)) # It can be counted, since the logical value TRUE corresponds to 1
[1] 55
> sum(!complete.cases(PE1)) # Counting the rows that are not complete
[1] 2
> sum(is.na(PE1)) # And this is simply a count of the number of missing values in the file:
[1] 2
> mean(PE1$Ci) # Calculating the mean (and much else) with NA is impossible:
[1] NA
> > # But it is possible to add the argument (from NA remove) that forces missing values to be ignored:
> mean(PE1$Ci, na.rm = TRUE)
[1] 37.65455
>
|
Note: the command sum(is.na( OBJECT )) counts the number of missing values, and sum(!complete.cases( OBJECT )) counts the number of rows with missing values. Will the commands sum(!complete.cases( OBJECT )) and sum(is.na( OBJECT )) always give the same result? No. If at least one row has more than one missing value, the number of incomplete rows will be less than the total number of missing values. How to generally remove rows with missing values from the PE1 data frame? This will become clear later (R-dialog 5.5.2)... 5.3. Vectors, their creation and selection of specific elements As you remember, a vector is a basic type of object for R. A vector is an object that has a name and contains an ordered set of data of a certain type (even if this set consists of a single element). Using vectors as an example, we can explain the properties of R objects and functions that will help when working with other objects. We have already used the c() function, which combines objects into a vector. R-dialog 5.3.1. Creating vectors and some operations with them
> > # Short commands can be written on one line separated by semicolons > > # To display a created element in the console, you can enclose the command in parentheses > > # Let us create a vector using the rep() command, which repeats a given content a specified number of times, combine it with another vector, merge them and display in the console — all in a single line > re <- rep(c(1, 0, NA), 2); ad <- c(1, 2, 3, 4); (v <- c(re, ad)) [1] 1 0 NA 1 0 NA 1 2 3 4 > unique(v) # Determine the unique elements of vector v [1] 1 0 NA 2 3 4 > table(v) # Frequencies of the elements of vector v v 0 1 2 3 4 2 3 1 1 1 > v1 <- v + 1 > # Arithmetic operations can be performed > v1 # NA elements are not changed in this case: [1] 2 1 NA 2 1 NA 2 3 4 5 > (v2 <- v * 2) [1] 2 0 NA 2 0 NA 2 4 6 8 > v2 > 5 # Logical operations can be performed [1] FALSE FALSE NA FALSE FALSE NA FALSE FALSE TRUE TRUE > v2[v2 > 5] # Elements can be selected using logical operations [1] NA NA 6 8 > v2[v2 > 5 & v2 < 7] # Different operations can be combined [1] NA NA 6 > |
As you can see, the rep() command ensures the repetition of a certain sequence. In addition to it, the seq() command, which generates a certain arithmetic sequence, can be useful for creating vectors. For example, the command se <- seq(from = 1, to = 24, by = 2), or its identical, more concise command se <- seq(1, 24, 2), creates a vector se, which starts from 1 and consists of elements, each of which is 2 greater than the previous one (the last value will be 23, do you understand why?). The unique() function removes duplicates and leaves unique values; the table() function displays the distribution of values in the object it analyzes. The main characteristics of a vector are its type mode() and length lenght(). Using the lenght() function, you can not only find out the length of a vector but also change it. The command length( VECTOR ) <- 10 will change the vector's length to the specified number of elements (if there were more, it will cut off the excess; if there were too few, it will add NA). Using the vector v, created in R-dialog 5.3.1, let's consider some more ways to work with vectors (R-dialog 5.3.2). R-dialog 5.3.2. Indexing vectors and removing NA
> v [1] 1 0 NA 1 0 NA 1 2 3 4 > v[2] > v[2] # To obtain an element of a vector, specify the index of that element: [1] 0 > v[2:4] # To get several elements you need a vector (2:4 is a vector!): [1] 0 NA 1 > > # The command v[2, 5, 11] will not execute; R will treat 2, 5 and 11 as coordinates > v[c(2, 5, 11)] # A vector can be created with the c() function [1] 0 0 NA > v%%2 == 1 # Logical constructs can be used [1] TRUE FALSE NA TRUE FALSE NA TRUE FALSE TRUE FALSE > is.na(v) # This construct selects the missing values [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE > (v3 <- v[!is.na(v)]) # And this one removes missing values [1] 1 0 1 0 1 2 3 4 > |
In the previous section, we discussed missing values using the PE1 data frame as an example. Can we remove missing rows from it in the same way as from a vector? No, because a data frame is a more complex object. How to do this will become clear after we discuss working with objects that have more than one dimension. Created vectors can, of course, be edited. In the previous R-dialog, we created the vector v3. Let's change it (R-dialog 5.3.3). R-dialog 5.3.3. Adding and removing elements from a vector; smallest, largest, repeated values
> v3 [1] 1 0 1 0 1 2 3 4 > ap <- c(8, 8, 2) # What is added to the vector should be converted into another vector > (v4 <- append(v3, ap)) # Thanks to the parentheses, the created vector is printed to the console [1] 1 0 1 0 1 2 3 4 8 8 2 > rem <- c(2:5, 10) # To remove something, you also need to create a vector > (v5 <- v4[-rem]) [1] 1 2 3 4 8 2 > which.max(v3) # Indicates the index of the largest element in the vector [1] 8 > which.min(v3) # Indicates the index of the smallest element in the vector [1] 2 > v3[c(7, 3, 5, 2, 7, 7, 7)] # Sets (with a vector!) the order in which the elements are output [1] 3 1 1 0 3 3 3 > v3[9:1] # A missing element's index will correspond to NA [1] NA 4 3 2 1 0 1 0 1 > (v3[c(-2, -3, -5)]) # Negative indices are a way to delete an element [1] 1 0 2 3 4 > duplicated(v4) # Finding repeated values: [1] FALSE FALSE TRUE TRUE TRUE FALSE FALSE FALSE FALSE TRUE TRUE > (v5 <- v4[!duplicated(v4)]) # Repeated values can be removed [1] 1 0 2 3 4 8 > v6 <- unique(v4) # Another way to do the same thing > v5 == v6 # Check [1] TRUE TRUE TRUE TRUE TRUE TRUE > |
As you can see, the rep() command provides repetition of a given sequence. In addition to it, the seq() command, which generates a given arithmetic sequence, may be useful for creating vectors. For example, the command se <- seq(from = 1, to = 24, by = 2), or the equivalent abbreviated command se <- seq(1, 24, 2), creates the vector se, which begins at 1 and consists of elements each of which is 2 greater than the previous one (the last element will be 23 — can you see why?).
The function unique() removes duplicates and retains unique values; the function table() displays the distribution of values in the object it analyses.
The key characteristics of a vector are its type mode() and its length length(). Using the function length(), one can not only determine the length of a vector but also change it. The command length(VECTOR) <- 10 will change the vector’s length to the specified number of elements (if there were more — excess elements are cut off; if there were fewer — NA values are added). Using the vector v created in R dialogue 5.3.1, let us examine a few more ways to work with vectors (R dialogue 5.3.2).
R dialogue 5.3.2. Vector indexing and removal of NA values
> v5 > v5 # We will experiment with this vector (R dialogue 5.3.3) [1] 1 2 3 4 8 2 > (v6 <- order(v5)) # Indicates the indices of the elements in the vector in ascending order [1] 1 2 6 3 4 5 > (v7 <- order(v5, decreasing = TRUE)) # And with this argument, in descending order [1] 5 4 3 2 6 1 > (v8 <- sort(v5)) # Sorts the elements of the vector in ascending order [1] 1 2 2 3 4 8 > (v9 <- v5[order(v5)]) # Alternatively, with the same result as in the previous case [1] 1 2 2 3 4 8 > (v10 <- rank(v5)) # Indicates the rank of the elements; for repeated elements the rank is a non-integer [1] 1.0 2.5 4.0 5.0 6.0 2.5 > (v11 <- rank(-v5)) # Reverse ranking order [1] 6.0 4.5 3.0 2.0 1.0 4.5 > |
Compare the functions order() and rank(). If a vector has no repeating elements, they give the same result; if such elements exist, they will be processed differently. If a vector (or other object) is text, the functions sub and gsub can be used for editing. If these functions are applied to a numeric vector, it will be converted to text, but there is nothing to prevent it from being converted back to numeric (R-dialog 5.3.5). R-dialog 5.3.5. Examples of replacing characters in a vector
> > # Replacing characters in text objects:
> (char_obj <- c("comp1", "comp2", "comp3", "comp4"))
[1] "comp1" "comp2" "comp3" "comp4"
> (char_obj1 <- sub("comp", "comp_", char_obj))
[1] "comp_1" "comp_2" "comp_3" "comp_4"
> > # The command sub replaces the first occurrence, gsub replaces all occurrences
> (char_obj2 <- gsub("comp", "comp_", char_obj))
[1] "comp_1" "comp_2" "comp_3" "comp_4"
> (num_obj <- 1:5)
[1] 1 2 3 4 5
> (num_obj1 <- sub("2", "3", num_obj))
[1] "1" "3" "3" "4" "5"
> > # The object num_obj1 has become textual; it must be converted back to numerical!
> (num_obj1 <- as.numeric(num_obj1))
[1] 1 3 3 4 5
> > # The same thing is appended to all elements; this can no longer be converted to numbers:
> (num_obj <- paste0(num_obj, "-th"))
[1] "1-th" "2-th" "3-th" "4-th" "5-th"
>
|
When values in a vector are repeated, the functions which.max() and which.min() will return the index of the first of the maximum or minimum values. An example of this can be seen in R dialogue 5.3.3 when applying the function which.min(): the minimum value, 0, is located at the second and fourth positions in vector v3; the answer to the command which.min(v3) is 2.
In many situations, various means of sorting elements are useful (R dialogue 5.3.4).
R dialogue 5.3.4. Various approaches to sorting, ordering, and ranking elements
> ma <- 1:28 # Create a vector
> dim(ma) <- c(4, 7) # Specify the matrix dimensions from the vector
> ma # Look at what we got
[,1] [,2] [,3] [,4] [,5] [,6] [,7]
[1,] 1 5 9 13 17 21 25
[2,] 2 6 10 14 18 22 26
[3,] 3 7 11 15 19 23 27
[4,] 4 8 12 16 20 24 28
> colnames(ma) # This matrix does not yet have row and column names
NULL
> rownames(ma)
NULL
> # These names can be set:
> (colnames(ma) <- c("One", "Two", "Three", "Four", "Five", "Six", "Seven"))
[1] "One" "Two" "Three" "Four" "Five" "Six" "Seven"
> (rownames(ma) <- c("First", "Second", "Third", "Fourth"))
[1] "First" "Second" "Third" "Fourth"
> dimnames(ma) # View the row and column names at the same time
[[1]]
[1] "First" "Second" "Third" "Fourth"
[[2]]
[1] "One" "Two" "Three" "Four" "Five" "Six" "Seven"
> ma # Now the matrix looks like this
One Two Three Four Five Six Seven
First 1 5 9 13 17 21 25
Second 2 6 10 14 18 22 26
Third 3 7 11 15 19 23 27
Fourth 4 8 12 16 20 24 28
>
|
A data frame can be compared to a list of relatively independent (containing different data) vectors (R-dialog 5.4.2): R-dialog 5.4.2. Examples of working with row and column names in a data frame
> df1 <- c("One", "Two", "Three", "Four", "Five", "Six", "Seven")
> df2 <- 8:14 ; df3 <- 15:21; df4 <- 22:28
> (df <- data.frame(df1, df2, df3, df4)) # Created a data frame
df1 df2 df3 df4
1 One 8 15 22
2 Two 9 16 23
3 Three 10 17 24
4 Four 11 18 25
5 Five 12 19 26
6 Six 13 20 27
7 Seven 14 21 28
> dimnames(df) # Row and column names
[[1]]
[1] "1" "2" "3" "4" "5" "6" "7"
[[2]]
[1] "First" "Second" "Third" "Fourth"
> df > df # If needed, colnames() and rownames() can be used
First Second Third Fourth
1 One 8 15 22
2 Two 9 16 23
3 Three 10 17 24
4 Four 11 18 25
5 Five 12 19 26
6 Six 13 20 27
7 Seven 14 21 28
>
|
5.5. Indexing data using the PelophylaxExample data frame as an example Indexing in R (and in similar cases) is the access to specific parts of an object. Indices (methods for selecting a specific part of the data) can be numeric, logical, or textual. For statistical analysis, it is often necessary to obtain and compare different fragments of a complete data frame with each other. For this, indexing methods shown in R-dialog 5.5.1 are required. R-dialog 5.5.1.
> PE <- read.csv('PelophylaxExamples.csv', sep = ";", dec = ",")
> PE[1, 6] # In square brackets: first the row, then the column
[1] female # For a factor, its values and levels will be shown:
Levels: female male
> PE[1, 9] # For a numeric variable, simply the values will be shown:
[1] 603
> PE[2, ] # If a coordinate is not specified, all values are selected
X Place East North Basin Sex DNA Genotyp L Ltc
2 LL_f_562 Chernetchina 35.13 50.05 Dnipro female 13.95 LL 562 187
Fm T Dp Ci Cs
2 266 249 62 41 152
> PE[2:5, c(9, 11, 13)] # Coordinates can be set with vectors of element indices
L Fm Dp
2 562 266 62
3 592 281 79
4 595 285 75
5 602 287 80
> PE[10, c('Sex', 'Genotyp')] # A vector with variable or row names can be created
Sex Genotyp
10 female LLR
> PE[which.max(PE$L), 'L'] # The largest body length (L) recorded in the data frame
[1] 930
> max(PE$L) == PE[which.max(PE$L), 'L'] # A simpler way to get the maximum
[1] TRUE
> PE[which.min(PE$L), 'L'] # The smallest value of the variable L
[1] 479
> PE[PE$Place == 'Zamulivka', c(6, 8)] # Conditions can be used!
Sex Genotyp
14 female LLR
15 female LLR
56 male RR
> PE_m <- subset(PE, Sex=="male") # Only part of the data can be selected
> > # Fairly complex logical constructs can be used:
> PE_f_2n <- subset(PE, Sex=="female" & (Genotyp=="LL" | Genotyp=="LR"| Genotyp=="RR"))
> PE_f_2n[c(4, 8, 12), c(1, 2, 6, 8)] # Let us look at the result:
X Place Sex Genotyp
5 LL_f_602 Chernetchina female LL
24 LR_f_877 SuhaGomol`sha female LR
46 RR_f_701 Lipci female RR
>
|
In R-dialog 5.3.3, the commands which.max() and which.min() are explained. Since they allow obtaining the index of the largest or smallest element, they can be used to obtain these values themselves, using commands such as: PE[which.max(PE$L), 'L']. However, the maximum and minimum values can be obtained in a simpler way, using the commands max(PE$L) and min(PE$L). Pay attention to the command in R-dialog 5.5.1: PE[PE$Place == 'Zamulivka, c(6, 8)]. In fact, this is the question: which frogs, by their sex and genotype, originate from Zamulivka in the studied sample? The first coordinate, the address by rows, is given here by a condition (originating from a specific locality), the second coordinate, the address by variables, is given by a vector that defines which attribute values should be obtained for cases that meet the specified condition. The command PART <- subset( DATAFRAME , CONDITION ) provides the broadest possibilities for selecting a part of the data from a data frame. PE_f_2n <- subset(PE, Sex=="female" & (Genotyp=="LL" | Genotyp=="LR" | Genotyp=="RR")) effectively instructs R to create an object that includes females having one of the three diploid genotypes (LL, LR, or RR). Earlier (R-dialog 5.2.1), we created a version of the data file that contained missing values. We learned how to remove missing values from vectors and calculate statistics for vectors without considering missing values. When working with matrices and data frames, it often makes sense to simply delete all rows containing missing values (R-dialog 5.5.2). R-dialog 5.5.2. Two ways to remove rows with missing values from a data frame
> PE1 <- read.csv('PelophylaxExamples.csv', sep = ";", dec = ",", na.strings = 42)
> > # Creating a frame that contains missing values, NA
> PE2 <- PE1[complete.cases(PE1), ] # The first way to remove rows containing NA
> PE3 <- na.omit(PE1) # The second way to remove rows containing NA
> dim(PE1) == dim(PE2) # The obtained results are equal in size
[1] TRUE TRUE
> > sum(PE2 != PE3) # Number of differences between two data frames
[1] 0 [1] 0 # The result demonstrates the complete identity of all elements of PE2 and PE3
>
|
5.6. Working with columns of the PelophylaxExample frame
An important part of working with data involves creating, modifying, and deleting variables themselves. Let us examine several useful commands. First of all, let us return to the example discussed in section 4.4. There we created two vectors: L and Ltc. To combine them into a data frame (let us call it temp), execute the command temp <- data.frame(L, Ltc).
Let us return to the frame PE. As you will recall, the first column had no name. R assigned it the name X. Incidentally, if we enter the command PE$X (i.e., FRAMENAME$VARIABLENAME), R will display in the console all 57 specimen codes. In that case, it makes logical sense to rename this column: names(PE)[1] <- "Code". The number in square brackets is the column index; this is how we tell R to change the name of the first variable specifically.
How can names be removed if they become unnecessary? Assign them the special operator NULL! This can be done, for example, as follows: colnames(DATAFRAME) <- NULL. The names will be converted to NA, and columns can then be accessed only by index.
A more significant restructuring of the PelophylaxExample frame concerns the transition from absolute values of metric traits to proportional ones. The point is that when comparing frogs that differ in size (above all in body length, L), there is no sense in comparing their metric traits relating to individual body parts. A frog whose head width (Ltc) is 200 units (20 mm) — does it have a narrow head or a wide one? If its body length is 50 mm, its head is wide; if its body length is 70 mm, its head is narrow. Therefore, in most analyses it is optimal to use body length in its absolute value, and all other metric traits in proportional form, as a fraction obtained by dividing the absolute value by body length. Incidentally, when studying other animals it may be appropriate to use some other trait as the “module” rather than body length. For example, in studies of birds and bats, the reference trait against which proportions are calculated is the length of the humerus.
There are several ways to add new columns with proportional traits to the frame PE. The simplest is as follows. In the script editor (or directly in the console), add and execute the command PE$Ltc_L <- PE$Ltc / PE$L and all analogous commands. When R receives such a command regarding a column that does not yet exist, it immediately both creates that column and performs the necessary calculations.
When creating new columns, logical expressions can also be used. Suppose we are interested in the relationship between head width and other traits. In the column Ltc_L we have calculated relative head width, but we wish to create an additional column, HeadWidth, in which frogs are divided into three groups: those with average head width (middle), narrow (narrow), and wide (wide). How should the boundaries between these classes be drawn? One approach is to use quartiles. Frogs falling in the first quartile are designated as narrow-headed, and those in the fourth quartile as wide-headed. How to do this is shown in R dialogue 5.6.1.
R dialogue 5.6.1. Creating new columns in the PelophylaxExample frame
> > PE <- read.csv('PelophylaxExamples.csv', sep = ";", dec = ",")
> names(PE)[1] <- "Code" # Assigning row names by the frog codes
> PE$Ltc_L <- PE$Ltc / PE$L # Creating columns with proportions
> PE$Fm_L <- PE$Fm/ PE$L
> PE$T_L <- PE$Ltc / PE$L
> PE$Dp_L <- PE$Dp / PE$L
> PE$Ci_L <- PE$Ci / PE$L
> PE$Cs_L <- PE$Cs / PE$L
> (su <- summary(PE$Ltc_L)) # More detail on the relative head width
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.2849 0.3345 0.3550 0.3554 0.3738 0.4343
> (as.vector(su)) # Convert the results output into a vector
[1] 0.2849462 0.3344538 0.3549669 0.3554291 0.3738318 0.4342688
> PE$HeadWidth[PE$Ltc_L >= su[5]] <- "wide"
> PE$HeadWidth[PE$Ltc_L < su[5] & PE$Ltc_L > su[2]] <- "middle"
> PE$HeadWidth[PE$Ltc_L <= su[2]] <- "narrow"
> table(PE$HeadWidth) # Look at the result
middle narrow wide
27 15 15
>
|
A data frame can be compared to a list of relatively independent vectors (which may contain different types of data) (R dialogue 5.4.2):
R dialogue 5.4.2. Examples of working with row and column names in a data frame
> df1 <- c("One", "Two", "Three", "Four", "Five", "Six", "Seven")
> df2 <- 8:14 ; df3 <- 15:21; df4 <- 22:28
> (df <- data.frame(df1, df2, df3, df4)) # Creating the frame
df1 df2 df3 df4
1 One 8 15 22
2 Two 9 16 23
3 Three 10 17 24
4 Four 11 18 25
5 Five 12 19 26
6 Six 13 20 27
7 Seven 14 21 28
> dimnames(df) # Row and column names
[[1]]
[1] "1" "2" "3" "4" "5" "6" "7"
[[2]]
[1] "First" "Second" "Third" "Fourth"
> df # If needed, colnames() and rownames() can be used
First Second Third Fourth
1 One 8 15 22
2 Two 9 16 23
3 Three 10 17 24
4 Four 11 18 25
5 Five 12 19 26
6 Six 13 20 27
7 Seven 14 21 28
>
5.5. Indexing data using the PelophylaxExample frame
Indexing in R (and in similar contexts) refers to accessing particular parts of an object. Indices (methods for selecting a specific portion of data) may be numeric, logical, or textual. Statistical analysis frequently requires obtaining and comparing different fragments of the full data frame. The indexing methods required for this purpose are shown in R dialogue 5.5.1.
R dialogue 5.5.1.
> PE <- read.csv('PelophylaxExamples.csv', sep = ";", dec = ",")
> PE[1, 6] # In square brackets: row first, then column
[1] female # For a factor, its value and levels will be shown:
Levels: female male
> PE[1, 9] # For a numerical variable, just the value will be shown:
[1] 603
> PE[2, ] # If a coordinate is not specified, all values are selected
X Place East North Basin Sex DNA Genotyp L Ltc
2 LL_f_562 Chernetchina 35.13 50.05 Dnipro female 13.95 LL 562 187
Fm T Dp Ci Cs
2 266 249 62 41 152
> PE[2:5, c(9, 11, 13)] # Coordinates can be specified as vectors of indices
L Fm Dp
2 562 266 62
3 592 281 79
4 595 285 75
5 602 287 80
> PE[10, c('Sex', 'Genotyp')] # A vector of variable or row names can be created
Sex Genotyp
10 female LLR
> PE[which.max(PE$L), 'L'] # Largest body length (L) recorded in the frame
[1] 930
> max(PE$L) == PE[which.max(PE$L), 'L'] # Simpler way to obtain the maximum
[1] TRUE
> PE[which.min(PE$L), 'L'] # Minimum value of variable L
[1] 479
> PE[PE$Place == 'Zamulivka', c(6, 8)] # Conditions can be used!
Sex Genotyp
14 female LLR
15 female LLR
56 male RR
> PE_m <- subset(PE, Sex=="male") # A subset of the data can be selected
> # Fairly complex logical constructs can be used:
> PE_f_2n <- subset(PE, Sex=="female" & (Genotyp=="LL" | Genotyp=="LR"| Genotyp=="RR"))
> PE_f_2n[c(4, 8, 12), c(1, 2, 6, 8)] # Let us see what we obtained:
X Place Sex Genotyp
5 LL_f_602 Chernetchina female LL
24 LR_f_877 SuhaGomol`sha female LR
46 RR_f_701 Lipci female RR
>
In R dialogue 5.3.3 the commands which.max() and which.min() were explained. Since they allow us to obtain the index of the element that is the largest or smallest, they can be used to retrieve those very values — for example with commands such as PE[which.max(PE$L), 'L']. However, the maximum and minimum values can also be obtained more simply, using the commands max(PE$L) and min(PE$L).
Note the command in R dialogue 5.5.1: PE[PE$Place == 'Zamulivka', c(6, 8)]. This is in fact a question: which frogs, by sex and genotype, originate in the studied sample from Zamulivka? The first coordinate, the row address, is specified here by a condition (origin from a particular locality); the second coordinate, the variable address, is specified by a vector that determines which attribute values should be retrieved for cases satisfying the given condition.
The command PART <- subset(DATAFRAME, CONDITION) provides the broadest possibilities for selecting a subset of data from a frame. PE_f_2n <- subset(PE, Sex=="female" & (Genotyp=="LL" | Genotyp=="LR" | Genotyp=="RR")) in effect instructs R to form an object containing only females with one of the three diploid genotypes (LL, LR, or RR).
Earlier (R dialogue 5.2.1) we created a version of the data file that contained missing values. We learned to remove missing values from vectors and to compute statistics for vectors while disregarding the missing entries. When working with matrices and data frames, it often makes sense to simply delete all rows that contain missing values (R dialogue 5.5.2).
R dialogue 5.5.2. Two methods for removing rows with missing values from a data frame
> PE1 <- read.csv('PelophylaxExamples.csv', sep = ";", dec = ",", na.strings = 42)
> # Creating a frame that contains missing values, NA
> PE2 <- PE1[complete.cases(PE1), ] # First method for removing rows containing NA
> PE3 <- na.omit(PE1) # Second method for removing rows containing NA
> dim(PE1) == dim(PE2) # The results are the same size
[1] TRUE TRUE
> > sum(PE2 != PE3) # Number of differences between the two frames
[1] 0 # The result demonstrates the complete identity of all elements of PE2 and PE3
>
5.6. Working with columns of the PelophylaxExample frame
An important part of working with data involves creating, modifying, and deleting variables themselves. Let us examine several useful commands. First of all, let us return to the example discussed in section 4.4. There we created two vectors: L and Ltc. To combine them into a data frame (let us call it temp), execute the command temp <- data.frame(L, Ltc).
Let us return to the frame PE. As you will recall, the first column had no name. R assigned it the name X. Incidentally, if we enter the command PE$X (i.e., FRAMENAME$VARIABLENAME), R will display in the console all 57 specimen codes. In that case, it makes logical sense to rename this column: names(PE)[1] <- "Code". The number in square brackets is the column index; this is how we tell R to change the name of the first variable specifically.
How can names be removed if they become unnecessary? Assign them the special operator NULL! This can be done, for example, as follows: colnames(DATAFRAME) <- NULL. The names will be converted to NA, and columns can then be accessed only by index.
A more significant restructuring of the PelophylaxExample frame concerns the transition from absolute values of metric traits to proportional ones. The point is that when comparing frogs that differ in size (above all in body length, L), there is no sense in comparing their metric traits relating to individual body parts. A frog whose head width (Ltc) is 200 units (20 mm) — does it have a narrow head or a wide one? If its body length is 50 mm, its head is wide; if its body length is 70 mm, its head is narrow. Therefore, in most analyses it is optimal to use body length in its absolute value, and all other metric traits in proportional form, as a fraction obtained by dividing the absolute value by body length. Incidentally, when studying other animals it may be appropriate to use some other trait as the “module” rather than body length. For example, in studies of birds and bats, the reference trait against which proportions are calculated is the length of the humerus.
There are several ways to add new columns with proportional traits to the frame PE. The simplest is as follows. In the script editor (or directly in the console), add and execute the command PE$Ltc_L <- PE$Ltc / PE$L and all analogous commands. When R receives such a command regarding a column that does not yet exist, it immediately both creates that column and performs the necessary calculations.
When creating new columns, logical expressions can also be used. Suppose we are interested in the relationship between head width and other traits. In the column Ltc_L we have calculated relative head width, but we wish to create an additional column, HeadWidth, in which frogs are divided into three groups: those with average head width (middle), narrow (narrow), and wide (wide). How should the boundaries between these classes be drawn? One approach is to use quartiles. Frogs falling in the first quartile are designated as narrow-headed, and those in the fourth quartile as wide-headed. How to do this is shown in R dialogue 5.6.1.
R dialogue 5.6.1. Creating new columns in the PelophylaxExample frame
It is sometimes useful to obtain a variable with the ordinal row numbers. What is the simplest way to do this? PE$Number <- 1:nrow(PE). Incidentally, if we execute the command PE$Ones <- 1, we simply obtain a column filled with ones.