# Chapter 2 Summarizing and visualizing data

This chapter focuses on the mechanics and construction of summary statistics and graphs. We use statistical software for generating the summaries and graphs presented in this chapter and book. However, since this might be your first exposure to these concepts, we take our time in this chapter to detail how to create them. Mastery of the content presented in this chapter will be crucial for understanding the methods and techniques introduced in rest of the book.

## 2.1 Exploring numerical data

In this section we will explore techniques for summarizing numerical variables. For example, consider the loan_amount variable from the loan50 data set, which represents the loan size for all 50 loans in the data set. This variable is numerical since we can sensibly discuss the numerical difference of the size of two loans. On the other hand, area codes and zip codes are not numerical, but rather they are categorical variables.

Throughout this section and the next, we will apply these methods using the loan50 and county data sets, which were introduced in Section 1.2. If you’d like to review the variables from either data set, see Tables 1.4 and 1.6.

The county data can be found in the usdata package and the loan50 data can be found in the openintro package.

### 2.1.1 Scatterplots for paired data

A scatterplot provides a case-by-case view of data for two numerical variables. In Figure 1.2, a scatterplot was used to examine the homeownership rate against the fraction of housing units that were part of multi-unit properties (e.g. apartments) in the county data set. Another scatterplot is shown in Figure 2.1, comparing the total income of a borrower total_income and the amount they borrowed loan_amount for the loan50 data set. In any scatterplot, each point represents a single case. Since there are 50 cases in loan50, there are 50 points in Figure 2.1. Figure 2.1: A scatterplot of loan_amount versus total_income for the loan50 data set.

Looking at Figure 2.1, we see that there are many borrowers with income below $100,000 on the left side of the graph, while there are a handful of borrowers with income above$250,000. Figure 2.2: A scatterplot of the median household income against the poverty rate for the county dataset. Data are from 2017. A statistical model has also been fit to the data and is shown as a dashed line.

Example 2.1 Figure 2.2 shows a plot of median household income against the poverty rate for 3142 counties. What can be said about the relationship between these variables?

The relationship is evidently nonlinear, as highlighted by the dashed line. This is different from previous scatterplots we have seen, which show relationships that do not show much, if any, curvature in the trend.

What do scatterplots reveal about the data, and how are they useful?34

Describe two variables that would have a horseshoe-shaped association in a scatterplot ($$\cap$$ or $$\frown$$)35

### 2.1.2 Dot plots and the mean

Sometimes we are interested in the distribution of a single variable. In these cases, a dot plot provides the most basic of displays. A dot plot is a one-variable scatterplot; an example using the interest rate of 50 loans is shown in Figure 2.3. Figure 2.3: A dot plot of interest_rate for the loan50 dataset. The rates have been rounded to the nearest percent in this plot, and the distribution’s mean is shown as a red triangle.

The mean, often called the average is a common way to measure the center of a distribution of data. To compute the mean interest rate, we add up all the interest rates and divide by the number of observations.

The sample mean is often labeled $$\bar{x}$$. The letter $$x$$ is being used as a generic placeholder for the variable of interest and the bar over the $$x$$ communicates we’re looking at the average interest rate, which for these 50 loans is 11.57%. It’s useful to think of the mean as the balancing point of the distribution, and it’s shown as a triangle in Figure 2.3.

Mean. The sample mean can be calculated as the sum of the observed values divided by the number of observations:

$\bar{x} = \frac{x_1 + x_2 + \cdots + x_n}{n}$

Examine the equation for the mean. What does $$x_1$$ correspond to? And $$x_2$$ Can you infer a general meaning to what $$n_i$$ might represent?36

What was $$n$$ in this sample of loans?37

The loan50 data set represents a sample from a larger population of loans made through Lending Club. We could compute a mean for this population in the same way as the sample mean. However, the population mean has a special label: $$\mu$$. The symbol $$\mu$$ is the Greek letter mu and represents the average of all observations in the population. Sometimes a subscript, such as $$_x$$, is used to represent which variable the population mean refers to, e.g. $$\mu_x$$. Often times it is too expensive to measure the population mean precisely, so we often estimate $$\mu$$ using the sample mean, $$\bar{x}$$.

The Greek letter $$\mu$$ is pronounced mu, listen to the pronunciation here.

Example 2.2 The average interest rate across all loans in the population can be estimated using the sample data. Based on the sample of 50 loans, what would be a reasonable estimate of $$\mu_x$$, the mean interest rate for all loans in the full data set?

The sample mean, 11.57, provides a rough estimate of $$\mu_x$$. While it is not perfect, this is our single best guess point estimate of the average interest rate of all the loans in the population under study. In Chapter 5 and beyond, we will develop tools to characterize the accuracy of point estimates, like the sample mean. As you might have guessed, point estimates based on larger samples tend to be more accurate than those based on smaller samples.

The mean is useful because it allows us to rescale or standardize a metric into something more easily interpretable and comparable. Suppose we would like to understand if a new drug is more effective at treating asthma attacks than the standard drug. A trial of 1500 adults is set up, where 500 receive the new drug, and 1000 receive a standard drug in the control group:

New drug Standard drug
Number of patients 500 1000
Total asthma attacks 200 300

Comparing the raw counts of 200 to 300 asthma attacks would make it appear that the new drug is better, but this is an artifact of the imbalanced group sizes. Instead, we should look at the average number of asthma attacks per patient in each group:

• New drug: $$200 / 500 = 0.4$$ asthma attacks per patient
• Standard drug: $$300 / 1000 = 0.3$$ asthma attacks per patient

The standard drug has a lower average number of asthma attacks per patient than the average in the treatment group.

Example 2.3 Provide another examples where the mean is useful for making comparisons.

### 2.1.9 Exercises

1. Mammal life spans. Data were collected on life spans (in years) and gestation lengths (in days) for 62 mammals. A scatterplot of life span versus length of gestation is shown below.48 1. What type of an association is apparent between life span and length of gestation?

2. What type of an association would you expect to see if the axes of the plot were reversed, i.e. if we plotted length of gestation versus life span?

3. Are life span and length of gestation independent? Explain your reasoning.

2. Associations. Indicate which of the plots show (a) a positive association, (b) a negative association, or (c) no association. Also determine if the positive and negative associations are linear or nonlinear. Each part may refer to more than one plot. 3. Reproducing bacteria. Suppose that there is only sufficient space and nutrients to support one million bacterial cells in a petri dish. You place a few bacterial cells in this petri dish, allow them to reproduce freely, and record the number of bacterial cells in the dish over time. Sketch a plot representing the relationship between number of bacterial cells and time.

4. Office productivity. Office productivity is relatively low when the employees feel no stress about their work or job security. However, high levels of stress can also lead to reduced employee productivity. Sketch a plot to represent the relationship between stress and productivity.

5. Parameters and statistics. Identify which value represents the sample mean and which value represents the claimed population mean.

1. American households spent an average of about $52 in 2007 on Halloween merchandise such as costumes, decorations and candy. To see if this number had changed, researchers conducted a new survey in 2008 before industry numbers were reported. The survey included 1,500 households and found that average Halloween spending was$58 per household.

2. The average GPA of students in 2001 at a private university was 3.37. A survey on a sample of 203 students from this university yielded an average GPA of 3.59 a decade later.

6. Sleeping in college. A recent article in a college newspaper stated that college students get an average of 5.5 hrs of sleep each night. A student who was skeptical about this value decided to conduct a survey by randomly sampling 25 students. On average, the sampled students slept 6.25 hours per night. Identify which value represents the sample mean and which value represents the claimed population mean.

7. Days off at a mining plant. Workers at a particular mining site receive an average of 35 days paid vacation, which is lower than the national average. The manager of this plant is under pressure from a local union to increase the amount of paid time off. However, he does not want to give more days off to the workers because that would be costly. Instead he decides he should fire 10 employees in such a way as to raise the average number of days off that are reported by his employees. In order to achieve this goal, should he fire employees who have the most number of days off, least number of days off, or those who have about the average number of days off?

8. Medians and IQRs. For each part, compare distributions A and B based on their medians and IQRs. You do not need to calculate these statistics; simply state how the medians and IQRs compare. Make sure to explain your reasoning.

1. A: 3, 5, 6, 7, 9; B: 3, 5, 6, 7, 20

2. A: 3, 5, 6, 7, 9; B: 3, 5, 7, 8, 9

3. A: 1, 2, 3, 4, 5; B: 6, 7, 8, 9, 10

4. A: 0, 10, 50, 60, 100; B: 0, 100, 500, 600, 1000

9. Means and SDs. For each part, compare distributions A and B based on their means and standard deviations. You do not need to calculate these statistics; simply state how the means and the standard deviations compare. Make sure to explain your reasoning. Hint: It may be useful to sketch dot plots of the distributions.

1. A: 3, 5, 5, 5, 8, 11, 11, 11, 13; B: 3, 5, 5, 5, 8, 11, 11, 11, 20

2. A: -20, 0, 0, 0, 15, 25, 30, 30; B: -40, 0, 0, 0, 15, 25, 30, 30

3. A: 0, 2, 4, 6, 8, 10; B: 20, 22, 24, 26, 28, 30

4. A: 100, 200, 300, 400, 500; B: 0, 50, 300, 550, 600

10. Histograms and box plots. Describe the distribution in the histograms below and match them to the box plots. 11. Air quality. Daily air quality is measured by the air quality index (AQI) reported by the Environmental Protection Agency. This index reports the pollution level and what associated health effects might be a concern. The index is calculated for five major air pollutants regulated by the Clean Air Act and takes values from 0 to 300, where a higher value indicates lower air quality. AQI was reported for a sample of 91 days in 2011 in Durham, NC. The relative frequency histogram below shows the distribution of the AQI values on these days.49 1. Estimate the median AQI value of this sample.

2. Would you expect the mean AQI value of this sample to be higher or lower than the median? Explain your reasoning.

3. Estimate Q1, Q3, and IQR for the distribution.

4. Would any of the days in this sample be considered to have an unusually low or high AQI? Explain your reasoning.

12. Median vs. mean. Estimate the median for the 400 observations shown in the histogram, and note whether you expect the mean to be higher or lower than the median. 13. Histograms vs. box plots. Compare the two plots below. What characteristics of the distribution are apparent in the histogram and not in the box plot? What characteristics are apparent in the box plot but not in the histogram? 14. Facebook friends. Facebook data indicate that 50% of Facebook users have 100 or more friends, and that the average friend count of users is 190. What do these findings suggest about the shape of the distribution of number of friends of Facebook users? (Backstrom 2011)

15. Distributions and appropriate statistics. For each of the following, state whether you expect the distribution to be symmetric, right skewed, or left skewed. Also specify whether the mean or median would best represent a typical observation in the data, and whether the variability of observations would be best represented using the standard deviation or IQR. Explain your reasoning.

1. Number of pets per household.

2. Distance to work, i.e. number of miles between work and home.

16. Distributions and appropriate statistics. For each of the following, state whether you expect the distribution to be symmetric, right skewed, or left skewed. Also specify whether the mean or median would best represent a typical observation in the data, and whether the variability of observations would be best represented using the standard deviation or IQR. Explain your reasoning.

1. Housing prices in a country where 25% of the houses cost below $350,000, 50% of the houses cost below$450,000, 75% of the houses cost below $1,000,000 and there are a meaningful number of houses that cost more than$6,000,000.

2. Housing prices in a country where 25% of the houses cost below $300,000, 50% of the houses cost below$600,000, 75% of the houses cost below $900,000 and very few houses that cost more than$1,200,000.

3. Number of alcoholic drinks consumed by college students in a given week. Assume that most of these students don’t drink since they are under 21 years old, and only a few drink excessively.

4. Annual salaries of the employees at a Fortune 500 company where only a few high level executives earn much higher salaries than all the other employees.

17. Income at the coffee shop. The first histogram below shows the distribution of the yearly incomes of 40 patrons at a college coffee shop. Suppose two new people walk into the coffee shop: one making $225,000 and the other$250,000. The second histogram shows the new income distribution. Summary statistics are also provided, rounded to the nearest whole number.

n

Min

Q1

Median

Mean

Max

SD

Before

40

$60,679$60,818

$65,238$65,089

$69,885$2,122

After

42

$60,679$60,838

$65,352$73,299

$250,000$37,321 1. Would the mean or the median best represent what we might think of as a typical income for the 42 patrons at this coffee shop? What does this say about the robustness of the two measures?

2. Would the standard deviation or the IQR best represent the amount of variability in the incomes of the 42 patrons at this coffee shop? What does this say about the robustness of the two measures?

18. Midrange. The midrange of a distribution is defined as the average of the maximum and the minimum of that distribution. Is this statistic robust to outliers and extreme skew? Explain your reasoning.

19. Commute times. The US census collects data on the time it takes Americans to commute to work, among many other variables. The histogram below shows the distribution of average commute times in 3,142 US counties in 2017. Also shown below is a spatial intensity map of the same data.50 1. Describe the numerical distribution and comment on whether or not a log transformation may be advisable for these data.

2. Describe the spatial distribution of commuting times using the map.

20. Hispanic population. The US census collects data on race and ethnicity of Americans, among many other variables. The histogram below shows the distribution of the percentage of the population that is Hispanic in 3,142 counties in the US in 2010. Also shown is a histogram of logs of these values.51  1. Describe the numerical distribution and comment on why we might want to use log-transformed values in analyzing or modeling these data.

2. What features of the distribution of the Hispanic population in US counties are apparent in the map but not in the histogram? What features are apparent in the histogram but not the map?

3. Is one visualization more appropriate or helpful than the other? Explain your reasoning.

## 2.2 Exploring categorical data

In this section, we will introduce tables and other basic tools for categorical data that are used throughout this book. The loan50 data set represents a sample from a larger loan data set called loans. This larger data set contains information on 10,000 loans made through Lending Club. We will examine the relationship between homeownership, which for the loans data can take a value of rent, mortgage (owns but has a mortgage), or own, and app_type, which indicates whether the loan application was made with a partner or whether it was an individual application.

The data can be found in the openintro package: loans_full_schema.

### 2.2.1 Contingency tables and bar plots

Table 2.3 summarizes two variables: application_type and homeownership. A table that summarizes data for two categorical variables in this way is called a contingency table. Each value in the table represents the number of times a particular combination of variable outcomes occurred. For example, the value 3496 corresponds to the number of loans in the data set where the borrower rents their home and the application type was by an individual. Row and column totals are also included. The row totals provide the total counts across each row and the column totals down each column. We can also create a table that shows only the overall percentages or proportions for each combination of categories, or we can create a table for a single variable, such as the one shown in Table 2.4 for the homeownership variable.

Table 2.3: A contingency table for application type and homeownership.
homeownership
application_type rent mortgage own Total
joint 362 950 183 1495
individual 3496 3839 1170 8505
Total 3858 4789 1353 10000
Table 2.4: A table summarizing the frequencies for each value of the homeownership variable: mortgage, own, and rent.
homeownership Count
mortgage 4789
own 1353
rent 3858
Total 10000

A bar plot is a common way to display a single categorical variable. The left panel of Figure 2.14 shows a bar plot for the homeownership variable. In the right panel, the counts are converted into proportions, showing the proportion of observations that are in each level. Figure 2.14: Two bar plots: the left panel shows the counts and the right panel shows the proportions of values of the homeownership variable.

### 2.2.2 Using a bar plot with two variables

We can display the distributions of two categorical variables on a bar plot concurrently. Such plots are generally useful for visualizing the relationship between two categorical variables. Figure 2.15 shows three such plots that visualize the relationship between homeownership and application_type variables. Plot A in Figure 2.15 is a stacked bar plot. This plot most clearly displays that loan applicants most commonly live in mortgaged homes. It is difficult to say, based on this plot alone, how different application types vary across the levels of homeownership. Plot B is a dodged bar plot. This plot most clearly displays that within each level of homeownership, individual applications are more common than joint applications. Finally, plot C is a standardized bar plot (also known as filled bar plot). This plot most clearly displays that joint applications are most common among applications who live in mortgaged homes, compared to renters and owners. This type of visualization is helpful in understanding the fraction of individual or joint loan applications for borrowers in each level of homeownership. Additionally, since the proportions of and vary across the groups, we can conclude that the two variables are associated for this sample. Figure 2.15: Three bar plots (stacked, dodged, and standardized) displaying homeownership and application type variables.

Example 2.10 Examine the three bar plots in Figure 2.15. When is the stacked, dodged, or standardized bar plot the most useful?

The stacked bar plot is most useful when it’s reasonable to assign one variable as the explanatory variable and the other variable as the response, since we are effectively grouping by one variable first and then breaking it down by the others.

Dodged bar plots are more agnostic in their display about which variable, if any, represents the explanatory and which the response variable. It is also easy to discern the number of cases in of the six different group combinations. However, one downside is that it tends to require more horizontal space; the narrowness of Plot B compared to the other two in Figure 2.15 makes the plot feel a bit cramped. Additionally, when two groups are of very different sizes, as we see in the group relative to either of the other two groups, it is difficult to discern if there is an association between the variables.

The standardized stacked bar plot is helpful if the primary variable in the stacked bar plot is relatively imbalanced, e.g. the category has only a third of the observations in the category, making the simple stacked bar plot less useful for checking for an association. The major downside of the standardized version is that we lose all sense of how many cases each of the bars represents.

### 2.2.3 Mosaic plots

A mosaic plot is a visualization technique suitable for contingency tables that resembles a standardized stacked bar plot with the benefit that we still see the relative group sizes of the primary variable as well.

To get started in creating our first mosaic plot, we’ll break a square into columns for each category of the variable, with the result shown in Plot A of Figure 2.16. Each column represents a level of homeownership, and the column widths correspond to the proportion of loans in each of those categories. For instance, there are fewer loans where the borrower is an owner than where the borrower has a mortgage. In general, mosaic plots use box areas to represent the number of cases in each category.

Plot B in Figure 2.16 displays the relationship between homeownership and application type. Each column is split proportional to the number of loans from individual and joint borrowers. For example, the second column represents loans where the borrower has a mortgage, and it was divided into individual loans (upper) and joint loans (lower). As another example, the bottom segment of the third column represents loans where the borrower owns their home and applied jointly, while the upper segment of this column represents borrowers who are homeowners and filed individually. We can again use this plot to see that the and variables are associated, since some columns are divided in different vertical locations than others, which was the same technique used for checking an association in the standardized stacked bar plot. Figure 2.16: The mosaic plots: one for homeownership alone and the other displaying the relationship between homeownership and applocation type.

In Figure 2.16, we chose to first split by the homeowner status of the borrower. However, we could have instead first split by the application type, as in Figure 2.17. Like with the bar plots, it’s common to use the explanatory variable to represent the first split in a mosaic plot, and then for the response to break up each level of the explanatory variable, if these labels are reasonable to attach to the variables under consideration. Figure 2.17: Mosaic plot where loans are grouped by homeownership after they have been divided into individual and joint application types.

### 2.2.4 Row and column proportions

In the previous sections we inspected visualisations of two categorical variables in bar plots and mosaic plots. However, we have not discussed how the values in the bar and mosaic plots that show proportions are calculated. In this section we will investigate fractional breakdown of one variable in another and we can modify our contingency table to provide such a view. Table 2.5 shows row proportions the for Table 2.3, which are computed as the counts divided by their row totals. The value 3496 at the intersection of and is replaced by $$3496 / 8505 = 0.411$$, i.e. 3496 divided by its row total, 8505. So what does 0.411 represent? It corresponds to the proportion of individual applicants who rent.

Table 2.5: A contingency table with row proportions for the application type and homeownership variables.
homeownership
application_type rent mortgage own Total
joint 0.242 0.635 0.122 1
individual 0.411 0.451 0.138 1
Total 0.653 1.087 0.260 1

A contingency table of the column proportions is computed in a similar way, where each is computed as the count divided by the corresponding column total. Table 2.6 shows such a table, and here the value 0.906 indicates that 90.6% of renters applied as individuals for the loan. This rate is higher compared to loans from people with mortgages (80.2%) or who own their home (86.5%). Because these rates vary between the three levels of homeownership (rent, mortgage, own), this provides evidence that app_type and homeownership variables may be associated.

Table 2.6: A contingency table with column proportions for the application type and homeownership variables.
homeownership
application_type rent mortgage own Total
joint 0.094 0.198 0.135 0.427
individual 0.906 0.802 0.865 2.573
Total 1.000 1.000 1.000 3.000

We could also have checked for an association between application_type and homeownership in Table 2.5 using row proportions. When comparing these row proportions, we would look down columns to see if the fraction of loans where the borrower rents, has a mortgage, or owns varied across the to application types.

(a) What does 0.451 represent in Table 2.5?

(b) What does 0.802 represent in Table 2.6?52

(a) What does 0.122 represent in Table 2.5?

(b) What does 0.135 represent in Table 2.6?53

Example 2.11 Data scientists use statistics to filter spam from incoming email messages. By noting specific characteristics of an email, a data scientist may be able to classify some emails as spam or not spam with high accuracy. One such characteristic is whether the email contains no numbers, small numbers, or big numbers. Another characteristic is the email format, which indicates whether or not an email has any HTML content, such as bolded text. We’ll focus on email format and spam status using the data set, and these variables are summarized in a contingency table in Table 2.7. Which would be more helpful to someone hoping to classify email as spam or regular email for this table: row or column proportions?

A data scientist would be interested in how the proportion of spam changes within each email format. This corresponds to column proportions: the proportion of spam in plain text emails and the proportion of spam in HTML emails.

If we generate the column proportions, we can see that a higher fraction of plain text emails are spam ($$209/1195 = 17.5\%$$) than compared to HTML emails ($$158/2726 = 5.8\%$$). This information on its own is insufficient to classify an email as spam or not spam, as over 80% of plain text emails are not spam. Yet, when we carefully combine this information with many other characteristics, we stand a reasonable chance of being able to classify some emails as spam or not spam with confidence. This example points out that row and column proportions are not equivalent. Before settling on one form for a table, it is important to consider each to ensure that the most useful table is constructed. However, sometimes it simply isn’t clear which, if either, is more useful.

Table 2.7: A contingency table for spam and format
spam HTML text Total
not spam 2568 986 3554
spam 158 209 367
Total 2726 1195 3921

Example 2.12 Look back to Table 2.5 and Table 2.6. Are there any obvious scenarios where one might be more useful than the other?

None that we thought were obvious! What is distinct about and vs the email example is that these two variables don’t have a clear explanatory-response variable relationship that we might hypothesize. Usually it is most useful to “condition” on the explanatory variable. For instance, in the email example, the email format was seen as a possible explanatory variable of whether the message was spam, so we would find it more interesting to compute the relative frequencies (proportions) for each email format.

### 2.2.5 Pie charts

A pie chart is shown in Figure 2.18 alongside a bar plot representing the same information. Pie charts can be useful for giving a high-level overview to show how a set of cases break down. However, it is also difficult to decipher details in a pie chart. For example, it;s not immediately obvious that there are more loans where the borrower has a mortgage than rent when looking at the pie chart, while this detail is very obvious in the bar plot. While pie charts can be useful, we prefer bar plots for their ease in comparing groups. Figure 2.18: A pie chart and a bar plot of homeownership.

### 2.2.6 Comparing numerical data across groups

Some of the more interesting investigations can be considered by examining numerical data across groups. In this section we will expand on a few methods we’ve already seen to make plots for numerical data from multiple groups on the same graph as well as introduce a few new methods for comparing numerical data across groups.

We will revisit the county dataset and compare the median household income for counties that gained population from 2010 to 2017 versus counties that had no gain. While we might like to make a causal connection here, remember that these are observational data and so such an interpretation would be, at best, half-baked.

We have data on 3142 counties in the United States. We are missing 2017 population data from 3, and of the remaining 3139 counties, in 1541 the population increased from 2010 to 2017 and in the remaining 1598. Table 2.8 shows a sample 5 observations from each group.

Table 2.8: The median household income from a random sample of 5 counties with population gain between 2010 to 2017 and another random sample of 5 counties with no population gain.
State County Change in population, % Gain / No gain Median household income
Arizona Navajo County 1.75 gain 38798
Louisiana Jefferson Davis Parish 0.57 gain 40744
Texas Victoria County 2.25 gain 55740
Virginia Shenandoah County 1.68 gain 53934
Wisconsin Clark County 0.36 gain 49131
Alabama Marengo County -3.63 no gain 32255
Iowa O’Brien County -1.73 no gain 56314
Kansas Geary County -7.95 no gain 46096
West Virginia Ohio County -3.86 no gain 45777
West Virginia Preston County -0.01 no gain 46673

Color can be used to split histograms for numerical variables by levels of a categorical variable. An example of this is shown in Plot A of Figure 2.19. The side-by-side box plot is another traditional tool for comparing across groups. An example is shown in Plot B of Figure 2.19, where there are two box plots, one for each group, placed into one plotting window and drawn on the same scale. Figure 2.19: Histograms (Plot A) and side by-side box plots (Plot B) for med_hh_income, where counties are split by whether there was a population gain or not.

Use the plots in Figure 2.19 to compare the incomes for counties across the two groups. What do you notice about the approximate center of each group? What do you notice about the variability between groups? Is the shape relatively consistent between groups? How many prominent modes are there for each group?54

What components of each plot in Figure 2.19 do you find most useful?55

Another useful visualisation for comparing numerical data across groups is ridge plots, which are density plots for various groups drawn on the same scale in a single plotting window. Figure 2.20 display ridge plots for the distribution of median household income in counties, split by whether there was a population gain or not. Figure 2.20: Ridge plot for for med_hh_income, where counties are split by whether there was a population gain or not.

What components of the ridge plots in Figure 2.19 do you find most useful compared to those in Figure 2.19?56

One last visualization technique we’ll highlight for comparing numerical data across groups is faceted plots. In this technique we split (facet) the graphical display of the data across plotting windows based on groups. Plot A in Figure 2.21 displays the same information as Plot A in Figure 2.19, however here the distributions of median household income for counties with and without population gain are faceted across two plotting windows. We preserve the same scale on the x and y axes for easier comparison. An advantage of this approach is that it extends to splitting the data across levels of two categorical variables, which allows for displaying relationships between three variables. In Plot B in Figure 2.21 we have now split the data into four groups using the pop_change and metro variables:

• top left represents counties that are not in a metropolitan area with population gain,
• top right represents counties that are in a metropolitan area with population gain,
• bottom left represents counties that are not in a metropolitan area without population gain, and finally
• bottom right represents counties that are in a metropolitan area without population gain. Figure 2.21: Distribution of median income in counties using faceted histograms: Plot A facets by whether there was a population gain or not and Plot B facets by both population gain and whether the county is in a metropolitan area.

We can continue building up on this visualisation to add one more variable, median_edu, which is the median education level in the county. In Figure 2.22, we represent median education level using color, where yellow (dotted line) represents counties where the median (dashed line) education level is Bachelor’s, green is some college degree, and blue (solid line) is high school diploma. Figure 2.22: Distribution of median income in counties using ridge plots, faceted by whether the county had a population gain or not as well as whether the county is in a metropolitan area and colored by the median education level in the county.

Based on Figure 2.22, what can you say about how median household income in counties vary depending on population gain/no gain, metropolitan area/not, and median degree?57

### 2.2.7 Exercises

1. Antibiotic use in children. The bar plot and the pie chart below show the distribution of pre-existing medical conditions of children involved in a study on the optimal duration of antibiotic use in treatment of tracheitis, which is an upper respiratory infection.58 1. What features are apparent in the bar plot but not in the pie chart?

2. What features are apparent in the pie chart but not in the bar plot?

3. Which graph would you prefer to use for displaying these categorical data?

2. Views on immigration. 910 randomly sampled registered voters from Tampa, FL were asked if they thought workers who have illegally entered the US should be (i) allowed to keep their jobs and apply for US citizenship, (ii) allowed to keep their jobs as temporary guest workers but not allowed to apply for US citizenship, or (iii) lose their jobs and have to leave the country. The results of the survey by political ideology are shown below.59

Response

Conservative

Liberal

Moderate

Total

Apply for citizenship

57

101

120

278

Guest worker

121

28

113

262

Leave the country

179

45

126

350

Not sure

15

1

4

20

Total

372

175

363

910

1. What percent of these Tampa, FL voters identify themselves as conservatives?

2. What percent of these Tampa, FL voters are in favor of the citizenship option?

3. What percent of these Tampa, FL voters identify themselves as conservatives and are in favor of the citizenship option?

4. What percent of these Tampa, FL voters who identify themselves as conservatives are also in favor of the citizenship option? What percent of moderates share this view? What percent of liberals share this view?

5. Do political ideology and views on immigration appear to be independent? Explain your reasoning.

6. What other variables might explain the potential relationship between these two variables.

3. Black Lives Matter. A Washington Post-Schar School poll conducted in the United States in June 2020, among a random national sample of 1,006 adults, asked respondents whether they support or oppose protests following George Floyd’s killing that have taken place in cities across the US. The survey also collected information on the age of the respondents. (n.d.a) The results are summarized in the stacked bar plot below. 1. Based on the stacked bar plot, do views on the protests and age appear to be independent? Explain your reasoning.

2. What other variables might explain the potential relationship between these two variables?

4. Raise taxes. A random sample of registered voters nationally were asked whether they think it’s better to raise taxes on the rich or raise taxes on the poor. The survey also collected information on the political party affiliation of the respondents. (n.d.b) 1. Based on the stacked bar plot shown above, do views on raising taxes and political affiliation appear to be independent? Explain your reasoning.

2. What other variables might explain the potential relationship between these two variables?

## 2.3 Effective data visualization

This section is currently under construction!

### 2.3.1 Keep it simple

We discussed earlier that pie charts do not tend to be useful when the categorical variable being displayed has many levels. In addition, there is little value added to coloring each pie. And definitely no value added to making the pie chart three dimensional. A simple bar plot can communicate the same information in a simpler, easier to digest way. vs. ### 2.3.2 Use color to draw attention

Avoid adding color just to add color, instead use it to draw attention. This doesn’t mean you shouldn’t think about how visually pleasing your visualization is, and if you’re adding color for making it visually pleasing without drawing attention to a particular feature, that might still be fine. But you should be critical of default coloring and explicitly decide whether to include color and how. Also note that not everyone sees color the same way, often it’s useful to add color and one more feature (e.g. pattern) so that you can refer to the features you’re drawing attention to in multiple ways. ### 2.3.3 Tell a story Figure 2.23: Credit: Angela Zoss and Eric Monson, Duke Data Visualization Services

### 2.3.4 Order matters

In September 2019, YouGov survey asked 1,639 GB adults the following question60:

In hindsight, do you think Britain was right/wrong to vote to leave EU?

• Right to leave
• Wrong to leave
• Don’t know

Alphabetical order is rarely ideal, so sometimes it’s better to order bars by frequency. We can improve this further by cleaning up axis labels. There may also be inherent ordering to levels of your categorical variables. If so, the visualization should respect that. ### 2.3.5 Put long categories on the y-axis

And clean up axis labels. ### 2.3.6 Pick a purpose

Segmented bar plots can be hard to read. Use faceting, avoid redundancy, and use informative labels (note the shortlink to the survey). ### 2.3.7 Select meaningful colors

Rainbow colors are not always the right choice. Viridis scale works well with ordinal data ### 2.3.8 Exercises

Exercises for this section are under construction!

## 2.4 Case study: malaria vaccine

Suppose your professor splits the students in class into two groups: students on the left and students on the right. If $$\hat{p}_{_L}$$ and $$\hat{p}_{_R}$$ represent the proportion of students who own an Apple product on the left and right, respectively, would you be surprised if $$\hat{p}_{_L}$$ did not exactly equal $$\hat{p}_{_R}$$? While the proportions would probably be close to each other, it would be unusual for them to be exactly the same. We would probably observe a small difference due to chance.

If we don’t think the side of the room a person sits on in class is related to whether the person owns an Apple product, what assumption are we making about the relationship between these two variables?

### 2.4.1 Variability within data

We consider a study on a new malaria vaccine called PfSPZ. In this study, volunteer patients were randomized into one of two experiment groups: 14 patients received an experimental vaccine and 6 patients received a placebo vaccine. Nineteen weeks later, all 20 patients were exposed to a drug-sensitive malaria virus strain; the motivation of using a drug-sensitive strain of virus here is for ethical considerations, allowing any infections to be treated effectively. The results are summarized in Table 2.9, where 9 of the 14 treatment patients remained free of signs of infection while all of the 6 patients in the control group patients showed some baseline signs of infection.

Table 2.9: Summary results for the malaria vaccine experiment.
treatment infection no infection Total
placebo 6 0 6
vaccine 5 9 14
Total 11 9 20

Is this an observational study or an experiment? What implications does the study type have on what can be inferred from the results?61

In this study, a smaller proportion of patients who received the vaccine showed signs of an infection (35.7% versus 100%). However, the sample is very small, and it is unclear whether the difference provides convincing evidence that the vaccine is effective.

Example 2.13 Data scientists are sometimes called upon to evaluate the strength of evidence. When looking at the rates of infection for patients in the two groups in this study, what comes to mind as we try to determine whether the data show convincing evidence of a real difference?

The observed infection rates (35.7% for the treatment group versus 100% for the control group) suggest the vaccine may be effective. However, we cannot be sure if the observed difference represents the vaccine’s efficacy or is just from random chance. Generally there is a little bit of fluctuation in sample data, and we wouldn’t expect the sample proportions to be exactly equal, even if the truth was that the infection rates were independent of getting the vaccine. Additionally, with such small samples, perhaps it’s common to observe such large differences when we randomly split a group due to chance alone!

This example is a reminder that the observed outcomes in the data sample may not perfectly reflect the true relationships between variables since there is random noise. While the observed difference in rates of infection is large, the sample size for the study is small, making it unclear if this observed difference represents efficacy of the vaccine or whether it is simply due to chance. We label these two competing claims, $$H_0$$ and $$H_A$$, which are spoken as “H-nought” and “H-A”:

• $$H_0$$: Independence model. The variables and are independent. They have no relationship, and the observed difference between the proportion of patients who developed an infection in the two groups, 64.3%, was due to chance.

• $$H_A$$: Alternative model. The variables are not independent. The difference in infection rates of 64.3% was not due to chance, and vaccine affected the rate of infection.

What would it mean if the independence model, which says the vaccine had no influence on the rate of infection, is true? It would mean 11 patients were going to develop an infection no matter which group they were randomized into, and 9 patients would not develop an infection no matter which group they were randomized into. That is, if the vaccine did not affect the rate of infection, the difference in the infection rates was due to chance alone in how the patients were randomized.

Now consider the alternative model: infection rates were influenced by whether a patient received the vaccine or not. If this was true, and especially if this influence was substantial, we would expect to see some difference in the infection rates of patients in the groups.

We choose between these two competing claims by assessing if the data conflict so much with $$H_0$$ that the independence model cannot be deemed reasonable. If this is the case, and the data support $$H_A$$, then we will reject the notion of independence and conclude the vaccine was effective.

### 2.4.2 Simulating the study

We’re going to implement simulation, where we will pretend we know that the malaria vaccine being tested does not work. Ultimately, we want to understand if the large difference we observed is common in these simulations. If it is common, then maybe the difference we observed was purely due to chance. If it is very uncommon, then the possibility that the vaccine was helpful seems more plausible.

Table 2.9 shows that 11 patients developed infections and 9 did not. For our simulation, we will suppose the infections were independent of the vaccine and we were able to rewind back to when the researchers randomized the patients in the study. If we happened to randomize the patients differently, we may get a different result in this hypothetical world where the vaccine doesn’t influence the infection. Let’s complete another randomization using a simulation.

In this simulation, we take 20 notecards to represent the 20 patients, where we write down “infection” on 11 cards and “no infection” on 9 cards. In this hypothetical world, we believe each patient that got an infection was going to get it regardless of which group they were in, so let’s see what happens if we randomly assign the patients to the treatment and control groups again. We thoroughly shuffle the notecards and deal 14 into a pile and 6 into a pile. Finally, we tabulate the results, which are shown in Table 2.10.

Table 2.10: Simulation results, where any difference in infection ratio is purely due to chance.
treatment placebo vaccine Total
infection 4 7 11
no infection 2 7 9
Total 6 14 20

How does this compare to the observed 64.3% difference in the actual data?62

### 2.4.3 Checking for independence

We computed one possible difference under the independence model in the previous Guided Practice, which represents one difference due to chance. While in this first simulation, we physically dealt out notecards to represent the patients, it is more efficient to perform this simulation using a computer.

Repeating the simulation on a computer, we get another difference due to chance: $\frac{2}{6{}} - \frac{9}{14{}} = -0.310$

And another: $\frac{3}{6{}} - \frac{8}{14{}} = -0.071$

And so on until we repeat the simulation enough times that we have a good idea of what represents the distribution of differences from chance alone.

Figure 2.24 shows a stacked plot of the differences found from 100 simulations, where each dot represents a simulated difference between the infection rates (control rate minus treatment rate). Figure 2.24: A stacked dot plot of differences from 100 simulations produced under the independence mode, $$H_0$$, where in these simulations infections are unaffected by te vaccine. Two of the 100 simulations had a difference of at least 64.3%, the difference observed in the study.

Note that the distribution of these simulated differences is centered around 0. We simulated these differences assuming that the independence model was true, and under this condition, we expect the difference to be near zero with some random fluctuation, where near is pretty generous in this case since the sample sizes are so small in this study.

Example 2.14 How often would you observe a difference of at least 64.3% (0.643) according to Figure 2.24? Often, sometimes, rarely, or never?

It appears that a difference of at least 64.3% due to chance alone would only happen about 2% of the time according to Figure 2.24. Such a low probability indicates a rare event.

The difference of 64.3% being a rare event suggests two possible interpretations of the results of the study:

• $$H_0$$: Independence model. The vaccine has no effect on infection rate, and we just happened to observe a difference that would only occur on a rare occasion.

• $$H_A$$: Alternative model. The vaccine has an effect on infection rate, and the difference we observed was actually due to the vaccine being effective at combatting malaria, which explains the large difference of 64.3%.

Based on the simulations, we have two options. (1) We conclude that the study results do not provide strong evidence against the independence model. That is, we do not have sufficiently strong evidence to conclude the vaccine had an effect in this clinical setting. (2) We conclude the evidence is sufficiently strong to reject $$H_0$$ and assert that the vaccine was useful. When we conduct formal studies, usually we reject the notion that we just happened to observe a rare event.[^This reasoning does not generally extend to anecdotal observations. Each of us observes incredibly rare events every day, events we could not possibly hope to predict. However, in the non-rigorous setting of anecdotal evidence, almost anything may appear to be a rare event, so the idea of looking for rare events in day-to-day activities is treacherous. For example, we might look at the lottery: there was only a 1 in 292 million chance that the Powerball numbers for the largest jackpot in history (January 13th, 2016) would be (04, 08, 19, 27, 34) with a Powerball of (10), but nonetheless those numbers came up! However, no matter what numbers had turned up, they would have had the same incredibly rare odds. That is, any set of numbers we could have observed would ultimately be incredibly rare. This type of situation is typical of our daily lives: each possible event in itself seems incredibly rare, but if we consider every alternative, those outcomes are also incredibly rare. We should be cautious not to misinterpret such anecdotal evidence.] So in this case, we reject the independence model in favor of the alternative. That is, we are concluding the data provide strong evidence that the vaccine provides some protection against malaria in this clinical setting.

One field of statistics, statistical inference, is built on evaluating whether such differences are due to chance. In statistical inference, data scientists evaluate which model is most reasonable given the data. Errors do occur, just like rare events, and we might choose the wrong model. While we do not always choose correctly, statistical inference gives us tools to control and evaluate how often these errors occur. In Chapter @ref(#intro-stat-inference), we give a formal introduction to the problem of model selection. We spend the next two chapters building a foundation of probability and theory necessary to make that discussion rigorous.

### 2.4.4 Exercises

1. Side effects of Avandia. Rosiglitazone is the active ingredient in the controversial type 2 diabetes medicine Avandia and has been linked to an increased risk of serious cardiovascular problems such as stroke, heart failure, and death. A common alternative treatment is Pioglitazone, the active ingredient in a diabetes medicine called Actos. In a nationwide retrospective observational study of 227,571 Medicare beneficiaries aged 65 years or older, it was found that 2,593 of the 67,593 patients using Rosiglitazone and 5,386 of the 159,978 using Pioglitazone had serious cardiovascular problems. These data are summarized in the contingency table below.63

treatment

No

Yes

Total

Pioglitazone

154592

5386

159978

Rosiglitazone

65000

2593

67593

Total

219592

7979

227571

1. Determine if each of the following statements is true or false. If false, explain why. Be careful: The reasoning may be wrong even if the statement’s conclusion is correct. In such cases, the statement should be considered false.

1. Since more patients on pioglitazone had cardiovascular problems (5,386 vs. 2,593), we can conclude that the rate of cardiovascular problems for those on a pioglitazone treatment is higher.

2. The data suggest that diabetic patients who are taking rosiglitazone are more likely to have cardiovascular problems since the rate of incidence was (2,593 / 67,593 = 0.038) 3.8% for patients on this treatment, while it was only (5,386 / 159,978 = 0.034) 3.4% for patients on pioglitazone.

3. The fact that the rate of incidence is higher for the rosiglitazone group proves that Rosiglitazone causes serious cardiovascular problems.

4. Based on the information provided so far, we cannot tell if the difference between the rates of incidences is due to a relationship between the two variables or due to chance.

2. What proportion of all patients had cardiovascular problems?

3. If the type of treatment and having cardiovascular problems were independent, about how many patients in the Rosiglitazone group would we expect to have had cardiovascular problems?

4. We can investigate the relationship between outcome and treatment in this study using a randomization technique. While in reality we would carry out the simulations required for randomization using statistical software, suppose we actually simulate using index cards. In order to simulate from the independence model, which states that the outcomes were independent of the treatment, we write whether or not each patient had a cardiovascular problem on cards, shuffled all the cards together, then deal them into two groups of size 67,593 and 159,978. We repeat this simulation 100 times and each time record the difference between the proportions of cards that say “Yes” in the Rosiglitazone and Pioglitazone groups. Use the histogram of these differences in proportions to answer the following questions.

1. What are the claims being tested?

2. Compared to the number calculated in part (b), which would provide more support for the alternative hypothesis, higher or lower proportion of patients with cardiovascular problems in the Rosiglitazone group?

3. What do the simulation results suggest about the relationship between taking Rosiglitazone and having cardiovascular problems in diabetic patients? 2. Heart transplants. The Stanford University Heart Transplant Study was conducted to determine whether an experimental heart transplant program increased lifespan. Each patient entering the program was designated an official heart transplant candidate, meaning that he was gravely ill and would most likely benefit from a new heart. Some patients got a transplant and some did not. The variable transplant indicates which group the patients were in; patients in the treatment group got a transplant and those in the control group did not. Of the 34 patients in the control group, 30 died. Of the 69 people in the treatment group, 45 died. Another variable called survived was used to indicate whether or not the patient was alive at the end of the study.64 1. Based on the stacked bar plot, is survival independent of whether or not the patient got a transplant? Explain your reasoning.

2. What do the box plots below suggest about the efficacy (effectiveness) of the heart transplant treatment.

3. What proportion of patients in the treatment group and what proportion of patients in the control group died?

4. One approach for investigating whether or not the treatment is effective is to use a randomization technique.

1. What are the claims being tested?

2. The paragraph below describes the set up for such approach, if we were to do it without using statistical software. Fill in the blanks with a number or phrase, whichever is appropriate.

We write alive on $$\rule{2cm}{0.5pt}$$ cards representing patients who were alive at the end of the study, and deceased on $$\rule{2cm}{0.5pt}$$ cards representing patients who were not. Then, we shuffle these cards and split them into two groups: one group of size $$\rule{2cm}{0.5pt}$$ representing treatment, and another group of size $$\rule{2cm}{0.5pt}$$ representing control. We calculate the difference between the proportion of cards in the treatment and control groups (treatment - control) and record this value. We repeat this 100 times to build a distribution centered at $$\rule{2cm}{0.5pt}$$. Lastly, we calculate the fraction of simulations where the simulated differences in proportions are $$\rule{2cm}{0.5pt}$$. If this fraction is low, we conclude that it is unlikely to have observed such an outcome by chance and that the null hypothesis should be rejected in favor of the alternative.

1. What do the simulation results shown below suggest about the effectiveness of the transplant program? ## 2.5 Chapter review

### 2.5.1 Terms

We introduced the following terms in the chapter. If you’re not sure what some of these terms mean, we recommend you go back in the text and review their definitions. We are purposefully presenting them in alphabetical order, instead of in order of appearance, so they will be a little more challenging to locate. However you should be able to easily spot them as bolded text.

 average filled bar plot nonlinear standard deviation bimodal first quartile outlier standardized bar plot box plot histogram point estimate symmetric column totals intensity map random noise tail contingency table interquartile range ridge plot third quartile data density IQR right skewed transformation deviation left skewed robust statistics unimodal distribution mean row totals variability dodged bar plot median scatterplot variance dot plot mosaic plot side-by-side box plot weighted mean faceted plot multimodal stacked bar plot whiskers

### 2.5.2 Chapter exercises

1. Make-up exam. In a class of 25 students, 24 of them took an exam in class and 1 student took a make-up exam the following day. The professor graded the first batch of 24 exams and found an average score of 74 points with a standard deviation of 8.9 points. The student who took the make-up the following day scored 64 points on the exam.

1. Does the new student’s score increase or decrease the average score?

2. What is the new average?

3. Does the new student’s score increase or decrease the standard deviation of the scores?

2. Infant mortality. The infant mortality rate is defined as the number of infant deaths per 1,000 live births. This rate is often used as an indicator of the level of health in a country. The relative frequency histogram below shows the distribution of estimated infant death rates for 224 countries for which such data were available in 2014.65 1. Estimate Q1, the median, and Q3 from the histogram.

2. Would you expect the mean of this data set to be smaller or larger than the median? Explain your reasoning.

3. TV watchers. College students in a statistics class were asked how many hours of television they watch per week, including online streaming services. This sample yielded an average of 8.28 hours, with a standard deviation of 7.18 hours. Is the distribution of number of hours students watch television weekly symmetric? If not, what shape would you expect this distribution to have? Explain your reasoning.

4. A new statistic. The statistic $$\frac{\bar{x}}{median}$$ can be used as a measure of skewness. Suppose we have a distribution where all observations are greater than 0, $$x_i > 0$$. What is the expected shape of the distribution under the following conditions? Explain your reasoning.

1. $$\frac{\bar{x}}{median} = 1$$

2. $$\frac{\bar{x}}{median} < 1$$

3. $$\frac{\bar{x}}{median} > 1$$

5. Oscar winners. The first Oscar awards for best actor and best actress were given out in 1929. The histograms below show the age distribution for all of the best actor and best actress winners from 1929 to 2019. Summary statistics for these distributions are also provided. Compare the distributions of ages of best actor and actress winners.66 Mean

SD

n

Best actor

43.8

8.8

92

Best actress

36.2

11.9

92

6. Exam scores. The average on a history exam (scored out of 100 points) was 85, with a standard deviation of 15. Is the distribution of the scores on this exam symmetric? If not, what shape would you expect this distribution to have? Explain your reasoning.

7. Stats scores. The final exam scores of twenty introductory statistics students, arranged in ascending order, as as follows: 57, 66, 69, 71, 72, 73, 74, 77, 78, 78, 79, 79, 81, 81, 82, 83, 83, 88, 89, 94. Suppose students who score above the 75th percentile on the final exam get an A in the class. How many students will get an A in this class?

8. Marathon winners. The histogram and box plots below show the distribution of finishing times for male and female winners of the New York Marathon between 1970 and 1999.67 1. What features of the distribution are apparent in the histogram and not the box plot? What features are apparent in the box plot but not in the histogram?

2. What may be the reason for the bimodal distribution? Explain.

3. Compare the distribution of marathon times for men and women based on the box plot shown below. 1. The time series plot shown below is another way to look at these data. Describe what is visible in this plot but not in the others. ### 2.5.3 Interactive R tutorials

Navigate the concepts you’ve learned in this chapter in R using the following self-paced tutorials. All you need is your browser to get started!

You can also access the full list of tutorials supporting this book here.

### 2.5.4 R labs

Further apply the concepts you’ve learned in this chapter in R with computational labs that walk you through a data analysis case study.

### References

Backstrom, Lars. 2011. Facebook Data Team’s Notes.

n.d.a.

n.d.b.

1. Answers may vary. Scatterplots are helpful in quickly spotting associations relating variables, whether those associations come in the form of simple trends or whether those relationships are more complex.↩︎

2. Consider the case where your vertical axis represents something “good” and your horizontal axis represents something that is only good in moderation. Health and water consumption fit this description: we require some water to survive, but consume too much and it become toxic and can kill a person.↩︎

3. $$x_1$$ corresponds to the interest rate for the first loan in the sample, $$x_2$$ to the second loan’s interest rate, and $$x_i$$ corresponds to the interest rate for the $$i^{th}$$ loan in the data set. For example, if $$i = 4$$, then we’re examining $$x_4$$, which refers to the fourth observation in the data set.↩︎

4. The sample size was $$n = 50$$.↩︎

5. Other ways to describe data that are right skewed: skewed to the right, skewed to the high end, or skewed to the positive end.↩︎

6. The interest rates for individual loans.↩︎

7. Unimodal Remember that uni stands for 1 (think unicycles). Similarly, bi stands for 2 (think bicycles). We are hoping a multicycle will be invented to complete this analogy.↩︎

8. There might be two height groups visible in the data set: one of the students and one of the adults. That is, the data are probably bimodal.↩︎

9. Figure 2.7 shows three distributions that look quite different, but all have the same mean, variance, and standard deviation. Using modality, we can distinguish between the first plot (bimodal) and the last two (unimodal). Using skewness, we can distinguish between the last plot (right skewed) and the first two. While a picture, like a histogram, tells a more complete story, we can use modality and shape (symmetry/skew) to characterize basic information about a distribution.↩︎

10. Since $$Q_1$$ and $$Q_3$$ capture the middle 50% of the data and the median splits the data in the middle, 25% of the data fall between $$Q_1$$ and the median, and another 25% falls between the median and $$Q_3$$.↩︎

11. These visual estimates will vary a little from one person to the next: $$Q_1 \approx$$ 8%, $$Q_3 \approx$$ 14%, IQR $$\approx$$ 14 - 8 = 6%.↩︎

12. (a) Mean is affected more. (b) Standard deviation is affected more.↩︎

13. If we are looking to simply understand what a typical individual loan looks like, the median is probably more useful. However, if the goal is to understand something that scales well, such as the total amount of money we might need to have on hand if we were to offer 1,000 loans, then the mean would be more useful.↩︎

14. Note: answers will vary. There is some correspondence between high earning and metropolitan areas, where we can see darker spots (higher median household income), though there are several exceptions. You might look for large cities you are familiar with and try to spot them on the map as dark spots.↩︎

15. The data used in this exercise can be found in the openintro package: mammals.↩︎

16. The data used in this exercise can be found in the openintro package: pm25_2011_durham.↩︎

17. The data used in this exercise can be found in the usdata package: county_complete.↩︎

18. The data used in this exercise can be found in the usdata package: county_complete.↩︎

19. (a) 0.451 represents the proportion of individual applicants who have a mortgage. (b) 0.802 represents the fraction of applicants with mortgages who applied as individuals.↩︎

20. (a) 0.122 represents the fraction of joint borrowers who own their home. (b) 0.135 represents the home-owning borrowers who had a joint application for the loan.↩︎

21. Answers may vary a little. The counties with population gains tend to have higher income (median of about $45,000) versus counties without a gain (median of about$40,000). The variability is also slightly larger for the population gain group. This is evident in the IQR, which is about 50% bigger in the gain group. Both distributions show slight to moderate right skew and are unimodal. The box plots indicate there are many observations far above the median in each group, though we should anticipate that many observations will fall beyond the whiskers when examining any data set that contain more than a couple hundred data points.↩︎

22. Answers will vary. The side-by-side box plots are especially useful for comparing centers and spreads, while the hollow histograms are more useful for seeing distribution shape, skew, and potential anomalies.↩︎

23. The ridge plots give us a better sense of the shape, and especially modality, of the data.↩︎

24. The ridge plots give us a better sense of the shape, and especially modality, of the data.↩︎

25. The data used in this exercise can be found in the openintro package: antibiotics.↩︎

26. The data used in this exercise can be found in the openintro package: immigration.↩︎

27. Source: YouGov Survey Results, retrieved Oct 7, 2019.↩︎

28. The study is an experiment, as patients were randomly assigned an experiment group. Since this is an experiment, the results can be used to evaluate a causal relationship between the malaria vaccine and whether patients showed signs of an infection.↩︎

29. $$4 / 6 - 7 / 14 = 0.167$$ or about 16.7% in favor of the vaccine. This difference due to chance is much smaller than the difference observed in the actual groups.↩︎

30. The data used in this exercise can be found in the openintro package: avandia.↩︎

31. The data used in this exercise can be found in the openintro package: heart_transplant.↩︎

32. The data used in this exercise can be found in the openintro package: cia_factbook.↩︎

33. The data used in this exercise can be found in the openintro package: oscars.↩︎

34. The data used in this exercise can be found in the openintro package: marathon.↩︎