Practice Problems

Hypothesis Tests

All of the data sets for this lab can be found in the stats20data package. Parts a and h require written English answers; d requires English answers and code; the other parts can be answered just with code, output, and plots.

Part I: Permutation Tests

  1. In order to study gender bias in promotion decisions, researchers took 48 resumes and randomly assigned them to 48 bank managers who were asked whether they would promote this candidate or not. The resumes were identical except the name: 24 of them had names generally associated with women, 24 of them had names generally associated with men. The data from this study can be found in the promote data frame. Use this data to determine whether gender played a role in promotion decisions.

    1. Write the null and alternative hypothesis.

      Null: Promotion decisions are independent of the gender of the name on the application. (or promotion proportions are equivalent)

      Alternative: Promotion decisions are dependent on the gender of the name on the application. (or promotion proportions are not equivalent)

    2. Compute the observed test statistic.

library(tidyverse)
library(stat20data)
library(infer)
data(promote)

obs_stat <- promote %>%
  specify(response = decision, explanatory = gender, success = "promote") %>%
  calculate(stat = "diff in props")
c. Visualize the observed data using an appropriate plot.
ggplot(promote, aes(x = gender, fill = decision)) +
  geom_bar(position = "fill")

d. Construct a plot featuring 9 subplots, each one featuring a visualization of a data set generated under the null hypothesis. Does your visualization of the observed data from the previous part look like it be one of these plots?

The plots below show generally closer proportions than in the data, but not by too much - it's a close call.
promote %>%
  specify(response = decision, explanatory = gender, success = "promote") %>%
  hypothesize(null = "independence") %>%
  generate(reps = 9, type = "permute") %>%
  ggplot(aes(x = gender, fill = decision)) +
  geom_bar(position = "fill") +
  facet_wrap(vars(replicate))

e. Construct and save the null distribution of statistics.
null <- promote %>%
  specify(response = decision, explanatory = gender, success = "promote") %>%
  hypothesize(null = "independence") %>%
  generate(reps = 500, type = "permute") %>%
  calculate(stat = "diff in props")
f. Visualize the null distribution.
visualize(null)

g. Compute the p-value.
get_p_value(null, obs_stat = obs_stat, direction = "both")
# A tibble: 1 × 1
  p_value
    <dbl>
1   0.036
h. Interpret the p-value. What does it say about the consistency between the null hypothesis and the observed data?

With a p-value of .056 on this seed (which is just a hair above the standard threshold of .05), its unclear whether the obsered difference in proportions can be attributed to change or not. With a different seed, it's possible this p-value could bounce above and below the .05 threshold.
  1. Instead of conducting a hypothesis test concerning the difference in these proportions, form a confidence interval for your estimate (what had been your observed test statistic). Use the bootstrap to form a 95% confidence interval for this statistic. Does it contain 0? How is this related to the result from question 1?
boot <- promote %>%
  specify(response = decision, explanatory = gender, success = "promote") %>%
  #hypothesize(null = "independence") %>%
  generate(reps = 500, type = "bootstrap") %>%
  calculate(stat = "diff in props")

get_ci(boot, level = .95)
# A tibble: 1 × 2
  lower_ci upper_ci
     <dbl>    <dbl>
1   -0.509  -0.0505

This particular interval does not contain 0, but it’s close. That means 0 is not really a very plausible value of the true difference in proportions, which leads to a similar conclusion as the hypothesis test.

Part II: Tests with a Point Null

  1. In the 2016 NBA season, it was noted that professional basketball player Steph Curry had a remarkable basket-shooting performance beyond 30 feet. The curry data frame contains his long range shooting performance across 27 attempts. By comparison, the long range shooting percentage of NBA players that season was 7.5%. Assess whether this data is consistent with the notion that Steph Curry has a long range shooting percentage that is no different from the average NBA player. Said another way, assess just how remarkable Curry’s shooting performance was in 2016.

    1. Write the null and alternative hypothesis.

    Null: Curry’s shooting rate is the same as the league average, where the probability of a basket is .075.

    Alternative: Curry’s shooting rate is different from the league average of .075.

    There will be some confusion/discussion here about whether this should be a one-sided test.

    1. Compute the observed test statistic.
data(curry)

obs_stat <- curry %>%
  specify(response = shot, success = "hit") %>%
  calculate(stat = "prop")
c. Visualize the observed data using an appropriate plot.
curry %>%
  ggplot(aes(x = shot)) +
  geom_bar()

d. Construct a plot featuring 9 subplots, each one featuring a visualization of a data set generated under the null hypothesis. Does your visualization of the observed data from the previous part look like it be one of these plots?

These plots look dramatically different than the plot of Curry's data. The shot rate is far lower than his.
curry %>%
  specify(response = shot, success = "hit") %>%
  hypothesize(null = "point", p = .075) %>%
  generate(reps = 9, type = "draw") %>%
  ggplot(aes(x = shot)) +
  geom_bar() +
  facet_wrap(vars(replicate))

e. Construct and save the null distribution of statistics.
null <- curry %>%
  specify(response = shot, success = "hit") %>%
  hypothesize(null = "point", p = .075) %>%
  generate(reps = 500, type = "draw") %>%
  calculate(stat = "prop")
f. Visualize the null distribution.
visualize(null)

g. Compute the p-value.
null %>%
  get_p_value(obs_stat = obs_stat, direction = "both")
# A tibble: 1 × 1
  p_value
    <dbl>
1       0
h. Interpret the p-value. What does it say about the consistency between the null hypothesis and the observed data?

A p-value of (approximately) zero indicates that Curry's performance is (very!) inconsistent with notion that he shoots like your average NBA player. His performance this season was truly exceptional.