Problems with endogenous sample selection
Consider an experiment with \(N\)
units. For each unit \(i\), we denote
the outcome as \(Y_i\), the binary
treatment status as \(D_i\), the binary
response indicator as \(S_i\), and
\(P\) pre-treatment covariates as \(\mathbf{X}_i = (X_{1i}, X_{2i}, \dots,
X_{Pi})\). The outcome \(Y_i\)
is observed only when \(S_i = 1\). The
variable \(S_i\) reflects whether the
unit opts to participate in the experiment or reports their outcome, and
its value may be influenced by treatment status. Under the potential
outcome framework, we have \[
Y_i = \begin{cases}
Y_i(1), D_i = 1 \\
Y_i(0), D_i = 0,
\end{cases} \text{ and } S_i = \begin{cases}
S_i(1), D_i = 1 \\
S_i(0), D_i = 0.
\end{cases}
\] For simplicity, we assume that the treatment is randomly
assigned with a positive and known probability, so that \(D_i\) is exogenous: \[
\begin{aligned}
& (Y_i(1), Y_i(0), S_i(1), S_i(0)) \perp D_i, \\
& \varepsilon < P(D_i = 1) < 1 - \varepsilon, \text{ with }
\varepsilon > 0.
\end{aligned}
\] Note that we can estimate the average outcome in either the
treatment group or the control group only by conditioning on the event
\(S_i = 1\). Since \(S_i\) is not randomly assigned, the
difference-in-means estimator may no longer be unbiased. As depicted in
the Directed Acyclic Graph (DAG) below, there may exist an unobservable
variable \(U_i\) influencing both \(S_i\) and \(Y_i\). In this case, the difference in the
average outcome across the treatment and control groups captures the
influence of both \(U_i\) and \(D_i\), and thus does not estimate the
average treatment effect accurately. Since \(S_i\) is an endogenous variable in the
relationship between \(Y_i\) and \((D_i, S_i)\), we refer to this issue as
“endogenous sample selection.”
This issue is common in social science and can also be interpreted through the lenses of post-treatment bias (Montgomery, Nyhan, and Torres 2018) or “phantom counterfactual” (Slough 2022). Conventionally, to mitigate the bias arising from conditioning on \(S_i\), researchers often turn to sample selection models (Heckman 1979) or various imputation methods (Honaker, King, and Blackwell 2011; Li et al. 2013; Blackwell, Honaker, and King 2017; Liu 2021). However, these methods generally assume that researchers have access to all variables influencing the selection process, a condition that is often unrealistic and contradicts the core rationale of conducting an experiment.
Researchers can also visualize results from CTTB() using the
plot() function, which produces a coefficient plot with
confidence intervals based on ggplot2. Additional elements can
be added to the generated plot to support further analysis. For example,
one can compare the estimated bounds against the ATE and the LATE for
the always-responders, as illustrated in the plot below.
Next, we estimate the conditional bounds using age as the moderator. The
results suggest that the effect may be larger among younger
subjects.
