CIMPOD Conference 2016

08:30 AM

Registration and Breakfast

09:00 AM

Welcome and Opening Remarks

Speaker: Yi Zhang, MTPPI

09:10 AM

Conference Introduction: the Importance of CI Methods for PCOR/CER

Speaker: Jason Gerson, Associate Director for CER Methods and Infrastructure team, PCORI (Slides)

Moderator: Julia Kim, MD, Assistant Professor, Pediatrics, John Hopkins School of Medicine

09:30 AM

Causal Inference as an attempt to emulate a target experiment

Speaker: Miguel Hernan, Harvard School of Public Health. (Slides)

Moderator: Julia Kim, MD, Assistant Professor, Pediatrics, John Hopkins School of Medicine

Abstract: Ideally, questions about comparative effectiveness or safety would be answered by an appropriately designed and conducted randomized experiment. When we cannot conduct the randomized experiment, we analyze observational data. Causal inference from large observational databases can be viewed as an attempt to emulate a randomized experiment---the target experiment or target trial--- that would answer the question of interest. When the goal is to guide decisions among several strategies, causal analyses of observational data need to be evaluated with respect to how well they emulate a particular target trial.

This talk outlines a framework for comparative effectiveness research using observational data that makes the target trial explicit. This framework channels counterfactual theory for comparing the effects of sustained treatment strategies, organizes analytic approaches, provides a structured process for the criticism of observational studies, and helps avoid common methodologic pitfall

10:10 AM

Fundamental issues: Understanding differences between observed and unobserved confounding

Speaker: Mike Baiocchi, Stanford University (Slides)

Moderator: Julia Kim, MD, Assistant Professor, Pediatrics, John Hopkins School of Medicine

Abstract: This talk will provide a foundational framework for understanding the primary concerns in estimating causal effects. We will discuss different sources of variation in your data that, if not addressed properly, might lead to erroneous conclusions. We will focus on the difference between observed and unobserved confounding - providing mathematical, intuitive and real-world examples - and then move on to how to pick methods that address problems arising from observed, unobserved or both kinds of confounding. The goal of this talk is to make you comfortable with the terminology and key ideas of confounding, as well as aware of the class of methodologies/study designs that appropriate to address your concerns.

10:50 AM

Tea Break

11:05 AM

Major threats to validity in observational studies

Speaker: Jay Kaufman, McGill University (Slides)

Moderator: Julia Kim, MD, Assistant Professor, Pediatrics, John Hopkins School of Medicine

Abstract: In observational studies in which the goal is etiologic inference, the investigator seeks to estimate the contrast between two potential interventions on some defined target population.  When this is based on observing associations or adjusted associations between exposure and outcome in the data set, the estimate is considered to be valid if a study with an increasing large sample size would converge to the true value of the causal effect.  There are several broad threats to the validity of the estimate -- improprieties that can lead the estimated value of the effect to be too high or too low.  The most common classification of these threats is into 1) confounding biases, 2) selection biases, and 3) information biases.  Each of these classes of potential bias has a unique structure that can be illustrated in a causal graph. 

Confounding bias involves common causes of the exposure and the outcome.  Selection bias, in contrast, involves conditioning in the design or analysis on consequences of the exposure and/or consequences of the outcome. Finally, information biases involve measurement error or misclassification of any of the study variables, including exposure, outcome or covariates.  Beyond these sources of bias, there are other potential errors in the analysis, for example the misspecification of the model or the inappropriate adjustment for factors affected by the exposure.   Taken together, these various challenges contribute to yielding an estimate that can deviate systematically from the true causal effect, which is the value that would be obtained under an actual intervention.  P-values and confidence intervals relate to only one alternative explanation for a study finding:  that it is a chance variation due to sampling variability.  These other sources of bias and error represent other alternative explanations, and also need to be considered and ruled out before one can extend much credibility to the estimated effects of an observational study.

11:45 AM

The 'best' or 'best possible' methods: controversies in the field

Speaker: Heejung Bang, University of California, Davis (Slides)

Moderator: Julia Kim, MD, Assistant Professor, Pediatrics, John Hopkins School of Medicine

Abstract: Causal inference is inherently complicated, yet a number of methods are available nowadays. Some methods are popularly adopted in practice, especially, with programming guide/software, while some methods are still considered unwieldy or some users do not feel full comfort and trust in use.  Despite high demand and increased acceptance, causal inference based on non-experimental data is still a controversial topic, including in statistical training/curriculum, where mechanical use of a method with inadequate considerations to qualitative issues may do more harm than good.

Causal inference naturally entails a number of assumptions, some of which are strong or unverifiable with ‘observed’ data.  Moreover, it can be a daunting task for practitioners or even quantitative researchers/data analysts to remember and check all of the assumptions and caveats in each step in implementation.  Also, we find that different methodology camps tend to claim their method is the best. In this talk, we will discuss how to choose best or best feasible methods for a given problem and report the findings. We will further discuss some controversies in causal inference, and see if we can harmonize the practice in wide but different fields

12:25 PM

Lunch Break

01:25 PM

Assumptions, sensitivity analyses, and software.

Speaker: Dylan Small, University of Pennsylvania (Slides)

Moderator: Martin Ho, FDA/CDRH/OSB/DBS; 

Abstract: Two fundamental assumptions for many observational study analysis methods are (1) that there is no unmeasured confounding (also called treatment ignorability, exchangeability, selection on observables or treatment exogeneity); (2) positivity -- that for every covariate value, there is a positive probability of being in both the treatment and control group.  We will discuss these assumptions, ways of dealing with violations of these assumptions through sensitivity analysis and limiting the study population, and associated software.

02:05 PM

Sensitivity analysis for untestable assumptions: formulation, implementation, interpretation

Speaker: Joseph Hogan: Brown University (Slides)

Moderator: Martin Ho, FDA/CDRH/OSB/DBS

Abstract: A rigorous analysis about causal effects includes some accounting for extra variation attributable to untestable assumptions. To fix ideas, this talk focuses on unmeasured confounding. Using an example from HIV epidemiology, we describe three specific approaches: (i) using bounds instead of point estimates to reflect lack of information about the unmeasured confounder(s); (ii) representing the unmeasured confounder as an unobserved variable that is associated with the outcome and treatment; and (iii) representing unmeasured confounding in terms of differences between potential outcomes distributions. Methods (ii) and (iii) can be used to generate graphical depictions, ‘tipping point’ analyses, and other representations of unmeasured confounding bias. The talk will compare and contrast each approach with focus on usage in practical settings.

02:45 PM

Tea Break

03:00 PM

Considerations on the choice of the statistical methods for PCOR questions

Speaker: Sara Lodi, Harvard School of Public Health (Slides)

Moderator: Martin Ho, FDA/CDRH/OSB/DBS

Abstract: PCOR questions often involve the comparison of complex treatment strategies sustained over long periods of time using data collected from routine clinical practice (or electronic health records). Traditional statistical methods (e.g. Cox or logistic regression analysis) do not always provide correct answers to these types of questions under valid assumptions. Using examples from the medical literature, we will illustrate when and why traditional methods fail. In particular, we will show that in the presence of time-updated treatments and/or confounders, a common situation when using routinely collected clinical data, an alternative class of methods, so-called g-methods become the most appropriate choice.

03:40 PM

A systematic road map for translating a CER question into a statistical analysis

Speaker: Maya Petersen, University of California, Berkley (Slides)

Moderator: Martin Ho,FDA/CDRH/OSB/DBS

Abstract: This session will describe a systematic road map for translating an applied causal question into a statistical analysis, highlighting the following key points in this process: 1) translation of a scientific question into formal causal query, with an emphasis on how to select a target counterfactual quantity (defined in terms of an ideal experiment) that comes as close as possible to answering the motivating scientific question; 2) translation of a causal query into a statistical estimation problem, with an emphasis on how to choose an appropriate statistical model and estimand (statistical target parameter); and, 3) estimation and inference, with an emphasis on how to choose the estimator that will perform best (in terms of robustness, precision, and reliable inference) in your data.

04:20 PM

Day 1 Wrap-up ; Day 2 Overview

Speaker: Yi Zhang, MTPPI.

04:30 PM

Adjournment