Our mission is to improve the design, analysis, and interpretation of research by providing resources for researchers and consumers of research alike. We aim to enhance the capacity of researchers to more effectively design and analyze research studies, and to communicate their findings. We also seek to enhance the capacity of consumers of research, including the public, clinicians, and policy makers to more readily interpret research findings in order to promote the wise incorporation of such findings into well informed decisions. We intend for our work to improve and enhance patient centered outcomes research, health services research, and quality improvement and quality of care research, among others.
Our work applies critical analysis to situations that regularly confront health care researchers and the consumers of research both. Our work spans from the development of new analytical frameworks, such as regression risk analysis, to our work that seeks to help patient centered outcomes researchers and comparative effectiveness researchers to develop and select appropriate conceptual frameworks that accommodate the complex nature of real-world patients.
Regression Risk Analysis
Our initial work resulted in the development of regression risk analysis (RRA), a novel statistical approach. Dr. Lawrence Kleinman and Prof. Edward Norton developed and validated this method as a solution to a longstanding challenge regarding how best to analyze and interpret multivariable models with dichotomous outcomes, using logistic regression as the primary example. RRA allows researchers to readily generate accurate estimates of both relative effect size (the adjusted risk ratio) and of absolute effect size (the adjusted risk difference) and their standard errors directly from nonlinear maximum likelihood models, such as logistic and probit regressions. Given frequent misgivings in the literature regarding interpretation of the more commonly used odds ratio, RRA thus allows researchers to present their findings in a way that both researchers and consumers of research can be expected to understand more intuitively than current methods.
Regression Risk Analysis and Rank Reversal
Because RRA allows for easy calculation of adjusted risk ratios and adjusted risk differences, researchers now have a choice of which measure to use in their analysis. During our early work with RRA, we contemplated what would be desirable practice for deciding between the different measures of effect. We noticed that, in studies indirectly comparing two or more treatments, the ranking of treatments may vary across measure choices. In other words, results reported as odds ratios may rank treatments differently than results reported as risk ratios. We have labeled this phenomenon “rank reversal”.
Rank reversal has important implications for researchers. Because the choice of risk measure may impact the conclusions drawn from research results, researchers must carefully consider which measure they select, and explicitly state the conceptual framework used for their study. Similarly, consumers of research should make note of the choice of measure when consulting research findings to guide decision making.
Our work detailing rank reversal is currently under peer-review. Updated information will be provided at a later time.
RRA and Comparative Effectiveness Research
Comparative Effectiveness Research (CER) seeks to improve the quality, effectiveness, and efficiency of health care and to help patients, health care professionals, and purchasers make informed decisions through direct and indirect comparisons of various treatment options for specified groups of patients.
We recognized that the power of RRA to enhance the nuance of analysis had specific importance when applied in the context of CER. When considering analyses involving heterogeneity of effect, layers of multiplicative models challenge both in terms of the need for meticulous housekeeping and in terms of interpreting effect sizes. RRA’s capacity to estimate the risk difference, a measure of absolute effect size, can be particularly valuable in this context.
A very typical instance when heterogeneity of effect becomes central to both the design and interpretation of CER is when considering treatment options in the presence of comorbid conditions. The base case for thinking about the implications of comorbidities is the construct of paired diseases, that is two illnesses that often (but not always) coexist with one another.
We have developed an analytical model to describe the ways that a pair of diseases or conditions can interact with one another. We have extended that model to handle consideration of a treatment that may impact one or both of the condition. This model can be used by researchers to help to design and interpret research that appropriately considers the impact of comorbidity when comparing the effectiveness of treatments for real world patients.
* Our work on Regression Risk Analysis is funded by the Agency for Healthcare Research and Quality, Grant Number 1R18 HS018032-01A1.
* Our work on Comparative Effectiveness Research and Comorbid Conditions is funded by the National Institutes of Health, Grant Number 3UL1RR029887-03S1.