Owen Yang
So to be more precise, when we just say random effect, I believe we normally mean random intercept, although technically random effect should refer to random slope or coefficient, or even wider concepts.
The ecology problem
Let us just say we plan to encourage patients to open up their sexual orientation to their doctors by simply placing a rainbow flag in the clinics. We may choose some days with the rainbow flag and some days without. We then need to collect information from a few clinic in order to get sufficient sample size to achieve conclusions.
Because clinics vary, we can imagine that none of the clinics are really ‘representative’. If we think in statistical term, there could be a larger ‘between-clinic variations’ than ‘within-clinic variations’ on the amount or the proportion of patients opening up their sexual orientations. Some clinic has a set of characters that may be more encouraging anyway, with or without the rainbow flag.
This can lead to a few confounding problems, especially if the characteristics of clinics are not distributed randomly and the chance of observing the outcome is not equal between the intervention group and the control groups (or exposure/non-exposure group). For example, in a study where a disproportionally large number of intervention group individuals are in a relatively encouraging clinic, then we may see an encouraging effect of the intervention even when there is no real effect of the intervention.
I think this is called an ‘ecological fallacy’ or something (not 100% sure). Ecology means a higher-level unit here, and here ‘the clinic level’ is a higher level than ‘the individual level.’
To run a matched study design
An obvious way to account for this problem is to control for the characteristics that cause the difference between clinics, but normally this only works partially because a lot of time we do not really know what is driving the difference between clinic, and so we do know which characteristic to control for.
So another smart way is a stratified study like a case-control design. So we would only want to compare patients on a like-for-like basis, just within each stratum, like a matched study, matching by clinic.
Random effects are opposed to fixed effects
Perhaps there are more than two ways to do the matched analysis, but the two big categories are fixed effect and random effects. Fixed effects basically mean run an analysis with an additional variable to adjust for the stratum, and so here it means adjusting for clinic (as a categorical variable). Random effects mean run an analysis that only cares about within-strata variations of the outcomes, and we as non-statisticians (or clinicians) can imagine a question like ‘compared to an average person in that clinic, does this intervention increase or reduce the likelihood of opening up their sexual orientation’?
The concept is that if we feel the difference between different clinics are relatively case-by-case, and generalising to a new clinic may not be appropriate, then a fixed effect model should fit our assumption better. If we feel the difference between different clinics can be seen as a high level of random variation, and somehow the results can be generalised to a new clinic, then a random effect model should fit our assumption better. In reality, both models calculate mini-coefficients in each stratum and assign some weights for each stratum to combine the mini-coefficients into a summative result. There is a tendency that the sample size has a smaller contribution to weight in random effect models compared to the equivalent fixed effect models.
Whether or not the variations are random can be quite relative because even in the most random scenario, randomisation only works in large numbers. So I really do not suggest we follow a certain protocol to decide, but really take care to consider what is going on in our study and our data.
Nevertheless, can you tell that random effect has a stronger assumption here? Because of this, there is a tendency that random effect provides greater statistical power (i.e. narrower confidence intervals).
Misleading concepts
If we just settle here with this simple scenario, there is a misleading concept that random effect has to be used to account for ecological bias. An ecological data structure does not necessarily lead to a bias. When there is a bias, random effect models are not the only way to deal with this structure.
What we are doing here in this simple scenario, fixed or random effects, has also assumed that the effect of the intervention is largely the same across all strata, and only the ‘baseline risk’ that varies across strata. It can get complicated when we start to assume this is not the case, and to worry that the effects are not the same in different clinics.
