Owen Yang
This is difficult to me because we have kind of already made up our mind when we ask. This is also not about whether we should choose a parametric or non-parametric test. If you are interested in my view about parametricity, I have moaned about this previously (see here).
The magic 6 in the lab world
I really think the lab world has moved on to the era that there is no longer a magic 6. The magic number varies but tend to be between 3 and 6. It is the number of repeats that a lab person believe they need to do to show the results. The number is magic in a sense that we would defend it so strongly despite there is little sense.
Well, there sort of is. It is a number where you start to feel the shape of the distribution of 6 numbers can be anything, and so have a bit feeling of randomness.
Source of heterogeneity
Repeating the same PCR 6 times using exactly the same sample is different from repeating PCR using 6 independent samples. We really need to step back and think carefully what it is that we are planning to prove to decide what the most sensible ‘repeat’ it should be, and what the most sensible ‘reliability’ these repeats should look like. If they are viral loads in blood samples from different patients, the chance is that there will be natural variation. If they are viral loads in the same blood sample, we probably would hope that the numbers in these same blood samples are identical, or if not, the variation is much smaller than the variation between samples.
A lot to time this within-sample and between-sample comparison has been done in your quality control process, but we have not processed them into our mind at all. If this is true, we will need some soul searching to realise that every step has its meaning. I hope there is some serendipity there.
Sometimes there have not been done before, and no one has suggested this should be done. Sometimes we feel this is irrelevant to the research question, or an assay is expensive or sample is precious. I do totally understand this frustration.
Larger sample size for less reliable repeats
We do need a larger sample size for less reliable (or more variable) repeats. If you have results from 6 patients, the chance is their numbers will vary because the essay is chosen initially because they are sensitive enough to test differences between samples. If you are comparing two different groups in small sample sizes (say 3 vs 3), then the difference will have to be so large that it transcend the natural variations. If the difference is not large but you believe there could be some moderate-sized difference, then you really need a large sample size to tell the statistical difference.
It is what statistical power means. The larger the sample size, the larger the power to detect (real) small differences. A lot of time we do have this knowledge, but we cannot believe one would actually quote something like this in reality. This is a little sad because some of us are not aware there are people in the world who actually have the resource to do things by the book, and also actually believe we should generate science by the book. I understand this sadness because in one way or another, I am in the same situation.
Box plots or bar charts
Most of you do know what I feel already about this question. The purpose of these plots is to represent the data. If we only have 6 data points, the most sensible choice is probably showing the 6 numbers in dots. Once you have done it you are free to choose whichever you like overlaying on it, being it a box plot or a bar chart. This is because no mean or standard deviation, or no median or interquartile range, is better than the 6 dots to represent the 6 dots themselves. When there are 40 dots in a group, it becomes very difficult for human eyes to comprehend the distribution of them and compare between two groups, and it is probably easier to digest the distribution using parameters (e.g. means and standard deviations), although you do need to use the right parameter that can best describe the distribution.
Stand up for science
Most lab sciences are still under some sort of apprentice-like climate, and we have no energy to challenge the protocols written up for us to follow. But when you feel empowered, please stand up for science. Stand up to your supervisor. Stand up to your funding body. A lot of times they just do not have the time to think about it, either, and can sometimes welcome your suggestion.
