Meta-analysis is not a form of rigorous independent verification in the vast majority of cases. It is usually a statistical veneer placed, lazily, on top of a smear of vastly different experimental results, giving the appearance of rigor.
Replication is a rigorous form of independent verification.
Question: why should we take your declarations of who is and isn't a scientist more seriously than the scientists themselves?
Lets take a typical cancer drug. It goes through safty, dosing, and efficiacy clinical trials. In lucky cases, it will show efficacy in a multi-year clinical trial across many geographic locations with hundreds of patients.
If the trial is successful, the FDA will eventually approve the drug, the drug can now be prescribed. The FDA mandates that follow up study is done continuously to learn more about the drug; better indications for use, contraindications for when it won't work, etc.
Where does "replication" come into any of this? Why in the world would somebody replicate a dosing study and generate the same data? That would be unethical, dangerous, and counterproductive. At best, one would throw out bad data that was improperly collected. At worst, one would just abandon the drug and move on to a different candidate drug.
When it comes to human studies that come at real cost to human life, not using all the best available information is unscientific, unethical, should result in civil penalties, and should probably result in criminal penalties as well.
This is the situation that the parent was asking about; by making short blanket statements about what a scientist is and is not, without considering the real issues at hand, makes it seem like you're not engaging the issue.
"Scientists replicate data" is a simple thing to say if you're looking at stars or running a particle collider or working on a new synthetic compound; taking that simple minded attitude is not appropriate for much of the most expensive research out there.
The question of what to replicate and when is a difficult one; it's the tradeoff between new discovery and making sure you're on the right path. If you can make a new discovery that simultaneously proves or disproves that you're on the right path, that's a smarter move, but it's not "replication."
THANK YOU. Nobody seems to pick up on the fact that those 3-5 trials cost (on average) in the hundreds of millions of dollars ($30-50M).
Not to mention the countless Institutional, Human Subject, and ethics review boards that must be satisfied before we can even begin to think about laying hands on a human to conduct a study of any sort - let alone one with an investigational new drug.
If you want to be a scientist whose good at their job, you re-run the statistics to put your replication in context, inform the priors you are using, etc.
> Or perhaps just obtain the raw, patient-level data from said company and analyze the data ourselves?
Data can be faked, manipulated, unless you can trace it back to its origin and ensure it was recorded properly. There are multiple cases (even recent ones) where investigators modified the results of their trials in order to make them look better than they were.
I don't think it's obvious that meta-review is better than nothing. It can lend credence to incorrect/fraudulent results, making actual replication appear less important.
No, you don't need to do meta-analysis. Without combining the results of multiple trials together, you can see if an observed effect in one trial is found in other trials through independent analysis.
But explaining whether or not "We didn't get the same answer" is the result of variation in the effect estimate, or because of a genuine failure to replicate, is one aspect of meta-analysis.
The purpose of meta-analysis goes beyond pooling. Indeed, one of the first steps in meta-analysis is "Is pooling even a good idea?"
Replication is a rigorous form of independent verification.