One of the most important elements of science is critical reading of papers. Most of us come into science as undergrads feeling somewhat naive about what we read in papers. Our default tendency is to believe most or all that we read as “true”.
As we get more experienced, we realize that in fact if anything it is the opposite. We have to view papers very skeptically.
And as we get wiser we realize the weaknesses present in even the best papers and how to spot papers that are weak overall relatively quickly. We realize the weaknesses in our own papers before we submit them too. We start reviewing papers for journals, which is very educational as we get to see what the other reviewers wrote as well. In short, we become critical readers of science literature; a key element of being a scientist who contributes to the community.
I believe it is more challenging for those outside of science to critically read papers without this kind of experience and exposure to so many papers. Over my 22 years in science, I’ve read hundreds of papers at least, I reviewed more than 100 I would guess, and I’ve published about 55 of my own papers. I hope I’ve learned from this process and from my mentors and colleagues.
A debate in the stem cell field is how ready adult stem cell therapies are for use in patients. For Multiple Sclerosis (MS), there is some promising pre-clinical animal data suggesting that some day stem cells may be proven safe and effective as treatments for some patients. I would argue that that day is not today and is unlikely to be a day next year or even the next. It’s going to take time.
An MS patient, SammyJo, who received treatment at Celltex and who was interviewed on NPR about her experience, took issue (voiced in a comment on my recent blog post) with my statement that such treatments are not proven safe and effective. She listed 6 papers to argue that in fact such treatments are safe and effective.
Below I’ve listed the 6 papers included with her comments and my own opinions of each paper. My general take is that these are very poor, weak papers. They all have serious shortcomings if one is to take them as supporting the use of such stem cells in people.
SammyJo’s comments are indented & italicized.
My specific responses are in bold. I also present an overall take on this at the very bottom.
“The reality is that the treatments discussed are not scientifically validated as safe or effected.” – Comment from Paul Knoepfler on NPR story.
I welcome input from a stem cell researcher on this. Here are 6 papers I reviewed that made me feel comfortable pursuing this course of treatment, for my untreatable MS. The review papers each cover further studies. Please point our why this science is lacking, for someone in my condition to rely upon:
1. “Mesenchymal stem cells as treatment for MS – progress to date” An exhaustive amount of preclinical data has shown that the intravenous administration of mesenchymal stem cells (MSC) effectively ameliorates experimental autoimmune encephalomyelitis (EAE), a model of multiple sclerosis (MS), through the release of anti-inflammatory and neuro protective molecules. Based on these results, several small pilot clinical trials in subjects with advanced MS have demonstrated that MSC administration is safe and provided an early signal of clinical effectiveness. - Multiple Sclerosis Journal http://msj.sagepub.com/content/early/2012/11/01/1352458512464686?elq=d5772cd1683c44a69ca3361a5408027e
Paul: This paper talks about mouse work mainly using an interesting, but imperfect mouse model of MS. The notably “small” human clinical trials presented provided what the authors claimed was an “early signal” of effectiveness, which is a weak statement if one were to use this paper to support human transplants. I would also quote the abstract (emphasis mine):
The current aim of clinicians and scientists interested in the development of MSC-based strategies for the treatment of MS is to have the ultimate demonstration in large clinical trials that MSC can inhibit CNS inflammation and foster tissue repair as realized clinically, with functional recovery, or visualized by magnetic resonance imaging (MRI).
The goal overall, they admit, is to have large clinical trials that prove effectiveness. Celltex began treating patients without FDA approval and without an IND or clinical trial context. Note that clinical trials also should not charge patients to participate.
2. “Safety of Cell Therapy with Mesenchymal Stromal Cells (SafeCell): A Systematic Review and Meta-Analysis of Clinical Trials” Based on the current clinical trials, MSC therapy appears safe. However, further larger scale controlled clinical trials with rigorous reporting of adverse events are required to further define the safety profile of MSCs. (Manoj et al 2012) http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3485008/ PMCID: MC3485008
Paul: Again, the authors admit that further larger scale controlled clinical trials are required before anyone can be sure on safety. Also the paper does report a whole range of adverse events in patients who get the transplants. The authors wisely noted major limitations of their study (emphasis mine):
“Our systematic review has several limitations. First, despite our comprehensive search strategy, there are a number of completed but unpublished industry sponsored studies and studies published in abstract form only that may alter the safety profile of MSCs. Second, we pooled adverse events across heterogeneous disease states. Given the limited number of clinical MSC studies, and the small sample sizes of each, it was important to pool data across trials to determine if any potential signals of harm existed. Previously, we have advocated this approach when individual trials are not adequately powered to detect potential harm.  However, we acknowledge that the occurrence, type, and severity of adverse events may vary significantly between different populations and according to different MSC characteristics (e.g. dose, type). The limited number of included RCTs precluded the conduct of these sensitivity analyses. Third, the majority of RCTs included in our analysis would be considered a high risk of bias. Although double blinding an MSC trial may be considered ethically unacceptable, it is difficult to justify the lack of concealment of the allocation of patients in many studies.”
3. Cleveland Clinic is several years into a human trial for MS using expanded bone marrow derived stem cells, and they have isolated a growth factor that is part of the anti inflammatory effect that offers such immediate symptom relief:
“Hepatocyte growth factor mediates mesenchymal stem cell–induced recovery in multiple sclerosis models.” http://www.ncbi.nlm.nih.gov/pubmed/22610068 PMID:22610068
“Recovery From Multiple Sclerosis By Growth Factor In Stem Cells” http://www.medicalnewstoday.com/articles/245816.php
Paul. All mouse experiments in these studies. They don’t address human safety at all. I don’t see how this study supports transplants into human patients, but as a stem cell scientist I find the data interesting.
The last 3 papers address the specific Celltex method:
4. Safety of Intravenous Infusion of Human Adipose Tissue-Derived Mesenchymal Stem Cells in Animals and Humans Dr JC Ra http://www.ncbi.nlm.nih.gov/pubmed?term=21303266
Paul: Mostly mouse studies. The few human studies are very weak, only following patients 3 months. Overall, a very unconvincing paper to support the idea that such treatments are safe to be used in human patients.
5. “Taking Stem Cells Beyond Discovery: A Milestone in the Reporting of Regulatory Requirements for Cell Therapy” Stem Cells and Development. August 2011, 20(8): 1295-1296. This article is by leaders in stem cell research at Duke, Tulane and LSU, refers to Dr JC Ra’s study ‘Safety of Intravenous Infusion of Human Adipose MSCs’, as a model for the demonstration that MSCs are safe, with praise for offering a model way to build regulatory frameworks. http://www.ncbi.nlm.nih.gov/pubmed?term=21510815
Paul: This paper is not really an independent paper. It is a comment on #4 paper above. No new data.
6. “Stem cell treatment for patients with autoimmune disease by systemic infusion of culture-expanded autologous adipose tissue derived mesenchymal stem cells.” Journal of Translational Medicine 2011, 9:181 Dr JC Ra, includes 5 case studies at the end. http://www.translational-medicine.com/content/9/1/181
Paul: Very small study, little data presented. Strangely most data discussed is not presented in the actual paper and no figures are shown. Just two small tables. No clear methods presented. Follow up only a few months.
Paul’s overall take on this: Taken together these six papers are, in my opinion, interesting, but from a clinical perspective they are so limited and weak that they do not give me any confidence in the safety or efficacy of adult stem cell treatments for MS patients today.
I realize and respect the fact that the drug therapies for MS today are highly imperfect and not only don’t work for some patients, but also in some cases make them sicker. I also recognize that there is risk from doing nothing as well and I understand the profound effects on quality of life due to MS. However, let’s not kid ourselves that the rationale supporting adult stem cell treatment for MS is strongly scientifically supported at this time. It just isn’t.
Some patients may chose to get the treatments anyway and I won’t judge them, but society generally, the FDA, and for-profit companies as well as the doctors and scientists at such companies have an ethical and importantly a legal responsibility to educate and protect patients.