In First Phase 3 Trial, the Tau Drug LMTM Did Not Work. Period.
Quick Links
Results of the eagerly awaited Phase 3 clinical trial of LMTM, a derivative of the dye methylene blue, generated consternation at the Alzheimer’s Association International Conference 2016, held July 22-28 in Toronto. Speaking to a packed audience on behalf of the study’s sponsor, TauRx Pharmaceuticals based in Aberdeen, Scotland, Serge Gauthier reported that LMTM failed to slow cognitive or functional decline in people with mild to moderate Alzheimer’s disease. On the main primary results slide, disease progression curves for both doses of drug and the placebo were practically identical. Scientists’ disappointment at this finding soon turned into disbelief when Gauthier went on to present a subgroup analysis that held no statistical credence yet purported to show a strong benefit on cognition and brain atrophy. LMTM is also known as TauRx0237 or LMT-X; Gauthier is at McGill University in Montreal, and chairs the scientific advisory board of TauRx.
Many researchers in the room, and later in the hallways, were dismayed not only at the way the data were being parsed, but also at media stories about the drug’s ostensible “success.” Outlets such as The Times, New Scientist, and the Huffington Post variously announced the arrival of the first drug to halt Alzheimer’s, or that the drug slowed disease by 80 percent, even while leading Alzheimer’s researchers at AAIC were challenging the company’s leader, Claude Wischik, for overstating the trial’s result.
Some media stories ran before the scientific session had even taken place and leaders in biostatistics had a chance to see and discuss the subgroup analysis. That is because press briefings at AAIC are held at 7 a.m. the day of the formal scientific presentation, which in this case happened at 4:30 p.m. Reporters who had attended the press briefing on the trial data and wrote more accurately were left scratching their heads at how their colleagues could have gotten the story so wrong.
Here is the bottom line on this trial: The drug missed its co-primary endpoints of slowing cognitive and functional decline in mild to moderate Alzheimer’s disease, as measured by the ADAS-cog and ADCS-ADL batteries. “The only important finding is that treatment was ineffective,” Paul Aisen, University of Southern California, San Diego, told Alzforum. “The results are disappointing,” said David Knopman, Mayo Clinic, Rochester, Minnesota, who chaired the press briefing but was not involved in the trial. “In my opinion, the only thing that counts is the primary outcome,” Knopman added. “Secondary results are interesting, but our experience in this field, and in trials in general, tells us that secondary analyses are fraught with hidden bias.” John Hardy of University College London was blunt with Alzforum, writing, “This was a misleading presentation of data which failed to show signs of achieving its primary endpoint.”
Suspect Subgroup Analysis
What was wrong with the subgroup analysis? Gauthier told the audience that about 15 percent of the 891 patients in the trial were not taking standard therapy, namely memantine or one of three acetylcholinesterase inhibitors. A pre-specified subgroup analysis suggested that among this group of 136 people, those taking 75 or 125 mg LMTM twice a day had a slower cognitive and functional decline over the 15 months of the trial. He said their brains atrophied less, as well, which he took as an indication that neurodegeneration was slowed. While this analysis prompted the sensational media reports, experts said that it was flawed in two important ways.
First, the researchers lacked any statistical basis for it. They had, in trial parlance, “spent their alpha on the primary analysis,” said Aisen. Wischik conceded that this was the case to biostatisticians and trialists who asked him directly after Gauthier’s talk, and he confirmed it with Alzforum.
What does this mean? Alpha refers to type 1 errors, i.e., false positive results that occur by chance. Researchers must take these potential errors into account, otherwise clinical trial results could be meaningless. Biostatisticians typically power trials to limit the chance of getting a false positive result to 5 percent or less. If they intend to run several different analyses with the trial data, the appropriate way is to pre-specify that, and then ensure the total “alpha” for all those analyses adds up to that 5 percent. There is no free shot at a second analysis. As Aisen explained, one could run analyses ad infinitum until a positive outcome came up by chance. “If you ran 20 analyses with an alpha of 5 percent for each, then one of those would, by chance, be positive," said Aisen, adding, “That would not be a true positive.” The correct approach splits the alpha among the 20 analyses, setting it to 0.25 percent for each. Because the TauRx researchers set their alpha for the primary analyses only, they left the secondary analyses open to false positives and as such, should have deemed the data “exploratory,” as other AD trial sponsors have done in the past.
That aside, the secondary analysis used an inappropriate placebo group, said clinicians at the conference. It compared people on LMTM therapy alone with the placebo group for the whole trial, which includes people taking standard AD drugs. “The correct analysis is to compare the LMTM monotherapy group against people on placebo who are also not on standard AD drugs,” said Aisen. Experts Alzforum spoke with at AAIC agreed that the comparison with the placebo group was inappropriate. “Comparing all control subjects to a subgroup of active subjects is like comparing apples to oysters,” wrote Suzanne Hendrix, Pentara Corporation, to Alzforum (see full comment below).
Gauthier explained that there were too few people in the placebo group who were not on AD drugs to run that analysis. His data slide showed that of 54 people who started in this group, 34 completed the trial. By comparison, 26 and 25 people on the low and high doses of LMTM monotherapy, respectively, completed the trial. Others commented to Alzforum that TauRx could have done the analysis, but chose not to because the results would have missed statistical significance. “If you were to publish this data, you’d have to do it with the correct placebo group,” said Aisen.
Researchers had further reasons to question the secondary analysis. Aisen noted that typically in AD trials, participants with a diagnosis of AD who are not on standard AD drugs may have very slowly progressing disease, or they may not have AD at all. “The need for treatment is an indicator of more rapid decline; for this reason the secondary analysis may simply reflect slow decliners,” he said (see Schneider et al., 2011). If that is what is happening here, then the group on LMTM monotherapy would be expected to show slower decline in any marker of disease, including atrophy, he added. Other leading trialists agreed that the monotherapy group may represent the slow progressors.
Aisen further pointed out that the fraction of people in the trial who carried an ApoE4 allele—48, 42, and 53 percent among the placebo, low-, and high-dose LMTM, respectively—seemed low. “In ADCS trials, ApoE4 prevalence among people with mild to moderate AD is typically 60-70 percent,” he said. “Since there was no biomarker to support the diagnosis of Alzheimer’s disease, the low ApoE4 numbers make one wonder about the accuracy of the diagnosis,” Aisen said. “If you rely solely on clinical diagnosis, then you had better have highly experienced clinicians,” he added. In ADNI, the bapineuzumab and first set of solanezumab Phase 3 trials, about a quarter of participants, especially the ApoE4 non-carriers, were later found to be amyloid-negative.
Reisa Sperling, Brigham and Women’s Hospital, Boston, asked if there was something unique about the people who were not on standard AD treatments. On this point, there seemed to be some confusion. During his conference presentation, Gauthier said nothing distinguished this group, but earlier during the press briefing he had said that many of them were from Eastern Europe and Malaysia. “I think it is very possible that people who entered the study who were not on standard care may have been from areas not receiving good health care, and when they entered the trial they experienced a placebo effect,” suggested Knopman. “There have been previous examples of that,” he added. Dimebon was reported to have had a treatment benefit in a Phase 2 trial in Russia before failing in Phase 3 trials conducted in the Americas, Europe, Australia, and New Zealand (see Mar 2010 news).
Others wondered if, as part of regular checkups in the context of a trial, those patients had begun to receive treatment for other conditions that may affect mental status, such as hypertension or diabetes. “That is certainly a valid consideration,” said Ron Petersen, who is also from the Mayo Clinic.
Gauthier reported that about 30 percent of participants had dropped out. Side effects were primarily gastrointestinal and urinary, and Gauthier called the drug’s safety profile acceptable. Among all treated patients, 25 percent reported diarrhea and 10 percent dysuria, a burning sensation related to the drug.
In an interview with Alzforum, Wischik acknowledged that the results were disappointing. He also accepted that because the alpha of 0.05 was expended in the primary analysis, the secondary analyses were therefore “formally hypothesis-generating and provide only nominal p-values (albeit corrected for multiple comparisons).” A TauRx press release stated “clinically meaningful and statistically significant reductions in the rate of disease progression were observed across three key measures in patients who were treated with LMTX® as their only Alzheimer’s disease medication.” The closing slide of Gauthier’s presentation to the field stated “LMTM as monotherapy is a safe and effective treatment for mild to moderate AD with larger effect size than currently available treatments.”
Where does this leave LMTM? Wischik told Alzforum that going forward, the most important step was to modify the primary analysis of a second, completed AD trial to make it a monotherapy analysis. He said that two-thirds of the patients in the second trial are from North America and one-third are from Europe, and that 20 percent of them are not taking cholinesterase inhibitors or memantine. Combining data from both trials should yield a large enough monotherapy placebo group for statistical comparison with LMTM monotherapy, he said. He said the second trial had the same subgroup result, namely that LMTM as monotherapy slowed decline and brain atrophy. “The first study generated that hypothesis, and the primary analysis of the second study was modified accordingly in the SAP [statistical analysis plan],” noted Wischik. This modified primary analyses would cut the number of patients available.
That data will be presented at the upcoming CTAD meeting in December. Data on the third trial with this drug—in frontotemporal dementia—was originally scheduled to be presented at AAIC, as well, but Wischik withdrew this second talk. The FTD results will instead be presented at the ICFTD meeting starting August 31 in Munich.
For prior coverage of this drug development program, see Aug 2008 conference news; Jul 2009 conference news; May 2015 news.—Tom Fagan
References
Therapeutics Citations
News Citations
- Dimebon Disappoints in Phase 3 Trial
- Chicago: Out of the Blue—A Tau-based Treatment for AD?
- Vienna (and Burkina Faso): What's New With Methylene Blue?
- Et Tu, Methylene Blue? Drug Only Works as Prophylactic
Conference Citations
- Clinical Trials on Alzheimer's Disease 2016
- International Conference on Frontotemporal Dementias 2016
Paper Citations
- Schneider LS, Insel PS, Weiner MW, . Treatment with cholinesterase inhibitors and memantine of patients in the Alzheimer's Disease Neuroimaging Initiative. Arch Neurol. 2011 Jan;68(1):58-66. PubMed.
Other Citations
External Citations
Further Reading
No Available Further Reading
Primary Papers
- Gauthier S, Feldman HH, Schneider LS, Wilcock GK, Frisoni GB, Hardlund JH, Moebius HJ, Bentham P, Kook KA, Wischik DJ, Schelter BO, Davis CS, Staff RT, Bracoud L, Shamsi K, Storey JM, Harrington CR, Wischik CM. Efficacy and safety of tau-aggregation inhibitor therapy in patients with mild or moderate Alzheimer's disease: a randomised, controlled, double-blind, parallel-arm, phase 3 trial. Lancet. 2016 Dec 10;388(10062):2873-2884. Epub 2016 Nov 16 PubMed.
Annotate
To make an annotation you must Login or Register.
Comments
Pentara Corp
Primary results from the Phase 3 study of LMTX were not significant, but a significant treatment effect was reported in a subgroup of approximately 15 percent of ACTIVE patients who were not taking concomitant ACHEI therapy. Unfortunately, the analysis did not actually make a comparison within the subgroup as was implied, but instead compared the active group patients who were not on ACHEI to the total placebo group, including both those on and those not on ACHEI therapy. This is a totally inappropriate comparison, since progression rates often differ between those on and not on ACHEI therapy. If the appropriate comparison of ACHEI non-users on active treatment to ACHEI non-users on placebo had been significant, it would have been shown. There are no positive results that have been presented yet, even for the subgroup.
Most criticism of these results is based on three opinions: 1. It's a post hoc subgroup analysis and only primary analyses should be considered. 2. Changing the primary analysis for the second Phase 3 study is "moving the goal posts." 3. Studies done outside of first-world countries may be suspect. Post hoc analyses can be appropriate, but they have to meet a higher standard than pre-specified analyses. Changing the primary analysis for an unblinded study may also be appropriate under the right circumstances. Studies may be done appropriately outside of first-world countries. But comparing all control subjects to a subgroup of active subjects is like comparing apples to oysters.
I have no business relationships with TauRx or any tau-based programs.
Syneos Health
The lack of using the proper placebo control group for this post-hoc analysis was the most glaring omission in the initial presentation and only brought to light when the questioning period started. Dr. Gauthier alluded the sister study had similar results, and I would hope they take a lesson here to present the data in a more acceptable way. If there truly is an effect, there needs to be a good hypothesis as to why. Again, this was not addressed in the presentation nor in any of the press releases so far. One could suppose this population may not have been truly demented or became more stabilized with the study procedures and oversight as surmised above.
Overall, while interesting, I think caution is warranted before making claims of "effective treatment" without the statistics to back it up. Even if the signal is there, a confirmatory monotherapy study will assuredly be required before the FDA is going to accept it.
AbbVie
Dr. Aisen correctly notes that the alpha was spent on the analysis of the co-primary outcome measures, meaning that any subsequent analyses carry the risk of an inflated type I error. The fact that this sub-group analysis was pre-specified suggests that it was not the result of a massive hunting expedition for a nominally significant result, but it’s not clear just how many post-hoc comparisons were carried out.
What was not discussed at the presentation was why there were patients diagnosed with mild to moderate AD who were not being treated despite there being approved therapies. Where did these subjects come from? Why were they not being treated? Were they more or less impaired than the ones being treated with acetylcholinesterase inhibitors/memantine?
One minor correction regarding type I error: If one carries out 20 comparisons using a p-value of 0.05, the probability of at least one comparison having a p-value less than 0.05 is approximately 64 percent.
Henan University of Chinese Medicine
It is great that the Alzforum journalists set the record straight. There have been a lot of hyped and misleading news releases on this topic. The methylene blue drug trial has failed. Few people in the research area are surprised by this.
At the end of the day, the scientists have the last word. The small effect of the monotherapy can only be classified as a pilot study. The stats are shaky, and we all have seen this before. One should not forget that there is a company behind all this that is trying to keep investments going.
Make a Comment
To make a comment you must login or register.