Lehallier B, Essioux L, Gayan J, Alexandridis R, Nikolcheva T, Wyss-Coray T, Britschgi M, Alzheimer’s Disease Neuroimaging Initiative. Combined Plasma and Cerebrospinal Fluid Signature for the Prediction of Midterm Progression From Mild Cognitive Impairment to Alzheimer Disease. JAMA Neurol. 2015 Dec 14;:1-10. PubMed.
Recommends
Please login to recommend the paper.
Comments
Rutgers Biomedical & Health Sciences
This is an interesting methodology paper that integrates different types of data (clinical, genetic, MRI features, and CSF and plasma proteins). Even though ADNI is a convenient sample that has been used for discovery purposes, the multi-centered nature of ADNI makes it a better validation set than a discovery set because it is more prone to variable signal-to-noise ratios from the different centers.
We have some experience validating discovery findings from ADNI. Among CSF and plasma analytes identified as associated with AD in ADNI, fewer than one-third of the associations have had some type of validation. A major challenge so far has been the technical validity of assays. While we and others use these assays to measure proteins in biological fluids, all the assays are most reproducible in buffer. There are many proteins in the fluids of interest (CSF or plasma) that can interfere with antibody-antigen binding, either by enhancing it or reducing it. Some of these processes can be stochastic (or are assumed to be), which makes precise (reproducible) measurements sometimes difficult, especially if the solute levels are low (as is often the case in CSF). Second, in ADNI the Myriad RBM (Luminex) assays are performed by third-party vendors, and we do not get control data of sufficient quality. This contrasts with in-house experiments that generate quite a bit of QC data that allow us to determine whether a particular plate/batch is good or bad. Third, the analytical approaches chosen (traditional statistics, modeling, network analysis, etc.) do not generate consistent outcomes because of intrinsic differences in the data structure of these biofluid proteins. Without independent discovery and validation sets, it is very easy for the combination of these factors to grossly overfit the data, which is possibly the most common cause for non-replication (even with cross-validation, which this paper did).
In these studies, people are likelier to believe a particular analyte is truly associated with Alzheimer's if there are reports to that effect, and they sometimes overlook the issue of how the assays perform. The likelihood of an association in ADNI is very high, because the 190 analytes measured in ADNI were assembled after a literature review of proteins previously associated with AD. In keeping with the increasing emphasis on rigor and reproducibility in the broader scientific community, we all share the responsibility of redirecting some of the precision-associated enthusiasm towards technical validation. There is now an NIH initiative that solicits journals to encourage reproducible, robust, and transparent data analysis and reporting (see NIH: Research and Reproducibility), but there is quite a bit of catching up to do in the field.
View all comments by William HuMake a Comment
To make a comment you must login or register.