Paper
- Alzforum Recommends
Janelidze S, Teunissen CE, Zetterberg H, Allué JA, Sarasa L, Eichenlaub U, Bittner T, Ovod V, Verberk IM, Toba K, Nakamura A, Bateman RJ, Blennow K, Hansson O. Head-to-Head Comparison of 8 Plasma Amyloid-β 42/40 Assays in Alzheimer Disease. JAMA Neurol. 2021 Nov 1;78(11):1375-1382. PubMed.
Recommends
Please login to recommend the paper.
Comments
Biomarkable bvba
This head-to-head comparison of the clinical performance of eight plasma Aβ assays is a step forward in the integration of blood-biomarker testing for patient stratification or diagnostic applications. Its use of the DeLong tests to compare ROC AUC between different assays is appropriate and effective. The paper provides information on the impact of analytical precision, both intra- and inter-assay variability, on the clinical outcome.
Janelidze et al. was done independently and in parallel with an fNIH study presented at AAIC this year. The study provides a reference to define acceptance criteria for precision for other or emerging technologies, including, but not limited to, chip-based technologies or point-of-care tests.
The clinical utility for using plasma Aβ to identify brain amyloidopathy (using CSF Aβ42/Aβ40 ratio as a reference) differs depending on the applied technology. Technological differences include, among others:
A detailed description of the assay format, its performance, and characteristics of the mAbs are absolutely necessary to enable the reader to correctly interpret the data and to make proposals for future optimization of the assay designs.
The lack in Janelidze et al. of good correlation between assays for Aβ proteins points to the fact that there are still many unknowns about the biology and value of using plasma Aβ. It is largely unknown whether binding proteins can change during the progression of the disease, or be different in function depending on the pathology. Several binding proteins can interfere with the measurement's accuracy. Environmental factors or activities of daily living, e.g., exercise, can result in a change of peripheral Aβ levels.
This paper appropriately identifies cohort differences as a potential culprit of the change. The outcome of this assays comparison might be compromised if the outliers, and number of outliers, during testing of samples are not identical for each technology (see info in Supplementary file). Most of the cohorts are very heavily biased toward APOE4, age, and MMSE. In contrast to the fNIH study mentioned above, here there is no information on a model with only age and APOE status. The effect of inclusion of APOE in the model for the BioFINDER samples with plasma Aβ42/Aβ40 was much higher for ELISA immunoassays from Euroimmun as compared to the other assay formats. Future studies can help to better asses the added value of any biomarker over demographic assessment within APOE4 noncarriers.
It has been difficult in the past to quantify Aβ isoforms accurately and precisely in blood samples. This was due in part to the complexity of the matrix, the low abundancy of selected proteins, risk of oligomerization or loss of materials due to adsorption at the time of sample collection and handling, and, last but not least, the absence of highly qualified monoclonal antibody pairs (mAbs) with proven affinity, specificity, and selectivity for the analyte of interest (Vanderstichele et al., 2019).
Previously, multiplex immunoassays for different Aβ isoforms (Aβ42, Aβ40, Aβ1, AβN) have been developed and validated on Luminex's xMAP technology for integration in clinical trials as a tool to follow up treatment efficiency (Lachno et al., 2012; Lachno et al., 2013). The assays used a combination of mAbs (21F12, 2G3, 3D6), which are nowadays also integrated in other commercial ELISAs, i.e., Roche, Euroimmun, and Simoa assays. The process for mAb production and purification depends on the vendor, and can explain some of the differences between technologies.
The assays were qualified for use in the Eli Lilly clinical trials with qualified standard operating procedures (SOP) for collection and storage of the plasma samples (Lachno et al., 2009). Janelidze et al. mentioned already the need to freeze samples shortly after collection, to take care of heterophilic interference, and to be sure to use recipients that are low in protein binding. Although these assays had good precision, their diagnostic value was limited (Hansson et al., 2010; Figurski et al., 2012).
The difference in concentration for Aβ-PET results, plotted here as [Aβ-PET(+)(pg/mL)/Aβ-PET(–)(pg/mL)] x 100 – 100) between Aβ-PET(+) and Aβ-PET(–)], is dependent on analyte (no decrease for Aβ40) and technology. The highest percentage difference for an assay is not always correlated with the best Area Under the Curve (AUC).The outcome of this head-to-head comparison in an identical set of samples is driven in part by the precision of the analysis. The intra- and inter-precision variability differ considerably between technologies, ranging from a few percentage points (ELISA) to more than 15.0 percent (Shimadzu). Intra-assay and inter-assay coefficients of variation (%CV) are higher for Aβ42 than for Aβ40 (UGOT, Shimadzu assay). A difference in precision for one analyte gets integrated into the imprecision data when using the ratio.
In addition, when plotting %CV in function of the AUC for all technologies, we see that part of the differences in AUC is driven by the %CV (as calculated using information in eTable 1). This seems to be more pronounced within a given assay than between assays. In studies where the absolute difference between Aβ-positive and -negative groups is limited to below 15 percent, Precision Qualified Assays (PQAs) are needed. Further improvements of analytical sensitivity will help us obtain better assay precision for analytes in the lower concentration range.
Another critical step for the future will be the generation of a standard operating procedure for collection and storage of the samples, as done recently for CSF (Hansson et al., 2021). Availability of certified reference materials or well-characterized sample sets could not only help to harmonize results between assay formats, but also give clues to possible differences between assays. Samples that do not correlate between technologies are of higher interest than those that do, because they can yield information on the biology behind these discrepancies.
Future assay optimization will have to focus on sensitivity and precision for testing biological samples. Each modification of the production process will have to be verified regarding its effect on the intended context of use with highly qualified samples for a specific context of use.
References:
Figurski MJ, Waligórska T, Toledo J, Vanderstichele H, Korecka M, Lee VM, Trojanowski JQ, Shaw LM, . Improved protocol for measurement of plasma β-amyloid in longitudinal evaluation of Alzheimer's Disease Neuroimaging Initiative study patients. Alzheimers Dement. 2012 Jul;8(4):250-60. PubMed.
Hansson O, Zetterberg H, Vanmechelen E, Vanderstichele H, Andreasson U, Londos E, Wallin A, Minthon L, Blennow K. Evaluation of plasma Abeta(40) and Abeta(42) as predictors of conversion to Alzheimer's disease in patients with mild cognitive impairment. Neurobiol Aging. 2010 Mar;31(3):357-67. Epub 2008 May 19 PubMed.
Hansson O, Batrla R, Brix B, Carrillo MC, Corradini V, Edelmayer RM, Esquivel RN, Hall C, Lawson J, Bastard NL, Molinuevo JL, Nisenbaum LK, Rutz S, Salamone SJ, Teunissen CE, Traynham C, Umek RM, Vanderstichele H, Vandijck M, Wahl S, Weber CJ, Zetterberg H, Blennow K. The Alzheimer's Association international guidelines for handling of cerebrospinal fluid for routine clinical measurements of amyloid β and tau. Alzheimers Dement. 2021 Mar 31; PubMed.
Lachno DR, Vanderstichele H, De Groote G, Kostanjevecki V, De Meyer G, Siemers ER, Willey MB, Bourdage JS, Konrad RJ, Dean RA. The influence of matrix type, diurnal rhythm and sample collection and processing on the measurement of plasma beta-amyloid isoforms using the INNO-BIA plasma Abeta forms multiplex assay. J Nutr Health Aging. 2009 Mar;13(3):220-5. PubMed.
Lachno DR, Emerson JK, Vanderstichele H, Gonzales C, Martényi F, Konrad RJ, Talbot JA, Lowe SL, Oefinger PE, Dean RA. Validation of a Multiplex Assay for Simultaneous Quantification of Amyloid-β Peptide Species in Human Plasma with Utility for Measurements in Studies of Alzheimer's Disease Therapeutics. J Alzheimers Dis. 2012 Aug 9; PubMed.
Lachno DR, Evert BA, Vanderstichele H, Robertson M, Demattos RB, Konrad RJ, Talbot JA, Racke MM, Dean RA. Validation of Assays for Measurement of Amyloid-β Peptides in Cerebrospinal Fluid and Plasma Specimens from Patients with Alzheimer's Disease Treated with Solanezumab. J Alzheimers Dis. 2013 Jan 1;34(4):897-910. PubMed.
Vanderstichele HM, Teunissen CE, Vanmechelen E. Critical Steps to be Taken into Consideration Before Quantification of β-Amyloid and Tau Isoforms in Blood can be Implemented in a Clinical Environment. Neurol Ther. 2019 Dec;8(Suppl 2):129-145. Epub 2019 Dec 12 PubMed.
View all comments by Hugo VandersticheleRoche
This publication references Aβ 42/40 prototype assays currently in feasibility testing as potential blood-based biomarkers for AD within Roche Diagnostics; these and other plasma assays are currently available in U.S. and EU for use in a research setting only.
Final decisions around Roche's IVD panel will be made based upon assessments of several parameters, including clinical performance (AUC values) as reported in this publication, and also other parameters not discussed in this publication. Our goal remains to develop an accurate and reliable test that is robust and suitable for a global deployment and use in routine clinical practice.
View all comments by Ivonne SuridjanVU University Medical Center
In the Netherlands, we use Quanterix’s N4PE test. There are practical reasons for that. The Simoa uses a small volume of sample and is high-throughput.
Most importantly, due to its multiplexing option, within N4PE we can simultaneously measure GFAP and NfL. Combining these markers led to an AUC of 0.88 (Verberk et al., 2020), which is comparable to that reported by the best-performing mass spec assays here.
Moreover, GFAP and NfL are related to cognitive decline upon follow-up in cognitively unimpaired individuals; thus we can generate information on several aspects of the disease within just one measurement.
The fact that amyloid levels are decreased only 10-15 percent in AD patients, irrespective of the test used, indicates that measuring amyloid isoforms in plasma with the current state of the art may not give robust results for individual patient decision-making.
For group analyses, such as in trials or cohort studies, other practical aspects that weigh in are the access to mass-spectrometry assay and the cost of it, besides the consideration of the best discrimination in the current selected cohorts.
I was surprised that in this study, all tests had somewhat similar AUCs to detect amyloid positivity, yet the correlations between the values was surprisingly low. This may suggest that even though the tests measure different isoforms of Aβ, these isoforms are all lowered, but not to the same extent in every individual.
References:
Verberk IM, Thijssen E, Koelewijn J, Mauroo K, Vanbrabant J, de Wilde A, Zwan MD, Verfaillie SC, Ossenkoppele R, Barkhof F, van Berckel BN, Scheltens P, van der Flier WM, Stoops E, Vanderstichele HM, Teunissen CE. Combination of plasma amyloid beta(1-42/1-40) and glial fibrillary acidic protein strongly associates with cerebral amyloid pathology. Alzheimers Res Ther. 2020 Sep 28;12(1):118. PubMed.
View all comments by Charlotte TeunissenAlzheimer's Therapeutic Research Institute, University of Southern California
There are many new plasma assays being developed now, and these are to meet the needs of trials that normally would rely on PET or CSF lumbar punctures to obtain biomarker information. A head-to-head comparison of new platforms is helpful, as it allows us to understand the differences in performance in these assays.
The issue typically is that these kinds of studies rely on retrospectively collected samples, and the populations being studied are usually different from what we are interested in looking at.
Right now ACTC is using the Quanterix Simoa platform for plasma, but we are actively looking into the utility of new mass-spectrometry assays for identifying AD biomarkers in asymptomatic AD populations. We have a study that soon will be submitted for review that compared the major mass-spec platforms in our A4 participant cohort. We are also looking into the possibility of using the mass-spec assays and/or new immunoassays in our current studies.
View all comments by Robert RissmanUniversity College London
The advent of accurate plasma biomarkers for Alzheimer’s disease has been one of the most exciting, and perhaps unexpected, developments in dementia research in the past few years. Moving away from reports exploring single assays, studies such as this, which perform direct head-to-head comparisons of multiple different techniques from different laboratories, are particularly important as the field looks to move from research to clinical application.
While all the plasma Aβ assays compared in this study distinguished amyloid-positive from amyloid-negative individuals with reasonable accuracy, mass-spectrometry techniques outperformed immunoassays—a finding consistent with previous studies including our work in the 1946 British Birth cohort (Keshavan et al., 2021).
While there are good theoretical reasons for quantifying Aβ in plasma directly, an alternative approach is to measure one of the many emerging immune-based assays of phosphorylated tau (Alawode et al., 2021). In Janelidze et al., it is notable that, in a subset of individuals where these techniques were compared directly, mass spectrometry measurement of Aβ42/40 numerically but not statistically outperformed plasma p-tau217 for detecting amyloid positivity.
An important question for clinical translation is whether any increased accuracy but likely additional complexities and costs of performing mass-spectrometry assays at scale outweigh the benefits of slightly less accurate but likely more easily implementable immune-based assays of phosphorylated tau; and also whether the pros/cons of these different approaches differ at different stages of the disease or in different, more diverse populations.
References:
Keshavan A, Pannee J, Karikari TK, Rodriguez JL, Ashton NJ, Nicholas JM, Cash DM, Coath W, Lane CA, Parker TD, Lu K, Buchanan SM, Keuss SE, James SN, Murray-Smith H, Wong A, Barnes A, Dickson JC, Heslegrave A, Portelius E, Richards M, Fox NC, Zetterberg H, Blennow K, Schott JM. Population-based blood screening for preclinical Alzheimer's disease in a British birth cohort at age 70. Brain. 2021 Mar 3;144(2):434-449. PubMed.
Alawode DO, Heslegrave AJ, Ashton NJ, Karikari TK, Simrén J, Montoliu-Gaya L, Pannee J, O Connor A, Weston PS, Lantero-Rodriguez J, Keshavan A, Snellman A, Gobom J, Paterson RW, Schott JM, Blennow K, Fox NC, Zetterberg H. Transitioning from cerebrospinal fluid to blood tests to facilitate diagnosis and disease monitoring in Alzheimer's disease. J Intern Med. 2021 Sep;290(3):583-601. Epub 2021 Jun 26 PubMed.
View all comments by Jonathan SchottRadboud University Nijmegen Medical Centre
Technological developments in recent years have allowed the quantification of Aβ peptides in plasma. Many different assays for the quantification of Aβ40 and Aβ42 have been developed since. When proven to be equally accurate in classifying Alzheimer’s disease patients as the currently available cerebrospinal fluid assays, these latter analyses could be replaced by quantification in plasma, which is more readily obtained from patients. A growing number of assays for the quantification of Aβ40 and Aβ42 have been developed, and in their manuscript, Janelidze and colleagues set out to compare the diagnostic accuracy of eight of these assays to discriminate “Aβ-positive” from “Aβ-negative” persons.
There is, however, in their manuscript, very little discussion of the biological meaning of the findings obtained using the eight assays, whereas several of their observations raise questions with respect to implications for this biological meaning.
First, figures D-F in the manuscript clearly show that correlation between CSF and plasma Aβ40 and Aβ42 assays is very poor (with r-values ranging from as low as 0.078 to a maximum of 0.28). This means that the percentage variance in CSF Aβ40 or Aβ42 that is explained by variance in plasma levels is maximally 8 percent, and mostly even lower for the majority of the eight assays. Whereas CSF Aβ42 correlates very well to signals on amyloid-PET scans and is relatively well associated with the clinical diagnosis of AD, these new findings suggest no relation between plasma and CSF Aβ40 and Aβ42, implying that plasma Aβ40 and Aβ42 levels are mainly derived from extracerebral sources and represent extracerebral rather than cerebral (pathological) biological mechanisms.
Second, the head-to-head comparisons of the eight plasma Aβ40 and Aβ42 assays also clearly show that the correlations between the various assays is low to very low (typically between 0.20 and 0.75; see table in Janelidze et al.). Although each of the manufacturers individually claims to quantify either Aβ40 or Aβ42 using their assays, these data show that the eight different plasma assays either detect different (combinations) of Aβ peptides or are differentially sensitive to pre-analytical, analytical, or biological variations.
Third, a closer look at the data obtained in the BIOFINDER and ADNI cohorts unravels some interesting, but as yet unexplained, inconsistencies (see Table below). Whereas one may expect some degree of variation in the absolute concentrations of compounds measured using the same assay in different cohorts, the results that were reported in eTables 5 and 9 show remarkably unexplained results. Whereas the IPMS-UGOT method reports a consistent, small (<20 percent) difference in mean levels of both plasma Aβ40 and Aβ42 in BIOFINDER vs. ADNI, the observed differences obtained with the other assays (IA-Elc, IA-N4PE for both Aβ40 and Aβ42, and IA-Quant for Aβ42) are >35 percent, and these are extremely discrepant for the IPMS-Shim assay. Moreover, within both the BIOFINDER and ADNI studies, four of six assays consistently yielded lower Aβ42 than Aβ40 concentrations. However, one assay (IP-WashU) yielded 1.3* higher Aβ42 than Aβ40 levels in ADNI, but reverse findings in BIOFINDER; Aβ42 levels were a factor 8 lower than Aβ40. Yet another assay, IA-N4PE, yielded slightly higher Aβ42 (factor 1.1) than Aβ40 levels in BIOFINDER, but inconsistently 25 times lower Aβ42 than Aβ40 levels in ADNI.
In conclusion, these data imply that much is still to be learned about the robustness of the assays, about which Aβ species are exactly quantified by the respective plasma assays, and what are the exact characteristics of each of the assays, before conclusions can be drawn on their clinical utility .
Table: Comparison of the measured plasma amyloid beta 40 and 42 levels in two different cohorts (BIOFINDER and ADNI) using the same quantification assays, for Aβ-negative cases only.
Numbers are copied from eTables 5 and 8 from Janelidze et al. Concentrations are in pg/ml except for Aβ42IPMS-Shim and Aβ40IPMS-Shim. For ease of comparison, only mean levels are shown. Outliers are depicted in red, results that deviate from the general picture are depicted in orange.
View all comments by Marcel VerbeekAraclon Biotech Ltd.
This head-to-head comparison confirms, generally speaking, the superior performance of MS-based assays relative to immunoassays. It is, however, important to evaluate the data of the different sub-cohorts studied here separately, as the absence of some assays in the whole cohort complicates interpretation of the results, as also stated by Hugo Vanderstichele. Results from different sub-cohorts are not directly comparable.
In addition, regarding biomarkers in which many unknowns remain about their biology, it has been largely demonstrated that there is a need for comprehensive standardization (from sample management to statistical analysis of the data) of this type of study in order to have the possibility of reaching valuable and extendable results.
A direct comparison between the best-performing methods is only possible in the IP-MS-Shim sub-cohort. It shows no statistical differences between WashU and Shimadzu, nor between Shimadzu and Araclon (see Table 2).
Our shared goal of integrating these blood biomarkers in routine clinical practice as a non-invasive tool for the most accurate stratification or even diagnostic applications, requires us to use the most accurate classification models. As is so often the case in AD, we need to take into account the contribution of genetic and demographic variables that are individual-dependent. Accordingly, the inclusion of APOE in the model resulted in no significant differences between the three best-performing methods, that is, WashU, Shimadzu, and Araclon, while immunoassays, including Elecsys, remained statistically inferior to the WashU assay.
To illustrate this point, see data from eTable7. It shows ROC analysis combined with APOE ε4 genotype for abnormal CSF Aβ42 and Aβ40 in BioFINDER; Sub-cohort with Aβ42/Aβ40IPMS-Shim, Aβ+/Aβ-, n 86/113:
Aβ42/Aβ40IPMS-WashU 0.902 [0.861-0.944]
Aβ42/Aβ40IPMS-Shim 0.868 [0.819-0.918]
Aβ42/Aβ40LCMS-Arc 0.863 [0.812-0.913]
Aβ42/Aβ40IA-Elc 0.834 [0.778-0.889] b
Aβ42/Aβ40IA-EI 0.816 [0.757-0.875] b
Aβ42/Aβ40IA-N4PE 0.798 [0.736-0.861] b
b: p<0.01 compared with Aβ42/Aβ40IPMS-WashU
It is true that MS-based methods present some practical limitations that hinder their accessibility in clinical trials or in routine clinical practice, such us their high complexity and cost, and limited throughput. That said, the novel MS method developed by Araclon Biotech mostly solves these drawbacks. Aβ peptides are directly extracted from plasma, with no immunoprecipitation nor digestion steps. This significantly reduces time and cost, while maintaining clinical performance in the top of the ranking, as has been demonstrated in several recent publications (Cullen et al., 2021; Janelidze et al., 2021).
Moreover, new data have been generated with an improved version of the Araclon MS assay in a large DPUK-KOREAN cohort, reaching AUCs of 0.91 for the model including Aβ ratio as well as other variables (Jang et al., Performance of plasma Aβ42/Aβ40 ratio, measured with a novel HPLC-MS/MS method, as a biomarker of amyloid-PET status in a DPUK-KOREAN cohort. Accepted in Alzheimer’s Research & Therapy).
Additional manuscripts supporting the clinical performance of Araclon’s MS-based method within different populations and diverse origins are in preparation. We continue working toward the goal of achieving a reliable and reproducible MS-based assay that is capable of accurately predicting β amyloid deposition in the brain.
To note, the test is currently being offered as a service for research use only through Araclon's ISO-certified central laboratory.
References:
Cullen NC, Leuzy A, Palmqvist S, Janelidze S, Stomrud E, Pesini P, Sarasa L, Allué JA, Proctor NK, Zetterberg H, Dage JL, Blennow K, Mattsson-Carlgren N, Hansson O. Individualized prognosis of cognitive decline and dementia in mild cognitive impairment based on plasma biomarker combinations. Nat Aging 1, 114–123 (2021).
Janelidze S, Palmqvist S, Leuzy A, Stomrud E, Verberk IM, Zetterberg H, Ashton NJ, Pesini P, Sarasa L, Allué JA, Teunissen CE, Dage JL, Blennow K, Mattsson-Carlgren N, Hansson O. Detecting amyloid positivity in early Alzheimer's disease using combinations of plasma Aβ42/Aβ40 and p-tau. Alzheimers Dement. 2021 Jun 20; PubMed.
View all comments by Sergio CastilloLund University
Lund University
Washington University School of Medicine
We would like to thank Dr. Marcel Verbeek for a thorough review of our manuscript and data. We believe that one important finding of our study is the consistency in the plasma Aβ42/Aβ40 ratio data (Table 1 and Table 3 in the published paper) between the BioFINDER and ADNI cohorts (see also table below).
With respect to the plasma Aβ42 and Aβ40 concentrations reported in supplementary eTables 5 and 8, we found calculation errors in the case of WashU and Shimadzu assays and we have sent corrections to the publishing journal. These calculation errors did not impact the Aβ42/Aβ40 ratios in the study, or the conclusions of the study, but they did impact the calculated concentrations of WashU and Shimadzu Aβ42 and Aβ40.
We would also like to address some of the other issues raised by Dr. Verbeek. We agree that the weak correlations between CSF and plasma Aβ42 and Aβ40 indicate that peripheral levels of these peptides might be to a large extent affected by production (and possibly degradation) outside the central nervous system. However, for the best-performing assays, we observed higher correlations between CSF and plasma Aβ42/Aβ40 (R 0.56-0.65), as well as relatively strong associations of the ratio with CSF Aβ and Aβ-PET status (AUC 0.81-0.87).
These findings suggest that plasma Aβ42/Aβ40 does to a large extent reflect intracerebral Aβ pathology (and that the effects of peripheral Aβ pathways might be mitigated when using the ratio).
We also agree with Dr. Verbeek that further research is needed to improve and optimize existing plasma Aβ assays. We hope that our study comparing the performance of currently available plasma Aβ assays/platforms would be informative for future work and would help the field to move forward.
Table. Comparison of plasma Aβ42/Aβ40 measured using different assays in Aβ-negative participants between BioFINDER and ADNI.
Make a Comment
To make a comment you must login or register.