Mail-In Genome Tests: Do They Measure Up?

The Food and Drug Administration (FDA) ruffled feathers late last year when it warned 23andMe—a company that sells personal genomic information directly to consumers—to stop providing health assessments until its tests were validated. Since then, consumers and scientists alike have weighed in on whether the FDA ought to regulate such tests. Two recently published commentaries—one in Nature and the other in the New England Journal of Medicine—come out for and against the FDA’s action, respectively. The editorials underscore the uncertainty surrounding new technologies that are commercialized before their consequences are fully appreciated. “Genetic testing is a double-edged sword,” said Murali Doraiswamy, Duke University, Durham, North Carolina. “We have to be extremely careful when disclosing personal genomic information.” 

Until November 22, 2013, a person could mail a tube of saliva to 23andMe in Mountain View, California, and for $99 the company would analyze single nucleotide polymorphisms in the DNA and send back a report on genetic risk for 254 diseases and conditions. However, as outlined in the FDA’s warning letter, the company has never supplied evidence that those tests measure what they claim to measure. An FDA spokeswoman declined to take a call but wrote to Alzforum that the agency is tasked with ensuring that health-related tests sold to the public are accurate—meaning that the measurements themselves are correct, and that companies can support the claims made in their interpretation of their measurements. According to the FDA, the evidence 23andMe provided was insufficient to allay either concern. The company needed to show that the genotype it reports is the one the patient truly has, and back up its health interpretations with its own studies or studies from the literature, wrote the spokeswoman. To protect the public, any diagnostic test—be it for plasma cholesterol or brain amyloid—must receive regulatory approval according to the Federal Food, Drug and Cosmetic Act. 

The FDA’s letter laid out examples of how errors could have serious consequences. A false-positive test for the breast cancer gene BRCA could lead to a needless mastectomy, while a negative one could lull a woman who may be at risk into a false sense of security.  A false-positive result for an SNP that increases sensitivity to the anticoagulant warfarin might prompt people to change their dose without consulting a doctor. In short, the FDA worried that people might react to inaccurate results. Since receiving the FDA’s letter, 23andMe has suspended health-related genetic tests, offering only raw genetic data and genealogical information to new customers. 

A Nature editorial penned January 16 by Robert Green, Brigham and Women’s Hospital, Boston, and Nita Farahany, Duke University, calls the clampdown unwarranted. They cite scientific evidence suggesting that consumer genomics leads neither to undue distress or inappropriate action. For instance, the Scripps Genomic Health Initiative, in a survey of 2,000 people who underwent genetic testing, found little change in anxiety or psychological health a year after people received test results (see Bloss et al., 2011). Similarly, preliminary data from the Impact of Personal Genomics Study, co-led by Green, finds no elevated anxiety or distress in 1,800 people a year after they received their genetic health assessment. This lines up with results from the REVEAL study, also led by Green, which found that people who learn, albeit in a research environment, that they carry an ApoE4 allele and are at risk for Alzheimer’s disease cope well, getting only mildly upset and only for a short time (see Jul 2009 news story).

Green and Farahany claim that most people do not act rashly after receiving personal genetic information. A survey from researchers at John’s Hopkins University found that fewer than 10 of 1,000 customers altered their medications without consulting a doctor (see Kaufman et al., 2012). In a survey by 23andMe, most female responders with a positive BRCA result sought a professional opinion (see Francke et al., 2013). Green and Farahany concede that the field needs to do more research to fully understand how consumers react to genetic information, but they worry that these studies cannot continue if direct-to-consumer genetic testing stops. “We believe that 23andMe should be more transparent about how accurate its genotyping chips are, and even more forthcoming about the limitations of its computer algorithms used for estimations of risk,” they said. “But regulatory constraints might stifle consumer genomics and other emerging products that could make society healthier and that do not fit neatly into the model of physician-driven health care.” 

Other scientists defend the FDA’s stance. In the February 12 New England Journal of Medicine, George Annas, Boston University, and Sherman Elias, Northwestern University, Chicago, maintain that far from depriving people of useful information, the FDA is doing its duty by ensuring that health-related tests sold to the public are safe and effective. Unless it is validated, “genomic information can be misleading—or just plain wrong—and this could cause more harm than good,” they wrote. Annas and Elias wrote that 23andMe originally framed genetic testing as consumer empowerment, giving people direct access to their information without requiring a go-between physician or genetic counselor. “We think the day will come when this framing is appropriate, but not until the diagnostic and prognostic capability of genomic information has been clinically validated,” they said.

Apart from showing validity, 23andMe needs to improve its consent and privacy protection measures, since the company plans to compile and sell data for research, Annas and Elias wrote. It should also allow people to request that certain genes—such as ApoE—not be sequenced, these authors wrote. They suggested that the moratorium will provide an opportunity to discuss these issues and set standards for 23andMe, the genetic testing industry, and future whole-genome-sequencing platforms. 

“Both editorials had good points,” Doraiswamy told Alzforum. “The truth probably lies somewhere in between.” He advocates for widely available genetic testing as long as labs produce high-quality data. However, he and others believe the FDA was right to step in because at the moment, genetic analyses offer limited value to the consumer. 

The Bigger Picture

Genotyping-based predictions may be on shaky ground for other reasons. They are often based on a few, or even just one genetic variant. Most geneticists agree that is fraught with danger. Many diseases, including Alzheimer's, are influenced by a plethora of genetic variants, some yet to be discovered. At this point, algorithms that predict health outcomes rely on the simplistic notion that one gene confers its own discrete risk, said Doraiswamy. “In reality, it’s the particular pattern of thousands of genes interacting together that determines risk,” he said. 

Case in point: Companies use different algorithms that can produce wildly divergent results. Last December in The New York Times, the author Kira Peikoff explained how she had her genome tested at three different direct-to-consumer genomics companies and got back opposite predictions in some cases. The company Genetic Testing Laboratories claimed her lowest risk was for rheumatoid arthritis and psoriasis, while 23andMe claimed her most elevated risk was for those same diseases (see Dec 2013 New York Times article). 

Alison Goate, Washington University in St. Louis, told Alzforum that for accurate, informative genetic testing, researchers have yet to figure out if multiple SNPs associated with a disease exert an additive or multiplicative influence on risk. In addition, SNP data has been collected mainly on white people of European descent. “Using that information to make risk estimates in other races and ethnic groups would be problematic and likely not scientifically sound,” she said. Though the quality of interpretations is improving all the time, at this point a good family history remains a more reliable indicator than individual SNP data, said Goate.

When it comes to Alzheimer’s, the data are especially murky, said Creighton Phelps from the National Institute on Aging, Bethesda, Maryland. “So many things—lifestyle, diet, and environmental exposure to toxins—can protect or lead to development of the disease,” he said. For sporadic AD, genomic screening is not informative enough right now, he said, but once particular risk variants are found to be predictive, not just associated with disease, they may prove more useful.

Even given these limitations, some believe that access to health-related genetic data is a basic right. “People should have access to their own data and its interpretation,” said Jose Bras, University College, London. He submitted a DNA sample to 23andMe a few years ago, and said the results simply added a layer of information to help guide his decisions about health. He now exercises a lot more. While almost all the risk that turned up in his genetic profile matched what he knew from his family history, the test gave him numbers. “I had only a vague notion of my risk before, but now I have a more accurate idea of what the possibilities are,” he said. 

Bras contends that the FDA is demanding the impossible. Validating the hundreds of thousands of SNPs that 23andMe analyzes is not feasible, he argued. It would take too long and require too many resources for one company. Goate agreed, saying that not all of them need to be rigorously tested. “One thing you might see come out of this is that the FDA will specify the subset of the most medically important tests it wants validated,” she said. Both the FDA and 23andMe declined to comment for this article.

Regardless of the short-term outcome, this debate advances discussion on a technology that will soon become commonplace, Goate said. “Clearly this is the wave of the future,” she told Alzforum. Doraiswamy agreed, noting that the field’s current struggles will raise industry standards. “Access to genetic testing information should be increased, but at the same time, it should be balanced by a greater disclosure of its limits and its potential downside,” he said. 

Doraiswamy has encouraged companies to better protect consumers who may become anxious about their results, a group that he believes is bigger than researchers realize because those people do not necessarily come forward on their own. Companies could bundle data for genes such as ApoE with online lectures and genetic counseling to provide more options for people who need follow-up, he suggested. Right now, 23andMe “locks” ApoE status until a customer indicates that he or she has read the limitations of the test and possible dangers of learning the information. “The field is swinging from one extreme to the other, but I think the pendulum will eventually settle in the middle,” said Doraiswamy.—Gwyneth Dickey Zakaib

 

Comments

No Available Comments

Make a Comment

To make a comment you must login or register.

References

News Citations

  1. Early ApoE4 Memory Effects, But Do You Really Want to Know?

Paper Citations

  1. Bloss CS, Schork NJ, Topol EJ. Effect of direct-to-consumer genomewide profiling to assess disease risk. N Engl J Med. 2011 Feb 10;364(6):524-34. Epub 2011 Jan 12 PubMed.
  2. Kaufman DJ, Bollinger JM, Dvoskin RL, Scott JA. Risky business: risk perception and the use of medical services among customers of DTC personal genetic testing. J Genet Couns. 2012 Jun;21(3):413-22. Epub 2012 Jan 26 PubMed.
  3. Francke U, Dijamco C, Kiefer AK, Eriksson N, Moiseff B, Tung JY, Mountain JL. Dealing with the unexpected: consumer responses to direct-access BRCA mutation testing. PeerJ. 2013;1:e8. Print 2013 PubMed.

External Citations

  1. warned
  2. Dec 2013 New York Times article

Further Reading

Papers

  1. Zettler PJ, Sherkow JS, Greely HT. 23andMe, the Food and Drug Administration, and the future of genetic testing. JAMA Intern Med. 2014 Apr;174(4):493-4. PubMed.
  2. Downing NS, Ross JS. Innovation, risk, and patient empowerment: the FDA-mandated withdrawal of 23andMe's Personal Genome Service. JAMA. 2014 Feb 26;311(8):793-4. PubMed.

Primary Papers

  1. Annas GJ, Elias S. 23andMe and the FDA. N Engl J Med. 2014 Mar 13;370(11):985-8. Epub 2014 Feb 12 PubMed.
  2. Green RC, Farahany NA. Regulation: The FDA is overcautious on consumer genomics. Nature. 2014 Jan 16;505(7483):286-7. PubMed.

Annotate

To make an annotation you must Login or Register.