Thursday, June 26, 2008

Gene Essence: what bad personal genomics looks like

I mentioned yesterday that one of the companies recently targeted by the California health department was an outfit called Gene Essence. Gene Essence is a 23andMe-style genome scan service launched back at the end of March (covered by Hsien) by Biomarker Pharmaceuticals, a company that aims to develop "genomic and proteomic aging-intervention technologies" to "provide interventions that will extend a healthy human lifespan by slowing the process of aging, and delaying the onset of age-related diseases."

I haven't heard anything about Gene Essence since its launch was announced, so I was curious to see how the service had evolved. The first signs were good: the company is using the Affymetrix 6.0 SNP array, a solid platform that provides information on around 1 million genetic variants throughout the genome - although I can't find any information on where the testing is actually being performed.

Unfortunately, it's all downhill from there. The company thoughtfully provides a demo report allowing customers to see what they'll be getting for their $1,195 - and based on what that shows me I can't imagine anyone purchasing this service, especially given that both 23andMe and deCODEme both offer infinitely superior products at a lower price.

To see just how bad Gene Effects' service is, first check out the 23andMe and deCODEme demo versions, here and here. Now take a look at the demo page where Gene Effects displays your "genetic susceptibility" for a set of different common diseases. Apparently the demo sample has a genetic susceptibility of 100% for atherosclerosis; but does that mean he is 100% likely to get atherosclerosis, that he has the maximum possible genetic risk for the disease, or simply that he has the riskier version all of the known polymorphisms associated with the disease? It's impossible to tell from the page itself - and if you click on the bar for that condition you end up on this absolutely incomprehensible "detail page", which is no help at all.

OK, so let's try reading the manual. The "How to Read This Report" page is unhelpful, but the "Sample Reports" page provides some useful detail - and after a bit of digging it becomes clear that the "genetic susceptibility" score is an indicator of how many of the known risk variants a person carries, scaled by the relative effect of each variant on disease risk. We're supposed to be using the "adjusted trend" column, which "
takes into account the fact that for each condition, a different proportion of the population will have a genetic trend value lower than yours". This doesn't make it any clearer what this actually means in terms of risk prediction; but rather than provide a useful clarification the page goes on to lay out a series of generic disclaimers (e.g. "SNPs are simply markers for the disease or condition and do not necessarily carry any predictive value in terms of assessing one’s susceptibility to a given disease").

In other words, the company provides a series of alarming and confusing predictions, and then simply tells you they don't necessarily mean anything.

There's no estimate of the individual's actual risk of the disease (as you would find in a 23andMe, deCODEme or Navigenics report), no indication of the fraction of the total variance in disease risk that is captured by the polymorphisms in your report, and no reference to whether the described associations are actually reliable and well-validated (for instance, it gives you information on 14 different variants associated with bipolar disease, not one of which has actually been independently replicated). That renders the information essentially meaningless and potentially seriously misleading for the typical customer.

By this stage anyone that Gene Essence somehow convinced to purchase their product would be desperately looking for a way to get some return on their $1,195 purchase. Well, they could always download their raw data and run it through Promethease, which would give them access to the information on each of their genetic variants from the public SNPedia database - except when they click on the "Complete SNP Data" page they'll find no straightforward way to download their entire data-set, but rather a box in which they can type a SNP identification number and download their genotypes one by one. A million of those is going to take some time, even if this feature was actually working (which it isn't, at least for the demo account).

In summary: based on the demo (which is all a potential customer would have to go on) this service is currently seriously bad; I find it hard to imagine that any potential personal genomics customer would pick it over its vastly more professional-looking, rigorous and user-friendly rivals. The fact that Gene Essence actually charges more for its service than 23andMe is utterly absurd. This is an example of what happens when a company tries to jump on the genome scan bandwagon without investing sufficiently in the knowledge-base and user interface required to present extremely complex data to a customer with a limited understanding of genetics.

This hardly inspires any further confidence in the company:
A person who declined to identify herself and who answered the phone at the number listed on Gene Essence's Web site said she didn't know anything about that business or Robert Danielzadeh, identified by the state as its chief executive.
I suggested yesterday that the best outcome in California might be a compromise, in which respectable personal genomics companies are allowed to continue operating (with slightly increased regulatory oversight) while amateurish efforts like Gene Essence slide quietly out of existence. While I don't agree with Steve Murphy that a doctor's explicit permission should be required to authorise a genome scan, there should be a minimum standard of validation and information reporting - and the current version of Gene Essence falls well below that standard.


Subscribe to Genetic Future.

8 comments:

Misha said...

Great picture...

Steve Murphy MD said...

That's the problem,
The government sees all of these as the same. I don't think the ideal situation will emerge. I have been speaking to several on the inside......they are not about to let well capitalized comapnies tell them how to interpret the law...The more they (DTC companies) try to sell, the worse it'll be.

I just hope they can change their business model in order to comply with current laws. THey do have some great people there....I am sure there has got to be at least one redeemable soul at Gene Essence too.....maybe not that guy, but someone.....

-Steve
www.thegenesherpa.blogspot.com
p.s. I stand by my opinion
SNP data not medicine
SNP data PLUS interpretation of medical risk = Medicine (7 days a week, 356 days a year)

Mr. Gunn said...

I don't think anyone is saying there shouldn't be regulations, just that a doctor's referral shouldn't be a blanket requirement for everyone who wants to know what's in their DNA.

It remains unfortunate that Mr. Gene Sherpa lumps bad providers in with good, genetic tests for specific conditions in with SNP profiling, and equates data mining with the practice of medicine.

I don't believe for a second that he's actually got someone "on the inside" who's leaking information to him. That kind of bluster is simply typical of his style of communication, and it doesn't take but a second to find situations where his positions become nonsensical. If SNPs + data analysis = medicine, what is happening when one company provides only data and another company provides only analysis of that data in the context of other profiles?

Which one is the one "practicing medicine", the lab that simply operates the machine, or the website that simply crunches the data?

I don't harbor any enmity towards Mr. Gene Sherpa, but it's quite obvious that he's out of his league here, and it's even more irritating because he's not even interested in a discussion, but rather making these rather foolish ex cathedra pronouncements.

Hugh said...

The notion that SNP data is not medicine is disingenious. Obviously people are motivated to pay for these tests with the motivation of interpreting the health implications. There is no distinction between the two. These tests, like most medical tests should be ordered by, or on the advice of a doctor.

Mr. Gunn said...

Hugh - So leaving aside the idea that people should be able to know what's in their DNA because it's their DNA, would you consider analysis of even non-disease related traits to be practicing medicine?

Would you like to take a stab at answering the questions I posed in my earlier comment, or are you going to take the Gene Sherpa approach of simply ignoring everyone else and stating your position over and over hoping that you'll eventually wear others down.

Hugh said...

Does it matter? The issue here is the regulation of genetic tests and their impact on pathology.

Mr. Gunn said...

Hugh, I'm asking these pointed questions to stimulate discussion. The issue of SNP profiling is an interesting one, because while it certainly has some medical implications, there are many applications which are non-medical. To me, this puts SNP profiling in a different category from DTC genetic tests used for the diagnosis of a medical condition. Some people who are discussing regulations haven't made this distinction due to what I think is a mistaken understanding of the technology. They're comparing it with the next closest thing they know of, which is DTC genetic testing. I'm wondering why someone with a clue doesn't point out to them that the two services are entirely different. I understand that information relating to disease risk is revealed and I understand the need for CLIA certification, validated methods, and protecting the consumer from false advertising, and I don't think anyone worth listening to is saying that there should be no regulation of the industry, simply that a doctor's order shouldn't be necessary, because much of the information imparted by SNP analysis is non-disease related, and a doctor wouldn't know how to interpret even the medically related stuff. Given that even Steve agrees with me on that last point, why is there such insistence on having a doctor's order?

It can't be to protect people from having adverse health events, because clearly knowing more about risk is protective, because knowing things about ancestry can be informative entirely aside from disease risk, and because a doctor wouldn't know what to do with the info anyways.

It can't be to protect people from getting scammed, because the things needed to achieve that, such as requiring the labs to be certified and use validated methods and not make false claims, can all be accomplished without a doctor's involvement.

It can't be, as erroneously claimed by the California regulators, to protect people from getting false information. SNPs aren't all that predictive right now, but the reputable companies aren't making claims beyond what's backed up by solid science(which, again, a GP won't know), so this could only be a valid reason if they had just targeted those who are making false claims, instead of everyone.

What is the reason for the insistence on having a doctor's order? Are they worried that they'll be exposed to malpractice claims if a patient presents them with risk information and they don't take action? Certainly the standards of care still apply, SNPs or no, so what's the problem there?

Like I said, I haven't heard a single person whom I'm convinced understands the technology argue the side of requiring a doctor's order. Let's be clear here: regulation is good for everyone, both the reputable, certified providers and consumers. The question therefore is not why should we have regulation, but why should a doctor's order be required?

John Adam said...

I have been seeing a lot of info on this subject, but none of them including this page doesn't say anything how the companies are comparing the data. Does any one know how the data generated from Affymetrix snp 6.0 is being compared to or which database is being used and where that can be found? Appreciate comments.