A quick update on my last post regarding the enormously controversial -- but completely unmentioned -- requirement to publicly report all versions of clinical trial protocols on ClinicalTrials.gov: The New England Journal of Medicine has weighed in with an editorial strongly in support of the TEST Act.
NEJM Editor-in-Chief
Jeffrey Drazen at least mentions the supporting documents requirement,
but only in part of one sentence, where he confusingly refers to the act "extending results reporting to include the deposition of consent and protocol documents approved by institutional review boards." The word "deposition" does not suggest actual publication, which the act clearly requires.
I don't think this qualifies as an
improvement in transparency about the impact the TEST Act, as written, would have. I'm not surprised when a trade publication like Center Watch recycles a press release into a news item. However, it wouldn't seem like too much to ask that NEJM editorials aspire to a moderately higher standard of critical inquiry.
Wednesday, August 8, 2012
Monday, August 6, 2012
Public Protocols? Burying the lede on the TEST Act
 |
| Not to be confused with the Test Act. (via Luminarium) |
Which is unfortunate, because nowhere in that release is there a single mention of the bill’s most controversial feature: publication of clinical trial "supporting documents", including the patient’s Informed Consent Form (ICF) and, incredibly, the entire protocol (including any and all subsequent amendments to the protocol).
How Rep. Markey and colleagues managed to put out a 1,000-word press release without mentioning this detail is nothing short of remarkable. Is the intent to try to sneak this through?
Full public posting of every clinical trial protocol would represent an enormous shift in how R&D is conducted in this country (and, therefore, in the entire world). It would radically alter the dynamics of how pharmaceutical companies operate by ripping out a giant chunk of every company’s proprietary investment – essentially, confiscating and nationalizing their intellectual property.
Maybe, ultimately, that would be a good thing. But that’s by no means clear ... and quite likely not true. Either way, however, this is not the kind of thing you bury in legislation and hope no one notices.
[Full text of the bill is here (PDF).]
[UPDATE May 17, 2013: Apparently, the irony of not being transparent with the contents of your transparency law was just too delicious to pass up, as Markey and his co-sponsors reintroduced the bill yesterday. Once again, the updated press release makes no mention of the protocol requirement.]
Tuesday, July 31, 2012
Clouding the Debate on Clinical Trials: Pediatric Edition
I would like to propose a rule for clinical trial benchmarks. This rule may appear so blindingly obvious that I run the risk of seeming simple-minded and naïve for even bringing it up.
The rule is this: if you’re going to introduce a benchmark for clinical trial design or conduct, explain its value.
That’s it. Just a paragraph explaining the rationale of why you’ve chosen to measure what you’re measuring. Extra credit if you compare it to other benchmarks you could have used, or consider the limitations of your new metric.
I would feel bad for bringing this up, were it not for two recent articles in major publications that completely fail to live up to this standard. I’ll cover one today and one tomorrow.
The first is a recent article in Pediatrics, Pediatric Versus Adult Drug Trials for Conditions With High Pediatric Disease Burden, which has received a fair bit of attention in the industry -- mostly due to Reuters uncritically recycling the authors’ press release.
It’s worth noting that the claim made in the release title, "Drug safety and efficacy in children is rarely addressed in drug trials for major diseases", is not at all supported by any data in the study itself. However, I suppose I can live with misleading PR. What is frustrating is the inadequacy of the measures the authors use in the actual study, and the complete lack of discussion about them.
To benchmark where pediatric drug research should be, they use the proportion of total "burden of disease" borne by children. Using WHO estimates, they look at the ratio of burden (measured, essentially, in years of total disability) between children and adults. This burden is further divided into high-income countries and low/middle-income countries.
This has some surface plausibility, but presents a host of issues. Simply looking at the relative prevalence of a condition does not really give us any insights into what we need to study about treatment. For example: number 2 on the list for middle/low income diseases is diarrheal illness, where WHO lists the burden of disease as 90% pediatric. There is no question that diarrheal diseases take a terrible toll on children in developing countries. We absolutely need to focus resources on improving prevention and treatment: what we do not particularly need is more clinical trials. As the very first bullet on the WHO fact sheet points out, diarrheal diseases are preventable and treatable. Prevention is mostly about improving the quality of water and food supplies – this is vitally important stuff, but it has nothing to do with pharmaceutical R&D.
In the US, the NIH’s National Institute for Child Health and Human Development (NICHD) has a rigorous process for identifying and prioritizing needs for pediatric drug development, as mandated by the BPCA. It is worth noting that only 2 of the top 5 diseases in the Pediatrics article make the cut among the 41 highest-priority areas in the NICHD’s list for 2011.
(I don’t even think the numbers as calculated by the authors are even convincing on their own terms: 3 of the 5 "high burden" diseases in wealthy countries – bipolar, depression, and schizophrenia – are extremely rare in very young children, and only make this list because of their increasing incidence in adolescence. If our objective is to focus on how these drugs may work differently in developing children, then why wouldn’t we put greater emphasis on the youngest cohorts?)
Of course, just because a new benchmark is at odds with other benchmarks doesn’t necessarily mean that it’s wrong. But it does mean that the benchmark requires some rigorous vetting before its used. The authors make no attempt at explaining why we should use their metric, except to say it’s "apt". The only support provided is a pair of footnotes – one of those, ironically, is to this article from 1999 that contains a direct warning against their approach:
Are we not putting enough resources into pediatric research, or have we over-incentivized risky experimentation on a vulnerable population? This is a critically important question in desperate need of more data and thoughtful analysis. Unfortunately, this study adds more noise than insight to the debate.
Tomorrow In a couple weeks, I’ll cover the allegations about too many trials being too small. [Update: "tomorrow" took a little longer than expected. Follow up post is here.]
[Note: the Pediatrics article also uses another metric, "Percentage of Trials that Are Pediatric", that is used as a proxy for amount of research effort being done. For space reasons, I’m not going to go into that one, but it’s every bit as unhelpful as the pediatric burden metric.]
Bourgeois FT, Murthy S, Pinto C, Olson KL, Ioannidis JP, & Mandl KD (2012). Pediatric Versus Adult Drug Trials for Conditions With High Pediatric Disease Burden. Pediatrics PMID: 22826574
The rule is this: if you’re going to introduce a benchmark for clinical trial design or conduct, explain its value.
| Are we not putting enough resources into pediatric research, or have we over-incentivized risky experimentation on a vulnerable population? This is a critically important question in desperate need of more data and thoughtful analysis. |
I would feel bad for bringing this up, were it not for two recent articles in major publications that completely fail to live up to this standard. I’ll cover one today and one tomorrow.
The first is a recent article in Pediatrics, Pediatric Versus Adult Drug Trials for Conditions With High Pediatric Disease Burden, which has received a fair bit of attention in the industry -- mostly due to Reuters uncritically recycling the authors’ press release.
It’s worth noting that the claim made in the release title, "Drug safety and efficacy in children is rarely addressed in drug trials for major diseases", is not at all supported by any data in the study itself. However, I suppose I can live with misleading PR. What is frustrating is the inadequacy of the measures the authors use in the actual study, and the complete lack of discussion about them.
To benchmark where pediatric drug research should be, they use the proportion of total "burden of disease" borne by children. Using WHO estimates, they look at the ratio of burden (measured, essentially, in years of total disability) between children and adults. This burden is further divided into high-income countries and low/middle-income countries.
This has some surface plausibility, but presents a host of issues. Simply looking at the relative prevalence of a condition does not really give us any insights into what we need to study about treatment. For example: number 2 on the list for middle/low income diseases is diarrheal illness, where WHO lists the burden of disease as 90% pediatric. There is no question that diarrheal diseases take a terrible toll on children in developing countries. We absolutely need to focus resources on improving prevention and treatment: what we do not particularly need is more clinical trials. As the very first bullet on the WHO fact sheet points out, diarrheal diseases are preventable and treatable. Prevention is mostly about improving the quality of water and food supplies – this is vitally important stuff, but it has nothing to do with pharmaceutical R&D.
In the US, the NIH’s National Institute for Child Health and Human Development (NICHD) has a rigorous process for identifying and prioritizing needs for pediatric drug development, as mandated by the BPCA. It is worth noting that only 2 of the top 5 diseases in the Pediatrics article make the cut among the 41 highest-priority areas in the NICHD’s list for 2011.
(I don’t even think the numbers as calculated by the authors are even convincing on their own terms: 3 of the 5 "high burden" diseases in wealthy countries – bipolar, depression, and schizophrenia – are extremely rare in very young children, and only make this list because of their increasing incidence in adolescence. If our objective is to focus on how these drugs may work differently in developing children, then why wouldn’t we put greater emphasis on the youngest cohorts?)
Of course, just because a new benchmark is at odds with other benchmarks doesn’t necessarily mean that it’s wrong. But it does mean that the benchmark requires some rigorous vetting before its used. The authors make no attempt at explaining why we should use their metric, except to say it’s "apt". The only support provided is a pair of footnotes – one of those, ironically, is to this article from 1999 that contains a direct warning against their approach:
Our data demonstrate how policy makers could be misled by using a single measure of the burden of disease, because the ranking of diseases according to their burden varies with the different measures used.If we’re going to make any progress in solving the problems in drug development – and I think we have a number of problems that need solving – we have got to start raising our standards for our own metrics.
Are we not putting enough resources into pediatric research, or have we over-incentivized risky experimentation on a vulnerable population? This is a critically important question in desperate need of more data and thoughtful analysis. Unfortunately, this study adds more noise than insight to the debate.
[Note: the Pediatrics article also uses another metric, "Percentage of Trials that Are Pediatric", that is used as a proxy for amount of research effort being done. For space reasons, I’m not going to go into that one, but it’s every bit as unhelpful as the pediatric burden metric.]
Bourgeois FT, Murthy S, Pinto C, Olson KL, Ioannidis JP, & Mandl KD (2012). Pediatric Versus Adult Drug Trials for Conditions With High Pediatric Disease Burden. Pediatrics PMID: 22826574
Subscribe to:
Posts (Atom)