Image via Wikipedia

The FDA has just released its new Draft guidance on clinical trial design for antibacterial drugs in the treatment of hospital-acquired pneumonia (HAP) and ventilator-associated pneumonia (VAP). As anticipated, they call for an endpoint of 28 day all cause mortality. But the devil is in the details. The FDA suggests that patients at high risk of dying (those with APACHE scores of 15 or greater) be enrolled to allow for a treated mortality rate of around 20%. That would allow for a non-inferiority margin of 10%. BUT – the analysis population is the microbiology intent to treat (MITT) population – that is, those patients enrolled and treated who have an identified bacterial pathogen at study entry.

Once again, in the appendix to the guidance, the FDA has gone to extraordinary and unscientific lengths to discount the treatment effect of antibiotics in VAP such that they arrive at an infeasible trial design. They demonstrate that inappropriate therapy for VAP is associated with a 62% mortality and that appropriate therapy is associated with a 20% mortality. The treatment effect of appropriate vs. inappropriate therapy is, therefore, 42%. Of course, inappropriate therapy is not no therapy, and 42% as a treatment effect is therefore already conservative. But, the FDA is not satisfied with that. They go on to apply their 95/95 rule using the upper bound of the 95% confidence interval for treatment effect of inappropriate therapy and the lower bound for appropriate therapy yielding a treatment effect of 29% instead of 42%. They then discount the treatment effect of 29% by an additional 30% for good measure. The FDA justifies their additional 30% discount by claiming that this is necessary to correct for “uncertainties” of the historical database. This brings the treatment effect or M1 down to 20%. How convenient! They now take 50% of that and call that a justified non-inferiority margin of 10%. (Is 10% starting to sound like a familiar number?) But since the margin simply has to be smaller than the treatment effect, the margin could just as easily have been 19%. But of course the entire discounting argument is overly conservative and irrational.

So, as an amateur developer, I’ve been trying to understand the practical consequences of this design. Lets take VAP since the FDA seems to focus on VAP in their guidance. In my calculations I have made the following assumptions:

Cure rate = 80% (20% mortality).

Evaluability – as far as I can tell, in modern trials, the MITT population is about 45% of the enrolled population.

NI margin = 10%

90% power (to exclude the chance of falsely concluding inferiority as much as possible).

I calculate that one would need to enroll over 1200 patients per trial. That would be at least 2400 patients in two trials.

I believe that the largest HAP/VAP trials ever performed were the ATTAIN-1 and -2 trials by Theravance/Astellas. They enrolled 1518 patients over about 28 months. I don’t know how many centers were used. They powered their trials to show a 20% non-inferiority for treatment of MRSA infection compared to vancomycin. Non-inferiority margins of 15-20% used to be common for VAP trials since these patients are so hard and so slow to enroll and since the trials are so expensive to run. Those were the days when we actually considered feasibility of trial designs before we issued guidance. Two trials at 1200 patients per trial in VAP is, once again, totally infeasible. Even just in terms of the time that would be required to complete such a trial.

I calculate, based on recent enrollment rates as cited by Steve Barriere of Theravance, that such a trial could take anywhere from 5 to 50 years to complete. In the current FDA guideline, the suggested severity of illness (APACHE score > 15) will slow enrollment. The number of centers participating in the trial and, as a corollary, the number of competing trials will all affect enrollment rates and costs. After even 5 years, the trial design and comparator might already be obsolete.

In terms of cost, VAP trials are probably the most expensive trials we currently undertake. I’m guess that costs now are in the order of $30-50,000 or more per enrolled patient. If the analysis population will now be the MITT population, these costs could be even higher. Frank Tally estimated that in Cubist’s failed (for lack of enrollment) trial of daptomycin in the treament of bacteremia caused by vancomycin-resistant Enterococcus, their costs were $250,000 per evaluable patient. They spent $20 million before jettisoning the trial. At $50,000 per enrolled patient, a 2400 patient set of trials in VAP would cost $120 million. No company, let me repeat that, NO COMPANY will take this on! The costs rapidly outstrip any return on investment.

What about superiority trials as a solution? We’ll address that in a future blog.

Have a great day!

Footnote: As far as I can tell, Theravance never published the full results of their trials. I can only find abstracts and a very brief and incomplete summary of their results on clintrials.gov. Now that it is clear what the FDA wants – will Theravance be able to provide it?