REPORT HIGHLIGHTS

DRUG SAFETY: Improvement Needed in FDA’s Postmarket Decision-making and
Oversight Process

GAO conducted an organizational review and case studies of four drugs with safety issues: Arava, Rezulin, Bextra, and Propulsid.

Bextra
There was sometimes a lack of consensus in our drug case studies, and we observed that ODS often performed a series of related analyses about the same safety concerns for OND over a significant period of time. As an illustration of this iterative decision-making process, OND requested in 2002 that ODS analyze cases of serious skin reactions associated with the pain reliever Bextra after the drug’s sponsor had communicated with OND about this potential risk. ODS staff searched the AERS database and found several related cases for review.

They estimated the occurrence of reported cases of serious skin reactions among Bextra users by using the cases and drug utilization data. On the basis of their analysis, ODS recommended that Bextra’s label be updated to include this risk, and OND followed the recommendation by working with the sponsor to update the label in 2002. Between 2002 and 2004, ODS staff conducted five other analyses of the occurrence of serious skin reactions associated with Bextra, including two that were requested by OND. In March 2004, ODS staff recommended that Bextra carry a boxed warning about its risks of serious skin reactions. The ODS staff based their recommendation on their finding that Bextra’s risk for serious skin reactions was 8 to 13 times higher than that for other similar drugs and 20 times higher than the incidence rate in the population. The ODS Division Directors who reviewed the analysis and recommendation agreed, but the OND review division responsible for Bextra did not initially agree. About 5 months later, the OND review division decided a boxed warning was warranted, after ODS performed another analysis requested by OND, comparing Bextra’s risk with several other similar drugs, including Mobic. ODS found no reported cases of serious skin reactions associated with Mobic. In 2005, a joint meeting of FDA’s Arthritis Advisory Committee and DSaRM was held to discuss the postmarket safety of several anti-inflammatory drugs including Bextra, with a focus on their cardiovascular risks. The committees recommended, after presentations by FDA staff and others, that Bextra should remain on the market. A few months later, FDA asked the sponsor to withdraw the drug from the market because, in part, its risk for serious skin reactions appeared to be greater than for other similar anti-inflammatory drugs.

FDA’s postmarket drug safety decision-making process has been limited by a lack of clarity, insufficient oversight by management, and data constraints. We observed that there is a lack of established criteria for determining what safety actions to take and when. Aspects of ODS’s role in the process are unclear, including its role in participating in scientific advisory committee meetings organized by OND. A lack of communication between ODS and OND’s review divisions and limited oversight of postmarket drug safety issues by ODS management has hindered the decision-making process. FDA relies primarily on three types of data sources—adverse event reports, clinical trial studies, and observational studies—in its postmarket decision making. Each data source has

While acknowledging the complexity of the postmarket drug safety decision-making process, we observed in our interviews with OND and ODS staff and in our case studies that the process lacked clarity about how drug safety decisions are made and about the role of ODS. If FDA had established criteria for certain postmarket drug safety decisions, then some of the disagreements we observed in our case studies could have possibly been resolved more quickly. For example, in the case of Bextra, as described earlier, ODS and OND staff disagreed about whether the degree of risk warranted a boxed warning, the most serious warning placed in the labeling of a prescription medication.

Propulsid
As another example, there were differing opinions over taking stronger actions against Propulsid, the nighttime heartburn medication which was associated with cardiovascular side effects, or whether to modify the label. Between 1995 and 1997, Propulsid’s label had been modified, including the addition of a boxed warning, to warn consumers and professionals about the cardiovascular side effects of the drug. In June 1997 a task force within FDA, including OND and ODS staff, was convened to further evaluate the efficacy and safety of Propulsid. FDA staff, including task force members, later met to discuss several regulatory options, including proposing further label modifications, presenting the agency’s concerns to an advisory committee, and proposing to withdraw approval of Propulsid. According to a former OND manager, as a result of this meeting, FDA decided to seek further label modifications. Some staff, from both OND and ODS, however, supported stronger actions at this time, including proceeding with proposing a withdrawal of approval. According to several FDA officials, in the absence of established criteria, decisions about safety actions are often based on the case-by-case judgments of the individuals reviewing the data.

Our observations are consistent with previous FDA reviews. In 2000, two internal CDER reports based on interviews that FDA conducted with staff indicated that an absence of established criteria for determining what safety actions to take, and when, posed a challenge for making postmarket drug safety decisions. The reports recognized the need to establish criteria to help guide such decisions. In a review of the safety issues concerning Propulsid, CDER staff recommended that a standardized approach to postmarket drug safety issues be established, by addressing various issues such as how to determine when to incorporate safety issues into labeling and when stronger actions should supersede further labeling changes. According to the report, several staff noted frustration with the numerous changes made to Propulsid’s label that were mostly ineffective in reducing the number of cardiovascular adverse events.

Rezulin
Similarly, after the diabetes drug Rezulin was removed from the market in 2000 because of its risk for liver toxicity, a CDER report focused on Rezulin also recommended that a consistent approach to postmarket drug safety be developed, including what regulatory actions should occur to address postmarket drug safety concerns, and when they should occur.

Arava
In addition to a lack of criteria for safety actions, we observed a lack of clarity related to ODS’s recommendations. In practice, ODS often makes written recommendations about safety actions to OND but there is some confusion over this role, according to several ODS managers, and there is no policy that explicitly states whether ODS’s role includes this responsibility. The case of Arava illustrates this confusion. In 2002, the OND review division responsible for Arava, a drug used to treat rheumatoid arthritis, requested that ODS review postmarket data for cases of serious liver toxicity associated with its use. The ODS staff who worked on this analysis recommended that Arava be withdrawn from the market because they concluded that the risk for serious liver toxicity exceeded its benefits. The OND Division Director responsible for Arava felt that ODS should not have included a recommendation in its consult because he argued that this was the responsibility of OND, not ODS. Some of the confusion may be the result of ODS’s evolving role in postmarket drug safety. A current and a former ODS manager told us that in the past, ODS’s safety consults were technical documents summarizing adverse events with minimal data analysis and few recommendations. Over time the consults have become more detailed with sophisticated data analyses and more recommendations about what safety action is needed (for example, label change, medication guide, drug withdrawal).

One staff member noted that the numerous labeling changes made it increasingly difficult to use Propulsid as labeled because of the numerous contraindications.

ODS’s role in scientific advisory committee meetings is also unclear. According to the OND Director, OND is responsible for setting the agenda for the advisory committee meetings, with the exception of DSaRM. This includes who is to present and what issues will be discussed by the advisory committees. For the advisory committees (other than DSaRM) it is unclear when ODS staff will participate. While ODS staff have presented their postmarket drug safety analyses during some advisory committee meetings, our case study of Arava, and another case involving antidepressant drugs, provide examples of the exclusion of ODS staff. For example, in March 2003, the Arthritis Advisory Committee met to review the efficacy of Arava, and its safety in the context of all available drugs to treat rheumatoid arthritis. The OND review division responsible for Arava presented its own analysis of postmarket drug safety data at the meeting, but did not allow the ODS staff—who had recommended that Arava be removed from the market—to present their analysis because it felt that ODS’s review did not have scientific merit. Specifically, the OND review division felt that some of the cases in the ODS review did not meet the definition of acute liver failure, the safety issue on which the review was focused. The OND division also believed that in some of the cases ODS staff inappropriately concluded that liver failure resulted from exposure to Arava. After the meeting, ODS epidemiologists and safety evaluators asked the ODS and OPaSS Directors to clarify ODS’s role involving postmarket drug safety issues, including its role at advisory committee meetings. According to an FDA official, there was no written response to this request.

As another example of ODS’s unclear role in scientific advisory committees, in February 2004 an ODS epidemiologist was not allowed to present his analysis of safety data at a joint meeting of the Psychopharmacologic Drugs Advisory Committee and the Pediatric Subcommittee of the Anti-Infective Drugs Advisory Committee that was held to discuss reports of suicidal thoughts and actions in children with major depressive disorder during clinical trials for various antidepressant drugs. According to statements by FDA officials at a congressional hearing, OND believed that the ODS staff member’s analysis, which showed a relationship between the use of antidepressants and suicidal thoughts and behaviors in children, was too preliminary to be presented in detail. The analysis was based on pediatric clinical trial data that FDA requested from the sponsors of several antidepressant drugs. FDA had asked the sponsors to identify suicide-related events using specific methods, and then ODS was asked to analyze all of the submitted data. OND later decided that the sponsors may have been inconsistent in their classification approaches and asked outside experts to perform additional reviews of all the cases by rating whether particular events could be classified as suicidal. The staff member who performed the ODS review, however, believed that the available data were sufficient to conclude a relationship between the use of antidepressants and suicidal thoughts and behaviors in pediatrics and to recommend further safety actions. In his consult, the ODS staff member also concluded that while additional analyses would yield valuable information, they would also take several more months to complete. In light of this delay, he recommended an interim plan to discourage the use of all but one antidepressant in the treatment of pediatric major depressive disorders. In December 2004, ODS epidemiologists communicated to the CDER Director their position that ODS’s role should include the responsibility of presenting all relevant ODS data at advisory committee meetings. According to an FDA official, there was no written response to this request. However, in our interviews, the Directors of CDER and OND told us that in retrospect they felt it was a mistake for FDA to have restricted the ODS epidemiologist from presenting his safety information at the meeting.

Several ODS managers that we interviewed told us that there is also a lack of clarity regarding the role of the epidemiologist in postmarket drug safety work. Despite the fact that ODS’s epidemiologists have some defined responsibilities, there appears to be some confusion about the scope of their activities and a lack of understanding on the part of OND about their role and capabilities. A prior review of postmarket drug safety identified similar issues. For example, in that review some epidemiologists indicated that they should be able to maintain an independent approach to their research and the publication of their research. However, some OND review division directors indicated that the work of the epidemiologists should be considered within the context of CDER’s overall regulatory mission. Further, the epidemiologists’ research conclusions do not necessarily reflect the conclusions of FDA but may be perceived as such by the medical community. ODS managers indicated that a current challenge for FDA is to determine how it should use its epidemiologists and what their work products should be. According to the current ODS Director, efforts are needed to help OND better understand what epidemiologists can do. The epidemiologists themselves have asked for greater clarity about their role and a stronger voice in decision making.

A lack of communication between ODS and OND’s review divisions and limited oversight of postmarket drug safety issues by ODS management have also hindered the decision-making process. The frequency and extent of communication between ODS and OND’s divisions on postmarket drug safety vary. ODS and OND staff often described their relationship with each other as generally collaborative, with effective communication. But both ODS and OND staff said sometimes there were communication problems, and this has been an ongoing concern. For example, according to some current and former ODS staff, OND does not always adequately communicate the key question or point of interest to ODS when it requests a consult, and as ODS works on the consult there is sometimes little interaction between the two offices. After a consult is completed and sent to OND, ODS staff reported that OND sometimes does not respond in a timely manner or at all. Several ODS staff characterized this as consults falling into a “black hole” or “abyss.” OND’s Director told us that OND staff probably do not “close the loop” in responding to ODS’s consults, which includes explaining why certain ODS recommendations are not followed. In some cases CDER managers and OND staff criticized the methods used in ODS consults and told us that the consults were too lengthy and academic.

FDA Postmarket Drug Safety

ODS management has not effectively overseen postmarket drug safety issues, and as a result, it is unclear how FDA can know that important safety concerns have been addressed and resolved in a timely manner. According to a former ODS Director, the small size of ODS’s management team has presented a challenge for effective oversight of postmarket drug safety issues. Another problem is the lack of systematic information on drug safety issues. According to the ODS Director, ODS currently maintains a database of consults that can provide certain types of information such as the total count, the types of consults that ODS staff conducted, and the ODS staff that wrote the consults. But it does not include information about whether ODS staff have made recommendations for safety actions and how the safety issues were handled and resolved, including whether recommended safety actions were implemented by OND. For example, ODS was unable to provide us with a summary of the recommendations for safety actions that its staff made in 2004 because it was not tracking such information.

Data constraints—such as weaknesses in data sources and limitations in requiring certain studies and obtaining data—contribute to FDA’s difficulty in making postmarket drug safety decisions. OND and ODS use three different sources of data to make postmarket drug safety decisions. They include adverse event reports, clinical trial studies, and observational studies. While data from each source have weaknesses that contribute to the difficulty in making postmarket drug safety decisions, evidence from more than one source can help inform the postmarket decision-making process. The availability of these data sources is constrained, however, because of FDA’s limited authority to require drug sponsors to conduct postmarket studies and its resources.

While decisions about postmarket drug safety are often based on adverse event reports, FDA cannot establish the true frequency of adverse events in the population with AERS data. The inability to calculate the true frequency makes it hard to establish the magnitude of a safety problem, and it makes comparisons of risks across similar drugs difficult.

In addition, it can be difficult to attribute adverse events to particular drugs when there is a relatively high incidence rate in the population for the medical condition.

For example, ODS staff analyzed adverse event reports of serious cardiovascular events among users of the anti-inflammatory drug Vioxx in a 2001 consult. However, because Vioxx was used to treat arthritis, which occurs more frequently among older adults, and because of the relatively high rate of cardiovascular events among the elderly, ODS staff concluded that the postmarket data available at that time were not sufficient to establish that Vioxx was causally related to serious cardiovascular adverse events.

With AERS data it is also difficult to attribute adverse events to the use of particular drugs because the AERS reports may be confounded by other factors, such as other drug exposures. For example, one AERS report described a patient who developed cardiac arrest after he was given the drug hyaluronidase with two local anesthetics in preparation for cataract surgery. Because local anesthetics can lead to cardiac events, the ODS safety evaluator who reviewed this case concluded that the causal role of hyaluronidase alone could not be established.

FDA may also use data from clinical trials and observational studies to support postmarket drug safety decisions, but each source has weaknesses that constrain the usefulness of the data provided. Clinical trials, in particular randomized clinical trials, are considered the “gold standard” for assessing evidence about efficacy and safety because they are considered the strongest method by which one can determine whether new drugs work.

However, clinical trials also have weaknesses. Clinical trials typically have too few enrolled patients to detect serious adverse events associated with a drug that occur relatively infrequently in the population being studied. They are usually carried out on homogenous populations of patients that often do not reflect the types of patients who will actually take the drugs, including those who have other medical problems or take other medications. In addition, clinical trials are often too short in duration to identify adverse events that may occur only after long use of the drug.48 This is particularly important for drugs used to treat chronic conditions where patients are taking the medications for the long term. Observational studies, which use data obtained from population-based sources, can provide FDA with information about the population effect and risk associated with the use of a particular drug. Because they are not controlled experiments, however, there is the possibility that the results can be biased or confounded by other factors.

Despite the weaknesses of clinical trials and observational studies, evidence from both types of studies helps inform FDA’s postmarket drug safety decision-making process. For example, clinical trials conducted by drug sponsors for their own purposes sometimes provide information for FDA’s evaluation of postmarket drug safety issues. For instance, drug sponsors sometimes conduct clinical trials for drugs already marketed in order to seek approval for a new or expanded use. These studies may also be conducted to support claims about the additional benefits of a drug, and their results sometimes reveal safety information about a marketed drug. For example, to support the addition of a claim for the lower risk of gastrointestinal outcomes (such as ulcers and bleeding), Vioxx’s sponsor conducted a clinical trial that found a greater number of heart attacks in patients taking Vioxx compared with another anti-inflammatory drug, naproxen. This safety information was later added to Vioxx’s labeling. In addition to relying on sponsors, ODS partners with researchers outside of FDA to conduct postmarket observational studies through cooperative agreements and contracts. For example, several cooperative agreements supported a study of Propulsid using population-based databases from two managed care organizations and one state Medicaid program, before and after warnings on contraindications were added to the drug’s label in 1998. The cooperative agreement researchers, which included ODS staff, measured the prevalence of contraindicated use of Propulsid, and found that a 1998 labeling change warning about the contraindication did not significantly decrease the percentage of users who should not have been prescribed this drug.
FDA’s access to postmarket clinical trial and observational data, however, is limited by its authority and available resources. As described previously, FDA does not have broad authority to require that a drug sponsor conduct an observational study or clinical trial for the purpose of investigating a specific postmarket safety concern. One senior FDA official and several outside drug safety experts told us that FDA needs greater authority to .require such studies. Long-term clinical trials may be needed to answer safety questions about risks associated with the long-term use of drugs, such as those that are widely used to treat chronic conditions. For example, during a February 2005 scientific advisory committee meeting, some FDA staff and members of the Arthritis Advisory Committee and DSaRM indicated that there was a need for better information on the long-term use of anti-inflammatory drugs and discussed how a long-term trial might be designed to study the cardiovascular risks associated with the use of these drugs. As another example, FDA approved Protopic and Elidel, both eczema creams, in December 2000 and December 2001, respectively. Since their approval, FDA has received reports of lymphoma and skin cancer in children and adults treated with these creams. In March 2005, FDA announced that it would require label changes for the creams, including a boxed warning about the potential cancer risk. An ODS epidemiologist told us that FDA has been trying for several years to get the sponsor to do long-term studies of these drugs, but that it has been difficult to negotiate.

In the absence of specific authority, FDA often relies on drug sponsors voluntarily agreeing to conduct such postmarket studies. But the postmarket studies that drug sponsors agree to conduct have not consistently been completed. For example, one study estimated that the completion rate of postmarket studies, including those that sponsors have voluntarily agreed to conduct, rose from 17 percent in the mid-1980s to 24 percent between 1991 and 2003.54 FDA has little leverage to ensure that these studies are carried out, for example, by imposing administrative penalties.

In terms of resource limitations, several FDA staff (including CDER managers) and outside drug safety experts told us that in the past ODS has not had enough resources for cooperative agreements to support its postmarket drug surveillance program. Annual funding for this program was less than $1 million from fiscal year 2002 through fiscal year 2005. In October 2005 FDA awarded four contracts to replace the cooperative agreements, and FDA announced that these contracts would allow FDA to more quickly access population-level data and a wider range of data sources. The total amount of the contracts, awarded from 2005 to 2010, is about $5.4 million, which averages about $1.1 million per year, a slight increase from fiscal year 2005 funding. The new contracts will provide access to data from a variety of health care settings including health maintenance organizations, preferred provider organizations, and state Medicaid programs.

According to an FDA official, FDA does not conduct its own clinical trials because of the high cost associated with carrying out such studies and because FDA does not have the infrastructure needed to conduct them. It was recently estimated that clinical trials designed to study long-term drug safety could cost between $3 million and $7 million per trial.55 The estimated cost of just one such trial would exceed the amount FDA has currently allocated ($1.1 million) for its contracts with researchers outside of FDA.