PDUFA V: BIO Comments on FDA's NME Review Program Assessment

Dear Sir/Madam:

The Biotechnology Industry Organization (BIO) thanks the Food and Drug Administration (FDA) for the opportunity to submit comments on the statement of work for the independent assessment of the Prescription Drug User Fee Act (PDUFA V) New Molecular Entity Review Program (‘the Program’).  BIO strongly supports the enactment of the new review model under PDUFA V, which is intended to facilitate earlier patient access to modern medicines and therapies by enhancing FDA-Sponsor communication and increasing first cycle approval rates.  A strong, independent, real-time assessment of the Program will help to introduce greater transparency and accountability into the Program and promote FDA and industry best practices that can further strengthen the drug and biologic evaluation process.

BIO represents more than 1,100 biotechnology companies, academic institutions, state biotechnology centers and related organizations across the United States and in more than 30 other nations. BIO members are involved in the research and development of innovative healthcare, agricultural, industrial and environmental biotechnology products, thereby expanding the boundaries of science to benefit humanity by providing better healthcare, enhanced agriculture, and a cleaner and safer environment. 

GENERAL COMMENTS:

1.    The Contractor should Maintain Independence in the Selection of Study Methodologies

When developing the workplan and assessment methodology, BIO believes that the Contractor will benefit from input and advice from FDA staff with experience in tracking review metrics and with hands-on technical expertise utilizing FDA databases.  We are pleased to see that there is ample opportunity for the Contractor to engage with FDA during the evaluation process.

However, we request that FDA clarify in the final statement of work that as an independent entity, the Contractor is not required to adopt FDA’s proposed revisions to the workplan, proposed data collection and analysis methodologies, interim and final assessments, or presentations to the public.  For the evaluation of the Program to be successful in the eyes of all stakeholders, it is important that the selected Contractor be viewed as an independent, credible, and objective party.  Indeed, the PDUFA V commitment letter states that “the Program described in Section IIA shall be evaluated by an independent Contractor with expertise in assessing the quality and efficiency of biopharmaceutical development and regulatory review programs.” (II.B., emphasis added) 

There are several phrases used throughout the document that suggest that the Contractor may be obligated to incorporate FDA’s proposed edits, or are unclear on this point.  For example (emphasis added):

·         Page 5, Task #1:“At this meeting, the Contractor shall present its proposed overall approach and workplan to FDA. The Contractor shall revise the proposed approach based on feedback from FDA.”

·         Page 6, Task #4: “The Contractor shall develop a proposed approach to the evaluation of FDA-applicant interactions in pre-submission meetings, mid-cycle communications, and late-cycle meetings as well as the quality and completeness of applicant submissions and FDA communications related to applications reviewed under the Program. The Contractor shall present the proposed approach to FDA and subsequently revised based on any FDA feedback.”

·         Pages 6-7, Task #5: “The Contractor shall develop a proposed approach to quantitative and qualitative analysis of all data collected on applications reviewed under the Program, including proposed evaluation methodologies for both qualitative and quantitative data. The Contractor shall present the proposed approach to FDA and subsequently revised based on any FDA feedback.”

·         Page 7, Task #8: “The Contractor shall develop a guide and a data collection instrument for conducting interviews of FDA review staff and Sponsors of applications reviewed under the Program. The draft guide and collection instrument shall be submitted to FDA and subsequently revised based on any FDA feedback.”

·         Page 8, Task #11:” Draft versions of the interim and final assessment shall be submitted to FDA and subsequently revised based on any FDA feedback.”

·         Page 8, Task #13:“The Contractor shall present the findings of the interim and final assessments at the public meetings conducted on each assessment. These presentations shall include the Contractor’s analysis of any comments submitted to the public docket. Draft presentation materials shall be submitted to FDA in advance of the public meetings and revised based on any FDA feedback.”

In order to maintain the independence, autonomy, and credibility of the Contractor, the statement of work should explicitly acknowledge that FDA does not have authority to overrule the decisions of the Contractor regarding the conduct and outcomes of the assessment.  Rather, the statement of work should state that the Contractor has discretion to incorporate FDA revisions to the proposed workplan, methodologies, or assessments on a case-by-case basis.

Should there be a disagreement between the FDA and the independent Contractor, we suggest that any public deliverables include an appendix outlining any areas of disagreement or divergence so that the FDA position can be adequately communicated and understood.  Such an approach has been used successfully used in Inspector General reports, for example.

2.    The Scope of the Assessment should Include RTFs and Missed PDUFA Goals

The statement of work also addresses the cohort of applications eligible for the assessment.  The document states that “because a key measurement of the Program’s success will be first cycle review performance, FDA has determined that only applications on which the Agency has taken at least a first cycle action will be included in each evaluation.” (Page 4)  In general, BIO agrees that the first cycle approval rate is a primary outcome measure for the Program.  However, there are other key variables that should be considered when evaluating the Program that may not be captured by the proposed cohort.  Because the Program is predicated on the submission of a complete application, it will also be important to evaluate whether the Program has contributed to an increase or decrease in the rate of refuse-to-file (RTF) determinations.  Furthermore, the Contractor should assess whether the Program may lead to an increase or decrease in missed first-cycle PDUFA goals.  By definition, applications that have experienced a RTF or missed PDUFA goal have not received a first cycle action and would be excluded from the cohort.  BIO believes that excluding these applications from the analysis will result in an incomplete picture of application outcomes under the Program. 

We suggest that the statement of work be revised to capture these application outcomes.  Alternatively, the proposal to limit the cohort to only those applications that have received first action could be viewed as a FDA recommendation only, and the independent Contractor would be afforded the discretion to choose the final cohort of applications included in the study.

3.    Specific Metrics to Capture on the Assessment of The Program

While the data collection tools and methodologies will ultimately be determined by the Contractor, it will be helpful for FDA and industry to recommend constructive and practical approaches on how to best track the information that will be captured. BIO proposes the use of a grid, such as the one we have included in Appendix A for example, to assist FDA and industry to align on the metrics that will be tracked to assess the Program.  Collecting information from both a quantitative and qualitative perspective will help to determine not only the timeliness, but also the quality of the activities conducted.   Further, establishing a pre-determined template that can be distributed to participating companies upon entering the Program will help Sponsors to internally collect information to share with the Contractor on a real-time basis, rather than attempting to gather the data retrospectively post-action.  We suggest that a similar example or grid be included with the final statement of work for illustrative purposes.

4.    Positive and Negative Case Studies should Complement Review Metrics

Performance metrics are important tools to track and evaluate performance, but data based on averages and medians can often overlook important lessons from the margins of the bell curve.  To better understand best practices and areas for improvement, we suggest that the Contractor also qualitatively evaluate individual case studies – both positive and negative – from the 5-10% of applications that underwent the shortest and longest review periods. 

For example, if FDA approves a drug or biologic in record time in a single review cycle, then what factors contributed to that success?  What lessons for both FDA and industry can be applied elsewhere?  On the other hand, if a product encounters two, three, or more review cycles or undergoes unnecessary delays, what can be drawn from that experience and avoided in the future?

While the lessons learned and best practices identified through case studies will be helpful for FDA and industry, the particular details should be blinded and anonymized prior to publication or dissemination, unless the Sponsor authorizes the release.

5.    Sponsor Feedback to the Contractor should be Blinded and Anonymized

BIO also requests that FDA clarify how Sponsor feedback to the independent Contractor will be anonymized and aggregated prior to FDA review and publication.  The PDUFA V commitment letter specifies that during the interim and final assessment public meetings “FDA will discuss the findings of the [interim/final] assessment, including anonymized aggregated feedback from the Sponsors and FDA review teams resulting from the independent contractor interviews.” (II.B.) However, it is unclear from the statement of work whether individual, attributable Sponsor responses will be shared with FDA prior to publication of the assessments. 

For example, the early drafts of the reports prepared by Booz Allen Hamilton for FDA review in PDUFA IV included data attributable to individual applications, while the final, publicly released reports had this information appropriately redacted.  While this approach was appropriate based on the information contained within that particular analysis, a similar approach is concerning for this assessment because of the nature of the interviews.  

To facilitate frank and constructive post-action conversations between the Sponsor and the independent Contractor, we encourage FDA to clarify that responses will be aggregated and anonymized prior to submission of the assessment to FDA.

CONCLUSION:

BIO appreciates this opportunity to comment on the Assessment of the Program for Enhanced Review Transparency and Communication for New Molecular Entity New Drug Applications and Original Biologics License Applications in Prescription Drug User Fee Act V.  Specific, detailed comments are included in the following chart.  We would be pleased to provide further input or clarification of our comments, as needed.