RWE: BIO Submits Comments on FDA Framework for a Real-World Evidence Program

February 5, 2019

Re: Docket No. FDA-2018-N-4000: Framework for a Real-World Evidence Program

Dear Sir/Madam:

The Biotechnology Innovation Organization (BIO) thanks the Food and Drug Administration (FDA) for the opportunity to submit comments to the Framework for a Real-World Evidence Program.

BIO is the world's largest trade association representing biotechnology companies, academic institutions, state biotechnology centers and related organizations across the United States and in more than 30 other nations. BIO members are involved in the research and development of innovative healthcare, agricultural, industrial, and environmental biotechnology products.

This important and anticipated Framework is well written and provides Sponsors and the community important information on FDA’s vision for advancing the use of Real-World Evidence (RWE). Below, we provide general comments and recommendations on the Framework. Additional detailed comments have also been included at the end of this document:

  • BIO is committed to increasing public learning and understanding of FDA’s perspectives on the acceptance of RWE to support regulatory decision-making. We applaud FDA for the work done in engaging external stakeholders, in particular through the workshops developed in collaboration with the Duke Margolis Center for Health Policy. We note that the Framework currently does not lay out plans for future workshops or external stakeholder engagement and education, nor does it set forth how the Agency plans to share experiences and learnings from its demonstration projects and other stakeholder engagements. BIO encourages FDA to consider more detailed plans on how it intends to engage with external stakeholders in order to educate the community at large. Publicly sharing the learnings by the FDA, industry Sponsors, academics and other relevant groups, would allow the community to advance the generation and use of credible RWE for regulatory decision-making. For example, FDA could hold public educational workshops during which the Agency, Sponsors and other stakeholders, as appropriate, could share the results and learnings of demonstration projects, pilot studies, and other relevant activities on the use of RWE for regulatory decision making.
  • The Framework implies, but nowhere explicitly states, that RWE in the appropriate regulatory context of use can be used to help to satisfy or be sufficient to meet the “substantial evidence” requirement. The concept of “adequate and well controlled investigations” is also not discussed. BIO encourages FDA to affirm that studies using RWE — in the appropriate regulatory context-of-use — may help satisfy or be sufficient to meet the ‘substantial evidence’ requirement under 505(D) of the FDCA when two-or-more RCTs are not necessary or feasible and may be considered ‘adequate and well controlled’. Formally documenting this approach, already employed by FDA in certain circumstances, will make it clear that the Agency considers the totality of the evidence and the context of the specific scenario when making regulatory decisions.
  • BIO recognizes the importance of validating methodology used in the generation of evidence for regulatory decision-making. However, a traditional randomized control trial will differ in material ways from RWE generation approaches, including observational studies. As such, the term “replication” should be avoided when referring to comparisons between RWE and traditional RCTs. BIO believes that “consistency in outcomes” may be a more appropriate way to describe the desired finding of these efforts.
  • The Framework emphasizes the importance ensuring that relevant and appropriate endpoints are captured in the RWD sources under consideration. BIO welcomes this mention of endpoints and would suggest FDA considers developing guidance on RWD endpoints and how to best establish the association between real world endpoints and traditional clinical trial endpoints.  
  • The Framework recognizes the importance of developing data standards and methods to maximize the utility of RWD and states FDA is “active in developing data standards for regulatory use and will continue to expand its work in this area. FDA will consider data standards along with the other critical aspects of the RWE Program” (pages 13-14). However, the Frameowrk does not describe how FDA plans to expand its efforst in this area. BIO suggests FDA elaborate on its objectives and workplan for continued efforts to develop data standards for regulatory use and requests opportunities to discuss how industry can assist with this work.
  • In regard to data quality and standards, although Sponsors have final responsibility for documenting the characteristics of the data used to generate RWE, data source owners (e.g., RWD originators, data collectors, and aggregators) can play a critical role in facilitating this process.[1] They can enhance data quality and reliability through improved documentation on, e.g., data provenance, cleaning processes, algorithmic transformations, and linking methods. BIO suggests that the FDA consider the important role data source owners can play in enhancing trust and credibility in RWD.
  • BIO recommends the Agency clarify whether data standards related to the submission of RWD are likely to become mandatory in a fashion similar to CDISC. Unlike clinical trials, Sponsors often do not own the data sources and have limited, if any, ability to impact data standards. Examples are registries, claims data and electronic medical records.
  • The Framework notes that future guidance will address considerations for the use of real-world data from other countries (pages 16-17). However, it does not indicate if guidance will address and supporting guidances should also address the use of foreign literature and data used to support ex-US approvals. This is particularly relevant for reducing regulatory burden for drugs with established safety and efficacy profiles in literature (e.g., off label standard of care, approved by other regulatory health authorities, particularly those where FDA has an MOU based on that country’s regulatory structure, such as those where there is a MRA for GMP acceptance). For example, FDA could draw from existing guidance documents such as “FDA Acceptance of Foreign Clinical Studies Not Conducted Under and IND Frequently Asked Questions.”
  • Lastly, BIO appreciates the amount of work FDA is undertaking as part of its implementation of the RWE program. However, it is unclear from the Framework the specific timelines FDA might be considering for the development of the suggested draft guidances, some of which will be iterative. Given the breadth of the work laid out in the Framework, BIO encourages FDA to communicate specific timing and/or prioritization for the development and release of draft guidances. BIO suggests the Agency consider providing an updated Framework document with the suggestions discussed in this document, or alternatively, provide regular updates on the priorities, progress, and timelines of the efforts listed under Program Items in the Framework. BIO welcomes the opportunity to partner with the Agency in providing information or resources throughout the guidance development process.

BIO welcomes this opportunity to submit comments on the Framework for a Real-World Evidence Program. We provide additional specific, detailed comments to improve the clarity of the Framework in the following chart. We would be pleased to provide further input or clarification of our comments, as needed.


[1] Duke-Margolis RWE Collaborative white paper, Characterizing RWD Quality and Relevance for Regulatory Purposes (Oct. 1, 2018)