Working Groups

The HTAi-DIA Working Group

Health Technology Assessment (HTA) is a multidisciplinary process to determine the value of a health technology at different points in its lifecycle, all in service of promoting an equitable, efficient, and high-quality healthcare system. By the very nature and complexity of health technologies, as well as methodological challenges associated with their assessments, uncertainty will always exist at some level in both regulatory and HTA informed decision-making. This has been underscored by the onset of the global health care crisis associated with COVID-19. The pandemic has disrupted attitudes, structures and processes – collectively leading to increased levels of uncertainty and complexity. A far more layered, nuanced and flexible response from all stakeholders in the health care ecosystem is required to achieve equity, efficiency and quality health care 


With the above in mind, the intent of the HTAi-DIA Working Group is to explore how to manage uncertainty in the regulatory-HTA interface across the health technology lifecycle. Specifically, we seek to create a platform and convene conversations between all key stakeholders in the healthcare system that share a common understanding – not simply information and data – to manage uncertainty in regulatory and HTA decision-making on drugs. The better and more effectively this is done, the greater the benefit for patients and healthcare systems. 

It is understood that the processes and methods for exploring and dealing with uncertainty are context and situation specific. Context includes jurisdiction, HTA structures and setting/remits which vary globally, for example regarding established social values or practices in a specific country, at institutional or individual level, or in a specific disease area. In some countries, situations where technologies are considered for children or disadvantaged populations, may also be allowed additional tolerance for greater uncertainty. This is all affected by the appetite for risk within a decision-making organization generally, as well as for the individuals deliberating on its behalf. 

During the 2021 HTAi Global Policy Forum, held virtually on February 22-26, 2021, key challenges and opportunities regarding uncertainty in HTA were discussed. Following the discussions, and building on recommendations made during the Forum, HTAi and the Drug Information Association (DIA) initiated a Working Group to provide guidance on management of uncertainty in the regulatory-HTA interface across the drug’s lifecycle.  


  • Wija Oortwijn
  • Inkatuuli Heikkinen 
  • Milou Hogervorst
  • Rick Vreman 
  • Indranil Bagchi 
  • Inaki Gutierrez-Ibarluzea 
  • Bettina Ryll 
  • Hans-Georg Eichler 
  • Elena Petelos 
  • Sean Tunis 
  • Claudine Sapede
  • Wim Goettsch 
  • Rosanne Janssens 
  • Isabelle Huys 
  • Liese Barbier 
  • Deirdre DeJean 
  • Valentina Strammiello 
  • Dimitra Lingri 
  • Melinda Goodall 
  • Magdalini Papadaki 
  • Massoud Toussi 
  • Despina Voulgaraki 
  • Ania Mitan 
  • Sharon Gorman 
  • Ansgar Hebborn 
  • Stefan Sauerland 
  • Ian Watson 


Panel 13 from the HTAi 2022 Annual Meeting in Utrecht, the Netherlands  

Date: June 27, 2022 


  • Inka Heikkinen, Denmark 
  • Wija Oortwijn, Netherlands 
  • Milou Hogervorst, Netherlands 
  • Iñaki Gutierrez-Ibarluzea 
  • Neil Bertelsen, Germany 
  • Dimitra Lingri, Greece 

Abstract: To accelerate access to innovations for patients, HTA bodies, regulators and stakeholders are facing increasingly uncertainty in their decision-making processes. To better understand the complexity of “uncertainty”, HTAi has formed a multistakeholder working group with DIA to advance this discussion.

Watch it on-demand

Date: March 23, 2023 


  • Wija Oortwijn, Netherlands 
  • Milou Hogervorst, Netherlands 
  • Bettina Ryll, Sweden 
  • Michael Berntgen, Netherlands 
  • Dimitra Lingri, Greece 
  • Wim Goettsch, the Netherlands 
  • Rick Vreman, the Netherlands 

Panel 31 from the HTAi 2023 Annual Meeting in Adelaide, Australia 

Date: June 27, 2023 


  • Wija Oortwijn, Netherlands 
  • Iñaki Gutierrez-Ibarluzea, Spain 
  • Hans Peter Dauben, Germany 

Abstract: The panel will describe results of the HTAi-DIA working group on uncertainty on how to gain trust and confidence as well as a proposal of how to follow a pathway to increase common action on trustful data as need for future actions in healthcare and health service industry. 

Publication Date: June 16, 2023 

Journal: International Journal of Technology Assessment in Health Care | open access article


  • Hogervorst, Milou; Utrecht University, Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences; Zorginstituut Nederland 
  • Vreman, Rick; Utrecht University, Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences 
  • Heikkinen, Inka; MSD, Global Regulatory Policy 
  • Bagchi, Indranil; GlaxoSmithKline, Global Pricing & Market Access 
  • Gutierrez-Ibarluzea, Iñaki; Basque Foundation for Health Innovation, Director of Managerial and Organisational Innovation 
  • Ryll, Bettina; Melanoma Patient Network Europe, 
  • Eichler, Hans-Georg; Association of Austrian social insurance bodies 
  • Petelos, Elena; Maastricht University Care and Public Health Research Institute, Department of Health Services Research; University of Crete Faculty of Medicine, Health and Society Lab 
  • Tunis, Sean; Tufts Medical Center, Center for Evaluation of Value and Risk in Health 
  • Sapede, Claudine; Novartis, Global Public Affairs 
  • Goettsch, Wim; Utrecht University Utrecht Institute for Pharmaceutical Sciences, Division of Pharmacoepidemiology and Clinical Pharmacology; Zorginstituut Nederland 
  • Janssens, Rosanne; KU Leuven, Department of Pharmaceutical and Pharmacological Sciences 
  • Huys, Isabelle; KU Leuven, Department of pharmaceutical and pharmacological sciences 
  • Barbier, Liese; KU Leuven, Department of Pharmaceutical and Pharmacological Sciences 
  • DeJean, Deirdre; Canadian Agency for Drugs and Technologies in Health 
  • Strammiello, Valentina; European Patients’ Forum 
  • Lingri, Dimitra; European Healthcare Fraud and Corruption Network; Aristotle University of Thessaloniki 
  • Goodall, Melinda; MSD (UK) Limited, Regulatory Affairs International 
  • Papadaki, Magdalini; MSD (UK) Limited, Regulatory Affairs International 
  • Toussi, Massoud; IQVIA, Real World Evidence Solutions 
  • Voulgaraki, Despina; Medtronic Ltd; Maastricht University 
  • Mitan, Ania; DIA 
  • Oortwijn, Wija; Radboud University Medical Center, Department for Health Evidence; Health Technology Assessment international 

Objectives: Uncertainty is a fundamental component of decision-making regarding access to, pricing and reimbursement of drugs. The context-specific interpretation and mitigation of uncertainty remain major challenges for decision-makers. Following the 2021 HTAi Global Policy Forum, a cross-sectoral, interdisciplinary HTAi-DIA Working Group (WG) was initiated to develop guidance to support stakeholder deliberation on the systematic identification and mitigation of uncertainties in the regulatory-HTA interface.  

Methods: Six online discussions among WG members (Dec 2021-Sep 2022) who examined the output of a scoping review, two literature-based case studies and a survey; application of the initial guidance to a real-world case study; and two international conference panel discussions.  

Results: The WG identified key concepts, clustered into twelve building blocks that were collectively perceived to define uncertainty: ‘unavailable’, ‘inaccurate, ‘conflicting’, ‘not understandable’, ‘random variation’, ‘information’, ‘prediction’, ‘impact’, ‘risk’, ‘relevance’, ‘context’, ‘judgement’. These were converted into a checklist to explain and define whether any issue constitutes a decision-relevant uncertainty. A taxonomy of domains in which uncertainty may exist within the regulatory-HTA interface was developed to facilitate categorization. The real-world case study was used to demonstrate how the guidance may facilitate deliberation between stakeholders and where additional guidance development may be needed.  

Conclusions: The systematic approach taken for the identification of uncertainties in this guidance has the potential to facilitate understanding of uncertainty and its management across different stakeholders involved in drug development and evaluation. This can improve consistency and transparency throughout decision processes. To further support uncertainty management, linkage to suitable mitigation strategies is necessary. 

Early HTA Terminology Working Group

Clear definition of the term “early HTA” is lacking and there is confusion with related terms such as “early advice”, “early dialogue”, and “development-focused HTA”. Reaching a consensus on what is meant by each of these terms will form a foundation for discussion and dialogue about the best methods to adopt and improve the profile of the work.


The HTAi Early HTA Terminology Working Group aims to reach consensus on the definition of early HTA and related terms (early dialogue, early advice, development-focused HTA), to clearly delineate what does not constitute early HTA and link proposed definitions with existing HTA and early awareness definitions in the HTA Glossary. The methods are as follows: 

  • Develop definitions and supporting materials for Delphi process, which includes:
    • Rapid review of the literature for existing definitions and frameworks
    • Core working group and Advisory Board develop definitions and supporting materials informed by the rapid review
  • A two-stage Delphi process
    • Initial round presenting definitions and supporting materials to individuals involved in evaluation of new health technologies from industry, agencies and academia.
    • Revised definitions and supporting materials circulated for rating. Consensus will be considered to be reached if 70% of respondents agree or strongly agree. If consensus is not reached on a particular aspect a further round may be considered.



  • Janneke Grutters
  • Janet Bouttell

Core Members

  • Neil Hawkins
  • Jon Karnon
  • Carla Fernandez
  • Michelle Tew
  • Wang Yi
  • Sulafa Ahmed
  • Laura Sampietro-Colom
  • Waseem Ahmed
  • Payam Abrishami
  • Erika Seyahian
  • Samuel Owusua Achiaw
  • Debjani Mueller
  • Nora Ibargoyen Roteta
  • Iñaki Gutierrez Ibarluzea 

Advisory Members

  • Yves Verboven
  • Chris McCabe
  • Andrew Partington
  • Dalia Dawoud
  • Rabia Sucu


Expected outcomes include definitions of the following terms:

  • Early HTA
  • Development-focused HTA
  • Early Advice
  • Early Dialogue

The terms will be proposed for inclusion in the HTA Glossary. A scientific article describing the work of the WG will be prepared for submission to IJTAHC.