Skip to main content

Systematic reviews of ten pharmaceutical pricing policies – a research protocol



High prices of pharmaceutical products are an increasing challenge in high- and low-income countries. Governments in many countries have implemented pricing policies to ensure affordability of medicines to patients and healthcare systems. The World Health Organization published in 2015 the Guideline on Country Pharmaceutical Pricing Policies, which was based on a series of evidence reviews in the preceding years.

As part of the ongoing update of this guideline, we present a protocol for 10 systematic literature reviews on pharmaceutical pricing policies to be covered by the updated guideline.


The systematic literature reviews will be undertaken according to the principles embodied in the Cochrane Handbook and Centre for Reviews and Dissemination. The interventions studied are pharmaceutical pricing policies implemented by public institutions or a group of purchasing organizations/individuals (e.g. health services). Studies reporting price, volume, availability and/or affordability as the primary outcomes will be eligible for inclusion. Studies in any country or jurisdiction, in any language and in any setting published in 2004 or later are eligible. Eligible study designs are randomized and non-randomized trials, and observational studies including cohort studies, panel data analyses, comparative time series design (including interrupted time-series and repeated measures studies), and controlled before-after studies. A list of 21 databases of peer-reviewed and grey literature will be searched, along with supplementary searches of relevant national and international organizational and governmental websites. Risk of bias will be assessed according to the Cochrane Effective Practice and Organisation of Care (EPOC) guidelines. A summary table according to the EPOC Worksheets for preparing a Summary of Findings table (SoF) using GRADE will be provided.


The results of the review will be used as part of the update of the WHO Guideline on Country Pharmaceutical Pricing Policies. The current protocol may serve as an example for performing systematic literature reviews to inform policy makers.


In recent years, high prices of pharmaceutical products have posed challenges in high- and low-income countries alike. In many instances, high prices of pharmaceutical products have led to significant financial hardship for individuals and negatively impacted on healthcare systems’ ability to provide population-wide access to essential medicines [1]. Governments in many countries have implemented pricing policies to ensure affordability of medicines to patients and healthcare systems.

In view of these problems and the overall mission of the World Health Organization (WHO), WHO has mandates to support countries in ensuring that medicines are affordable, by providing policy guidance on pricing of pharmaceutical products, as requested by Member States. These mandates include World Health Assembly (WHA) decision WHA71(8), which requested the Director-General to elaborate a road map report outlining the programming of WHO’s work on access to medicines and vaccines for the period 2019–2023. Guidance on pricing policy is one of the key milestones specified in this road map [2]. The WHO 13th General Programme of Work includes various areas of work relating to pricing to improve access to medicines, vaccines and health products such as fair pricing, and WHO Regional Committees Resolutions relating to access to medicines have noted the importance of having robust policies on the pricing of health products [3]. Finally the United Nations (UN) Sustainable Development Goals (SDGs) recognise the importance of achieving universal health coverage, through “financial risk protection, access to quality essential health-care services and access to safe, effective, quality and affordable essential medicines and vaccines for all” (SDG 3.8) [4].

In 2015, WHO published a guideline on country pharmaceutical pricing policies (“2015 Guideline”) [5]. This guideline was developed based on an evidence review conducted in 2010 and will be updated in 2020. Since 2010, the body of literature documenting the effects of various government pricing policies has increased. To ensure the updated guideline recommendations reflect current evidence base, WHO has commissioned this systematic review to provide an updated synthesis of the relevant evidence for the existing guideline, and to answer additional policy questions relating to pricing of pharmaceutical products.

The overall objective of this review is to assess the effects of 10 pharmaceutical pricing policies implemented individually or in combination, by an institution or a group of organizations, on price, volume, availability, and affordability of pharmaceutical products.

The specific objectives are (1) to the extent possible, estimate the effect size of pricing policy or policies on each prespecified outcome, (2) to the extent possible, develop Grade of Recommendation, Assessment, Development and Evaluation (GRADE) evidence profiles for each research question, with a view to assessing the overall strength, direction and quality of the evidence, and (3) to the extent possible, describe any practical and contextual considerations, based on an assessment of qualitative evidence pertaining to countries’ experiences, that might impact the implementation of pricing policies.


The systematic reviews will be undertaken according to the principles of systematic reviewing embodied in the Cochrane Handbook and guidance document published by the Centre for Reviews and Dissemination (CRD) which offers approaches for a range of study designs beyond randomised clinical trials (RCTs) [6, 7].

Types of interventions

The interventions are pharmaceutical pricing policies, which are defined as a set of written principles or requirements agreed or adopted by a public institution (e.g. a government) or a group of purchasing organizations/individuals (e.g. health services) for managing the prices of pharmaceutical products for human use. A pharmaceutical product, commonly referred interchangeably with drug, medicine or pharmaceutical, is defined as any manufactured or refined substance for human or veterinary use that is intended to modify or explore physiological systems or pathological states for the benefit of the recipient (adapted from WHO Glossary [8]). For the purpose of this review, the scope includes medicines (both small molecules and biological products) and vaccines for human use. There are 10 eligible policies or strategies within the scope of this review, as defined in Table 1. Single policies, or combinations of policies, are considered eligible. The review scope includes studies comparing interventions against other interventions, options or strategies, or a counterfactual in the absence of comparator interventions, including historical comparisons. Studies not describing any comparator or counterfactual will not be eligible for inclusion in the reviews.

Table 1 Definitions of pricing policies and strategies

Types of outcome measures

Studies reporting price, volume, availability and/or affordability as primary outcomes will be eligible for inclusion in the systematic reviews. Studies that report the primary outcomes will also be assessed for information on any secondary outcome, including transparency, system efficiencies, and adverse outcomes (shortages, quality issues, safety issues, unethical conduct, illegal conduct, equity). Definitions of primary and secondary outcomes are provided in Tables 2 and 3, respectively.

Table 2 Definitions of the primary outcomes
Table 3 Definitions of secondary outcomes

Contextual factors

Descriptions of the contextual factors or prerequisites (barriers or facilitators) in included studies, which have contributed to the success or failure of a policy intervention, will be identified and extracted. This includes strategies that were used to implement an intervention successfully. Studies reporting descriptive and qualitative information on the implementation of effective/successful interventions, in addition to the primary outcome(s), will be eligible.

Study settings

Studies conducted in any country or other jurisdictions (e.g. administrative regions) will be eligible. A subgroup of interest is studies that focus on middle and low-income countries. Outcomes in public, private and mixed public-private settings are of interest.

Study designs

The following study designs comparing interventions to at least one comparator or counterfactual are eligible:

  • randomized trial

  • non-randomized trial

  • observational studies, including:

    • cohort studies or panel data analysis

    • comparative time series design, including interrupted time series (ITS) and repeated measures (RM) study

    • controlled before-after study

Existing systematic reviews on relevant pharmaceutical pricing policies will not be included in the present review due to potential differences in scope and methodology, but any reviews identified will be discussed in the present reviews, and references from recent reviews (published in the last 5 years) will be checked for eligibility. A summary of all inclusion and exclusion criteria is provided in Table 4.

Table 4 Summary of the inclusion and exclusion criteria

Search strategy

The resources and supplementary search approaches proposed reflect the topic area and the anticipated importance of grey literature. Literature on pharmaceutical pricing policies has relevance to both healthcare, economics and policy research. A base strategy will be developed in MEDLINE (OvidSP) and translated to equivalent searches in other databases; the final Ovid MEDLINE strategy (Supplementary materials in Annex) will be peer-reviewed by a second Information Specialist for errors in spelling, syntax and line combinations.

The search will include the following databases: Ovid MEDLINE(R) and Epub Ahead of Print, In-Process & Other Non-Indexed Citations and Daily; Embase; Cochrane Database of Systematic Reviews (CDSR); Epistemonikos; Database of Abstracts of Reviews of Effects (DARE); Health Technology Assessment (HTA) Database; Social Science Citation Index; LILACS; EconLit; NHS Economic Evaluation Database (NHS EED); INRUD (International Network for Rational Use of Drugs) Bibliography; OECD iLibrary Books (freely accessible content only); International Political Science Abstracts (IPSA); WHO IRIS (Institutional Repository for Information Sharing); World Bank Documents & Reports; World Bank Open Knowledge Repository; World Bank eLibrary (freely accessible content only); IDEAS; Essential Medicines and Health Products Information Portal (WHO); Open Grey; Global Index Medicus.

Searches will be conducted using each database or resource listed, translating the Ovid MEDLINE strategy appropriately. Translation includes consideration of differences in database interfaces and functionality, in addition to variation in indexing languages and thesauri.

In addition, in the context of the balanced bibliographic database search approach, the main database searches will be complemented by supplementary search approaches: a range of governmental and intergovernmental websites will be searched, and experts (WHO Guideline Development Group, project expert panel) will be contacted for additional reference suggestions.

Since the topic context is one where developing a robust strategy without retrieving large numbers of records is challenging, a number of pragmatic decisions have been taken when developing the strategy, which can be found in Supplementary Material in the Annex.

Where possible, the results of searches will be downloaded in a tagged format and loaded into bibliographic management software (EndNote). The results will be deduplicated using several algorithms and the deduplicated references held in a duplicates EndNote database for checking if required. Results from resources which do not allow export in a format compatible with EndNote will be saved in Word or Excel documents as appropriate and manually deduplicated.

Study selection

Record assessment involves a number of stages. First, a single researcher will assess the search results according to their relevance in providing information on the review, and will remove the obviously irrelevant records based on titles and abstracts such as those that are about treatment effectiveness rather than pricing policy. This will be undertaken within Endnote. Records will be tagged if excluded at the title/abstract stage. Then, the titles and abstracts of remaining records will be assessed for relevance against the protocol criteria by double independent reviewer selection with disagreements adjudicated by a third reviewer. This will be undertaken within Covidence. Covidence allows record tagging which means at this stage records will be initially categorised by the review question they may be able to inform. Once completed we will go back and tag the EndNote library with the include/exclude decisions and group records within EndNote by exclusion reason. Finally, the full text of potentially relevant studies will be obtained and these will be assessed for eligibility against the protocol criteria by double independent reviewer selection with disagreements adjudicated by a third reviewer. This will be undertaken in Covidence. Studies will be tagged by the review question they will inform, so that they can be grouped for assessment at the next stage. Finally, references of included studies will be screened for further potentially eligible studies.

The number of records included and removed at each stage will be recorded in a PRISMA flow diagram. Studies excluded after assessment of the full document for each review will be listed in an excluded studies table with the reason for exclusion. Where results for a study are reported in more than one publication (for example a full paper and a conference abstract), all related publications will be identified and grouped together to ensure that studies are not double counted.

Data extraction and risk of bias assessment

Included studies will be extracted using a standard data extraction form, including information on study details, population/setting/subjects, interventions, outcomes and results. One researcher will extract the data, and the extraction will be checked by a second researcher. Risk of bias will be assessed according to the Cochrane Effective Practice and Organisation of Care (EPOC) guidelines [12]. Bias assessment criteria are adapted to study design (randomized-, non-randomized trials and controlled before-after studies assessed in the same way; interrupted time-series and repeated measures studies assessed in the same way; and a set of assessment criteria applied to all study types). Two researchers will assess risk of bias, resolving conflicts through discussion.

Data synthesis and analysis

We will provide a summary table according to the EPOC Worksheets for preparing a Summary of Findings table (SoF) using GRADE, which will summarise strength of evidence for each primary outcome. Any evidence identified as reviews/synthesis of qualitative evidence will be assessed using GRADE-CERQual [13, 14].

As it is expected that there will be substantial differences in the characteristics and contexts of included studies, we do not plan to undertake a meta-analysis. If several studies report on the same outcome, a mean/median and a range will be given. Otherwise a narrative summary describing the quality of the studies, the relationship between studies, and patterns that we have discerned in the data will be provided.

Where applicable, results will be presented according to subgroups of interest which are studies by type of medicines (medicines listed in National Essential Medicines Lists, high-priced medicines), studies in low- and middle-income countries, and studies in the private, public or mixed sectors.


These 10 different systematic reviews will inform the WHO Guideline Development Group to update the Guidelines for pharmaceutical pricing policies. A strength of the search methodology described is that it is very broad, making use of multiple scientific literature databases, grey literature and including both English and non-English articles. This will result in limited reporting bias. However, because search terms need to cover ten topics and terminology is not always straightforward the search results might result in an unprecise output. This would mean that it is possible that many titles need to be screened at title and abstract level.

From experience the number of good quality studies is low in policy research [15]. Limited or no randomized studies exist and often studies present before and after measurements around an intervention without taking into account any form of control. These studies will be excluded and this possibly results in a limited number of papers that can be used for the review. However, it was decided to only include study designs that can provide conclusions with limited chance of bias, as we believe that evidence-based decision making is of the same importance for policy makers as it is for clinicians.

Availability of data and materials

Not applicable.


  1. 1.

    Wirtz VJ, Hogerzeil HV, Gray AL, Bigdeli M, de Joncheere CP, Ewen MA, et al. Essential medicines for universal health coverage. Lancet. 2017;389:403–76.

    Article  Google Scholar 

  2. 2.

    World Health Organization. Medicines, vaccines and health products. Exec Board 144th Sess EB144/17. 2018. Available from: Accessed 23 Jan 2020.

  3. 3.

    World Health Organization. Thirteenth General Programme of Work, 2019–2023: WHO Results framework. 2019;14. Available from: Accessed 23 Jan 2020.

  4. 4.

    United Nations. Goal 3: Sustainable Development Knowledge Platform. Available from: Accessed 23 Jan 2020.

  5. 5.

    World Health Organization. WHO guideline on country pharmaceutical pricing policies. Geneva: World Health Organization; 2015.

  6. 6.

    Higgins JPT, Green S (eds). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. 2011. Available from:

  7. 7.

    Khan KS, Ter Riet G, Glanville J, Sowden AJ, Kleijnen J. Undertaking systematic reviews of research on effectiveness: CRD’s guidance for those carrying out or commissioning reviews. CRD Report 4 [2nd edition]. York: Centre for Reviews and Dissemination (CRD); 2001.

  8. 8.

    World Health Organization. WHO glossary. Available from: Accessed 21 Oct 2019.

  9. 9.

    WHO Collaborating Centre for Pharmaceutical Pricing and Reimbursement Policies. Glossary of pharmaceutical terms. Available from: Accessed 18 Nov 2019.

  10. 10.

    World Health Organization. Technical report: pricing of cancer medicines and its impacts: a comprehensive technical report for the World Health Assembly Resolution 70.12: operative paragraph 2.9 on pricing approaches and their impacts on availability and affordability of medicines. Geneva: World Health Organization; 2018. 112 p., XL.

  11. 11.

    Oxford Dictionary. Available from: Accessed 21 Oct 2019.

  12. 12.

    Cochrane Effective Practice and Organisation of Care (EPOC) Group. Suggested risk of bias criteria for EPOC reviews. Cochrane Effective Practice and Organisation of Care. 2017. Available from: Accessed 19 Dec 2019.

  13. 13.

    Cochrane Effective Practice and Organisation of Care (EPOC) Group. EPOC worksheets for preparing a Summary of Findings (SoF) table using GRADE [Internet]. Cochrane Effective Practice and Organisation of Care. 2017. Available from: Accessed 21 Oct 2019.

  14. 14.

    Lewin S, Booth A, Glenton C, Munthe-Kaas H, Rashidian A, Wainwright M, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implement Sci. 2018;13(S1):2. Available from: Accesssed 24 Jan 2020.

  15. 15.

    Jacobs TG, Robertson J, van den Ham HA, et al. Assessing the impact of law enforcement to reduce over-the-counter (OTC) sales of antibiotics in low- and middle-income countries; a systematic literature review. BMC Health Serv Res. 2019;19:536.

    Article  Google Scholar 

Download references


The authors would like to thank members of the Guideline Development Group for the update of 2015 WHO Guideline on Country Pharmaceutical Policies and WHO Steering Committee for their constructive feedback in formulating the scope, defining the terms and commenting on the methodology described in this systematic review protocol. The authors are also grateful to KiuTay-Teo for his assistance and thorough review of the protocol, to Lynn Al-Tayara and Iris Joosse for editorial assistance in drafting this protocol and to Mick Arber for his contribution to the search methodology, including search strategy development.

The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the members of the Guideline Development Group, WHO Steering Committee or WHO.


This project is commissioned and funded by the WHO Department of Essential Medicines and Health Products (EMP).

Author information




DT, HAvdH and AKMT conceptualised the research questions, outcomes, eligibility criteria and analysis plan in collaboration with WHO counterparts, and JG developed and tested the search strategy based on these. All authors contributed to defining the work flow. Tools for data extraction, risk of bias and quality assessment were developed/selected by DT, HAvdH and AKMT with input from JG. All authors contributed to writing the protocol, and approved the manuscript prior to submission.

Corresponding author

Correspondence to Aukje K. Mantel-Teeuwisse.

Ethics declarations

Ethics approval and consent to participate

Ethical approval will not be necessary for this systematic review of policy interventions.

Consent for publication

Not applicable.

Competing interests

JG declares that she has, within the past 4 years, undertaken consulting work for and received research support (e.g. grants, collaborations, sponsorships or other funding) from commercial entities or other organizations with an interest related to the subject of the work. DT declares that he has, within the past 4 years, undertaken consulting work for commercial entities or other organizations with an interest related to the subject of the work. AKMT has received, within the past 4 years, funding from de Dutch Health Care Institute. HAvdH does not declare any competing interest.

All authors declare that these interests have not and will not influence the work undertaken.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tordrup, D., van den Ham, H.A., Glanville, J. et al. Systematic reviews of ten pharmaceutical pricing policies – a research protocol. J of Pharm Policy and Pract 13, 22 (2020).

Download citation


  • Pharmaceutical pricing policies
  • Protocol
  • Systematic literature review
  • Risk of bias