Global overview

Problem

Health Policy and Systems Research (HPSR) is a field that aims to understand and improve the ways in which societies organize themselves to achieve collective health goals, and how different actors interact in the policy and implementation processes to contribute to policy outcomes. HPSR is, by its nature, interdisciplinary: a rich blend of economics, sociology, anthropology, political science, public health and epidemiology.  These different disciplines together draw a comprehensive picture of how health systems respond and adapt to health policies, and how health policies can shape − and be shaped by − health systems and the broader determinants of health (Alliance for Health Policy and Systems Research, 2012).

Despite the recognized growth and evolution of the field of HPSR, there are no known data sources to systematically monitor it, at either global or national levels. Recent studies underscore the lack of data on HPSR funding. As such, this Report has three core objectives: (i) to build the field of HPSR by benchmarking common indicators of HPSR relevant to countries, (ii) to enable the monitoring of national-level HPSR growth over time, and (iii) to advocate for further HPSR investments.

---

Methods

The Alliance convened country teams to develop the country profiles from January to April 2021.  Four initial domains were identified as relevant for describing the evolution of HPSR in countries:

• knowledge generation;
• academic and institutional capacity (institutions, defined here as “groups of collaborating professionals dedicated to HPSR within a legal entity or a unit of a larger legal entity” (Gonzalez Block and Mills, 2003);
• engagement with policy-makers and the public; and
• HPSR financing.

These indicators were elaborated collectively by the country teams  through an iterative process. During January and February 2021, country teams met virtually to define and improve indicators under each of these domains and to refine the templates to gather country and institutional-level information. While the indicators aimed to be as comprehensive as possible across diverse settings, it became clear during data collection that some indicators needed further refinement and were, at times, either too broad or too narrow for a given country context. This Report covers the three-year period from 2018 to 2020.

For detailed definitions of the indicators used, please see Annex 1.

Each country team collected their country data in March 2021. This involved the co-development of a shorter country profile template containing indicators related to nationally available information. This template captured: the number of known academic and non-academic institutions with a stated mandate to generate HPSR; the national health research budget; and narratives to describe the overarching state of knowledge generation, instances of policy influence, institutional capacity, and funding for HPSR within the country. A longer institutional profile template was also developed to capture indicators related to available information from institutions, including: the number of reports produced, number of formal meetings held, number of faculty teaching and/or conducting HPSR, number of participants enrolled in HPSR-focused short-courses, estimated amount of institutional budget dedicated to HPSR, and number of local newspaper articles (of any type) authored by faculty, and media appearances by faculty (For templates used, please see Annex 2) . It is important to note that in some countries, the institutional-level template was sent directly to the HPSR institutions identified by the country teams, while other country teams used alternative mechanisms to capture data and then subsequently complete the forms on the basis of the information gathered.

Country and institutional data were inputted into common dashboards, from May to June 2021, and iteratively cleaned for errors, duplications, incompleteness and aggregation or disaggregation where appropriate. Some variables, like institution type, were also re-categorized to ensure consistency across countries. While most countries provided data on budgets and finances in US$, some provided information in their local currency. In those cases, figures were converted using an average annual conversion rate for the year in which the information was provided. No funding figures have been adjusted for inflation. All figures were provided by country teams.

---

Results

This Report shows the increasing production of HPSR knowledge and stronger engagement between researchers, policy-makers and the general public across the 16 countries. There are many examples of the positive impact of HPSR on policy – from supporting the establishment of laws and national plans, to the improvement of health programmes.

The Report demonstrates that HPSR knowledge generation is increasing. In all, 288 institutions provided information about the number of reports they published in 2020: a total of 5808, or an average of about 20 reports per institution­ – an overall increase since 2018, when institutions identified just under 5000 published reports. In terms of institutional capacity for HPSR, 242 institutions provided information about the number of faculty or staff involved in HPSR activities in 2020, with an average of 29 among those institutions. There were, on average, 132 students participating in HPSR-related short courses among the 128 institutions that reported offering such training in 2020. This is a marked decrease from 2019 and is likely related to restrictions put in place due to the COVID-19 pandemic. It is important to note that these figures are likely to be under-estimates, as short courses may under-report the number of participants enrolled in other (i.e., formal graduate degree) programmes, and in some countries, such data are not recorded.

Despite significant challenges collecting information on HPSR financing, information provided for 2020 from 149 institutions shows that, collectively across these 16 countries, an estimated US$ 260 million was invested in HPSR activities – roughly the same amount as in 2019 (US$ 261 million), but a large jump from 2018 (US$ 87 million). This is less than the global annual average of US$ 433 million from 2010-2014 identified in the World HPSR Report in 2017, though more countries and institutions were included in that report. It proved difficult to generate estimates for HPSR financing in countries for many reasons. First, many countries simply do not have national health research budgets, and where they do exist, the budget line for HPSR is rarely, if ever, specified. As a result, attempts to collate estimates have to contend with the complexity of merging financing from various and often aggregated sources that may (or may not) include governmental research and training budgets, as well as funds from global donors, which might be channelled through government, or directly to institutions through grants (this is addressed further under Challenges). One thing is clear, however: international funding supports a substantial share of HPSR in the countries where these data are available, ranging from 45% to 69%.

Total expenditure reported by institutions

---

Challenges

During the elaboration of the indicators and collection of data, several issues emerged related to the interpretation of key terms, including institutions, reports and meetings.

Identifying HPSR institutions

When identifying HPSR institutions, we sought to include academic and non-academic entities that generate training, research or policy products that are relevant to HPSR. This meant identifying those institutions that mention HPSR specifically in their mission or vision. However, it became apparent while gathering data that many institutions in several countries engage in HPSR, whether or not they state this explicitly as part of their overall mission or vision. In other cases, some institutions classified themselves as both academic (i.e., offering training and providing degrees/certification) and non-academic.

Country teams used a variety of methods to address this identification challenge, depending on the size and heterogeneity of the country concerned. These methods included convenience sampling, snowball sampling and purposive sampling, identifying institutions through searches (of national databases and accredited teaching programmes, as well as the scientific literature) and through professional networks. In most cases, the institutions that were identified were known to the country teams through their existing networks and deep knowledge of actors in their own settings. These were institutions that were known to conduct work focused on some aspect of the health systems, rather than work that was biomedical or clinical in focus. In a small number of cases, country teams used searches to identify authors from country institutions that generate HPSR. As a result, the institutions included in this Report are broadly identified as those doing research and teaching in relation to HPSR.

There was also diversity in the institutional level at which HPSR activities occur, particularly within academic institutions, depending on country contexts and how universities are structured. As a result of this variation, HPSR institutions are considered to include small research groups within faculties, whole departments, and/ or schools or even whole universities, as relevant to the country context.

Identifying reports

In defining reports, we sought to identify national-level reports produced by government or HPSR institutions that were relevant to health systems strengthening. We have not included subnational reports (i.e., reports aimed at a particular district or region within a country) and reports that were not widely disseminated or were not considered by the country teams or the institutions surveyed within the country as being nationally influential were not included. In addition, there was a risk of double counting in a small number of cases where a report was produced jointly by multiple institutions.

Identifying meetings

We initially defined meetings as policy dialogues. However, this terminology failed to recognize that in certain countries, other forms of exchange between researchers and policy- and decision-makers takes place through a variety of knowledge translation models. Several countries have no centralized record-keeping of meetings, and the poor documentation of meetings meant that the ones identified were limited to personal recall. In other countries, such meetings were seen as a routine occurrence and were, therefore, too numerous to count. Ultimately, we refined the definition to formal meetings organized at national level.

Data availability

The lack of available data was the most significant challenge. It is worth reiterating that all figures contained in the 2021 Partners’ HPSR Report, as collected by country teams, are estimates only and that the estimates were to a great extent shaped by data that were non-existent, non-disaggregated or non-disclosed. Therefore, caution must be urged in the interpretation of the figures of each country profile. However, while these data may not reflect the comprehensiveness of the state of HPSR across countries, their imperfections do reinforce and add urgency to the call for:

• sharper definitions of HPSR within countries that delineate which activities are (or not) included;
• information systems (within institutions and across the country) that consistently capture HPSR-related indicators are needed to measure progress; and
• financing for HPSR that is committed and benchmarked, backed by continuous advocacy for the use and value of domestic resources.

We encountered four main challenges in attempting to retrieve the data required for this Report: the lack of publicly available data; the lack of data on financing; the lack of data on media engagement; and the short timeframe for data collection.

First, publicly available data for national-level resources were accessed wherever possible, but many countries did not have any readily available information on national health research budgets. Where these did not exist, it was a challenge to estimate various sources and mechanisms of government funding for health research, for example as a subset of national health expenditure, or other research and development flows. This points to the fact that in many countries, notions of a ‘national health research system’ are relatively new, and the many actors, institutions and disciplines that need to come together to form such a system have yet to cohere. Leading on from this, most countries did not have specific budget lines for HPSR. In some cases, HPSR activities might also be embedded within disease-specific health programmes, making it difficult to identify them on their own.

Second, and most significantly, it was difficult to trace financing data related to academic and institutional capacities. In particular, many institutions conducting HPSR are not solely dedicated to HPSR, meaning that institutional budgets are rarely disaggregated for HPSR-specific activities, or the amounts allocated to HPSR are not always significant enough to identify. One challenge, for example, was that many academic institutions have shared budgets and HPSR research groups are often part of broader institutional units. As a result, they do not have their own disaggregated budgets and share staff costs with other units, or cross-subsidize HPSR activities from general budgets.

In addition, sizeable amounts of academic institution budgets are co-financed from the institution by not charging faculty salaries, but rather counting them as in-kind contributions against external sources. In many cases, institutions did not have budget lines for HPSR because the bulk of funding came from external sources, and was not, therefore, counted within the institutional budget. These barriers to the identification of financing data may result in the underestimation of HPSR financing in-countries. It should be noted that challenges in accessing institutional budget data arose where it was not publicly available, and related to data sensitivity, data accessibility and reporting ability. Where institutional data were easily accessed, this was because of trusted relationships between the institutions being surveyed and the country teams.

Third, it was difficult to quantify data on media engagements. This was because the availability of and access to media tracking databases varies widely across countries and institutions, and very few institutions track their media activities. It was, therefore, a challenge to account for newspaper articles, radio and television appearances by staff. In addition, it was very difficult to unpack the content of media appearances and, in some cases, determine their relevance to HPSR. 

---

Limitations

In the 2021 Partners’ HPSR Report, identifying published papers where the first or last author was institutionally based within the country is likely to under-represent the full complement of papers relevant to the country’s HPSR knowledge generation. This excludes papers with country-based authors who are not first or last co-authors, as well as papers generated by country nationals based at institutions outside of the country.

Our original desire was to conduct country-specific bibliometric analyses of HPSR peer-reviewed articles. However, the lack of a global consensus on a standardized approach to conducting bibliometric analyses for HPSR studies meant that we excluded this for the 2021 Report. The Alliance recognizes the broader challenge of a lack of a standardized approach for conducting bibliometric analyses for HPSR studies (including the use of diverse search terms and strategies, common HPSR terms that are not catalogued as ‘MeSH’ terms, and more recent alternative metrics, such as AltMetric). This is an important area that needs urgent attention; the Alliance plans to convene an expert working group on this in the near future.

The exclusion of subnational meetings and reports is a limitation to the Report, especially in countries that are decentralized and thus undertake significant HPSR activities at local level.

The focus on national-level HPSR does not account for the contributions that countries make to global-level HPSR. In certain countries with well-established institutions that have been generating HPSR for many years, this is significant.

Finally, it should be noted that the information gathered for the Report was limited by the short timeframes in which data were collected. Data collection took place at a time when many institutions were trying to rebound from the initial waves of COVID-19, and therefore other priorities prevented them from fully engaging in the process. In addition, movement restrictions caused by the response to the COVID-19 pandemic further hindered much of the in-person data gathering needed.

---

Reflections and conclusions

This first edition of a collaborative Partners’ HPSR Report presents only a snapshot of the state of HPSR in a selection of countries. While the information captured in this Report is not exhaustive, it nonetheless conveys a compelling image of HPSR development and establishment in the countries that are included. Our hope is that data quality and availability will improve in the coming years and that more institutions and more countries will be featured in future iterations.

Nearly 20 years ago, the Alliance sought to document institutional capacities to fund and generate HPSR across select LMICs. While the methods used for this 2021 Report differ to those for the survey conducted by the Alliance in 2003 to assess LMIC-based HPSR knowledge production, stakeholder engagement and institutional capacity, the research teams encountered similar challenges when defining indicators and accessing relevant data.

Nevertheless, it is possible to draw some tentative conclusions on the evolution of the HPSR field since 2003. There has been little or no change in some areas: in 2003, financing for HPSR in LMICs was deemed to be very low, with the greater proportion (approximately 60%) coming from international sources, and a negligible amount coming from domestic spending as a share of total health expenditure. The 2021 Report finds that this is still the case.

The 2021 Partners’ HPSR Report also affirms findings from both the 2012 WHO Strategy on HPSR, (World Health Organisation, 2012), and the 2017 World Report on HPSR. Significant progress appears to have been made on increasing capacities in countries to support HPSR teaching and training. The 2017 report noted that, while much of the focus on HPSR progress had taken a global view, there were tensions between global and national preoccupations that shape HPSR, and a need to balance universal relevance with country specificity (Bennett et al., 2018). Linked to this, the 2021 Partners’ HPSR Report reveals several important distinctions in relation to how HPSR is organized in countries. Much of the debate about HPSR to date has been about its boundaries and terrain, in terms of the kinds of questions it asks and the accompanying methods and approaches to answer those questions (Sheikh et al., 2011). There has been less focus on the importance of specific contexts for HPSR, for the way in which HPSR knowledge generation, public and policy engagement, institutional capacity and funding take place in countries. This Report demonstrates that there are wide variations in how HPSR is organized in countries, despite the broadly accepted definition of what HPSR is – and in this case, the collective agreement on the indicators for its measurement. HPSR is often shaped by a country’s political, administrative, historical and fiscal contexts, as well as the structure, finance and staffing of its constellation of education, policy and research institutions and their capacity to provide training in, and conduct, HPSR. This implies the need for further work within countries to define and measure their national and subnational HPSR progress to ensure the necessary precision of HPSR indicators; global standards may be broadly useful, but lack the specificity required.

We did not examine in-depth whether the location of HPSR within institutions supports or challenges the applied and interdisciplinary nature of HPSR. There may, for example, be qualitative differences in the way it is practiced within an academic department of management or epidemiology, compared to its practice within a policy unit of a think tank. This consideration has been raised elsewhere (Bennett et al., 2011) and deserves further study.

In addition, we recognize the need to broaden the scope of what is considered HPSR. This Report did not cover activities such as community engagement projects, or advocacy, and it is important to ensure that robust mechanisms are in place to capture and measure these activities. Another factor in the variations observed is the maturity of the institutions involved. In general, for example, we found that a focus on HPSR-related short courses was less relevant in more established settings where HPSR may have greater recognition and that can, therefore, offer full academic post-graduate training, but this may not be the case in all settings. This signals the dynamic evolution of HPSR institutions and – once again – the need to consider measures of HPSR growth that can account for these changes over time.

Finally, while we did not collect data on COVID-19 specifically, it is impossible to discuss the current state of HPSR without mentioning the impact of the pandemic. Notably, the pandemic appears to have had a dual effect on HPSR generation within countries. On the one hand, it may well have hindered institutional capacities to train and conduct research as a result of restrictions on in-person gatherings and mobility, and it put pressure on research budgets in some countries as funding was shifted towards the COVID-19 response. On the other hand, however, there seems to have been an increase in meetings and media appearances, with experts called upon to provide evidence or comment on unfolding national COVID-19 responses (though it is unclear the degree to which this was HPSR-specific). Ongoing monitoring will be needed to understand the medium to long-term effects of COVID-19 on HPSR, nationally and globally.

---