Skip to main content

Adherence to metformin in adults with type 2 diabetes: a combined method approach

Abstract

Background

Medication adherence, one of the most important aspects in the process of optimal medicines use, is unfortunately still a major challenge in modern healthcare, and further research is required into how adherence can be assessed and optimised. The aim of this study was to use a combined method approach of self-report and dried blood spot (DBS) sampling coupled with population pharmacokinetic (PopPK) modelling, to assess adherence to metformin in adult patients with type 2 diabetes. Further aims were to assess metformin exposure levels in patients, determine factors associated with non-adherence with prescribed metformin, and to explore the relationship between adherence and therapeutic outcomes.

Methods

A combined method approach was used to evaluate metformin adherence in patients with type 2 diabetes who had been prescribed metformin for a minimum period of 6 months. Patients were recruited from consultant-led diabetic outpatient clinics at three hospitals in Northern Ireland, UK. Data collection involved self-reported questionnaires [Medication Adherence Report Scale (MARS), Beliefs about Medicines Questionnaire and Centre for Epidemiologic Studies Depression Scale], direct measurement of metformin concentration in DBS samples, and researcher-led patient interviews. The DBS sampling approach was coupled with population pharmacokinetic (PopPK) modelling, which took account of patient characteristics, metformin dosage and type of formulation prescribed (immediate or sustained release).

Results

The proportion of patients considered to be adherent to their prescribed metformin, derived from self-reported MARS scores and metformin concentration in DBS samples, was 61.2% (74 out of 121 patients). The majority (n = 103, 85.1%) of recruited patients had metformin exposure levels that fell within the therapeutic range. However, 17 patients (14.1%) had low exposure to metformin and one patient (0.8%) had undetectable metformin level in their blood sample (non-exposure). Metformin self-administration and use of a purchased adherence pill box significantly increased the probability of a patient being classified as adherent based on logistic regression analysis. Both HbA1c and random glucose levels (representing poor glycaemic control) in the present research were, however, not statistically linked to non-adherence to metformin (P > 0.05).

Conclusions

A significant proportion of participating patients were not fully adherent with their therapy. DBS sampling together with the use of a published PopPK model was a useful, novel, direct, objective approach to estimate levels of adherence in adult patients with type 2 diabetes (61.2%).

Background

Good medication adherence is one of the most important aspects of medicines optimisation and a critical element in the medicines use process. It is one of the expected outcomes of the effective delivery of medicines optimisation [1]. Patients with type 2 diabetes who cannot achieve glycaemic control through lifestyle modification (e.g. diet and exercise) require pharmacological treatment such as oral antidiabetic agent(s) (OAAs) and/or insulin treatment. These treatments are not only important for glycaemic control but also for preventing or delaying diabetes complications [2, 3]. Unfortunately, poor medication adherence remains an important issue in diabetes patients and is a key factor in the failure to achieve treatment goals. It is associated with poor glycaemic control, increased incidence of hospital admissions, long-term complications, increased mortality rates and increased healthcare costs [4,5,6,7]. A systematic review which included 27 studies has reported that type 2 diabetes medication adherence ranged between 38.5 and 93.1% [4].

The cause of poor adherence to diabetes medication is often multifactorial. Systematic reviews of therapy adherence in patients with type 2 diabetes reported that dose regimen complexity, polypharmacy, the need to use injectable medications, and associated side-effects (e.g. hypoglycaemia, weight gain, cardiovascular problems) can all influence adherence. Perceptions relating to treatment safety and efficacy (necessity belief and concerns about medication), depression, economic considerations and the relationship between the patient and healthcare provider have also been identified as factors which can influence medication adherence [4, 8,9,10]. A number of studies assessing adherence to diabetes medications in patients with type 2 diabetes have reported that metformin has the lowest adherence rates when compared with other OAAs [11,12,13,14,15]. Published pharmacoepidemiological studies, which have assessed the influence of metformin adherence on patient outcomes, have recorded adherence rates ranging between 22.0% and 88.6% [11,12,13,14,15,16,17,18]. The research to date has involved indirect assessments of adherence to metformin therapy, which can overestimate or underestimate true adherence. It is strongly recommended to use a number of different approaches to assess medication adherence, and to triangulate the results obtained. This multi-method approach increases the validity and reliability of adherence data [19, 20].

Quantification of medicines and/or metabolites in dried blood spot (DBS) samples is a new direct approach to adherence assessment [21]. This method utilises very small volumes of blood (typically 15 μL) usually taken from a finger prick. It is a more convenient way of blood sampling especially in children, older people and psychiatric patients. Alternatively, DBS samples can be obtained via an aliquot of venous blood taken for another purpose. Samples collected are spotted onto, and dried on, an absorbent card (Guthrie card). When dried, this card can be easily handled, transported to the laboratory, stored without refrigeration and analysed, typically using high-performance liquid chromatography (HPLC) [21,22,23,24,25,26]. A validated approach for the determination of metformin in DBS samples has been developed within our laboratory [22].

Population pharmacokinetics (PopPK) is the pharmacokinetic study of a specific drug within a target patient population. The aim of PopPK studies is to build a model that relates drug dosage and covariates to blood concentrations [27,28,29]. Typical examples of covariates studied include patient demographics (e.g. age, sex), physiological and pathological factors (e.g. organ function), environmental factors (e.g. diet, smoking) and drug interaction [28]. PopPK modelling mainly uses a non-linear mixed effects modelling approach [30]. Available PopPK models for a target population can be used to provide estimates of expected plasma concentrations and can be applied to assess adherence by comparing actual with expected blood concentrations [21, 31, 32].

It is important to monitor adherence to metformin to prevent unneeded dose increases or the prescription of additional or replacement diabetes medication for those who are poorly adherent [22].

The aim of the present study was to evaluate adherence to metformin in patients with type 2 diabetes attending outpatient clinics in the South Eastern Health & Social Care Trust, Northern Ireland, using a combined method approach i.e. self-report and dried blood spot (DBS) sampling coupled with population pharmacokinetic (PopPK) modelling. Further aims were to assess metformin exposure levels in patients, determine factors associated with non-adherence with prescribed metformin, and to explore the relationship between adherence and therapeutic outcomes.

Methods

Study patients and study sites

Patient recruitment for the study was carried out at consultant-led diabetic outpatient clinics at the Ulster, Ards and Bangor Hospitals within the South Eastern Health & Social Care Trust, Northern Ireland.

Patients who attended a diabetic outpatient clinic at a study site hospital and who were aged ≥ 18 years, diagnosed with type 2 diabetes mellitus and prescribed metformin for a minimum period of 6 months were invited to join the study. Patients were excluded from the study if they were unable to give written informed consent, e.g. Alzheimer’s disease or learning disability, unable to communicate/complete study paperwork in English.

Patients interested in the study were given a proposed date for attending a research session at the hospital or if participants expressed a preference for the research session to be conducted on the same day during their visit to the hospital, this was accommodated.

Sample size

Based on the assumption that adherence to medication in diabetes patients was approximately 50%, 196 subjects were needed to accurately estimate the percentage of medication adherence. This sample size provides 95% certainty that the estimate is within ± 7% of the exact population proportion [33]. We planned to recruit a target number of 220 patients to mitigate against missing data.

Sampling method

Patients were recruited using convenience sampling, as described above, taking account of the study inclusion and exclusion criteria.

Administration of self-completed questionnaires and collection of dried blood spot (DBS) samples

The Medication Adherence Report Scale (MARS) [34], the Beliefs about Medicines Questionnaire (BMQ) [35] and The Centre for Epidemiologic Studies Depression Scale (CES-D) [36] were given by the researcher (NFS) to each patient during a research visit to the hospital, and they were asked to complete these. The total MARS score ranges from 5 to 25. Higher scores represent better adherence levels. In this study, a cut‐off point of 90% [37] was used and the patients who had total scores ≥ 23 were considered adherent. The total BMQ scores of the necessity scale and the concern scale range from 5 to 25. Higher scores for the necessity scale are associated with greater patient perceived need for their medication to control their disease and maintain their health. On the other hand, higher scores for the concern scale are associated with higher patient concerns about harmful or adverse effects that are possible from long-term use of the medicine in question and worries regarding becoming dependent on their medications [35]. The CES-D was used as a tool for screening symptoms of depressed mood which can contribute to medication non-adherence. The CES-D score ranges from 0 to 60, with higher scores associated with greater depressive symptoms. A score of 16 or greater indicates depressed mood [36].

Having completed the questionnaires, a finger prick blood sample (automatic, disposable lancet) was taken by the researcher (NFS) from each patient and used to spot a Guthrie card with 15-uL aliquot using a disposable pipette. Date and time of collection were recorded for each sampling time. The blood spot samples were dried overnight in the dark at room temperature before storage at − 80 °C prior to analysis. As part of the process for the determination of metformin in DBS samples, haematocrit (Hct) values in the finger prick samples were measured during the research visit using Hemo_Control®.

Quantification of metformin in DBS samples

The DBS samples were analysed for content of metformin using HPLC with UV detection, based on a validated method developed in house [22]. The method utilised 8‐mm disks (incorporating the full 15‐μL sample) punched from the Guthrie cards. Preparation of the DBS samples involved extraction using 1 mL of methanol. After centrifugation, the supernatant was transferred to a glass tube and dried under nitrogen at 40 °C for 30 min. This dried extract was then reconstituted with 120 μL of pure deionised water, vortexed for 5 min and then subjected to HPLC with UV detection (236 nm). The assay limit of quantification in DBS samples was 0.075 μg/mL. The intraday and interday accuracy and precision were within the limits recommended by International Council for Harmonisation (ICH) guidelines (± 15%).

Population pharmacokinetic (PopPK) modelling

Prediction of metformin plasma concentrations associated with full adherence was based on a PopPK model developed by Bardin et al. [38]. To determine whether the patients were adherent, a computer simulation method was utilised to estimate the 95% interval of predicted plasma concentrations for metformin at the time of sampling relative to time of dose administration (n = 1000 sets of simulations using the non-linear mixed effect modelling software package, NONMEM version 7.4.4; Icon Development Solutions, Ellicott City, MD, USA).

Literature values of PopPK parameters for metformin were employed [38, 39]. In addition, significant covariates reported to influence PK parameters (e.g. age, lean body weight, daily dose and serum creatinine) were incorporated into the simulation models. For each patient, the measured DBS concentrations were transformed to corresponding plasma concentrations based on each individual patient's haematocrit level.

Patients with an undetectable metformin concentration or a calculated plasma concentration that fell outside the 95% confidence interval of the predicted full adherence plasma level [based on daily dose, time of last dose and time of blood spot collection] as determined by PopPK modelling [38], were classified as non-adherent. Patients were considered adherent if their metformin equivalent plasma concentration fell within the calculated 95% prediction intervals (2.5th and 97.5th percentiles) [21].

Metformin equivalent plasma concentrations were also used to assess the metformin exposure level by relating this value to the therapeutic range for metformin in plasma. If metformin plasma concentration was lower or higher than the metformin therapeutic range (1–5 μg/mL) [40,41,42], this was categorised as low or high exposure, respectively. Metformin levels below the limit of quantification in DBS samples were categorised as non-exposure.

Collection of routine patient data

Demographic data (age, weight, height, sex and ethnic origin), medical history, current medications (and dosage regimes), and biochemical data (including serum creatinine, blood sugar levels and HbA1c levels) were recorded for each enrolled patient. These data were obtained from each patient’s medical records. All participating patients were interviewed by the researcher about their metformin and its management (e.g. dose regimes, how to remember to take medicine, any problems with medicine administration and potentially side-effects) using a study guide.

Combined data analysis

Adherence for individual patients was determined based on combination of data derived from the concentration of metformin in DBS samples, together with the total score achieved from the MARS questionnaires. A patient was considered non-adherent if the MARS or the plasma level indicated that the patient was not following the prescribed dosage schedule [21].

Statistical analyses were carried out using SPSS version 25.0, SPSS Inc, USA. Descriptive statistics were used to describe patient demographics/characteristics and percentage of adherent and non-adherent patients. The Kappa coefficient (κ) was used to assess the extent of agreement between the adherence assessment methods [43].

Parametric and non-parametric tests were conducted as appropriate based on the normality of the data distribution. Adherent/non-adherent group differences were analysed using Chi-square analysis or Fisher’s exact test for categorical variables and t-test analysis for continuous variables. Spearman’s bivariate correlation was utilised to determine the relationship between adherence and continuous variables. Logistic regression analysis was also performed to determine the relative contribution of patient factors, including matters raised by patients during the patient interviews, to adherence with prescribed metformin.

Results

A total of 277 patients met the inclusion criteria and 121 patients were recruited (43.6%). The total number of patients recruited was lower than anticipated since recruitment had to be terminated prematurely as a result of the cancellation of face-to-face outpatient clinics due to the COVID-19 pandemic. The pandemic also did not allow planned home DBS sampling to be implemented. Patient recruitment flow is presented in Fig. 1. A considerable number of patients (n = 125) were not referred by the clinicians to the researcher.

Fig. 1
figure 1

Flowchart of patient recruitment

Patient characteristics

Characteristics of the 121 patients recruited are presented in Table 1.

Table 1 Patient characteristics

On average, recruited patients had been living with type 2 diabetes for approximately 15 years and took, on average, three antidiabetic medicines. Almost all (98.3%) patients had other medical conditions and the mean ± SD number of all prescribed medicines was 10.8 ± 4.4. The mean ± SD levels of study population HbA1c and random glucose levels were 67.9 ± 17.9 mmol/mol (8.4 ± 3.8%) and 9.9 ± 4.1 mmol/L, respectively.

Metformin regimens

More than half (60.3%) of recruited patients took immediate-release metformin tablets. Patients generally took metformin twice a day (79.3%) at a dosage level of 2000 mg per day (85.1%) (Table 2). During the interviews, it was observed that approximately half (n = 62, 51.2%) of recruited patients were taking metformin before meals instead of after meals (recommended direction).

Table 2 Metformin regimes taken by recruited patients

Adherence based on self-reported MARS

All participating patients completed the MARS questionnaire. The questionnaire revealed that 110 patients (90.9%) scored ≥ 90% (23 points or greater) i.e. described themselves as adherent. Unintentional non-adherence was reported by 13 patients, who reported forgetting to take metformin at least sometimes, often, or always (Fig. 2) over a month.

Fig. 2
figure 2

Number of patients who sometimes, often, or always engaged in non-adherent behaviours to metformin stated in the MARS questionnaire

Beliefs about medicines and depressed mood

Three-quarters of patients (75.2%) had a BMQ necessity score that was above the scale midpoint, indicating that they had a strong belief that metformin was necessary for managing diabetes, while just under one-third of patients (31.4%) had concerns regarding the potential harmful effects of metformin prescribed for them. The mean ± SD total scores for the necessity and concern subscales were 17.7 ± 3.4 and 13.4 ± 3.6, respectively.

The majority of patients (n = 83, 68.6%), as measured by the CES-D questionnaire, scored < 16 indicating that they did not have a depressed mood or had a mild depressed mood. However, 18 patients (14.9%) had a moderate depressed mood, and 20 patients (16.5%) had a total score of > 23 which was indicative of severe depressed mood.

Adherence based on metformin concentration in DBS samples

DBS samples were obtained from all recruited patients (n = 121) during their attendance at the hospital-based research session. According to the DBS sampling method, 75 patients (62.0%) were classified as adherent, whereas 46 patients (38.0%) were classified as non-adherent. From the non-adherent patient group, a total of 27 and 19 patients had metformin levels below and higher than the predicted plasma concentrations respectively. Examples of the PopPK model derived data for participating patients who were taking immediate-release and sustained-release formulations of metformin are shown in Additional file 1: Fig. S1 and Additional file 2: Fig. S2.

The majority (n = 103, 85.1%) of recruited patients had metformin exposure levels that fell within the therapeutic range. However, 17 patients (14.1%) had low exposure to metformin and one patient (0.8%) had metformin levels below the limit of quantification (non-exposure) (Fig. 3).

Fig. 3
figure 3

Level of metformin exposure in recruited patients

Comparison of the different measures of adherence

As described earlier, a patient was classified as non-adherent if designated as such by one or both of the adherence measures (MARS and/or DBS approach) [21]. After recruited patients were categorised into adherent and non-adherent, the Kappa coefficient (κ) was utilised to assess the extent of agreement between adherence assessment methods (MARS and DBS samples). There was only a slight agreement between both measures since κ was 0.157.

The assessment method involving metformin concentration in blood captured higher percentages of non-adherence (38.0%) when compared with MARS (9.1%). Upon combining two adherence assessment methods, a total of 74 (61.2%) patients were considered adherent overall (Fig. 4).

Fig. 4
figure 4

Comparison of results of metformin adherence classification of adult patients with type 2 diabetes using the different methods of assessment. MARS: medication adherence report scale; DBS: dried blood spot

Linking non-adherence to patient outcomes

HbA1c and random glucose levels are deemed to be clear markers of poor clinical outcomes in patients with type 2 diabetes. Both HbA1c and random glucose levels representing poor glycaemic control in the present research were, however, not statistically linked to overall non-adherence to metformin (P > 0.05) (Table 3). These outcome parameters were also not statistically linked to non-adherence assessed by individual methods (MARS and DBS). In addition HbA1c and random glucose levels were not statistically linked to the level of exposure to metformin (Table 4).

Table 3 HbA1c and random glucose levels as outcomes for non-adherence
Table 4 HbA1c and random glucose levels as outcomes for exposure

Factors that influence adherence

Univariate analysis was performed for available variables in patients who were overall adherent and non-adherent to metformin (Table 5). Two variables were significantly associated with adherence to metformin (P < 0.05) using univariate analysis. These factors were (i) metformin self-administration (i.e. not being dependant on others to help with medication administration) and (ii) patient utilising a purchased adherence pill box. Logistic regression analysis indicated that both these factors were significantly and independently associated with adherence. Metformin self-administration and use of a purchased adherence pill box increased the probability of a patient being classified as adherent by 7.5 and 2.8 times, respectively (Table 6).

Table 5 Univariate analysis of possible factors affecting metformin adherence
Table 6 Factors independently linked to metformin adherence using logistic regression analysis

Discussion

Since medication adherence is best assessed using a number of different approaches and triangulating the results obtained, the present study used a combination of approaches to increase the validity and reliability of the adherence data collected [19, 20]. It also demonstrated the utility of the finger prick DBS sampling approach to assess metformin adherence in adult patients.

Intraindividual and interindividual variability related to measured concentrations in blood can, however, be a limitation in the use of blood levels for adherence assessment [44]. In the present study, the application of PopPK models was used to minimise these uncertainties, and to increase the robustness of the overall approach. The PopPK of metformin in adults has been reported in previous research [38, 39, 45]. The PopPK model developed by Bardin et al. [38] was selected as the most suitable for use in the present study population, because the clearance model includes age, lean body weight and serum creatinine level as covariates—all of which are of particular relevance to the present population. This model was appropriate for immediate-release formulations and also works well for sustained-release formulations [39]. Concomitant use of some medications such as bictegravir, cimetidine, dolutegravir, guanfacine, mexiletine, pitolisant, ribociclib, telaprevir, topiramate and vandetanib [46] can increase metformin concentrations, but in this present research no patients were taking any of these medications.

The proportion of patients in this study who were adherent measured by the MARS (90.9%) was approximately the same as the proportion of adherent patients reported in a previous study of diabetes medication (89.5%) where the MARS score of 23 was used as the cut off to categorise patients as adherent [37]. Overestimation of the adherence rate may, however, result from patient reluctance and unwillingness to disclose their non-adherent behaviour or report not taking medicines as prescribed [47, 48].

In the present study, the proportion of patients who were classified as being adherent to metformin using the DBS approach was much lower than that reported using the MARS. Some of the patients were classified using the current DBS/PopPK approach as over-adherent, i.e. had higher blood levels than estimated by the PopPK modelling. This could indicate a shortcoming in the modelling or “white coat adherence” in which a patient may have taken an extra dose of metformin, i.e. exceeded the prescribed amount, prior to their visit to the research session [49,50,51].

Regarding the exposure to metformin, only one patient had undetectable metformin level in their blood sample (non-exposure). This indicated that the patient had discontinued metformin.

In this present research, the HbA1c levels of those patients classified as over-adherent were higher, although not statistically significant, (mean ± SD: 69.6 ± 21.1 mmol/mol) than in adherent patients (mean ± SD: 66.3 ± 17.6 mmol/mol), highlighting the possibility of patients taking more metformin than prescribed before coming to the clinic to mask previous non-adherence [49,50,51]. This has been reported by previous studies in patients with diabetes [52,53,54].

Comparison of the different measures of adherence

The proportion of patients considered overall adherent to metformin, derived from the subjective (self-reported MARS) and objective (metformin concentration in DBS samples) measures, was 61.2%, which is within the internationally reported range of the proportion of patients adherent to metformin in a type 2 diabetes population [11,12,13,14,15,16,17,18, 55, 56]. A study assessing medication adherence using a combination of subjective (self-reported questionnaire) and objective (pill counts) measures in adult patients receiving OAAs, lipid lowering, antihypertensive and/or antiplatelet agents, reported that 81.3% of patients were classified as adherent based on self-reported questionnaire results, while only 35.4% of patients were classified as adherent based on pill counts [57].

Linking non-adherence to clinical outcomes

In the present study, significant correlations (P < 0.05) were not found between adherence classification and HbA1c or random glucose concentrations. The association between adherence to diabetes medication and glycaemic control has been reported in many published studies [5, 7, 17, 58,59,60]. In the present study, although not statistically significant, non-adherent patients had slightly higher levels of HbA1c (mean ± SD: 69.1 ± 17.5 mmol/mol) compared with adherent patients (mean ± SD 67.1 ± 18.2 mmol/mol).

The level of metformin exposure was also not statistically linked to both HbA1c and random blood glucose levels. The mean levels of HbA1c in patients who were exposed to metformin within its therapeutic range were considered high (67.5 mmol/mol). However, the fact that almost all patients were prescribed more than one diabetes medication at the time of their engagement with the study and that patients may have had considerable variation in adherence with lifestyle measures recommended in diabetes management, add additional challenges to clinical outcome data interpretation. What is clear is that metformin adherence or exposure do not appear to be major drivers of glycaemic control in this patient population.

Factors influencing adherence

Metformin self-administration and the patient reporting their use of a purchased adherence pill box were variables that independently (P < 0.05) predicted metformin adherence in the present study (logistic regression modelling). Metformin self-administration in the present research refers to patients themselves being responsible for the storage and administration of their own metformin, without any help from others. Positive associations between metformin self-administration and adherence to metformin could be linked to the longer duration of diabetes. The majority of patients who self-administered had been living with diabetes for a long duration indicating they had good knowledge about their metformin and its dose regimes. A large study assessing determinants of diabetes medication adherence reported that good medication adherence was associated with longer diabetes duration [61]. The Capability, Opportunity and Motivation (COM) B model of behaviour developed by Michie et al. [62] shows that to undertake medication-taking, individuals should have the physical and psychological capability as well motivation to do so; in particular, to adhere to the instructions related to their medication. Capability and motivation can be achieved through increasing knowledge and understanding.

The use of a purchased adherence pill box was linked with adherence in the present research. Systematic reviews have found evidence that pillboxes are an effective intervention to address medication adherence issues [63,64,65]. They can be an effective strategy to improve medication adherence as they assist patients to remember to take their medicines and simplify complicated medication regimens [66, 67].

The present study did not confirm the hypothesis that necessity belief about metformin was positively associated with metformin adherence, or that concerns about metformin was negatively associated with adherence (as measured by MARS or DBS sampling method individually or combined). Other studies, including systematic reviews on medication adherence in patients with type 2 diabetes, have reported that believing that medications are important or necessary was positively associated with medication adherence, while having concerns about the medication negatively impacted adherence [8, 68, 69]. The majority of patients (75.2%) in the present study had a BMQ necessity score above the scale midpoint, indicating that they had a strong belief that metformin was necessary for managing diabetes, while about only one-third of patients (31.4%) had concerns regarding the potential harmful effects of metformin prescribed for them. This indicates that medical team have done a good job in this regard.

Again, there was no significant association between the CES-D score and non-adherence to metformin. A number of previous studies have found that depression is a predictor for non-adherence to diabetes medication [4, 70,71,72,73]. Patients with diabetes have been found to be 1.4–3 times more likely to have depression compared to non-diabetic patients. The prevalence of major depression in patients with DM is mostly estimated around 12% (ranging from 8 to 18%) [73]. Compared with the latter study results, the number of patients in the present study who had a moderate-to-severe depressed mood as measured by the CES-D questionnaire (31.4%) was considered high.

Limitations

The present study had a number of limitations. Firstly, the sample size was smaller than planned, due to the COVID-19 pandemic cutting short recruitment, thus reducing the statistical power of the study. The pandemic also did not allow planned home DBS sampling to be implemented. Furthermore, there is a risk of recall bias associated with the self-reported MARS scores, which may impact the high adherence revealed using this measure compared to that reported with DBS samples. Since adult patients with type 2 diabetes are usually prescribed a combination of hypoglycaemic agents, the clinical outcomes evaluated in the present study, i.e. HbA1c and random glucose concentrations could be associated with the use of other diabetes medications and of course adherence to the advice on diet and exercise provided to all type 2 diabetic patients. Adherence assessed by DBS sampling during research visit and the MARS over a month might not be reflective of cumulative HBA1C over the last 3 months.

Conclusions

DBS sampling together with the use of a published PopPK model to estimate metformin concentrations was shown to be a useful and novel, direct, objective approach to estimate levels of adherence in adult patients with type 2 diabetes. Being classified as non-adherent to metformin did not significantly influence clinical outcomes such as HbA1c and random glucose level. Metformin self-administration and use of a purchased adherence pill box by a patient were associated with increased probability of a patient being classified as adherent based on logistic regression analysis. The results suggest that attention should be given to the education of patients receiving metformin, with involvement of a caregiver as needed, to ensure that the patient is capable of self-administration of medication and recommend use of an adherence pill box. Further investigations of adherence utilising the DBS method should be carried out for other diabetes medications.

Availability of data and materials

The data used to support the findings of this study are included in this published article.

References

  1. Royal Pharmaceutical Society Medicine optimisation: Helping patient to make the most of medicine. 2013. https://www.rpharms.com/promoting-pharmacy-pdfs/mo---evidence-in-practice.pdf. Accessed 26 May 2016

  2. American Diabetes Association (ADA). 6. Glycemic targets: standards of medical care in diabetes—2018. Diabetes Care. 2018;41:S55–64. https://doi.org/10.2337/dc18-S006.

    Article  Google Scholar 

  3. National Institute for Health and Clinical Excellence. Managing blood glucose in adults with type 2 diabetes. 2019a. https://pathways.nice.org.uk/pathways/type-2-diabetes-in-adults/managing-blood-glucose-in-adults-with-type-2-diabetes. Accessed 25 Sep 2019.

  4. Krass I, Schieback P, Dhippayom T. Adherence to diabetes medication: a systematic review. Diabet Med. 2015;32:725–37. https://doi.org/10.1111/dme.12651.

    Article  PubMed  CAS  Google Scholar 

  5. Egede LE, Gebregziabher M, Echols C, Lynch CP. Longitudinal effects of medication nonadherence on glycemic control. Ann Pharmacother. 2014;48:562–70.

    Article  Google Scholar 

  6. Khunti K, Seidu S, Kunutsor S, Davies M. Association between adherence to pharmacotherapy and outcomes in type 2 diabetes: a meta-analysis. Diabetes Care. 2017;40:1588–96. https://doi.org/10.2337/dc16-1925.

    Article  PubMed  Google Scholar 

  7. Rodgers LR, Weedon MN, Henley WE, Hattersley AT, Shields BM. Cohort profile for the MASTERMIND study: using the Clinical Practice Research Datalink (CPRD) to investigate stratification of response to treatment in patients with type 2 diabetes. BMJ Open. 2017;7: e017989. https://doi.org/10.1136/bmjopen-2017-017989.

    Article  PubMed  PubMed Central  Google Scholar 

  8. García-Pérez LE, Alvarez M, Dilla T, Gil-Guillén V, Orozco-Beltrán D. Adherence to therapies in patients with type 2 diabetes. Diabetes Ther. 2013;4:175–94. https://doi.org/10.1007/s13300-013-0034-y.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Capoccia K, Odegard P, Letassy N. Medication adherence with diabetes medication a systematic review of the literature. Diabetes Educ. 2016;42:34–71.

    Article  Google Scholar 

  10. McGovern A, Tippu Z, Hinton W, Munro N, Whyte M, de Lusignan S. Comparison of medication adherence and persistence in type 2 diabetes: a systematic review and meta-analysis. Diabetes Obes Metab. 2018;20:1040–3.

    Article  CAS  Google Scholar 

  11. Shenolikar RA, Balkrishnan R, Camacho FT, Whitmire JT, Anderson RT. Race and medication adherence in medicaid enrollees with type-2 diabetes. J Natl Med Assoc. 2006;98:1071–7.

    PubMed  PubMed Central  Google Scholar 

  12. Haupt D, Weitoft GR, Nilsson JL. Refill adherence to oral antihyperglycaemic drugs in Sweden. Acta Diabetol. 2009;46:203–8. https://doi.org/10.1007/s00592-008-0076-1.

    Article  PubMed  CAS  Google Scholar 

  13. White AJ, Kellar I, Prevost AT, Kinmonth AL, Sutton S, Canny M, Griffin SJ. Adherence to hypoglycaemic medication among people with type 2 diabetes in primary care. Prim Care Diabetes. 2012;6:27–33.

    Article  Google Scholar 

  14. Quilliam BJ, Ozbay AB, Sill BE, Kogut SJ. The association between adherence to oral anti-diabetic drugs and hypoglycaemia in persons with type 2 diabetes. Diabet Med. 2013;30:1305–13.

    Article  CAS  Google Scholar 

  15. Lee CS, Tan J, Sankari U, Koh Y, Tan NC. Assessing oral medication adherence among patients with type 2 diabetes mellitus treated with polytherapy in a developed Asian community: a cross-sectional study. BMJ Open. 2017;7: e016317. https://doi.org/10.1136/bmjopen-2017-016317.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Martono D, Hak E, Heerspink H, Wiffert B, Denig P. Predictors of HbA1c levels in patient initiating metformin. Curr Med Res Opin. 2016;32:2021–8.

    Article  CAS  Google Scholar 

  17. Farmer AJ, Rodgers LR, Lonergan M, Shields B, Weedon MN, Donnelly L, Holman RR, Pearson ER, Hattersley AT. Adherence to oral glucose-lowering therapies and associations with 1-year HbA1c: a retrospective cohort analysis in a large primary care database. Diabetes Care. 2016;39:258–63. https://doi.org/10.2337/dc15-1194.

    Article  PubMed  CAS  Google Scholar 

  18. Min JY, Griffin MR, Chipman J, Hackstadt AJ, Greevy RA, Grijalva CG, Hung AM, Roumie CL. Recent metformin adherence and the risk of hypoglycaemia in the year following intensification with a sulfonylurea. Diabet Med. 2019;36:482–90. https://doi.org/10.1111/dme.13853.

    Article  PubMed  CAS  Google Scholar 

  19. Nunes V, Neilson J, O’Flynn N, Calvert N, Kuntze S, Smithson H, Benson J, Blair J, Bowser A, Clyne W, C Crome P, Haddad P, Hemingway S, Horne R, Johnson S, Kelly S, Packham B, Patel M, Steel J. Clinical guidelines and evidence review for medicines adherence: involving patients in decisions about prescribed medicines and supporting adherence. London: National Collaborating Centre for Primary Care and Royal College of General Practitioners. 2009. https://www.nice.org.uk/guidance/cg76. Accessed 1 Mar 2017.

  20. Lehmann A, Aslani P, Ahmed R, Celio J, Gauchet A, Bedouch P, Bugnon O, Allenet B, Schneider MP. Assessing medication adherence: options to consider. Int J Clin Pharm. 2013. https://doi.org/10.1007/s11096-013-9865-x.

    Article  PubMed  Google Scholar 

  21. Shah N, Hawwa A, Millership J, Collier P, Ho P, Tan M, Peake D, Tirupathi S, Bothwell J, Bailie N, Shepherd C, Craig J, McElnay JC. Adherence to antiepileptic medicines in children: a multiple-methods assessment involving dried blood spot sampling. Epilepsia. 2013;54:1020–7.

    Article  CAS  Google Scholar 

  22. AbuRuz S, Millership J, McElnay J. Dried blood spot liquid chromatography assay for therapeutic drug monitoring of metformin. J Chromatogr B. 2006;832:202–7.

    Article  CAS  Google Scholar 

  23. McDade TW, Williams S, Snodgrass JJ. What a drop can do: dried blood spots as a minimally invasive method for integrating biomarkers into population-based research. Demography. 2007;44:899–925.

    Article  Google Scholar 

  24. Suyagh M, Collier PS, Millership JS, Iheagwaram G, Millar M, Halliday HL, McElnay JC. Metronidazole population pharmacokinetics in preterm neonates using dried blood-spot sampling. Pediatrics. 2011;127:e367–734.

    Article  Google Scholar 

  25. Li P, Lee M, editors. Dried blood spots applications and techniques. Wiley Series on Pharmaceutical Science. Hoboken: John Wiley Sons Inc; 2014.

    Google Scholar 

  26. Hawwa AF, AlBawab A, Rooney M, Wedderburn LR, Beresford MW, McElnay JC. Methotrexate polyglutamates as a potential marker of adherence to long-term therapy in children with juvenile idiopathic arthritis and juvenile dermatomyositis: an observational, cross-sectional study. Arthritis Res Ther. 2015;17:295. https://doi.org/10.1186/s13075-015-0814-z.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  27. Ette EI, Williams PJ. Population pharmacokinetics I: background, concepts, and models. Ann Pharmacother. 2004;38:1702–6.

    Article  Google Scholar 

  28. Mould DR, Upton RN. Basic concepts in population modeling, simulation, and model-based drug development. CPT Pharmacometrics Syst Pharmacol. 2012;1: e6. https://doi.org/10.1038/psp.2012.4.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  29. Bonate PL. Pharmacokinetic-pharmacodynamic modelling and simulation. New York: Springer; 2006.

    Google Scholar 

  30. Charles B. Population pharmacokinetics: an overview. Aust Prescr. 2014;37:210–3.

    Article  Google Scholar 

  31. Zhang C, Denti P, van der Walt JS, Ren Y, Smith P, Karlsson MO, McIlleron H. Population pharmacokinetic model for adherence evaluation using lamivudine concentration monitoring. Ther Drug Monit. 2012;34:481–4.

    Article  CAS  Google Scholar 

  32. Almardini R, Taybeh EO, Alsous MM, Hawwa AF, McKeever K, Horne R, McElnay JC. A multiple methods approach to determine adherence with prescribed mycophenolate in children with kidney transplant. Br J Clin Pharmacol. 2019;85:1434–42.

    Article  CAS  Google Scholar 

  33. Daniel WW, Cross CL. Biostatistics: a Foundation for Analysis in the Health Sciences. 10th ed. New York: John Wiley & Sons; 2013.

    Google Scholar 

  34. Horne R, Weinman J. Self-regulation and self-management in asthma: exploring the role of illness perceptions and treatment beliefs in explaining non-adherence to preventer medication. Psychol Health. 2002;17:17–32.

    Article  Google Scholar 

  35. Horne R, Weinman J. Patients’ beliefs about prescribed medicines and their role in adherence to treatment in chronic physical illness. J Psychosom Res. 1999;47:555–67.

    Article  CAS  Google Scholar 

  36. Radloff LS. The CES-D scale a Self-Report Depression Scale for research in the general population. Appl Psychol Meas. 1977;1:385–401.

    Article  Google Scholar 

  37. Vluggen S, Hoving C, Schaper NC, De Vries H. Psychological predictors of adherence to oral hypoglycaemic agents: an application of the ProMAS questionnaire. Psychol Health. 2020;35:387–404. https://doi.org/10.1080/08870446.2019.1672873.

    Article  PubMed  Google Scholar 

  38. Bardin C, Nobecourt E, Larger E, Chast F, Treluyer JM, Urien S. Population pharmacokinetics of metformin in obese and non-obese patients with type 2 diabetes mellitus. Eur J Clin Pharmacol. 2012;68:961–8.

    Article  CAS  Google Scholar 

  39. Duong JK, Kumar SS, Kirkpatrick CM, Greenup LC, Arora M, Lee TC, Timmins P, Graham GG, Furlong TJ, Greenfield JR, Williams KM, Day RO. Population pharmacokinetics of metformin in healthy subjects and patients with type 2 diabetes mellitus: simulation of doses according to renal function. Clin Pharmacokinet. 2013;52:373–84.

    Article  CAS  Google Scholar 

  40. The International Association of Forensic Toxicologists (TIAFT).Reference blood level list of therapeutic and toxic substances. London: The Association; 1995–2015. 2004. http://www.tiaft.org. Accessed 10 Sep 2019.

  41. Food and Drug Administration. FDA label approved (revised) on 07/19/2013 for Metformin Hydrochloride, ANDA no. 091664. 2013. http://www.accessdata.fda.gov/drugsatfda_docs/label/2013/091664Orig1s000lbl.pdf Accessed 12 Sep 2020.

  42. Kajbaf F, De Broe ME, Lalau JD. Therapeutic concentrations of metformin: a systematic review. Clin Pharmacokinet. 2016;55:439–59. https://doi.org/10.1007/s40262-015-0323-x.

    Article  PubMed  CAS  Google Scholar 

  43. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.

    Article  CAS  Google Scholar 

  44. Paschal AM, Hawley SR, St Romain T, Ablah E. Measures of adherence to epilepsy treatment: review of present practices and recommendations for future directions. Epilepsia. 2008;49:1115–22. https://doi.org/10.1111/j.1528-1167.2008.01645.x.

    Article  PubMed  Google Scholar 

  45. Barranco Garduno LM, Neri Salvador JC, Juarez Olguin H, Lugo Goytia G, Carrasco Portugal M, Flores MF. Population pharmacokinetics of metformin in Mexican patients with type 2 diabetes mellitus. Int J Pharmacol. 2015;11:632–7.

    Article  CAS  Google Scholar 

  46. British National Formulary (BNF). 2019. https://www.pharmpress.com/assets/docs/bnf%2077_9780857113436.pdf

  47. Culig J, Leppée M. From Morisky to Hill-bone; self-reports scales for measuring adherence to medication. Coll Antropol. 2014;38:55–62.

    PubMed  Google Scholar 

  48. Lam WY, Fresco P. Medication adherence measures: an overview. Biomed Res Int. 2015. https://doi.org/10.1155/2015/217047.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Osterberg L, Blaschke T. Adherence to medication. N Eng J Med. 2005;353:487–97.

    Article  CAS  Google Scholar 

  50. Jimmy B, Jose J. Patient medication adherence: measures in daily practice. Oman Med J. 2011;26:155–9.

    Article  Google Scholar 

  51. Brown MT, Sinsky CA. Medication adherence: we didn’t ask and they didn’t tell. 2013. https://www.aafp.org/fpm/2013/0300/fpm20130300p25.pdf . Accessed 15 May 2019.

  52. Winkler A, Teuscher AU, Mueller B, Diem P. Monitoring adherence to prescribed medication in type 2 diabetic patients treated with sulfonylureas. Swiss Med Wkly. 2002;132:379–85.

    PubMed  Google Scholar 

  53. Driscoll KA, Wang Y, Johnson SB, Gill E, Wright N, Deeb LC. White coat adherence occurs in adolescents with type 1 diabetes receiving intervention to improve insulin pump adherence behaviors. J Diabetes Sci Technol. 2017;11:455–60. https://doi.org/10.1177/1932296816672691.

    Article  PubMed  Google Scholar 

  54. McConville A, Noser AE, Clements MA, Patton SR. Mealtime insulin BOLUS score increases prior to clinic visits in youth with type 1 diabetes. BMJ Open Diabetes Res Care. 2020;8: e001348. https://doi.org/10.1136/bmjdrc-2020-001348.

    Article  PubMed  PubMed Central  Google Scholar 

  55. Chong E, Wang H, King-Shier KM, Quan H, Rabi DM, Khan NA. Prescribing patterns and adherence to medication among South-Asian, Chinese and white people with type 2 diabetes mellitus: a population-based cohort study. Diabet Med. 2014;31:1586–93. https://doi.org/10.1111/dme.12559.

    Article  PubMed  CAS  Google Scholar 

  56. Horsburgh S, Barson D, Zeng J, Sharples K, Parkin L. Adherence to metformin monotherapy in people with type 2 diabetes mellitus in New Zealand. Diabetes Res Clin Pract. 2019;158: 107902. https://doi.org/10.1016/j.diabres.2019.107902.

    Article  PubMed  CAS  Google Scholar 

  57. Mateo JF, Gil-Guillen VF, Mateo E, Orozco D, Carbayo JA, Merino J. Multifactorial approach and adherence to prescribed oral medications in patients with type 2 diabetes. Int J Clin Pract. 2006;60:422–8.

    Article  CAS  Google Scholar 

  58. Tiv M, Viel JF, Mauny F, Eschwège E, Weill A, Fournier C, Fagot-Campagna A, Penfornis A. Medication adherence in type 2 diabetes: the ENTRED study 2007, a French Population-Based Study. PLoS ONE. 2012;7: e32412. https://doi.org/10.1371/journal.pone.0032412.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  59. McAdam-Marx C, Bellows BK, Unni S, Wygant G, Mukherjee J, Ye X, Brixner DI. Impact of adherence and weight loss on glycemic control in patients with type 2 diabetes: cohort analyses of integrated medical record, pharmacy claims, and patient-reported data. J Manag Care Spec Pharm. 2014;20:691–700.

    PubMed  Google Scholar 

  60. Gordon J, McEwan P, Idris I, Evans M, Puelles J. Treatment choice, medication adherence and glycemic efficacy in people with type 2 diabetes: a UK clinical practice database study. BMJ Open Diabetes Res Care. 2018;6: e000512. https://doi.org/10.1136/bmjdrc-2018-000512.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Kirkman MS, Rowan-Martin MT, Levin Fonseca VA, Schmittdiel JA, Herman WA, Aubert RE. Determinants of adherence to diabetes medications: findings from a large pharmacy claims database. Diabetes Care. 2015;38(4):604–9.

    Article  Google Scholar 

  62. Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:42. https://doi.org/10.1186/1748-5908-6-42.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Boeni F, Spinatsch E, Suter K, Hersberger KE, Arnet I. Effect of drug reminder packaging on medication adherence: a systematic review revealing research gaps. Syst Rev. 2014;3:29–29. https://doi.org/10.1186/2046-4053-3-29.

    Article  PubMed  PubMed Central  Google Scholar 

  64. Bartlett Ellis RJ, Knisely MR, Boyer K, Pike C. Pillbox intervention fidelity in medication adherence research: a systematic review. Nurs Outlook. 2017;65(4):464–76. https://doi.org/10.1016/j.outlook.2016.12.011.

    Article  PubMed  Google Scholar 

  65. Conn VS, Ruppar TM, Chan KC, Dunbar-Jacob J, Pepper GA, De Geest S. Packaging interventions to increase medication adherence: systematic review and meta-analysis. Curr Med Res Opin. 2015;31:145–60. https://doi.org/10.1185/03007995.2014.978939.

    Article  PubMed  Google Scholar 

  66. Schwartz JK. Pillbox use, satisfaction, and effectiveness among persons with chronic health conditions. Assistive Technol. 2017;29(4):181–7. https://doi.org/10.1080/10400435.2016.1219884.

    Article  Google Scholar 

  67. Morello CM, Chynoweth M, Kim H, Singh RF, Hirsch JD. Strategies to improve medication adherence reported by diabetes patients and caregivers: results of a taking control of your diabetes survey. Ann Pharmacother. 2011;45(2):145–53. https://doi.org/10.1345/aph.1P322.

    Article  PubMed  Google Scholar 

  68. Aikens JE, Piette JD. Longitudinal association between medication adherence and glycaemic control in type 2 diabetes. Diabet Med. 2013;30:338–44.

    Article  CAS  Google Scholar 

  69. Polonsky WH, Henry RR. Poor medication adherence in type 2 diabetes: recognizing the scope of the problem and its key contributors. Patient Prefer Adherence. 2016;10:1299–307. https://doi.org/10.2147/PPA.S106821.

    Article  PubMed  PubMed Central  Google Scholar 

  70. Hoogendoorn CJ, Shapira A, Roy JF, Walker EA, Cohen HW, Gonzalez JS. Depressive symptom dimensions and medication non-adherence in suboptimally controlled type 2 diabetes. J Diabetes Complic. 2019;33:217–22. https://doi.org/10.1016/j.jdiacomp.2018.12.001.

    Article  Google Scholar 

  71. Mendesa R, Martins S, Fernandes L. Adherence to medication, physical activity and diet in older adults with diabetes: its association with cognition, anxiety and depression. J Clin Med Res. 2019;11:583–92.

    Article  Google Scholar 

  72. Vieira JFF, Santos P. Medication adherence in type 2 diabetes mellitus patients: a cross-sectional study. Rev Port Med Geral Fam. 2020;36:104–12.

    Google Scholar 

  73. Andreoulakis E, Hyphantis T, Kandylis D, Lacovides A. Depression in diabetes mellitus: a comprehensive review. Hippokratia. 2012;16:205–14.

    PubMed  PubMed Central  CAS  Google Scholar 

Download references

Acknowledgements

The authors wish to thank to patients who participated; the diabetes team at Ulster, Ards and Bangor hospitals for the assistance and support in facilitating the conduct of the research, in particular, Dr Frederick McElwaine, Dr Darren McLaughlin, Dr Tom Trinnick, Dr Paul McMullan, Anita Lawther, Phyl Reynolds, Natasha Bailie, Joan Calvert, Stephanie Oates and Kathryn Stringer; Professor Robert Horne from University College London for his permission to use the MARS and BMQ; and the Indonesia Endowment Fund for Education (LPDP) for financial support.

Funding

NFS was sponsored by the Indonesia Endowment Fund for Education (LPDP) (20160422046233).

Author information

Authors and Affiliations

Authors

Contributions

JCMcE, MS, GF, AH, RH, CM, RD, JH and NFS conceived and designed the study. RD, RH, JH approached potential participants. NFS recruited participants and collected DBS samples and data. NFS performed the pharmacokinetic modelling, quantification of metformin in DBS samples, and data analysis with support from AH and GC. NFS drafted the manuscript with support from JCMcE and CP. All authors provided substantial feedback and helped shape the research, analysis and manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to James C. McElnay.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the Health Research Authority (HRA), London-Bromley Research Ethic Committees (19/LO/0804). Research governance approval was obtained from the South Eastern Health and Social Care Trust, N. Ireland (SET.19.16). Human blood sample collection for use in developing calibration standards for metformin assays was approved by the School of Pharmacy Research Ethics Committee, Queen’s University Belfast (013PMY2009).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Fig. S1.

Example of simulated metformin concentrations derived from PopPK model for patient aged 76 years old with lean body weight 47 kg and serum creatinine level 124 µmol/L who was receiving an immediate-release formulation of metformin at a dose of 1 g twice daily.

Additional file 2: Fig. S2.

Example of simulated metformin concentrations derived from PopPK model for patient aged 70 years old with lean body weight 49 kg and serum creatinine level 129 µmol/L who was receiving a sustained-release formulation of metformin at a dose of 1 g twice daily.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Syafhan, N.F., Donnelly, R., Harper, R. et al. Adherence to metformin in adults with type 2 diabetes: a combined method approach. J of Pharm Policy and Pract 15, 61 (2022). https://doi.org/10.1186/s40545-022-00457-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40545-022-00457-5

Keywords