Volume 86, Issue 4 p. 687-697
REVIEW
Free Access

How the EMERGE guideline on medication adherence can improve the quality of clinical trials

Lina Eliasson

Corresponding Author

Lina Eliasson

Sprout Behaviour Change Ltd, London, UK

Correspondence

Lina Eliasson, Sprout Behaviour Change Ltd, Wisteria Grange Barn, Pikes End, Pinner, London, HA5 2EX.

Email: [email protected]

Search for more papers by this author
Sarah Clifford

Sarah Clifford

Sprout Behaviour Change Ltd, London, UK

Search for more papers by this author
Amy Mulick

Amy Mulick

London School of Hygiene and Tropical Medicine, London, UK

Search for more papers by this author
Christina Jackson

Christina Jackson

Sprout Behaviour Change Ltd, London, UK

Search for more papers by this author
Bernard Vrijens

Bernard Vrijens

AARDEX Group, Seraing, Belgium

Liège University, Liège, Belgium

Search for more papers by this author
First published: 08 February 2020
Citations: 21

Abstract

Medication adherence in drug trials is suboptimal, affecting the quality of these studies and adding significant costs. Nonadherence in this setting can lead to null findings, unduly large sample sizes and the need for dose modification after a drug has been approved. Despite these drawbacks, adherence behaviours are not consistently measured, analysed or reported appropriately in trial settings. The ESPACOMP Medication Adherence Reporting Guideline (EMERGE) offers a solution by facilitating a sound protocol design that takes this crucial factor into account. This article summarises key evidence on traditional and newer measurements of adherence, discusses implementation in clinical trial settings and makes recommendations about the analysis and interpretation of adherence data. Given the potential benefits of this approach, the authors call on regulators and the pharmaceutical industry to endorse the EMERGE guideline.

1 INTRODUCTION

Adherence to treatment should be reported in all clinical trials. This call has been repeated for more than 20 years1-4 but unfortunately proper consideration of adherence remains the exception rather than the rule. Too often, adherence is neglected at every stage of the research process, from trial design and conduct through to data analysis and reporting in the literature. The consequences are concerning. If adherence is prevalent and unrecognised in a trial, it can potentially confound safety and efficacy signals, and obscure exposure-response estimations.1-4 If we do not know whether patients followed their treatment accurately during a trial – and why they did so, or why they did not – our understanding of how the drug will work in clinical practice may be dangerously flawed.

Until recently, the lack of concise guidance on how adherence should be measured, analysed and reported in clinical trials has been a barrier. Now, the International Society for Medication Adherence (ESPACOMP) Medication Adherence Reporting Guideline (EMERGE) offers a solution.5

This is a tool that can readily be applied alongside recommendations from the US Food and Drug Administration (FDA)6 and the European Medicines Agency (EMA).7 EMERGE has also been designed to complement the Consolidated Standards of Reporting Trials (CONSORT) for randomised controlled trials (RCTs)8 and the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) standards.9

In addition to complementing existing clinical trial guidelines, EMERGE aims to increase transparency and consistent reporting on adherence by offering a list of 21 items that include minimum reporting criteria, and more detailed recommendations for each element of a research report (the abstract; introduction; methods including study design and participants, measurement, intervention and statistical analysis; results; and discussion).5

The EMERGE development methodology has been previously reported.5 In brief, the guideline was developed through a Delphi process involving an international panel of experts. The group followed the recommended procedure outlined by the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) group.10 Like other reporting guidelines, while EMERGE offers a list of essential criteria, it does not extend to providing methodological recommendations.

This paper thus has a dual aim. It provides a comprehensive summary of recommendations for measurement, analysis and interventions related to treatment adherence in drug trial settings. When used in combination with EMERGE, this paper can guide trial sponsors and researchers as they develop drug trial protocols aligned with other key guidelines that specify that participants' adherence to investigational products and trial protocols should be adequately reported.8, 11 It is also a call to pharmaceutical regulators to take decisive action to officially endorse EMERGE and to give appropriate consideration to adherence and its impact on drug development.5 Clear guidance from regulatory approval organisations is key to driving change in the pharmaceutical industry.

2 BACKGROUND

Nonadherence to appropriately prescribed treatments is a substantial and costly problem that can severely impact both trials and practice through reduced clinical benefit and increased risk of morbidity and mortality.12, 13 In clinical practice, up to 50% of patients do not take their medications as prescribed14 and estimates point to nonadherence accounting for up to 48% of asthma deaths, an 80% increased risk of death in diabetes and a 3.8-fold increased risk of death in the year following a heart attack.15 The related costs to healthcare systems across the world are substantial: the cost of unused medicines has been estimated to reach £300 million per year in the United Kingdom.16

Adherence in clinical trial settings is also suboptimal. Blaschke et al. compiled adherence data captured by electronic measurements from 16 907 participants across 95 clinical trials. They found an initial drop of 4% due to noninitiation. By day 100, 20% of participants were nonpersistent (ie had stopped taking treatment against protocol specifications) and, among persistent patients, daily, a further 12% displayed suboptimal implementation.17

However, assessment and reporting of adherence and drug discontinuation in clinical trials are lacking. One recent systematic review of medication adherence in RCTs evaluating cardiovascular or mortality outcomes in dialysis patients found that of 22 clinical trials that met the inclusion criteria, only five reported measuring individual-level medication adherence.18 All five of these trials also presented results demonstrating negative study outcomes for the medication under investigation.

A striking example of the impact of adherence on efficacy comes from a set of RCTs assessing pre-exposure prophylaxis (PrEP) to reduce the risk of acquiring human immunodeficiency virus (HIV). Corneli et al. report that, after PrEP had been found to be effective in three separate RCTs in high-risk populations, two placebo-controlled RCTs conducted with women at higher risk of acquiring HIV failed to show effectiveness and were therefore closed early.19

However, when the data were re-analysed using a composite adherence score, based on plasma and intracellular drug concentrations, it was apparent that only 12% of participants had achieved good adherence throughout their study participation.19 This finding raises the possibility that participants in the trial were not taking enough of the prescribed dose to see a therapeutic effect.

The authors used drug concentration as a measure of adherence, instead of pill count or self-reporting, because they had previously observed that adherence was strongly overestimated with these measures. Good adherence was defined as plasma and intracellular concentrations that could be expected if a participant had taken the study drug four or more times each week over the preceding 28 days.

Medication nonadherence in clinical trials also results in increased variability and decreased effect size. The consequences vary, depending on the study design. In placebo-controlled trials, nonadherence results in decreased power, increasing the risk of type II error (the failure to show that an effective drug is effective). In positive controlled trials, there is an increased risk of type I error (erroneously claiming that drugs are equivalent).

Reduced power is often mitigated by increasing the sample size, which comes at a cost to the sponsor, or by increasing the dose, which can increase unexpected adverse reactions. Partial adherence may also cause adverse events due to rebound effects.

Trials that fail to show effectiveness because of nonadherence may be a widespread issue, and this suggestion is supported by the published evidence.20-23 However, because adherence has not been measured and reported appropriately in trial settings, and because there is a tendency towards under-publication of failed trials, the data are obscured and we cannot truly know how many compounds have been affected by this problem throughout the history of drug development.

Nonadherence may also confound dose–response estimations and skew results towards overestimation of dosing requirements, which increases the risk of post-approval dose reductions.3 Indeed, it has been estimated that 20–33% of all drugs approved since 1980 have been dose-adjusted after market authorisation had been granted, and that 60–80% of the adjustments were dose reductions.24, 25

When a compound fails to reach regulatory approval as a result of nonadherence in clinical trials, the cost is likely to run into hundreds of millions of dollars annually in the United States, and this cost is primarily carried by the pharmaceutical sponsor. An analysis by the Tufts Center for the Study of Drug Development suggested that the average cost of getting a drug approved is $2.6 billion.26 Wasted costs are incurred when additional participants are enrolled to account for nonadherence. There may be opportunity costs due to missed therapeutic effect. There will also be a knock-on effect on patients and healthcare systems because otherwise effective treatments are lost. To the best of our knowledge, the downstream financial impact of post-approval dose reductions, and suspended and revoked licences has not yet been estimated. However, these costs are likely to add up to substantial losses for the licence holders.27

3 IDENTIFYING APPROPRIATE ADHERENCE MEASURES

As outlined above, comprehensive evidence supports the argument that adherence to investigational drugs should be appropriately measured, analysed and reported in all clinical trials. This section provides guidance to researchers on how to achieve this in alignment with EMERGE guideline.5 It will discuss different approaches to measuring adherence, and explain how EMERGE could be applied in each case.

There are advantages and disadvantages with all measures of adherence, and identifying the measurement most appropriate to any specific trial requires thorough consideration of the overall objectives, study design and patient population, and the points where adherence may have an impact on the trial objectives.

3.1 The ABC taxonomy

The first step is to identify the specific aspect(s) of adherence that must be captured. The ABC taxonomy, which is fundamental in EMERGE, defines adherence as a three-phased process by which patients take their medications as prescribed.5, 28 Each phase is unique and must be specifically defined, measured and analysed:
  • (A) Initiation is when the patient takes the first dose of a prescribed drug; the process then continues with
  • (B) implementation, which is defined as the extent to which a patient's actual dosing corresponds to the prescribed dosing regimen, from initiation until the last dose is taken, and
  • (C) discontinuation marks the end of therapy, with the period between initiation and discontinuation referred to as persistence.
Nonadherence can thus arise in a number of situations, including late or noninitiation of prescribed treatment, suboptimal implementation of the dosing regimen and nonpersistence/early discontinuation that is not specified by the trial protocol or prescription (see Figure 1, adapted to clinical trials based on Vrijens et al.28).
Details are in the caption following the image
Illustration of the process of medication adherence according to the ABC taxonomy28 adapted to a clinical trial setting. The example is illustrating a twice-a-day dosing schedule

The appropriate operational definition(s) of adherence, and how this can be captured, must be decided. Researchers also need to consider whether it is important to understand any reasons for nonadherence within their trial populations, which can only be captured by using patient-reported outcome measures (PROs).

3.2 Specific measurements of adherence

The second step is to identify and review potential methods of measurement, taking into consideration the validity, reliability and potential bias of each measure, as well as the burden on participants and investigators. Table 1 presents a summary of each measurement approach, noting key considerations. Further description is provided in the specific sections below.

Table 1. Potential adherence measures to assess initiation, implementation and/or persistence, as well as reasons for nonadherence
Adherence measure Initiation date Implementation Discontinuation date Reasons for nonadherence
Directly observed therapy ✓✓

✓✓

(full dosing history)

✓✓ X
Pill count X

(only aggregate dosing summary) subject to social desirability bias

X X
Patient-reported outcome measures (self-reported adherence)

Subject to desirability bias

Subject to recall bias

Subject to desirability bias

✓✓

In particular when combined with a sound measure of adherence

Drug or drug-metabolite monitoring X

Sparse data and subject to white coat adherence bias

X X
Electronic detection of package entry

✓✓

Requires activation at patient initiation in the trial

✓✓

(full dosing history)

✓✓ X
Ingestible sensor, eg breathalysers smart pill

✓✓

Requires activation at patient initiation in the trial

✓✓

(full dosing history) perceived intrusiveness and directions for using sensor should be considered

✓✓ X
Pharmacy claims and refill data

(date of first fill may differ from formal treatment initiation)

(only an aggregate dosing summary)

(not precise, typically 90 days uncertainty)

X
  • Note. ✓, suitable; ✓✓, very suitable; X, not suitable.

3.2.1 Directly observed therapy

This is one of the only real-time measures of adherence that can guarantee drug ingestion or administration. Directly observed therapy (DOT) is resource intensive and primarily viable where participants are staying in a clinical setting during the trial period. DOT is therefore often used in Phase I trials with healthy volunteers, but rarely used in Phase II–Phase IV trials with ambulatory patients. McCann et al. summarised “DOT-proxy” methods, including using camera/video-enabled mobile phone technology where participants record ingestion and transmit the images to investigators for observation29 (eg AiCure Technologies, Inc.), which may be somewhat burdensome for both parties. DOT methods could be used to assess all three phases of nonadherence in the clinical trial setting, as defined by EMERGE. However, the high burden on participants, and the intrusiveness into their lives, precludes widespread use.

3.2.2 Pill count

Counting returned tablets is the standard practice for drug accountability in clinical trials; it is also the most commonly used method to measure adherence. However, it is easily censored by participants and only provides an aggregate summary of adherence between visit periods. It does not allow precise estimates of initiation, implementation or persistence.30, 31

3.2.3 Patient-reported outcome measures

Self-reported adherence data is also common and is typically based on a patient-reported outcome (PRO) scale or categorical measure, or as a patient diary. There are many self-report measures and different methods of administration, with large variation in validity and reliability burden to participants and investigators,32, 33 and large variation in licensing fees. PROs are sensitive to recall and social desirability biases and, like pill count, have often been found to overestimate adherence.17 Classic administration traditionally uses pen and paper, which may have increased such biases, but there has been a move in recent years towards electronic data capture in clinical trials.

A great advantage of self-report, which is often overlooked in the literature on adherence measurements, is that it is the only way to explore why a participant has acted in a certain way regarding treatment and/or trial protocol adherence.30, 32 This approach can be helpful in determining reasons for nonadherence during initiation of therapy, implementation and ongoing persistence, providing valuable and timely insight to the drug development and study design team. Structured or semi-structured PROs can be designed to capture participants' reasons for nonadherence.32 However, measures that use double-barrelled items to capture both the act and cause of nonadherence in one item are not recommended as the data would be difficult to interpret.

Exit interviews are increasingly used to assess adherence, and the reasons for nonadherence, in a clinical trial setting. A study by Nunn et al. provides a useful example. In people diagnosed with HIV, who were stabilised on antiretroviral treatment, this trial assessed the risks and benefits of stopping cotrimoxazole (a prophylaxis against opportunistic infections).34 Exit interviews were used to assess participants' adherence to the study procedures. It emerged that some participants were nonadherent to the trial protocol because they obtained and used open-label cotrimoxazole, although not to the extent that the trial results were seriously compromised.34

3.2.4 Drug or drug metabolite monitoring

Monitoring drug or drug metabolites in plasma, urine or hair can provide a snapshot of adherence. However, such monitoring is sensitive to bias when participants ingest the investigational drug before a trial visit, even if they may have been nonadherent in the period running up to the visit (white-coat adherence).35 Monitoring the investigational drug in biologic fluids is also restricted to active arms of the trial, unless an ultralow dose of the drug, which is detectable but below the lower end of the therapeutic range, is used in the control arm.29 An alternative is to use other pharmacologically inactive biologic markers that are ingested simultaneously with the investigational or placebo compounds. The ideal marker should have an appropriate pharmacokinetic profile in terms of dosing schedule, low variability within and between participants, few drug–drug or food interactions, and should be excreted in urine or saliva. It is also essential that the marker is generally considered safe or licensed for use by regulators, and uncommon in dietary sources, supplements and pharmaceuticals.36 Another potential drawback is that the biological marker would need to be added to the formulation of the investigational drug.3 This method could be used as a measure of adherence implementation, given the drawbacks mentioned above are addressed, but it cannot be used to determine initiation or persistence, unless daily sampling is undertaken.

3.2.5 Electronic detection of package entry

Electronic monitoring incorporated in medication packaging, pioneered by the Medication Event Monitoring System (MEMS®, AARDEX Group37), allows for date- and time-stamped recordings when the participant opens the pharmaceutical package. MEMS was originally designed as an electronic cap to capture the opening and closing of a standard pharmaceutical bottle. The principle can easily be extended to other drug-delivery systems, including blister packaging, injections and inhalers. To date, more than 810 papers in peer-reviewed journals report diverse uses of MEMS in research settings and a multitude of companies offer smart packaging.

It is assumed that a correct dose of the investigational drug is ingested each time the packaging is opened, and that the continuous data captured will clearly display adherence patterns over time. These recording devices require minimal management from participants and investigators. Electronic detection of package entry is an indirect measure of dose intake and there could be instances where the package is activated but a dose is not taken.31 Studies comparing MEMS data with drug concentrations show that there is 97% accuracy between opening the pharmaceutical package and time of ingestion of the prescribed dose.3

In drug trials, this method can be used to precisely identify the time of initiation and the time of eventual treatment discontinuation. Furthermore, it allows reliable and sound compilation of each patient's implementation of the dosing regimen.

3.2.6 Ingestible sensors (breath inhaler, smart pill) and electronic detection of ingestion

Technological approaches to measuring adherence are emerging at a rapid rate. These include breathalysers (eg SMART®, Xhale, Inc.), monitors for inhalers (eg the Inhaler Compliance Assessment, INCA™) and ingestible sensors (eg Proteus Digital Health, Inc. and eTectRx, Inc.). Each can be used to assess adherence across the initiation, implementation and persistence stages of the clinical trial data collection process that are described in EMERGE. Many can confirm that the drug has been ingested by the participant, although some sensors have technical limitations and, alternatively, allow users to record an event by pressing a button.38 These measures may also provide insights into patterns of adherence and nonadherence, although they may not be feasible when investigating specific drugs or in all contexts of use. For example, ingestible sensors can only be used with solid oral medication and require drug reformulation. In addition, where a skin patch is required to detect the signal, the user must maintain the patches over the treatment period: skin irritation and inflammation are commonly reported.38 Ingestible sensors may also be perceived as intrusive by users.39

3.2.7 Claims and refills

Measures such as pharmacy refill and insurance claims data, which are used in adherence research, are not appropriate in a Phase I–III clinical trial setting where participants tend to be given the investigational drug during trial visits and the cost of the treatment is funded by the sponsor. This approach may only be relevant for studies in the post-approval stage, that is, Phase IV registry studies. In addition, such data do not provide information on any errors related to dose and timing of administration, which is crucial in terms of understanding exposure-response.3

3.2.8 Multiple complementary measures

A trial design that aligns with EMERGE by considering appropriate definitions of the aspect(s) of adherence to be captured (initiation, implementation and persistence) may prompt the need for multiple complementary adherence measures. The use of complementary measures is distinct from measurement triangulation, in that each measure should measure a distinct element of adherence, and each should be defined and reported according to EMERGE.5

4 ADHERENCE INTERVENTIONS IN TRIAL SETTINGS

Regulators have taken notice of the potential benefit of supporting adherence in trial settings. The FDA's guidance for Enrichment Strategies for Clinical Trials to Support Approval of Human Drugs and Biological Products (2019) acknowledges that good adherence in a trial can increase the power of a study – its ability to demonstrate a treatment effect if one is present – by decreasing variability that is unrelated to the study drug.6 One strategy is to identify and select patients who are likely to adhere to treatment, but only prior to randomisation, not afterwards. Otherwise, this step could impact the statistical validity and conclusions of the trial; indeed, adherence alone, even to placebo, has been linked to better outcomes in a trial. In addition, a protocol that is partially masked to participants and investigators during a run-in period can reduce the risk that adherence may be overly and artificially encouraged.

Other measures discussed include making patients aware of the conditions and demands of the trial, avoiding overly rapid initial titrations to reduce initial side effects, and using adherence prompts and feedback, for example using dosing history data from smart bottles to encourage good adherence. These strategies can also be used in safety assessments.6

It is also advisable to consider complexity and stringency when designing a trial protocol to facilitate adherence.36

SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) is an international initiative that aims to improve the quality of clinical trial protocols by defining an evidence-based set of items that should be included in a clinical trial protocol.40, 41 One component of this is a section on adherence that provides guidance on procedures and strategies for monitoring and improving adherence and reporting this clearly in the study protocol.

Adherence interventions in a trial setting mainly fall into two categories: (a) approaches to identify adherence issues in the trial preparation and run-in phases; and (b) approaches to directly improve adherence to the trial protocol and/or the investigational drug.

4.1 Trial preparation and run-in phases

Previous participation in another trial, within a specific time period, is a common criterion for participant exclusion. However, some participants have been shown to mislead investigators about this point.29 One solution is to use participant registries, and this precaution may reduce the risk of enrolling participants who are involved in multiple trials.36 In some markets, including France and the United Kingdom, registration of Phase I volunteers is mandatory. It is increasingly apparent that this requirement is needed for Phase II–IV trials.

Some pharmaceutical sponsors use participant identifiers to identify duplicate enrollers in their own trials. However, these techniques do not identify them across sponsors. There are also various systems, including Verified Clinical Trials, CTSdatabase and Dupcheck, which use General Data Protection Regulation (GDPR)- and Health Insurance Portability and Accountability Act (HIPAA)-compliant databases to identify duplicate enrollers and thus verify eligibility.36

Shiovitz et al. recommend sharing only limited inclusion and exclusion criteria during recruitment to ensure participants do not have the information they would need to feign inclusion characteristics. Other recommendations include ensuring stipends are reasonable, to discourage “professional” subjects and reduce the risk of coercing participation, and requiring site investigators to enrol at least half of participants from internal databases that offer a known source of eligible participants. The authors further recommend that investigators should be reimbursed appropriately in terms of screen-fail ratios, as a low screen-fail ratio may encourage investigators to screen in ineligible participants in order to secure payment.36

As discussed above, the use of single-blind placebo run-in periods with adherence monitoring has been recommended to identify participants who are inclined to respond to placebo and those inclined to be nonadherent to the investigational drug and/or protocol requirements.6, 29 These participants can thus be excluded according to predefined criteria or accounted for in the analysis of the trial. As noted in the FDA guidance, it is important that identification is done before randomisation.6

The FDA guidance describes the run-in period approach as an enrichment strategy that has the potential to decrease heterogeneity (nondrug-related variability) and thus increase study power to detect a real drug effect.6 However, investigators must carefully consider exactly how to analyse data from participants who, during the run-in period, have been identified as placebo responders or low adherers. This decision must be guided by the objective of the study.3 It may not always be appropriate to simply exclude these participants. Instead, the information may be used to inform the analysis of subpopulations.

Despite recommending them, the FDA guidance recognises that enrichment strategies may limit the generalisability and applicability of the study results.6 Therefore, care should be taken to ensure that sufficient data are collected on the full range of individuals who may later receive the drug post approval.

4.2 Approaches to directly improve adherence

The FDA guidance also describes a range of methods that can be used to positively affect adherence.6 However, the ethics of directly encouraging ingestion of an investigational drug need to be carefully considered, in particular before efficacy and safety have been established, as a trial participant must remain free to discontinue treatment without explanation and prejudice, and must also feel free to report any adverse reactions or experiences of taking the drug.29 FDA guidance recommends adherence feedback at the point of care to avoid this ethical issue. To avoid potential source of bias, it is recommended to provide standard adherence monitoring and support to all participants in both active and placebo arms.

McCann et al. recommend financial incentives, not to increase adherence but rather to encourage participants to report adherence accurately, whether the drug has been taken or not.29 However, this approach carries the risk of biasing a participant's response because they are incentivised to give a response at each point of measurement.

One strategy that has great potential is patient-centred trial design. This can encourage adherence not only to the requirements of the study period but also throughout follow-up, which is often lengthy. This approach includes steps such as seeking patient input at the design stage, to ensure patient-friendly study materials and processes are used to help explain and implement the clinical trial; choosing outcomes that are meaningful to patients; and minimising participant burden from clinic visits via greater use of at-home electronic data capture.42

5 ANALYSING AND INTERPRETING ADHERENCE DATA IN CLINICAL TRIALS

This section links to items 5–8 in the EMERGE guideline, which describe the trial design and statistical analysis elements that should be considered a priori and reported at the end of the trial.5

Nonadherence to a planned treatment regimen in placebo-controlled trials can diminish the causal treatment effect (ie efficacy) and make a treatment look less effective than it really is.36, 43, 44 This causes many problems, some of which have been discussed above.

In placebo-controlled trial analysis, nonadherence fundamentally alters what is meant by the “treatment effect”. If nonadherence is ignored in the statistical analysis, the true efficacy of treatment will be diminished, resulting in overly conservative estimates of efficacy. In contrast, in safety analyses and in analyses of noninferiority and equivalence trials, the diminishing effect of nonadherence is usually anticonservative, which means a harmful treatment is more likely to be accepted as harmless and equivalency or noninferiority are more likely to be declared.44, 45 There is also some evidence that patterns and causes of nonadherence can influence the treatment effect in different ways, making it difficult to predict the overall effect.46, 47

A recent review found that a majority of trials are subject to various forms of nonadherence to the treatment protocol, and investigators attempt to deal with nonadherence using a variety of statistical methods and analysis populations. However, they rarely consider the potential for the introduction of bias.45 Here, we present recent research and recommendations on how to account for nonadherence in trial analyses to better estimate treatment efficacy.

5.1 Planned approaches

If nonadherence is foreseeable in a trial, investigators should plan for this at the design stage both to demonstrate transparency and to ensure all data that will be necessary for analysis are collected. Dodd et al. have published a useful set of recommendations to enable adequate data collection.43 For analysis, current draft International Conference on Harmonisation (ICH) guidelines (E9(R1) 2017) state that clear trial objectives should be translated into key scientific questions of interest, and suitable estimands (the target of estimation for these questions, see box) chosen to address them, before an appropriate statistical analysis plan is specified. EMERGE sections 7 and 8 can guide the selection of an estimand by helping to define the parameters of nonadherence.5 After the estimands have been settled, an appropriate statistical analysis can be specified.7

5.1.1 Intention-to-treat

It is generally recommended to perform the primary efficacy analysis under the intention-to-treat (ITT) principle,43 where all participants are included in the analysis and treatments are compared between randomised arms, regardless of which treatment the participants actually received. This preserves the element of randomisation, ensuring that no baseline confounding is expected to affect the analysis. Current ICH guidelines describe five ITT strategies, each for different estimands of subtly different treatment effects of interest.7 The first is treatment policy, which is the strategy most commonly associated with ITT analyses and the one prone to underestimating efficacy. This estimand is useful in pragmatic trials that are looking for the population effect of implementing a treatment policy, with real-world deviations from the protocol taken into account.

5.1.2 Additional strategies

However, when interest lies in more precisely estimating efficacy, and appropriate data on adherence are available, the remaining four strategies can be employed in addition to (or instead of, depending on the aims of the trial) the treatment policy strategy. Briefly, these are:
  1. The composite strategy, where nonadherence is integrated into the (adverse) outcome of interest. With this strategy we assume nonadherence adds meaningful information about the overall effect of the treatment. As an example, perhaps participants did not adhere to the treatment because they could not tolerate it, or they experienced other adverse events that had an influence on the treatment effect. The outcome would be expanded to include either the event under study or failure to adhere to or withdrawal from treatment, and randomised groups would be compared in an appropriate statistical analysis.
  2. The hypothetical strategy, where the treatment effect is that which would have been seen if everyone had adhered perfectly to treatment through to the end of follow-up. Strong assumptions are required for this effect to be meaningful, including that medication nonadherence was unrelated to tolerability (see ICH E9 (R1) guidelines for further discussion). In this strategy, outcome measurements that were not observed due to nonadherence can be imputed under plausible missingness assumptions, and an appropriate statistical analysis can be performed on the imputed dataset(s).
  3. The principle stratum strategy, where the treatment effect is that which would be seen in the stratum of people who (a) would be able to perfectly adhere to either treatment and (b) would actually do so. This subset of randomised participants cannot be directly known because valid reasons for nonadherence (such as medication intolerability) cannot be predicted in advance, and they cannot be inferred after the trial has been completed as participants can only receive one treatment. However, investigators can attempt to infer membership in this stratum from covariates and perform the statistical analysis on data from these participants only. At the design stage investigators may also consider specifying markers of good adherence and restricting participation with relevant inclusion or exclusion criteria and/or they may consider using special trial designs such as run-in, enrichment or randomised withdrawal, though these strategies may decrease the generalisability of the results.
  4. The while on treatment strategy, where the treatment effect is a weighted average of the outcomes seen while adhering to treatment (rather than the effect seen by a prespecified time point). An appropriate statistical analysis that accounts for time in adherence can then be performed.
Which strategy to use will depend on the estimand of interest. Since all of these strategies are underpinned by assumptions, it is best to use one or more sensitivity analyses to test them. For a thorough discussion, see the ICH Harmonised Guideline for Estimands and Sensitivity Analyses in Clinical Trials.7

5.2 Less preferable strategies

Other analysis principles, such as per-protocol (PP) and as-treated (AT) analyses, attempt to address questions of adherence and efficacy. However, they are not usually suitable. PP analyses typically exclude or censor participants who deviate from the treatment protocol before comparing treatments between randomised arms. However, because the treatment protocols are likely to present different challenges to adherence, the participant groups may not be comparable.44 AT analyses include all participants, but analyse them according to the treatment they actually received, discarding randomisation. Both of these analyses are prone to selection bias, so confounding must be considered and appropriately adjusted for in statistical analysis. As it is usually not possible to completely adjust for all possible sources of confounding, neither of these is preferable to an appropriately designed ITT analysis.

5.3 Unplanned approaches

If nonadherence was not considered during trial design (so no planned analyses incorporate it), but relevant data are available for analysis, investigators may decide to perform one of the above analyses post hoc. However, care must still be taken in selecting an appropriate estimand and, as with all post hoc investigations, investigators must use caution and avoid overinterpreting results from unplanned analyses.

More advanced statistical techniques can also estimate causal treatment effects and account for nonadherence to treatment, while maintaining the balance produced by randomisation. Some examples include randomisation-based efficacy estimation,48 instrumental variable49 and complier average causal effect (CACE)50 methods. Such analysis techniques are not yet commonly used, as they tend to be more complex and/or computationally intensive than the previously discussed analyses. They also rely on potentially unverifiable assumptions, which are often stronger than those required for ITT analyses. However, this area of research is growing and some35 are calling for increased awareness and uptake of these methods.

Although it may be possible to estimate causal treatment effects after a trial has started, or even after it has ended, preplanned statistical analyses always give weightier and more transparent results than post hoc analyses. Therefore, identifying potential adherence issues early in the clinical trial programme is crucial.

5.4 Safety analyses

Although much research has gone into methods of accounting for nonadherence in estimating treatment efficacy, there is to our knowledge no research and guidance on the appropriate analysis population to use for harms outcomes in the presence of such nonadherence.45 This is likely due to a more widespread lack of good practice around collecting, reporting and analysing harms in clinical trials. A recent systematic review51 of trial results reported in four high-impact journals found that these items were inconsistently undertaken and concluded that statistical analysis, in particular, was often deemed inappropriate and suboptimal, potentially leading to missed harm signals and unsafe treatments being declared safe. In addition, because adverse events are unpredictable in advance, safety analyses are not often well-powered. The problem of low power is exacerbated by nonadherence, which could preclude the observation of adverse events that would otherwise have occurred; further, nonadherence could itself be a consequence of side effects that were on the path towards an adverse event. More research is needed on how to deal with nonadherence in safety analyses.

6 REPORTING ADHERENCE DATA FROM CLINICAL TRIALS

The EMERGE guidelines5 should be used in harmony with reporting guidelines for clinical trials, such as STROBE9 and CONSORT,8 to ensure that all the relevant aspects of the adherence data are reported in a relevant and appropriate manner. The minimum reporting criteria include (a) the phases of adherence being studied; (b) the operational definition of each adherence phase; (c) the measurement method/s used; and (d) the results of the analysis appropriate for each phase being studied.5

An estimand is simply a treatment effect “of interest” for a clinical question. The treatment effect in a perfect setting, where 100% of trial participants would adhere to their treatment throughout the entire follow-up period, is a pure effect of the treatment. It is how the outcome of treatment, taken exactly as prescribed, compares to what would have happened to the same subjects had they all taken the alternative (or no) treatment exactly as prescribed. Often this is the effect “of interest”; however, non-adherence reduces or increases the amount of exposure to the treatment and muddles this interpretation. In this situation there are several possible “treatment effects” that could be obtained, each answering slightly different questions.

The EMERGE guideline highlights the information required to define an estimand for a clinical trial that factors adherence into the treatment effect. Estimands have four elements7:

  1. the target population (ie the stratum of participants who would adhere to treatment if they could tolerate it: EMERGE items 5a–5c elicit information to help define this5)
  2. the variable or endpoint to be obtained for each participant (eg treatment failure defined as non-response or treatment discontinuation: EMERGE item 6a5)
  3. the specification of how to account for nonadherence and other treatment-altering events (see text for a description of potential strategies: EMERGE items 7a,b5)
  4. the population-level summary for the variable, which can be compared between treatment conditions (eg mean change in outcome or proportion of treatment failures: EMERGE items 8a,b, 9a,b5).

7 CONCLUSIONS

Unidentified nonadherence in clinical trial settings is a substantial and costly challenge that directly or indirectly affects all stakeholders. Nonadherence in clinical trials can obscure exposure–response estimations and confound safety and efficacy signals. Addressing this issue offers benefits to all. For drug developers, it is likely to be more cost-effective to identify and manage noninformative data or use trial enrichment strategies to increase study power than to manage the effect of nonadherence by increasing sample size.36 Patients stand to benefit from safer and more effective dosing regimens, which have been established based on adherence-informed results.3 The EMERGE guideline5 can help drug developers and researchers to better understand how adherence data should be collected, analysed and reported in clinical trial settings, and can be used to drive improvement in addressing the pervasive issue of nonadherence in clinical trials. To align with previously published guidance specifying that adherence to investigational products and trial protocols should be adequately reported,6, 7 pharmaceutical regulators need to officially endorse EMERGE5 to drive positive change, and ensure clinical trial sponsors and research organisations can achieve appropriate alignment with these guidelines.

ACKNOWLEDGEMENTS

The authors would like to thank the members of the Centre for Business Innovation's (CfBI's) Medical Adherence and Digital Health Consortium for contributing to this manuscript through meeting discussions and feedback on the manuscript drafts. The consortium has representatives from pharmaceutical companies, medical device providers, systems integrators, healthcare information handlers, private and public healthcare purchasers and providers, and leading academic experts in the field of treatment adherence. The consortium members work collaboratively to advance knowledge and promote best practices in the field of medication adherence. The views presented in this paper are those of the authors and are not necessarily representative of the affiliated organisations. Colleen Shannon, Freelance Medical Writer, assisted with editing of the manuscript.

    COMPETING INTERESTS

    The authors declare the following conflicts of interest: LLE and CAJ are owners and directors of Sprout Behaviour Change Ltd. BV owns shares and is a director at AARDEX Group. SC and AM have no conflicts of interest to declare. No funding was received for writing this manuscript.