Lost your password? (close)

FAQ

View All Answers

FAQ – QUALIFICATION-RELATED (FDA, EMA)

1._ When do I use the EXACT® vs. the E-RS™?

The 14-item EXACT was developed to standardize the symptomatic evaluation of acute exacerbations of COPD and chronic bronchitis. The total score is used to characterize the severity of medically-treated exacerbations (MTEs) and track symptom recovery over time. Threshold rules are used to identify unreported, symptom-defined events which can be described in terms of their frequency, severity, and duration. The EXACT is used to evaluate outcomes in trials testing the effect of treatment on acute exacerbations or to prevent exacerbations of COPD.

The 11-item E-RS quantifies the severity of respiratory symptoms of COPD, including breathlessness, cough, sputum, chest congestion, chest discomfort, and chest tightness. The E-RS was developed to evaluate the effect of treatment on respiratory symptom severity of COPD, i.e., to test the extent to which treatments improve respiratory symptoms when patients are stable/non-exacerbating.

Because the E-RS is embedded in the EXACT, either or both may be used in a trial, with the underlying hypotheses pre-specified during trial design. Context of use and example endpoint models are included in the User Manual for each measure.

Several example scenarios are as follows:

  • A 28-day acute intervention trial testing the effect of an anti-infective therapy
    • Patients are enrolled during an acute state. The EXACT is used to track symptomatic recovery. The E-RS is not used.
  • A 12-month exacerbation prevention trial
    • Patients are enrolled in a stable state. The EXACT is used to evaluate the effect of treatment on the symptomatic severity of MTEs and the frequency of unreported symptom defined events.
    • The E-RS is used to evaluate the effect of treatment on respiratory symptoms during non-exacerbating states, with sensitivity analyses over the trial period.
  • A 12-week COPD treatment trial
    • Patients are enrolled in a stable state. The E-RS is used to evaluate the effect of treatment on respiratory symptoms over the study period.
    • The EXACT is not used, or the EXACT is used as an exploratory endpoint to understand the effect of treatment on the frequency of symptom-defined events.

2._ Is there, and should there be, a perfect correspondence between health care resource utilization (HCRU) and symptom-based (EXACT®) definitions of exacerbation?

  • No, there is not, nor should there be. The HCRU definition of exacerbation is a dichotomous outcome based on the presence of a clinic visit or hospitalization and clinician-report of the event and change in treatment. The patient makes a decision to seek care and is seen by the clinician who evaluates the patient and prescribes treatment (Event Occurred). A symptom-based definition of exacerbation using a daily diary is derived from a continuous variable tracking patient symptoms over time. In order to yield a dichotomous outcome (Event Occurred), a scoring threshold is needed, indicating the patient experienced an event—defined as sustained worsening of COPD beyond normal day-to-day variability.
  • Several key points:
    • The EXACT scoring threshold is used to identify unreported symptom-defined events (an Event Occurred). It is NOT used to validate or verify HCRU visits.
    • To quantify symptom severity during HCRU events, the continuous EXACT scores are used, covering the full range of symptom experience, with higher scores indicating more severe symptoms.
  • Research has shown a strong relationship between continuous EXACT scores and HCRU events over time, with EXACT scores rising and falling with these events (Leidy et al. 2014; Jones et al. 2014).
  • The relationship between HCRU events and EXACT-defined events (threshold scoring) provides interesting insight into the nature of HCRU events. Research suggests that approximately 50% of medically treated events (MTEs) exceed the threshold of an EXACT-defined event (persistent increase in symptoms of at least 9 points); conversely, 50% do not reach the threshold used to identify unreported events (Leidy et al. 2014; Jones et al. 2014). Note that this does NOT mean EXACT scores did not rise; rather, it means scores did not reach the 9-point threshold. One should expect this result for many reasons, including:
    • Patients choose to call or see their clinician for a variety of reasons, including clinician instructions, change in other health parameters, and timing (day of the week, holidays, etc.), all of which are not part of the symptomatic definition of exacerbation.
    • Clinicians see patients and select treatments for a variety of reasons, including professional preferences, past experiences with a particular patient, comorbid conditions, family concerns, and/or the patient’s reduced capacity for self-care or need for professional observation.
  • It is interesting to note that analyses of three independent trials showed that the correspondence between alternative indicators of patient worsening (airflow, rescue medication use) was equivalent or better for symptom (EXACT)-defined events than for HCRU events (Leidy et al. 2014).

3._ Can the EXACT® be used as a primary, co-primary, or key-secondary endpoint in exacerbation prevention trials?

  • Pharmaceutical sponsors are free to decide how best to select and position EXACT-defined endpoints in their trials. Sponsors using the instrument in drug development program should consult with the appropriate regulatory agency (FDA, EMA) regarding their trial design and endpoint model.
  • Regulatory agencies (FDA, EMA) define COPD exacerbations as HCRU or “medically treated events” (MTEs), i.e., clinic or emergency room visits, hospitalizations, accompanied by a change in treatment. The EXACT was not designed to replace this definition. Rather, the EXACT was created to supplement this endpoint, providing standardized, quantitative data on the severity of symptoms associated with MTEs, as well as data on the frequency, severity, and duration of unreported symptom-defined events. (See Table 1; this table is also provided in the EXACT User Manual).

4._ Are there logical inconsistencies in individual patient-report on items related to cough (i.e., items 2, 3, 4) and/or those related to breathlessness (i.e., items 7–11)?

  • No. A post-hoc analysis was conducted to address this question. Specifically, cross-tabulations were performed with data from the first validation study (N=410), deriving counts of logical inconsistencies across the 28 study day period. Results showed a very low percentage of logical inconsistencies (<0.6%), which remained low over time. These results were confirmed using data from two 3-month trials and one 6-month study.

5._ How does the E-RS™ complement and extend existing PRO measures?

  • Respiratory symptoms are an important component of how patients with COPD feel and function. Despite consensus on the defining respiratory symptoms characteristic of COPD and the importance of symptom relief to patients and clinicians, there is no standardized, validated diary for evaluating symptom outcomes in clinical trials.
    • The Breathlessness, Cough, and Sputum Scale (BCSS) (Leidy et al. 2003, is a daily symptom diary that has been tested for reliability and validity and used in clinical trials of COPD, but was not developed according to the United States (US) Food and Drug Administration (FDA) PRO Guidance (FDA 2009).
  • Several multi-dimensional health status questionnaires administered periodically during the course of a trial include information on respiratory symptoms and their impact, but do not capture this information on a daily or weekly basis. Further, they do not include subscales that specifically measure the severity of breathlessness, cough and sputum, and chest symptoms.
  • There are also a number of symptom-specific questionnaires that capture dyspnea or cough. These instruments do not assess each of the defining respiratory symptoms of COPD on a daily basis.
  • The E-RS was designed to quantify the severity of respiratory symptoms of COPD (breathlessness, cough and sputum, chest symptoms) on a daily basis; thereby capturing day-to-day variability of the patient experience. Developed and validated according to FDA PRO Guidelines, daily and mean weekly scores can be used to characterize respiratory symptom experience and response to treatment.

6._ Why should the 11-item E-RS™ never be administered independently of the 14-item EXACT®?

  • Instructions to administer the 11-item E-RS as part of the 14-item EXACT assures consistent context for item completion across studies. It also permits users to run post-hoc exploratory or secondary analyses using the EXACT scoring algorithms to inform future research.
  • Subject completion time for the 14-item diary is less than three minutes. We concluded that the advantages of consistent context and dual analytical opportunities (E-RS and EXACT) more than outweigh any marginal effects of reducing the diary by three items.

FAQ – GENERAL

1._ What is the EXACT®?

  • The EXAcerbations of Chronic pulmonary disease Tool (EXACT®) is a 14-item daily diary used to quantify exacerbations of chronic obstructive pulmonary disease (COPD), including chronic bronchitis and emphysema, in natural history studies and clinical trials.
  • See home page and instrument descriptions for more information.

2._ Why was the EXACT® developed?

  • Historically there has been no standardized method for quantifying exacerbations of COPD in clinical studies.
    • Exacerbation frequency has been estimated using definitions that are event-based (counting clinic or emergency department visits and/or hospitalizations for COPD), and/ or symptom-based (assessing change in symptoms through a patient diary card).
    • Severity and duration have been estimated based on the type of resource use, duration of hospitalization or pharmacologic treatment, and/or clinician global impression.
    • Each of the approaches used to date has its unique set of limitations. For example:
    • Event-based definitions cannot account for regional differences in health policy or unreported events and do not standardize the magnitude of change in the patients’ underlying condition.
    • Symptom diaries have varied in content and structure, and do not demonstrate all of the important measurement characteristics (reliability and validity) needed to make precise decisions based on their assessment of COPD.
  • The EXACT provides a single, standardized approach for assessing the symptomatic manifestations of COPD exacerbations, with a development history consistent with guidelines proposed by the FDA, EMA, and well-known measurement principles.
  • The E-RS, a derivative measure using a subset of items from the EXACT, addresses the need for a standardized respiratory symptom diary with a development history consistent with guidelines proposed by the FDA, EMA, and well-known measurement principles.

3._ How is the EXACT® different?

  • The EXACT was developed and validated using a systematic process consistent with the US FDA Guidance for Developing Patient-Reported Outcome Measures for Use in Medical Product Development to Support Labeling Claims.
  • It was designed with and for patients with COPD.
  • It was designed specifically for completion on an electronic device-PDA, smart phones, and similar electronic devices-for ease of use and to permit real-time data tracking.
  • The EXACT is the first instrument to undergo PRO drug development tool qualification review by the FDA.
  • The EXACT and E-RS minimize investigator and subject burden by providing two validated uses for 1 diary:
    • Assessment of COPD exacerbations via EXACT Total Score
    • Quantification of respiratory symptoms in stable COPD using RS-Total and subscale scores

4._ How was the EXACT® developed?

  • See information on Development here.

5._ Is the EXACT® reliable and valid?

  • Instruments are evaluated for reliability and validity for their specific context of use. See the FDA PRO and DDT Guidance documents.
  • The EXACT scoring properties have been tested in subjects with COPD, with inclusion/exclusion criteria based on historical eligibility standards for clinical trials in this target population.
  • Data used to evaluate EXACT reliability, validity, and responsiveness were from:
    • A prospective natural history study of acute and stable patients
    • A 6-month Phase II trial conducted in the US
    • Two 3-month Phase II trials conducted internationally
  • Information on study samples, methods, and results of these performance studies are provided in their respective publications here.

6._ What is the minimally important difference (MID) for the EXACT®?

The 14-item EXACT was designed to quantify exacerbations of COPD, including the identification of unreported events, the severity of those seen in the clinic and the severity around hospitalizations. The question about an “MID” for the EXACT can be interpreted in several ways. The “MID” for determining the presence of a symptom-defined event is based on both the magnitude and duration of score change: 9 point increase (worsening) from the usual score (e.g., stable baseline) sustained for 3 days or 12 point increase sustained for 2 days. So – 9 point changes are meaningful, and sustained increases of this magnitude are indicative of a symptom-defined exacerbation. An overview of the research supporting these values is included in the supplement of the paper published in Leidy et al. 2014.

Regarding day-to-day interpretation of the EXACT scores, if a patient reports an increase of 9 points, this would clearly be significant to the patient (and if sustained for 3 days, would constitute an unreported symptom-defined exacerbation). Increases > 5 points may also be significant, however the data suggest COPD patients can vary by 3-5 points on a day-to-day basis, suggesting this is “normal variability” (See Table 1 of Leidy et al. 2014 – Information on SEM and Intra-individual variability during stable/baseline period across 3 trials). This variability does not control for other factors that may be measured or unmeasured in the study (e.g., seasonality).

A related question that may be of interest to some sponsors is the extent to which other factors (e.g., seasonality, air quality) contributes to exacerbations, both reported and unreported. Reported events are straightforward – clinic visit, urgent care, hospitalization. If a sponsor seeks to use the EXACT for questions of this nature, the “MID” values would be related to frequency, severity and duration of symptom-defined exacerbations. Clearly, fewer, milder, and shorter durations are preferred. To date, specific MIDs for these 3 endpoints have not been proposed or tested.

7._ What is the minimally important difference (MID) for the E-RS™?

The E-RS is a subset of items from the EXACT and includes 3 subscales in addition to the E-RS total score: Breathlessness, Cough & Sputum, Chest Symptoms. Proposed MID values for the RS-Total score and 3 subscales are found in Leidy et al. 2014. These have been used to identify responders in randomized trials (paper under review) and seem to work quite well.

8._ How should the EXACT® be administered?

  • The EXACT was designed as an electronic patient-reported outcome (ePRO) to be completed on a personal digital assistant (PDA) or smart phone each evening before bedtime.
  • The diary takes less than three minutes to complete.
  • An EXACT ePRO Certification Program was developed to:
    • Optimize consistency across devices and ePRO vendors;
    • Provide instrument users with a choice of informed ePRO vendors;
    • Click here for more information on the Certification Program.
  • Although a paper-pen booklet is available, the paper-pen method of administration is not recommended or endorsed.
    • No data are available on the performance characteristics of the EXACT administered via pen-paper diary.
    • There is evidence that subjects can be non-compliant with paper-pen diaries, entering multiple days of data at a single sitting. This is not possible with an eDiary.

9._ Has the EXACT® been translated for use in other languages?

  • See information on translations here.

10. What is the regulatory (qualification) status of the EXACT®?

11. How should I select an eDiary/ePRO vendor for my trial?

  • See information on ePRO here.

12. What if I need a language that is not currently available?

13. What timelines should I consider when planning to administer the EXACT® in a clinical trial?

  • There are two important time-limiting factors to consider — translations and device programming.
    • For new translations, allow three to five months for this process.
    • Plan on one to four months for device programming, IT validation, and uploading/formatting of the EXACT in the selected languages.

14. Can the EXACT® be used to evaluate respiratory symptoms in a non-exacerbating state?

  • The E-RS (Evaluating Respiratory Symptoms) is a scoring algorithm for a subset of 11 items within the EXACT designed for this purpose. See a description of the E-RShere.
  • The E-RS is always administered as part of the 14-item EXACT. It is never used alone.
  • The EXACT and E-RS minimize investigator and subject burden by providing two validated uses for 1 diary:
    • Assessment of COPD exacerbations via EXACT Total Score
    • Quantification of respiratory symptoms in stable COPD using RS-Total and subscale scores

15. Can the EXACT® be used for estimating utility?

  • Dr. Jennifer Petrillo developed a preference-based scoring algorithm to estimate utilities of an exacerbation from a subset of items in the EXACT for use in cost-effectiveness models.
  • This scoring algorithm, known as the “EXACT-U” was developed according to NICE guidelines and there is evidence to suggest EXACT-U scores distinguish stable and acute COPD states, differentiate clinician-rated levels of exacerbation severity and are responsive to change.
  • See Petrillo & Cairns 2011 for more information.

16. Can the EXACT® be used to evaluate outcomes in other respiratory diseases?

  • Although the EXACT was developed and validated for use in COPD, there has been interest in evaluating the suitability of the instrument for evaluating exacerbations and respiratory symptoms in other conditions with similar respiratory attributes, including interstitial pulmonary fibrosis (IPF), cystic fibrosis, and alpha-1 antitrypsin deficiency.
  • Additional qualitative and quantitative work will be required to test and support the use of the EXACT in any new populations to meet the standards for PROs outlined by the US FDA Guidance for Developing Patient-Reported Outcome Measures for Use in Medical Product Development to Support Labeling Claims.
  • A poster describing the qualitative evaluation of the EXACT for use in IPF was presented at the ISPOR conference in 2013.
    • Murray L, Vernon M, O’Quinn S, Martinez F, Parker J, Leidy NK. The EXACT in idiopathic pulmonary fibrosis (IPF): content validity of an existing patient-reported outcome measure for use in a rare disease. Poster presented at: International Society for Pharmacoeconomics and Outcomes Research (ISPOR); May, 2013; New Orleans, LA.
  • A poster describing the qualitative evaluation of the EXACT for use in CF was presented at the NACFC conference in 2009.
    • Ajayi T, Murray L, Oresanya J, Richardson P, Miller L, Leidy NK. A cognitive debriefing study of a patient-reported outcome tool (EXACT) for quantification of pulmonary exacerbations in CF. Poster presented at: North American Cystic Fibrosis Conference (NACFC); October, 2009; Minneapolis MN.
  • Please contact Evidera at exactpro@evidera.com if you are interested in different contexts of use for the EXACT or E-RS.
© 2017 Evidera     +1 301 654 9729 (US)     +44 (0) 208 576 5000 (EU)