Development and content validation of two new patient-reported outcome measures for endometriosis: the Endometriosis Symptom Diary (ESD) and Endometriosis Impact Scale (EIS)

Background: Endometriosis is a common, chronic, impactful condition in women of reproductive age. In the absence of established sensitive and specific biomarkers, disease severity is determined by patient-reported symptoms and impacts. This article details the development of two new patient-reported outcome (PRO) measures designed to assess efficacy endpoints in clinical studies: The Endometriosis Symptom Diary (ESD) and the Endometriosis Impact Scale (EIS).

The ESD and EIS were developed according to best practice and scientific standards (including the Food and Drug Administration (FDA) PRO Guidance) and with extensive input from women with surgically-confirmed endometriosis. Research included: a review of published qualitative literature; concept elicitation interviews in the US, Germany and France ( n = 45) to explore the experiences of women with endometriosis and to inform ESD and EIS development; and cognitive interviews in the US and Germany ( n = 31) to assess relevance and understanding of the ESD and EIS and usability of administration using an electronic handheld device. The FDA and the European Medicines Agency (EMA) as well as PRO and clinical experts were consulted throughout the process.

Pelvic pain was identified as the most frequent, severe and bothersome symptom for women with endometriosis. Pain was reported to be greatest during menstruation (dysmenorrhea) and during or after sexual intercourse (dyspareunia). Pain resulted in significant impairments in physical activities, work/study, social/leisure activities, household activities and sexual functioning. All women highlighted the emotional impact of endometriosis. Descriptions of pain and associated impacts were largely consistent across participants from the US and Europe, with the most notable differences being the words used to describe the location of pain (e.g., ‘pelvis’ vs. ‘abdomen’). Testing during cognitive interviews indicated that the ESD and EIS were well understood and consistently interpreted. Furthermore, all participants found the ePRO devices easy to use and no issues regarding visual presentation, selection of responses or navigation were identified.

Evidence from extensive qualitative research supports the content validity of the ESD and EIS as patient-reported measures of the disease-defining symptoms of endometriosis and the associated impact on women’s lives. Future research will seek to establish the measurement properties of the measures. (Continued on next page) © The Author(s). 2020 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. * Correspondence: [email protected] 1Adelphi Values, Cheshire, UK Full list of author information is available at the end of the article Journal of Patient- Reported Outcomes Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 https://doi.org/10.1186/s41687-020-0177-3 (Continued from previous page)

Endometriosis, Patient-reported outcomes (PROs), Endometriosis associated pelvic pain (EAPP), Endometriosis symptom diary (ESD), Endometriosis impact scale (EIS), Health-related quality of life (HRQoL), Development, Qualitative, Content validity

Endometriosis is a common chronic condition estimated to affect as many as 10% of women of reproductive age [1]. The condition is characterised by chronic pelvic pain, dysmenorrhea and dyspareunia. Past research has indicated that women experience significant functional disability and deficits in health-related quality of life (HRQoL) [ 2–9] as a result of these symptoms. Accord- ingly, the costs associated with endometriosis in terms of direct healthcare expenditure and indirect costs (e.g., reduced work productivity) are considerable [ 10, 11]. While changes in the number and size of endometriotic lesions have traditionally been used to assess the efficacy of treatments for endometriosis [ 12–15], studies have sug- gested that the extent of lesions is only weakly associated with the severity of pain [ 16–18]. In the absence of estab- lished sensitive and specific biomarkers, the key symptoms and impacts associated with endometriosis can only be measured by direct reports from women themselves [ 19]. Therefore, there is a need for reliable and well-defined PRO measures that can be used to determine the clinical benefit of medical interventions. Evidence of content validity (i.e., the extent to which the content of an instrument is an adequate reflection of the construct to be measured) [ 20] and is measuring what is important to patients within the intended context of use is often regarded as the most important measurement prop- erty of PRO measures [ 21]. To ensure content validity, concepts assessed by PRO measures should be informed by members of the target patient population and measures should be worded in such a way that is relevant, meaning- ful and consistently understood by this population. The Biberoglu and Behrman (B&B) scale has traditionally served as the standard clinical outcome assessment for endometriosis symptoms (including pelvic pain) in both clinical trials and clinical practice [ 22]. However, review of the B&B reveals a number of critical limitations (e.g., the B&B was developed primarily by clinicians with little to no direct involvement of women with endometriosis) that question the content validity of the measure and the ex- tent to which it can be considered a reliable, valid and sen- sitive assessment of patients' experiences of endometriosis [23, 24]. To address these limitations, two new electronic PRO (ePRO) measures have been developed based on exten- sive involvement of women suffering from endometriosis in close accordance with the FDA PRO Guidance [ 21] and best practices established by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) PRO Good Research Practices Task Force [ 25, 26]. The Endometriosis Symptom Diary (ESD) is a patient-reported daily diary assessing the key symptoms of endometriosis, while the Endometriosis Impact Scale (EIS) assesses the impact of endometriosis symptoms over the past 7 days. Multinational qualitative research conducted to inform the initial development of the ESD and EIS and to provide evidence of the content validity of these measures is summarised.

Development, refinement and confirmation of the valid- ity of the ESD and EIS were conducted in stages, con- sistent with accepted best practice [ 21, 25, 26] (Fig. 1). The FDA and the European Medicines Agency (EMA) as well as PRO and clinical experts were consulted dur- ing the process. Stage I: targeted literature review A targeted review of qualitative research studies in women with endometriosis was conducted to identify concepts that are relevant and important to women with endometriosis. Articles were identified via keyword searches conducted in MEDLINE, EMBASE and PSY- CINFO. Searches comprised a combination of disease (e.g., ‘endometriosis’), data collection (e.g., ‘interviews’, ‘focus groups ’) and analysis (e.g., ‘thematic analysis ’, ‘grounded theory ’, ‘discourse’, ‘phenomenological’) terms limited to adult participants and articles published in English in the past 10 years (2004 –2014). Qualitative re- search articles exploring the symptoms and associated impacts of endometriosis were reviewed in full. Articles were excluded if qualitative methods or analysis were not used and if abstracts were not related to the experi- ences of women with endometriosis. Articles selected for full-text review were evaluated and salient information pertaining to study aim(s), sample demographic charac- teristics, methodology, and results were summarized. Key concepts relating to women ’s experience of symp- toms and impacts of endometriosis were used to inform the development of interview guides for subsequent con- cept elicitation interviews. Stage II: concept elicitation interviews Semi-structured interviews were conducted with 45 women who had a surgically confirmed diagnosis of endometriosis to comprehensively understand the Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 2 of 17 experience of endometriosis symptoms and the impact of these symptoms on various aspects of the womens' daily lives (e.g. physical activities, emotional well-being, sexual activities, paid work or study). Recruitment Women were recruited from the United States (US; n = 15), Germany ( n = 15) and France ( n = 15). A sample of 15 participants for each country was targeted with the aim of achieving conceptual saturation in each country. Conceptual saturation has previously been shown to be achievable in 12 –15 individual interviews [ 27, 28]. EU countries with Latin-derived (i.e. France) and Germanic (i.e. Germany) languages were selected to promote lin- guistic and cultural diversity. Participants were recruited via referrals from treating physicians. Eligibility criteria for participation in the interviews were reflective of the criteria typically employed in clinical endometriosis studies. Specifically, all participants were required to have been diagnosed with stage I to IV endometriosis (according to revised American Society for Reproductive Medicine score classification) [ 29], as determined by laparoscopy or laparotomy in the past 5 years. Partici- pants were also required to have recently experienced pain due to endometriosis – as verified by a participant- reported score of ≥3 on an 11-point numeric rating scale (NRS) assessing worst endometriosis-associated pain in the last 24 h at the time of screening (0 = no pain; 10 = pain as bad as you can imagine). Recruitment quotas were employed to ensure demographic and clinical di- versity in the study sample. Interview procedure Interviews (lasting approximately 1 h) were all con- ducted face-to-face by female interviewers with extensive experience of conducting qualitative interviews among people with a variety of health conditions. Interviews were conducted in local language using a semi- structured interview guide (which was developed in US- English and formally translated for use in France and Germany). All interviewers received a detailed briefing on the study objectives, content of the interview guides Fig. 1 Overview of research stages to develop and evaluate the content validity of the Endometriosis Symptom Diary (ESD) and Endometriosis Impact Scale (EIS) Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 3 of 17 and adverse event reporting procedures. All interviewers also engaged in a mock interview prior to the com- mencement of interviews. Broad, open-ended questions were asked initially, with care taken not to lead or direct participant responses and to provide every opportunity for concepts to be men- tioned ‘spontaneously’. Focused probes were only used to elicit feedback on potentially relevant concepts that did not arise spontaneously during the course of the interview. Prior to the interview, participants were asked to create a collage that ‘represented their experience of endometri- osis’ which was subsequently discussed during the inter- view and used as a means to facilitate further spontaneous (i.e., patient-directed) elicitation of concepts. Analysis All interviews were digitally recorded, transcribed verba- tim in the language in which they were conducted, and (for German and French transcripts) subsequently trans- lated into US-English. A software package (Atlas.Ti) was used to facilitate the storage and qualitative analysis of interview transcripts using thematic analysis [ 30]. Each transcript was coded individually with the first two tran- scripts in each country used to create a coding scheme to be used throughout the analysis process by two separate analysts (the content and the data quality reviewer) to monitor and establish consensus in the coding scheme. As new codes emerged throughout the process, transcripts were reread and analysed to ensure all codes were consist- ently applied. All reported qualitative data were verified through a review of source data from the transcripts. Saturation, defined as the point at which no new rele- vant or important information emerges with the collec- tion of more data, was assessed to confirm that the concepts elicited by participants in each country had been fully explored [ 21, 28, 31]. Sequential sets of inter- views (i.e. , interviews 1 –5 vs. 6 –10 vs. 11 –15) were com- pared to one another. If no new concepts were elicited during the final set of interviews (i.e., interviews 11 –15) then saturation was said to have been achieved. Stage III development of the ESD and the EIS Based on information derived from the targeted litera- ture review and concept elicitation interviews, draft items, instructions, response options and hypothesised conceptual frameworks for US-English, French and Ger- man versions of the ESD and EIS were developed. Input was sought from expert clinicians (to ensure the clinical relevance of items), linguistic validation specialists (to ensure the cross-cultural validity and translatability of the items) and ePRO vendors (to ensure ease of imple- mentation of measures on electronic devices). Formal translatability and lexibility assessments were conducted to determine the appropriateness of the draft measures for adaptation to other languages and for use in respon- dents with low levels of literacy. The ESD and EIS were developed for completion on a handheld electronic device. This has notable advantages over traditional pen-and-paper methods in terms of pro- viding confidence as to when questionnaires have been completed (preventing back-filling or forward-filling of questionnaires), implementing safeguards for avoiding missed completions (e.g. through use of alarms) and minimising time and potential errors associated with subsequent manual entry of questionnaire data [ 32]. Stage IV: cognitive interviews Semi-structured cognitive interviews and pilot testing were conducted with women with endometriosis to evaluate the relevance and participant understanding of draft items, instructions and response options. The us- ability of the handheld ePRO device (TrialMax Touch eDiary; CRF Health, Plymouth Meeting, Pennsylvania, US) was also assessed during these interviews. Recruitment An independent sample (i.e., not including those women who participated in concept e licitation interviews) of 31 women with endometriosis in the US ( n = 19) and Germany ( n = 12) were recruited for participation in the cognitive interviews. Prior research has recommended cog- nitive interview sample sizes of 30 or more be preferred in order to achieve a reasonable power to detect prevalent problems [33]. Participants were subject to the same eligi- bility criteria employed during the concept elicitation inter- views and recruitment quotas were implemented to ensure a diverse sample. Note that cognitive interviews were not performed in France due to difficulties identifying women eligible for participation during the prior concept elicitation interviews. Furthermore, a greater proportion of US partici- pants were targeted for recr uitment to account for demo- graphic diversity in the US population. Procedure Each participant attended two study visits (Fig. 2). Dur- ing visit 1, the participant ’s understanding and compre- hension of the EIS was assessed using a “think aloud ” technique, whereby participants were asked to speak aloud their thoughts while responding to the EIS ques- tions. Following the “think aloud ” exercise, participants were asked about the relevance and understanding of EIS items, instructions and response options. The partic- ipants were also trained on use of the ePRO device, which they were required to take home for completion of the ESD on a daily basis for 7 –10 days and comple- tion of the EIS at the end of Day 7. This was designed to mimic how the questionnaires would be implemented in a clinical study. At visit 2, participants provided feedback Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 4 of 17 regarding comprehension and understanding of the ESD using the “think aloud ” method described above. Their experience of completing the ESD and EIS, including us- ability of the ePRO device was also explored. For both visits, a semi-structured interview guide was used to en- sure that all areas of the ESD and EIS were discussed. Interviews were conducted in two separate rounds to allow implementation of modifications to the measures following round 1 (US, n = 5; Germany, n = 4), before testing in round 2 (US, n = 14; Germany, n = 8). Analysis Visit 1 and 2 interviews were digitally recorded and tran- scribed verbatim in the language in which they were conducted (with German transcripts subsequently trans- lated into US-English). Atlas.Ti was used to facilitate analysis with participant quotes used to determine un- derstanding/clarity and the relevance of each instruction, item and response option for each participant.

Stage I: targeted literature review A total of 14 articles met the pre-specified criteria for in- clusion in the review [ 2–8, 23, 34–39]. Pain in the pelvic region was identified as the predominant symptom of endometriosis [ 2–8, 23, 35, 38, 39]. Pain was reported to be experienced at any time, although pain specifically as- sociated with menstrual bleeding [ 7, 23, 38] and sexual intercourse [ 2, 3, 5, 23, 39] was commonly reported. Women with endometriosis characterized pain using a variety of sensory descriptors which can be broadly cate- gorized as either continuous/constant or intermittent/ short-term [ 7, 37]. Endometriosis-associated pain was reported to have a sig- nificant impact on numerous facets of women ’sl i v e s including physical functioning [7, 38, 39], ability to work [ 2, 6, 7, 38, 39] and to carry out activiti es of daily living (e.g., housework) [7], social functioning and personal relationships [2, 5–8, 38, 39]. Endometriosis was also reported to have a considerable emotional impact on women, with women feel- ing both depressed and irritable/moody [7]. In addition, dys- pareunia was reported to significantly impact women ’s sexual relationships; women frequently reported avoiding intercourse because of expected or experienced pain, and in- dicate that this puts strain on the relationship with their partner [2, 3, 5, 7]. Further impacts reported include im- paired sleep, inability to concentrate and reduced appetite [7]. Stage II: concept elicitation interviews Table 1 shows the demographic and the clinical charac- teristics of women who participated in the concept elicitation interviews. Pain was mentioned by all 45 participants (96% of who mentioned this spontaneously) and was described by the vast majority of participants as being the most frequent (92%), most severe (92%) and most bothersome (86%) symptom that they experience: /C15“The pain, that ’s for sure. The nausea, the fatigue, and the dizziness and all, I ’m sure I could deal with a lot better if they were alone ” (101)1 /C15“If the pain could be wiped out, then nothing else has really been troublesome. It ’s just the pain. ” (104) 1Number represents unique participant ID. IDs with a prefix of 1 –3 present interviews with US participants (Philadelphia, Los Angeles and New Orleans). Prefixes of 4 and 5 represent participants from Germany and France, respectively. Fig. 2 Overview of cognitive interview process Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 5 of 17 Participants used a variety of sensory descriptors to describe their experiences of pain, such as sharp/shoot- ing/stabbing ( n = 32), cramping/contractions ( n = 26), and dull/aching pain ( n = 16). Two distinct types of pain were identified ( ‘constant’ and ‘short-term’ pain). The sensory descriptors used to describe these types of pain were not mutually exclusive, rather these types of pain were instead differentiated by temporal characteristics. While pain was typically considered to be at its greatest during menstruation, participants reported experiencing pain throughout the entire menstrual cycle (including pre- menstrual pain, menstrual pain, post-menstrual pain and non-menstrual pain). Participants indicated that pain occurring outside a menstrual period frequently could not be differentiated from cyclical pain, nor dissociated from pain with periods. Pain was also reported both during (n = 24) and following sexual intercourse (n =1 6 ) . When asked about the location of their pain, partici- pants most commonly referred to the pain occurring in the pelvic region (including uterus, ovaries, and bladder; n = 37), abdominal region (including stomach; n =4 0 ) , and lower back ( n = 36). Pain in the legs was also men- tioned by participants ( n = 24); however, this was largely described as being a result of pain radiating down from the pelvic region. Pain descriptions were largely consistent across participants from the US and Europe; the most Table 1 Demographic and clinical characteristics of study samples used to establish the content validity of the ESD and EIS Demographic and clinical characteristics Concept Elicitation ( n = 45) Cognitive Debriefing ( n = 31) Age Mean (Range) 33.9 (18 –44) 36.3 (21 –45) Country n (%) United States 15 (33.3) 19 (61.3) Germany 15 (33.3) 12 (38.7) France 15 (33.3) N/A Ethnic Background n (%) a Caucasian or White 10 (66.7) 8 (42.1) African American 3 (20.0) 9 (47.4) Other 2 (13.3) 2 (10.5) Education Level n (%) High school diploma or GED 4 (8.9) 3 (9.7) Some years of college 8 (17.8) 5 (16.1) Certificate program 7 (15.6) 9 (29.0) College or University degree 1 (2.2) 10 (32.3) Graduate or professional degree 25 (55.6) 4 (12.9) Experience of pain at screening n (%) b No pain (0) 0 (0.0) 1 (3.2) d Mild (1–4) 16 (35.6) 9 (29.0) d Moderate (5–6) 14 (31.1) 8 (25.8) Severe (7–10) 15 (33.3) 13 (41.9) Mean (range) 5.8 (3 –10) 5.9 (0 –10) Current treatment n (%) c Painkillers 22 (48.9) 19 (61.3) Oral contraceptives 13 (28.9) 11 (35.5) Other hormonal contraceptives 3 (6.7) 5 (16.1) GnRH analogue 4 (8.9) 5 (16.1) Progestin 3 (6.7) 1 (3.2) Other 3 (6.7) 1 (3.2) aData not available for European sample; therefore, percentages calculated based on US sample only (concept elicitation: n = 15, cognitive debriefing: n = 19) bSelf-rated assessment of endometriosis-associated pain at its worst in the last 24 h (using a 0 –10 NRS where 0 = no pain and 10 = pain as bad as you can imagine). Classifications of pain based on Serlin et al. (1995) [ 40] cCounts not mutually exclusive dOf note: Two patients from the US sample had NRS scores < 3 Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 6 of 17 notable differences being the words used to describe pain location (e.g., ‘pelvis’ vs. ‘abdomen’). In addition to pain, many women ( n = 33) referred to vaginal bleeding, including heavy menstrual bleeding (n = 30) and unpredictable bleeding or spotting outside of their usual menstrual cycle ( n = 14) (Table 2). A range of other symptoms were reported by women with endometriosis including: tiredness ( n =4 0 ) ; h e a d - aches ( n = 29); abdominal bloating ( n =2 6 ) ; n a u s e a (n = 24); constipation ( n = 14); vomiting ( n =1 2 ) ; d i z z i n e s s (n =1 2 ) ; d i a r r h o e a (n = 9); lack of energy ( n = 9); loss of appetite ( n = 8); bloody stools ( n = 7); frequent need to urinate (n = 7); feeling of heaviness ( n = 6); painful breasts (n = 6); weight loss/weight gain ( n = 6); and fever ( n =5 ) . These symptoms, however, were mentioned much less frequently than pain associated with endometriosis and bleeding irregularities, and in the majority of participants were not considered to be linked to their endometriosis. As such, these may be considered secondary rather than primary symptoms of endometriosis (something later con- firmed via discussions with expert clinicians). At its worst, endometriosis-related pain was extremely de- bilitating for women, impacting many facets of their lives: “Because it’s painful. It hurts. I don ’t like the way it feels. It d i s r u p t sm yw h o l el i f ea tt h a tt i m et h a ti t’s going on.” (304). Participants spoke in general about how the pain affected their usual tasks and activities on a daily basis: “See, the stabbing pains are bothersome because they affect all my daily activities. Not only what I do, but what I would want to do” (306). Such impairments manifested in impaired ability to participate in: physical activities (n = 44), work and study (n = 39), social and leisure activities (n = 35) and household activities (n = 31) and sexual activity (n = 30/41). Participants also reported an impaired ability to sleep ( n =3 6 ) , c o n c e n - trate (n = 28) and eat (n =8 ) .U s eo fp r e s c r i p t i o no ro v e rt h e counter pain medications was common among patients (n = 39/45). Indeed, reports from patients implied that pain medi- cation was an integral part of managing their condition and minimizing the impact of endometriosis on their daily lives: “And then, I take analgesics… otherwise I can’tw o r k ,i nf a c t .” (508). All 45 women interviewed referred to the significant emotional impact of endometriosis (Table3). Consideration of qualitative data obtained during the in- terviews revealed that no new concepts were elicited dur- ing the final set of interviews and that saturation was achieved within this sample. Ninety percent of all concepts were elicited in the first round of five interviews in each of the three countries (US, Germany and France). Further- more, all concepts identified were elicited in each country. Stage III: development of the ESD and the EIS ESD The ESD was developed as a patient-reported electronic d i a r yt oa s s e s st h ek e ys y m p t o m sa s s o c i a t e dw i t h Table 2 Concept elicitation interviews: overview of key symptoms reported by women with endometriosis ( n = 45) Concept Sub-concept Example Quote (Patient ID#) Pain (n = 45) Constant pain ( n = 40) “The consistent ache - you see, because the consistent ache, the dull ache is there every single - it ’s hurting right now as I ’m sitting here talking to you. ” (306) Short-term pain ( n = 31) “I’ll get a pain in my pelvic area but it ’s short-term. The duration isn ’t like it is when my cycle is on. ” (104) Dysmenorrhea Pain during period ( n = 43) “But on - during my menstrual cycle, when I was having my period, the pain was excruciating. At times it literally felt like a sword had been pierced right through my side, my left side ” (306) “…and sometime the pain is just - you know, especially around my menstrual period, it’s very painful, that all I can do is lay on the couch and just - and I might take Tylenol, but sometimes that doesn ’t even help. ” (206) Pre-menstrual pain ( n = 28) “I have a lot of pelvic pain and it gets worse, I think, about the week before my period, it starts to get really bad. I get a lot of stomach pains, a lot of lower back pain and then I get the period and the bleeding. “(106) “Well, with my symptoms, they ’re usually worse like the week before my period and during my period. So usually like the week before, I ’m just in bed, very fatigued, feeling - just hurting, like cramping, like as if you would have menstrual cramps. I ’m like that the week before my period, and then during my period, it ’s like 100 times worse ”. (204) Dyspareunia Pain during intercourse ( n = 24) “I’ve had intercourse a couple of times, and sometimes that ’s been very painful … It’s like a sharp pain. And then it ’s a sharp shooting pain, and then it ’s throbbing together afterward.” (206) Pain after intercourse ( n = 16) “The same type of pain, it sometimes stayed for 24 h, sometimes a few hours, sometimes … sometimes 2 or 3 days and then … that’s it.” (501) Vaginal bleeding ( n = 33) Heavy bleeding ( n = 30) “They’re extremely heavy, like bad heavy. It doesn ’t lighten up until like the last day. ” (207) Spotting (n = 14) “I’ll actually sometimes, like if I push myself too far, I ’ll start bleeding. And sometimes it may just be like spotty bleeding. ” (302) Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 7 of 17 endometriosis: pelvic pain, dysmenorrhea and dyspareunia. The ESD is designed to be completed once daily, with a re- call period of the past 24 h. This recall period was selected to account for the day-to-day variability in the presentation (i.e., menstrual pain and non-menstrual pain, event-driven dyspareunia) and severity of endometriosis symptoms and to minimize recall error associated with asking respondents to recall their experiences over a long period of time [ 21, 41]. The draft ESD comprised 12 items. ESD pain items instruct respondents to rate their pain ‘at its worst’, as there is evidence suggesting that ratings of worst pain are more reliable than reports of average pain and are most representative of the burden of pain [ 24, 42]. ESD items assessing pain utilise a numeric rating scale (NRS) ranging from 0 ( ‘no pain’)t o1 0( ‘pain as bad as you can imagine ’); a widely used measure of pain intensity [43–46] recommended for assessment of pain in clinical trials [47] and measurement and assessment of pain asso- ciated with endometriosis [ 24, 48]. Past research indicates that NRSs are sensitive to changes in levels of pain and a r ee a s i l yu n d e r s t o o db yr e s p o n d e n t s[49]. PRO measures historically implemented in endometri- osis clinical trials (e.g., B&B) make specific reference to the location of pain experienced by women (for example: ‘pelvic pain’ or ‘abdominal pain’). It is important to differ- entiate pain that may realistically be associated with endo- metriosis from other types of pain that may be experienced by women with endometriosis (e.g., head- aches, breast pain). However, as demonstrated by findings from the concept elicitation interviews, women with endo- metriosis frequently experience pain in more than one re- gion and use a variety of different terms to describe the location of pain. Such differences in terminology were notably evident when comparing data between countries. For example, when provided with a diagram on which to circle their areas of pain, women in each country highlighted similar areas. However, when describing the pain location verbally, US participants more commonly used the term pelvic, while pain in the abdomen was the most prominent location descriptor used by participants in Germany and France. Therefore, to facilitate compre- hension and understanding, the ESD and EIS include a diagram (depicting front and rear-view body maps) that highlights the areas in which women with endometriosis typically experience pain associated with their endometri- osis. This area is referred as the ‘target area ’ and refer- enced throughout ESD and EIS items and instructions in place of verbal descriptors of pain location. The use of the term ‘target area’ avoids the use of specific clinical termin- ology which may be difficult for low literacy respondents to interpret and may vary across languages and cultures. Dysmenorrhea is widely recognised as a cardinal symp- tom of endometriosis and included in historical measures of endometriosis symptom severity (e.g. B&B). Feedback obtained during the concept elicitation interviews highlighted concerns regarding participants’ ability to reli- ably attribute their pain to bleeding or differentiate be- tween menstrual pain and non-menstrual pain. Therefore, the initial draft of the ESD included items assessing ‘worst pain due to your period ’ as well as a daily assessment of vaginal bleeding (‘none’, ‘spotting’, ‘light’, ‘normal’, ‘heavy’). The inclusion of a daily assessment of bleeding is consist- ent with recommendations from the Art of Science Endo- metriosis meeting [ 24] and prior feedback from the FDA and facilitates additional assessment of dysmenorrhea and non-menstrual pelvic pain based on independent assess- ments of pain (i.e., ESD item 1) and bleeding without rely- ing on respondent attribution. Items assessing dyspareunia were developed based on the findings from the literature review and qualitative in- terviews. It is important that any daily assessment only accounts for days where the respondent did engage in sexual activities and is therefore able to provide a rating of dyspareunia. For that reason, an initial item asking whether the respondent had (or did not have) sexual intercourse was developed. Assessment of use of analgesics and pain-reliving medi- cations is key for understanding womens ’ experiences of pain and to help demonstrate treatment benefit in endo- metriosis [24] and such measures are included within gen- eric pain assessments (e.g. Brief Pain Inventory-Short Form) and traditional assessments of endometriosis symp- toms (e.g. B&B). Given the intended use of the ESD within the context of a clinical trial, the ESD includes items assessing use of both protocol-specified supportive pain medication as well as use of additional pain medication. The conceptual framework of the ESD and example screenshots are provided in Figs. 3 and 4, respectively. EIS The EIS was developed as a PRO measure providing a comprehensive assessment of the impact of endometriosis pain on various facets of women ’sl i v e s .I na c c o r d a n c e with findings from the concept elicitation interviews and IMMPACT recommendations for core outcome domains to be assessed in chronic pain clinical trials [ 50], the pri- mary focus of the EIS is assessment of the impact of endo- metriosis symptoms on women ’s physical activities, emotional well-being and sexual activities. The EIS is designed to be completed using an ePRO de- vice, once weekly with a recall period of the past 7 days. This recall period was selected as the optimum compromise between potential recall problems and responder burden. Subsequent research has demonstrated that for assessment of the impact of endometriosis pain on physical activities, a 7-day recall period provides data that is consistent with daily administration of the same items (with a 24 h recall period) over the same period [51]. Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 8 of 17 The draft EIS comprised 32 items. All items in the EIS use a 5-point verbal rating scale ( ‘Not at all ’, ‘Slightly’, ‘Moderately’, ‘A lot ’, ‘Extremely’), with an optional ‘does not apply ’ option included for concepts that may not be relevant to respondents (e.g., sexual activities). This is consistent with other PRO measures investigating the impact of endometriosis / comparable conditions such as the Endometriosis Health Profile-30 (EHP-30) [ 52] and the Menorrhagia Impact Questionnaire [ 53]. The conceptual framework of the EIS and example screenshots are provided in Figs. 5 and 6, respectively. Stage IV: cognitive interviews Demographic and clinical characteristics of the 31 women with endometriosis who participated in the cog- nitive interviews are summarised in Table 1. Conceptual coverage Concepts and descriptions to emerge from open-ended

with participants during the cognitive inter- views were in line with findings from the concept elicitation interviews. When asked during the cognitive interviews, participants indicated that the ESD and EIS captured all symptoms and impacts associated with their experience of endometriosis. There were no concepts within the ESD and EIS that were deemed to not be rele- vant, although some conceptual overlap between items (particularly those comprising the emotional well-being domain of the EIS) was noted. Understanding and interpretation of the ESD and EIS Feedback from participants indicated that ESD and EIS instructions, items and response options were well understood and consistently interpreted. All par- ticipants demonstrated understanding of the term ‘tar- get area ’ and all participants reported that the target area depicted on the diagram covered those areas where they experience pain related to their endomet- riosis. Participants were able to select responses using both the 0 –10 NRS employed by the majority of ESD items and the 5-point verbal rating scale used by EIS items. Feedback from pa rticipants during the ‘think- Fig. 3 Endometriosis Symptom Diary (ESD) conceptual framework Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 9 of 17 aloud’ exercise highlighted correct use of ESD and EIS recall periods with participants thinking back to the past 24 h and past week for the ESD and EIS, respectively. Of note, no differences in understanding and comprehension were observed between US and German participants. Fig. 4 Endometriosis Symptom Diary (ESD) example US-English screenshots Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 10 of 17 Table 3 Concept elicitation interviews: overview of key impacts reported by women with endometriosis ( n = 45) Concept Sub-concept Example Quote (Patient ID#) Physical activities ( n = 44)a Lie down ( n = 42) “It was intense pain in … at times I really felt too much pain in my abdomen, and I felt that the only way … for it to go was to lie down … and I had to lie down. ” (503) Walking (n = 22) “I can hardly walk, and as I said, without pain is out of - pain killers - is out of question. ” (406) Standing (n =2 5 ) “And some days I was like I can ’t even stand up because I ’m in pain. ” (305) Sitting (n =1 4 ) “I can ’t drive for long periods of time. I can ’t sit up straight for long periods of time. ” (306) Lifting (n = 16) “Yeah, that when I get the pain, I ’m like I can ’t pick kids up. It hurts. My stomach hurts. ” (301) Carrying (n =1 6 ) “There are heavy things, let ’s say, when my abdomen hurts too much, anything heavy I don ’t carry it. ” (505) Exercise (n = 28) “Running, jogging, that ’s absolutely out of the question. I used to run two miles a day, and weight train for an hour a day, and I can ’t do that anymore ” (306) Emotional state ( n = 45) Sad ( n = 21) “Yes, the sadness because my kids want to do things, and we don ’t go out and stuff sometimes. ” (207) Depressed (n =2 2 ) “Because it makes you depressed, because you are in bed. ” (204) Irritable (n =1 3 ) “Like it [the pain] just makes me irritable, like I said. ” (207) Bad mood ( n = 23) “When I am in pain, I have mood swings and I know the pain causes them. ” (513) Angry (n = 15) “I just get - when the pain comes on I get angry ” (305) Annoyed (n = 19) “it was frustrating and annoying, because I had to leave work constantly to go to doctor appointments all the time ” (301) Anxious (n = 23) “Just depression and anxiety because of pain. ” (305) Stressed (n = 10) “I feel stressed, just waiting at some point of the day to be ambushed by an excessive amount of pain ” (306) Sexual activities ( n =3 0 )b Avoidance (n =2 3 ) “I final stopped (having sex) altogether - because it was just no longer tolerable. ” (410) Limited enjoyment ( n =1 2 ) “At each penetration … when it’s painful. So, it ’s quite unpleasant, it ’s unpleasant for me ” (509) Work and study ( n =3 9 ) “Because when - I couldn ’t even go to work if I ’m having severe pain, so I ’d have to call in sick. ” (206) “At work or after, I used to be able to do more with my time but now it feels like I ’m slowing down ” (201) Social and leisure activities ( n = 36) “Well, when the pain was very severe I would have liked to curl up like a hedgehog somehow, that one - that I be left alone - I do not want to have to deal with anyone ” (403) “…my partner and a bunch of friends were having a big potluck dinner at their house. And they live across town, and it would have been a bike ride and I wasn ’t feeling well, so I didn ’t go.” (101) Household activities ( n =2 7 ) “…when I’m not feeling good, then it just kind of sits there and then when I feel better, I go in and do it. ” (106) “It slows me down like with all the usual things I have to do at home. ” (302) Impaired sleep ( n = 36) “A lot of times I ’ll wake up in pain. If I ’ve slept through my next opportunity to take another dose of pain medication, then the pain will wake me up and I ’m just sort of awake after that. ” (101) Impaired concentration (28/45) “I mean, trying to drive a car and you ’re in pain. Well, that ’s taking your - you ’re distracted. You ’re thinking about that, not what you should be. You ’re not thinking about your job or everyday life activities. ” (301) Eat less (8/45) “Because the pain ’s so bad I feel sick to my stomach. I feel like I can ’t eat.” (301) a Number of participants reporting impact during CE interview bNote that four study participants commented that they were not sexually active at the time of the study Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 11 of 17 In light of the feedback from participants during the first round of cognitive debriefing some minor changes in the wording of ESD instructions and items were im- plemented and tested during round 2. ePRO device usability All participants found the ePRO devices easy to use. Of particular note, no issues regarding visual presentation, selection of responses or navigation were identified. Fur- thermore, good compliance, low levels of missing data and short average completion times were observed for daily completion of the ESD (2.5 min) and weekly completion of the EIS (5.45 min) during the 7 –10-day completion period which allay any potential concerns re- garding responder burden.

Establishing content validity is critical for any PRO measure. This is especially the case for any PRO meas- ure intended to be used in clinical studies to support product labelling claims regarding treatment benefit, as evidence of other types of validity (e.g., construct valid- ity) or reliability (e.g., reproducibility of scores) will not overcome problems with content validity [ 21]. The Fig. 5 Endometriosis Impact Scale (EIS) conceptual framework Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 12 of 17 extensive evidence compiled from qualitative research among women with endometriosis and presented here serves as a critical foundation for the content validity of the ESD and EIS. Specifically, findings confirm that the ESD and EIS assess the key symptoms and impacts of relevance and importance to women with endometriosis and that both measures are understood and interpreted consistently by respondents. Exploration of linguistic Fig. 6 Endometriosis Impact Scale (EIS) example US-English screenshots Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 13 of 17 and cultural differences in the way in which women with endometriosis talk about their experiences within the published literature are limited, but findings from the present study revealed the conceptualizations of women’s experiences of endometriosis to be very similar across the US, Germany and France. The simultaneous development of the ESD and EIS in the US and Europe is a key strength, with care taken to ensure the wording of instructions, items and response options are appropri- ate for women regardless of language spoken or literacy levels. Originally developed in US-English, French, and German, the ESD and EIS have subsequently been trans- lated and linguistically validated for use in approximately 30 languages across Europe, North America, South America, Africa, Asia and the Middle East. The ESD and EIS have both been employed in non- interventional (NCT01643122) and interventional re- search studies (NCT02203331; NCT01822080 [ESD only]). Preliminary insights from these studies have fur- ther supported the usability and feasibility of daily as- sessment of endometriosis symptoms and weekly assessments of endometriosis impacts using ePRO mea- sures, with high levels of compliance and low levels of missing data observed [ 54]. As the next step in the de- velopment and validation of the ESD and EIS, data from these studies is to be used to evaluate item performance and to determine preliminary scoring algorithms for the ESD and EIS. The scope of these analyses will be to identify any poorly performing or redundant items and to understand the relationship between item scores to determine optimal derivation of domain or total scores. Once final item content and provisional scoring algo- rithms for the ESD and EIS have been established then further analyses will be conducted to evaluate the meas- urement properties and psychometric validity of such scores. In particular the ability of ESD and EIS scores to produce consistent scores overtime in a stable patient population (test-retest reliability), the extent to which ESD and EIS scores reflect scores for other PROs meas- uring similar/dissimilar concepts (concurrent validity), are able to discriminate between groups according to key indicators e.g. severity (known groups validity) and are able to detect change when the clinical status of re- spondents has changed (responsiveness) will be ex- plored. Furthermore, definitions of meaningful changes in ESD and EIS scores will be explored. There are unique challenges associated with the devel- opment, psychometric evaluation and implementation of daily diaries such as the ESD [ 55]. In particular, despite being favoured by instrument developers, the process by which daily diary assessments are scored and translated into meaningful and responsive endpoints has received little attention in the literature. This is particularly im- portant in the current context where the daily assessment of pain and bleeding represents numerous options for understanding patient experiences of symp- toms over time (e.g., average pain over observation period, worst pain over observation period, pain on bleeding days, pain on non-bleeding days) as well as change in these symptoms over time (absolute change in symptom severity scores vs relative change in symptom severity scores). As such, alongside research activities de- signed to evaluate the measurement properties of such scores, additional qualitative research is also on-going to explore patient and clinician perspectives regarding der- ivation of scores and definitions of clinically important differences. PRO development can be a lengthy process and, as an area of considerable unmet need, it is not surprising that during the development and validation of the ESD and EIS, similar efforts were underway by other researchers and study sponsors to develop PRO measures for use in this area. For example, evidence regarding the develop- ment and content validity testing of another daily diary, the Endometriosis Pain Daily Diary (EPDD), has recently been published [ 56]. Encouragingly, concepts assessed by the EPDD and ESD are remarkably similar, support- ing the content validity of both measures. However, des- pite these similarities there are notable differences between the ESD and EPDD. For example, a key feature of the ESD (and the EIS) is graphical depictions to aid respondents in reliably and consistency identifying endometriosis-related pain. The ESD also includes as- sessment of continuous and short-term pain as well as assessment of vaginal bleeding severity (rather than just the presence or absence of bleeding) which may be valu- able depending on the mechanism of action for investi- gative products under evaluation for the treatment of endometriosis. That the ESD is complemented by the EIS is also a key strength given the absence of compre- hensive PRO assessments of the impact of endometriosis on women ’s lives that do not overburden patients and meet current regulatory expectations. Finally, while the ESD and EIS have been developed to meet the guidelines and expectations for PRO measures intended to assess endpoints to evaluate the efficacy of new treatments in clinical studies, these measures may also have utility for use in clinical practice in the future. For example, many women with endometriosis experi- ence significant delays in receiving a formal diagnosis of endometriosis (approximately 7 –12 years) [ 57, 58]. As- sessment of the core symptoms of endometriosis from the patient-perspective could facilitate earlier diagnosis of endometriosis, however there are currently no vali- dated measures routinely used by healthcare profes- sionals for this purpose [ 59]. Similarly, the ESD and EIS may have use for monitoring response to treatment in clinical practice and providing additional insights into Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 14 of 17 the burden of disease beyond generic HRQoL (e.g., SF- 36) and legacy PRO measures (e.g., EHP-30) commonly used for this purpose [ 60].

The ESD and EIS are newly-developed PRO measures that have demonstrated content validity for assessment of endometriosis-associated key symptoms and impacts. Developed in accordance with the scientific best practice (including the FDA PRO Guidance), these measures are expected to have important applications for use to assess clinical trial endpoints to support regulatory label claims and for use in clinical practice to inform treatment decisions. Abbreviations B&B: Biberoglu and Behrman Scale; EHP-30: Endometriosis Health Profile-30; EIS: Endometriosis Impact Scale; EMA: European Medicines Agency; ePRO: Electronic patient-reported outcome; ESD: Endometriosis Symptom Diary; EU: European Union; FDA: Food and Drug Administration; HRQoL: Health-related quality of life; ISPOR: International Society for Pharmacoeconomics and Outcomes Research; NRS: Numeric rating scale; PRO: Patient-reported outcome; SF-36 : 36-item Short Form Health Survey; US: United States

We thank all the women and gynecologists who dedicated their time to take part of this study. We would also like to thank Dr. Thomas D ’Hooghe, Dr. David Olive and Dr. Isabelle Streuli for their contributions as part of the expert panel. Thanks are also extended to Sabine Bielfeldt, Isabelle Guillemin and Khadra Benmedjahed (for their help in conducting the interviews in France and Germany). Finally, we extend our thanks to Julia Knierim (formerly of Bayer AG) for her contributions during the initial scoping of this research. Authors’ contributions AG led the overall design and conduct of the research. FT contributed to the conduct and analysis of the literature review, concept elicitation and cognitive debriefing interviews. CS and CG provided review and input throughout all phases of the research. KW provided review and input during cognitive debriefing of the ESD and EIS. AG and CH prepared the first draft of the manuscript which was reviewed and approved by all authors. Funding This study was funded by Bayer AG. Availability of data and materials Written transcripts of participant discussions that have been transcribed and translated verbatim are the primary source of data supporting the initial development of the ESD and EIS. While consent signed by participants indicated that quotes and excerpts may be made publicly available (i.e., via publication), consent to share entire transcripts publicly was not obtained. Therefore, it is not considered appropriate to make the data for this study publicly available. Ethics approval and consent to participate This research was conducted in accordance with the Declaration of Helsinki and was approved by Independent Review Boards in the US (Copernicus Group Independent Review Board) and Germany (Freiburger Ethik- Kommission International). Ethics approval in France was not required for this research, but this study was conducted in compliance with local ethics rules relating to the French personal data protection law stipulated by the data protection authority - Commission nationale de l ’informatique et des libertés. Written informed consent was obtained from all study participants before taking part in this research. Consent for publication Data presented is anonymised and all identifiable information has been removed. Declarations of consent signed by participants made it clear that data may be used for the purposes of publication. Competing interests AG and FT are employees of Adelphi Values, a health-outcomes consultancy commissioned to conduct this research on behalf of Bayer AG. CS, CG, KW and CH are Bayer AG employees. The authors have no other conflicts of interest regarding the content of this article. Author details 1Adelphi Values, Cheshire, UK. 2Adelphi Values, Boston, USA. 3Bayer AG, Berlin, Germany. 4Obstetrics, Gynecology and Reproductive Medicine, University of Saarland, Homburg, Saar, Germany. Received: 10 June 2019 Accepted: 3 February 2020

1. Eskenazi, B., & Warner, M. L. (1997). Epidemiology of endometriosis. Obstetrics and Gynecology Clinics of North America, 24 (2), 235 –258. 2. Denny, E. (2004). Women ’s experience of endometriosis. Journal of Advanced Nursing, 46 (6), 641 –648. https://doi.org/10.1111/j.1365-2648.2004. 03055.x. 3. Denny, E. (2004). ‘You are one of the unlucky ones ’: Delay in the diagnosis of endometriosis. Diversity in Health & Social Care, 1 (1), 39 –44. 4. Denny, E., & Khan, K. S. (2006). Systematic reviews of qualitative evidence: What are the experiences of women with endometriosis? Journal of Obstetrics and Gynaecology, 26 (6), 501 –506. https://doi.org/10.1080/ 01443610600797301. 5. Denny, E., & Mann, C. H. (2007). Endometriosis-associated dyspareunia: The impact on women ’s lives. Journal of Family Planning and Reproductive Health Care, 33(3), 189 –193. https://doi.org/10.1783/147118907781004831. 6. Huntington, A., & Gilmour, J. A. (2005). A life shaped by pain: Women and endometriosis. Journal of Clinical Nursing, 14 (9), 1124 –1132. https://doi.org/ 10.1111/j.1365-2702.2005.01231.x. 7. Jones, G., Jenkinson, C., & Kennedy, S. (2004). The impact of endometriosis upon quality of life: A qualitative analysis. Journal of Psychosomatic Obstetrics & Gynecology, 25 (2), 123 –133. 8. Strzempko Butt, F., & Chesla, C. (2007). Relational patterns of couples living with chronic pelvic pain from endometriosis. Qualitative Health Research, 17(5), 571 –585. https://doi.org/10.1177/1049732307299907. 9. Gao, X., Yeh, Y. C., Outley, J., Simon, J., Botteman, M., & Spalding, J. (2006). Health-related quality of life burden of women with endometriosis: A literature review. Current Medical Research and Opinion, 22 (9), 1787 –1797. https://doi.org/10.1185/030079906X121084. 10. Simoens, S., Meuleman, C., & D'Hooghe, T. (2011). Non-health-care costs associated with endometriosis. Human Reproduction, 26 (9), 2363 –2367. https://doi.org/10.1093/humrep/der215. 11. Simoens, S., Dunselman, G., Dirksen, C., Hummelshoj, L., Bokor, A., Brandes, I., et al. (2012). The burden of endometriosis: Costs and quality of life of women with endometriosis and treated in referral centres. Human Reproduction, 27(5), 1292 –1299. https://doi.org/10.1093/humrep/des073. 12. O'Shea, R. T., & Jones, W. R. (1985). Danazol: Objective assessment in the treatment of endometriosis. Clinical Reproduction and Fertility, 3 (3), 205 –206. 13. Salat-Baroux, J., Giacomini, P., & Antoine, J. M. (1988). Laparoscopic control of danazol therapy on pelvic endometriosis.Human Reproduction, 3(2), 197–200. 14. Bulletti, C., Flamigni, C., Polli, V., Giacomucci, E., Albonetti, A., Negrini, V., et al. (1996). The efficacy of drugs in the management of endometriosis. The Journal of the American Association of Gynecologic Laparoscopists, 3(4), 495–501. 15. Selak, V., Farquhar, C., Prentice, A., & Singla, A. (2007). Danazol for pelvic pain associated with endometriosis. Cochrane Database of Systematic Reviews , (4), CD000068. https://doi.org/10.1002/14651858.CD000068.pub2. 16. The American Fertility Society. (1985). Revised American fertility society classification of endometriosis: 1985. Fertility and Sterility, 44 (2), 7 –8. 17. Vercellini, P., Trespidi, L., De Giorgi, O., Cortesi, I., Parazzini, F., & Crosignani, P. G. (1996). Endometriosis and pelvic pain: Relation to disease stage and localization. Fertility and Sterility, 65 (2), 299 –304. Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 15 of 17 18. Rodgers, A. K., & Falcone, T. (2008). Treatment strategies for endometriosis. Expert Opinion on Pharmacotherapy, 9 (2), 243 –255. https://doi.org/10.1517/ 14656566.9.2.243. 19. Fassbender A, Burney RO, O DF, D'Hooghe T, Giudice L (2015) Update on biomarkers for the detection of endometriosis. BioMed Research International 2015:130854. doi: https://doi.org/10.1155/2015/130854. 20. Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., et al. (2010). The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. Journal of Clinical Epidemiology, 63(7), 737 –745. https://doi.org/10.1016/j.jclinepi.2010.02.006. 21. US Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research. (2009). Guidance for industry: patient reported outcome measures: use in medical product development to support labeling claims . Silver Spring, MD. 22. Biberoglu, K. O., & Behrman, S. J. (1981). Dosage aspects of danazol therapy in endometriosis: Short-term and long-term effectiveness. American Journal of Obstetrics and Gynecology, 139 (6), 645 –654. 23. Fauconnier, A., Staraci, S., Huchon, C., Roman, H., Panel, P., & Descamps, P. (2013). Comparison of patient- and physician-based descriptions of symptoms of endometriosis: A qualitative study. Human Reproduction, 28(10), 2686–2694. https://doi.org/10.1093/humrep/det310. 24. Vincent, K., Kennedy, S., & Stratton, P. (2010). Pain scoring in endometriosis: Entry criteria and outcome measures for clinical trials. Report from the art and science of endometriosis meeting. Fertility and Sterility, 93 (1), 62–67. https://doi.org/10.1016/j.fertnstert.2008.09.056. 25. Patrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E., et al. (2011). Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 1--eliciting concepts for a new PRO instrument. Value in Health, 14 (8), 967–977. https://doi.org/10.1016/j.jval.2011.06.014. 26. Patrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E., et al. (2011). Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 2--assessing respondent understanding. Value in Health, 14 (8), 978 –988. https://doi.org/10.1016/j.jval.2011.06.013. 27. Francis, J. J., Johnston, M., Robertson, C., Glidewell, L., Entwistle, V., Eccles, M. P., et al. (2010). What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychology and Health, 25 (10), 1229–1245. https://doi.org/10.1080/08870440903194015. 28. Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18(1), 59–82. 29. American Society for Reproductive Medicine. (1997). Revised American society for reproductive medicine classification of endometriosis: 1996.Fertility and Sterility, 67(5), 817–821. https://doi.org/10.1016/s0015-0282(97)81391-x. 30. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3 (2), 77 –101. 31. Glaser, B. G., & Strauss, A. L. (1967). The constant comparative method of qualitative analysis. The discovery of grounded theory: Strategies for qualitative research (Vol. 101, p. 158). 32. Coons, S. J., Gwaltney, C. J., Hays, R. D., Lundy, J. J., Sloan, J. A., Revicki, D. A., et al. (2009). Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient- reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value in Health, 12 (4), 419 –429. https://doi.org/10.1111/j. 1524-4733.2008.00470.x. 33. Perneger, T. V., Courvoisier, D. S., Hudelson, P. M., & Gayet-Ageron, A. (2015). Sample size for pre-tests of questionnaires. Quality of Life Research, 24 (1), 147–151. https://doi.org/10.1007/s11136-014-0752-2. 34. Denny, E., & Mann, C. H. (2008). Endometriosis and the primary care consultation. European Journal of Obstetrics, Gynecology, and Reproductive Biology, 139(1), 111 –115. https://doi.org/10.1016/j.ejogrb.2007.10.006. 35. Denny, E. (2009). I never know from one day to another how I will feel: Pain and uncertainty in women with endometriosis. Qualitative Health Research, 19(7), 985 –995. https://doi.org/10.1177/1049732309338725. 36. Ballard, K., Lowton, K., & Wright, J. (2006). What ’s the delay? A qualitative study of women ’s experiences of reaching a diagnosis of endometriosis. Fertility and Sterility, 86 (5), 1296 –1301. https://doi.org/10.1016/j.fertnstert. 2006.04.054. 37. Deal, L. S., DiBenedetti, D. B., Williams, V. S., & Fehnel, S. E. (2010). The development and validation of the daily electronic endometriosis pain and bleeding diary. Health and Quality of Life Outcomes, 8 , 64. https://doi.org/10. 1186/1477-7525-8-64. 38. Manderson, L., Warren, N., & Markovic, M. (2008). Circuit breaking: Pathways of treatment seeking for women with endometriosis in Australia. Qualitative Health Research, 18 (4), 522 –534. https://doi.org/10.1177/1049732308315432. 39. Moradi, M., Parker, M., Sneddon, A., Lopez, V., & Ellwood, D. (2014). Impact of endometriosis on women ’s lives: A qualitative study. BMC Women’s Health, 14, 123. https://doi.org/10.1186/1472-6874-14-123. 40. Serlin, R. C., Mendoza, T. R., Nakamura, Y., Edwards, K. R., & Cleeland, C. S. (1995). When is cancer pain mild, moderate or severe? Grading pain severity by its interference with function. Pain, 61 (2), 277 –284. 41. Norquist, J. M., Girman, C., Fehnel, S., DeMuro-Mercon, C., & Santanello, N. (2012). Choice of recall period for patient-reported outcome (PRO) measures: Criteria for consideration. Quality of Life Research, 21 (6), 1013 – 1020. https://doi.org/10.1007/s11136-011-0003-8. 42. Shi, Q., Wang, X. S., Mendoza, T. R., Pandya, K. J., & Cleeland, C. S. (2009). Assessing persistent cancer pain: A comparison of current pain ratings and pain recalled from the past week. Journal of Pain and Symptom Management, 37(2), 168 –174. https://doi.org/10.1016/j.jpainsymman.2008.02. 009. 43. Cleeland, C. (1989). Measurement of pain by subjective report. Advances in Pain Research and Therapy, 12 , 391–403. 44. Cleeland, C. S. (1990). Assessment of pain in cancer. Advances in Pain Research and Therapy, 16 ,4 7 –55. 45. Cleeland, C. (1991). Research in cancer pain. What we know and what we need to know. Cancer, 67(3 Suppl), 823 –827. 46. Cleeland, C. S., & Ryan, K. M. (1994). Pain assessment: Global use of the brief pain inventory. Annals of the Academy of Medicine, Singapore, 23 (2), 129 – 138. 47. Dworkin, R. H., Turk, D. C., Farrar, J. T., Haythornthwaite, J. A., Jensen, M. P., Katz, N. P., et al. (2005). Core outcome measures for chronic pain clinical trials: IMMPACT recommendations. Pain, 113(1–2), 9 –19. https://doi.org/10. 1016/j.pain.2004.09.012. 48. Bourdel, N., Alves, J., Pickering, G., Ramilo, I., Roman, H., & Canis, M. (2015). Systematic review of endometriosis pain assessment: How to choose a scale? Human Reproduction Update, 21 (1), 136 –152. https://doi.org/10.1093/ humupd/dmu046. 49. Jensen, M. P., Karoly, P., & Braver, S. (1986). The measurement of clinical pain intensity: A comparison of six methods. Pain, 27 (1), 117 –126. 50. Turk, D. C., Dworkin, R. H., Allen, R. R., Bellamy, N., Brandenburg, N., Carr, D. B. , et al. (2003). Core outcome domains for chronic pain clinical trials: IMMPACT recommendations. Pain, 106 (3), 337 –345. 51. Gater, A., Wichmann, K., Seitz, C., Gerlinger, C., Chen, W., & Filonenko, A. (2014). Assessing patient-reported impact of endometriosis pain using a daily versus 7-day recall period. Qual Life Res (2014) 23:1 –184. https://doi. org/10.1007/s11136-014-0769-6. 52. Jones, G., Kennedy, S., Barnard, A., Wong, J., & Jenkinson, C. (2001). Development of an endometriosis quality-of-life instrument: The endometriosis health Profile-30. Obstetrics and Gynecology, 98 (2), 258 –264. 53. Bushnell, D. M., Martin, M. L., Moore, K. A., Richter, H. E., Rubin, A., & Patrick, D. L. (2010). Menorrhagia impact questionnaire: Assessing the influence of heavy menstrual bleeding on quality of life. Current Medical Research and Opinion, 26(12), 2745–2755. https://doi.org/10.1185/03007995.2010.532200. 54. Seitz, C., Lanius, V., Lippert, S., Gerlinger, C., Haberland, C., Oehmke, F., et al. (2018). Patterns of missing data in the use of the endometriosis symptom diary. BMC Women ’s Health, 18 (1), 88. https://doi.org/10.1186/s12905-018- 0578-0. 55. Gater, A., Coon, C. D., Nelsen, L. M., & Girman, C. (2015). Unique challenges in development, psychometric evaluation, and interpretation of daily and event diaries as endpoints in clinical trials. Therapeutic Innovation & Regulatory Science, 49 (6), 813 –821. https://doi.org/10.1177/ 2168479015609649. 56. van Nooten, F. E., Cline, J., Elash, C. A., Paty, J., & Reaney, M. (2018). Development and content validation of a patient-reported endometriosis pain daily diary. Health and Quality of Life Outcomes, 16 (1), 3. https://doi.org/ 10.1186/s12955-017-0819-1. 57. Nnoaham, K. E., Hummelshoj, L., Webster, P., d'Hooghe, T., de Cicco Nardone, F., de Cicco Nardone, C., et al. (2011). Impact of endometriosis on quality of life and work productivity: A multicenter study across ten Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 16 of 17 countries. Fertility and Sterility, 96 (2), 366 –73.e8. https://doi.org/10.1016/j. fertnstert.2011.05.090. 58. Hudelist, G., Fritzer, N., Thomas, A., Niehues, C., Oppelt, P., Haas, D., et al. (2012). Diagnostic delay for endometriosis in Austria and Germany: Causes and possible consequences. Human Reproduction, 27 (12), 3412–3416. https://doi.org/10.1093/humrep/des316. 59. Surrey, E., Carter, C. M., Soliman, A. M., Khan, S., DiBenedetti, D. B., & Snabes, M. C. (2017). Patient-completed or symptom-based screening tools for endometriosis: A scoping review. Archives of Gynecology and Obstetrics, 296(2), 153 –165. https://doi.org/10.1007/s00404-017-4406-9. 60. Bourdel, N., Chauvet, P., Billone, V., Douridas, G., Fauconnier, A., Gerbaud, L., et al. (2019). Systematic review of quality of life measures in patients with endometriosis. PLoS One, 14 (1), e0208464. https://doi.org/10.1371/journal. pone.0208464. Publisher’sN o t e Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Gater et al. Journal of Patient-Reported Outcomes (2020) 4:13 Page 17 of 17