Psychometric Validation of a Moroccan Version of the 22-Item Sino-Nasal
Outcome Test
Otolaryngology–
Head and Neck Surgery 1–7
ÓAmerican Academy of Otolaryngology—Head and Neck Surgery Foundation 2016 Reprints and permission:
sagepub.com/journalsPermissions.nav DOI: 10.1177/0194599816650477 http://otojournal.org
Choaib Adnane, MD
1, Taoufik Adouly, MD
1,
Tarek Oubahmane, MD
1, Abdelhay Zouak, MD
1, Sami Rouadi, PhD
1, Reda Lah Abada, PhD
1, Mohamed Roubal, PhD
1, and
Mohamed Mahtar, PhD
1No sponsorships or competing interests have been disclosed for this article.
Abstract
Objective. The objective of this study paper was to culturally adapt and validate the 22-item Sino-Nasal Outcome Test (SNOT- 22) questionnaire for Moroccan Arabic speaking patients.
Study Design.Prospective cohort study.
Setting.Tertiary referral center; Ibn Rochd University Hospital, Casablanca, Morocco.
Subjects and Methods.The SNOT-22 was conducted in patients with chronic rhinosinusitis (CRS) undergoing sinonasal surgery and in healthy volunteers, from January 2012 to December 2013. It was translated into Moroccan Arabic language. To evalu- ate this questionnaire, internal consistency, test-retest reliability, responsiveness to treatment, and validity were analyzed.
Results. Preoperative SNOT-22 scores were completed by 88 patients. Six-month postoperative SNOT-22 scores were available for 74 patients. The Cronbach’s alpha coefficient for the SNOT-22 was 0.968, indicating high internal consis- tency. The test-retest reliability coefficient was 0.993, indi- cating high reliability when administering the instrument on repeated occasions. The Moroccan version of the SNOT-22 was able to highly discriminate between patients with CRS and group of healthy volunteers (P\.0001). There was a statistically significant reduction in patient reported SNOT- 22 scores at 6 months after surgery (P\.0001).
Conclusion. The present study has found the Moroccan ver- sion of SNOT-22 to be valid and easy to use with good relia- bility, validity, and responsiveness. It can be used to measure the impact of CRS on the patient’s quality of life and may also be used to evaluate CRS treatment.
Keywords
chronic rhinosinusitis, Sino-Nasal Outcome Test 22, SNOT 22, Moroccan language
Received March 19, 2015; revised May 5, 2015; accepted June 12, 2015.
Introduction
Chronic rhinosinusitis (CRS) is a common complaint condi- tion of a multifactorial origin that has a huge impact on quality of life (QOL).1 This impact has been demonstrated using global measures of quality of life, such as the Short Form 36 Health Survey Questionnaire (SF-36).2The disad- vantages of such global scores is that they may fail to cap- ture most important aspects of disease and may also lack the sensitivity to detect response to treatment. For this reason, disease-specific measure tools are often preferable.3 In the past several years, numerous scoring systems have been developed to assess the specific impact of nasal func- tion on QOL. The Sino-Nasal Outcome Test (SNOT) is one of these disease-specific QOL measures of outcome in sino- nasal disorders such as CRS.1,3 The SNOT-20 contains 20 questions and was developed first from the 31-question Rhinosinusitis Outcome Measure (RSOM-31) by removing 11 redundant items.4The addition of 2 items (nasal obstruc- tion and olfaction) formed the SNOT-22, each item of which is rated from 0 to 5 (0 = no problem, 1 = very mild problem, 2 = mild or slight problem, 3 = moderate problem, 4 = severe problem, 5 = problem as bad as it can be). The total score can range from 0 to 110. Lower total scores sug- gest better QOL and symptom severity. The SNOT-22 has demonstrated good reliability, validity, and responsiveness.5 Nowadays, many clinicians and centers have adopted this questionnaire to evaluate patient sino-nasal diseases and their evolution after medical or chirurgical treatments.6The SNOT-22 has been translated and validated in several lan- guages, including Lithuanian, Danish, German, Portuguese, Greek, and French but not in the Moroccan or other Arabic languages.7-12 The Moroccan dialect is used by Moroccan population but also by some Maghreb countries. The aim of
1Department of ENT, 20 Aouˆt Hospital, Ibn Rochd University Hospital, Casablanca, Morocco
Corresponding Author:
Choaib Adnane, 20 Aouˆt Hospital, Ibn Rochd University Hospital, Casablanca, Morocco.
Email: adnanechoaib@gmail.com
the present paper was to culturally adapt and validate the 22- Item Sino-Nasal Outcome Test questionnaire for Moroccan Arabic speaking patients (see Appendix 1 at www.otojournal .org/supplemental).
Methods
Study Design and Participants
This study is a continuation of the previously published work from our group.13 It is a prospective cohort study of 88 patients undergoing functional endoscopic sinus surgery (FESS) for CRS from January 2012 to December 2013 in Ibn Rochd University Hospital in Casablanca. Fifty-one healthy volunteers were randomly selected from escorting members to participate as a control group. The research pro- tocol was approved by the Ethics in Research with Human Beings Committee of the local institution (Ibn Rochd University Hospital).
Inclusion criteria for patients with CRS were as follows:
adult patients (18 years or older) who had undergone FESS for CRS (with or without polyps) as reported by the European Position Paper on Rhinosinusitis and Nasal Polyps (EPOS 2012).14 All patients had preoperative endoscopic examination and sinus computed tomography (CT) scan.
Patients were excluded if the CRS medical therapy was changed or recently administrated within 6 weeks before surgery. Patients were also excluded if they had prior sinus surgery, bronchial asthma, aspirin intolerance, immunodefi- ciency, pregnancy, secondary CRS, or psychiatric disorders that affect QOL. On the other hand, healthy volunteers were included if they had objective findings in nasal cavity scored 1 or lower without any psychiatric disorders and chronic or acute diseases that affect their QOL.
Computed tomography scan of the sinuses was classified using the Lund-Mackay scoring system, and the endoscopic examination was scored using the Lund-Kennedy scoring system.15,16
Medical treatment was the first therapy introduced at least for 3 months. It was based on topical steroid applica- tions (budenoside spray) on each nostril every 12 hours and short courses of oral steroid (prednisolone) for 7 days (between 2 and 6 cures). Board spectrum antibiotics were used at least for 2 weeks.
Functional endoscopic sinus surgery was performed under general anesthesia after failure of medical therapy protocol. The extent of surgery was based on CRS CT scan extension and type: for CRS with nasal polyps (CRSwNP) bilateral maxillary antrostomy with functional total ethmoı¨dectomy was performed, but for CRS without nasal polyps (CRSsNP) surgical extension was based on CT scan extension. Septoplasty and/or inferior turbinoplasty were performed when indicated.
Postoperative care was as follows: the nasal packing was removed 2 days postoperatively; all patients received amoxicil- lin and clavulanic acid for 10 days and topical nasal steroid on each nostril every 12 hours for at least 3 months; saline irriga- tion and endoscopic dressing were performed for 6 weeks.
The eligible patients (patients with CRS and healthy volunteers) were asked to complete the translated version of SNOT-22 questionnaire at time 1 (SNOT-22 preoperative 1) and after 1 week (SNOT-22 preoperative 2). Patients who had finished the follow-up were asked to complete the same questionnaire 6 months after surgery (SNOT-22 postopera- tive). Another sino-nasal–specific questionnaire (rhinosinu- sitis disability index; RSDI) was performed at the same time as a reference for comparative measure.17
The 22 items of the SNOT-22 score were recategorized and summarized into 5 distinct domains, including rhinologic symptoms, extranasal rhinologic symptoms, ear/facial symp- toms, psychological dysfunction, and sleep dysfunction.18
Methodology
Translation and cultural adaptation of the SNOT-22.
Step 1: All the items of the English version of the SNOT-22 were forward-translated by 2 translators separately into the Moroccan dialect. The transla- tors avoided word-for-word translations and used simple terms that could be easily understood by the subjects. The final draft was created and finally reviewed by an ENT specialist for medical viewpoint.
Step 2: The draft obtained was back-translated into English by 2 translators different from those who translated the first forward translation.
Step 3: The English draft obtained was reviewed and compared with the original version. It was con- firmed that similar expressions had been used.
Assessment of reliability, validity, and sensibility or responsiveness.
Reliability.Reliability is defined as the extent to which the SNOT-22 produces the same results on repeated trials. It is the stability of scores over time assessed through a test- retest reproducibility by measuring correlations between the SNOT-22 at time 1 and at time 2 (1 week after) for the same patients with CRS. Internal consistency or homogene- ity concerns the extent to which items on the test are mea- suring the same thing.
Validity. Validity is defined as the extent to which the SNOT-22 measures what it purports to measure. Content validity was assessed in the development of the original form and maintained by the process of forward and back- ward translation. Criterion validity compares test score per- formance to a gold standard, which in the case of the QOL scales does not exist.
Construct validity is the degree to which the SNOT-22 measures the theoretical construct that it is intended to mea- sure and includes convergent and discriminant or divergent validity. Convergent validity was assessed by comparing the 2 scores: SNOT-22 and RSDI. A convergent validity was assessed because we desired to see how closely the SNOT- 22 was related to another questionnaire (RSDI) of the same construct. Discriminant validity, which is the ability to
discriminate between known groups, was determined by comparing 2 groups: patients with CRS and healthy volunteers.
Concurrent validity was assessed by comparing the preo- perative SNOT-22 score (at time 1) with 2 preoperative objective scores: endoscopic Lund-Kennedy and CT scan Lund-Mackay scoring systems.
Responsiveness or sensibility. The scores before (at time 1) and 6 months after FESS were compared to detect if they were significant clinical change after surgical intervention.
Statistical analysis. The data sets were analyzed using SPSS 20.0. The normal distribution was assessed by using Shapiro-Wilk test and skewness kurtosis z-values. The demographic characteristics were summarized by means, percentages, and standard deviations (SD).
The internal consistency was measured by Cronbach’s alpha, and the minimal acceptable score was 0.7.19 The test-retest reliability and concurrent and convergent valid- ities were calculated by Spearman’s correlation. The inter- pretation of correlation coefficient was: excellent correlation (.0.91), good correlation (0.71-0.90), moderate correlation (0.51-0.70), acceptable correlation (0.31-0.50), and low cor- relation (\0.30).19
For discriminant validity, the unpaired Mann-Whitney U test was used to determine if the questionnaire could detect a difference between healthy volunteers and patients with CRS. Furthermore, the ROC curve was used to assess how well the SNOT-22 indicates discriminancy between the 2 groups. The factors that were used to create the ROC curve were sensitivity and specificity. The area under this curve was defined as ROC-AUC. The evaluation standard has been established as: 0.90 to 1.00 = excellent, 0.80 to 0.90 = good, 0.70 to 0.80 = fair, and 0.50 to 0.60 = fail.20
The responsiveness was assessed by using the Wilcoxon signed-rank test and the effect size; the mean different score divided by baseline SD. An effect size can be small (.0.2), moderate (.0.5), and large (.0.8) for the improvement in health-related QOL.21 The minimal important difference (MID) was estimated statistically as a change of more than or equal to half standard deviation (SD) of the baseline SNOT-22 score.22
Stepwise method was chosen for multiple regression models. A P value under .05 (5%) was considered statisti- cally significant for all analyses.
Results
Participant Characteristics
Eighty-eight patients with CRS were operated in the ENT department; only 74 patients (84%) completed the study; the other 14 patients were lost to follow-up (16%). Fifty-one healthy volunteers participated in the study. For the patients with CRS, the mean age was 38.66 years, the median was 40 years (IQR = 19 and SD = 13.37), and the sex-ratio was 0.85 (34 male and 40 female). For the healthy volunteers, the mean age was 35.1 years, and the sex-ratio was 0.82 (23 male and
28 female). There were no significant differences between the 2 groups in age (P= .957) and gender (P= .925).
Chronic rhinosinusitis without nasal polyps was more prevalent (60.8%). Maxillary antrostomy was performed for all patients, total ethmoidectomy in 58.1% and partial eth- moidectomy in 41.9% of patients (Table 1). Characteristics of patients in the study are described inTable 2.
Table 1.Repartition of Surgical Procedures.
Surgical Procedures CRSwNP, No. (%) CRSsNP, No. (%) Maxillary antrostomy 29 (39.2) 45 (60.8) Ethmoidectomy
Partial — 31 (41.9)
Total 29 (39.2) 14 (18.9)
Sphenoidotomy — 4 (5.4)
Frontal sinusotomy — 3 (4)
Septoplasty 2 (2.7) 8 (10.8)
Inferior turbinoplasty — 11 (14.8)
Abbreviations: CRSsNP, chronic rhinosinusitis without nasal polyps;
CRSwNP, chronic rhinosinusitis with nasal polyps.
Table 2. Characteristics of Patients with CRS and Healthy Volunteers.
Characteristics Patients Controls
Total, No. 74 51
Age, y (mean6SD) 38.66613.37 35.1611.9 Gender, No. (%)
Male 34 (45.9) 23 (45.1)
Female 40 (54.1) 28 (54.9)
Preoperative diagnostic testing (mean6SD)
Lund-Kennedy endoscopy score
5.9263.682 —
Lund-Mackay CT score 11.4967.907 — Type of CRS, No. (%)
With polyps 29 (39.2) —
Without polyps 45 (60.8) —
SNOT-22 (mean6SD)
SNOT-22 preop 1 50.39621.132 14.4965.143
SNOT-22 preop 2 50.66620.953 —
SNOT-22 postop 23.43619.125 —
AbsoluteDa 26.96612.956 —
RSDI (mean6SD)
RSDI preop 73.51620.829 —
RSDI postop 32.78620.021 —
AbsoluteD 40.73616.689 —
Abbreviations: CRS, chronic rhinosinusitis; CT, computed tomography;
RSDI, Rhinosinusitis Disability Index; SNOT-22, 22-item Sino-Nasal Outcome Test.
aD = absolute change value of RSDI score and SNOT-22 (preop 1 and postop).
Reliability
Internal consistency.The Cronbach’s alpha scores for the new version of the SNOT-22 (Moroccan version) indicate high internal consistency (Table 3).
Test-retest reliability.The Spearman correlation coefficient for test-retest reliability was 0.993 (P\.0001), indicating high reliability of repeated measures and high stability over time.
According to the Bland-Altman plot, most of the times the differences in scores between the first and the second assessments were within the levels of agreement of 95%
(mean61.963SD) (Figure 1).
Validity
For concurrent validity, correlation between the SNOT-22 scores and the Lund-Kennedy endoscopic scores showed
Spearman r = 0.71. Correlation between the SNOT-22 scores and CT scan Lund-Mackay scores showed Spearman r = 0.695. The 2 preoperative objective scores were highly correlated with the SNOT-22 (P\.0001).
In the study of convergent validity, Spearman’s rank cor- relation coefficient was r = 0.73, indicating a high correla- tion between preoperative SNOT-22 and RSDI scores at time 1 (P\.0001).
Discriminant validity revealed that the SNOT-22 was able to highly discriminate between patients with CRS and healthy volunteers (50.39 [IQR = 26] vs 14.49 [IQR = 9];
Mann-Whitney U test, U = 24.5, z =29.36, r =20.84,P\ .0001). Furthermore, when the discriminant validity was assessed using a receiver operating characteristic (ROC) curve, it was found that ROC area under the curve (AUC) was 0.994 (Figure 2).
Responsiveness at 6 Months
A strongly statistically significant reduction was seen between the scores attained pre- and postoperatively on SNOT-22 (50.39 [IQR = 26] vs 23.43 [IQR = 20], Wilcoxon signed-rank test,t= 0, z =27.48, r =20.87,P\.0001).
After 6 months, the overall effect size in all patients was 1.27 (Table 4). The effect size was very large. The MID was defined as a decrease of 10.57 points.
The Mann-Whitney U test was used to determine the dif- ference between the SNOT-22 scores in healthy volunteers and in patients with CRS after undergoing FESS. The differ- ence was not significant (U = 1560.5, z = 21.643, r = 20.15,P= .1).
Table 3. Cronbach’s Alpha Scores for the SNOT-22 and Its Subscales.a
SNOT-22 Preoperative 1 Preoperative 2 Postoperative
Total 0.968 0.968 0.977
Rhinologic symptoms 0.828 0.834 0.927 Extranasal rhinologic
symptoms
0.869 0.851 0.860
Ear/facial symptoms 0.815 0.819 0.807 Psychological
dysfunction
0.974 0.971 0.971
Sleep dysfunction 0.973 0.973 0.973
Abbreviation: SNOT-22, 22-item Sino-Nasal Outcome Test.
aCronbach’s alpha was superior to 0.7 for all subscales, which means a high internal consistency.
Figure 1.Bland-Altman plot test-retest reproducibility. On the Y axis, the difference between the 2 scores, on the X axis, their mean values. Most of the times, the differences between test- retest scores were within the levels of agreement of 95% (mean6 1.963SD).
Figure 2.Discriminant validity using ROC curve of the SNOT-22 mean scores (ROC-AUC was 0.994, indicating an excellent dis- crimination).
Abbreviations: ROC, receiver operating characteristic. SNOT-22, 22-item Sino-Nasal Outcome Test.
Predictive Factors
Eight preoperative factors were used for multiple regression model: nasal polyps, Lund-Kennedy endoscopy score, Lund-Mackay CT score, and the 5 domains of preoperative SNOT-22 score. Only 3 preoperative characteristics pre- dicted the SNOT-22 evolution after FESS (Table 5).
Preoperative ear/facial and extranasal rhinologic symptoms appeared to provide predictive information such that patients with higher ear, facial, and extranasal rhinologic symptoms were more likely to improve after FESS. Also patients with nasal polyposis had worst QOL evolution after surgery. This model was able to explain 40.6% of the change in SNOT-22 score (R2 = 0.406 and adjusted R2 = 0.380). There was no collinearity within our data (VIF\10, tolerance statistics\0.1).
Discussion
According to prior studies, CRS-specific QOL is the main driver of patients’ choice of management strategy, so the use of the SNOT-22 improves patient understanding of their surgical outcomes while they make decisions regarding FESS.23 For Hopkins et al,5the test-retest reliability coeffi- cient of SNOT-22 was 0.93. In many adapted versions of
SNOT-22, test-retest reliability coefficient was between 0.70 and 0.91.7,10,11In the present study, correlation coeffi- cient was 0.993, suggesting a strong correlation between the scores of the initial test and the retest examination.
Considering Cronbach’s alpha, Hopkins et al5 showed an alpha of 0.91, which is in agreement with the result of this study (0.968). The high reliability coefficients may be expli- cated by the patient’s severity disease due to their selection after medical treatment failure.
The Moroccan version of the SNOT-22 has excellent dis- criminant validity and responsiveness after surgery, and the estimated MID was 10.57 points. Therefore, a change of less than 10.5 points cannot be perceived as a real improve- ment. This was superior to the MID calculated by Hopkins et al.5 The results of this study matched the validation of the original questionnaire by Hopkins et al5 and the other studies.8,10,11
Three preoperative predictors were found in multiple regression model: nasal polyps, ear/facial symptoms, and extranasal rhinologic symptoms. Patients with nasal polypo- sis have poorer outcome after FESS, as previous studies have already shown.24DeConde et al25found that psycholo- gical and sleep dysfunction were significantly more likely to have a greater relative influence on patients electing Table 4.Responsiveness for Each Group of CRS.a
Nasal Polyps CRS without Polyps All Patients
Measure No. Mean (SD) No. Mean (SD) No. Mean (SD)
Preoperative SNOT-22 29 70.97 (17.81) 45 37.13 (8.95) 74 50.39 (21.13)
Postoperative SNOT-22 29 41 (19.71) 45 12.11 (5.02) 74 23.43 (19.12)
AbsoluteD 29 29.97 (17.91) 45 25.02 (8.03) 74 26.96 (12.95)
Effect size 2.79 1.68 1.27
Abbreviations: CRS, chronic rhinosinusitis; SNOT-22, 22-item Sino-Nasal Outcome Test.
aThe evolution of the SNOT-22 scores after functional endoscopic sinus surgery for both CRS with and without polyps was statistically significant (P\.001).
Dis the absolute change value of SNOT-22 (preop 1 and postop).
Table 5.Predictors of Disease-Specific Quality of Life Improvement after Functional Endoscopic Sinus Surgery (Stepwise Method).a
Predictor B SE B ß P R
Step 1
Constant 14.785 2.442
Ear/facial symptoms (preop) 1.678 0.289 0.564 \.0001 0.319
Step 2
Constant 13.879 2.406
Ear/facial symptoms (preop) 2.199 0.362 0.740 \.0001
Nasal polyps –7.348 3.208 –0.279 .025 0.366
Step 3
Constant 8.457 3.428
Ear/facial symptoms (preop) 1.609 0.445 0.541 .001
Nasal polyps –9.894 3.340 –0.375 .004
Extranasal rhinologic symptoms (prep) 1.456 0.671 0.336 .033 0.406
aPreoperative ear, facial, and extranasal rhinologic symptoms had a positive impact on the 22-item Sino-Nasal Outcome Test (SNOT-22) score evolution, and patients with nasal polyps had the worst SNOT-22 score evolution.
FESS than other symptom domains (rhinologic, extranasal rhinologic, ear/facial symptoms). In the present study, 2 dif- ferent symptom domains (ear/facial, extranasal rhinologic symptoms) were found to have a positive significant impact on the QOL outcomes after FESS.
This study presents a few limitations. One of them is the fact that the study was carried out at a tertiary referral center and all the patients who underwent FESS had already been submitted to longstanding medical therapy to no avail, typi- cally representing the more severe disease spectrum. So the preoperative scores were high, and the internal consistence of this group was elevated (Cronbach’s alpha up to 0.9). Another is the reduced sample size with the study of both CRS cate- gories (with and without nasal polyps). Also, due to the absence of a group that compares surgery to medical therapy, there was no evaluation of the score for medical treatment.
The strength of this study includes the prospective nature, the use of validated instruments (SNOT-22), a low rate of lost patients, the use of control group, and the low rate of comorbid conditions (asthma, allergy, revision sur- gery) in the CRS and control groups that may reduce response bias.
Conclusion
Research on QOL is gaining more weight within otolaryn- gology. The present study demonstrates that the Moroccan version of the SNOT-22 is a good instrument to evaluate the QOL of patients with CRS. This study facilitates the systematic incorporation of quality of life measures into international trials in rhinology.
Acknowledgments
We would like to thank the translators and all patients and control subjects enrolled in this cohort study.
Author Contributions
Choaib Adnane, conception and design, data collection, analysis and interpretation, writing the manuscript, statistical expertise;
Taoufik Adouly, data collection, analysis and interpretation, writ- ing the manuscript;Tarek Oubahmane, data collection, analysis and interpretation, writing the manuscript;Abdelhay Zouak, data collection, analysis and interpretation, writing the manuscript;
Sami Rouadi, conception and design, supervision, critical revision of the manuscript; Reda Lah Abada, conception and design, supervision, critical revision of the manuscript;Mohamed Roubal, conception and design, supervision, critical revision of the manu- script;Mohamed Mahtar, conception and design, statistical exper- tise, obtaining funding, administrative, technical, or material support, supervision, critical revision of the manuscript.
Disclosures
Competing interests:None.
Sponsorships:None.
Funding source:None.
Supplemental Material
Additional supporting information may be found at http://otojournal .org/supplemental.
References
1. Piccirillo JF, Merritt MG, Richards ML. Psychometric and clinimetric validity of the 20-item Sino-Nasal Outcome Test (SNOT-20).Otolaryngol Head Neck Surg. 2002;126:41-47.
2. Glicklich RE, Metson R. The health impact of chronic sinusitis in patients seeking otolaryngologic care. Otolaryngol Head Neck Surg. 1995;113:104-109.
3. Hopkins C. Patient rated outcome measures in rhinology.
Rhinology. 2009;40:10-17.
4. Piccirillo JF, Edwards D, Haiduk A, et al. Psychometric and clinimetric validity of the 31-item Rhinosinusitis Outcome Measure (RSOM-31).Am J Rhinol Allergy. 1995;9:297-306.
5. Hopkins C, Gillett S, Slack R, et al. Psychometric validity of the 22-item Sinonasal Outcome Test.Clin Otolaryngol. 2009;
34:447-454.
6. Browne JP, Hopkins C, Slack R, et al. Health-related quality of life after simple nasal polypectomy or nasal polypectomy with additional sino-nasal surgery: a prospective cohort study with three year follow up.Laryngoscope. 2006;116:297-302.
7. Lange B, Thilsing T, Al-kalemji A, et al. The sino-nasal out- come test 22 validated for Danish patients. Dan Med Bul.
2011;58:A 4235.
8. Kosugi EM, Chen VG, da Fonseca VM, et al. Translation, cross- cultural adaptation and validation of SinoNasal Outcome Test (SNOT): 22 to Brazilian Portuguese. Braz J Otorhinolaryngol.
2011;77:663-669.
9. Caminha GP, Melo Junior JT, Hopkins C, et al. SNOT-22: psy- chometric properties and cross-cultural adaptation into the Portuguese language spoken in Brazil.Braz J Otorhinolaryngol.
2012;78:34-39.
10. Vaitkus S, Padervinskis E, Balsevicius T, et al. Translation, cross-cultural adaptation, and validation of the sino-nasal out- come test (SNOT)-22 for Lithuanian patients. Eur Arch Otorhinolaryngol. 2013;270:1843-1848.
11. Lachanas VA, Tsea M, Tsiouvaka S, et al. The sino-nasal out- come test (SNOT)-22: validation for Greek patients.Eur Arch Otorhinolaryngol. 2014;271:2723-2728.
12. De Dorlodot C, Horoi M, Lefebvre P, et al. French adaptation and validation of the SinoNasal Outcome Test (SNOT)-22: a prospective cohort study on quality of life among 422 subjects.
Clin Otolaryngol. 2015;40:29-35.
13. Adnane C, Adouly T, Zouak A, et al. Quality of life outcomes after functional endoscopic sinus surgery for nasal polyposis.
Am J Otolaryngol. 2015;36:47-51.
14. Fokkens WJ, Lund VJ, Mullol J, et al. EPOS 2012: European position paper on rhinosinusitis and nasal polyps. A summary for otorhinolaryngologists.Rhinology. 2012;50:1-12.
15. Lund VJ, Mackay IS. Staging in rhinosinusitis. Rhinology.
1993;31:183-184.
16. Lund V, Kennedy D. Quantification for staging sinusitis.Ann Otol Rhinol Laryngol. 1995;104(suppl. 10):1-31.
17. Benninger MS, Senior BA. The development of the Rhinosinusitis Disability Index. Arch Otolaryngol Head Neck Surg. 1997;123:
1175-1179.
18. DeConde AS, Bodner TE, Mace JC, et al. Response shift in quality of life after endoscopic sinus surgery for chronic
rhinosinusitis.JAMA Otolaryngol Head Neck Surg. 2014;140:
712-719.
19. Cronbach LJ. Coefficient alpha and the internal structure of tests.Psychometrika. 1951;16:297-306.
20. Swets JA. Measuring the accuracy of diagnostic systems.
Science. 1988;240:1285-1293.
21. Cohen J.Statistical Power Analysis for the Behavioral Sciences.
2nd ed.Englewood Cliffs, NJ: Lawrence Erlbaum; 1988.
22. Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation.Medical Care. 2003;41:582-592.
23. Rudmik L, Soler ZM, Mace JC, et al. Using preoperative SNOT-22 score to inform patient decision for endoscopic sinus surgery.Laryngoscope. 2015;125(7):1517-1522.
24. Watelet JB, Annicq B, Van Cauwenberge P, et al. Objective outcome after functional endoscopic sinus surgery: prediction factors.Laryngoscope. 2004;114:1092-1097.
25. DeConde AS, Mace JC, Bodner T, et al. SNOT-22 quality of life domains differentially predict treatment modality selection in chronic rhinosinusitis. Int Forum Allergy Rhinol. 2014;4:
972-979.
