The Quality of Life Scale (QOLS): Reliability, Validity, and Utilization

Carol S Burckhardt^1* and Kathryn L Anderson²

¹ School of Nursing Oregon Health & Science University, Portland, Oregon, USA
² School of Nursing, Seattle University, Seattle, Washington, USA

Received:	22 July 2003
Accepted:	23 October 2003
Published:	23 October 2003

© 2003 Burckhardt and Anderson; licensee BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original URL.

Abstract

The Quality of Life Scale (QOLS), created originally by American psychologist John Flanagan in the 1970's, has been adapted for use in chronic illness groups. This paper reviews the development and psychometric testing of the QOLS. A descriptive review of the published literature was undertaken and findings summarized in the frequently asked questions format. Reliability, content and construct validity testing has been performed on the QOLS and a number of translations have been made. The QOLS has low to moderate correlations with physical health status and disease measures. However, content validity analysis indicates that the instrument measures domains that diverse patient groups with chronic illness define as quality of life. The QOLS is a valid instrument for measuring quality of life across patient groups and cultures and is conceptually distinct from health status or other causal indicators of quality of life.

Keywords:

Quality of Life Scale; QOLS; chronic illness outcomes; quality of life evaluation

Why assess Quality of Life in chronic illness?

Quality of life (QOL) measures have become a vital and often required part of health outcomes appraisal. For populations with chronic disease, measurement of QOL provides a meaningful way to determine the impact of health care when cure is not possible. Over the past 20 years, hundreds of instruments have been developed that purport to measure QOL [1]. With few exceptions, these instruments measure what Fayers and colleagues [2,3] have called causal indicators of QOL rather than QOL itself. Health care professionals need to be clear about the conceptual definition of QOL and not to confound it with functional status, symptoms, disease processes, or treatment side-effects [4-7]. Although the definition of QOL is still evolving, Revicki and colleagues define QOL as "a broad range of human experiences related to one's overall well-being. It implies value based on subjective functioning in comparison with personal expectations and is defined by subjective experiences, states and perceptions. Quality of life, by its very natures, is idiosyncratic to the individual, but intuitively meaningful and understandable to most people [[8], p. 888]." This definition denotes a meaning for QOL that transcends health. The Quality of Life Scale (QOLS) first developed by American psychologist, John Flanagan, [9,10] befits this definition of QOL.

What does the Quality of Life Scale (QOLS) measure?

The QOLS was originally a 15-item instrument that measured five conceptual domains of quality of life: material and physical well-being, relationships with other people, social, community and civic activities, personal development and fulfillment, and recreation. After descriptive research that queried persons with chronic illness on their perceptions of quality of life, the instrument was expanded to include one more item: Independence, the ability to do for yourself. Thus, the QOLS in its present format contains 16 items. See Table 1 for the individual items within each conceptual category in the original Flanagan version of the scale.

Table 1. Flanagan Quality of Life Scale (QOLS) original conceptual categories and scale items.

How was the QOLS developed?

The original work on the QOLS was undertaken in the United States in the mid-1970's. Using the critical incident technique, nearly 3,000 people of various ages, ethnic groups, and backgrounds from all parts of the United States were asked to contribute experiences that were important or satisfying to them. Substantial efforts were made to include ethnic minorities, rural inhabitants, senior citizens, and low income groups. As Flanagan stated, "The purpose of using the regional samples and diverse groups was not to obtain accurate estimates of frequencies but rather to insure that differing points of view and types of experience were represented [[9], p. 138]."

With the possible exception of Cantril's ladder [11], no other QOL instrument currently in general circulation has been developed with such extensive attention to diversity and individual perspective. The original QOLS contained 15 items representing 5 conceptual domains of QOL that were empirically derived from the 6500 critical incidents that Flanagan and his team collected.

In a second step, Flanagan used the instrument to survey a total of 3,000 people, ages 30, 50, and 70, using 5-point scales of "importance" and "needs met." The results of this national survey revealed that most people of both genders and all three ages felt that the items were important to them. The only exceptions were in the areas of participating in local and national government and public affairs (Item #8) which a majority of 30-year olds did not think was important, and creative expression (Item #12), socializing (Item #13) and passive recreation (Item #14) which less than a majority of men endorsed as important. Nevertheless, a majority of all people of both genders and all age groups was satisfied that their needs were being met in all areas [10].

Item Scaling

The original work by Flanagan [9] used two five-point scales of "importance" and "needs met." No reliability of this scaling was reported at the time. Earlier work by Andrews and Crandall [12] had suggested that a 7-point scale anchored with the words "delighted" and "terrible" was more sensitive and less negatively skewed than a 5-point satisfaction scale for quality of life assessment, probably because it allowed for a broader range of affective responses to QOL items. The seven responses were "delighted" (7), "pleased" (6), "mostly satisfied" (5), "mixed" (4), "mostly dissatisfied" (3), "unhappy" (2), "terrible" (1). For all work undertaken to adapt the scale for use in American chronic illness populations, the 7-point delighted-terrible scale was used to measure satisfaction with an item. The 5-point importance scale was used only for determining content validity in the initial chronic illness study [13].

How was the QOLS validated?

Flanagan did not report internal consistency reliability (Cronbach's alpha) estimates in his instrument development work. Estimates from the first study of 240 American patients with chronic illness (diabetes, osteoarthritis, rheumatoid arthritis and post-ostomy surgery) indicated that the 15-item QOLS satisfaction scale was internally consistent (α = .82 to .92) and had high test-retest reliability over 3-weeks in stable chronic illness groups (r = 0.78 to r = 0 .84) [13]. Other researchers have reported similar reliability estimates for the 16-item scale [14-17].

The quality and quantity of descriptive work with large numbers of Americans provided strong evidence for content validity of the QOLS during its early development. However, Flanagan, himself, reasoned that some adaptations for persons with chronic conditions or disabilities might be needed and that different rating scales might produce divergent results. In 1981 Professor Flanagan gave the first author permission to adapt the scale for patients with chronic illness. Over the intervening years the QOLS has been called the "Adapted Quality of Life Scale" or "Flanagan Quality of Life Scale." In this paper it will be called simply the QOLS and always refer to the 16-item scale as adapted by Burckhardt and colleagues for persons with chronic illness.

When the 240 Americans with chronic illness were asked open-ended questions about what the term "quality of life" meant to them and what was important to their QOL, they generated words and phrases that were very similar to those used by the general population that Flanagan had studied. The importance of material comforts and security, health, relationships with both family and friends, understanding of themselves, as well as the ability to socialize, participate in activities and have satisfying work experiences were all apparent in their descriptions. However, they also generated a list of phrases that could be best described as "efforts to remain independent" using words and phrases, such as "independence", "able to care for myself", and "being physically active". This item was added to the QOLS to make a 16-item scale: "Independence, ability to do for oneself" [13]. In addition, during this process the wording of item #8 "activities related to local and national government" was broadened to "participating in organizations and public affairs." This rewording was based on the qualitative responses from the people with chronic illness who were interviewed. Few of them participated in political activities or local government affairs; but they did participate in clubs, religious groups and other organizations.

In all, 207 of the 240 patients also rated the importance of the 15 items on the original Flanagan QOLS. Table 2 summarizes the findings by total group and gender. More than 50% of the patients rated all items except civic activities as important or very important to their quality of life. Men and women differed on four items with men rating relationships with parents, siblings and other relatives, relationships with friends, helping and encouraging others, and socializing as significantly less important to their quality of life.

Table 2. Importance of the QOLS items by total sample and gender. Percent rating the item as important or very important (In parentheses, the percent rating the item as unimportant).

Convergent and discriminant construct validity of the QOLS in chronic illness groups was evidenced first by the high correlations between the QOLS total score and the Life Satisfaction Index-Z (LSI-Z) [18] (r = 0.67 to 0.75) and its low to moderate correlations with the Duke-UNC Health Profile (DUHP) [19] physical health status subscale (r = 0.25 to 0.48) and a disease impact measure, the Arthritis Impact Measurement Scales (AIMS) [20] (r = 0.28 to 0.44) [13]. Later, Burckhardt and colleagues offered evidence that the QOLS could discriminate levels of QOL in populations that would be expected to differ. A group of healthy adults as well as groups with more stable chronic illnesses, such as post-ostomy surgery, osteoarthritis, and rheumatoid arthritis, were shown to have significantly higher scores than groups of patients with the persistent painful condition, fibromyalgia, life-threatening COPD, or insulin-dependent diabetes [21].

More recently, a sample of 1241 chronically ill and healthy adults from American and Swedish databases was used to generate factor analyses for both the 15-item original QOLS and the 16-item chronic illness adaptation. Analysis of the data suggested that the QOLS has three factors in the healthy sample and across chronic conditions, two languages and gender. Factors that could be labeled (1) Relationships and Material Well-Being, (2) Health and Functioning, and (3) Personal, Social and Community Commitment were identified [22].

In which populations has the QOLS been used?

The QOLS has been used in studies of healthy adults and patients with rheumatic diseases, fibromyalgia, chronic obstructive pulmonary disease, gastrointestinal disorders, cardiac disease, spinal cord injury, psoriasis, urinary stress incontinence, posttraumatic stress disorder, and diabetes. Although some researchers have questioned whether the instrument is appropriate for children, to our knowledge it has not been used and is probably not appropriate. It has, however, been used to measure the quality of life of young adults (mean age = 21 years) with juvenile rheumatoid arthritis [23-25].

Which translations are available?

The QOLS was originally developed and validated for English-speaking populations in the United States. It has been translated into at least 16 different languages: Arabic, Danish, Farsi, French, German, Greek, Hebrew, Icelandic, Italian, Mandarin Chinese, Norwegian, Portuguese (Portugal and Brazil), Spanish (Spain and Mexico), Swedish, Thai and Turkish.

Validated and published translations of the 16-item QOLS exist in Swedish, Norwegian and Hebrew [15-17]. A validated version in Mandarin Chinese exists in thesis format [26]. These translations used standardized methods of translation, backtranslation, pilot testing and critique by patients who were the intended subjects of the questionnaire.

In all the English language work, the 7-point "delighted-terrible" scaling format referenced above was used. In studies using a translated version of the instrument, a 7-point satisfaction scale anchored by "very satisfied" and "very dissatisfied" was used because the words "delighted" and "terrible" could not be meaningfully translated into the other languages. As shown in Table 3, when the means and standard deviations of the QOLS items between the English language original and the three translated versions were compared, the 7-point satisfaction scale appears to have as much variance as the "delighted-terrible" scale.

Table 3. A comparison of the means and standard deviations of the scale items in the original English language version using the 7-point delighted-terrible scale and three validated translations using the 7-point satisfaction-dissatisfaction scale.

What are the applications of the QOLS?

Over the past 20 years, a number of researchers have used the QOLS to gather quantitative QOL information from diverse groups of people with chronic illnesses. These illnesses include diabetes mellitus [13], osteoarthritis [13,27], gastrointestinal disorders [13,28], rheumatoid arthritis and systemic lupus erythematosus [13,15,29-33], chronic obstructive pulmonary disease (COPD) [14], fibromyalgia [14,21,33-36], psoriasis [38], heart disease [39,40], spinal cord injury [25], stress incontinence [41], posttraumatic stress disorder [42], and low back pain [43]. Some researchers have also found it useful for measuring the QOL of parents of children with juvenile rheumatoid arthritis [44,45] and relatives of patients with fibromyalgia [35].

How is the QOLS administered and how long does it take to complete?

The QOLS is usually self-administered either by completing the questionnaire in a clinic setting or by mail. It can also be completed by interview format. If the interview format is chosen, patients should be given a copy of the 7-point response scale to refer to when making their decision as to the most appropriate point on the scale. The QOLS can be completed in about 5 minutes.

How is the QOLS scored?

The QOLS is scored by adding up the score on each item to yield a total score for the instrument. Scores can range from 16 to 112. There is no automated administration or scoring software for the QOLS.

How are the QOLS' scores interpreted?

The QOLS scores are summed so that a higher score indicates higher quality of life. Average total score for healthy populations is about 90. For rheumatic disease groups, the average score ranges are 83 for rheumatoid arthritis, 84 for systemic lupus erythematosus, 87 for osteoarthritis, and 92 for young adults with juvenile rheumatoid arthritis. Average total scores for other conditions range from 61 for Israeli patients with posttraumatic stress disorder, to 70 for fibromyalgia, to 82 for psoriasis, urinary incontinence and chronic obstructive pulmonary disease. All of these means come from descriptive studies or experimental pretest data. And like many QOL instruments, the means tend to be quite negatively skewed with most patients reporting some degree of satisfaction with most domains of their lives.

Is the QOLS responsive to change and what is a meaningful change for the QOLS?

There is preliminary evidence that the QOLS is reponsive to change as a result of specific treatments. Five studies, recently reviewed, yielded effect sizes (mean of the treated group minus the mean of the control group divided by the pooled standard deviation) ranging from .16 to .51 when treated groups were compared to control groups and the effects of differences at pretest were accounted for [41,46-49]. The mean effect size was .24 which Cohen would call a small effect [50]. A 6-month fibromyalgia treatment program study with a comparison group who did not participate in the program showed an effect size for the treated group of .41, which Cohen would classify as moderate, for the QOLS after 6 months of multidisciplinary treatment [51]. At 2 years, the effect size for the treated group was .48 while the non-treated group showed no change in their QOLS scores.

Further data analysis of the above study [51] has shown that the average patient who completed the program rated their symptoms as 60% better than on entry. Scores ranged from 25% worse to 100% better. When these scores were condensed into three groups: -25% to 25%, 30% to 65% and 70% to 100%, mean total scores on the QOLS were 67.8 (CI 61.1 to -74.6), 79.1 (CI 75.4 to 82.8) and 82.1 (CI 77.5–86.5). QOLS mean change scores for the three groups were -1.1 (CI -6.4 to 4.1), 8.1 (CI 4.3 to 11.9) and 7.1 (CI 3.9 to 10.3) respectively. Thus, it is reasonable to expect that patients who participate in a treatment program and rate their symptoms as improved by 60% or more will gain 7 to 8 points on the QOLS total score. However, it should be noted that this applies to group means only, as the QOLS has not been used for individual patient assessment.

Who may I contact by e-mail to obtain a copy of the QOLS?

The QOLS is copyrighted by Carol Burckhardt. However, it is considered to be in the public domain. You may contact Carol Burckhardt at burckhac@ohsu.edu for a free copy of the English language version which you may duplicate and use in research or clinical practice. We ask that you cite relevant QOLS articles if you publish findings from studies. Alternatively, you may download a copy of the English language version or obtain contact information for the Swedish, Norwegian and Hebrew translations from the MAPI site http://www.qolid.org webcite if you are a member.

Conclusions

The QOLS is a reliable and valid instrument for measuring quality of life from the perspective of the patient. It focuses on domains that come from the qualitative descriptions of a wide range of adults across gender, cultural and language groups. Although Flanagan speculated that people with chronic illnesses might have different quality of life priorities or concerns, no evidence of that has ever been uncovered. Thus, the scale can be used with confidence in chronic illness groups.

References

1. Berzon RA, Donnelly MA, Simpson RL Jr, Simeon GP, Tilson HH: Quality of life bibliography and indexes: 1994 update.
   Qual Life Res 1995, 4:547-569. PubMed Abstract
   
2. Fayers PM, Hand DJ: Factor analysis, causal indicators and quality of life.
   Qual Life Res 1997, 6:139-150. PubMed Abstract
   
3. Fayers PM, Hand DJ, Bjordal K, Groenvold M: Causal indicators in quality of life research.
   Qual Life Res 1997, 6:393-406. PubMed Abstract | Publisher Full Text
   
4. Anderson KL, Burckhardt CS: Conceptualization and measurement of quality of life as an outcome variable for health care intervention and research.
   J Adv Nurs 1999, 29:298-306. PubMed Abstract | Publisher Full Text
   
5. Leplege A, Hunt S: The problem of quality of life in medicine.
   JAMA 1997, 278:47-50. PubMed Abstract | Publisher Full Text
   
6. Smith KW, Avis NE, Assmann SF: Distinguishing between quality of life and health status in quality of life research: a meta-analysis.
   Qual Life Res 1999, 8:447-459. PubMed Abstract | Publisher Full Text
   
7. Haas BK: Clarification and integration of similar quality of life concepts.
   Image: J Nurs Scholar 1999, 31:215-220.
   
8. Revicki DA, Osoba D, Fairclough D, Barofsky I, Berzon R, Leidy NK, Rothman M: Recommendations on health-related quality of life research to support labeling and promotional claims in the United States.
   Qual Life Res 2000, 9:887-900. PubMed Abstract | Publisher Full Text
   
9. Flanagan JC: A research approach to improving our quality of life.
   Am Psychologist 1978, 33:138-147. Publisher Full Text
   
10. Flanagan JC: Measurement of the quality of life: Current state of the art.
    Arch Phys Med Rehabil 1982, 63:56-59. PubMed Abstract
    
11. Cantril H: The Pattern of Human Concerns. In New Brunswick. NJ: Rutgers University Press; 1965.
    
12. Andrews RM, Crandall R: The validity of measures of self-reported well-being.
    Soc Indicator Res 1976, 3:1-19.
    
13. Burckhardt CS, Woods SL, Schultz AA, Ziebarth DM: Quality of life of adults with chronic illness: A psychometric study.
    Res Nurs Health 1989, 12:347-354. PubMed Abstract
    
14. Anderson KL: The effect of chronic obstructive pulmonary disease on quality of life.
    Res Nurs Health 1995, 18:547-556. PubMed Abstract
    
15. Burckhardt CS, Archenholtz B, Bjelle A: Measuring quality of life of women with rheumatoid arthritis or systemic lupus erythematosus: A Swedish version of the Quality of Life Scale (QOLS).
    Scand J Rheumatol 1992, 21:190-195. PubMed Abstract
    
16. Neumann L, Buskila D: Measuring the quality of life of women with fibromyalgia: a Hebrew version of the quality of life scale (QOLS).
    J Musculoskel Pain 1997, 5:5-17.
    
17. Wahl A, Burckhardt CS, Wiklund I, Hanestad BR: The Norwegian version of the Quality of Life Scale (QOLS-N). A validity and reliability study in patients suffering from psoriasis.
    Scand J Caring Sci 1998, 12:215-222. PubMed Abstract | Publisher Full Text
    
18. Wood V, Wylie ML, Sheafor B: An analysis of a short self-report measure of life satisfaction: Correlation with rater judgements.
    J Gerontol 1969, 24:465-469. PubMed Abstract
    
19. Parkerson GR, Gehbach SH, Wagner EH, James SA, Clapp NE, Muhlbaier LH: The Duke-UNC health profile: An adult health status instrument for primary care.
    Med Care 1981, 19:806-828. PubMed Abstract
    
20. Meenan RF, Gertman PM, Mason JH: Measuring health status in arthritis: The arthritis impact measurement scales.
    Arthritis Rheum 1980, 23:146-152. PubMed Abstract
    
21. Burckhardt CS, Clark SR, Bennett RM: Fibromyalgia and quality of life: A comparative analysis.
    J Rheumatol 1993, 20:475-479. PubMed Abstract
    
22. Burckhardt CS, Anderson KL, Archenholtz B, Hägg O: The Flanagan Quality of Life Scale: Evidence of Construct Validity.
    Health Qual Life Outcome, in press.
    
23. Archenholtz B, Nordborg E, Bremell T: Lower level of education in young adults with arthritis starting in early childhood.
    Scand J Rheumatol 2001, 30:353-355. PubMed Abstract | Publisher Full Text
    
24. Ruperto N, Levinson JE, Ravelli A, Shear ES, Tague BL, Murray K, Martini A, Giannini EH: Longterm health outcomes and quality of life in American and Italian inception cohorts of patients with juvenile rheumatoid arthritis. I. Outcome status.
    J Rheumatol 1997, 24:945-951. PubMed Abstract
    
25. Ruperto N, Ravelli A, Levinson JE, Shear ES, Murray K, Tague BL, Martini A, Glass DN, Giannini EH: Longterm health outcomes and quality of life in American and Italian inception cohorts of patients with juvenile rheumatoid arthritis. II. Early Predictors of Outcome.
    J Rheumatol 1997, 24:952-958. PubMed Abstract
    
26. Han S-H: Quality of life of persons with sprinal cord injury living in Taipei: A psychometric study. In School of Nursing. Portland: Oregon Health Sciences University; 1995.
    
27. Blixen CE, Kippes C: Depression, social support and quality of life in older adults with osteoarthritis.
    Image: J Nurs Scholar 1999, 31:221-226.
    
28. Sperber AD, Atzmon Y, Neumann L, Weisberg I, Shalit Y, Abu-Shakra M, Fich A, Buskila D: Fibromyalgia in the irritable bowel syndrome; Studies of prevalence and clinical implications.
    Am J Gastroenterol 1999, 94:3541-3546. PubMed Abstract | Publisher Full Text
    
29. Abu-Shakra M, Mader R, Langevitz P, Firger M, Codish S, Neumann L, Buskila D: Quality of life in systemic lupus erythematosus: A controlled study.
    J Rheumatol 1999, 26:306-309. PubMed Abstract
    
30. Archenholtz B, Burckhardt CS, Segesten K: Quality of life of women with systemic lupus erythematosus or rheumatoid arthritis: Domains of importance and dissatisfaction.
    Qual Life Res 1999, 8:411-416. PubMed Abstract | Publisher Full Text
    
31. Bradbury VL, Catanzaro M-L: The quality of life in a male population suffering from arthritis.
    Rehabil Nurs 1989, 14:187-190. PubMed Abstract
    
32. Burckhardt CS, Archenholtz B, Bjelle A: Quality of life of women with systemic lupus erythematosus: A comparison with women with rheumatoid arthritis.
    J Rheumatol 1993, 20:977-981. PubMed Abstract
    
33. Wikstrom I, Isacsson A, Jacobsson LTH: Leisure activities in rheumatoid arthritis: change after disease onset and associated factors.
    Br J Occup Ther 2001, 64:87-92.
    
34. Henriksson C, Burckhardt CS: Impact of fibromyalgia on everyday life: a study of women in the USA and Sweden.
    Disability Rehab 1996, 18:241-248.
    
35. Neumann L, Buskila D: Quality of life and physical functioning of relatives of fibromyalgia patients.
    Sem Arthritis Rheumatol 1997, 26:834-839.
    
36. Neumann L, Berzak A, Buskila D: Measuring health status in Israeli patients with fibromyalgia syndrome and widespread pain and healthy individuals: Utility of the Short Form 36-Item Health Survey (SF-36).
    Sem Arthritis Rheumatol 2000, 29:400-408.
    
37. Burckhardt CS, Clark SR, O'Reilly CA, Bennett RM: Pain-coping strategies of women with fibromyalgia: Relationship to pain, fatigue and quality of life.
    J Musculoske Pain 1997, 5:5-21.
    
38. Wahl A, Hanestad BR, Wiklund I, Moum T: Coping and quality of life in patients with psoriasis.
    Qual Life Res 1999, 8:427-433. PubMed Abstract | Publisher Full Text
    
39. Dantas RAP, Motzer SA, Ciol MA: The relationship between quality of life, sense of coherence and self-esteem in persons after coronary artery bypass graft surgery.
    Internat J Nurs Studies 2002, 39:745-755. Publisher Full Text
    
40. Motzer SA, Stewart BJ: Sense of coherence as a predictor of quality of life in persons with coronary heart disease surviving cardiac arrest.
    Res Nurs Health 1996, 19:287-298. PubMed Abstract | Publisher Full Text
    
41. Bo K, Talseth T, Vinsnes A: Randomized controlled trial on the effect of pelvic floor muscle training on quality of life and sexual problem in genuine stress incontinent women.
    Acta Obstet Gynecol Scand 2000, 79:598-603. PubMed Abstract | Publisher Full Text
    
42. Amir M, Kaplan Z, Neumann L, Sharabani R, Shani N, Buskila D: Posttraumatic stress disorder, tenderness and fibromyalgia.
    J Psychosom Res 1997, 42:607-613. PubMed Abstract | Publisher Full Text
    
43. Hägg O, Burckhardt CS, Fritzell P, Nordwall A: Quality of life in chronic low back pain: A comparison with fibromyalgia and the general population.
    J Musculoskel Pain 2003, 11:31-38.
    
44. Press J, Neumann L, Abu-Shakra M, Bolotin A, Buskila D: Living with a child with familial Mediterranean fever: Does it affect the quality of life of the parents?
    Clin Experiment Rheumatol 2000, 18:103-106.
    
45. Press J, Neumann L, Uziel Y, Bolotin A, Buskila D: Assessment of quality of life of parents of children with juvenile chronic arthritis.
    Clin Rheumatol 2002, 21:280-283. PubMed Abstract | Publisher Full Text
    
46. Brattberg G: Connective tissue massage in the treatment of fibromyalgia.
    Eur J Pain 1999, 3:235-245. PubMed Abstract | Publisher Full Text
    
47. Burckhardt CS, Mannerkorpi K, Hedenberg L, Bjelle A: A randomized, controlled clinical trial of education and physical training for women with fibromyalgia.
    J Rheumatol 1994, 21:714-720. PubMed Abstract
    
48. Jones KD, Burckhardt CS, Clark SR, Bennett RM, Potempa KM: A randomized controlled trial of muscle strengthening versus flexibility training in fibromyalgia.
    J Rheumatol 2002, 29:1041-1048. PubMed Abstract | Publisher Full Text
    
49. Mannerkorpi K, Nyberg B, Ahlmen M, Ekdahl C: Pool exercise combined with an education program for patients with fibromyalgia syndrome. A prospective, randomized study.
    J Rheumatol 2000, 27:2473-2481. PubMed Abstract
    
50. Cohen J: Statistical Analysis for the Behavioral Sciences. In Hillsdale. NJ: L Erlbaum Associates; 1988.
    
51. Bennett RM, Burckhardt CS, Clark SR, O'Reilly CA, Wiens AN, Campbell SM: Group treatment of fibromyalgia: a 6-month outpatient program.
    J Rheumatol 1996, 23:521-528. PubMed Abstract