Abstract. Context: Acupuncture effectiveness should have to be demonstrated by consistent research results and require explanations consistent with modern knowledge. However, clinical trials, meta-analyses, and reviews of acupuncture efficacy have shown conflicting results and have yielded conflicting recommendations. This overall review of meta-analyses and reviews asseses overall efficacy of acupuncture as a method.
Objectives: We collected reviews and meta-analyses of acupuncture for various symptoms published since 1990 in order to estimate overall general effectiveness as well as effectiveness for specific conditions.
Data Sources: We recorded 33 literature reviews for 17 clinical conditions, containing over 200 evaluable clinical trials from 1990 to the present using MEDLINE as a primary database source. Three recent review books were secondary sources. We then recorded the review authors’ conclusions from each review.
Study selection: All systematic reviews and metaanalyses found were included in the review.
Data extraction: Outcome of each review was classified according to authors’ conclusions of effective, indeterminate effectiveness, or ineffective. No attempt was made to grade the quality of each review. No attempt was made to pool data or to compare reviews within the specified symptom categories.
Data synthesis: Review of 33 meta-analyses and systematic reviews showed a positive effect in a majority of trials for only 2 symptoms (nausea and dental pain) and then only by a slim majority and under specific conditions. For the remainder of conditions tested, the majority of trials showed either inconclusive results or no effect.
Conclusion: Effectiveness could not be established with confidence for any condition studied. Taken as a group, reviews of clinical studies published since 1990 on the clinical efficacy of acupuncture do not support the notion that acupuncture is effective for any variety of conditions and cast doubt on efficacy for some specific conditions for which acupuncture has been reported as effective.
For approximately the past 30 years, peer-reviewed literature has accumulated hundreds of clinical investigations into various applications for acupuncture. For each application, individual studies exist both supporting and refuting effectiveness. We undertook this study of systematic reviews and metaanalyses in order to determine the likelihood of acupuncture’s overall effectiveness and its effectiveness in specific conditions.
MATERIALS AND METHODS
Studies were located using the MEDLINE database with the following search criteria: acupuncture, acupuncture meta-analysis, acupuncture systematic review, and acupuncture pain. Additional data were obtained from the books Ernst E and White A, eds., Acupuncture: A Scientific Appraisal (Oxford: Butterworth Heinemann, 1999); Filshie J and White A, eds., Medical Acupuncture: A Western Scientific Approach (Edinburgh: Churchill Livingstone, 1998); and Waddell G, The Back Pain Revolution (Edinburgh: Churchill Livingstone, 1998).
In 1994 the United States Agency for Health Care Policy and Research screened 24 articles pertaining to acupuncture for back pain.1 It noted the poor methodological quality of most studies and concluded that “invasive needle acupuncture and other dry needling techniques are not recommended for treating patients with acute low back problems.”
A 1998 meta-analysis of randomized, controlled trials found that in trials that were both sham controlled and patient and evaluator blinded, acupuncture was superior to no treatment but that it was not superior to sham.2
A 1999 systematic review concluded that acupuncture could not be recommended as a treatment for human low back pain.3
A 2000 systematic review of 13 acupuncture trials in the treatment of back and neck pain assessed analgesic efficacy and adverse effects of acupuncture compared with placebo. In addition, the study attempted to develop a new tool to measure validity of findings from randomized, controlled trials (RCTs), to enable ranking of trial findings according to validity within qualitative reviews (the Oxford Pain Validity Scale [OPVS]. Statistical analyses were carried out on the OPVS scores to assess the relationship between trial finding (positive or negative) and validity. The review found that the conclusions of the authors of individual studies did not always agree with their data. Higher validity scores were associated with negative findings. The review concluded that there is no convincing evidence for the analgesic efficacy of acupuncture for back or neck pain.4
Nausea and vomiting
A 1996 review analyzed 33 controlled trials in which the P6 acupuncture point was stimulated for treatment of nausea and/or vomiting of chemotherapy, pregnancy, or postsurgery status. Authors concluded that positive results were found except when the acupuncture was administered under anesthesia.5
A 1998 meta-analysis of the use of nonpharmacological techniques to prevent postoperative nausea and vomiting looked at 19 trials including acupuncture, electroacupuncture, and acupressure. The outcomes were nausea, vomiting, or both, 0 to 6 hours (early efficacy) or 0 to 48 hours (late efficacy) after surgery. Results were mixed. Nonpharmacological techniques were found to be equivalent to commonly used antiemetic drugs within six hours of surgery and were 20 to 25% more effective than placebo. Acupuncture was not effective for late vomiting, nor was it effective in children.6
A 1997 review of acupuncture for osteoarthritis found the 11 reviewed studies to be “highly contradictory.” Methodological flaws were noted in most of the trials. The authors concluded that “real” acupuncture was not superior to sham needling and that both may have specific or nonspecific effects.7
A 1990 systematic review identified 51 controlled trials. These studies were reviewed using a list of 18 methodological criteria. The quality of even the better studies was described as “mediocre.” The study concluded that the efficacy of acupuncture for chronic pain was doubtful.8
A 2000 systematic review evaluated 51 studies that met inclusion criteria. Seventy-five percent of the studies were of low quality, and low quality was associated with positive results (p = 0.05). The study concluded that there is limited evidence that acupuncture is more effective than no treatment for chronic pain; the evidence that acupuncture is more effective than placebo, sham acupuncture, or standard care is inconclusive.9
A 1999 systematic review identified 12 trials of manual acupuncture (MA) for analgesia, 10 of which were sham controlled. The overall result of the review “provides no support for an analgesic effect of MA.”10
A 1998 systematic review identified 12 trials of electroacupuncture (EA) analgesia but did not report whether the trials were sham controlled. The review found that EA raised the pain threshold by an average of 20 to 30%—too small a difference to be clinically useful.11
Dental and temporomandibular joint pain
A 1998 review recorded 48 papers on temporomandibular dysfunction (TMD) and on dental analgesia. The study concluded “the value of acupuncture as an analgesic must be questioned” but that there seemed to be a real effect in TMD and facial pain.12
A 1998 review on acute dental pain11 evaluated 16 controlled trials. The reviewers found the majority of the trials to be positive in dental analgesia; however, they called for future investigations to define “optimal” acupuncture technique and to compare its effectiveness with conventional methods.
A 1999 review evaluated 6 reports and suggested that acupuncture might be an effective therapy for TMD. However, none of the studies was controlled for placebo effects. The reviewers stated that more rigorous investigations were required.12
A 1996 review of acupuncture in stroke rehabilitation evaluated all published controlled trials. The authors found the trials to be uniformly positive for functional recovery. However, all studies failed to account for placebo effect and contained other methodological flaws. The review concluded that “the evidence that acupuncture is a useful adjunct for stroke rehabilitation is encouraging but not compelling” and called for more trials.13
A 1991 review analyzed 13 controlled trials and concluded that “claims that acupuncture is effective in the treatment of asthma are not based on the results of wellperformed clinical trials.”14
A 1998 review summarized reports from diverse sources and weighted the validity of the conclusions based on design, number of subjects, duration of studies, types and number of controls, and statistical analyses used. Authors concluded that “data presently in the literature do not provide sufficient support for a useful role for acupuncture in asthma management.”15
A 2000 review of 7 trials involving 174 subjects noted that the trial quality varied and that results were inconsistent. No statistically significant or clinically relevant effect was found for acupuncture compared to sham. The reviewers concluded that there was not enough evidence to recommend acupuncture in asthma.16
A 1997 systematic evaluating acupuncture for weight reduction included 4 studies. The 2 more rigorous studies evaluated showed no benefit, while the 2 less rigorous studies suggested some benefit. Claims for the effectiveness of acupuncture for weight loss were not based on evidence from well-conducted clinical trials.17
A 1991 systematic review evaluating alternative treatments, including acupuncture, in the treatment of rheumatic diseases (rheumatoid arthritis, osteoarthritis, “soft tissue rheumatism,” and “fibromyalgia”) found no convincing proof that acupuncture was more effective than control or placebo.18
A 1999 review evaluated 7 studies and found only 1 that was of “high” methodological quality. That trial suggested that “real” acupuncture was superior to sham. However, the duration of benefit was not known. In addition, some patients reported no benefit and others reported an exacerbation of symptoms. The positive conclusions of this review were based on a single study and the reviewers called for more high-quality trials to provide more robust data on effectiveness.19
Another 1999 review of acupuncture for fibromyalgia found 2 trials with both subjective and objective evidence of pain reduction, but 3 that documented exacerbation of symptoms.20
A 1998 systematic review identified 5 sham-controlled studies. Of those, 1 was positive and 4 were negative. The review concluded that the data were not sufficient to state with certainty that acupuncture is superior to sham acupuncture or to other controls for the treatment of neck pain.21
A 1999 analysis of randomized clinical trials evaluated 27 studies. The authors found that the quality of the studies was high, but acupuncture had either no effect or a negative effect on outcome.22
A 2000 review of treatments for mechanical neck disorders reviewed randomized and controlled trials of multiple physical medicine methods, including acupuncture. The study concluded that not enough scientific testing exists to clearly determine the effectiveness of acupuncture for this indication.23
A previously cited 2000 review of acupuncture for the treatment of neck pain concluded that there was no convincing evidence for the efficacy of acupuncture.4
A 1999 review evaluated 22 randomized or “quasi-randomized” trials comparing acupuncture with any type of control. Meta-analysis of specific outcome data was not possible because of heterogeneity and insufficient reporting. The investigators concluded that acupuncture has a role in the treatment of recurrent headaches but that the quality and amount of evidence is not fully convincing.24
A 1995 analysis of smoking cessation evaluated 188 randomized, controlled clinical trials, including acupuncture, and concluded that acupuncture is ineffective.25
A 1999 review of smoking addiction identified 10 sham-controlled, randomized trials. Overall, the quality of the studies was poor. Acupuncture was not shown to be superior to sham control and no particular acupuncture technique was associated with a positive effect.26
A 2000 review identified 18 randomized trials involving 20 comparisons of acupuncture with either sham acupuncture, another intervention, or no intervention. Acupuncture was not superior to sham. When acupuncture was compared with other antismoking interventions, there were no differences in outcome. The authors concluded that there is no clear evidence that acupuncture is effective for smoking cessation.27
A 2000 systematic review identified 6 randomized, controlled trials. Two unblinded studies showed a positive result, whereas 4 blinded studies showed no significant effect. The authors concluded that acupuncture has not been demonstrated to be efficacious as a treatment for tinnitus.28
A 1990 meta-analysis of 3 types of addiction evaluated 22 controlled trials. It found that design of the trials was generally poor. Overall, 14 studies were negative and 8 were positive—the positive ones all inferior in quality. For smoking, it found that the studies with negative outcomes were overwhelmingly negative. In one of the best of the positive studies, 90% of the control patients dropped out. The sum of the best studies was more negative than the sum of the group as a whole. The report concluded that “claims that acupuncture is efficacious as a therapy for these addictions are thus not supported by results from sound clinical research.”29
A 1996 review evaluated the efficacy 12 trials covering of a number of treatments for this condition, including acupuncture. It found that most of the therapies evaluated, including acupuncture, were ineffective.30
The NIH report
Although not a formalized systematic review, one other report bears mentioning. In 1997 a Consensus Development Conference of the United States National Institutes of Health delivered a consensus statement on acupuncture.31 The conference was organized and supported by the Office of Alternative Medicine and the Office of Medical Applications of Research of the NIH. Contrary to common belief, the committee report was not the opinion or position of the NIH, but that of the people invited and some NIH staff members. A list of 2302 references was provided. The panel concluded that many studies provided equivocal results because of design, sample size, and other factors. In addition, it found that the issue was complicated by difficulties in the use of appropriate controls, such as placebos and sham acupuncture groups. The panel conceded that placebo effects may be operative in acupuncture while accepting the proposition that acupuncture has more specific, nonplacebo physiological effects as well. However, such specific mechanisms were not identified, nor have any been commonly accepted as valid. The panel also noted that relatively few high-quality randomized, controlled trials have been published on the effects of acupuncture.
In spite of these difficulties, the panel concluded that acupuncture showed efficacy for adult postoperative and chemotherapy-associated nausea and vomiting, and for postoperative dental pain. It was considered “an adjunct treatment or an acceptable alternative” to be considered in such things as addiction, stroke rehabilitation, headache, menstrual cramps, tennis elbow, fibromyalgia, myofascial pain, osteoarthritis, low back pain, carpal tunnel syndrome, and asthma, despite the fact that the evidence for those recommendations was poor to contradicting. The panel concluded that acupuncture was not effective for smoking cessation and that the evidence was insufficient to evaluate treatment of other conditions for which acupuncture is used. It also arrived at an inconsistent, paradoxical conclusion that since even sham acupuncture has effects, studies showing no difference between “real” and sham cannot imply lack of effect.
The conclusions of the panel were not convincing to some.32 Reviews usually reflect the bias of the reviewers.33 The conference invited only token authors of studies with negative results. The organizing committee was also made up largely of NIH staff and extramural members interested in “alternative” methods. A subsequent report evaluated the 12 conditions for which acupuncture was found “useful” by the NIH Consensus Conference. It found “partial support” for only 4 of those conditions.34 We found some previously published reviews that contradict those 4 indications.
Semiquantitative assessment of the efficacy of a method by reviewing reviews may not be ideal, but it seems to be a reasonable way to evaluate the vast amount of data in multiple trials. The relatively recent technique of metaanalysis, though questioned by some,35,36,37,38 appears to be most useful when the primary literature is of good quality, there is little heterogeneity in the response to treatment, the objective is a critical outcome, the reports are graded as to quality, and the persons performing the meta-analysis are expert in the subject matter.36 Systematic analyses of acupuncture trials rarely fit all the above criteria. Meta-analysis techniques have also produced results that may be misleading—results may disagree with the those of later randomized, controlled clinical trials.39 Grading individual trials and looking for inconsistencies and errors should increase the reliability of meta-analyses.
We did not attempt to grade the individual reports or the reviews and analyses, but recorded only the conclusions of the reviews’ authors. In absence of undertaking individual trial and review quality, an undertaking outside the scope of this study, we had to assume that reviews used reliable techniques and assumed any bias would more likely have shown up as favoring acupuncture. Finally, it should be noted that many of the published reviews on acupuncture come from one group; it is unknown how or if this might bias any conclusions.
Some generalizations appear to be justifiable. First, reviews of acupuncture do not support the notion that acupuncture is an effective treatment for the wide variety of conditions tested (Table).
Second, important questions remain regarding any beneficial effects seen. For example, it is curious that while acupuncture is found to be effective for some types of nausea, it is not effective for all types of nausea. It appears to diminish nausea occurring less than 6 hours after a surgical procedure, but not more than 6 hours afterward. It is ineffective in preventing nausea in children and when it is performed under general anesthesia. These inconsistencies suggest that inadequate control of expectation, suggestion, and other nonspecific effects resulted in more positive than negative studies. Also, in any situation in which large numbers of studies are performed, a certain number will be positive by chance, and publication bias would result in publication of more positive than negative studies. There is no sensible reason why acupuncture should be effective in selected subgroups, and finding positive results in subgroups can also be due to chance.
Similarly, acupuncture is claimed to be effective for dental pain, but meta-analysis has shown ineffectiveness for neck and back pain. This inconsistency again suggests that for dental pain, results can be explained by nonspecific effects caused by defective blinding, or by chance or publication bias. Not commonly appreciated is the fact that almost all trials on acupuncture have been for various symptoms, rather than for altering the course of disease. Disease outcomes are more objectively measurable and less affected by nonspecific effects.
Third, if there is a useful physiological effect of acupuncture, it is not clear that precise needle placement is important. Most studies fail to support the hypothesis that “real” acupuncture is superior to “sham” acupuncture.40 If precise needle placement is largely irrelevant and if needling is an effective therapy, then it may be that acupuncture training is unduly complicated and that “certification” is unnecessary.
The common call for more high-quality research may by this time be seriously questioned. Historically, acupuncture has been tried and discarded in 3 previous waves of interest.41 Thirty years of active acupuncture research have failed to provide clear evidence of its efficacy.42 The significant number of negative well-designed and well-implemented trials suggests that acupuncture effects are nonspecific—that is, due to counterirritation, expectation, suggestion, conditioning, and other effects having nothing to do with acupuncture itself.43,44
Common sense and scientific custom indicate that truly effective methods should show consistency across trials. In a meta-analysis of 44 trials of tamoxifen effectiveness in postoperative breast cancer, 25 trials containing over 95% of the patients were positive, while only 5 small trials were negative.45 Preoperative radiotherapy for resectable rectal cancer was beneficial in 11 trials and showed no effect in only 4.46 Unfortunately, there is no agreed-upon number or percentage of negative trials that differentiates effectiveness from ineffectiveness. Evaluation remains here a matter of judgment.
However, the overwhelming evidence in acupuncture clinical trials is negative. Serious consideration must be given to reduction or cessation of acupuncture investigations and to replacement of acupuncture by more effective methods. There seems to be no reason to teach the method to students and physicians.
- Agency for Health Care Policy and Research Clinical Practice Guideline No. 14, AHCPR Publication No. 950642:1994.
- Ernst E, White AR. Acupuncture for back pain: a meta-analysis of randomized controlled trials. Arch Int Med. 1998; 158: 2235–2241.
- van Tulder MW, Cherkin DC, Berman B, et al. The effectiveness of acupuncture in the management of acute and chronic low back pain. A systematic review within the framework of the Cochrane Collaboration Back Review Group. Spine. 1999; 24: 1113–1123.
- Smith LA, Oldman AD, McQuay HJ, et al. Teasing apart quality and validity in systematic reviews: an example from acupuncture trials in chronic neck and back pain. Pain. 2000; 86: 119–132.
- Vickers AJ. Can acupuncture have specific effects on health? A systematic review of acupuncture antiemesis trials. J R Soc Med. 1996; 89: 303–311.
- Lee A, Done ML. The use of nonpharmacologic techniques to prevent postoperative nausea and vomiting: a meta-analysis. Anesth Analg. 1999; 88: 1362–1369.
- Ernst E. Acupuncture as a symptomatic treatment of osteoarthritis. A systematic review. Scand J Rheumatol. 1997; 26: 444–447
- ter Riet G, Kleijnen J, Knipschild P. Acupuncture and chronic pain: a criteria-based meta-analysis. J Clin Epidemiol. 1990; 43: 1191–1199.
- Ezzo J, Berman B, Hadhazy VA, et al. Is acupuncture effective for the treatment of chronic pain? A systematic review. Pain. 2000; 86: 217–225.
- White A. Acupuncture: A Scientific Appraisal. Oxford, England: Butterworth-Heinemann, 1999: 76–78.
- Rosted P. The use of acupuncture in dentistry: a review of the scientific validity of published papers. Oral Dis. 1998; 4: 100–104.
- Ernst E, White A. Acupuncture as a treatment for temporomandibular joint dysfunction: a systematic review of randomized trials. Arch Otolaryngol Head Neck Surg. 1999; 125: 269–272.
- Ernst E, White AR. Acupuncture as an adjuvant therapy in stroke rehabilitation? Wien Med Wochenschr. 1996; 146: 556–558.
- Kleijnen J, ter Riet G, Knipschild P. Acupuncture and asthma: a review of controlled trials. Thorax. 1991; 46: 799–802.
- Davis PA, Chang C, Hackman RM, et al. Acupuncture in the treatment of asthma: a critical review. Allergol Immunopathol. 1998; 26: 263–271.
- Linde K, Jobst K, Panton J. Acupuncture for chronic asthma. Cochrane Database Syst Rev. 2000; 2: CD000008.
- Ernst E. Acupuncture/acupressure for weight reduction? A systematic review. Wien Kiln Wochenschr. 1997; 109: 60–62.
- Jacobs JW, Rasker JJ, Van Riel PL, et al. Alternative treatment methods in rheumatic diseases: a literature review. Ned Tijdschr Geneeskd. 1991; 135: 317–322.
- Berman BM, Ezzo J, Hadhazy V, et al. Is acupuncture effective in the treatment of fibromyalgia? J Fam Pract. 1999; 48: 213–218.
- Berman BM, Swyers JP. Complementary medicine treatments for fibromyalgia syndrome. Balliere’s Clin Rheumatol. 1999; 13: 487–492.
- White AR, Ernst E. A systematic review of randomized controlled trials of acupuncture for neck pain. Rheumatology. 1999; 38: 143–147.
- Kjellman GV, Skargren EI, Oberg BE. A critical analysis of randomised clinical trials on neck pain and treatment efficacy. Scand J Rehabil Med. 1999; 31: 139–152.
- Gros AR, Aker PD, Goldsmith CH, et al. Physical medicine modalities for mechanical neck disorders. Cochrane Database Syst Rev. 2000; 2: CD000961.
- Melchar D, Linde K, Fischer P, et al. Acupuncture for recurrent headaches: a systematic review of randomized controlled trials. Cephalalgia. 1999; 19: 779–786.
- Law M, Tang JL. An analysis of the effectiveness of interventions intended to help people stop smoking. Arch Intern Med. 1995; 155: 1933–1941.
- White AR, Resch KL, Ernst E. A meta-analysis of acupuncture techniques for smoking cessation. Tob Control. 1999; 8: 393–897.
- White AR, Rampes H, Ernst E. Acupuncture for smoking cessation. Cochrane Database Syst Rev. 2000; 2: CD000009.
- Park J, White AR, Ernst E. Efficacy of acupuncture as a treatment for tinnitus: a systematic review. Arch Otolaryngol Head Neck Surg. 2000; 126: 489–492.
- ter Riet, G, Kleijnen J, Knipschild P. Ameta-analysis of studies into the effect of acupuncture on addiction. Br J Gen Pract. 1990; 40: 379–382.
- Volmink J, Lancaster T, Gray S, et al. Treatments for postherpetic neuralgia—a systematic review of randomized controlled trials. Fam Pract. 1996; 13: 84–91.
- NIH Consensus Conference. Acupuncture. JAMA. 1998; 280: 1518–1524.
- Sampson W. On the National Institute of Drug Abuse Consensus Conference on Acupuncture. Sci Rev Alt Med. 1998; 2(1): 54–55.
- Egger M, Schneider M, Smith GD. Spurious precision? Meta-analysis of observational studies. BMJ. 1998; 316: 140–144.
- Mayer D. Acupuncture: an evidence-based review of the clinical literature. Ann Rev Med. 2000; 51: 49–63.
- Bailar, JC III. The practice of meta-analysis. J Clin Epidemiol. 1995; 48: 1491–1557.
- Bailar JC III. The promise and problems of metaanalysis. N Engl J Med. 1997; 337: 559–561.
- Egger M, Smith GD. Meta-Analysis. Potentials and promise. BMJ. 1997; 315: 1371–1374.
- Herbison P. Problems with meta-analysis. N Z Med J. 1999; 112: 38–41.
- LeLorier J, Gregoire G, Benhaddad A, et al. Discrepancies between meta-analyses and subsequent randomized, controlled trials. N Engl J Med. 1997; 337: 536–542.
- Sanchez AM. Does the choice of placebo determine the results of clinical studies on acupuncture? Forsch Komplementarmed. 1998; S1: 8–11.
- Unschuld P. Chinese Medicine. Brookline, Mass: Paradigm; 1998.
- Sampson W. National Council against Health Fraud position paper on acupuncture. Clin J Pain. 1991; 7: 162–166.
- Moroz A. Issues in acupuncture research: the failure of quantitative methodologies and the possibilities for viable, alternative solutions. Am J Acupunct. 1999; 27: 95–103.
- Beyerstein B. Social and judgemental biases that make inert treatments seem to work. Sci Rev Alt Med. 1999; 3(2): 20–33.
- Early Breast Cancer Trialists Collaborative Group. Systemic treatment of early breast cancer by hormonal, cytotoxic, or immune therapy. Lancet. 1992; 339: 1–15.
- Camma C, Giunta M, Fiorica F, et al. Preoperative radiotherapy for resectable rectal cancer, a meta-analysis. JAMA. 2000; 284: 1008–1015.