|
|
|
|
Criteria for Assessing the Quality of Health Information on the Internet - Policy Paper Background | Evaluation Criteria | Summary | Action Plan Supported in part by the Agency for Health Care Policy and Research, Conference Grant #R13 HS09549-01 This policy paper presents a set of seven criteria developed for use in evaluating the quality of health information provided on the Internet. It is hoped that organizations and individuals will become involved in this important effort at some level, whether it be supporting and/or endorsing the policy paper, adopting the use of the set of criteria, joining the Health Summit Working Group, or educating others on this topic. As health information proliferates on the Internet, there is a growing need for objective, reproducible, widely accepted criteria that can be used to evaluate the quality of the information. To address this need, the Health Information Technology Institute of Mitretek Systems convened a Health Summit Working Group. Participants included representatives of the general public, as well as health-care providers, medical librarians and related information resources professionals, and website developers affiliated with numerous organizations in the health-care and information communities. The group held three Health Summit Meetings over a period of 18 months (November 1996–May 1998). The first and a portion of the second meeting were focused on creating and refining a set of criteria for assessing the quality of health information on the Internet. Broad input and outside review of these criteria were solicited at a number of medical and scientific meetings, and an interim white paper was posted on the Web for comment. The revised set of criteria that resulted from that process is presented in this document. The second and third Health Summit Meetings centered on implementing these criteria into a Web-based, interactive tool. The tool development, the testing process used to validate the tool, and the test results are beyond the scope of this paper, but will be presented in a subsequent paper. The Problem The Internet can be a valuable resource for users seeking health information. The quality of this information is critically important as it could potentially affect health outcomes for millions. Yet the quality of health information on the Internet is extremely variable and difficult to assess. Thus the choice of appropriate evaluation criteria for the information is both crucial and challenging. This problem is generally recognized. There is no consensus, however, on how to resolve the problem, and there remain no uniform guidelines for quality assessment of Web-based health information for consumers. A number of websites display rating schemes, and there are "stamps of approvals," but there is little explanation of how those schemes were developed or how the ratings have been applied. The Health Summit Working Group addressed this problem by developing a set of criteria for use in assessing the quality of health information on the Internet. Purpose The purpose of this paper is to provide a brief overview and explanation of the criteria developed by Health Summit Working Group. The ultimate aim of the group’s efforts is to improve the quality of health information on the Internet, and the group regards the development of these criteria—designed to serve as the basis for decision making and tool development—as an essential step toward that goal. The intended audience for this document includes the general public, policymakers, health professionals, and providers of Web-based health information. It is hoped that organizations and individuals reviewing these criteria will use them to assess the quality of health information on the Internet. Scope The criteria presented herein were developed by focusing on health information websites intended primarily for the general public, and do not address information on mailing lists, listservs, bulletin boards, chat rooms, etc., where it is easier to post information with greater anonymity. The emphasis is on the quality of the information provided. Methods and Process The Health Information Technology Institute convened a one-day Health Summit Meeting at Mitretek Systems in McLean, Virginia, on Friday, November 22, 1996. To ensure objectivity in the development of the criteria, 18 individuals with diverse backgrounds, including leaders of medical schools, libraries, pharmaceutical associations, government agencies, professional medical associations, and consumer groups, were invited to attend. The meeting consisted of a series of presentations, open discussions, and breakout groups, and resulted in the initial identification of potential criteria and their strengths and weaknesses, as well as a list of items for future action. Additionally, participants in the meeting developed the preliminary draft of this policy paper that was presented for comment as described above (http://hitiweb.mitretek.org/docs/criteria.htm) The next step in the process was to rank these initial criteria in order of importance. Members of the Health Summit Working Group and other individuals and organizations were invited to provide their suggested rankings. The criteria were categorized as either "Essential," "Important," or "Desirable," and these ratings served as a starting point for further refinement of the criteria at the second Health Summit Meeting. The second Health Summit Meeting was convened on Friday, October 17, 1997, at Mitretek Systems. Participants included those who attended the first meeting, as well as many newly recruited individuals. The purpose of this meeting was to develop an implementable set of criteria (based on the policy paper) for use by the general public. The participants first reviewed and identified those criteria capable of being implemented, and then further refined these criteria into a usable question-answer rating format to be incorporated into a tool developed by the Health Information Technology Institute. In breakout sessions, participants worked on an implementation strategy and identified next steps, which included testing of the tool by volunteers from each of four working groups comprising consumers, librarians, health-care providers, and website developers, respectively. The results of the preliminary tool testing and refinement were then presented at the third Health Summit Meeting, held May 18, 1998, at Mitretek Systems. The focus of this meeting was on the refinement, analysis, and testing of the tool, as well as initial steps toward the creation of a "gold standard" for high-quality health information websites. Evaluation Criteria Back to Top The Health Summit Working Group selected, defined, ranked, and evaluated seven major criteria for assessing the quality of Internet health information: credibility, content, disclosure, links, design, interactivity, and caveats (advisories). Criteria for Evaluating Internet Health Information * Credibility: includes the source, currency, relevance/utility, and editorial review process for the information. * Content: must be accurate and complete, and an appropriate disclaimer provided. * Disclosure: includes informing the user of the purpose of the site, as well as any profiling or collection of information associated with using the site. * Links: evaluated according to selection, architecture, content, and back linkages. * Design: encompasses accessibility, logical organization (navigability), and internal search capability. * Interactivity: includes feedback mechanisms and means for exchange of information among users. * Caveats: clarification of whether site function is to market products and services or is a primary information content provider. Credibility To determine the credibility of Internet health information, one must consider its source, currency, relevance/utility, and editorial review process. Source. The source of medical information is the premier criterion for its credibility and quality. Therefore, a site should display the name and logo of the institution or organization responsible for the information, as well as the name and the title of the author(s), if relevant. Additionally, the qualifications/credentials of the organization and author(s), along with any relevant personal or financial associations or other real or potential sources of bias, should be disclosed. Disclosing sponsorship and the nature of such support can help consumers assess the motivations of the provider of the information (e.g., if the site is advertising a product or service) and any potential conflicts of interest. Currency. The date of the original document on which the information is based and the date of posting on the Web should be provided so the user can judge the timeliness of the information. Although the date of posting does not necessarily show that the information provided is correct or up to date, it does serve as an indicator of currency. Relevance. Users need to know the content of a site corresponds to the information it purports to offer. Site Evaluation. In an academic community, the peer review process is used to ensure the validity and quality of the information presented in papers and reports. The general public, however, is more likely to understand a "seal of approval" from an individual or group commonly perceived as credible. Sites should indicate whether the information provided has been subjected to review, and if so, describe the process and the individuals involved. Content The content of health information on the Internet must be accurate and complete; an appropriate disclaimer should also be provided. Accuracy. Given that accuracy of content is based on evidence and its verification, the site should identify the data that underlies the conclusions presented. Clinical or scientific evidence that supports a position should be clearly stated. The framework of the study should also be described in language the lay person can understand. And users need to be aware that testimonials are not evidence. Disclaimer. A disclaimer describing the limitations, purpose, scope, authority, and currency of the information should be provided. To ensure accuracy and avoid plagiarism and copyright violation, sources of the information should also be disclosed. The disclaimer should emphasize as well that the content is general health information, not medical advice. Completeness. The treatment of a topic should be comprehensive and balanced. Users should be wary of one-sided views with critical information missing. Pertinent facts, negative results, and a statement of any information not known about the subject should be included. Disclosure Websites should provide appropriate disclosures, including the purpose of the site, as well as any profiling/collecting of information associated with using the site, so users can understand the intent of the organization or individual in providing the information. Purpose. The mission or purpose of the site should be clearly stated, and the information provided should be appropriate to that mission or purpose. Profiling/Collection of Information. Websites request and use information for purposes of which the user may be unaware. It is critical that users be informed of the collection, use, and dissemination of any information they may be providing in visiting the site. Only then can they make an informed decision to provide the information and/or approve of its eventual use. Links Especially critical to the quality of an Internet site are its external links—connections to other internal pages or to external sites that form the web-like structure of information searches within and among sites. There are four criteria for evaluating the quality of links: selection, architecture, content, and back linkages. Selection. The selection of links is made at the originating site. The person or group responsible for link selection should have the expertise and credentials to evaluate critically the appropriateness of those links. It is also important that the original and linked sites target a set of readers with similar characteristics. Architecture. The architecture or design of pointers to linked sites is important for ease of navigation: whether there are timely escape mechanisms during side searches, whether the user can easily find his or her way backwards and forwards, and whether the structure is apparent and logical to the reader. Image-based icons and textual identifiers should be meaningful and consistent. Content. The content of the links should be accurate, current, credible, and relevant. The content of the originating site is enhanced if it includes links to high-quality sites; on the other hand, links to poor-quality sites indicate a lower-quality originating site. Users should be alerted when they are about to view an external site. Means that can be used for this purpose include information relating to the linked source, provided before the user clicks to the site, and use of transition screens. Back Linkages. Back linkages are links to one website from another. Many websites track and publish back linkages for the purpose of enhancing their credibility and marketability. The best way to evaluate back linkages is to examine the context in which they are used, that is, their purpose, relevance, credibility, and authority, as well as any associated bias. Design The design or layout of the website, including graphics and text, as well as links, is important to the effective delivery and use of any Web-based information, even though it does not affect the quality of the information per se. The design of websites can be evaluated in terms of accessibility, logical organization (navigability), and internal search capability. Access. Websites should be accessible by the lowest-level available browser technology. Other features to improve access include options for accessing the information when multimedia browsers are not available, as well as options for enabling use by the hearing and seeing impaired. Logical Organization (Navigability). The best websites are clearly focused on their purpose and target audience. They are simple, internally consistent, and easy to use. Cross-references are provided to aid the user in comprehending the overall structure of the information. The composition of the information reflects an awareness of reading level, language, and the need for a balance of text and graphics, color and sound. Internal Search Capability. An internal search engine is a highly desirable component for most websites with depth and breadth of content. The scope and function of the search engine—what it covers and how it works—should be clearly described. The search engine should be capable of searching specified content by keyword or search string and retrieving only relevant materials. It should also have a user interface that is easy to understand and use. Interactivity Websites should include a feedback mechanism for users to offer their comments, corrections, and criticisms, and raise questions about the information provided. This makes the website accountable to its users. If a site provides a chat room, allowing information to be exchanged among many individuals, an indication of whether a moderator is present should be provided, together with a warning that the information may not be accurate. If a moderator is present, his/her expertise and affiliations, as well as the source of his/her compensation should be identified. When a website provides an interactive service, such as tailoring information to the user based on clinical algorithms, the algorithm used should be posted, including its developer and the site’s affiliation with the developer. Caveats Sites that market services and products have different agenda then those that are primary content providers. Summary Back to Top The Internet presents a powerful mechanism for helping users improve their health-care decision making by providing easy and rapid access, exchange, and dissemination for enormous amounts of health information. Yet users must be aware of the potential for misinformation and recognize the critical need to assess the quality of the information provided. Content providers must be encouraged to develop and post high-quality information, and policymakers and health-care professionals must be educated on this important health issue. The Health Summit Working Group has developed this set of criteria to address this critical need. These criteria are intended as a resource for users seeking health-related information on the Internet, and should aid in evaluating information to determine whether it is usable and credible. Action Plan Back to Top Individuals and organizations reviewing this policy paper are encouraged to become involved in this important effort at some level, whether it be supporting and/or endorsing this policy paper, adopting the use of the criteria, joining the Health Summit Working Group, educating others on this topic, or participating in the tool development and testing process. Contact Information: For more information on the Health Summit Working Group, visit our homepage or email hiti@mitretek.org Acknowledgements Chair: Helga Rippen, MD, PhD, MPH, Health Information Technology Institute, Mitretek Systems Lead - Librarian Working Group: Roger Guard, University of Cincinnati and Association of Academic Health Sciences Libraries Lead - Consumer Working Group: Marshall Kragen, JD; Next lead to be announced Lead – Health Care Provider Working Group: Patricia Byrns, BSN, MD, University of Colorado Health Sciences Center Bill Silberg, Journal of the American Medical Associaiton Lead – Developer Working Group: Denise Silber, MBA Lead – Products and Services Working Group: Richard Cleland, Federal Trade Commission Patricia A. Abbott, MS, RN-C, University of Maryland School of Nursing & American Nursing Association John Ambre+, MD, PhD, American Medical Association Susan Anderson, American Medical Association Ralph Arcari, University of Connecticut Health Center Thomas G. Basler, PhD, Medical University of South Carolina Suzy Buckovich, JD, MPH, Health Information Technology Institute Jose T. M. Carnerio, MPA, EdD, Office of Minority Health Resource Center Betty Chang, UCLA School of Nursing Richard Cleland, Federal Trade Commission William F. Decker, American Association of Retired Persons Mary Jo Deering, PhD, Office of Disease Prevention and Health Promotion, HHS Charlie Dragovich, American Pharmaceutical Association Thomas Eng, VMD, MPH, Office of Disease Prevention and Health Promotion, HHS J. Michael Fitzmaurice, PhD, Agency for Healthcare Policy and Research Jean Fourcroy, MD, PhD, American Medical Women’s Association Gilles J. Frydman, Association of Cancer Online Resources Linda Golodner, National Consumer's League Gordon Hay, Life with Cancer William Hersh, MD, Oregon Health Sciences University Douglas Hoey, RPh, MBA, National Community Pharmacists Association Gary Holden, DSW, NYU - School of Social Work Carole Hudgings, PhD, Office of Alternative Medicine, NIH Marco Johnson, Defense Medical Information Management Barbara D. Kerlin+, PhD, Health Information Technology Institute Carol A. Kimbrough, Food and Drug Administration Allan J. Lazar, Agency for Health Care Policy and Research Tom Linden, Author and Internet Medical Consultant Douglas S. Lloyd, MD, MPH, Center for Public Health Practice, HHS David Mark, MD, MPH, Journal of the American Medical Association Anthony J. Mazzaschi, Association of American Medical Colleges Susan Molloy-Hubbard, Nation Cancer Institute Russell E. Morgan, SPRY Foundation Frances M. Perveiler, Joint Commission on Accreditation of Healthcare Organizations Michael Pollack, American Pharmaceutical Association Susan Probyn, American Association of Health Plans John Renner, MD, Consumer Health Information Research Institute David Rodbard, MD, Association of American Medical Colleges Michael J. Rozen, MD, The National Emergency Health Data Center N. Lee Rucker, MSPH, National Council on Patient Information and Education Lucia Ruggiero, PAHO/WHO Virginia Saba, EdD, RN, FAAN, Georgetown University School of Nursing Margaret Savage, MD, MPH, American College of Preventive Medicine, Koop Foundation Samantha Jane Scolamiero, Brain Trust, Inc. Terry Shannon, MPH, Agency for Health Care Policy and Research Lynne Siemers, Washington Hospital Center C. Richard Talley, American Society of Health-System Pharmacists Anne Thurn, PhD, National Cancer Institute James Tierny, NCQA Joseph G. Valentino, JD, United States Pharmacopoeia |
|
|
Copyright © 2007 dmim.com Powered by Engineer Partner The One Stop Outsource
|