Background Previous research indicates that variations in patient populations and practice settings often warrant tailoring of clinical-practice guidelines (CPGs). Despite such needs, local tailoring of CPGs requires experts in medicine and evidence synthesis who may be unavailable in many practice settings. We developed a computer-based system that enables developers and users to create, disseminate, and tailor CPGs, using normative decision models (DMs). Methods We developed alchemist, a web-based system that analyzes a DM, creates a CPG in the form of an annotated algorithm, and displays for the guideline user the optimal strategy. alchemist’s interface enables remote users to tailor the guideline by changing underlying input variables for the guideline and observing the new annotated algorithm that is developed automatically. We used a DM that evaluates strategies for staging nonsmall-cell lung cancer to perform a pilot evaluation of the system. Subjects (n=15) compared the automatically created guideline to published guidelines for staging nonsmall cell cancer and critiqued both using a previously developed instrument that assesses guideline quality. Each subject rated the CPGs for usability, accountability, and accuracy on a scale of 0 (worst) to 2 (best), with greater scores reflecting higher quality. Results The mean overall score for the alchemist CPG was 1.502, compared to the published-CPG score of 0.987 (p = 0.002). The alchemist CPG scores for usability, accountability, and accuracy were 1.683, 1.393, and 1.430. respectively; the published CPG scores were 1.192, 0.941, and 0.830 (each comparison p < 0.05). On a scale of 1 (worst) to 5 (best), usersí mean rating of alchemist's ease of use, usefulness of content, and presentation format was 4.76, 3.98, and 4.64, respectively. Conclusions Our study demonstrates the feasibility of a web-based system that automatically analyzes a DM and creates a CPG as an annotated algorithm. The system enables remote users to develop site-specific CPGs tailored to their practice settings. Our pilot evaluation indicates that guidelines produced by alchemist both meet established criteria for quality and compare favorably to guidelines published by national organizations. The high ratings for usability and usefulness suggest that such computer-based systems can provide an important tool for guideline development.
