When the Agency for Healthcare Quality and Research (AHRQ) shut down its National Guideline Clearinghouse (NGC) in July, medical professionals who relied on the database, hosted at guideline.gov, reacted with alarm. For nearly 20 years, AHRQ’s repository of medical guidelines had served as the gold standard for clinicians, helping guide day to day care by keeping medical professionals informed of the most current research in virtually every field, and helping them evaluate the quality of guidelines published by a range of organizations.

The vast archive of guidelines that the NGC once held — and the concise summaries of sometimes lengthy documents that made it a critical resource — is, for all practical purposes, inaccessible. AHRQ has not allowed any organization to take control of the materials, and has not allocated the resources required to maintain even a static archive of the records created over the past two decades.

The nonprofit that originally created the database under contract with AHRQ, Pennsylvania-based ECRI Institute, recently launched what they hope will resemble a reconstructed NGC. But it’s far from complete and remains a work in progress. Five months after the NGC’s disappearance, professionals who relied on it have to work harder to find the guidelines they need.

According to Valerie King, a professor and Director of Research at the Center for Evidence-based Policy at Oregon Health & Science University, one of the NGC’s greatest strengths was that it centralized guidelines from hundreds of publishers — professional organizations, research institutes and others — in an easily searchable repository. Without that central destination, she and her staff have started going directly to professional organizations and other guideline authors “one by one by one.”

“We’ve just patchworked it together,” King says of the search process they now employ. It’s not always obvious which organizations may have published materials on a particular topic, and there’s a risk of missing important information. It’s also far more time consuming. “We do have other things to do with that time.”

Several organizations pledged to repost the NGC’s content in a static form after its retirement, but without the sophisticated search capability that was one of the site’s strengths. The ECRI Institute, the nonprofit organization that developed the original NGC under contract for AHRQ, was among the first to announce a plan to carry the agency’s work forward with full functionality. Without the previous NGC material, however, the nonprofit is in the unenviable position of duplicating years of work as it also tries to update and add new guidelines and guideline assessments.

The agency maintains that it can’t give ECRI permission to use that content — content created by its own staff under an AHRQ contract — because of concerns about copyright. (Legal experts who previously spoke with Sunlight have cast doubt on that explanation.)

“The guideline developers hold the copyright and granted AHRQ permission to post the guidelines on the NGC per agreement with each guideline developer,” an AHRQ spokesperson said via email. “That permission is not transferable to another organization or entity, such as ECRI.”

After some delays, ECRI launched its Guidelines Trust site in mid-November. But, as ECRI’s Karen Schoelles told us in an interview in late October, it will be a while before the site can match the breadth of the NGC. For the time being, it holds about 450 guideline summaries — the NGC by contrast held more than 1,300 — and only around 200 are in a completed form, including an assessment of reliability, or what ECRI is calling its “TRUST Scorecard.” The scorecard addresses things like conflicts of interest and adherence to Institute of Medicine standards — the same standards relied upon by the NGC — helping users gauge the credibility of guidelines in much the same way the defunct database did.

The process of gathering and summarizing is time consuming, Schoelles said. Without access to the AHRQ materials, her team has to contact guideline publishers individually for permission to summarize their work, or in some cases, re-summarize guidelines they had worked on under their contract with AHRQ. With a team of about 25 people contributing to the project, Schoelles said, they’re hoping to add steadily to their collection of completed materials.

Peace Ossom Williamson, Director of Research Data Services at the University of Texas at Arlington Libraries and an early user of Guidelines Trust, said she found ECRI’s site easy to use and a welcome resource, but noted that it offered much less robust search functions than the NGC. That’s something ECRI will be working on in the coming months, according to Schoelles. Although ECRI previously expected that it would have to set up a fee or subscription model, the site is free thus far; Williamson appreciated that too.

Schoelles said the organization may introduce paid services, in addition to the basic summaries, at some point in the future, but said she hopes to secure funding sources that would forestall that. The organization is also hoping to improve on aspects of the NGC. Better navigational features, planned for the coming months, will help users go directly to information they’re most interested in, like the populations served by a given guideline or conflict of interest disclosures that help users assess reliability.  ECRI also plans to produce more concise summaries than those published on the NGC website by AHRQ.

“We’re calling ours guideline briefs,” Schoelles said, “and over time we’re going to streamline them even more” to offer users a “bite sized” option of a few pages, if users would  prefer those to longer summaries.

AHRQ recently announced that it has hired the MITRE Corporation, a government-chartered nonprofit that often performs research for federal agencies, to conduct a twelve-month study to “help identify new models for disseminating and accessing evidence-based clinical practice guidelines,” in the words of an AHRQ spokesperson. The spokesperson offered few details about what that might entail, and a MITRE representative directed questions back to AHRQ. The MITRE study will also identify “potential partners for making guidelines available,” the spokesperson said, suggesting that other organizations may be contracted for that work, though he denied that AHRQ had been unhappy with ECRI’s performance.

A spokesperson for AHRQ said the MITRE study will cost $400,000. That’s about what it would cost to maintain the NGC as a static, searchable archive for one year, according to an estimate offered by another AHRQ official in an interview with WIP this summer, who estimated that it would cost a “few hundred thousand” dollars annually. Asked about the apparent discrepancy, a spokesperson did not elaborate.

“Shifting priorities as reflected in AHRQ’s FY 2018 budget did not allow for the continuation of the NGC,” the spokesperson said.