Data and Research Intern Alex Engler wrote this post.

The regulatory process is a politically charged arena, where the perception of over-regulating, or not regulating enough, can become a political liability.  Whether it’s Tom Donohue of the Chamber of Commerce warning of the oncoming “tsunami” of regulations from President Obama, or the National Resource Defense Council striking at the Bush administration for an “assault on our clean air protections,” there can be no doubt that the perceived level of regulation matters.

However, one should look skeptically towards assertions about the degree of rulemaking, especially when those assertions include specific numbers.  These claims are often based on research that can be structured so as to intentionally mislead. And beyond the political motivation in how one measures regulatory action, there are also many opportunities for genuine methodological error.

Despite recent improvements, the presentation of information on federal regulations is currently very convoluted: a handful of different government websites present overlapping and incomplete pictures of the regulatory process.  Sunlight Labs has been trudging through four of these sites for months, and we’d like to share what we’ve learned.

You should know the basics of federal regulations going into this: after Congress passes a law, the executive branch implements it through regulations, which are much more specific. The regulatory process is also intended to be public and participatory; almost all regulations allow for a period of public comments between when they are proposed and when they are finalized. (If you’re looking for more information, check out the EPA’s basic primer, or OMB’s Reg Map for a more detailed examination). In practice, however, this process can be difficult to follow.  Consequently, it is important to both know what sources are available and to understand their individual strengths and weaknesses.

FederalRegister.gov

For regulatory research, we strongly recommend starting at federalregister.gov (FR.gov), which offers every proposed and final regulation in a rich presentation.  FR.gov, which is run by the Office of the Federal Register, also provides advanced search features and makes available all of its data in structured form (JSON).

The site’s well-designed presentation and search make FR.gov the best source for looking up individual regulations, and FR.gov’s data is also the best source for raw numbers of regulations.

However, even if you are using federalregister.gov to look at a comprehensive list of proposed and final regulations, finding useful metrics can prove to be very challenging.  You cannot simply count all documents listed as a “Rule” (rather than “Proposed Rule” or “Notice”) and determine that this accurately represents the total number of regulations enacted.  The function of these documents varies dramatically.

For example, take the Amendment of Class D and Class E Airspace; Lakehurst, NJ.  This document is referred to as “Final Rule; Technical Amendment.”, which is a somewhat ambiguous classification.  Upon reading the summary of the regulation, it becomes very clear that this “rule” from the Federal Aviation Administration does nothing except change the name of an airport.

Examining the Establishment of Area Navigation (RNAV) Roytes; Southwestern United States also illustrates this point.  This document, referred to as a “Final Rule; Correction.”,  changes incorrect latitude/longitude coordinates in a previous rule.

Any generic count of “Rules” on federalregister.gov would treat both of these documents as equivalent to the recent blockbuster regulation that defines what financial instruments in the 650 trillion dollar swaps market will face new rules under the 2010 Dodd-Frank financial reform law.  Other documents listed under “Rules” include requests for comment, extensions of compliance dates, and even temporary rules.  This makes simply counting the total number of “rules” a fairly useless endeavor.

Reginfo.gov

So despite the comprehensive nature of FR.gov, we are left without reasonable measures of regulatory action.  This is where reginfo.gov becomes very useful.  Reginfo.gov has information for all regulations reviewed by the Office of Information and Regulatory Affairs (OIRA), which maintains this regulatory review dashboard.  Reginfo.gov provides information on regulations currently under review, and breaks down these documents by agency and type.  This site also provides historical reports of all reviewed regulations that one can filter by agency and year, as well as aggregated counts of all reviewed regulations that can be searched for by specified dates.

These are both useful tools, but once again, some context is necessary.  OIRA’s primary mission is regulatory analysis, which it reserves for regulations designated as “significant.”  There are essentially two ways to meet the criteria for this designation: economic significance and other significance.  Economic significance is defined as having an annual effect on the economy of $100 million or more, or otherwise “adversely affect[ing]” the economy. It’s worth noting that the $100 million bar is adjusted for inflation and as of 2008 the qualifying amount was approximately $133 million. The other criteria are even more vague; if there is a clear-cut process to determine what regulations qualify as significant under them, we are unaware of it.

This is all to say that what regulations OIRA reviews is essentially a product of what OIRA, as an extension of the White House, feels capable of and interested in reviewing – with the exception that regulations of economic significance must be reviewed.  During the Obama Administration, for example, OIRA has formally reviewed just over 16% of all final rules, which illustrates how relying on reginfo.gov tallies for overall assessments of government rulemaking activities can be misleading.

Using reginfo.gov also presents a second, more familiar problem.  Not all significant regulations are equally significant. Some regulations may have costs and benefits that just barely hit the $100 million dollar threshold or don’t even hit this threshold at all, while others drastically exceed this mark.

When the Obama administration used selective metrics to say that they had regulated less than their predecessors, they were criticized for cherry picking by Factcheck.org — appropriately so, in our view. Even though Obama’s OIRA had reviewed fewer rules than George W. Bush’s over the same time period, the regulations coming from the federal agencies under Obama were far more likely to be “economically significant”. Factcheck.org’s research points out that the total economic impact of regulations from the Obama administration was also much higher than the economic impact (both positive and negative) during the equivalent period under President Bush.

Equally important, in addition to the varying importance of regulations, is that there is clearly some political attention paid to OIRA reviews, which may act as a bias on the total number of reviews itself. This paper from the Harvard Law Review also details how agencies and their administrators have incentives to avoid (and occasionally seek) OIRA review.

Regulations.gov

Next we examine regulations.gov, which is managed by the eRulemaking Program Management Office (an office housed in the EPA through an accident of history).  Regulations.gov lists proposed and final regulations for every federal agency, but the primary purpose of the site is to act as a centralized location for public comments.  According to their website, 176 agencies, including all of those at the Cabinet level, cooperate with regulations.gov by receiving comments from and posting comments to this single platform.

There are 127 non-participating agencies that may receive comments through this site, but do not post any information.  Although many of the larger and more prominent agencies do work with regulations.gov, it would be a mistake to ignore the agencies that are omitted.  Major agencies like the Federal Reserve, FEC, FCC, CIA, SEC, CFTC, GAO and NEC, among others, are non-participating.  Regulatory documents for non-participating agencies are still available on regulations.gov; it is just the public comments that are missing.

Regulations.gov is also limited by its very sparse aggregated data: the site only publishes the top five agencies and top five documents in terms of comments posted, as well as running year to date totals of each type of document.  You can learn, for instance, that regulations.gov has received over 250,000 publicly submitted comments this year. Beyond this, the advanced search function allows for some estimation of regulations.  Searching over specified agency, dates, affected sector, and document type can provide for a general idea of how many and what kind of regulations have been enacted.

gpo.gov

The Government Printing Office’s (GPO) website makes available every complete issue of the Federal Register from 1994 to today. For issues before 1994, you need access to HeinOnline or LexisNexis – both of which have electronic versions of the Federal Register going back to its inception in 1936.

The Federal Register, published by the National Archives and Records Administration, brands itself as the “Daily Journal of the United States Government.”  Every issue includes that day’s proposed rules, final rules and public notices from all federal agencies, in addition to all Presidential actions, such as executive orders.  GPO’s site offers each complete issue in XML or PDF, or, alternatively, split up by agency and presented as PDF or text. However, gpo.gov does not offer any search function, so unless you know exactly what you’re looking for and when it was published in the Federal Register, this is hardly a useful tool.

It is worth mentioning that GPO also makes available the annual edition of the Code of Federal Regulations (CFR) – which is the codification of all current permanent regulations, just as the United States Code is a codification of current law.

Which of these source you should use depends on what you’re trying to do.  If you want raw unprocessed copies of the Federal Register, GPO can help.  For anything related to comments, look to regulations.gov.  In terms of measuring the total number of regulations – regardless of content – then you should be analyzing the data on FederalRegister.gov.  Lastly, if you want to perform analysis that sorts out more meaningful regulations, reginfo.gov is a good starting point.

However, it cannot be overstated that all of these sources provide imperfect metrics.  Simply finding a tally of regulations enacted cannot truly provide a useful picture of an administration’s regulatory action.

*A previous version of this article incorrectly stated that the $100 million qualification for economic significant regulations is not adjusted for inflation.