Methodology

USASpending.gov contains information about two main types of spending: contracts and direct assistance, which consists of grants, direct payments, loans and loan guarantees. These two types of spending are tracked using two very different reporting systems. One of them is the Federal Awards and Assistance Data System PLUS (FAADS-PLUS) and the other is the Federal Procurement Data System - Next Generation (FPDS-NG). For the purposes of this analysis, we have focused exclusively on FAADS PLUS, the dataset that contains information about direct assistance. The data used for this analysis was obtained in October 2012 from the bulk download section of USASpending.gov.

Data Sources and Limitations

The metrics we calculated relied on data in USASpending.gov and yearly program obligation estimates provided in the Catalogue of Federal Domestic Assistance (CFDA). The CFDA is published online and contains spending estimates for the three most recent fiscal years for all grant and loan programs. It also defines the de facto organizational unit for reporting on direct assistance at the federal level. While budget authorizations and appropriations are grouped together into broad program areas, federal agencies exercise their own discretion in organizing obligations under smaller, more specific, CFDA programs. There is no clear link between budget appropriations for program areas and funds appropriated by agencies to each CFDA program. Because of this, we measure consistency between two different systems for tracking obligations. We are unable to offer an accuracy measurement due to the difficulty of tracing an obligation’s origin to the budget.

The Consistency Metric

The consistency metric is a comparison between two reports of annual federal obligations. The first is an obligation estimate that is reported annually by each program to the Catalogue of Federal Domestic Assistance. The second is an aggregate of obligations reported by each program to USASpending.gov, summed by fiscal year. Obligations are taken from the full CFDA program directory located on the CFDA's FTP server. The difference in the obligations from both sources, weighted as a percentage of the CFDA total, is used as the primary indicator of consistency.

Programs that report obligations in the CFDA but do not have any records in USASpending.gov are designated as nonreporting and are not used when calculating the variance of underreporting or overreporting programs. Programs that report some, but not all, obligations in USASpending.gov are designated as underreporting programs; and those that report more in USASpending.gov than in the CFDA are overreporting programs. Statistics for underreporting and overreporting programs are calculated separately.

Since CFDA program obligations are annual estimates, there are very few programs whose reported CFDA obligation total is exactly equal to the aggregate obligations in USASpending.gov. As a result, the presence of over- or underreporting cannot be assumed to represent an error. The programs with the closest numbers in their reporting generally have a variation of about 25% of the program's total funding. The colors of the consistency metrics indicate the performance of a program or agency with regard to a specific consistency measurement (over-, under- or nonreporting). Programs or agencies that misreport between 0% and 25% of their total program value are considered good and are highlighted in green. Those that misreported between 25% and 50% are considered average and are highlighted in yellow. Those that misreport more than 50% of their total program value are highlighted in red and are considered to be poor at fulfilling their reporting obligations.

Some CFDA programs have had manual corrections made to their CFDA obligation amounts. In cases where we observed obvious order of magnitude errors (e.g. reporting in millions instead billions) we corrected the obligation to allow for a more meaningful analysis. All corrections are listed on the corrections page with the original and corrected obligations figures. Programs wishing to update their CFDA obligation amount may contact us directly using the contact form.

Only certain types of programs are present in this analysis. Many CFDA programs provide non-financial services, which may include dissemination of information, investigation of complaints, and advisory or counseling services. These programs may report obligations in the CFDA obligation field, but they are almost always designated as salaries and are ignored for the sake of this analysis. Only programs with the following assistance types are included in our evaluation of consistency:

  1. Formula Grants
  2. Project Grants
  3. Direct Payments for a Specified Use
  4. Direct Payments with Unrestricted Use
  5. Direct Loans
  6. Guaranteed/Insured Loans
  7. Insurance

Note that in the results, programs that provide mixed assistance, relying on both grants and loans, will be split and have a listing for both grant reporting and loan/loan guarantee reporting. This is because the reporting guidances on loan obligations are not as concrete as those for grants; previous research on this topic has led us to believe they should be considered separately.

Some grant programs included in our analysis designate certain obligations reported in the CFDA as salaries. These obligations are ignored in this analysis.

The Federal Funding and Accountability and Transparency Act (FFATA) does not require that grants to individual recipients be reported. As a result, many programs have been designated as nonreporting, such as Social Security payments, Medicare insurance, direct small business loans and unemployment insurance payments. While these programs are still in compliance with the law, their sheer magnitude warrants a look at what portion of federal spending is not being reported.

The Completeness Metric

To measure the completeness of the records in USASpending.gov, each record was systematically checked to see if it contained well-formatted data for 18 specific fields required by OMB Memorandum M-09-19. Some fields were only checked to see if they contained any value. Other fields were checked for formatting and validity. The fields checked for content include:

  1. Recipient Type
  2. Recipient Name
  3. Recipient State Code
  4. Recipient County Code
  5. Recipient County Name
  6. Recipient City Code
  7. Recipient City Name
  8. Recipient Congressional District
  9. Record Type
  10. Action Type
  11. Assistance Type
  12. Federal Award ID
  13. Principal Place of Performance County Code
  14. Principal Place of Performance State Code
  15. Federal Agency Code
  16. Federal Funding Amount

The obligation action date field was checked for content and checked for proper date formatting. The CFDA program number was also checked for a properly formatted CFDA number and excluded common placeholders that appear in the data (such as 00.000 and MU.LTI). For each record that failed to complete one or more of these fields, the total obligated funds for that record were added to a total of obligations failing to meet the reporting standard. Programs with less than 50% of their obligations being reported completely are considered poorly performing programs and are designated in red on the scorecard. All others are considered well performing and are shown in green.

The Timeliness Metric

The timeliness metric measures the time between the obligation date and the date it was reported to USASpending.gov. The obligation date is a mandated field. It is also a FFATA requirement that all obligations be reported to USASpending.gov within 30 days. The reported date appears in the name of the data files that individual agencies submit to USASpending.gov. These files, which contain original, agency-submitted data, are posted on the USASpending.gov site. We parsed the date from the file name and compared it with the obligation date of the transactions present in that particular file. Transactions that appear in multiple files, either as corrections or updates, are only counted the first time they appear. Because not all transactions are available in the original, agency submitted data, our timeliness metric evaluates a sample of the USASpending.gov data and cannot be considered to be comprehensive. Transactions that do not appear in any of the available agency files are not measured. If the difference between the obligated date and the reported date is greater than 45 days, the obligation for that transaction is counted as late and added to a total of late obligations. Programs that have less than 50% of their obligations reported within 45 days are considered poorly performing programs and are highlighted in red. All others are considered well performing and are highlighted in green.

For this year's report, here are our sample sizes for each year, in dollars.

Fiscal YearSpending in SampleTotal Spending%
2009$440,067,148,059$1,625,257,115,43927%
2010$332,087,816,641$1,437,032,472,47423%
2011$306,287,843,420$1,639,826,026,45219%

Loans

Federal loans and loan guarantees are outliers among the data examined by Clearspending. A significant portion of federal assistance is provided through loans, loan guarantees and other types of insurance. These funds differ from grants in that they only cost taxpayers in the case of a default or insurance payout. Because of this their cost depends on the riskiness of the lending and the expected rate of default. Reporting systems (including USASpending.gov) track what is know as the "subsidy cost," an estimate of potential risk based on the overall performance of the loan or guarantee program. The total value of the loans and guarantees is also reported but from a budgetary perspective the projected risk determines the "cost" of the loan program.

The loan data in USASpending.gov is of substantially lower quality than the grant data. It is often the case that only defaults are reported despite the FFATA’s requirement that all loans be reported. Because loan data is sparse, it is not included in our timeliness metric and is considered separately in our consistency metric. Aggregate figures for the consistency measurement do not include loans and loan guarantees, though these figures are available in the program detail view. Since only the face value of the loan is reliably available in both CFDA and USASpending.gov we use this figure in our analysis and not the subsidy cost.

Why doesn't Clearspending Analyze Contracts?

Clearspending uses the Catalog of Federal Domestic Assistance (CFDA) as a comparison point for the obligations in USASpending.gov. The CFDA includes only direct assistance, such as grants and loans. Contracts are not reported to the CFDA. Because we do not have another comparison point for contract obligations, we have not ported this analysis to the procurement space. We do however, have a pending Freedom of Information Act request for this data and hope to post an analysis soon.

How does the federal government report its spending?

The flow of federal spending reporting can get complicated. We've tried to simplify it for you with a step-by-step visualization.