When it comes to the data in USASpending.gov, we’ve been pretty vocal about the quality and usefulness of it. Last Fall we undertook a project to assess the quality of the data based on three metrics: consistency, completeness and timeliness. The project, Clearspending.com, garnered attention from politicians and watchdogs for its shocking top line number — $1.3 trillion in broken spending reporting. We’ve often looked at the macro perspective with this data, but what if we followed the transactions of a single program? Would we be able to understand and follow the data easily and provide citizen oversight, as was the intent of the legislation behind USASpending.gov? In the course of updating Clearspending with new data, we tried to do just that.

For instance, what if you wanted to look at spending for the CHIP and Medicaid programs, in fiscal year 2010, to the state of California? Agencies submit data files to USASpending.gov, which are then validated and displayed on the site. Agencies can also re-submit transactions at a later date with a different action type, such as to indicate a correction to the funding amount. To analyze late spending, we go through the raw agency submission files to determine the first date of reporting for an award. Let’s start with the first transactions for award CA20103. On April 5th, 2010, the Department of Health and Human Services (HHS) submitted a file with these transactions:

Action type ‘A’ indicates the initial award. Fifteen days later, on April 20th, they followed up with a continuing assistance transaction (action type ‘B’) to adjust their spending downward:

After the adjustment on April 20th, this is $9.35 billion. In FY2010, the entire CHIP program had $12.5 billion in appropriations, according to the Catalog of Federal Domestic Assistance (CFDA). California is very large, but surely it isn’t getting 75% of all CHIP appropriations, is it? I wouldn’t bet on it, but on June 21st HHS doubled down by submitting these transactions:

The funding amounts are identical. However the FAIN (a.k.a. award ID) has lost it’s -T21 suffix (CHIP is authorized by title 21 of the Social Security Act). Similarly the program description for the earlier transactions was “MEDICAID ENTITLEMENT FOR CALIFORNIA – FY 2010 QUARTER 3 – T21”. These new transactions have a description of “MEDICAID ENTITLEMENT FOR CALIFORNIA – FY 2010 QUARTER 3 – T19”. Title 19 of the Social Security Act authorizes the Medicaid Assistance program, which had $275 billion in appropriations for FY2010. $9.35 billion is 3.4% the $275 billion appropriated for Medicaid Assistance. That’s a far more reasonable portion to be allocated to California. It appears HHS realized their mistake but in their attempt to correct it they made another mistake; they forgot to update the CFDA number, so these transactions still pertain to the CHIP program. The effect is that they mis-reported this spending twice. At the end of June, the spending reports would have shown California receiving 150% of the funds appropriated in FY2010 for the CHIP program.

Thankfully HHS realized their error in failing to change the CFDA number in their June 21st submission. On July 20th they submitted these transactions:

The first 4 transactions report spending for the Medicaid Assistance program (CFDA 93.778) while the last 4 transactions delete the previously, erroneously reported spending for the CHIP program (note the ‘D’ in the correction indicator column). Unfortunately it took HHS until September 20th to fix their original reporting error, which was for award CA20103-T21. Here are the now-familiar delete transactions:

So let’s recap the timeline. In April they mistakenly reported Medicaid spending as CHIP spending. In June they tried to correct this by changing the description and the award ID but they failed to change the CFDA program number and they failed to delete the previous transactions. In July they fixed the CFDA program number. Finally, in September they deleted the original erroneous transactions.

By the end of the fiscal year in September, the data was nearly accurate (in November, they made normal year-end adjustments). Today, USASpending will not show any CA20103 spending under the CHIP program. However, the data was inaccurate between April 5th and September 20th. That is nearly seven months, over half of the fiscal year. The value of frequent reporting is severely diminished if the accuracy of the data cannot be relied upon. How many misconceptions were formed by people viewing the data during that time? How many decisions were affected?

The law requires agencies to report transactions within 30 days. HHS originally reported this spending on April 5th, just 4 days after the obligation date of April 1st. By doing so they met the letter of the law requiring them to report their spending in a timely fashion. However they failed to report it accurately. Timely reporting of inaccurate data does not satisfy the spirit of the reporting requirements. There is a natural tension between timeliness and accuracy. Some situations will surely arise where it’s infeasible for an agency to meet both goals. But in this case HHS had another 25 days before they had to report this spending. Why didn’t they slow down and get it right the first time?

In addition to diminishing the viability of citizen oversight, this causes conceptual difficulties when trying to measure the timeliness of an agency’s reporting. Award IDs are meant to identify spending. Since the original reporting and the corrections used different award IDs, our analysis considers this spending twice. While the initial reporting was within the 30 day threshold required by law, the corrections were late, even beyond the 45 day threshold we use in our methodology. Thus we consider the first report timely and the second tardy. This is a fair and useful measurement, but it makes calculating the percentage of their spending reported late more complicated. Counting the spending twice inflates the denominator, giving the appearance that the agency is reporting a higher proportion of their spending in a timely manner than they really are, only because they erroneously reported more spending than they had the authority to spend. This is one example of why timeliness cannot be considered in isolation; you must consider it in concert with completeness and consistency. This is exactly what we we do with our Clearspending project, which you can use to find many other examples of poor reporting on other programs and by other agencies.