The Obama administration unveiled its “IT Dashboard” last week—to significant fanfare, including a write-up in The New York Times—as a symbol of its commitment to openness and understanding of technology.

But is it more than a flashy gizmo?

The Web site is only the first step in a broader effort to make government contracting data available online, and the tech-minded workers tasked with designing a system made the natural choice to start with what they know best: IT contracts.

And as an early (and rushed) prototype, we won’t fault it for the bugs and crashes that were prevalent for the first several days, most of which appear to be fixed.

But if the challenge for central agencies is coordinating across the wide federal bureaucracy, the sometimes narrow focus of technophiles may be the site’s shortcoming. There is little explanatory language placing information in context—understandable given that it’s being drawn from many different departments.

At the same time, by relying so heavily on Web design bells and whistles in an attempt to be user-friendly, the site risks becoming eye candy, bloated with colorful animated charts and lacking when it comes to methodology information. To the casual observer, the Dashboard might seem to be the epitome of “Web 2.0”: interactive, loaded with fancy plug-ins such as Flash, and sporting a crisp, modern design.

That interface can become a barrier to, not a conduit for, the information itself, however. The gauge-style graphic isn’t really suited for displaying cost information, for example; it’s unclear how this metric is assessed: Is a contractor penalized for coming it at under projected cost? Much of the data that drives the IT Dashboard is not available in raw form, either as a live feed from the site or downloadable. The three datasets pointed to by Data.gov, the new Web site that aims to catalog government data, are of little interest, as are the “data feeds” provided by the Dashboard.

The goal is to disseminate information to the citizenry, including technological neophytes, so the urge to simplify is understandable: a massive government database is overwhelming and incomprehensible to most. The tricky line being walked is that an over-reliance on summarization means the government is giving us analysis rather than information itself.

With the raw data, I could conduct my own analysis, as complex or basic as I liked. It was clear that there was a large set of data driving the site, popping up in selective tidbits as I clicked through a confusing and poorly documented maze, but it seemed to be almost purposely withheld.

So I wrote a computer program to “scrape” the site, expending a couple hours to literally reverse what Dashboard programmers had spent six weeks to do: Turn an expansive and informative, if dry, database into a graphical interface.

Once I had my overwhelming but potentially informative database, I could run statistics such as average ratings (the average cost rating is 6.1, while the average timeliness rating is 2.9).

But quickly, I ran into the same problem that has so often riddled government data: Poor quality. The name of the government contractor was missing 78 percent of the time. And contracting companies had no unique identifier, meaning a person doing research on, say, Booz Allen Hamilton, is out of luck: That company appears variously as BOOZ ALLEN HAMILTON INCORPORATED (3626), BOOZ ALLEN & HAMILTON and BOOZ ALLEN HAMILTON INC.

Based on IT Dashboard data, Booz Allen has 13 contracts with the government, more than any other firm. But a search for contracts with the firm on USASpending.gov returns 8,900 contacts.

The other obvious use of the data was to examine which companies had the most contracts by dollar. North West Geomatics Limited appeared to top that list with… $4.5 trillion dollars, or more than one-third the GDP of the United States. But click another link on that page and you’re sent to a page on USASpending.gov showing the contract is closer to $4.5 million.

Even a one-character error—a B for billion instead of M for million—is a big deal that throws any other analysis into further question. If this data isn’t clean and reliable, it’s useless to reporters, government workers and others who require an authoritative source.

These are administrative shortcomings, not programming errors—largely an absence of communication between various government agencies. But a “high-tech” display of low-quality data is vastly less preferable than a low-tech incarnation of high-quality data.

By the metrics of the Web designers who built the IT Dashboard, it may be successful site, laying out information in a way that’s easy to navigate. But to the journalists and policy wonks to whom information means people and decisions, not bits and bytes, that’s the easy part. If the agencies don’t report numbers correctly and the Dashboard team doesn’t run basic quality checks, update regularly and—most importantly, tell us what the numbers mean—the Dashboard application could be a window to nowhere. The bottom line is that visualizations might enhance, but do not replace, raw data, and that increasingly making a useful modern Web site is, perhaps paradoxically, about a lot more than technology.

Download the complete data Sunlight Labs scraped from IT Dashboard: projects, contracts, ratings, costs.