To comply with the Open Government Directive, the Defense Department designated three high-value datasets last week, among them a listing of those requesting more transparency from the Pentagon. DoD released details on the 4,000 Freedom of Information requests it has received as well as datasets with information on service members gender and race, U.S. state, and marriage statistics.

Interestingly, the three datasets that Defense said were its high value releases, fulfilling a requirement of the first stage of the directive, were not marked as agency-reported high-value dataset (with an asterisk) on Data.gov. This seems like a small oversight compared to what the Pentagon did not disclose: all the datasets they released are already publicly available on various Defense Department web sites.

While there was some value added by presenting data in an excel format (which was available only in a PDF format before this), having spreadsheets of this old information isnt a huge leap in transparency for a Department that receives more than 20 percent of the U.S. budget. Its relatively easy to convert simple files from PDFs into a text file or spreadsheet. Several free programs do this and optical character recognition software has gotten much better.

But making spreadsheets available does allow users to sort and filter fields and make their own analysis about the data released. In the case of the FOIA log, which includes the identification number, requester information, the date the request was received and closed, and the nature of the request from FY2007-2008, you can find out that the good folks at the National Security Archive made 16 percent of all the 4,300-plus requests, and individuals made up 42 percent.

Among those individuals seeking information was Harmeet Sooden, a Canadian activist who was kidnapped in Baghdad in 2005 along with three others, one of whom, Tom Fox, was subsequently killed. Sooden wrote: I request a copy of all records pertaining to the capture, detention and judicial processing of our alleged kidnappers.

The database also shows that the average time from request date to close date is 38 days, however more than 1,400 entries had no closure date at all. The data also doesnt show whether the requests were closed after being granted or denied, a major omission.

The DoD certainly tracks this information, as they publish an annual FOIA report (sadly only in PDF form) that lists the number of requests granted, partially denied and denied or closed for other reasons such as lack of records, referral to another agency, or lack of specificity. In FY2008 for example, 40 percent of requests were granted in full, 35 percent were closed for other reasons, 21 percent were partially denied, and 4 percent were completely denied. Of the denied and partially-denied requests, 57 percent were due to privacy issues.

The other three DoD datasets released last week, on statistics of service members by race, state location of assigned unit, and marital status by pay grade, were also previously released in an annual report published by Defense on population representation of the military services but in a PDF format. As with the FOIA dataset, having this data in a spreadsheet allows for easy sorting and filtering for analysis on members of the armed services.

When downloading the personnel data on Data.gov, keep in mind that they come from the Defense Manpower Data Center, which is normally only for use by current members of the DoD, Coast Guard and Uniformed Services. The first time you download one of these spreadsheets using Firefox, users will get a warning screen that asks users to accept a certification request with the ominous headline: This Connection is Untrusted. Users must agree to make an exception to download the data. Thankfully you only have to do it once. (A similar warning page is returned by Internet Explorer.)

The marital status report for active duty personnel as of November 2009 actually includes a breakdown by military pay grade and gender, which you can use to find that the higher the pay grade, the fewer the women.  Of the 1.4 million service members on active duty, women make up 15 percent of that total while men makeup 86 percent. At the enlisted level, 14 percent are women, at the officer level 16 percent are women, but at the warrant officer level only 8 percent are women.

The data also shows that 33 percent of active duty service members are single men without children, 6 percent are single women without children, 4 percent are single men with children, 2 percent are single women with children, 6 percent are service members married to another service member, 46 percent are men married to a civilian and 3 percent are women married to a civilian.

The data on the gender and race of service members as of September 2009 includes a header that is coded and unexplained and there is also no technical documentation to accompany this data to explain what the codes mean.

The data on the state location of the 1.3 million reserve military members in the United States shows that as of Nov. 2009, 28 percent are in an unknown location meaning that they are in the Individual Ready Reserve, are on standby reserve and retired reserve without assigned unit states. The state with the largest percent of reserves is Tennessee, which houses 8 percent of all reserves. In contrast, California and Texas each house 4 percent of the nations reserves.
When the data was released last week, the DoD said that it had met the first milestone with the postings on data.gov. It added that there were three officials named as agency designates accountable for the quality of the information that will be publicly disseminated. They are:

* Ms. Elizabeth McGrath, senior high-level official for overall implementation (OSD-DCMO)

* Mr. Mike McCord, senior high-level official to verify accuracy of federal spending data (OSD-C)

* Mr. John Conger, Green Initiatives working group (Assistant Deputy Under Secretary of Defense – Installations & Environment – OSD-AT&L)

If you liked or disliked what you saw from the DoD on data.gov, now you know who to contact.