As of midnight last night, candidates for federal office were to have filed their campaign finance disclosure reports with the Federal Election Commission. These reports contain crucial information that lets voters know which special interests, big-money lobbyists or out-of-state donors may be funding a candidate’s campaign. The reports are supposed to be public, but if you try to find Senate candidates’ reports today, you will be out of luck. Why? Because the Senate has exempted itself from filing directly with the FEC, instead using the Secretary of the Senate as an intermediary. And instead of filing their reports electronically, like House candidates and presidential candidates have been doing for years, Senators and Senate candidates mail or hand-deliver paper printouts of their electronically generated reports.

After receiving the reports, the Secretary of the Senate must scan, page by painstaking page, thousands of pages of campaign finance reports before transmitting them to the FEC. It may be days or weeks before the FEC receives the reports—longer for the ones that are mailed rather than hand delivered, as the mailed reports don’t even arrive at the Secretary of the Senate’s office until they have been processed off site.

But wait, there’s more. After it receives the scanned documents, the FEC must then spend about $450,000 in taxpayer dollars and untold hours having the records typed in, line-by-line, to the FEC’s databases. It will take at least three weeks before the information is publicly available—longer in the middle of a busy election season. The process isn’t just inefficient. It denies citizens timely access to information that can help shape and inform their opinions about their candidates and elected officials.

Senator Tester and Cochran have repeatedly introduced legislation to streamline the process and make electronic filing mandatory. This Congress, a bipartisan group of 30 senators have cosponsored S. 375, the Senate Campaign Disclosure Parity Act, with many others voicing support for the measure.

Despite its overwhelming support, the bill has not become law because some in the Senate have chosen to make it a political pawn. That is why we urge every Senator who supports transparency and government efficiency, as well as every one who opposes government waste, to cosponsor the bill. Overwhelming, demonstrated support may be the best chance this common sense piece of legislation to pass.

Every day in Washington, DC, thousands of government officials are at work writing, revising, and publishing regulations affecting everything from food safety, to personal finances, to the environment and beyond.

But as our Reporting team reports today, with the help of data crunched by Sunlight Labs, the rulemaking process, though ostensibly a public one that anyone can participate in, is largely a rarified one. But it doesn’t have to be that way.

This year 89% percent of the regulations that drew at least one comment received 100 or fewer, according to an analysis of information available on regulations.gov -- the government's one-stop hub for information about the regulatory process -- that we made using Docket Wrench, a new tool in development by our Labs team. Only 102 regulatory proposals drew 1,000 comments or more.

In contrast, the regulations that drew the most participation were largely concerned with controversial matters that are patrolled by groups with well organized constituencies eager to act when prompted by an action alert. For example, the regulation that received the most comments—more than 63,000—concerned contraceptive benefits under the Affordable Care Act. Much of the action appeared to be organized by Catholic groups opposed on religious grounds.

We are hoping that Sunlight’s Docket Wrench, which is based on data collected by regulations.gov, will draw more people in to the process. Our Scout tool also offers alerts on regulations as published in the Federal Register.

However, our tools can only be as good as the data underlying them. While a long list of federal agencies participate in regulations.gov, technically independent agencies are not required to do so. These include such high profile agencies as the Securities and Exchange Commission, the Federal Communications Commission, and the Federal Elections Commission. While these agencies individually may post the comments they receive on pending regulations, very few have an “API” that allows developers to pull the data into a new interface as Labs does with Docket Wrench. Without the participation of these agencies in regulations.gov, our new Docket Wrench tool will be limited.

In addition, the data collected at regulations.gov is often imperfect. There’s little standardization of fields, such as “employer,” that would allow a user to easily search for a particular entity. When our reporting group sought to find out, before the election, where the U.S. Chamber of Commerce was putting its regulatory efforts, we found ourselves limited because the data would not easily allow a thorough search on the name of the organization. Ideally each organization filing a comment should use unique corporate identifiers that would help make such searches comprehensive.

It should be mandatory that all of the federal agencies -- including the independent ones -- should participate in regulations.gov. Of course the agencies should receive the funding necessary to make this happen. If we can improve the data, that’s one step closer to improving participation.

With the end of the 2012 election season, so too comes the conclusion of a seemingly infinite number of campaign advertisements. Of course, there is a finite number of political ads that ran this cycle and we at Ad Hawk have placed that figure at more than 4,000. That number is based on our work gathering and identifying the ads in order to help the public learn more about the sources of funding behind them.

The Ad Hawk project collected and indexed advertisements released by campaigns in order to inform voters about the ads’ sponsors and funding sources. As a result, we have amassed a comprehensive and searchable database of this year's presidential, congressional and third-party advertisements focused on federal issues and candidates.

Every day, Ad Hawk scans more than 300 of public feeds associated with campaigns and committees. Each time one of those organizations posts a new video, we determine if it is an ad, and if so, we mark it in our database. So far, we’ve considered nearly 8,000 videos, marking more than half as ads. As the election approaches, we’ve found that organizations are releasing ads at an accelerating rate. It’s enough to keep us from turning on the television in our free time.

We knew we were collecting a lot, but one question still nagged at us: were we collecting enough? It's difficult to estimate how extensive our coverage is, since there is no publicly available record of the advertisements that are published during a campaign season. We have noticed, however, that the Ad Hawk system's collection rate seems to be consistent with the recent explosion in outside spending in the push before election day.

Using an environment for statistical computing called “R,” we performed a cross-correlation (R's ccf() function) of ads released and outside spending to find out exactly how well the correlate. We also checked to see if there is a lag between spending and ad releases, to account for the possibility that ads come out days after funds are spent. As it turned out, though, spending and ad releases had nearly no lag difference.

This seems to support the notion that money spent by outside groups is immediately seen in the form of new messages being pushed to the public. This, we believe, demonstrates the importance of projects like Ad Hawk and Political Ad Sleuth, which try to show the money trail between advertisements and the people, organizations, and interests behind them. All the more reason, we would argue, for the FCC to require records of advertising contracts signed between media outlets and committees immediately available to the public online and in machine readable format.

Data and R scripts here

Transparency advocates suffered a setback today, when the US Court of Appeals for the D.C. Circuit in Van Hollen v. FEC overturned an earlier ruling that favored transparency.

At issue is a provision of the Bipartisan Campaign Reform Act (also known as McCain/Feingold) that required disclosure of the names and addresses of every individual who contributed $1,000 or more to an individual or group that made an electioneering communication.

When it drafted its regulations on that provision of the law, the FEC took what was, to us anyway, clear disclosure language and narrowed it so that only the names and addresses of donors who specifically made contributions to the organization “for the purpose of furthering electioneering communications” would have to be disclosed. That opened a gaping loophole that prevented any disclosure of dark money contributions to groups airing political ads.

A lower court agreed that the language in BICRA was unambiguous and held that the FEC was wrong when it narrowed the situations to which disclosure applied. Unfortunately, the appellate court overturned the lower court’s ruling. It held that the statute’s language was less than clear, opening up the possibility that the FEC’s regulations might have been reasonable. The issue isn’t over, as the appellate court remanded the case back to the district court for further consideration.

The district court ruled in favor of transparency once, and we hope it will do so again. Congress made clear that the public deserves to know who is paying for its elections and it is not up to the FEC to obfuscate that information.

Harried campaign finance dorks who spend the 20th of each month huddled over a terminal window hoping the next 43,558-page election disclosure report doesn't break their parsing code have one less thing to worry about going wrong: today the Federal Election Commission announced they're providing an RSS feed of the latest filings.

It's a small step forward--and, full disclosure--one that the Sunlight Foundation asked for a few months ago at a public meeting. But it's important nonetheless because it acknowledges that time-sensitive data needs to be presented in a convenient machine readable format. Until now, real-time campaign finance sites (like Follow the Unlimited Money) had to regularly ignore the FEC's policy against web crawlers querying their campaign finance search and parse a page of human readable results. A few overzealous bots from a variety of news organizations were blocked in recent months for reading data too quickly. And the reliance on human-readable results meant that any formatting change might break a campaign finance scraper.

The RSS feed is a modest step, but it's certainly nice to see an agency going out of their way by not only hosting a forum on public data, but actually implementing some of the suggestions.

For most Americans, the incessant barrage of political ads on TV are still a few months away. But in Wisconsin, the electioneering is well underway with exorbitant fundraising spent on political ads in the recall election of Governor Scott Walker scheduled for next Tuesday.

In the post-Citizens United era of unlimited fundraising (and spending) by super PACs, the elections in Wisconsin are just a preview of what is to come this fall. While much of this ‘dark money’ is hidden from detection by the Federal Election Commission there is a one place where they have to leave a paper trail: at the TV stations where they buy their political ads. For the first time, the Federal Communications Commission (FCC) has ruled to require television stations in the top 50 media markets to put their public and political files online. These files contain information on political ad spending which are crucial to the public interest, and provide a glimpse into who is trying to influence our vote.

Not surprisingly, the broadcasters have filed a lawsuit against the disclosure requirement. Even if the broadcasters are forced to comply with the ruling, the top 50 media markets still only cover a fraction of the swing states where the majority of political ad spending will occur.

That is why Sunlight Foundation is working with Free Press, New America Foundation and other coalition members to recruit volunteers, journalists and allies across the country to inspect political files at TV stations and post our findings online. With the national attention (and fundraising dollars) focused on Wisconsin, we are starting here to shine light on political ad spending by unlocking the political files at the broadcast stations. But we need your help!

Sign up to become a political ad sleuth and we will send you all the information you need to inspect the files at your local station.

Yesterday, a three-judge panel of the D.C. Circuit Court of Appeals demonstrated that it gets the need for greater transparency of money in politics. In a victory for transparency advocates, the court denied a motion to stay a lower court ruling that requires comprehensive disclosure of “electioneering communications.” In non-lawyer language, that means that nonprofit groups like the Chamber of Commerce, Crossroads GPS and Priorities USA that want to run political ads right before an election will have to disclose their donors.

The case is Van Hollen v. FEC and the issue is whether the FEC properly narrowed its own disclosure rules, leaving donors in the dark about who is funding campaign ads. Under the Bipartisan Campaign Reform Act, (also known as the McCain-Feingold law) certain disclosure requirements applied to organizations making “electioneering communications”—ads that refer to a candidate and run within 30 days of a primary election or 60 days of a general. Under the law, the group running the ad must identify any person, corporation or labor union that contributed $1000 to the organization. (It should be noted that the old rule also permits the group to set up a separate segregated bank account to make electioneering communications. If it does that, only donors to that account need to be disclosed. This is not unlike provisions in the DISCLOSE Act.)

The FEC gutted the old rule, limiting disclosure to only those contributions made “for the purpose of furthering electioneering communications.” In other words, the Commission created a gaping disclosure loophole that meant donors behind some of the nastiest political ads that flood the airwaves would remain secret. Van Hollen sued, and the lower court agreed with him that the original rule should remain in place. By that time, however, two other groups joined the case as “intervenors” and urged the court to “stay”—or hold off—enforcing the decision that would require immediate disclosure.

The panel of D.C. Court of Appeals agreed with the lower court and refused to grant the stay, recognizing the importance of disclosure. The lower court judge based her decision in part on the Supreme Court’s observation that, “the disclosure requirements serve an important public function because they ‘provide the electorate with information about the sources of election-related spending’ and help citizens ‘make informed choices in the political marketplace.’ (Citations omitted) The result of the ruling is that the original, broad disclosure requirement is in effect and groups should be on notice that they are required to disclose their donors if they plan to run electioneering communications.

But, don’t expect to see a database of donors pop up on the FEC’s website tomorrow. Groups will try to avoid disclosure and have already begun to craft ads they think won’t trigger disclosure requirements. (Think ads that say “Congress” instead of “Congressman Smith.”) The groups will also ask the FEC to provide “Advisory Opinions” (AOs) as to whether disclosure rules apply in particular circumstances. How the FEC responds to these requests for AOs may signal how serious the commission is about enforcing the new, old rule. The FEC, with its even split between Democrats and Republicans may well deadlock rather than move forward on disclosure and enforcement.

The judicial system is likely not done with the case either. It is possible that the decision denying the stay will be appealed, in which case we would hope the judicial system continues to favor disclosure over secrecy. Yesterday's decision gives us reason to believe that the judicial branch will continue to trend toward greater transparency.