It’s hard to evaluate the effectiveness of our representatives. This is never more true than at the state level. Their work is often done out of the public’s view and legislation is written in frustratingly complex language. The end result is a massive barrier to civic engagement. Most people don’t even know who their representatives are, let alone, their voting record.

In Colorado, the Open Media Foundation (OMF) is striving to overcome the barriers that prevent people from following their representative’s daily work. We have operated the Colorado Channel for the last eight years, attempting to not only to increase the transparency of our capital, but to build literacy in the legislative process overall.

Every day during the Colorado state legislative session, OMF produces a live broadcast aired throughout the state. These sessions are archived – which dates back to 2008 – and available on demand online. The website features a live agenda stamping tool that allows broadcasters to label events like bill readings or votes. Users may then jump between these events in any session.

These tools were so successful for the Colorado state legislature that OMF abstracted them into a software as a service product, the Open Media Project for Governments. OMF now has many Colorado municipal governments and several out of state governmental entities leveraging the service.

For the Open Media Project for Governments, OMF has leveraged YouTube’s live transcription service to make these sessions searchable by keyword. From topics such as gun rights to the legalization of marijuana, the archives may be searched and recordings found where key decisions were made. The search is performed on speech that is often in simpler terms than the complex titles of bills. The archive and searchable transcripts are a huge step forward in connecting constituents to their representatives.

During the 2015 legislative session, we realized that despite the versatility and exhaustiveness of the session archive, it was still difficult to separate meaningful content from the daily monotony. Connecting who was speaking in a clip to other metadata like bill and subject area, seemed like a great avenue to approach. Listing the times a legislator spoke and on what topic would provide a simple entry point.

Going forward it would be simple to add this metadata as the session was recorded. “Senator X spoke about bill Y that day.” The real challenge was finding a way to add this metadata to the robust archive of video the Colorado Channel team had already collected.

The team decided to explore innovative and experimental approaches to make this resource more accessible. Nascent advances in speaker recognition bore some hope for identifying speakers. Unfortunately, nobody on the team was an audio expert or statistician. But we were fortunate enough to win a Prototype Grant from the Knight Foundation to explore this area.

Through the prototype work, we were able to learn about statistical audio analysis and developed a toolset for identifying speakers in video files. Video files have noise and nonspeech segments removed. The toolset then assigns IDs to speakers and allows users to confirm their identity. This drastically reduces the amount of time required to identify when representatives speak in the files. While rudimentary, the toolkit is open source and publicly available. We are excited by the potential of this work both in and out of the civic sector.

OMF synthesized our bill timestamps, the 2015 session speaker timestamps and bill and vote data from the Sunlight Foundation’s Open States project to create She Said, He Said (SSHS). SSHS is a prototype app that allows users to search by legislator, bill or address to find related video and voting records. Visitors can see every time their representative spoke on the floor and what they said.

OMF is still in the process of evaluating the future of SSHS, but we plan on integrating the SSHS toolset into the Colorado Channel and OMP for Government workflows. Through Google and custom analytics, we hope to evaluate the usefulness of this extra metadata in improving overall transparency.

Interested in writing a guest blog for Sunlight? Email us at guestblog@sunlightfoundation.com