On March 10, U.S. Chief Information Officer Tony Scott asked the public for feedback on a draft policy for improved access to the source code developed by the federal government. The policy is a commitment that United States made in late 2014, the Second Open Government National Action Plan.

Today, the Sunlight Foundation filed answers to the considerations posed at sourcecode.cio.gov on Github, where more than 100 issues have been filed to date; our contribution follows below. If you feel strongly about this issue, take the time to leave feedback before the comment period closes at midnight tonight.

To what extent is the proposed pilot an effective means to fuel innovation, lower costs, benefit the public, and meet the operational and mission needs of covered agencies?

We believe that making more of the software built by or for the federal government open source will improve how the federal government works and save agencies money.

Open sourcing a given software project enables a wider community of developers to access the project and contribute freely. It enables other agencies or nongovernmental entities to reuse the code written. Internally, this is where the primary cost savings would result. Externally, some projects may have much broader societal relevance, like the NASA Nebula project that grew into OpenStack, or the Accumulo software from the NSA. Selected projects may also be relevant to state and local agencies: The GSA’s Web analytics dashboard was adopted by the city of Philadelphia.

Open source code also provides public benefits with respect to government transparency. The public can not only review the end results of government spending but also learn from and build upon what the government produces.

Government agencies can also meet their mission goals through soliciting distributed community involvement through improving the software, either through direct submissions of code, quality assurance and/or pointing out potential errors or flaws. This last benefit is only realizable to the extent that agencies are willing to implement improvements provided by the community.

Would a different minimum percentage be more or less effective in achieving the goals above?

We believe that a higher minimum would be a stronger lever to encourage adoption.

Would an “open source by default” approach that required all new federal custom code to be released as open source software (with the exception of things like national security) be more or less effective in achieving the goals above?

We believe that the source code for software that is used by a regulator or agency to make determinations about procurement, human resources or the performance of public programs should be open by default. The effectiveness of an “open source by default” policy, however, depends upon the ability of agencies to capitalize on the particular benefits that being open source affords. Poor understanding and poor management can easily reduce the effectiveness of any approach, open or not.

If agencies were to be receptive to community feedback and contributions from the beginning, it could dramatically improve the quality and relevance of the resulting software.

What are the advantages and disadvantages associated with implementing this type of pilot program?

The federal government can compare and contrast the outcomes of different approaches to building, buying and maintaining proprietary software, open source software and software-as-a-service.

To what extent could this policy have an effect on the software development market? For example, could such a policy increase or decrease competition among vendors, dollar amounts bid on federal contracts, or total life-cycle cost to the federal government?

We would expect to see a shift toward contractors that specialize in maintaining and improving systems, as opposed to building new products for each engagement. We also would expect to see increased participation by new, geographically distributed entities in such development and maintenance, including people far outside of Washington.

How could it impact new products being developed or transparency in quality of vendor-produced code?

We would expect to see significant improvements in the transparency and quality of vendor-produced code.

What metrics should be used to determine the impact and effectiveness of the pilot proposed in this draft policy and of an open source policy more generally?

Some metrics could be: magnitudes of actual project timelines (start-to-finish); size/complexity of the finished codebase; ratio of errors to code size/complexity; relative amount of code contributed by the community; and number of errors detected by the community. These metrics should applied across the government to assess software development performance as soon as possible.

What opportunities and challenges exist in Government-wide adoption of an open source policy?

Such a policy could improve extradepartmental, interdepartmental and even intradepartmental awareness of what software is being produced or is available from each agency.

One of the challenges might be maintaining security over sensitive information, such as system authentication credentials. There is potential within an undisciplined process to leak sensitive items in publishing project code.

How broadly should an open source policy apply across the government? Would a focus on particular agencies be more or less effective?

Open source software policy should be governmentwide. The positive and negative experiences of scientific agencies that have adopted or shared open source projects should be given special attention, along with the Department of Defense’s experience. The code, collaboration and outcomes of open source development at 18F in the GSA should also be given more attention.

This policy addresses custom code that is created by federal government employees as well as custom code that is federally procured. To what extent would it be appropriate and desirable for aspects of this draft policy to be applied in the context of federal grants and cooperative agreements?

Code produced for federal grants and agreements should be covered by the policy.

How can the policy achieve its objectives for code that is developed with government funds while at the same time enabling federal agencies to select suitable software solutions on a case-by-case basis to meet the particular operational and mission needs of the agency?

Federal agencies should weigh the choice between purchasing proprietary software and adopting and developing open source code themselves carefully, particularly for new projects. In some cases, working with a closed source project and a given vendor may accomplish mission goals more quickly and effectively.

When a government agency is developing a digital service in-house, government technologists and contractors should have discretion to evaluate and adopt the approaches that best meet their needs.

Wherever possible, agencies should take steps to insulate themselves against the risk posed by developing a proprietary system that results in code that can only be maintained by one vendor, or contracts where the code itself is owned by the vendor instead of the government. Another risk is that of proprietary formats.

Regardless of which vendors are chosen, any agency should still be able to access all of its data even without the services of the vendor. This means that vendors should provide all data in open source formats, or at least in formats that are widely accepted and accessible within the particular domains that those data inhabit.

How should agencies consider factors such as performance, total life-cycle cost of ownership, security and privacy protections, interoperability, ability to share or reuse, resources required to later switch vendors, and availability of support?

We believe that all of these factors should be carefully weighed against one another. Interoperability through open standards, security and the availability of support are critical considerations that will vary depending on the context.