ACCURATE Comments on the VSTCP Manual, v2.0
By Joseph Hall on 31 Jan 2011.
The Election Assistance Commission put the 2nd version of their Voting System Testing and Certification Program (VSTCP) Manual out for comment in late November of last year. Today was the due date for comments and ACCURATE submitted a public comment lauding the EAC for how the Testing and Certification Program and the associated Manual have evolved in positive directions. You can read ACCURATE’s commentary here (PDF).
We offered a few suggestions for further improving v2.0 of the VSTCP Manual:
- The term “malfunction” should be explicitly defined in the VSTCP Manual and the conditions for triggering manufacturer reporting of malfunctions to EAC better specified along with a more detailed set of reporting requirements. We endorsed the recommendations of the Brennan Center for Justice at New York University School of Law in their report “Voting System Failures: A Database Solution”.
- The requirement for source code review of 1% of Lines of Code (LOC) during the new Test Readiness Review, where a voting system must pass a few basic tests before being allowed to undergo more extensive testing, needs to be better specified to be effective; we proposed a few ways this could be improved.
- There should be explicit recognition that an important goal of the test plan and test report is to facilitate reproducibility of certification testing. We cited the difficulty of reproducing certain tests ACCURATE PIs and researchers faced during the California Top-To-Bottom Review and the Ohio EVEREST voting system review.
- The procedure for dealing with modifications to software in relation to the trusted build process needs to be better specified to handle each possibility of availability/unavailability of the original build environment and/or file signatures. The bottom line is if an unmodified file can pass signature verification or can be manually compared to a bona-fide unmodified file, then it doesn’t have to undergo testing again; otherwise, there’s no basis to know if the file has been unmodified.
We look forward to working further with EAC, vendors, advocates and experts to ensure the Testing and Certification Program remains healthy, efficient and robust.
