ES&S ExpressVote Usability Study

Summary

In this project, I was part of a team of 5 researchers who planned and conducted a usability study, and performed data analysis of the study results for one of Election Systems & Software's newest voting machine-ExpressVote EVS5200. ES&S, one of the largest manufacturers and sellers of voting machine equipment and services, adheres to regulations and guidelines set forth by the Election Assistance Commission. As a result, for a new voting system to be used in a national or local election, it undergoes certification by the EAC. Our usability testing report was for the purpose of submission to the EAC certification process.

Problem

The primary purpose of this study was to examine the usability of the ExpressVote EVS5200 voting machine. The EVS5200 voting system includes a touch-screen display, an audio-tactile interface, and an integrated card reader and printer. The audio-tactile interface includes three assistive technologies: two position switches and a keypad. The ExpressVote system was designed to accommodate voters in the general voting population, including voters with cognitive, dexterity, auditory, and visual impairments. For this study, participants in each of those populations tested either the visual/touchscreen or the audio-tactile modality and observational data were collected to compute effectiveness, efficiency, and voter satisfaction for each participant group and all participants combined.

Process

The study was conducted over a period of 5 months (see figure below). We met weekly to discuss usability requirement material provided by ES&S, draft a study protocol and data collection forms, and complete required documents for Institutional Review Board (IRB) submission. After obtaining IRB approval, we spent about 3 months recruiting participants, conducting a pilot study, amending the original IRB documents, conducting the usability tests, collecting data, analyzing the data, and drafting the usability report.

Project Timeline

Product

Overall, all participants (n = 68) were able to cast their ballot successfully. Approximately, 92.14% of the voting tasks were completed without error. Thirty-seven ballots were cast with no errors and 31 of the ballots cast contained one or more errors. The highest average session time was among the Blind/Low-Vision group (mean = 9) and lowest among the Deaf/Hearing (mean = 4.82). The system usability score for the ExpressVote system was 72.09 which indicates a higher than average level of user satisfaction among participants. The lowest mean SUS score (mean = 62.50) was among participants with cognitive impairments. Most voters were neutral or confident they could use the system in a real election. Overall, these results suggested that the ExpressVote is a usable and accessible voting technology.

Mean Voter Confidence (5 Point Likert Scale)