Can People Self-Report Security Accurately? Agreement Between Self-Report and Behavioral Measures
by: Rick Wash, Emilee Rader, and Chris Fennell
It is common for researchers to use self-report measures (e.g. surveys) to measure people’s security behaviors. In the computer security community, we don’t know what behaviors people understand well enough to self-report accurately, or how well those self-reports correlate with what people actually do. In a six week field study, we collected both behavior data and survey responses from 122 subjects. We found that a rela- tively small number of behaviors – mostly related to tasks that require users to take a specific, regular action – have non-zero correlations. Since security is almost never a user’s primary task for everyday computer users, several important security behaviors that we directly measured were not self-reported accurately. These results suggest that security research based on self-report is only reliable for certain behaviors. Additionally, a number of important security behaviors are not sufficiently salient to users that they can self-report accurately.
Rick Wash, Emilee Rader, and Chris Fennell. “Can People Self-Report Security Accurately? Agreement Between Self-Report and Behavioral Measures” CHI 2017. Denver, CO. May 2017.