Panel: Data Sources and How Much to Trust Them presented at 8th International Workshop on Security Measurements and Metrics (MetriSec) 2012

by Peter Gutmann, Fabio Massacci, Stephan Neuhaus, Laurie Williams,

Summary : "One of the biggest problems in empirical studies about computer security is the data. Usually you can't control the data acquisition process yourself; instead, you need to take other people's work and use that. For example, you could be using Mozilla Foundation Security Advisories, or the National Vulnerability Database. Then the question is, to what extent can you trust this information to be complete and unbiased? The answer is that you cannot, at least not without knowing the process by which these databases are created. For example, many researchers have for years believed that the NVD constitutes some kind of ground truth. If that were true, then one would expect that entries that have been in the NVD for some time will in general not change. Work currently being done here at ETH indicates, however, that the amount of change, or churn, in the NVD is quite high, and that even very old entries get changed!"