ICMC18: SP800-90B Testing Process, Result Bounds and Current Issues

SP800-90B: Testing Process, Result Bounds, and Current Issues (G11c) Joshua Hill, Information Security Scientist, UL, United States

We started out with just hand waving  for evaluating entropy - now, after several iterations, we have NISTs SP800-90B final - YAY! But, I had to submit 20 pages of comments.

Major comments: High-entropy noise sources fail the Restart Sanity Check much more than expected. Entropy and noise sources are required to have the same entropy rate across all process characteristic's and all environmental conditions and required to be stationary!
By more than expected... 60x more frequently than expected! There's a statistical test problem

No noise/entropy source behaves the same way across variations or temperature or voltage changes - nobody could comply!

We need to characterize what are the entropy-relevant parameters and asses appropriately.

We also have issues with noise source definition - output can be the XOR of the output of multiple copies of the same physical noise source. Think of deterministic ring oscillators - fixed period. ... get a nice flat, low entropy. Statistically looks good, but not in reality. (don't do that!)

We've constructed various simulated noise sources and models - the work is a larger scale version of DJ Johnston's 2017 work using NIST's reference python implementation. This testing occurred using only the full set of non-IID tests. First pass - looks much better!

Set up a bunch of other models, including one where you might have bad grounding, etc. (Narrow Gaussian noise source, 8-bit ADC, Sinusoidal Bias), and an idealized ring oscillator.

The models are somewhat complicated, and can return a range of entropy values for each parameter set. The lower end off the modeled range is the value that out to be used in our assessments.

It's vital to test only raw data, and to filter out extraneous signals. Don't perform statistical testing on conditional data!