BHUSA2017: Automated Testing of Crypto Software Using Differential Fuzzing

JP Aumasson and Yolan Romailler

JP just released a new book on cryptography - check it out. Yolan is working on his masters.

What do we want to accomplish? We want to prove valid functionality works and that the program cannot be abused and secrets won't leak. They are testing code against code. For example, if you're porting from one language to another, they should be able to do the same things. (the assumption is that the reference code is correct - not always true!).  Additionally, want to test the code against the specifications, though those are sometimes not even complete or leave exercises to the users.

Automated testing can cover static analyzers, test vectors, dub fuzzing, smart fuzzing and formal verification. With things like test vectors, the more vectors you have, the better your coverage. That's a lot of testing, so need to think about how to maximize the efficiency (ease of use x coverage).

There are limitations on the current methods, like randomness quality, timing leaks and test vectors focus on valid inputs.

The researchers came up with a new tool, CDF, to do crypto differential fuzzing. It's a command line tool written in Go, portable to WIndows/Linux/MacOS and made it fast so it won't be a bottleneck. This tool will check for both correctness and security of implemetnations and interoperability between implementations. It will check for insecure parameters, non-compliance with standards (e.g. FIPS) and edge cases for specific algorithms.

This is similar to WycheProof, but different. The two tools will complement each other.

One of their checks for ECDSA will make sure that sending 00 hash or 00 secret key will not send it into an infinite loop. For RSA they do various checks for timing leaks.  With their testing, they found potential DoS with OpenSSL.

Discovered that most libraries are not testing the DSA parameters, found issues with almost every library tested. Even though they did not find issues with DSA in crypto++, it doesn't mean they don't exist - just the test did not trip over it.

Several libraries got an infinite loop with generator set to 0 in DSA. It's not allowed by the  standard, but tripped over several libraries.

Still working on adding additional tests and making the suite more robust.