While trying to add support for variable output lengths for shake128 and shake256 to Node.js, I noticed a number of segmentation faults on some of our CI platforms such as s390x. We are using OpenSSL 1.1.c.
One test case calls EVP_DigestFinalXOF with a requested output size of zero bytes. I expected OpenSSL not to write to the supplied buffer in this case, and that is true on some platforms. On other platforms such as s390x, SHA3_squeeze causes a segmentation fault, maybe trying to access the buffer which is NULL.
It is possible that other inputs are affected as well, even though I was not able to produce any segmentation faults for inputs other than 0.
Arguably, calling EVP_DigestFinalXOF with an output size of 0 bytes makes little sense, but it still should not cause a segmentation fault in my opinion. The C implementation of SHA3_squeeze seems to handle this case nicely.
While trying to add support for variable output lengths for shake128 and shake256 to Node.js, I noticed a number of segmentation faults on some of our CI platforms such as s390x. We are using OpenSSL 1.1.c.
One test case calls
EVP_DigestFinalXOFwith a requested output size of zero bytes. I expected OpenSSL not to write to the supplied buffer in this case, and that is true on some platforms. On other platforms such as s390x,SHA3_squeezecauses a segmentation fault, maybe trying to access the buffer which isNULL.It is possible that other inputs are affected as well, even though I was not able to produce any segmentation faults for inputs other than
0.Arguably, calling
EVP_DigestFinalXOFwith an output size of0bytes makes little sense, but it still should not cause a segmentation fault in my opinion. The C implementation ofSHA3_squeezeseems to handle this case nicely.