tjexample.c: Prevent integer overflow

Because width, height, and tjPixelSize[] are signed integers, signed
integer overflow will occur if width * height *
tjPixelSize[pixelFormat] > INT_MAX, which would cause an incorrect value
to be passed to tj3Alloc().  This commit modifies tjexample.c in the
following ways:

- Implicitly promote width, height, and tjPixelSize[pixelFormat] to
  size_t before multiplying them.

- Use malloc() rather than tj3Alloc() to allocate the uncompressed image
  buffer.  (tj3Alloc() is only necessary for JPEG buffers that will
  potentially be reallocated by the TurboJPEG API library.)

- If size_t is 32-bit, throw an error if width * height *
  tjPixelSize[pixelFormat] would overflow the data type.

Since tjexample is not installed or packaged, the worst case for this
issue was that a downstream application might interpret tjexample.c
literally and introduce a similar overflow issue into its own code.
However, it's worth noting that such issues could also be introduced
when using malloc().
This commit is contained in:
DRC
2023-06-01 13:11:14 -04:00
parent 2192560d74
commit 6b506ed397

View File

@@ -338,7 +338,12 @@ int main(int argc, char **argv)
outSubsamp = inSubsamp;
pixelFormat = TJPF_BGRX;
if ((imgBuf = tj3Alloc(width * height * tjPixelSize[pixelFormat])) == NULL)
if ((unsigned long long)width * height * tjPixelSize[pixelFormat] >
(unsigned long long)((size_t)-1))
THROW("allocating uncompressed image buffer", "Image is too large");
if ((imgBuf =
(unsigned char *)malloc(sizeof(unsigned char) * width * height *
tjPixelSize[pixelFormat])) == NULL)
THROW_UNIX("allocating uncompressed image buffer");
if (tj3Decompress8(tjInstance, jpegBuf, jpegSize, imgBuf, 0,