tjexample.c: Prevent integer overflow

Because width, height, and tjPixelSize[] are signed integers, signed
integer overflow will occur if width * height *
tjPixelSize[pixelFormat] > INT_MAX or jpegSize > INT_MAX, which would
cause an incorrect value to be passed to tjAlloc().  This commit
modifies tjexample.c in the following ways:

- Use malloc() rather than tjAlloc() to allocate the uncompressed image
  buffer.  (tjAlloc() is only necessary for JPEG buffers that will
  potentially be reallocated by the TurboJPEG API library.)

- Implicitly promote width, height, and tjPixelSize[pixelFormat] to
  size_t before multiplying them.

- If size_t is 32-bit, throw an error if width * height *
  tjPixelSize[pixelFormat] would overflow the data type.

- Throw an error if jpegSize would overflow a signed integer (which
  could only happen on a 64-bit machine.)

Since tjexample is not installed or packaged, the worst case for this
issue was that a downstream application might interpret tjexample.c
literally and introduce a similar overflow issue into its own code.
However, it's worth noting that such issues could also be introduced
when using malloc().
This commit is contained in:
DRC
2023-06-01 13:11:14 -04:00
parent 2e1b8a462f
commit be678e5a1b

View File

@@ -36,6 +36,7 @@
#define _CRT_SECURE_NO_DEPRECATE
#endif
#include <limits.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
@@ -274,6 +275,8 @@ int main(int argc, char **argv)
if (size == 0)
THROW("determining input file size", "Input file contains no data");
jpegSize = (unsigned long)size;
if (jpegSize > (unsigned long)INT_MAX)
THROW("allocating JPEG buffer", "Input file is too large");
if ((jpegBuf = (unsigned char *)tjAlloc(jpegSize)) == NULL)
THROW_UNIX("allocating JPEG buffer");
if (fread(jpegBuf, jpegSize, 1, jpegFile) < 1)
@@ -331,8 +334,12 @@ int main(int argc, char **argv)
outSubsamp = inSubsamp;
pixelFormat = TJPF_BGRX;
if ((imgBuf = (unsigned char *)tjAlloc(width * height *
tjPixelSize[pixelFormat])) == NULL)
if ((unsigned long long)width * height * tjPixelSize[pixelFormat] >
(unsigned long long)((size_t)-1))
THROW("allocating uncompressed image buffer", "Image is too large");
if ((imgBuf =
(unsigned char *)malloc(sizeof(unsigned char) * width * height *
tjPixelSize[pixelFormat])) == NULL)
THROW_UNIX("allocating uncompressed image buffer");
if (tjDecompress2(tjInstance, jpegBuf, jpegSize, imgBuf, width, 0, height,