Lossless decomp: Range-limit 12-bit samples

12-bit is the only data precision for which the range of the sample data
type exceeds the valid sample range, so it is possible to craft a 12-bit
lossless JPEG image that contains out-of-range 12-bit samples.
Attempting to decompress such an image using color quantization or merged
upsampling (NOTE: libjpeg-turbo cannot generate YCbCr or subsampled
lossless JPEG images, but it can decompress them) caused segfaults or
buffer overruns when those algorithms attempted to use the out-of-range
sample values as array indices.  This commit modifies the lossless
decompressor so that it range-limits the output of the scaler when using
12-bit samples.

Fixes #670
Fixes #672
Fixes #673
Fixes #674
Fixes #675
Fixes #676
Fixes #677
Fixes #678
Fixes #679
Fixes #681
Fixes #683
This commit is contained in:
DRC
2023-04-04 13:53:21 -05:00
parent 3f43c6a310
commit 9f756bc67a
2 changed files with 20 additions and 1 deletions

View File

@@ -8,6 +8,13 @@ subsampling, which allows losslessly transposed or rotated 4:1:1 JPEG images to
be losslessly cropped, partially decompressed, or decompressed to planar YUV be losslessly cropped, partially decompressed, or decompressed to planar YUV
images. images.
2. Fixed various segfaults and buffer overruns that occurred when attempting to
decompress various specially-crafted malformed 12-bit-per-component lossless
JPEG images. These issues were caused by out-of-range sample values that were
not range-limited before being used as array indices. The issues were specific
to 12-bit data precision, since that is the only data precision for which the
range of the sample data type exceeds the valid sample range.
2.1.91 (3.0 beta2) 2.1.91 (3.0 beta2)
================== ==================

View File

@@ -6,7 +6,7 @@
* Lossless JPEG Modifications: * Lossless JPEG Modifications:
* Copyright (C) 1999, Ken Murchison. * Copyright (C) 1999, Ken Murchison.
* libjpeg-turbo Modifications: * libjpeg-turbo Modifications:
* Copyright (C) 2022, D. R. Commander. * Copyright (C) 2022-2023, D. R. Commander.
* For conditions of distribution and use, see the accompanying README.ijg * For conditions of distribution and use, see the accompanying README.ijg
* file. * file.
* *
@@ -217,7 +217,15 @@ simple_upscale(j_decompress_ptr cinfo,
JDIFFROW diff_buf, _JSAMPROW output_buf, JDIMENSION width) JDIFFROW diff_buf, _JSAMPROW output_buf, JDIMENSION width)
{ {
do { do {
#if BITS_IN_JSAMPLE == 12
/* 12-bit is the only data precision for which the range of the sample data
* type exceeds the valid sample range. Thus, we need to range-limit the
* samples, because other algorithms may try to use them as array indices.
*/
*output_buf++ = (_JSAMPLE)((*diff_buf++ << cinfo->Al) & 0xFFF);
#else
*output_buf++ = (_JSAMPLE)(*diff_buf++ << cinfo->Al); *output_buf++ = (_JSAMPLE)(*diff_buf++ << cinfo->Al);
#endif
} while (--width); } while (--width);
} }
@@ -226,7 +234,11 @@ noscale(j_decompress_ptr cinfo,
JDIFFROW diff_buf, _JSAMPROW output_buf, JDIMENSION width) JDIFFROW diff_buf, _JSAMPROW output_buf, JDIMENSION width)
{ {
do { do {
#if BITS_IN_JSAMPLE == 12
*output_buf++ = (_JSAMPLE)((*diff_buf++) & 0xFFF);
#else
*output_buf++ = (_JSAMPLE)(*diff_buf++); *output_buf++ = (_JSAMPLE)(*diff_buf++);
#endif
} while (--width); } while (--width);
} }