78a36f6dc3
Regression introduced by16bd984557and5b177b3cabThe pre-computed absolute values used in encode_mcu_AC_first() and encode_mcu_AC_refine() were stored in a JCOEF (signed short) array. When attempting to losslessly transform a specially-crafted malformed 12-bit JPEG image with a coefficient value of -32768 into a progressive 12-bit JPEG image, the progressive Huffman encoder attempted to store the absolute value of -32768 in the JCOEF array, thus overflowing the 16-bit signed data type. Therefore, at this point in the code:8c5e78ce29/jcphuff.c (L889)the absolute value was read as -32768, which caused the test at8c5e78ce29/jcphuff.c (L896)to fail, falling through to8c5e78ce29/jcphuff.c (L908)with an overly large value of r (46) that, when shifted left four places, incremented, and passed to emit_symbol(), exceeded the maximum index (255) for the derived code tables. Fortunately, the buffer overrun was fully contained within phuff_entropy_encoder, so the issue did not generate a segfault or other user-visible errant behavior, but it did cause a UBSan failure that was detected by OSS-Fuzz. This commit introduces an unsigned JCOEF (UJCOEF) data type and uses it to store the absolute values of DCT coefficients computed by the AC_first_prepare() and AC_refine_prepare() methods. Note that the changes to the Arm Neon progressive Huffman encoder extensions cause signed 16-bit instructions to be replaced with equivalent unsigned 16-bit instructions, so the changes should be performance-neutral. Based on:bbf61c0382Closes #628
51 lines
1.8 KiB
C
51 lines
1.8 KiB
C
/*
|
|
* jchuff.h
|
|
*
|
|
* This file was part of the Independent JPEG Group's software:
|
|
* Copyright (C) 1991-1997, Thomas G. Lane.
|
|
* libjpeg-turbo Modifications:
|
|
* Copyright (C) 2022, D. R. Commander.
|
|
* For conditions of distribution and use, see the accompanying README.ijg
|
|
* file.
|
|
*
|
|
* This file contains declarations for Huffman entropy encoding routines
|
|
* that are shared between the sequential encoder (jchuff.c) and the
|
|
* progressive encoder (jcphuff.c). No other modules need to see these.
|
|
*/
|
|
|
|
/* The legal range of a DCT coefficient is
|
|
* -1024 .. +1023 for 8-bit data;
|
|
* -16384 .. +16383 for 12-bit data.
|
|
* Hence the magnitude should always fit in 10 or 14 bits respectively.
|
|
*/
|
|
|
|
#if BITS_IN_JSAMPLE == 8
|
|
#define MAX_COEF_BITS 10
|
|
#else
|
|
#define MAX_COEF_BITS 14
|
|
#endif
|
|
|
|
/* The progressive Huffman encoder uses an unsigned 16-bit data type to store
|
|
* absolute values of coefficients, because it is possible to inject a
|
|
* coefficient value of -32768 into the encoder by attempting to transform a
|
|
* malformed 12-bit JPEG image, and the absolute value of -32768 would overflow
|
|
* a signed 16-bit integer.
|
|
*/
|
|
typedef unsigned short UJCOEF;
|
|
|
|
/* Derived data constructed for each Huffman table */
|
|
|
|
typedef struct {
|
|
unsigned int ehufco[256]; /* code for each symbol */
|
|
char ehufsi[256]; /* length of code for each symbol */
|
|
/* If no code has been allocated for a symbol S, ehufsi[S] contains 0 */
|
|
} c_derived_tbl;
|
|
|
|
/* Expand a Huffman table definition into the derived format */
|
|
EXTERN(void) jpeg_make_c_derived_tbl(j_compress_ptr cinfo, boolean isDC,
|
|
int tblno, c_derived_tbl **pdtbl);
|
|
|
|
/* Generate an optimal table definition given the specified counts */
|
|
EXTERN(void) jpeg_gen_optimal_table(j_compress_ptr cinfo, JHUFF_TBL *htbl,
|
|
long freq[]);
|