Since libjpeg-turbo does not support Exif, the only way it can embed
density information in a JPEG image is by using the JFIF marker, which
is only written if the JPEG colorspace is YCbCr or grayscale.
(Referring to the conversation under #793, we may need to further
restrict that to 8-bit-per-sample JPEG images, because the JFIF spec
requires 8-bit data precision.)
- Due to an oversight, a113506d17
(libjpeg-turbo 1.4 beta1) effectively made the call to
std_huff_tables() in jpeg_set_defaults() a no-op if the Huffman tables
were previously defined, which made it impossible to disable Huffman
table optimization or progressive mode if they were previously enabled
in the same API instance. std_huff_tables() retains its previous
behavior for decompression instances, but it now force-enables the
standard (baseline) Huffman tables for compression instances.
- Due to another oversight, there was no way to disable lossless mode
if it was previously enabled in a particular API instance.
jpeg_set_defaults() now accomplishes this, which makes
TJ*PARAM_LOSSLESS behave as intended/documented.
- Due to yet another oversight, setCompDefaults() in the TurboJPEG API
library permanently modified the value of TJ*PARAM_SUBSAMP when
generating a lossless JPEG image, which affected subsequent lossy
compression operations. This issue was hidden by the issue above and
thus does not need to be publicly documented.
Fixes#792
The target data precision isn't known at the time that the calling
program sets TJPARAM_LOSSLESSPT, so tj3Set() needs to allow all possible
values (from 0 to 15.) jpeg_enable_lossless(), which is called within
the body of tj3Compress*(), will throw an error if the point transform
value is greater than {data precision} - 1.
Since LLVM/Windows emulates Visual Studio, CMake sets
CMAKE_C_SIMULATE_ID="MSVC" but does not set MSVC. Thus, our build
system needs to enable most (but not all) of the Visual Studio features
when CMAKE_C_SIMLUATE_ID="MSVC". Support for LLVM/Windows is currently
undocumented because it isn't a standalone build environment. (It
requires the Visual Studio and Windows SDK headers and link libraries.)
Closes#786
This just improves code readability by emphasizing that we don't care
about the destination image's level of subsampling unless
TJPARAM_NOREALLOC is set or lossless cropping will be performed.
With respect to tj3Transform(), this addresses an oversight from
bb1d540a80.
Note to self: A convenience function/method for computing the worst-case
transformed JPEG size for a particular transform would be nice.
- When transforming, the worst-case JPEG buffer size depends on the
subsampling level used in the destination image, since a grayscale
transform might have been applied.
- Parentheses Police
tj*Transform() relied upon the underlying transupp API to check the
cropping region. However, transupp uses unsigned integers for the
cropping region, whereas the tjregion structure uses signed integers.
Thus, casting negative values from a tjregion structure produced very
large unsigned values. In the case of the left and upper boundary, this
was innocuous, because jtransform_request_workspace() rejected the
values as being out of bounds. However, jtransform_request_workspace()
did not always reject very large width and height values, because it
supports expanding the destination image by specifying a cropping region
larger than the source image. In certain cases, it allowed those
values, and the libjpeg memory manager subsequently ran out of memory.
NOTE: Prior to this commit, image expansion technically worked with
tj*Transform() as long as the cropping width and height were valid and
automatic JPEG buffer (re)allocation was used. However, that behavior
is not a documented feature of the TurboJPEG API, nor do we have any way
of testing it at the moment. Official support for image expansion can
be added later, if there is sufficient demand for it.
Similarly, this commit modifies tj3SetCroppingRegion() so that it
explicitly checks for left boundary values exactly equal to the scaled
image width and upper boundary values exactly equal to the scaled image
height. If the specified cropping width or height was 0 (which is
interpreted as {scaled image width} - {left boundary} or
{scaled image height} - {upper boundary}), then such values caused a
cropping width or height of 0 to be passed to the libjpeg API. In the
case of the width, this was innocuous, because jpeg_crop_scanline()
rejected the value. In the case of the height, however, this caused
unexpected and hard-to-diagnose errors farther down the pipeline.
jcopy_markers_execute() has historically ignored its option argument,
which is OK for jpegtran, but tj*Transform() needs to be able to save a
set of markers from the source image and write a subset of those markers
to each destination image. Without that ability, the function
effectively behaved as if TJ*OPT_COPYNONE was not specified unless all
transforms specified it.
These images were only used with tjbenchtest and tjexampletest, but I
was concerned about introducing additional licensing provisions into the
libjpeg-turbo source tree. Thus, this commit replaces them with images
that I own and can thus make available under the libjpeg-turbo licenses
with no additional provisions.
Put all general functions at the top of the list, and ensure that all
functions are defined before they are mentioned. Also consistify the
function ordering between turbojpeg.h and turbojpeg.c
Extending the Bayer matrix past Column 80 seems like a lesser
readability sin than not putting a space after each comma, especially
since we had to explicitly whitelist the file in checkstyle.
- Eliminate unnecessary "www."
- Use HTTPS.
- Update Java, MSYS, tdm-gcc, and NSIS URLs.
- Update URL and title of Agner Fog's assembly language optimization
manual.
- Remove extraneous information about MASM and Borland Turbo Assembler
and outdated NASM URLs from the x86 assembly headers, and mention
Yasm.
Lossless cropping is performed after other lossless transform
operations, so the cropping region must be specified relative to the
destination image dimensions and level of chrominance subsampling, not
the source image dimensions and level of chrominance subsampling.
More specifically, if the lossless transform operation swaps the X and Y
axes, or if the image is converted to grayscale, then that changes the
cropping region requirements.
The JPEG-1 spec never uses the term "MCU block". That term is rarely
used in other literature to describe the equivalent of an MCU in an
interleaved JPEG image, but the libjpeg documentation uses "iMCU" to
describe the same thing. "iMCU" is a better term, since the equivalent
of an interleaved MCU can contain multiple DCT blocks (or samples in
lossless mode) that are only grouped together if the image is
interleaved.
In the case of restart markers, "MCU block" was used in the libjpeg
documentation instead of "MCU", but "MCU" is more accurate and less
confusing. (The restart interval is literally in MCUs, where one MCU
is one data unit in a non-interleaved JPEG image and multiple data units
in a multi-component interleaved JPEG image.)
In the case of 9b704f96b2, the issue was
actually with progressive JPEG images exactly two DCT blocks wide, not
two MCU blocks wide.
This commit also defines "MCU" and "MCU row" in the description of the
various restart marker options/parameters. Although an MCU row is
technically always a row of samples in lossless mode, "sample row" was
confusing, since it is used in other places to describe a row of samples
for a single component (whereas an MCU row in a typical lossless JPEG
image consists of a row of interleaved samples for all components.)
TJ*PARAM_RESTARTBLOCKS technically works with lossless compression, but
it is not useful, since the value must be equal to the number of samples
in a row. (In other words, it is no different than
TJ*PARAM_RESTARTINROWS, except that it requires the user to do more
math.)
For reasons I can't recall, fc01f4673b
(the TurboJPEG 3 API overhaul) changed the C version of TJBench, but not
the Java version, so that it requires an exact scaling factor match (as
opposed to allowing unreduced scaling factors, as djpeg does and prior
versions of TJBench did.) That might have been temporary testing code
that was accidentally committed.
Because the crop spec was parsed using unsigned 32-bit integers,
negative numbers were interpreted as values ~= UINT_MAX (4,294,967,295).
This had the following ramifications:
- If the cropping region width was negative and the adjusted width + the
adjusted left boundary was greater than 0, then the 32-bit unsigned
integer bounds checks in djpeg and jpeg_crop_scanline() overflowed and
failed to detect the out-of-bounds width, jpeg_crop_scanline() set
cinfo->output_width to a value ~= UINT_MAX, and a buffer overrun and
subsequent segfault occurred in the upsampling or color conversion
routine. The segfault occurred in the body of
jpeg_skip_scanlines() --> read_and_discard_scanlines() if the cropping
region upper boundary was greater than 0 and the JPEG image used
chrominance subsampling and in the body of jpeg_read_scanlines()
otherwise.
- If the cropping region width was negative and the adjusted width + the
adjusted left boundary was 0, then a zero-width output image was
generated.
- If the cropping region left boundary was negative, then an output
image with bogus data was generated.
This commit modifies djpeg and jpeg_crop_scanline() so that the
aforementioned bounds checks use 64-bit unsigned integers, thus guarding
against overflow. It similarly modifies jpeg_skip_scanlines(). In the
case of jpeg_skip_scanlines(), the issue was not reproducible with
djpeg, but passing a negative number of lines to jpeg_skip_scanlines()
caused a similar overflow if the number of lines +
cinfo->output_scanline was greater than 0. That caused
jpeg_skip_scanlines() to read past the end of the JPEG image, throw a
warning ("Corrupt JPEG data: premature end of data segment"), and fail
to return unless warnings were treated as fatal. Also, djpeg now parses
the crop spec using signed integers and checks for negative values.
Due to an oversight in the multi-precision feature,
TJCompressor.srcBuf12 and TJCompressor.srcBuf16 were not set to null
in TJCompressor.setSourceImage(YUVImage) or
TJCompressor.setSourceImage(BufferedImage, ...). Thus, if an
application set a 12-bit or 16-bit packed-pixel buffer as the source
image then set a BufferedImage with integer pixels as the source image,
TJCompress.compress() would compress from the 12-bit or 16-bit
packed-pixel buffer instead of the BufferedImage. The odds of an
application actually doing that are very slim, however.
There are two approaches to handling abbreviated command-line options:
1. If a new option is introduced that begins with the same letters as an
existing option, require a longer abbreviation for the existing option
in order to ensure that abbreviations are always unique.
2. Require a unique abbreviation only for new options, and match all
non-unique abbreviations with existing options, thus maintaining
backward compatibility.
keymatch() supports either approach, and Tom Lane historically seemed to
prefer Approach 2, whereas both approaches have been applied
inconsistently in the years since. This commit consistently applies
Approach 2.
More specific notes:
We unnecessarily required 'cjpeg -progressive' to be abbreviated as
'cjpeg -pro' rather than 'cjpeg -p' when the -precision option was
introduced in libjpeg-turbo 3.0 beta.
The IJG unnecessarily required 'cjpeg -scans' to be abbreviated as
'cjpeg -scan' rather than 'cjpeg -sc' when the -scale option was
introduced in jpeg-7. We even more unnecessarily adopted that
requirement, even though we never adopted the -scale option.
We unnecessarily required 'djpeg -scale' to be abbreviated as
'djpeg -sc' rather than 'djpeg -s' when the -skip option was introduced
in libjpeg-turbo 1.5 beta.
The IJG unnecessarily required 'jpegtran -copy' to be abbreviated as
'jpegtran -co' rather than 'jpegtran -c' when the -crop option was
introduced in jpeg-7.
The IJG unnecessarily required 'jpegtran -progressive' to be abbreviated
as 'jpegtran -pr' rather than 'jpegtran -p' when the -perfect option was
introduced in jpeg-7.
We need to use tj3Alloc() (which, when ZERO_BUFFERS is defined, calls
calloc() instead of malloc()) to allocate all destination buffers.
Otherwise, if the compression/decompression/transform operation fails,
then the buffer checksum (which is computed to prevent the compiler from
optimizing out the whole test, since the destination buffer is never
used otherwise) will depend upon values in the destination buffer that
were never written, and MSan will complain.
- "Optimized baseline entropy coding" = "Huffman table optimization"
"Optimized baseline entropy coding" was meant to emphasize that the
feature is only useful when generating baseline (single-scan lossy
8-bit-per-sample Huffman-coded) JPEG images, because it is
automatically enabled when generating Huffman-coded progressive
(multi-scan), 12-bit-per-sample, and lossless JPEG images. However,
Huffman table optimization isn't actually an integral part of those
non-baseline modes. You can forego Huffman table optimization with
12-bit data precision if you supply your own Huffman tables. The spec
doesn't require it with progressive or lossless mode, either, although
our implementation does. Furthermore, "baseline" describes more than
just the type of entropy coding used. It was incorrect to say that
optimized "baseline" entropy coding is automatically enabled for
Huffman-coded progressive, 12-bit-per-sample, and lossless JPEG
images, since those are clearly not baseline images.
- "Progressive entropy coding" = "Progressive JPEG"
"Progressive" describes more than just the type of entropy coding
used. (In fact, both Huffman-coded and arithmetic-coded images can be
progressive.)
- Mention that TJPARAM_OPTIMIZE/TJ.PARAM_OPTIMIZE can be used with
lossless transformation as well.
- General wordsmithing
- Formatting tweaks
Referring to
https://sourceforge.net/p/libjpeg-turbo/bugs/48,
https://sourceforge.net/p/libjpeg-turbo/bugs/82,
#15, #238, #253, and #619,
valgrind and MSan have failed to properly detect data initialization by
libjpeg-turbo's x86 SIMD extensions for the entire 14 years that
libjpeg-turbo has been a project, resulting in false positives unless
libjpeg-turbo is built with WITH_SIMD=0 or run with JSIMD_FORCENONE=1.
This commit introduces a new C preprocessor macro (ZERO_BUFFERS) that,
if set, causes libjpeg-turbo to zero certain buffers in order to work
around the specific valgrind/MSan test failures caused by the
aforementioned false positives. This allows us to more closely
approximate the production configuration of libjpeg-turbo when testing
with valgrind or MSan.
Closes#781
The dimensions in the PPM header of the output file generated by
'example decompress' were always 640 x 480, regardless of the size of
the JPEG image being decompressed. Our regression tests (which this
commit also fixes) missed the bug because they decompressed the
640 x 480 image generated by 'example compress'.
Fixes#778
