TJ/xform: Check crop region against dest. image

Lossless cropping is performed after other lossless transform
operations, so the cropping region must be specified relative to the
destination image dimensions and level of chrominance subsampling, not
the source image dimensions and level of chrominance subsampling.

More specifically, if the lossless transform operation swaps the X and Y
axes, or if the image is converted to grayscale, then that changes the
cropping region requirements.

ChangeLog.md

@@ -43,6 +43,14 @@ color conversion routine.  Both bounds checks now use 64-bit integers to guard
 against overflow, and djpeg now checks for negative numbers when it parses the
 crop specification from the command line.
 
+7. Fixed an issue whereby the TurboJPEG lossless transformation function and
+methods checked the specified cropping region against the source image
+dimensions and level of chrominance subsampling rather than the destination
+image dimensions and level of chrominance subsampling, which caused some
+cropping regions to be unduly rejected when performing 90-degree rotation,
+270-degree rotation, transposition, transverse transposition, or grayscale
+conversion.
+
 
 3.0.3
 =====

doc/html/group___turbo_j_p_e_g.html

@@ -1124,7 +1124,7 @@ scalingFactor)</code>. </p>
 <tr><td class="fieldname"><a id="gga1d047060ea80bb9820d540bb928e9074ac124fa8f6cb41147e3d670dfbdfb7173" name="gga1d047060ea80bb9820d540bb928e9074ac124fa8f6cb41147e3d670dfbdfb7173"></a>TJSAMP_UNKNOWN&#160;</td><td class="fielddoc"><p>Unknown subsampling. </p>
 <p>The JPEG image uses an unusual type of chrominance subsampling. Such images can be decompressed into packed-pixel images, but they cannot be</p><ul>
 <li>decompressed into planar YUV images,</li>
-<li>losslessly transformed if <a class="el" href="group___turbo_j_p_e_g.html#ga9c771a757fc1294add611906b89ab2d2" title="Enable lossless cropping.">TJXOPT_CROP</a> is specified, or</li>
+<li>losslessly transformed if <a class="el" href="group___turbo_j_p_e_g.html#ga9c771a757fc1294add611906b89ab2d2" title="Enable lossless cropping.">TJXOPT_CROP</a> is specified and <a class="el" href="group___turbo_j_p_e_g.html#ga3acee7b48ade1b99e5588736007c2589" title="Discard the color data in the source image, and generate a grayscale destination image.">TJXOPT_GRAY</a> is not specified, or</li>
 <li>partially decompressed using a cropping region. </li>
 </ul>
 </td></tr>

doc/html/structtjregion.html

@@ -146,7 +146,7 @@ Data Fields</h2></td></tr>
 </div><div class="memdoc">
 
 <p>The left boundary of the cropping region. </p>
-<p>This must be evenly divisible by the iMCU width (see <a class="el" href="group___turbo_j_p_e_g.html#ga9e61e7cd47a15a173283ba94e781308c" title="iMCU width (in pixels) for a given level of chrominance subsampling">tjMCUWidth</a>.) </p>
+<p>For lossless transformation, this must be evenly divisible by the iMCU width (see <a class="el" href="group___turbo_j_p_e_g.html#ga9e61e7cd47a15a173283ba94e781308c" title="iMCU width (in pixels) for a given level of chrominance subsampling">tjMCUWidth</a>) of the destination image. For decompression, this must be evenly divisible by the scaled iMCU width of the source image. </p>
 
 </div>
 </div>
@@ -163,7 +163,7 @@ Data Fields</h2></td></tr>
 </div><div class="memdoc">
 
 <p>The upper boundary of the cropping region. </p>
-<p>For lossless transformation, this must be evenly divisible by the iMCU height (see <a class="el" href="group___turbo_j_p_e_g.html#gabd247bb9fecb393eca57366feb8327bf" title="iMCU height (in pixels) for a given level of chrominance subsampling">tjMCUHeight</a>.) </p>
+<p>For lossless transformation, this must be evenly divisible by the iMCU height (see <a class="el" href="group___turbo_j_p_e_g.html#gabd247bb9fecb393eca57366feb8327bf" title="iMCU height (in pixels) for a given level of chrominance subsampling">tjMCUHeight</a>) of the destination image. </p>
 
 </div>
 </div>

java/doc/org/libjpegturbo/turbojpeg/TJ.html

@@ -932,7 +932,8 @@ extends java.lang.Object</pre>
  images can be decompressed into packed-pixel images, but they cannot be
  <ul>
  <li> decompressed into planar YUV images,
- <li> losslessly transformed if <a href="TJTransform.html#OPT_CROP"><code>TJTransform.OPT_CROP</code></a> is specified,
+ <li> losslessly transformed if <a href="TJTransform.html#OPT_CROP"><code>TJTransform.OPT_CROP</code></a> is specified
+ and <a href="TJTransform.html#OPT_GRAY"><code>TJTransform.OPT_GRAY</code></a> is not specified,
  or
  <li> partially decompressed using a cropping region.
  </ul></div>

java/doc/org/libjpegturbo/turbojpeg/TJTransform.html

@@ -818,9 +818,10 @@ extends java.awt.Rectangle</pre>
 <dl>
 <dt><span class="paramLabel">Parameters:</span></dt>
 <dd><code>x</code> - the left boundary of the cropping region.  This must be evenly
- divisible by the iMCU width (see <a href="TJ.html#getMCUWidth(int)"><code>TJ.getMCUWidth()</code></a>)</dd>
+ divisible by the iMCU width (see <a href="TJ.html#getMCUWidth(int)"><code>TJ.getMCUWidth()</code></a>)
+ of the destination image.</dd>
 <dd><code>y</code> - the upper boundary of the cropping region.  This must be evenly
- divisible by the iMCU height (see <a href="TJ.html#getMCUHeight(int)"><code>TJ.getMCUHeight()</code></a>)</dd>
+ divisible by the iMCU height (see <a href="TJ.html#getMCUHeight(int)"><code>TJ.getMCUHeight()</code></a>) of the destination image.</dd>
 <dd><code>w</code> - the width of the cropping region.  Setting this to 0 is the
  equivalent of setting it to (width of the source JPEG image -
  <code>x</code>).</dd>

java/org/libjpegturbo/turbojpeg/TJ.java

@@ -112,7 +112,8 @@ public final class TJ {
    * images can be decompressed into packed-pixel images, but they cannot be
    * <ul>
    * <li> decompressed into planar YUV images,
-   * <li> losslessly transformed if {@link TJTransform#OPT_CROP} is specified,
+   * <li> losslessly transformed if {@link TJTransform#OPT_CROP} is specified
+   * and {@link TJTransform#OPT_GRAY} is not specified,
    * or
    * <li> partially decompressed using a cropping region.
    * </ul>

java/org/libjpegturbo/turbojpeg/TJTransform.java

@@ -165,10 +165,11 @@ public class TJTransform extends Rectangle {
    *
    * @param x the left boundary of the cropping region.  This must be evenly
    * divisible by the iMCU width (see {@link TJ#getMCUWidth TJ.getMCUWidth()})
+   * of the destination image.
    *
    * @param y the upper boundary of the cropping region.  This must be evenly
    * divisible by the iMCU height (see {@link TJ#getMCUHeight
-   * TJ.getMCUHeight()})
+   * TJ.getMCUHeight()}) of the destination image.
    *
    * @param w the width of the cropping region.  Setting this to 0 is the
    * equivalent of setting it to (width of the source JPEG image -

turbojpeg-jni.c

@@ -1141,7 +1141,7 @@ JNIEXPORT jintArray JNICALL Java_org_libjpegturbo_turbojpeg_TJTransformer_transf
   size_t *dstSizes = NULL;
   tjtransform *t = NULL;
   jbyteArray *jdstBufs = NULL;
-  int i, jpegWidth = 0, jpegHeight = 0, jpegSubsamp;
+  int i, jpegWidth = 0, jpegHeight = 0, srcSubsamp;
   jintArray jdstSizes = 0;
   jint *dstSizesi = NULL;
   JNICustomFilterParams *params = NULL;
@@ -1154,8 +1154,7 @@ JNIEXPORT jintArray JNICALL Java_org_libjpegturbo_turbojpeg_TJTransformer_transf
     THROW_ARG("JPEG header has not yet been read");
   if ((jpegHeight = tj3Get(handle, TJPARAM_JPEGHEIGHT)) == -1)
     THROW_ARG("JPEG header has not yet been read");
-  if ((jpegSubsamp = tj3Get(handle, TJPARAM_SUBSAMP)) == TJSAMP_UNKNOWN)
+  srcSubsamp = tj3Get(handle, TJPARAM_SUBSAMP);
-    THROW_ARG("TJPARAM_SUBSAMP must be specified");
 
   n = (*env)->GetArrayLength(env, dstobjs);
   if (n != (*env)->GetArrayLength(env, tobjs))
@@ -1214,6 +1213,7 @@ JNIEXPORT jintArray JNICALL Java_org_libjpegturbo_turbojpeg_TJTransformer_transf
 
   for (i = 0; i < n; i++) {
     int w = jpegWidth, h = jpegHeight;
+    int dstSubsamp = (t[i].options & TJXOPT_GRAY) ? TJSAMP_GRAY : srcSubsamp;
 
     if (t[i].op == TJXOP_TRANSPOSE || t[i].op == TJXOP_TRANSVERSE ||
         t[i].op == TJXOP_ROT90 || t[i].op == TJXOP_ROT270) {
@@ -1223,7 +1223,7 @@ JNIEXPORT jintArray JNICALL Java_org_libjpegturbo_turbojpeg_TJTransformer_transf
     if (t[i].r.h != 0) h = t[i].r.h;
     BAILIF0(jdstBufs[i] = (*env)->GetObjectArrayElement(env, dstobjs, i));
     if ((size_t)(*env)->GetArrayLength(env, jdstBufs[i]) <
-        tj3JPEGBufSize(w, h, jpegSubsamp))
+        tj3JPEGBufSize(w, h, dstSubsamp))
       THROW_ARG("Destination buffer is not large enough");
   }
   BAILIF0NOEC(jpegBuf = (*env)->GetPrimitiveArrayCritical(env, jsrcBuf, 0));

turbojpeg.c

@@ -2665,7 +2665,7 @@ DLLEXPORT int tj3Transform(tjhandle handle, const unsigned char *jpegBuf,
   static const char FUNCTION_NAME[] = "tj3Transform";
   jpeg_transform_info *xinfo = NULL;
   jvirt_barray_ptr *srccoefs, *dstcoefs;
-  int retval = 0, i, saveMarkers = 0;
+  int retval = 0, i, saveMarkers = 0, srcSubsamp;
   boolean alloc = TRUE;
   struct my_progress_mgr progress;
 
@@ -2733,20 +2733,31 @@ DLLEXPORT int tj3Transform(tjhandle handle, const unsigned char *jpegBuf,
       (unsigned long long)dinfo->image_width * dinfo->image_height >
       (unsigned long long)this->maxPixels)
     THROW("Image is too large");
-  this->subsamp = getSubsamp(&this->dinfo);
+  srcSubsamp = getSubsamp(&this->dinfo);
 
   for (i = 0; i < n; i++) {
+    int dstSubsamp = (t[i].options & TJXOPT_GRAY) ? TJSAMP_GRAY : srcSubsamp;
+
     if (!jtransform_request_workspace(dinfo, &xinfo[i]))
       THROW("Transform is not perfect");
 
     if (xinfo[i].crop) {
-      if (this->subsamp == TJSAMP_UNKNOWN)
+      if (dstSubsamp == TJSAMP_UNKNOWN)
         THROW("Could not determine subsampling level of JPEG image");
-      if ((t[i].r.x % tjMCUWidth[this->subsamp]) != 0 ||
+      if (t[i].op == TJXOP_TRANSPOSE || t[i].op == TJXOP_TRANSVERSE ||
-          (t[i].r.y % tjMCUHeight[this->subsamp]) != 0)
+          t[i].op == TJXOP_ROT90 || t[i].op == TJXOP_ROT270) {
-        THROWI("To crop this JPEG image, x must be a multiple of %d\n"
+        if ((t[i].r.x % tjMCUHeight[dstSubsamp]) != 0 ||
-               "and y must be a multiple of %d.", tjMCUWidth[this->subsamp],
+            (t[i].r.y % tjMCUWidth[dstSubsamp]) != 0)
-               tjMCUHeight[this->subsamp]);
+          THROWI("To crop this JPEG image, x must be a multiple of %d\n"
+                 "and y must be a multiple of %d.", tjMCUHeight[dstSubsamp],
+                 tjMCUWidth[dstSubsamp]);
+      } else {
+        if ((t[i].r.x % tjMCUWidth[dstSubsamp]) != 0 ||
+            (t[i].r.y % tjMCUHeight[dstSubsamp]) != 0)
+          THROWI("To crop this JPEG image, x must be a multiple of %d\n"
+                 "and y must be a multiple of %d.", tjMCUWidth[dstSubsamp],
+                 tjMCUHeight[dstSubsamp]);
+      }
     }
   }
 
@@ -2754,6 +2765,7 @@ DLLEXPORT int tj3Transform(tjhandle handle, const unsigned char *jpegBuf,
 
   for (i = 0; i < n; i++) {
     int w, h;
+    int dstSubsamp = (t[i].options & TJXOPT_GRAY) ? TJSAMP_GRAY : srcSubsamp;
 
     if (!xinfo[i].crop) {
       w = dinfo->image_width;  h = dinfo->image_height;
@@ -2765,7 +2777,7 @@ DLLEXPORT int tj3Transform(tjhandle handle, const unsigned char *jpegBuf,
       w = xinfo[i].crop_width;  h = xinfo[i].crop_height;
     }
     if (this->noRealloc) {
-      alloc = FALSE;  dstSizes[i] = tj3JPEGBufSize(w, h, this->subsamp);
+      alloc = FALSE;  dstSizes[i] = tj3JPEGBufSize(w, h, dstSubsamp);
     }
     if (!(t[i].options & TJXOPT_NOOUTPUT))
       jpeg_mem_dest_tj(cinfo, &dstBufs[i], &dstSizes[i], alloc);
@@ -2843,22 +2855,13 @@ DLLEXPORT int tjTransform(tjhandle handle, const unsigned char *jpegBuf,
   int i, retval = 0;
   size_t *sizes = NULL;
 
-  GET_DINSTANCE(handle);
+  GET_TJINSTANCE(handle, -1);
   if ((this->init & DECOMPRESS) == 0)
     THROW("Instance has not been initialized for decompression");
 
   if (n < 1 || dstSizes == NULL)
     THROW("Invalid argument");
 
-  if (setjmp(this->jerr.setjmp_buffer)) {
-    /* If we get here, the JPEG code has signaled an error. */
-    retval = -1;  goto bailout;
-  }
-
-  jpeg_mem_src_tj(dinfo, jpegBuf, jpegSize);
-  jpeg_read_header(dinfo, TRUE);
-  if (getSubsamp(dinfo) == TJSAMP_UNKNOWN)
-    THROW("Could not determine subsampling level of JPEG image");
   processFlags(handle, flags, COMPRESS);
 
   if ((sizes = (size_t *)malloc(n * sizeof(size_t))) == NULL)

turbojpeg.h

@@ -188,7 +188,8 @@ enum TJSAMP {
    * The JPEG image uses an unusual type of chrominance subsampling.  Such
    * images can be decompressed into packed-pixel images, but they cannot be
    * - decompressed into planar YUV images,
-   * - losslessly transformed if #TJXOPT_CROP is specified, or
+   * - losslessly transformed if #TJXOPT_CROP is specified and #TJXOPT_GRAY is
+   * not specified, or
    * - partially decompressed using a cropping region.
    */
   TJSAMP_UNKNOWN = -1
@@ -1050,13 +1051,16 @@ typedef struct {
  */
 typedef struct {
   /**
-   * The left boundary of the cropping region.  This must be evenly divisible
+   * The left boundary of the cropping region.  For lossless transformation,
-   * by the iMCU width (see #tjMCUWidth.)
+   * this must be evenly divisible by the iMCU width (see #tjMCUWidth) of the
+   * destination image.  For decompression, this must be evenly divisible by
+   * the scaled iMCU width of the source image.
    */
   int x;
   /**
    * The upper boundary of the cropping region.  For lossless transformation,
-   * this must be evenly divisible by the iMCU height (see #tjMCUHeight.)
+   * this must be evenly divisible by the iMCU height (see #tjMCUHeight) of the
+   * destination image.
    */
   int y;
   /**