Catch libjpeg errors in tjDecompressToYUV2()
Even though tjDecompressToYUV2() is mostly just a wrapper for
tjDecompressToYUVPlanes(), tjDecompressToYUV2() still calls
jpeg_read_header(), so it needs to properly set up the libjpeg error
handler prior to making this call. Otherwise, under very esoteric (and
arguably incorrect) use cases, a program could call tjDecompressToYUV2()
without first checking the JPEG header using tjDecompressHeader3(), and
if the header was corrupt, then the libjpeg API would invoke
my_error_exit(). my_error_exit() would in turn call longjmp() on the
previous value of myerr->setjmp_buffer, which was probably set in a
previous TurboJPEG function, such as tjInitDecompress(). Thus, when a
libjpeg error was triggered within the body of tjDecompressToYUV2(), the
PC would jump to the error handler of the previous TurboJPEG function,
and this usually caused stack corruption in the calling program (because
the signature and return type of the previous TurboJPEG function
probably wasn't the same as that of tjDecompressToYUV2().)
diff --git a/ChangeLog.txt b/ChangeLog.txt
index 4de0955..68261e8 100644
--- a/ChangeLog.txt
+++ b/ChangeLog.txt
@@ -38,6 +38,14 @@
not security bugs, since they were confined to the cjpeg program and did not
affect any of the libjpeg-turbo libraries.
+[7] Fixed an issue whereby attempting to decompress a JPEG file with a corrupt
+header using the tjDecompressToYUV2() function would cause the function to
+abort without returning an error and, under certain circumstances, corrupt the
+stack. This only occurred if tjDecompressToYUV2() was called prior to calling
+calling tjDecompressHeader3(), or if the return value from
+tjDecompressHeader3() was ignored (both cases represent incorrect usage of the
+TurboJPEG API.)
+
1.4.2
=====
diff --git a/turbojpeg.c b/turbojpeg.c
index b20272a..d0e9704 100644
--- a/turbojpeg.c
+++ b/turbojpeg.c
@@ -1886,6 +1886,12 @@
|| !isPow2(pad) || height<0)
_throw("tjDecompressToYUV2(): Invalid argument");
+ if(setjmp(this->jerr.setjmp_buffer))
+ {
+ /* If we get here, the JPEG code has signaled an error. */
+ return -1;
+ }
+
jpeg_mem_src_tj(dinfo, jpegBuf, jpegSize);
jpeg_read_header(dinfo, TRUE);
jpegSubsamp=getSubsamp(dinfo);
