You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pdfbox.apache.org by Tim Allison <ta...@apache.org> on 2019/09/20 12:43:00 UTC
Re: build failure in master
PDFBox Colleagues,
Do you know of any diffs between Java 8 and 11 that would affect the
extraction of images from PDFs? Dan is getting a build failure
because of a hash mismatch.
Thank you.
Best,
Tim
On Fri, Sep 20, 2019 at 8:39 AM Dan Becker <db...@kududyn.com> wrote:
>
> When one installs "sudo apt install -y default-jre" on Ubuntu 16.04, the
> Tika build will be successful with the following tools:
> vagrant@ubuntu-xenial:~/tika$ javac -version
> javac 1.8.0_222
> vagrant@ubuntu-xenial:~/tika$ java -version
> openjdk version "1.8.0_222"
>
> When one installs "sudo apt install -y default-jre" on Ubuntu 18.04, the
> Tika build will FAIL with the following tools:
> vagrant@ubuntu-bionic:~/tika$ javac -version
> javac 11.0.4
> vagrant@ubuntu-bionic:~/tika$ java -version
> openjdk version "11.0.4" 2019-07-16
>
> When one installs "sudo apt install -y openjdk-8-jdk" on Ubuntu 18.04, the
> Tika build will FAIL with the following tools (Note the compiler has been
> changed to jdk 8, but java has not):
> vagrant@ubuntu-bionic:~/tika$ javac -version
> javac 1.8.0_222
> vagrant@ubuntu-bionic:~/tika$ java -version
> openjdk version "11.0.4" 2019-07-16
>
> If you switch the Java version with "echo 2 | sudo update-alternatives
> --config java" (Note "2" works for a clean bionic vagrant VM, but YMMV),
> then the Tika build will be successful with the following tools:
> vagrant@ubuntu-bionic:~/tika$ javac -version
> javac 1.8.0_222
> vagrant@ubuntu-bionic:~/tika$ java -version
> openjdk version "1.8.0_222"
>
> I suspect that there is some new issue with running Tika under Open JDK
> 11.0.4. I will continue to look for the root cause of that next week.
>
>
> Dan
> C: 301-524-8899
>
>
> On Thu, Sep 19, 2019 at 1:00 PM Dan Becker <db...@kududyn.com> wrote:
>
> > It is a clean checkout and build with no local changes.
> >
> > I tested against the stock Ubuntu 16.04, and all the tests passed. The
> > only difference with the command sequence listed in the first email is
> > "vagrant init ubuntu/xenial64".
> >
> > I retested Ubuntu 18.04 on a different host (different version of vagrant,
> > etc), and I got the same error, so the problem does "repro".
> >
> > I will try to debug it further to determine the root cause.
> >
> > Dan
> > C: 301-524-8899
> >
> >
> > On Thu, Sep 19, 2019 at 5:07 AM Nick Burch <ap...@gagravarr.org> wrote:
> >
> >> On Wed, 18 Sep 2019, Dan Becker wrote:
> >> > I am trying to build the master branch from Ubuntu 18.04, but I am
> >> getting
> >> > the following error:
> >> >
> >> > [ERROR] Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> >> > 1.409 s <<< FAILURE! - in org.apache.tika.server.UnpackerResourceTest
> >> > [ERROR] testPDFImages(org.apache.tika.server.UnpackerResourceTest) Time
> >> > elapsed: 0.366 s <<< FAILURE!
> >> > org.junit.ComparisonFailure:
> >> expected:<[7c2f14acbb737672a1245f4ceb50622a]>
> >> > but was:<[58b8269d1a584b7e8c1adcb936123923]>
> >> > at
> >> >
> >> org.apache.tika.server.UnpackerResourceTest.testPDFImages(UnpackerResourceTest.java:208)
> >>
> >> Have you made any local changes first? Anything that might've been merged
> >> in locally?
> >>
> >> I'm building on Ubuntu 18.04 with Java 11, and the build completes fine
> >> for me with no errors. Pretty sure some/most of our build servers are
> >> Ubuntu too. So, not sure what's wrong for you...
> >>
> >> Nick
> >>
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: build failure in master
Posted by Tim Allison <ta...@apache.org>.
Thank you, Tilman! Let me know if there's anything I can do to help.
Cheers,
Tim
On Fri, Sep 20, 2019 at 12:04 PM Tilman Hausherr <TH...@t-online.de> wrote:
>
> After looking at the sources, I suspect that this test saves into a PNG
> image and then compares this.
>
> I now found out that when using PDFToImage to conert to PNG images, the
> files are much larger with jdk12 than with jdk8 :-(
> This may be related to the compression value in ImageIOUtils. In the
> past this didn't play any role => I will investigate. I don't have jdk11
> on this system (although on another), but this is very suspicious.
>
> jdk8: 214 KB at 240dpi
> jdk12: 14789 KB at 240dpi
>
> Alternatively, change the test so that the pixels are compared.
>
> Tilman
>
> Am 20.09.2019 um 14:43 schrieb Tim Allison:
> > PDFBox Colleagues,
> > Do you know of any diffs between Java 8 and 11 that would affect the
> > extraction of images from PDFs? Dan is getting a build failure
> > because of a hash mismatch.
> > Thank you.
> >
> > Best,
> >
> > Tim
> >
> > On Fri, Sep 20, 2019 at 8:39 AM Dan Becker <db...@kududyn.com> wrote:
> >> When one installs "sudo apt install -y default-jre" on Ubuntu 16.04, the
> >> Tika build will be successful with the following tools:
> >> vagrant@ubuntu-xenial:~/tika$ javac -version
> >> javac 1.8.0_222
> >> vagrant@ubuntu-xenial:~/tika$ java -version
> >> openjdk version "1.8.0_222"
> >>
> >> When one installs "sudo apt install -y default-jre" on Ubuntu 18.04, the
> >> Tika build will FAIL with the following tools:
> >> vagrant@ubuntu-bionic:~/tika$ javac -version
> >> javac 11.0.4
> >> vagrant@ubuntu-bionic:~/tika$ java -version
> >> openjdk version "11.0.4" 2019-07-16
> >>
> >> When one installs "sudo apt install -y openjdk-8-jdk" on Ubuntu 18.04, the
> >> Tika build will FAIL with the following tools (Note the compiler has been
> >> changed to jdk 8, but java has not):
> >> vagrant@ubuntu-bionic:~/tika$ javac -version
> >> javac 1.8.0_222
> >> vagrant@ubuntu-bionic:~/tika$ java -version
> >> openjdk version "11.0.4" 2019-07-16
> >>
> >> If you switch the Java version with "echo 2 | sudo update-alternatives
> >> --config java" (Note "2" works for a clean bionic vagrant VM, but YMMV),
> >> then the Tika build will be successful with the following tools:
> >> vagrant@ubuntu-bionic:~/tika$ javac -version
> >> javac 1.8.0_222
> >> vagrant@ubuntu-bionic:~/tika$ java -version
> >> openjdk version "1.8.0_222"
> >>
> >> I suspect that there is some new issue with running Tika under Open JDK
> >> 11.0.4. I will continue to look for the root cause of that next week.
> >>
> >>
> >> Dan
> >> C: 301-524-8899
> >>
> >>
> >> On Thu, Sep 19, 2019 at 1:00 PM Dan Becker <db...@kududyn.com> wrote:
> >>
> >>> It is a clean checkout and build with no local changes.
> >>>
> >>> I tested against the stock Ubuntu 16.04, and all the tests passed. The
> >>> only difference with the command sequence listed in the first email is
> >>> "vagrant init ubuntu/xenial64".
> >>>
> >>> I retested Ubuntu 18.04 on a different host (different version of vagrant,
> >>> etc), and I got the same error, so the problem does "repro".
> >>>
> >>> I will try to debug it further to determine the root cause.
> >>>
> >>> Dan
> >>> C: 301-524-8899
> >>>
> >>>
> >>> On Thu, Sep 19, 2019 at 5:07 AM Nick Burch <ap...@gagravarr.org> wrote:
> >>>
> >>>> On Wed, 18 Sep 2019, Dan Becker wrote:
> >>>>> I am trying to build the master branch from Ubuntu 18.04, but I am
> >>>> getting
> >>>>> the following error:
> >>>>>
> >>>>> [ERROR] Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> >>>>> 1.409 s <<< FAILURE! - in org.apache.tika.server.UnpackerResourceTest
> >>>>> [ERROR] testPDFImages(org.apache.tika.server.UnpackerResourceTest) Time
> >>>>> elapsed: 0.366 s <<< FAILURE!
> >>>>> org.junit.ComparisonFailure:
> >>>> expected:<[7c2f14acbb737672a1245f4ceb50622a]>
> >>>>> but was:<[58b8269d1a584b7e8c1adcb936123923]>
> >>>>> at
> >>>>>
> >>>> org.apache.tika.server.UnpackerResourceTest.testPDFImages(UnpackerResourceTest.java:208)
> >>>>
> >>>> Have you made any local changes first? Anything that might've been merged
> >>>> in locally?
> >>>>
> >>>> I'm building on Ubuntu 18.04 with Java 11, and the build completes fine
> >>>> for me with no errors. Pretty sure some/most of our build servers are
> >>>> Ubuntu too. So, not sure what's wrong for you...
> >>>>
> >>>> Nick
> >>>>
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
> > For additional commands, e-mail: dev-help@pdfbox.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: dev-help@pdfbox.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: build failure in master
Posted by Tilman Hausherr <TH...@t-online.de>.
After looking at the sources, I suspect that this test saves into a PNG
image and then compares this.
I now found out that when using PDFToImage to conert to PNG images, the
files are much larger with jdk12 than with jdk8 :-(
This may be related to the compression value in ImageIOUtils. In the
past this didn't play any role => I will investigate. I don't have jdk11
on this system (although on another), but this is very suspicious.
jdk8: 214 KB at 240dpi
jdk12: 14789 KB at 240dpi
Alternatively, change the test so that the pixels are compared.
Tilman
Am 20.09.2019 um 14:43 schrieb Tim Allison:
> PDFBox Colleagues,
> Do you know of any diffs between Java 8 and 11 that would affect the
> extraction of images from PDFs? Dan is getting a build failure
> because of a hash mismatch.
> Thank you.
>
> Best,
>
> Tim
>
> On Fri, Sep 20, 2019 at 8:39 AM Dan Becker <db...@kududyn.com> wrote:
>> When one installs "sudo apt install -y default-jre" on Ubuntu 16.04, the
>> Tika build will be successful with the following tools:
>> vagrant@ubuntu-xenial:~/tika$ javac -version
>> javac 1.8.0_222
>> vagrant@ubuntu-xenial:~/tika$ java -version
>> openjdk version "1.8.0_222"
>>
>> When one installs "sudo apt install -y default-jre" on Ubuntu 18.04, the
>> Tika build will FAIL with the following tools:
>> vagrant@ubuntu-bionic:~/tika$ javac -version
>> javac 11.0.4
>> vagrant@ubuntu-bionic:~/tika$ java -version
>> openjdk version "11.0.4" 2019-07-16
>>
>> When one installs "sudo apt install -y openjdk-8-jdk" on Ubuntu 18.04, the
>> Tika build will FAIL with the following tools (Note the compiler has been
>> changed to jdk 8, but java has not):
>> vagrant@ubuntu-bionic:~/tika$ javac -version
>> javac 1.8.0_222
>> vagrant@ubuntu-bionic:~/tika$ java -version
>> openjdk version "11.0.4" 2019-07-16
>>
>> If you switch the Java version with "echo 2 | sudo update-alternatives
>> --config java" (Note "2" works for a clean bionic vagrant VM, but YMMV),
>> then the Tika build will be successful with the following tools:
>> vagrant@ubuntu-bionic:~/tika$ javac -version
>> javac 1.8.0_222
>> vagrant@ubuntu-bionic:~/tika$ java -version
>> openjdk version "1.8.0_222"
>>
>> I suspect that there is some new issue with running Tika under Open JDK
>> 11.0.4. I will continue to look for the root cause of that next week.
>>
>>
>> Dan
>> C: 301-524-8899
>>
>>
>> On Thu, Sep 19, 2019 at 1:00 PM Dan Becker <db...@kududyn.com> wrote:
>>
>>> It is a clean checkout and build with no local changes.
>>>
>>> I tested against the stock Ubuntu 16.04, and all the tests passed. The
>>> only difference with the command sequence listed in the first email is
>>> "vagrant init ubuntu/xenial64".
>>>
>>> I retested Ubuntu 18.04 on a different host (different version of vagrant,
>>> etc), and I got the same error, so the problem does "repro".
>>>
>>> I will try to debug it further to determine the root cause.
>>>
>>> Dan
>>> C: 301-524-8899
>>>
>>>
>>> On Thu, Sep 19, 2019 at 5:07 AM Nick Burch <ap...@gagravarr.org> wrote:
>>>
>>>> On Wed, 18 Sep 2019, Dan Becker wrote:
>>>>> I am trying to build the master branch from Ubuntu 18.04, but I am
>>>> getting
>>>>> the following error:
>>>>>
>>>>> [ERROR] Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
>>>>> 1.409 s <<< FAILURE! - in org.apache.tika.server.UnpackerResourceTest
>>>>> [ERROR] testPDFImages(org.apache.tika.server.UnpackerResourceTest) Time
>>>>> elapsed: 0.366 s <<< FAILURE!
>>>>> org.junit.ComparisonFailure:
>>>> expected:<[7c2f14acbb737672a1245f4ceb50622a]>
>>>>> but was:<[58b8269d1a584b7e8c1adcb936123923]>
>>>>> at
>>>>>
>>>> org.apache.tika.server.UnpackerResourceTest.testPDFImages(UnpackerResourceTest.java:208)
>>>>
>>>> Have you made any local changes first? Anything that might've been merged
>>>> in locally?
>>>>
>>>> I'm building on Ubuntu 18.04 with Java 11, and the build completes fine
>>>> for me with no errors. Pretty sure some/most of our build servers are
>>>> Ubuntu too. So, not sure what's wrong for you...
>>>>
>>>> Nick
>>>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: dev-help@pdfbox.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: build failure in master
Posted by Tilman Hausherr <TH...@t-online.de>.
Hi all,
The changes I made won't help because tika passes the "ocrImageQuality"
value 1 to ImageIOUtil.write().
It may be possible to pass the build tests on jdk11 and later by
changing the configuration. The "ocrImageQuality" in ParserConfig.java
should be 0, not 1 for "png".
I tested this with PDFToImage and the result png files are now
identical. But this could be different on other jdk versions.
So in the long run you should check size and pixel equality, like done
in PDFBox TestPDFToImage.java:
/**
* Create an image; the part between the smaller and the larger
image is painted black, the rest
* in white
*
* @param minWidth width of the smaller image
* @param minHeight width of the smaller image
* @param maxWidth height of the larger image
* @param maxHeight height of the larger image
*
* @return
*/
private BufferedImage createEmptyDiffImage(int minWidth, int
minHeight, int maxWidth, int maxHeight)
{
BufferedImage bim3 = new BufferedImage(maxWidth, maxHeight,
BufferedImage.TYPE_INT_RGB);
Graphics graphics = bim3.getGraphics();
if (minWidth != maxWidth || minHeight != maxHeight)
{
graphics.setColor(Color.BLACK);
graphics.fillRect(0, 0, maxWidth, maxHeight);
}
graphics.setColor(Color.WHITE);
graphics.fillRect(0, 0, minWidth, minHeight);
graphics.dispose();
return bim3;
}
/**
* Get the difference between two images, identical colors are set
to white, differences are
* xored, the highest bit of each color is reset to avoid colors
that are too light
*
* @param bim1
* @param bim2
* @return If the images are different, the function returns a diff
image If the images are
* identical, the function returns null If the size is different, a
black border on the botton
* and the right is created
*
* @throws IOException
*/
private BufferedImage diffImages(BufferedImage bim1, BufferedImage
bim2) throws IOException
{
int minWidth = Math.min(bim1.getWidth(), bim2.getWidth());
int minHeight = Math.min(bim1.getHeight(), bim2.getHeight());
int maxWidth = Math.max(bim1.getWidth(), bim2.getWidth());
int maxHeight = Math.max(bim1.getHeight(), bim2.getHeight());
BufferedImage bim3 = null;
if (minWidth != maxWidth || minHeight != maxHeight)
{
bim3 = createEmptyDiffImage(minWidth, minHeight, maxWidth,
maxHeight);
}
for (int x = 0; x < minWidth; ++x)
{
for (int y = 0; y < minHeight; ++y)
{
int rgb1 = bim1.getRGB(x, y);
int rgb2 = bim2.getRGB(x, y);
if (rgb1 != rgb2
// don't bother about small differences
&& (Math.abs((rgb1 & 0xFF) - (rgb2 & 0xFF)) > 3
|| Math.abs(((rgb1 >> 8) & 0xFF) - ((rgb2 >> 8)
& 0xFF)) > 3
|| Math.abs(((rgb1 >> 16) & 0xFF) - ((rgb2 >>
16) & 0xFF)) > 3))
{
if (bim3 == null)
{
bim3 = createEmptyDiffImage(minWidth,
minHeight, maxWidth, maxHeight);
}
int r = Math.abs((rgb1 & 0xFF) - (rgb2 & 0xFF));
int g = Math.abs((rgb1 & 0xFF00) - (rgb2 & 0xFF00));
int b = Math.abs((rgb1 & 0xFF0000) - (rgb2 &
0xFF0000));
bim3.setRGB(x, y, 0xFFFFFF - (r | g | b));
}
else
{
if (bim3 != null)
{
bim3.setRGB(x, y, Color.WHITE.getRGB());
}
}
}
}
return bim3;
}
Tilman
Am 21.09.2019 um 14:45 schrieb Tim Allison:
> Thank you, Tilman!
>
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/PDFBOX-4655
>
> We should probably change our unit test to avoid requiring digest equality?
> :( Other simple options?
>
> On Fri, Sep 20, 2019 at 11:57 AM Sergey Beryozkin <sb...@gmail.com>
> wrote:
>
>> Is it the message digest signatures of some PDF content ? May be to do
>> with some MessageDigest enhancements in Java 11 ? I haven't found anything
>> specific, but this one possible line. Or may be the default hashCode() has
>> changed, which can also affect the collection hashCode() and the total
>> digest too
>>
>> Cheers, Sergey
>>
>> On Fri, Sep 20, 2019 at 1:43 PM Tim Allison <ta...@apache.org> wrote:
>>
>>> PDFBox Colleagues,
>>> Do you know of any diffs between Java 8 and 11 that would affect the
>>> extraction of images from PDFs? Dan is getting a build failure
>>> because of a hash mismatch.
>>> Thank you.
>>>
>>> Best,
>>>
>>> Tim
>>>
>>> On Fri, Sep 20, 2019 at 8:39 AM Dan Becker <db...@kududyn.com> wrote:
>>>> When one installs "sudo apt install -y default-jre" on Ubuntu 16.04, the
>>>> Tika build will be successful with the following tools:
>>>> vagrant@ubuntu-xenial:~/tika$ javac -version
>>>> javac 1.8.0_222
>>>> vagrant@ubuntu-xenial:~/tika$ java -version
>>>> openjdk version "1.8.0_222"
>>>>
>>>> When one installs "sudo apt install -y default-jre" on Ubuntu 18.04, the
>>>> Tika build will FAIL with the following tools:
>>>> vagrant@ubuntu-bionic:~/tika$ javac -version
>>>> javac 11.0.4
>>>> vagrant@ubuntu-bionic:~/tika$ java -version
>>>> openjdk version "11.0.4" 2019-07-16
>>>>
>>>> When one installs "sudo apt install -y openjdk-8-jdk" on Ubuntu 18.04,
>>> the
>>>> Tika build will FAIL with the following tools (Note the compiler has
>>> been
>>>> changed to jdk 8, but java has not):
>>>> vagrant@ubuntu-bionic:~/tika$ javac -version
>>>> javac 1.8.0_222
>>>> vagrant@ubuntu-bionic:~/tika$ java -version
>>>> openjdk version "11.0.4" 2019-07-16
>>>>
>>>> If you switch the Java version with "echo 2 | sudo update-alternatives
>>>> --config java" (Note "2" works for a clean bionic vagrant VM, but
>>> YMMV),
>>>> then the Tika build will be successful with the following tools:
>>>> vagrant@ubuntu-bionic:~/tika$ javac -version
>>>> javac 1.8.0_222
>>>> vagrant@ubuntu-bionic:~/tika$ java -version
>>>> openjdk version "1.8.0_222"
>>>>
>>>> I suspect that there is some new issue with running Tika under Open JDK
>>>> 11.0.4. I will continue to look for the root cause of that next week.
>>>>
>>>>
>>>> Dan
>>>> C: 301-524-8899
>>>>
>>>>
>>>> On Thu, Sep 19, 2019 at 1:00 PM Dan Becker <db...@kududyn.com> wrote:
>>>>
>>>>> It is a clean checkout and build with no local changes.
>>>>>
>>>>> I tested against the stock Ubuntu 16.04, and all the tests passed.
>>> The
>>>>> only difference with the command sequence listed in the first email is
>>>>> "vagrant init ubuntu/xenial64".
>>>>>
>>>>> I retested Ubuntu 18.04 on a different host (different version of
>>> vagrant,
>>>>> etc), and I got the same error, so the problem does "repro".
>>>>>
>>>>> I will try to debug it further to determine the root cause.
>>>>>
>>>>> Dan
>>>>> C: 301-524-8899
>>>>>
>>>>>
>>>>> On Thu, Sep 19, 2019 at 5:07 AM Nick Burch <ap...@gagravarr.org>
>>> wrote:
>>>>>> On Wed, 18 Sep 2019, Dan Becker wrote:
>>>>>>> I am trying to build the master branch from Ubuntu 18.04, but I am
>>>>>> getting
>>>>>>> the following error:
>>>>>>>
>>>>>>> [ERROR] Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time
>>> elapsed:
>>>>>>> 1.409 s <<< FAILURE! - in
>>> org.apache.tika.server.UnpackerResourceTest
>>>>>>> [ERROR]
>>> testPDFImages(org.apache.tika.server.UnpackerResourceTest) Time
>>>>>>> elapsed: 0.366 s <<< FAILURE!
>>>>>>> org.junit.ComparisonFailure:
>>>>>> expected:<[7c2f14acbb737672a1245f4ceb50622a]>
>>>>>>> but was:<[58b8269d1a584b7e8c1adcb936123923]>
>>>>>>> at
>>>>>>>
>>> org.apache.tika.server.UnpackerResourceTest.testPDFImages(UnpackerResourceTest.java:208)
>>>>>> Have you made any local changes first? Anything that might've been
>>> merged
>>>>>> in locally?
>>>>>>
>>>>>> I'm building on Ubuntu 18.04 with Java 11, and the build completes
>>> fine
>>>>>> for me with no errors. Pretty sure some/most of our build servers are
>>>>>> Ubuntu too. So, not sure what's wrong for you...
>>>>>>
>>>>>> Nick
>>>>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: build failure in master
Posted by Tim Allison <ta...@apache.org>.
Thank you, Tilman!
https://issues.apache.org/jira/plugins/servlet/mobile#issue/PDFBOX-4655
We should probably change our unit test to avoid requiring digest equality?
:( Other simple options?
On Fri, Sep 20, 2019 at 11:57 AM Sergey Beryozkin <sb...@gmail.com>
wrote:
> Is it the message digest signatures of some PDF content ? May be to do
> with some MessageDigest enhancements in Java 11 ? I haven't found anything
> specific, but this one possible line. Or may be the default hashCode() has
> changed, which can also affect the collection hashCode() and the total
> digest too
>
> Cheers, Sergey
>
> On Fri, Sep 20, 2019 at 1:43 PM Tim Allison <ta...@apache.org> wrote:
>
>> PDFBox Colleagues,
>> Do you know of any diffs between Java 8 and 11 that would affect the
>> extraction of images from PDFs? Dan is getting a build failure
>> because of a hash mismatch.
>> Thank you.
>>
>> Best,
>>
>> Tim
>>
>> On Fri, Sep 20, 2019 at 8:39 AM Dan Becker <db...@kududyn.com> wrote:
>> >
>> > When one installs "sudo apt install -y default-jre" on Ubuntu 16.04, the
>> > Tika build will be successful with the following tools:
>> > vagrant@ubuntu-xenial:~/tika$ javac -version
>> > javac 1.8.0_222
>> > vagrant@ubuntu-xenial:~/tika$ java -version
>> > openjdk version "1.8.0_222"
>> >
>> > When one installs "sudo apt install -y default-jre" on Ubuntu 18.04, the
>> > Tika build will FAIL with the following tools:
>> > vagrant@ubuntu-bionic:~/tika$ javac -version
>> > javac 11.0.4
>> > vagrant@ubuntu-bionic:~/tika$ java -version
>> > openjdk version "11.0.4" 2019-07-16
>> >
>> > When one installs "sudo apt install -y openjdk-8-jdk" on Ubuntu 18.04,
>> the
>> > Tika build will FAIL with the following tools (Note the compiler has
>> been
>> > changed to jdk 8, but java has not):
>> > vagrant@ubuntu-bionic:~/tika$ javac -version
>> > javac 1.8.0_222
>> > vagrant@ubuntu-bionic:~/tika$ java -version
>> > openjdk version "11.0.4" 2019-07-16
>> >
>> > If you switch the Java version with "echo 2 | sudo update-alternatives
>> > --config java" (Note "2" works for a clean bionic vagrant VM, but
>> YMMV),
>> > then the Tika build will be successful with the following tools:
>> > vagrant@ubuntu-bionic:~/tika$ javac -version
>> > javac 1.8.0_222
>> > vagrant@ubuntu-bionic:~/tika$ java -version
>> > openjdk version "1.8.0_222"
>> >
>> > I suspect that there is some new issue with running Tika under Open JDK
>> > 11.0.4. I will continue to look for the root cause of that next week.
>> >
>> >
>> > Dan
>> > C: 301-524-8899
>> >
>> >
>> > On Thu, Sep 19, 2019 at 1:00 PM Dan Becker <db...@kududyn.com> wrote:
>> >
>> > > It is a clean checkout and build with no local changes.
>> > >
>> > > I tested against the stock Ubuntu 16.04, and all the tests passed.
>> The
>> > > only difference with the command sequence listed in the first email is
>> > > "vagrant init ubuntu/xenial64".
>> > >
>> > > I retested Ubuntu 18.04 on a different host (different version of
>> vagrant,
>> > > etc), and I got the same error, so the problem does "repro".
>> > >
>> > > I will try to debug it further to determine the root cause.
>> > >
>> > > Dan
>> > > C: 301-524-8899
>> > >
>> > >
>> > > On Thu, Sep 19, 2019 at 5:07 AM Nick Burch <ap...@gagravarr.org>
>> wrote:
>> > >
>> > >> On Wed, 18 Sep 2019, Dan Becker wrote:
>> > >> > I am trying to build the master branch from Ubuntu 18.04, but I am
>> > >> getting
>> > >> > the following error:
>> > >> >
>> > >> > [ERROR] Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time
>> elapsed:
>> > >> > 1.409 s <<< FAILURE! - in
>> org.apache.tika.server.UnpackerResourceTest
>> > >> > [ERROR]
>> testPDFImages(org.apache.tika.server.UnpackerResourceTest) Time
>> > >> > elapsed: 0.366 s <<< FAILURE!
>> > >> > org.junit.ComparisonFailure:
>> > >> expected:<[7c2f14acbb737672a1245f4ceb50622a]>
>> > >> > but was:<[58b8269d1a584b7e8c1adcb936123923]>
>> > >> > at
>> > >> >
>> > >>
>> org.apache.tika.server.UnpackerResourceTest.testPDFImages(UnpackerResourceTest.java:208)
>> > >>
>> > >> Have you made any local changes first? Anything that might've been
>> merged
>> > >> in locally?
>> > >>
>> > >> I'm building on Ubuntu 18.04 with Java 11, and the build completes
>> fine
>> > >> for me with no errors. Pretty sure some/most of our build servers are
>> > >> Ubuntu too. So, not sure what's wrong for you...
>> > >>
>> > >> Nick
>> > >>
>> > >
>>
>
Re: build failure in master
Posted by Tim Allison <ta...@apache.org>.
Thank you, Tilman!
https://issues.apache.org/jira/plugins/servlet/mobile#issue/PDFBOX-4655
We should probably change our unit test to avoid requiring digest equality?
:( Other simple options?
On Fri, Sep 20, 2019 at 11:57 AM Sergey Beryozkin <sb...@gmail.com>
wrote:
> Is it the message digest signatures of some PDF content ? May be to do
> with some MessageDigest enhancements in Java 11 ? I haven't found anything
> specific, but this one possible line. Or may be the default hashCode() has
> changed, which can also affect the collection hashCode() and the total
> digest too
>
> Cheers, Sergey
>
> On Fri, Sep 20, 2019 at 1:43 PM Tim Allison <ta...@apache.org> wrote:
>
>> PDFBox Colleagues,
>> Do you know of any diffs between Java 8 and 11 that would affect the
>> extraction of images from PDFs? Dan is getting a build failure
>> because of a hash mismatch.
>> Thank you.
>>
>> Best,
>>
>> Tim
>>
>> On Fri, Sep 20, 2019 at 8:39 AM Dan Becker <db...@kududyn.com> wrote:
>> >
>> > When one installs "sudo apt install -y default-jre" on Ubuntu 16.04, the
>> > Tika build will be successful with the following tools:
>> > vagrant@ubuntu-xenial:~/tika$ javac -version
>> > javac 1.8.0_222
>> > vagrant@ubuntu-xenial:~/tika$ java -version
>> > openjdk version "1.8.0_222"
>> >
>> > When one installs "sudo apt install -y default-jre" on Ubuntu 18.04, the
>> > Tika build will FAIL with the following tools:
>> > vagrant@ubuntu-bionic:~/tika$ javac -version
>> > javac 11.0.4
>> > vagrant@ubuntu-bionic:~/tika$ java -version
>> > openjdk version "11.0.4" 2019-07-16
>> >
>> > When one installs "sudo apt install -y openjdk-8-jdk" on Ubuntu 18.04,
>> the
>> > Tika build will FAIL with the following tools (Note the compiler has
>> been
>> > changed to jdk 8, but java has not):
>> > vagrant@ubuntu-bionic:~/tika$ javac -version
>> > javac 1.8.0_222
>> > vagrant@ubuntu-bionic:~/tika$ java -version
>> > openjdk version "11.0.4" 2019-07-16
>> >
>> > If you switch the Java version with "echo 2 | sudo update-alternatives
>> > --config java" (Note "2" works for a clean bionic vagrant VM, but
>> YMMV),
>> > then the Tika build will be successful with the following tools:
>> > vagrant@ubuntu-bionic:~/tika$ javac -version
>> > javac 1.8.0_222
>> > vagrant@ubuntu-bionic:~/tika$ java -version
>> > openjdk version "1.8.0_222"
>> >
>> > I suspect that there is some new issue with running Tika under Open JDK
>> > 11.0.4. I will continue to look for the root cause of that next week.
>> >
>> >
>> > Dan
>> > C: 301-524-8899
>> >
>> >
>> > On Thu, Sep 19, 2019 at 1:00 PM Dan Becker <db...@kududyn.com> wrote:
>> >
>> > > It is a clean checkout and build with no local changes.
>> > >
>> > > I tested against the stock Ubuntu 16.04, and all the tests passed.
>> The
>> > > only difference with the command sequence listed in the first email is
>> > > "vagrant init ubuntu/xenial64".
>> > >
>> > > I retested Ubuntu 18.04 on a different host (different version of
>> vagrant,
>> > > etc), and I got the same error, so the problem does "repro".
>> > >
>> > > I will try to debug it further to determine the root cause.
>> > >
>> > > Dan
>> > > C: 301-524-8899
>> > >
>> > >
>> > > On Thu, Sep 19, 2019 at 5:07 AM Nick Burch <ap...@gagravarr.org>
>> wrote:
>> > >
>> > >> On Wed, 18 Sep 2019, Dan Becker wrote:
>> > >> > I am trying to build the master branch from Ubuntu 18.04, but I am
>> > >> getting
>> > >> > the following error:
>> > >> >
>> > >> > [ERROR] Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time
>> elapsed:
>> > >> > 1.409 s <<< FAILURE! - in
>> org.apache.tika.server.UnpackerResourceTest
>> > >> > [ERROR]
>> testPDFImages(org.apache.tika.server.UnpackerResourceTest) Time
>> > >> > elapsed: 0.366 s <<< FAILURE!
>> > >> > org.junit.ComparisonFailure:
>> > >> expected:<[7c2f14acbb737672a1245f4ceb50622a]>
>> > >> > but was:<[58b8269d1a584b7e8c1adcb936123923]>
>> > >> > at
>> > >> >
>> > >>
>> org.apache.tika.server.UnpackerResourceTest.testPDFImages(UnpackerResourceTest.java:208)
>> > >>
>> > >> Have you made any local changes first? Anything that might've been
>> merged
>> > >> in locally?
>> > >>
>> > >> I'm building on Ubuntu 18.04 with Java 11, and the build completes
>> fine
>> > >> for me with no errors. Pretty sure some/most of our build servers are
>> > >> Ubuntu too. So, not sure what's wrong for you...
>> > >>
>> > >> Nick
>> > >>
>> > >
>>
>
Re: build failure in master
Posted by Sergey Beryozkin <sb...@gmail.com>.
Is it the message digest signatures of some PDF content ? May be to do with
some MessageDigest enhancements in Java 11 ? I haven't found anything
specific, but this one possible line. Or may be the default hashCode() has
changed, which can also affect the collection hashCode() and the total
digest too
Cheers, Sergey
On Fri, Sep 20, 2019 at 1:43 PM Tim Allison <ta...@apache.org> wrote:
> PDFBox Colleagues,
> Do you know of any diffs between Java 8 and 11 that would affect the
> extraction of images from PDFs? Dan is getting a build failure
> because of a hash mismatch.
> Thank you.
>
> Best,
>
> Tim
>
> On Fri, Sep 20, 2019 at 8:39 AM Dan Becker <db...@kududyn.com> wrote:
> >
> > When one installs "sudo apt install -y default-jre" on Ubuntu 16.04, the
> > Tika build will be successful with the following tools:
> > vagrant@ubuntu-xenial:~/tika$ javac -version
> > javac 1.8.0_222
> > vagrant@ubuntu-xenial:~/tika$ java -version
> > openjdk version "1.8.0_222"
> >
> > When one installs "sudo apt install -y default-jre" on Ubuntu 18.04, the
> > Tika build will FAIL with the following tools:
> > vagrant@ubuntu-bionic:~/tika$ javac -version
> > javac 11.0.4
> > vagrant@ubuntu-bionic:~/tika$ java -version
> > openjdk version "11.0.4" 2019-07-16
> >
> > When one installs "sudo apt install -y openjdk-8-jdk" on Ubuntu 18.04,
> the
> > Tika build will FAIL with the following tools (Note the compiler has been
> > changed to jdk 8, but java has not):
> > vagrant@ubuntu-bionic:~/tika$ javac -version
> > javac 1.8.0_222
> > vagrant@ubuntu-bionic:~/tika$ java -version
> > openjdk version "11.0.4" 2019-07-16
> >
> > If you switch the Java version with "echo 2 | sudo update-alternatives
> > --config java" (Note "2" works for a clean bionic vagrant VM, but YMMV),
> > then the Tika build will be successful with the following tools:
> > vagrant@ubuntu-bionic:~/tika$ javac -version
> > javac 1.8.0_222
> > vagrant@ubuntu-bionic:~/tika$ java -version
> > openjdk version "1.8.0_222"
> >
> > I suspect that there is some new issue with running Tika under Open JDK
> > 11.0.4. I will continue to look for the root cause of that next week.
> >
> >
> > Dan
> > C: 301-524-8899
> >
> >
> > On Thu, Sep 19, 2019 at 1:00 PM Dan Becker <db...@kududyn.com> wrote:
> >
> > > It is a clean checkout and build with no local changes.
> > >
> > > I tested against the stock Ubuntu 16.04, and all the tests passed. The
> > > only difference with the command sequence listed in the first email is
> > > "vagrant init ubuntu/xenial64".
> > >
> > > I retested Ubuntu 18.04 on a different host (different version of
> vagrant,
> > > etc), and I got the same error, so the problem does "repro".
> > >
> > > I will try to debug it further to determine the root cause.
> > >
> > > Dan
> > > C: 301-524-8899
> > >
> > >
> > > On Thu, Sep 19, 2019 at 5:07 AM Nick Burch <ap...@gagravarr.org>
> wrote:
> > >
> > >> On Wed, 18 Sep 2019, Dan Becker wrote:
> > >> > I am trying to build the master branch from Ubuntu 18.04, but I am
> > >> getting
> > >> > the following error:
> > >> >
> > >> > [ERROR] Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time
> elapsed:
> > >> > 1.409 s <<< FAILURE! - in
> org.apache.tika.server.UnpackerResourceTest
> > >> > [ERROR] testPDFImages(org.apache.tika.server.UnpackerResourceTest)
> Time
> > >> > elapsed: 0.366 s <<< FAILURE!
> > >> > org.junit.ComparisonFailure:
> > >> expected:<[7c2f14acbb737672a1245f4ceb50622a]>
> > >> > but was:<[58b8269d1a584b7e8c1adcb936123923]>
> > >> > at
> > >> >
> > >>
> org.apache.tika.server.UnpackerResourceTest.testPDFImages(UnpackerResourceTest.java:208)
> > >>
> > >> Have you made any local changes first? Anything that might've been
> merged
> > >> in locally?
> > >>
> > >> I'm building on Ubuntu 18.04 with Java 11, and the build completes
> fine
> > >> for me with no errors. Pretty sure some/most of our build servers are
> > >> Ubuntu too. So, not sure what's wrong for you...
> > >>
> > >> Nick
> > >>
> > >
>
Re: build failure in master
Posted by Sergey Beryozkin <sb...@gmail.com>.
Is it the message digest signatures of some PDF content ? May be to do with
some MessageDigest enhancements in Java 11 ? I haven't found anything
specific, but this one possible line. Or may be the default hashCode() has
changed, which can also affect the collection hashCode() and the total
digest too
Cheers, Sergey
On Fri, Sep 20, 2019 at 1:43 PM Tim Allison <ta...@apache.org> wrote:
> PDFBox Colleagues,
> Do you know of any diffs between Java 8 and 11 that would affect the
> extraction of images from PDFs? Dan is getting a build failure
> because of a hash mismatch.
> Thank you.
>
> Best,
>
> Tim
>
> On Fri, Sep 20, 2019 at 8:39 AM Dan Becker <db...@kududyn.com> wrote:
> >
> > When one installs "sudo apt install -y default-jre" on Ubuntu 16.04, the
> > Tika build will be successful with the following tools:
> > vagrant@ubuntu-xenial:~/tika$ javac -version
> > javac 1.8.0_222
> > vagrant@ubuntu-xenial:~/tika$ java -version
> > openjdk version "1.8.0_222"
> >
> > When one installs "sudo apt install -y default-jre" on Ubuntu 18.04, the
> > Tika build will FAIL with the following tools:
> > vagrant@ubuntu-bionic:~/tika$ javac -version
> > javac 11.0.4
> > vagrant@ubuntu-bionic:~/tika$ java -version
> > openjdk version "11.0.4" 2019-07-16
> >
> > When one installs "sudo apt install -y openjdk-8-jdk" on Ubuntu 18.04,
> the
> > Tika build will FAIL with the following tools (Note the compiler has been
> > changed to jdk 8, but java has not):
> > vagrant@ubuntu-bionic:~/tika$ javac -version
> > javac 1.8.0_222
> > vagrant@ubuntu-bionic:~/tika$ java -version
> > openjdk version "11.0.4" 2019-07-16
> >
> > If you switch the Java version with "echo 2 | sudo update-alternatives
> > --config java" (Note "2" works for a clean bionic vagrant VM, but YMMV),
> > then the Tika build will be successful with the following tools:
> > vagrant@ubuntu-bionic:~/tika$ javac -version
> > javac 1.8.0_222
> > vagrant@ubuntu-bionic:~/tika$ java -version
> > openjdk version "1.8.0_222"
> >
> > I suspect that there is some new issue with running Tika under Open JDK
> > 11.0.4. I will continue to look for the root cause of that next week.
> >
> >
> > Dan
> > C: 301-524-8899
> >
> >
> > On Thu, Sep 19, 2019 at 1:00 PM Dan Becker <db...@kududyn.com> wrote:
> >
> > > It is a clean checkout and build with no local changes.
> > >
> > > I tested against the stock Ubuntu 16.04, and all the tests passed. The
> > > only difference with the command sequence listed in the first email is
> > > "vagrant init ubuntu/xenial64".
> > >
> > > I retested Ubuntu 18.04 on a different host (different version of
> vagrant,
> > > etc), and I got the same error, so the problem does "repro".
> > >
> > > I will try to debug it further to determine the root cause.
> > >
> > > Dan
> > > C: 301-524-8899
> > >
> > >
> > > On Thu, Sep 19, 2019 at 5:07 AM Nick Burch <ap...@gagravarr.org>
> wrote:
> > >
> > >> On Wed, 18 Sep 2019, Dan Becker wrote:
> > >> > I am trying to build the master branch from Ubuntu 18.04, but I am
> > >> getting
> > >> > the following error:
> > >> >
> > >> > [ERROR] Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time
> elapsed:
> > >> > 1.409 s <<< FAILURE! - in
> org.apache.tika.server.UnpackerResourceTest
> > >> > [ERROR] testPDFImages(org.apache.tika.server.UnpackerResourceTest)
> Time
> > >> > elapsed: 0.366 s <<< FAILURE!
> > >> > org.junit.ComparisonFailure:
> > >> expected:<[7c2f14acbb737672a1245f4ceb50622a]>
> > >> > but was:<[58b8269d1a584b7e8c1adcb936123923]>
> > >> > at
> > >> >
> > >>
> org.apache.tika.server.UnpackerResourceTest.testPDFImages(UnpackerResourceTest.java:208)
> > >>
> > >> Have you made any local changes first? Anything that might've been
> merged
> > >> in locally?
> > >>
> > >> I'm building on Ubuntu 18.04 with Java 11, and the build completes
> fine
> > >> for me with no errors. Pretty sure some/most of our build servers are
> > >> Ubuntu too. So, not sure what's wrong for you...
> > >>
> > >> Nick
> > >>
> > >
>