You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pdfbox.apache.org by Tilman Hausherr <TH...@t-online.de> on 2025/01/04 09:46:25 UTC

Re: PDFBox 2.0.33 release

https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33.tar.xz

definitively some work to do; one exception in CMap and some content 
differences / losses.

The 2.0.33 snapshot version is from 31.12, I forgot to build 2.0. 
However there have been no changes related parsing or text extraction

Tilman


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org


Re: PDFBox 2.0.33 release

Posted by Andreas Lehmkühler <an...@lehmi.de.INVALID>.

Am 12.01.25 um 13:58 schrieb sahyoun@fileaffairs.de:
> Am Sonntag, dem 12.01.2025 um 13:24 +0100 schrieb Andreas Lehmkühler:
>>
>>
>> Am 08.01.25 um 04:56 schrieb Tilman Hausherr:
>>> On 07.01.2025 15:00, Tilman Hausherr wrote:
>>>> - mysterious: govdocs1/838/838013.pdf has "ion: 4 | name: 4 |
>>>> creatinga: 3 | ram: 3 | anand: 2 | jec: 2 | message: 2 | oc: 2 |
>>>> ons:
>>>> 2 | 0or: 1", "creatinga" and "anand" DO NOT APPEAR in ordinary
>>>> text
>>>> extractions, not even with Tika from the command like. But they
>>>> appear
>>>> in the tika extraction JSON file on the machine. I'll try to
>>>> investigate this.
>>>
>>> It turns out that it happens with happens with PDFBox ExtractText
>>> only
>>> on the regression test machine. And with rendering too.
>>>
>>> The cause is that the machine has no fonts, so our Liberation Sans
>>> is
>>> used. That font is slightly larger. So instead of rendering like
>>> this
>>>
>>> We get this
>>>
>>> And text extraction uses these positions too.
>>>
>>> The appearance of "creatinga" in the "B" column of the excel file
>>> is
>>> because there are 3 more than in the "A" run.
>>>
>>> So we should install fonts on the test machine, see
>> I've tried to do so, but my user isn't a member of the sudo group :-o
> 
> I added you to the group - please give it a try and let me know if
> there are issues
Thanks for the fast response. Works like a charm. I've installed the 
mentioned ttf-mscorefonts-installer package, so that the missing fonts 
are available now.

Hope this helps

Andreas

> 
> BR
> Maruan
> 
>>
>> @Maruan or @Tim
>> Please install those missing fonts or add me to the sudo group
>>
>> Thanks
>> Andreas
>>>
>>> https://pdfbox.apache.org/3.0/faq.html#what-fonts-do-i-need-on-my-system%3F
>>>
>>> Tilman
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
>> For additional commands, e-mail: dev-help@pdfbox.apache.org
>>
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: dev-help@pdfbox.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org


Re: PDFBox 2.0.33 release

Posted by "sahyoun@fileaffairs.de" <sa...@fileaffairs.de>.
Am Sonntag, dem 12.01.2025 um 13:24 +0100 schrieb Andreas Lehmkühler:
> 
> 
> Am 08.01.25 um 04:56 schrieb Tilman Hausherr:
> > On 07.01.2025 15:00, Tilman Hausherr wrote:
> > > - mysterious: govdocs1/838/838013.pdf has "ion: 4 | name: 4 | 
> > > creatinga: 3 | ram: 3 | anand: 2 | jec: 2 | message: 2 | oc: 2 |
> > > ons: 
> > > 2 | 0or: 1", "creatinga" and "anand" DO NOT APPEAR in ordinary
> > > text 
> > > extractions, not even with Tika from the command like. But they
> > > appear 
> > > in the tika extraction JSON file on the machine. I'll try to 
> > > investigate this. 
> > 
> > It turns out that it happens with happens with PDFBox ExtractText
> > only 
> > on the regression test machine. And with rendering too.
> > 
> > The cause is that the machine has no fonts, so our Liberation Sans
> > is 
> > used. That font is slightly larger. So instead of rendering like
> > this
> > 
> > We get this
> > 
> > And text extraction uses these positions too.
> > 
> > The appearance of "creatinga" in the "B" column of the excel file
> > is 
> > because there are 3 more than in the "A" run.
> > 
> > So we should install fonts on the test machine, see
> I've tried to do so, but my user isn't a member of the sudo group :-o

I added you to the group - please give it a try and let me know if
there are issues

BR
Maruan

> 
> @Maruan or @Tim
> Please install those missing fonts or add me to the sudo group
> 
> Thanks
> Andreas
> > 
> > https://pdfbox.apache.org/3.0/faq.html#what-fonts-do-i-need-on-my-system%3F
> > 
> > Tilman
> > 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: dev-help@pdfbox.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org


Re: PDFBox 2.0.33 release

Posted by Andreas Lehmkühler <an...@lehmi.de.INVALID>.

Am 08.01.25 um 04:56 schrieb Tilman Hausherr:
> On 07.01.2025 15:00, Tilman Hausherr wrote:
>> - mysterious: govdocs1/838/838013.pdf has "ion: 4 | name: 4 | 
>> creatinga: 3 | ram: 3 | anand: 2 | jec: 2 | message: 2 | oc: 2 | ons: 
>> 2 | 0or: 1", "creatinga" and "anand" DO NOT APPEAR in ordinary text 
>> extractions, not even with Tika from the command like. But they appear 
>> in the tika extraction JSON file on the machine. I'll try to 
>> investigate this. 
> 
> It turns out that it happens with happens with PDFBox ExtractText only 
> on the regression test machine. And with rendering too.
> 
> The cause is that the machine has no fonts, so our Liberation Sans is 
> used. That font is slightly larger. So instead of rendering like this
> 
> We get this
> 
> And text extraction uses these positions too.
> 
> The appearance of "creatinga" in the "B" column of the excel file is 
> because there are 3 more than in the "A" run.
> 
> So we should install fonts on the test machine, see
I've tried to do so, but my user isn't a member of the sudo group :-o

@Maruan or @Tim
Please install those missing fonts or add me to the sudo group

Thanks
Andreas
> 
> https://pdfbox.apache.org/3.0/faq.html#what-fonts-do-i-need-on-my-system%3F
> 
> Tilman
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org


Re: PDFBox 2.0.33 release

Posted by Tilman Hausherr <TH...@t-online.de>.
On 07.01.2025 15:00, Tilman Hausherr wrote:
> - mysterious: govdocs1/838/838013.pdf has "ion: 4 | name: 4 | 
> creatinga: 3 | ram: 3 | anand: 2 | jec: 2 | message: 2 | oc: 2 | ons: 
> 2 | 0or: 1", "creatinga" and "anand" DO NOT APPEAR in ordinary text 
> extractions, not even with Tika from the command like. But they appear 
> in the tika extraction JSON file on the machine. I'll try to 
> investigate this. 

It turns out that it happens with happens with PDFBox ExtractText only 
on the regression test machine. And with rendering too.

The cause is that the machine has no fonts, so our Liberation Sans is 
used. That font is slightly larger. So instead of rendering like this

We get this

And text extraction uses these positions too.

The appearance of "creatinga" in the "B" column of the excel file is 
because there are 3 more than in the "A" run.

So we should install fonts on the test machine, see

https://pdfbox.apache.org/3.0/faq.html#what-fonts-do-i-need-on-my-system%3F

Tilman

Re: PDFBox 2.0.33 release

Posted by Tilman Hausherr <TH...@t-online.de>.
On 07.01.2025 14:10, Tilman Hausherr wrote:
> latest:
>
> https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-6.tar.xz 

So this is pretty good now. Here's what I found:

- superscript degradation ("1 coupled" becomes "1coupled"): annoying, 
but should be solved separately some day with an algorithm improvement. 
Having correct space detection in ordinary texts has a higher priority.

- spaced texts degradation ("METAMORPHOSE" becomes "M E T A M O R P H O 
S E"): that's because these texts look like that in the original.

- angled degradation: these are differences, but both extractions are 
bad. That's what the angle option is for (maybe use this option in the 
future?)

- mysterious: govdocs1/838/838013.pdf has "ion: 4 | name: 4 | creatinga: 
3 | ram: 3 | anand: 2 | jec: 2 | message: 2 | oc: 2 | ons: 2 | 0or: 1", 
"creatinga" and "anand" DO NOT APPEAR in ordinary text extractions, not 
even with Tika from the command like. But they appear in the tika 
extraction JSON file on the machine. I'll try to investigate this.

- PDFBOX-5384 - we'll probably need more time for that one.

Besides that, lots of improvements, and the tests really helped finding 
the flaws in PDFBOX-5920.

Tilman


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org


Re: PDFBox 2.0.33 release

Posted by Tilman Hausherr <TH...@t-online.de>.
latest:

https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-6.tar.xz


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org


Re: PDFBox 2.0.33 release

Posted by Tilman Hausherr <TH...@t-online.de>.
On 06.01.2025 10:19, Tilman Hausherr wrote:
>>
>>
>> https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-4.tar.xz 
>>
>
>
> This is starting to look much better. I'm doing another "B" run that 
> includes the last change from PDFBOX-5920. It does not yet include the 
> revert from PDFBOX-5384. 

https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-5.tar.xz

Re: PDFBox 2.0.33 release

Posted by Tilman Hausherr <TH...@t-online.de>.
> On 05.01.2025 18:17, Tilman Hausherr wrote:
>> The last commit also fixes the changes in 2 italian files, e.g. 
>> P5ZY75DGZMAP3VYLGTWRWTIOZN7IPXJO. However it turns out that their 
>> text extraction was terrible before. The "La storia" part below the 
>> blue bar in fine print is also difficult to read for a human.
>>
>> I'm gonna start another "B" run so we'll have less regressions. 
>
>
> https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-4.tar.xz


This is starting to look much better. I'm doing another "B" run that 
includes the last change from PDFBOX-5920. It does not yet include the 
revert from PDFBOX-5384.

Tilman


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org


Re: PDFBox 2.0.33 release

Posted by Tilman Hausherr <TH...@t-online.de>.
On 05.01.2025 18:17, Tilman Hausherr wrote:
> The last commit also fixes the changes in 2 italian files, e.g. 
> P5ZY75DGZMAP3VYLGTWRWTIOZN7IPXJO. However it turns out that their text 
> extraction was terrible before. The "La storia" part below the blue 
> bar in fine print is also difficult to read for a human.
>
> I'm gonna start another "B" run so we'll have less regressions. 


https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-4.tar.xz



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org


Re: PDFBox 2.0.33 release

Posted by Tilman Hausherr <TH...@t-online.de>.
The last commit also fixes the changes in 2 italian files, e.g. 
P5ZY75DGZMAP3VYLGTWRWTIOZN7IPXJO. However it turns out that their text 
extraction was terrible before. The "La storia" part below the blue bar 
in fine print is also difficult to read for a human.

I'm gonna start another "B" run so we'll have less regressions.

Tilman


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org


Re: PDFBox 2.0.33 release

Posted by Andreas Lehmkühler <an...@lehmi.de.INVALID>.
Hi,

thanks for running the tests.

I'm going to have a look as well.

Andreas

Am 05.01.25 um 13:47 schrieb Tilman Hausherr:
> On 04.01.2025 21:26, Tilman Hausherr wrote:
>>>>
>>>> After that, I'll do another "B" run but with ActualText disabled, 
>>>> because this is responsible for most of the differences. Some are 
>>>> improvements, some are not.
>>
>>
>> https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-3.tar.xz 
> 
> Currently investigating 862271, 054514 p6, 
> Y5U2XZCKG2U6TO3FC36NCGOZECHQA2PY p39 and GQBOZYKKDEXTGJKCNZMYB5NIGXGDVTX2.
> 
> The regressions seem to be related to the changes in getSpaceWidth(). I 
> have created a reduced file of 862271.
> 
> Tilman
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: dev-help@pdfbox.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org


Re: PDFBox 2.0.33 release

Posted by Tilman Hausherr <TH...@t-online.de>.
On 04.01.2025 21:26, Tilman Hausherr wrote:
>>>
>>> After that, I'll do another "B" run but with ActualText disabled, 
>>> because this is responsible for most of the differences. Some are 
>>> improvements, some are not.
>
>
> https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-3.tar.xz 

Currently investigating 862271, 054514 p6, 
Y5U2XZCKG2U6TO3FC36NCGOZECHQA2PY p39 and GQBOZYKKDEXTGJKCNZMYB5NIGXGDVTX2.

The regressions seem to be related to the changes in getSpaceWidth(). I 
have created a reduced file of 862271.

Tilman


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org


Re: PDFBox 2.0.33 release

Posted by Tilman Hausherr <TH...@t-online.de>.
>>
>> After that, I'll do another "B" run but with ActualText disabled, 
>> because this is responsible for most of the differences. Some are 
>> improvements, some are not.


https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-3.tar.xz


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org


Re: PDFBox 2.0.33 release

Posted by Tilman Hausherr <TH...@t-online.de>.
On 04.01.2025 14:27, Tilman Hausherr wrote:
> On 04.01.2025 10:46, Tilman Hausherr wrote:
>> https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33.tar.xz
>>
>> definitively some work to do; one exception in CMap and some content 
>> differences / losses. 
>
>
> I'm currently doing another "B" run to be sure that there are no 
> further exceptions.

https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-2.tar.xz

Tilman



>
> After that, I'll do another "B" run but with ActualText disabled, 
> because this is responsible for most of the differences. Some are 
> improvements, some are not.




---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org


Re: PDFBox 2.0.33 release

Posted by Tilman Hausherr <TH...@t-online.de>.
On 04.01.2025 10:46, Tilman Hausherr wrote:
> https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33.tar.xz
>
> definitively some work to do; one exception in CMap and some content 
> differences / losses. 


I'm currently doing another "B" run to be sure that there are no further 
exceptions.

After that, I'll do another "B" run but with ActualText disabled, 
because this is responsible for most of the differences. Some are 
improvements, some are not.

Tilman


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org