You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pdfbox.apache.org by Tilman Hausherr <TH...@t-online.de> on 2025/01/04 09:46:25 UTC
Re: PDFBox 2.0.33 release
https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33.tar.xz
definitively some work to do; one exception in CMap and some content
differences / losses.
The 2.0.33 snapshot version is from 31.12, I forgot to build 2.0.
However there have been no changes related parsing or text extraction
Tilman
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: PDFBox 2.0.33 release
Posted by Andreas Lehmkühler <an...@lehmi.de.INVALID>.
Am 12.01.25 um 13:58 schrieb sahyoun@fileaffairs.de:
> Am Sonntag, dem 12.01.2025 um 13:24 +0100 schrieb Andreas Lehmkühler:
>>
>>
>> Am 08.01.25 um 04:56 schrieb Tilman Hausherr:
>>> On 07.01.2025 15:00, Tilman Hausherr wrote:
>>>> - mysterious: govdocs1/838/838013.pdf has "ion: 4 | name: 4 |
>>>> creatinga: 3 | ram: 3 | anand: 2 | jec: 2 | message: 2 | oc: 2 |
>>>> ons:
>>>> 2 | 0or: 1", "creatinga" and "anand" DO NOT APPEAR in ordinary
>>>> text
>>>> extractions, not even with Tika from the command like. But they
>>>> appear
>>>> in the tika extraction JSON file on the machine. I'll try to
>>>> investigate this.
>>>
>>> It turns out that it happens with happens with PDFBox ExtractText
>>> only
>>> on the regression test machine. And with rendering too.
>>>
>>> The cause is that the machine has no fonts, so our Liberation Sans
>>> is
>>> used. That font is slightly larger. So instead of rendering like
>>> this
>>>
>>> We get this
>>>
>>> And text extraction uses these positions too.
>>>
>>> The appearance of "creatinga" in the "B" column of the excel file
>>> is
>>> because there are 3 more than in the "A" run.
>>>
>>> So we should install fonts on the test machine, see
>> I've tried to do so, but my user isn't a member of the sudo group :-o
>
> I added you to the group - please give it a try and let me know if
> there are issues
Thanks for the fast response. Works like a charm. I've installed the
mentioned ttf-mscorefonts-installer package, so that the missing fonts
are available now.
Hope this helps
Andreas
>
> BR
> Maruan
>
>>
>> @Maruan or @Tim
>> Please install those missing fonts or add me to the sudo group
>>
>> Thanks
>> Andreas
>>>
>>> https://pdfbox.apache.org/3.0/faq.html#what-fonts-do-i-need-on-my-system%3F
>>>
>>> Tilman
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
>> For additional commands, e-mail: dev-help@pdfbox.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: dev-help@pdfbox.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: PDFBox 2.0.33 release
Posted by "sahyoun@fileaffairs.de" <sa...@fileaffairs.de>.
Am Sonntag, dem 12.01.2025 um 13:24 +0100 schrieb Andreas Lehmkühler:
>
>
> Am 08.01.25 um 04:56 schrieb Tilman Hausherr:
> > On 07.01.2025 15:00, Tilman Hausherr wrote:
> > > - mysterious: govdocs1/838/838013.pdf has "ion: 4 | name: 4 |
> > > creatinga: 3 | ram: 3 | anand: 2 | jec: 2 | message: 2 | oc: 2 |
> > > ons:
> > > 2 | 0or: 1", "creatinga" and "anand" DO NOT APPEAR in ordinary
> > > text
> > > extractions, not even with Tika from the command like. But they
> > > appear
> > > in the tika extraction JSON file on the machine. I'll try to
> > > investigate this.
> >
> > It turns out that it happens with happens with PDFBox ExtractText
> > only
> > on the regression test machine. And with rendering too.
> >
> > The cause is that the machine has no fonts, so our Liberation Sans
> > is
> > used. That font is slightly larger. So instead of rendering like
> > this
> >
> > We get this
> >
> > And text extraction uses these positions too.
> >
> > The appearance of "creatinga" in the "B" column of the excel file
> > is
> > because there are 3 more than in the "A" run.
> >
> > So we should install fonts on the test machine, see
> I've tried to do so, but my user isn't a member of the sudo group :-o
I added you to the group - please give it a try and let me know if
there are issues
BR
Maruan
>
> @Maruan or @Tim
> Please install those missing fonts or add me to the sudo group
>
> Thanks
> Andreas
> >
> > https://pdfbox.apache.org/3.0/faq.html#what-fonts-do-i-need-on-my-system%3F
> >
> > Tilman
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: dev-help@pdfbox.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: PDFBox 2.0.33 release
Posted by Andreas Lehmkühler <an...@lehmi.de.INVALID>.
Am 08.01.25 um 04:56 schrieb Tilman Hausherr:
> On 07.01.2025 15:00, Tilman Hausherr wrote:
>> - mysterious: govdocs1/838/838013.pdf has "ion: 4 | name: 4 |
>> creatinga: 3 | ram: 3 | anand: 2 | jec: 2 | message: 2 | oc: 2 | ons:
>> 2 | 0or: 1", "creatinga" and "anand" DO NOT APPEAR in ordinary text
>> extractions, not even with Tika from the command like. But they appear
>> in the tika extraction JSON file on the machine. I'll try to
>> investigate this.
>
> It turns out that it happens with happens with PDFBox ExtractText only
> on the regression test machine. And with rendering too.
>
> The cause is that the machine has no fonts, so our Liberation Sans is
> used. That font is slightly larger. So instead of rendering like this
>
> We get this
>
> And text extraction uses these positions too.
>
> The appearance of "creatinga" in the "B" column of the excel file is
> because there are 3 more than in the "A" run.
>
> So we should install fonts on the test machine, see
I've tried to do so, but my user isn't a member of the sudo group :-o
@Maruan or @Tim
Please install those missing fonts or add me to the sudo group
Thanks
Andreas
>
> https://pdfbox.apache.org/3.0/faq.html#what-fonts-do-i-need-on-my-system%3F
>
> Tilman
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: PDFBox 2.0.33 release
Posted by Tilman Hausherr <TH...@t-online.de>.
On 07.01.2025 15:00, Tilman Hausherr wrote:
> - mysterious: govdocs1/838/838013.pdf has "ion: 4 | name: 4 |
> creatinga: 3 | ram: 3 | anand: 2 | jec: 2 | message: 2 | oc: 2 | ons:
> 2 | 0or: 1", "creatinga" and "anand" DO NOT APPEAR in ordinary text
> extractions, not even with Tika from the command like. But they appear
> in the tika extraction JSON file on the machine. I'll try to
> investigate this.
It turns out that it happens with happens with PDFBox ExtractText only
on the regression test machine. And with rendering too.
The cause is that the machine has no fonts, so our Liberation Sans is
used. That font is slightly larger. So instead of rendering like this
We get this
And text extraction uses these positions too.
The appearance of "creatinga" in the "B" column of the excel file is
because there are 3 more than in the "A" run.
So we should install fonts on the test machine, see
https://pdfbox.apache.org/3.0/faq.html#what-fonts-do-i-need-on-my-system%3F
Tilman
Re: PDFBox 2.0.33 release
Posted by Tilman Hausherr <TH...@t-online.de>.
On 07.01.2025 14:10, Tilman Hausherr wrote:
> latest:
>
> https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-6.tar.xz
So this is pretty good now. Here's what I found:
- superscript degradation ("1 coupled" becomes "1coupled"): annoying,
but should be solved separately some day with an algorithm improvement.
Having correct space detection in ordinary texts has a higher priority.
- spaced texts degradation ("METAMORPHOSE" becomes "M E T A M O R P H O
S E"): that's because these texts look like that in the original.
- angled degradation: these are differences, but both extractions are
bad. That's what the angle option is for (maybe use this option in the
future?)
- mysterious: govdocs1/838/838013.pdf has "ion: 4 | name: 4 | creatinga:
3 | ram: 3 | anand: 2 | jec: 2 | message: 2 | oc: 2 | ons: 2 | 0or: 1",
"creatinga" and "anand" DO NOT APPEAR in ordinary text extractions, not
even with Tika from the command like. But they appear in the tika
extraction JSON file on the machine. I'll try to investigate this.
- PDFBOX-5384 - we'll probably need more time for that one.
Besides that, lots of improvements, and the tests really helped finding
the flaws in PDFBOX-5920.
Tilman
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: PDFBox 2.0.33 release
Posted by Tilman Hausherr <TH...@t-online.de>.
latest:
https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-6.tar.xz
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: PDFBox 2.0.33 release
Posted by Tilman Hausherr <TH...@t-online.de>.
On 06.01.2025 10:19, Tilman Hausherr wrote:
>>
>>
>> https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-4.tar.xz
>>
>
>
> This is starting to look much better. I'm doing another "B" run that
> includes the last change from PDFBOX-5920. It does not yet include the
> revert from PDFBOX-5384.
https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-5.tar.xz
Re: PDFBox 2.0.33 release
Posted by Tilman Hausherr <TH...@t-online.de>.
> On 05.01.2025 18:17, Tilman Hausherr wrote:
>> The last commit also fixes the changes in 2 italian files, e.g.
>> P5ZY75DGZMAP3VYLGTWRWTIOZN7IPXJO. However it turns out that their
>> text extraction was terrible before. The "La storia" part below the
>> blue bar in fine print is also difficult to read for a human.
>>
>> I'm gonna start another "B" run so we'll have less regressions.
>
>
> https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-4.tar.xz
This is starting to look much better. I'm doing another "B" run that
includes the last change from PDFBOX-5920. It does not yet include the
revert from PDFBOX-5384.
Tilman
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: PDFBox 2.0.33 release
Posted by Tilman Hausherr <TH...@t-online.de>.
On 05.01.2025 18:17, Tilman Hausherr wrote:
> The last commit also fixes the changes in 2 italian files, e.g.
> P5ZY75DGZMAP3VYLGTWRWTIOZN7IPXJO. However it turns out that their text
> extraction was terrible before. The "La storia" part below the blue
> bar in fine print is also difficult to read for a human.
>
> I'm gonna start another "B" run so we'll have less regressions.
https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-4.tar.xz
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: PDFBox 2.0.33 release
Posted by Tilman Hausherr <TH...@t-online.de>.
The last commit also fixes the changes in 2 italian files, e.g.
P5ZY75DGZMAP3VYLGTWRWTIOZN7IPXJO. However it turns out that their text
extraction was terrible before. The "La storia" part below the blue bar
in fine print is also difficult to read for a human.
I'm gonna start another "B" run so we'll have less regressions.
Tilman
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: PDFBox 2.0.33 release
Posted by Andreas Lehmkühler <an...@lehmi.de.INVALID>.
Hi,
thanks for running the tests.
I'm going to have a look as well.
Andreas
Am 05.01.25 um 13:47 schrieb Tilman Hausherr:
> On 04.01.2025 21:26, Tilman Hausherr wrote:
>>>>
>>>> After that, I'll do another "B" run but with ActualText disabled,
>>>> because this is responsible for most of the differences. Some are
>>>> improvements, some are not.
>>
>>
>> https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-3.tar.xz
>
> Currently investigating 862271, 054514 p6,
> Y5U2XZCKG2U6TO3FC36NCGOZECHQA2PY p39 and GQBOZYKKDEXTGJKCNZMYB5NIGXGDVTX2.
>
> The regressions seem to be related to the changes in getSpaceWidth(). I
> have created a reduced file of 862271.
>
> Tilman
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: dev-help@pdfbox.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: PDFBox 2.0.33 release
Posted by Tilman Hausherr <TH...@t-online.de>.
On 04.01.2025 21:26, Tilman Hausherr wrote:
>>>
>>> After that, I'll do another "B" run but with ActualText disabled,
>>> because this is responsible for most of the differences. Some are
>>> improvements, some are not.
>
>
> https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-3.tar.xz
Currently investigating 862271, 054514 p6,
Y5U2XZCKG2U6TO3FC36NCGOZECHQA2PY p39 and GQBOZYKKDEXTGJKCNZMYB5NIGXGDVTX2.
The regressions seem to be related to the changes in getSpaceWidth(). I
have created a reduced file of 862271.
Tilman
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: PDFBox 2.0.33 release
Posted by Tilman Hausherr <TH...@t-online.de>.
>>
>> After that, I'll do another "B" run but with ActualText disabled,
>> because this is responsible for most of the differences. Some are
>> improvements, some are not.
https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-3.tar.xz
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: PDFBox 2.0.33 release
Posted by Tilman Hausherr <TH...@t-online.de>.
On 04.01.2025 14:27, Tilman Hausherr wrote:
> On 04.01.2025 10:46, Tilman Hausherr wrote:
>> https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33.tar.xz
>>
>> definitively some work to do; one exception in CMap and some content
>> differences / losses.
>
>
> I'm currently doing another "B" run to be sure that there are no
> further exceptions.
https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33-2.tar.xz
Tilman
>
> After that, I'll do another "B" run but with ActualText disabled,
> because this is responsible for most of the differences. Some are
> improvements, some are not.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org
Re: PDFBox 2.0.33 release
Posted by Tilman Hausherr <TH...@t-online.de>.
On 04.01.2025 10:46, Tilman Hausherr wrote:
> https://home.snafu.de/tilman/tmp/reports_pdfbox_2.0.32_vs_2.0.33.tar.xz
>
> definitively some work to do; one exception in CMap and some content
> differences / losses.
I'm currently doing another "B" run to be sure that there are no further
exceptions.
After that, I'll do another "B" run but with ActualText disabled,
because this is responsible for most of the differences. Some are
improvements, some are not.
Tilman
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: dev-help@pdfbox.apache.org