You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pdfbox.apache.org by Zeev Sands <ze...@gmail.com> on 2014/04/19 20:57:10 UTC
Discrepancy between rendered and extracted characters.
Hello,
I am new to pdfbox and pdf format in general, so I apologize, if my
questions are uninformed.
I am trying to extract text from a pdf file and some of the characters
correctly rendered on the screen (via acrobat) are coming out funny. 99%
of the characters from the pdf are extracted correctly, but in one
place, for example, what appears like a letter X on the screen is
extracted as '}{', in another place two paragraph symbols (¶¶) are
extracted as 'iii!'.
After poking around PDFStreamEngine and PDFStreamParser, I can see that
the string rendering as 'X' on the screen is coming out of the pdf
stream as <007D007B0020> which is 00 + '}' + 00 + '{' 00 + ' ', so that
is what is extracted, yet on the screen it is clearly an X with the
backward slash thicker then the forward slash, set in a nice serif font,
so as far as I understand, inside the pdf it *is* '}{ ', but it renders
as 'X' on the screen.
Is there any way I can get that X? or more importantly those¶¶? Where in
the pdfbox code can I look to figure it out? Perhaps, I am missing the
basic understanding of how character rendering works in pdf. Could
someone, please, point me in the right direction? references, links etc?
Thank you,
-ZS
Re: Discrepancy between rendered and extracted characters.
Posted by Tres Finocchiaro <tr...@gmail.com>.
In my experience these issues with OCR are unavoidable. If you have
frequent issues with certain characters you could replace them with a regex
string replacement, but unfortunately to do proper context sensitive
spelling and grammatical corrections in combination with OCR would have to
be invested at the scanning end, or you would have to find a third party
plugin with reactively corrects this.
What I've seen more of is words like MEMO translating to "M EM 0".
Spinning your own if/else helpers for the common occurrences are probably
the fastest way from the receiving end unless someone knows of a better
way... :)
-Tres
Re: Discrepancy between rendered and extracted characters.
Posted by Zeev Sands <ze...@gmail.com>.
On 04/19/2014 03:28 PM, Tres Finocchiaro wrote:
> @ZS,
>
> Is the text part of the original PDF or has it been created with OCR?
>
> That sounds similar to an OCR issue where the scanner that scanned in the
> document made the mistake.
>
> -Tres
>
I obtained the document from a 3rd party, so I am not sure, but looking
at the "producer" field in it's meta data I see 'Adobe Acrobat Pro
11.0.6 Paper Capture Plug-in'. So it appears, you are correct, the
document might have been scanned. Ouch!
What are my options for extracting an error-free text? Using a better
OCR software? I have just started using pdfbox, so I haven't compiled
any statistics on the variety or frequency of these errors, How do
people deal with this issue? Is it possible to write a set of rules for
a few characters?
Thank you,
-ZS
Re: Discrepancy between rendered and extracted characters.
Posted by Tres Finocchiaro <tr...@gmail.com>.
@ZS,
Is the text part of the original PDF or has it been created with OCR?
That sounds similar to an OCR issue where the scanner that scanned in the
document made the mistake.
-Tres
Re: Discrepancy between rendered and extracted characters.
Posted by Tilman Hausherr <TH...@t-online.de>.
Indeed. If you can, use 600dpi, and take care that the paper is properly
aligned. And clean your scanner.
Tilman
Am 22.04.2014 15:26, schrieb Tres Finocchiaro:
> I've also noticed scanning the document at higher resolutions helps with
> these types of issues as well, but I've never had a "perfect" OCR scan.
> They always have spaces or unrecognized characters in some place of the
> document. :)
>
> -Tres
>
Re: Discrepancy between rendered and extracted characters.
Posted by Tres Finocchiaro <tr...@gmail.com>.
I've also noticed scanning the document at higher resolutions helps with
these types of issues as well, but I've never had a "perfect" OCR scan.
They always have spaces or unrecognized characters in some place of the
document. :)
-Tres
Re: Discrepancy between rendered and extracted characters.
Posted by Zeev Sands <ze...@gmail.com>.
When I perform the test you described, the pasted text matches the text
extracted by pdfbox (both with bad characters).
In this particular case, I now believe that the document was scanned. A
supporting evidence of that is a horizontal line rendered as
'==================' is extracted as '==~======~===~===' (using both the
acrobat test and pdfbox). This seems random and consistent with scanner
errors.
You make an interesting point, perhaps the font on the scanning end has
no way of recognizing the character (¶). This character in particular is
missed consistently. '¶' is extracted as 'ii' and '¶¶' as 'iii!'. This
gives me some hope that I might be able to do a better job on the
images in the file by using a different OCR software. Although, 'Acrobat
11.0.6 Paper Capture Plug-in' that presumably produced the document
sounds serious...hopes diminishing.
Thank you to all for quick responses,
Best Regards,
-ZS
On 04/22/2014 07:01 AM, Andreas Lehmkühler wrote:
>
> There are several reasons for that behaviour.
> - the pdf doesn't provide all necessary information to map the rendered text to
> a readable one
> - the pdf uses a Type3 font, which most likely can't be extracted as text
> - there is a bug or a special case isn't yet supported within PDFBox
>
> But without having a hand on the pdf , these are just guesses.
>
> First of all, perform the Acrobat test. To do so, open the pdf in quesiotn
> using acrobat and try to copy and paste the text. What do you get?
Re: Discrepancy between rendered and extracted characters.
Posted by Andreas Lehmkühler <an...@lehmi.de>.
Hi,
> Zeev Sands <ze...@gmail.com> hat am 19. April 2014 um 20:57 geschrieben:
>
>
> Hello,
>
> I am new to pdfbox and pdf format in general, so I apologize, if my
> questions are uninformed.
>
> I am trying to extract text from a pdf file and some of the characters
> correctly rendered on the screen (via acrobat) are coming out funny. 99%
> of the characters from the pdf are extracted correctly, but in one
> place, for example, what appears like a letter X on the screen is
> extracted as '}{', in another place two paragraph symbols (¶¶) are
> extracted as 'iii!'.
>
> After poking around PDFStreamEngine and PDFStreamParser, I can see that
> the string rendering as 'X' on the screen is coming out of the pdf
> stream as <007D007B0020> which is 00 + '}' + 00 + '{' 00 + ' ', so that
> is what is extracted, yet on the screen it is clearly an X with the
> backward slash thicker then the forward slash, set in a nice serif font,
> so as far as I understand, inside the pdf it *is* '}{ ', but it renders
> as 'X' on the screen.
>
> Is there any way I can get that X? or more importantly those¶¶? Where in
> the pdfbox code can I look to figure it out? Perhaps, I am missing the
> basic understanding of how character rendering works in pdf. Could
> someone, please, point me in the right direction? references, links etc?
There are several reasons for that behaviour.
- the pdf doesn't provide all necessary information to map the rendered text to
a readable one
- the pdf uses a Type3 font, which most likely can't be extracted as text
- there is a bug or a special case isn't yet supported within PDFBox
But without having a hand on the pdf , these are just guesses.
First of all, perform the Acrobat test. To do so, open the pdf in quesiotn
using acrobat and try to copy and paste the text. What do you get?
> Thank you,
> -ZS
BR
Andreas Lehmkühler