You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pdfbox.apache.org by Andrew Phillips <an...@gmail.com> on 2013/12/06 23:05:07 UTC

Issue with PDFTextStripper.java and text positions

Working with the PDFTextStripper.class, i found a bug in the code.  I’d love to contribute the fix, but not sure the best way to do that.   I am an experienced programmer, but have never contributed to open source activities (yet, although I should consider I take advantage of such).

So, I found in a PDF I was pulling text from by using a custom PDFTextStripper subclass that overrides writeString(String text, List<TextPosition> textPositions) that i was getting the wrong textPositions that were not lined up with the text.   I found that the test position of all “words” in a line always come over as the “last” text positions of the last word in the line.   I found the issue in the PDFTextStripper class

So here is the Code Issue:

    /**
     * Used within {@link #normalize(List, boolean, boolean)} to handle a {@link TextPosition}.
     * @return The StringBuilder that must be used when calling this method.
     */
    private StringBuilder normalizeAdd(LinkedList<WordWithTextPositions> normalized,
            StringBuilder lineBuilder, List<TextPosition> wordPositions, TextPosition text)
    {
        if (text instanceof WordSeparator) 
        {
            normalized.add(createWord(lineBuilder.toString(), wordPositions));
            lineBuilder = new StringBuilder();
            wordPositions.clear();
        }
        else 
        {
            lineBuilder.append(text.getCharacter());
            wordPositions.add(text);
        }
        return lineBuilder;
    }


When the normalizeAdd method, you create a new word passing the wordPositions.   A reference to the wordPositions is stored in the new WordWithTextPositions in the normalized linked list, but in the next line, you clear().   Since the last wordPositions was passed as a reference, the wordPositions is cleared in the WordWithTextPositions you just created.

Soo, i would suggest you do the following:

    /**
     * Used within {@link #normalize(List, boolean, boolean)} to handle a {@link TextPosition}.
     * @return The StringBuilder that must be used when calling this method.
     */
    private StringBuilder normalizeAdd(LinkedList<WordWithTextPositions> normalized,
            StringBuilder lineBuilder, List<TextPosition> wordPositions, TextPosition text)
    {
        if (text instanceof WordSeparator) 
        {
            normalized.add(createWord(lineBuilder.toString(), wordPositions));
            lineBuilder = new StringBuilder();
            wordPositions = new ArrayList<TextPosition>();
        }
        else 
        {
            lineBuilder.append(text.getCharacter());
            wordPositions.add(text);
        }
        return lineBuilder;
    }


This will fix the issue.   I would be more than happy to add this, but as I mentioned, I am not really experienced in contributing to open source projects.

Thanks!
Andy Phillips

Re: Issue with PDFTextStripper.java and text positions

Posted by Andreas Lehmkühler <an...@lehmi.de>.
Hi,

> Andrew Phillips <an...@gmail.com> hat am 9. Dezember 2013 um 15:02
> geschrieben:
>
>
> Thanks, i found that i had a mistake in my suggested fix, the correct way to
> fix is as follows (i wrote the email before trying it):
>
>     /**
>      * Used within {@link #normalize(List, boolean, boolean)} to handle a
>{@link TextPosition}.
>      * @return The StringBuilder that must be used when calling this method.
>      */
>     private StringBuilder normalizeAdd(LinkedList<WordWithTextPositions>
>normalized,
>             StringBuilder lineBuilder, List<TextPosition> wordPositions,
>TextPosition text)
>     {
>         if (text instanceof WordSeparator)
>         {
>             normalized.add(createWord(lineBuilder.toString(), new
>ArrayList<TextPosition>(wordPositions)));
>             lineBuilder = new StringBuilder();
>             wordPositions.clear();
>         }
>         else
>         {
>             lineBuilder.append(text.getCharacter());
>             wordPositions.add(text);
>         }
>         return lineBuilder;
>     }
>
>
> I’ll be more than happy to create an account and add the fix.   What is the
> link to JIRA for this project? 
Have a look at the footnote at the end of my former email ....

> Thanks!
> Andy P
>
> On Dec 9, 2013, at 5:01 AM, Andreas Lehmkühler <an...@lehmi.de> wrote:
>
> > Hi,
> >
> >> Andrew Phillips <an...@gmail.com> hat am 6. Dezember 2013 um
> >> 23:05
> >> geschrieben:
> >>
> >>
> >> Working with the PDFTextStripper.class, i found a bug in the code.  I’d
> >> love
> >> to contribute the fix, but not sure the best way to do that.   I am an
> >> experienced programmer, but have never contributed to open source
> >> activities
> >> (yet, although I should consider I take advantage of such).
> >
> > Thanks for your interest in PDFBox and your offer to help. We are using JIRA
> > [1]
> > to handle any changes,
> > such as issues, improvements etc. YOu have to create an user (it's free) and
> > create an issue. Choose a
> > reasonable title, add a description and attach a sample pdf if possible.
> > Patches
> > should be created as
> > diff against the current trunk and attached to the issue as well. That's it.
> >
> >>
> >> So, I found in a PDF I was pulling text from by using a custom
> >> PDFTextStripper
> >> subclass that overrides writeString(String text, List<TextPosition>
> >> textPositions) that i was getting the wrong textPositions that were not
> >> lined
> >> up with the text.   I found that the test position of all “words” in a line
> >> always come over as the “last” text positions of the last word in the line.
> >>  I found the issue in the PDFTextStripper class
> >>
> >> So here is the Code Issue:
> >>
> >>      /**
> >>       * Used within {@link #normalize(List, boolean, boolean)} to handle a
> >> {@link TextPosition}.
> >>       * @return The StringBuilder that must be used when calling this
> >>method.
> >>       */
> >>      private StringBuilder normalizeAdd(LinkedList<WordWithTextPositions>
> >> normalized,
> >>              StringBuilder lineBuilder, List<TextPosition> wordPositions,
> >> TextPosition text)
> >>      {
> >>          if (text instanceof WordSeparator)
> >>          {
> >>              normalized.add(createWord(lineBuilder.toString(),
> >>wordPositions));
> >>              lineBuilder = new StringBuilder();
> >>              wordPositions.clear();
> >>          }
> >>          else
> >>          {
> >>              lineBuilder.append(text.getCharacter());
> >>              wordPositions.add(text);
> >>          }
> >>          return lineBuilder;
> >>      }
> >>
> >>
> >> When the normalizeAdd method, you create a new word passing the
> >> wordPositions.   A reference to the wordPositions is stored in the new
> >> WordWithTextPositions in the normalized linked list, but in the next line,
> >> you
> >> clear().   Since the last wordPositions was passed as a reference, the
> >> wordPositions is cleared in the WordWithTextPositions you just created.
> >>
> >> Soo, i would suggest you do the following:
> >>
> >>      /**
> >>       * Used within {@link #normalize(List, boolean, boolean)} to handle a
> >> {@link TextPosition}.
> >>       * @return The StringBuilder that must be used when calling this
> >>method.
> >>       */
> >>      private StringBuilder normalizeAdd(LinkedList<WordWithTextPositions>
> >> normalized,
> >>              StringBuilder lineBuilder, List<TextPosition> wordPositions,
> >> TextPosition text)
> >>      {
> >>          if (text instanceof WordSeparator)
> >>          {
> >>              normalized.add(createWord(lineBuilder.toString(),
> >>wordPositions));
> >>              lineBuilder = new StringBuilder();
> >>              wordPositions = new ArrayList<TextPosition>();
> >>          }
> >>          else
> >>          {
> >>              lineBuilder.append(text.getCharacter());
> >>              wordPositions.add(text);
> >>          }
> >>          return lineBuilder;
> >>      }
> >>
> >>
> >> This will fix the issue.   I would be more than happy to add this, but as I
> >> mentioned, I am not really experienced in contributing to open source
> >> projects.
> >
> > Sounds reasonable!
> >
> >> Thanks!
> >> Andy Phillips
> >
> > BR
> > Andreas Lehmkühler
> >
> >
> > [1] https://issues.apache.org/jira/browse/PDFBOX
>

BR
Andreas Lehmkühler

Re: Issue with PDFTextStripper.java and text positions

Posted by Andrew Phillips <an...@gmail.com>.
Thanks, i found that i had a mistake in my suggested fix, the correct way to fix is as follows (i wrote the email before trying it):

    /**
     * Used within {@link #normalize(List, boolean, boolean)} to handle a {@link TextPosition}.
     * @return The StringBuilder that must be used when calling this method.
     */
    private StringBuilder normalizeAdd(LinkedList<WordWithTextPositions> normalized,
            StringBuilder lineBuilder, List<TextPosition> wordPositions, TextPosition text)
    {
        if (text instanceof WordSeparator) 
        {
            normalized.add(createWord(lineBuilder.toString(), new ArrayList<TextPosition>(wordPositions)));
            lineBuilder = new StringBuilder();
            wordPositions.clear();
        }
        else 
        {
            lineBuilder.append(text.getCharacter());
            wordPositions.add(text);
        }
        return lineBuilder;
    }


I’ll be more than happy to create an account and add the fix.   What is the link to JIRA for this project?  

Thanks!
Andy P

On Dec 9, 2013, at 5:01 AM, Andreas Lehmkühler <an...@lehmi.de> wrote:

> Hi,
> 
>> Andrew Phillips <an...@gmail.com> hat am 6. Dezember 2013 um 23:05
>> geschrieben:
>> 
>> 
>> Working with the PDFTextStripper.class, i found a bug in the code.  I’d love
>> to contribute the fix, but not sure the best way to do that.   I am an
>> experienced programmer, but have never contributed to open source activities
>> (yet, although I should consider I take advantage of such).
> 
> Thanks for your interest in PDFBox and your offer to help. We are using JIRA [1]
> to handle any changes,
> such as issues, improvements etc. YOu have to create an user (it's free) and
> create an issue. Choose a
> reasonable title, add a description and attach a sample pdf if possible. Patches
> should be created as
> diff against the current trunk and attached to the issue as well. That's it.
> 
>> 
>> So, I found in a PDF I was pulling text from by using a custom PDFTextStripper
>> subclass that overrides writeString(String text, List<TextPosition>
>> textPositions) that i was getting the wrong textPositions that were not lined
>> up with the text.   I found that the test position of all “words” in a line
>> always come over as the “last” text positions of the last word in the line. 
>>  I found the issue in the PDFTextStripper class
>> 
>> So here is the Code Issue:
>> 
>>      /**
>>       * Used within {@link #normalize(List, boolean, boolean)} to handle a
>> {@link TextPosition}.
>>       * @return The StringBuilder that must be used when calling this method.
>>       */
>>      private StringBuilder normalizeAdd(LinkedList<WordWithTextPositions>
>> normalized,
>>              StringBuilder lineBuilder, List<TextPosition> wordPositions,
>> TextPosition text)
>>      {
>>          if (text instanceof WordSeparator)
>>          {
>>              normalized.add(createWord(lineBuilder.toString(), wordPositions));
>>              lineBuilder = new StringBuilder();
>>              wordPositions.clear();
>>          }
>>          else
>>          {
>>              lineBuilder.append(text.getCharacter());
>>              wordPositions.add(text);
>>          }
>>          return lineBuilder;
>>      }
>> 
>> 
>> When the normalizeAdd method, you create a new word passing the
>> wordPositions.   A reference to the wordPositions is stored in the new
>> WordWithTextPositions in the normalized linked list, but in the next line, you
>> clear().   Since the last wordPositions was passed as a reference, the
>> wordPositions is cleared in the WordWithTextPositions you just created.
>> 
>> Soo, i would suggest you do the following:
>> 
>>      /**
>>       * Used within {@link #normalize(List, boolean, boolean)} to handle a
>> {@link TextPosition}.
>>       * @return The StringBuilder that must be used when calling this method.
>>       */
>>      private StringBuilder normalizeAdd(LinkedList<WordWithTextPositions>
>> normalized,
>>              StringBuilder lineBuilder, List<TextPosition> wordPositions,
>> TextPosition text)
>>      {
>>          if (text instanceof WordSeparator)
>>          {
>>              normalized.add(createWord(lineBuilder.toString(), wordPositions));
>>              lineBuilder = new StringBuilder();
>>              wordPositions = new ArrayList<TextPosition>();
>>          }
>>          else
>>          {
>>              lineBuilder.append(text.getCharacter());
>>              wordPositions.add(text);
>>          }
>>          return lineBuilder;
>>      }
>> 
>> 
>> This will fix the issue.   I would be more than happy to add this, but as I
>> mentioned, I am not really experienced in contributing to open source
>> projects.
> 
> Sounds reasonable!
> 
>> Thanks!
>> Andy Phillips
> 
> BR
> Andreas Lehmkühler
> 
> 
> [1] https://issues.apache.org/jira/browse/PDFBOX


Re: Issue with PDFTextStripper.java and text positions

Posted by Andreas Lehmkühler <an...@lehmi.de>.
Hi,

> Andrew Phillips <an...@gmail.com> hat am 6. Dezember 2013 um 23:05
> geschrieben:
>
>
> Working with the PDFTextStripper.class, i found a bug in the code.  I’d love
> to contribute the fix, but not sure the best way to do that.   I am an
> experienced programmer, but have never contributed to open source activities
> (yet, although I should consider I take advantage of such).

Thanks for your interest in PDFBox and your offer to help. We are using JIRA [1]
to handle any changes,
such as issues, improvements etc. YOu have to create an user (it's free) and
create an issue. Choose a
reasonable title, add a description and attach a sample pdf if possible. Patches
should be created as
diff against the current trunk and attached to the issue as well. That's it.

>
> So, I found in a PDF I was pulling text from by using a custom PDFTextStripper
> subclass that overrides writeString(String text, List<TextPosition>
> textPositions) that i was getting the wrong textPositions that were not lined
> up with the text.   I found that the test position of all “words” in a line
> always come over as the “last” text positions of the last word in the line. 
>  I found the issue in the PDFTextStripper class
>
> So here is the Code Issue:
>
>     /**
>      * Used within {@link #normalize(List, boolean, boolean)} to handle a
>{@link TextPosition}.
>      * @return The StringBuilder that must be used when calling this method.
>      */
>     private StringBuilder normalizeAdd(LinkedList<WordWithTextPositions>
>normalized,
>             StringBuilder lineBuilder, List<TextPosition> wordPositions,
>TextPosition text)
>     {
>         if (text instanceof WordSeparator)
>         {
>             normalized.add(createWord(lineBuilder.toString(), wordPositions));
>             lineBuilder = new StringBuilder();
>             wordPositions.clear();
>         }
>         else
>         {
>             lineBuilder.append(text.getCharacter());
>             wordPositions.add(text);
>         }
>         return lineBuilder;
>     }
>
>
> When the normalizeAdd method, you create a new word passing the
> wordPositions.   A reference to the wordPositions is stored in the new
> WordWithTextPositions in the normalized linked list, but in the next line, you
> clear().   Since the last wordPositions was passed as a reference, the
> wordPositions is cleared in the WordWithTextPositions you just created.
>
> Soo, i would suggest you do the following:
>
>     /**
>      * Used within {@link #normalize(List, boolean, boolean)} to handle a
>{@link TextPosition}.
>      * @return The StringBuilder that must be used when calling this method.
>      */
>     private StringBuilder normalizeAdd(LinkedList<WordWithTextPositions>
>normalized,
>             StringBuilder lineBuilder, List<TextPosition> wordPositions,
>TextPosition text)
>     {
>         if (text instanceof WordSeparator)
>         {
>             normalized.add(createWord(lineBuilder.toString(), wordPositions));
>             lineBuilder = new StringBuilder();
>             wordPositions = new ArrayList<TextPosition>();
>         }
>         else
>         {
>             lineBuilder.append(text.getCharacter());
>             wordPositions.add(text);
>         }
>         return lineBuilder;
>     }
>
>
> This will fix the issue.   I would be more than happy to add this, but as I
> mentioned, I am not really experienced in contributing to open source
> projects.

Sounds reasonable!

> Thanks!
> Andy Phillips

BR
Andreas Lehmkühler


[1] https://issues.apache.org/jira/browse/PDFBOX