You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pdfbox.apache.org by Andreas Hubold <an...@coremedia.com> on 2018/03/20 21:35:38 UTC
OutOfMemoryError in PDExtendedGraphicsState#getLineDashPattern
Hi,
I'm getting an OutOfMemoryError from PDFBox when parsing a certain PDF
using the Apache Tika App v 1.17 - which uses PDFBox 2.0.8 internally.
This is reproducible even with 8GB heap.
The OutOfMemoryError happens in
org.apache.pdfbox.pdmodel.graphics.state.PDExtendedGraphicsState#getLineDashPattern,
which contains this piece of suspicious code:
COSArray dp = (COSArray) dict.getDictionaryObject( COSName.D );
if( dp != null )
{
COSArray array = new COSArray();
dp.addAll(dp);
The last line seems to wrong? It appends all elements from 'dp' to 'dp'
again, effectively duplicating the elements in the list. Maybe it should
be 'array.addAll(dp)' or something like that?
Can you confirm this being a bug? Should I open a JIRA ticket for this
problem?
Do you know a workaround to avoid the crash, e.g. an option to skip some
parts of the file for text extraction?
Here's the stacktrace:
[Full GC (Allocation Failure) 4225609K->4224664K(5989888K), 32,9544686
secs]
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3210)
at java.util.Arrays.copyOf(Arrays.java:3181)
at java.util.ArrayList.grow(ArrayList.java:261)
at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:235)
at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:227)
at java.util.ArrayList.addAll(ArrayList.java:579)
at org.apache.pdfbox.cos.COSArray.addAll(COSArray.java:124)
at
org.apache.pdfbox.pdmodel.graphics.state.PDExtendedGraphicsState.getLineDashPattern(PDExtendedGraphicsState.java:280)
at
org.apache.pdfbox.pdmodel.graphics.state.PDExtendedGraphicsState.copyIntoGraphicsState(PDExtendedGraphicsState.java:89)
at
org.apache.pdfbox.contentstream.operator.state.SetGraphicsStateParameters.process(SetGraphicsStateParameters.java:61)
at
org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:838)
at
org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:495)
at
org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:469)
at
org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:150)
at
org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
at
org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
at org.apache.tika.parser.pdf.PDF2XHTML.processPage(PDF2XHTML.java:147)
at
org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
at
org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:117)
at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:168)
at
org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
at
org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
at
org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)
at org.apache.tika.cli.TikaCLI$OutputType.process(TikaCLI.java:205)
at org.apache.tika.cli.TikaCLI.process(TikaCLI.java:486)
at org.apache.tika.cli.TikaCLI.main(TikaCLI.java:145)
I'm not yet sure if I can share the PDF. If needed, I can check that.
Best regards,
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
Re: OutOfMemoryError in PDExtendedGraphicsState#getLineDashPattern
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Andreas,
On 3/20/18 5:35 PM, Andreas Hubold wrote:
> I'm getting an OutOfMemoryError from PDFBox when parsing a certain
> PDF using the Apache Tika App v 1.17 - which uses PDFBox 2.0.8
> internally. This is reproducible even with 8GB heap.
>
> The OutOfMemoryError happens in
> org.apache.pdfbox.pdmodel.graphics.state.PDExtendedGraphicsState#getLi
neDashPattern,
>
>
which contains this piece of suspicious code:
>
> COSArray dp = (COSArray) dict.getDictionaryObject( COSName.D ); if(
> dp != null ) { COSArray array = new COSArray(); dp.addAll(dp);
>
> The last line seems to wrong?
That certainly looks wrong to me.
> It appends all elements from 'dp' to 'dp' again, effectively
> duplicating the elements in the list. Maybe it should be
> 'array.addAll(dp)' or something like that?
>
> Can you confirm this being a bug? Should I open a JIRA ticket for
> this problem?
>
> Do you know a workaround to avoid the crash, e.g. an option to skip
> some parts of the file for text extraction?
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQJRBAEBCAA7FiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlqxgDYdHGNocmlzQGNo
cmlzdG9waGVyc2NodWx0ei5uZXQACgkQHPApP6U8pFj/Ew/7BqHbZpfLea7necmh
zY6oOLIgLRwoarm61rWt8Kz6+Z+SGgU/8x5exQvJoZh8UhBG/sJ3OBIpdx5utMVM
/XsvEj8k0CEMPLnvhq5D+akszJbfB3GWZgwZVdhUq6tMbWKPrXVqlJ4/boLBlWYY
gOdkIkkULFuJtdk8rQ8GctbBmMnraSCyEvShLuuVOOi/m0MOMJnHIO6Ul6odWxWr
gDLVsT4UXVb6G2fDDeTx9LkadOalAFDAbSNlH+MwI/uoA3L9o9Vs7Hz8LE5pt4ds
ATBMS44hm+mk46t41VCD+dWP5adsJyZdzcZW+td0TUVGskeTHGfQ1uqDbBlFWyyA
n06sqi5xFnJvO/nCAl8lX0P8xPhJG1xi1/oF4vHAr3LzwxELE5U5oV+l2Qk06Sdc
RUNMuEyruiDlxj0Xm4xOnyy0X08RWjIp0XPyYW7DpGNIFxd+Wq/RC2ybUtSi2Ek7
2b5bd4rvk1jXdkEoBol/UB2rhNYDQUyqNPwU1ManA1coaHhqPRpDo8j4J0+ika9p
+qsdsgRqOu5oIzBHE8uLnW+ViuAuuFDNGySWgbxdelrARXGj/1MgTaFqQUKjNwHg
qFdZ9P29Kwv+oqQvJdkPpre9YoP2EJI49gV5EBakerM5/6BY+4wV03pNhtwoSL0r
tr/qb0cGpzAr+2kKZsohQYDjEa0=
=OFd7
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
Re: OutOfMemoryError in PDExtendedGraphicsState#getLineDashPattern
Posted by Andreas Hubold <an...@coremedia.com>.
Thank you, I've opened https://issues.apache.org/jira/browse/PDFBOX-4162
I will check if I can share the PDF and attach it, if possible.
Tilman Hausherr schrieb am 20.03.2018 um 22:46:
> Yes this is a bug. Yes please open a ticket and attach your PDF file.
>
> No I don't have an idea how to bypass this.
>
> Tilman
>
> Am 20.03.2018 um 22:35 schrieb Andreas Hubold:
>> Hi,
>>
>> I'm getting an OutOfMemoryError from PDFBox when parsing a certain
>> PDF using the Apache Tika App v 1.17 - which uses PDFBox 2.0.8
>> internally. This is reproducible even with 8GB heap.
>>
>> The OutOfMemoryError happens in
>> org.apache.pdfbox.pdmodel.graphics.state.PDExtendedGraphicsState#getLineDashPattern,
>> which contains this piece of suspicious code:
>>
>> COSArray dp = (COSArray) dict.getDictionaryObject( COSName.D );
>> if( dp != null )
>> {
>> COSArray array = new COSArray();
>> dp.addAll(dp);
>>
>> The last line seems to wrong? It appends all elements from 'dp' to
>> 'dp' again, effectively duplicating the elements in the list. Maybe
>> it should be 'array.addAll(dp)' or something like that?
>>
>> Can you confirm this being a bug? Should I open a JIRA ticket for
>> this problem?
>>
>> Do you know a workaround to avoid the crash, e.g. an option to skip
>> some parts of the file for text extraction?
>>
>> Here's the stacktrace:
>>
>> [Full GC (Allocation Failure) 4225609K->4224664K(5989888K),
>> 32,9544686 secs]
>> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>> at java.util.Arrays.copyOf(Arrays.java:3210)
>> at java.util.Arrays.copyOf(Arrays.java:3181)
>> at java.util.ArrayList.grow(ArrayList.java:261)
>> at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:235)
>> at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:227)
>> at java.util.ArrayList.addAll(ArrayList.java:579)
>> at org.apache.pdfbox.cos.COSArray.addAll(COSArray.java:124)
>> at
>> org.apache.pdfbox.pdmodel.graphics.state.PDExtendedGraphicsState.getLineDashPattern(PDExtendedGraphicsState.java:280)
>> at
>> org.apache.pdfbox.pdmodel.graphics.state.PDExtendedGraphicsState.copyIntoGraphicsState(PDExtendedGraphicsState.java:89)
>> at
>> org.apache.pdfbox.contentstream.operator.state.SetGraphicsStateParameters.process(SetGraphicsStateParameters.java:61)
>> at
>> org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:838)
>> at
>> org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:495)
>> at
>> org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:469)
>> at
>> org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:150)
>> at
>> org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
>> at
>> org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
>> at
>> org.apache.tika.parser.pdf.PDF2XHTML.processPage(PDF2XHTML.java:147)
>> at
>> org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
>> at
>> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
>> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:117)
>> at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:168)
>> at
>> org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
>> at
>> org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
>> at
>> org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)
>> at org.apache.tika.cli.TikaCLI$OutputType.process(TikaCLI.java:205)
>> at org.apache.tika.cli.TikaCLI.process(TikaCLI.java:486)
>> at org.apache.tika.cli.TikaCLI.main(TikaCLI.java:145)
>>
>> I'm not yet sure if I can share the PDF. If needed, I can check that.
>>
>> Best regards,
>> Andreas
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
>> For additional commands, e-mail: users-help@pdfbox.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
>
> .
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
Re: OutOfMemoryError in PDExtendedGraphicsState#getLineDashPattern
Posted by Tilman Hausherr <TH...@t-online.de>.
Yes this is a bug. Yes please open a ticket and attach your PDF file.
No I don't have an idea how to bypass this.
Tilman
Am 20.03.2018 um 22:35 schrieb Andreas Hubold:
> Hi,
>
> I'm getting an OutOfMemoryError from PDFBox when parsing a certain PDF
> using the Apache Tika App v 1.17 - which uses PDFBox 2.0.8 internally.
> This is reproducible even with 8GB heap.
>
> The OutOfMemoryError happens in
> org.apache.pdfbox.pdmodel.graphics.state.PDExtendedGraphicsState#getLineDashPattern,
> which contains this piece of suspicious code:
>
> COSArray dp = (COSArray) dict.getDictionaryObject( COSName.D );
> if( dp != null )
> {
> COSArray array = new COSArray();
> dp.addAll(dp);
>
> The last line seems to wrong? It appends all elements from 'dp' to
> 'dp' again, effectively duplicating the elements in the list. Maybe it
> should be 'array.addAll(dp)' or something like that?
>
> Can you confirm this being a bug? Should I open a JIRA ticket for this
> problem?
>
> Do you know a workaround to avoid the crash, e.g. an option to skip
> some parts of the file for text extraction?
>
> Here's the stacktrace:
>
> [Full GC (Allocation Failure) 4225609K->4224664K(5989888K),
> 32,9544686 secs]
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Arrays.java:3210)
> at java.util.Arrays.copyOf(Arrays.java:3181)
> at java.util.ArrayList.grow(ArrayList.java:261)
> at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:235)
> at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:227)
> at java.util.ArrayList.addAll(ArrayList.java:579)
> at org.apache.pdfbox.cos.COSArray.addAll(COSArray.java:124)
> at
> org.apache.pdfbox.pdmodel.graphics.state.PDExtendedGraphicsState.getLineDashPattern(PDExtendedGraphicsState.java:280)
> at
> org.apache.pdfbox.pdmodel.graphics.state.PDExtendedGraphicsState.copyIntoGraphicsState(PDExtendedGraphicsState.java:89)
> at
> org.apache.pdfbox.contentstream.operator.state.SetGraphicsStateParameters.process(SetGraphicsStateParameters.java:61)
> at
> org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:838)
> at
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:495)
> at
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:469)
> at
> org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:150)
> at
> org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
> at
> org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
> at
> org.apache.tika.parser.pdf.PDF2XHTML.processPage(PDF2XHTML.java:147)
> at
> org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
> at
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:117)
> at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:168)
> at
> org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
> at
> org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
> at
> org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)
> at org.apache.tika.cli.TikaCLI$OutputType.process(TikaCLI.java:205)
> at org.apache.tika.cli.TikaCLI.process(TikaCLI.java:486)
> at org.apache.tika.cli.TikaCLI.main(TikaCLI.java:145)
>
> I'm not yet sure if I can share the PDF. If needed, I can check that.
>
> Best regards,
> Andreas
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org