You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@tika.apache.org by Slava G <sl...@gmail.com> on 2019/05/15 13:49:26 UTC
Corrupted PDF file causing severe OOM
I have small pdf file (142kb) while I'm trying to parse it with TIKA my
entire app is crashing on OOM with heap dump on 36gb (nothing else in the
code, hust parsing this PDF).
With possible error : FlateFilter: stop reading corrupt stream due to a
DataFormatException
And stack trace (at the moment of OOM):
"main" #1 prio=5 os_prio=0 tid=0x00007f6460009000 nid=0x4876 waiting for
monitor entry [0x00007f646680d000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.util.HashMap.newNode(HashMap.java:1734)
at java.util.HashMap.putVal(HashMap.java:630)
at java.util.HashMap.put(HashMap.java:611)
at org.apache.fontbox.cmap.CMap.addCharMapping(CMap.java:191)
at
org.apache.fontbox.cmap.CMapParser.parseBeginbfrange(CMapParser.java:398)
at org.apache.fontbox.cmap.CMapParser.parse(CMapParser.java:136)
at
org.apache.pdfbox.pdmodel.font.CMapManager.parseCMap(CMapManager.java:75)
at org.apache.pdfbox.pdmodel.font.PDFont.readCMap(PDFont.java:197)
at org.apache.pdfbox.pdmodel.font.PDFont.<init>(PDFont.java:137)
at
org.apache.pdfbox.pdmodel.font.PDType0Font.<init>(PDType0Font.java:176)
at
org.apache.pdfbox.pdmodel.font.PDFontFactory.createFont(PDFontFactory.java:83)
at
org.apache.pdfbox.pdmodel.PDResources.getFont(PDResources.java:146)
at
org.apache.pdfbox.contentstream.operator.text.SetFontAndSize.process(SetFontAndSize.java:60)
at
org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:848)
at
org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:503)
at
org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:477)
at
org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:150)
at
org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
at
org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
at
org.apache.tika.parser.pdf.PDF2XHTML.processPage(PDF2XHTML.java:147)
at
org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
at
org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:117)
at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:172)
Please advise how can I detect that this can happen and skip such file from
the parsing. Or this is a bug ?
Thanks
Re: Corrupted PDF file causing severe OOM
Posted by Slava G <sl...@gmail.com>.
Well, seems that It'll be fixed in PDFBox 2.0.16
On Wed, May 15, 2019 at 5:35 PM Slava G <sl...@gmail.com> wrote:
> Will definitely try, is this rc available via maven?
>
> On Wed, May 15, 2019, 17:20 Tim Allison <ta...@apache.org> wrote:
>
>> Yay! Tilman and colleagues on PDFBox really are _that_fast. :)
>>
>> You can try Tika’s integration w 2.0.15 in our 1.21-rc2:
>>
>> https://lists.apache.org/thread.html/2c027535156cc6862149490b289552d72ba5a9bff985fb7cce794e21@%3Cdev.tika.apache.org%3E
>>
>> On Wed, May 15, 2019 at 10:01 AM Slava G <sl...@gmail.com> wrote:
>>
>> > Sure, I can share it privately.
>> > But seems that in PDFBox 2.0.15 it's already fixed as, when I run
>> tika-app
>> > (1.20) it's caused same issue, but when I ran extractText in PDFBox
>> 2.0.15
>> > I got next :
>> > May 15, 2019 4:59:11 PM org.apache.pdfbox.filter.FlateFilter decompress
>> > WARNING: FlateFilter: premature end of stream due to a
>> DataFormatException
>> > May 15, 2019 4:59:11 PM org.apache.pdfbox.filter.FlateFilter decode
>> > SEVERE: FlateFilter: stop reading corrupt stream due to a
>> > DataFormatException
>> > Exception in thread "main" java.io.IOException:
>> > java.util.zip.DataFormatException: invalid literal/lengths set
>> > at org.apache.pdfbox.filter.FlateFilter.decode(FlateFilter.java:58)
>> > at org.apache.pdfbox.filter.Filter.decode(Filter.java:87)
>> > at org.apache.pdfbox.cos.COSInputStream.create(COSInputStream.java:77)
>> > at org.apache.pdfbox.cos.COSStream.createInputStream(COSStream.java:175)
>> > at org.apache.pdfbox.cos.COSStream.createInputStream(COSStream.java:163)
>> > at org.apache.pdfbox.pdmodel.PDPage.getContents(PDPage.java:170)
>> > at
>> >
>> org.apache.pdfbox.pdfparser.PDFStreamParser.<init>(PDFStreamParser.java:91)
>> > at
>> >
>> >
>> org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:495)
>> > at
>> >
>> >
>> org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:479)
>> > at
>> >
>> >
>> org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:152)
>> > at
>> >
>> >
>> org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
>> > at
>> >
>> >
>> org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
>> > at
>> >
>> >
>> org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
>> > at
>> >
>> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
>> > at
>> org.apache.pdfbox.tools.ExtractText.extractPages(ExtractText.java:375)
>> > at
>> >
>> org.apache.pdfbox.tools.ExtractText.startExtraction(ExtractText.java:272)
>> > at org.apache.pdfbox.tools.ExtractText.main(ExtractText.java:96)
>> > at org.apache.pdfbox.tools.PDFBox.main(PDFBox.java:60)
>> > Caused by: java.util.zip.DataFormatException: invalid literal/lengths
>> set
>> > at java.util.zip.Inflater.inflateBytes(Native Method)
>> > at java.util.zip.Inflater.inflate(Inflater.java:259)
>> > at java.util.zip.Inflater.inflate(Inflater.java:280)
>> > at org.apache.pdfbox.filter.FlateFilter.decompress(FlateFilter.java:83)
>> > at org.apache.pdfbox.filter.FlateFilter.decode(FlateFilter.java:50)
>> > ... 17 more
>> >
>> >
>> > On Wed, May 15, 2019 at 4:54 PM Tim Allison <ta...@apache.org>
>> wrote:
>> >
>> > > Sounds like it might be a bug.
>> > >
>> > > PDFBox colleagues, any recs?
>> > >
>> > > Slava, if you’re able to share the file even if only privately,
>> that’ll
>> > > help.
>> > >
>> > > On Wed, May 15, 2019 at 9:49 AM Slava G <sl...@gmail.com> wrote:
>> > >
>> > > > I have small pdf file (142kb) while I'm trying to parse it with
>> TIKA my
>> > > > entire app is crashing on OOM with heap dump on 36gb (nothing else
>> in
>> > the
>> > > > code, hust parsing this PDF).
>> > > > With possible error : FlateFilter: stop reading corrupt stream due
>> to a
>> > > > DataFormatException
>> > > > And stack trace (at the moment of OOM):
>> > > > "main" #1 prio=5 os_prio=0 tid=0x00007f6460009000 nid=0x4876 waiting
>> > for
>> > > > monitor entry [0x00007f646680d000]
>> > > > java.lang.Thread.State: BLOCKED (on object monitor)
>> > > > at java.util.HashMap.newNode(HashMap.java:1734)
>> > > > at java.util.HashMap.putVal(HashMap.java:630)
>> > > > at java.util.HashMap.put(HashMap.java:611)
>> > > > at
>> org.apache.fontbox.cmap.CMap.addCharMapping(CMap.java:191)
>> > > > at
>> > > >
>> >
>> org.apache.fontbox.cmap.CMapParser.parseBeginbfrange(CMapParser.java:398)
>> > > > at
>> > org.apache.fontbox.cmap.CMapParser.parse(CMapParser.java:136)
>> > > > at
>> > > >
>> >
>> org.apache.pdfbox.pdmodel.font.CMapManager.parseCMap(CMapManager.java:75)
>> > > > at
>> > > org.apache.pdfbox.pdmodel.font.PDFont.readCMap(PDFont.java:197)
>> > > > at
>> > org.apache.pdfbox.pdmodel.font.PDFont.<init>(PDFont.java:137)
>> > > > at
>> > > >
>> org.apache.pdfbox.pdmodel.font.PDType0Font.<init>(PDType0Font.java:176)
>> > > > at
>> > > >
>> > >
>> >
>> org.apache.pdfbox.pdmodel.font.PDFontFactory.createFont(PDFontFactory.java:83)
>> > > > at
>> > > > org.apache.pdfbox.pdmodel.PDResources.getFont(PDResources.java:146)
>> > > > at
>> > > >
>> > >
>> >
>> org.apache.pdfbox.contentstream.operator.text.SetFontAndSize.process(SetFontAndSize.java:60)
>> > > > at
>> > > >
>> > >
>> >
>> org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:848)
>> > > > at
>> > > >
>> > >
>> >
>> org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:503)
>> > > > at
>> > > >
>> > >
>> >
>> org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:477)
>> > > > at
>> > > >
>> > >
>> >
>> org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:150)
>> > > > at
>> > > >
>> > >
>> >
>> org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
>> > > > at
>> > > >
>> > >
>> >
>> org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
>> > > > at
>> > > > org.apache.tika.parser.pdf.PDF2XHTML.processPage(PDF2XHTML.java:147)
>> > > > at
>> > > >
>> > >
>> >
>> org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
>> > > > at
>> > > >
>> > >
>> >
>> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
>> > > > at
>> > > org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:117)
>> > > > at
>> > org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:172)
>> > > >
>> > > >
>> > > > Please advise how can I detect that this can happen and skip such
>> file
>> > > > from the parsing. Or this is a bug ?
>> > > >
>> > > > Thanks
>> > > >
>> > >
>> >
>>
>
Re: Corrupted PDF file causing severe OOM
Posted by Slava G <sl...@gmail.com>.
Will definitely try, is this rc available via maven?
On Wed, May 15, 2019, 17:20 Tim Allison <ta...@apache.org> wrote:
> Yay! Tilman and colleagues on PDFBox really are _that_fast. :)
>
> You can try Tika’s integration w 2.0.15 in our 1.21-rc2:
>
> https://lists.apache.org/thread.html/2c027535156cc6862149490b289552d72ba5a9bff985fb7cce794e21@%3Cdev.tika.apache.org%3E
>
> On Wed, May 15, 2019 at 10:01 AM Slava G <sl...@gmail.com> wrote:
>
> > Sure, I can share it privately.
> > But seems that in PDFBox 2.0.15 it's already fixed as, when I run
> tika-app
> > (1.20) it's caused same issue, but when I ran extractText in PDFBox
> 2.0.15
> > I got next :
> > May 15, 2019 4:59:11 PM org.apache.pdfbox.filter.FlateFilter decompress
> > WARNING: FlateFilter: premature end of stream due to a
> DataFormatException
> > May 15, 2019 4:59:11 PM org.apache.pdfbox.filter.FlateFilter decode
> > SEVERE: FlateFilter: stop reading corrupt stream due to a
> > DataFormatException
> > Exception in thread "main" java.io.IOException:
> > java.util.zip.DataFormatException: invalid literal/lengths set
> > at org.apache.pdfbox.filter.FlateFilter.decode(FlateFilter.java:58)
> > at org.apache.pdfbox.filter.Filter.decode(Filter.java:87)
> > at org.apache.pdfbox.cos.COSInputStream.create(COSInputStream.java:77)
> > at org.apache.pdfbox.cos.COSStream.createInputStream(COSStream.java:175)
> > at org.apache.pdfbox.cos.COSStream.createInputStream(COSStream.java:163)
> > at org.apache.pdfbox.pdmodel.PDPage.getContents(PDPage.java:170)
> > at
> >
> org.apache.pdfbox.pdfparser.PDFStreamParser.<init>(PDFStreamParser.java:91)
> > at
> >
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:495)
> > at
> >
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:479)
> > at
> >
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:152)
> > at
> >
> >
> org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
> > at
> >
> >
> org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
> > at
> >
> >
> org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
> > at
> >
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
> > at org.apache.pdfbox.tools.ExtractText.extractPages(ExtractText.java:375)
> > at
> > org.apache.pdfbox.tools.ExtractText.startExtraction(ExtractText.java:272)
> > at org.apache.pdfbox.tools.ExtractText.main(ExtractText.java:96)
> > at org.apache.pdfbox.tools.PDFBox.main(PDFBox.java:60)
> > Caused by: java.util.zip.DataFormatException: invalid literal/lengths set
> > at java.util.zip.Inflater.inflateBytes(Native Method)
> > at java.util.zip.Inflater.inflate(Inflater.java:259)
> > at java.util.zip.Inflater.inflate(Inflater.java:280)
> > at org.apache.pdfbox.filter.FlateFilter.decompress(FlateFilter.java:83)
> > at org.apache.pdfbox.filter.FlateFilter.decode(FlateFilter.java:50)
> > ... 17 more
> >
> >
> > On Wed, May 15, 2019 at 4:54 PM Tim Allison <ta...@apache.org> wrote:
> >
> > > Sounds like it might be a bug.
> > >
> > > PDFBox colleagues, any recs?
> > >
> > > Slava, if you’re able to share the file even if only privately, that’ll
> > > help.
> > >
> > > On Wed, May 15, 2019 at 9:49 AM Slava G <sl...@gmail.com> wrote:
> > >
> > > > I have small pdf file (142kb) while I'm trying to parse it with TIKA
> my
> > > > entire app is crashing on OOM with heap dump on 36gb (nothing else in
> > the
> > > > code, hust parsing this PDF).
> > > > With possible error : FlateFilter: stop reading corrupt stream due
> to a
> > > > DataFormatException
> > > > And stack trace (at the moment of OOM):
> > > > "main" #1 prio=5 os_prio=0 tid=0x00007f6460009000 nid=0x4876 waiting
> > for
> > > > monitor entry [0x00007f646680d000]
> > > > java.lang.Thread.State: BLOCKED (on object monitor)
> > > > at java.util.HashMap.newNode(HashMap.java:1734)
> > > > at java.util.HashMap.putVal(HashMap.java:630)
> > > > at java.util.HashMap.put(HashMap.java:611)
> > > > at org.apache.fontbox.cmap.CMap.addCharMapping(CMap.java:191)
> > > > at
> > > >
> > org.apache.fontbox.cmap.CMapParser.parseBeginbfrange(CMapParser.java:398)
> > > > at
> > org.apache.fontbox.cmap.CMapParser.parse(CMapParser.java:136)
> > > > at
> > > >
> > org.apache.pdfbox.pdmodel.font.CMapManager.parseCMap(CMapManager.java:75)
> > > > at
> > > org.apache.pdfbox.pdmodel.font.PDFont.readCMap(PDFont.java:197)
> > > > at
> > org.apache.pdfbox.pdmodel.font.PDFont.<init>(PDFont.java:137)
> > > > at
> > > >
> org.apache.pdfbox.pdmodel.font.PDType0Font.<init>(PDType0Font.java:176)
> > > > at
> > > >
> > >
> >
> org.apache.pdfbox.pdmodel.font.PDFontFactory.createFont(PDFontFactory.java:83)
> > > > at
> > > > org.apache.pdfbox.pdmodel.PDResources.getFont(PDResources.java:146)
> > > > at
> > > >
> > >
> >
> org.apache.pdfbox.contentstream.operator.text.SetFontAndSize.process(SetFontAndSize.java:60)
> > > > at
> > > >
> > >
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:848)
> > > > at
> > > >
> > >
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:503)
> > > > at
> > > >
> > >
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:477)
> > > > at
> > > >
> > >
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:150)
> > > > at
> > > >
> > >
> >
> org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
> > > > at
> > > >
> > >
> >
> org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
> > > > at
> > > > org.apache.tika.parser.pdf.PDF2XHTML.processPage(PDF2XHTML.java:147)
> > > > at
> > > >
> > >
> >
> org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
> > > > at
> > > >
> > >
> >
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
> > > > at
> > > org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:117)
> > > > at
> > org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:172)
> > > >
> > > >
> > > > Please advise how can I detect that this can happen and skip such
> file
> > > > from the parsing. Or this is a bug ?
> > > >
> > > > Thanks
> > > >
> > >
> >
>
Re: Corrupted PDF file causing severe OOM
Posted by Tim Allison <ta...@apache.org>.
Yay! Tilman and colleagues on PDFBox really are _that_fast. :)
You can try Tika’s integration w 2.0.15 in our 1.21-rc2:
https://lists.apache.org/thread.html/2c027535156cc6862149490b289552d72ba5a9bff985fb7cce794e21@%3Cdev.tika.apache.org%3E
On Wed, May 15, 2019 at 10:01 AM Slava G <sl...@gmail.com> wrote:
> Sure, I can share it privately.
> But seems that in PDFBox 2.0.15 it's already fixed as, when I run tika-app
> (1.20) it's caused same issue, but when I ran extractText in PDFBox 2.0.15
> I got next :
> May 15, 2019 4:59:11 PM org.apache.pdfbox.filter.FlateFilter decompress
> WARNING: FlateFilter: premature end of stream due to a DataFormatException
> May 15, 2019 4:59:11 PM org.apache.pdfbox.filter.FlateFilter decode
> SEVERE: FlateFilter: stop reading corrupt stream due to a
> DataFormatException
> Exception in thread "main" java.io.IOException:
> java.util.zip.DataFormatException: invalid literal/lengths set
> at org.apache.pdfbox.filter.FlateFilter.decode(FlateFilter.java:58)
> at org.apache.pdfbox.filter.Filter.decode(Filter.java:87)
> at org.apache.pdfbox.cos.COSInputStream.create(COSInputStream.java:77)
> at org.apache.pdfbox.cos.COSStream.createInputStream(COSStream.java:175)
> at org.apache.pdfbox.cos.COSStream.createInputStream(COSStream.java:163)
> at org.apache.pdfbox.pdmodel.PDPage.getContents(PDPage.java:170)
> at
> org.apache.pdfbox.pdfparser.PDFStreamParser.<init>(PDFStreamParser.java:91)
> at
>
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:495)
> at
>
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:479)
> at
>
> org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:152)
> at
>
> org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
> at
>
> org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
> at
>
> org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
> at
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
> at org.apache.pdfbox.tools.ExtractText.extractPages(ExtractText.java:375)
> at
> org.apache.pdfbox.tools.ExtractText.startExtraction(ExtractText.java:272)
> at org.apache.pdfbox.tools.ExtractText.main(ExtractText.java:96)
> at org.apache.pdfbox.tools.PDFBox.main(PDFBox.java:60)
> Caused by: java.util.zip.DataFormatException: invalid literal/lengths set
> at java.util.zip.Inflater.inflateBytes(Native Method)
> at java.util.zip.Inflater.inflate(Inflater.java:259)
> at java.util.zip.Inflater.inflate(Inflater.java:280)
> at org.apache.pdfbox.filter.FlateFilter.decompress(FlateFilter.java:83)
> at org.apache.pdfbox.filter.FlateFilter.decode(FlateFilter.java:50)
> ... 17 more
>
>
> On Wed, May 15, 2019 at 4:54 PM Tim Allison <ta...@apache.org> wrote:
>
> > Sounds like it might be a bug.
> >
> > PDFBox colleagues, any recs?
> >
> > Slava, if you’re able to share the file even if only privately, that’ll
> > help.
> >
> > On Wed, May 15, 2019 at 9:49 AM Slava G <sl...@gmail.com> wrote:
> >
> > > I have small pdf file (142kb) while I'm trying to parse it with TIKA my
> > > entire app is crashing on OOM with heap dump on 36gb (nothing else in
> the
> > > code, hust parsing this PDF).
> > > With possible error : FlateFilter: stop reading corrupt stream due to a
> > > DataFormatException
> > > And stack trace (at the moment of OOM):
> > > "main" #1 prio=5 os_prio=0 tid=0x00007f6460009000 nid=0x4876 waiting
> for
> > > monitor entry [0x00007f646680d000]
> > > java.lang.Thread.State: BLOCKED (on object monitor)
> > > at java.util.HashMap.newNode(HashMap.java:1734)
> > > at java.util.HashMap.putVal(HashMap.java:630)
> > > at java.util.HashMap.put(HashMap.java:611)
> > > at org.apache.fontbox.cmap.CMap.addCharMapping(CMap.java:191)
> > > at
> > >
> org.apache.fontbox.cmap.CMapParser.parseBeginbfrange(CMapParser.java:398)
> > > at
> org.apache.fontbox.cmap.CMapParser.parse(CMapParser.java:136)
> > > at
> > >
> org.apache.pdfbox.pdmodel.font.CMapManager.parseCMap(CMapManager.java:75)
> > > at
> > org.apache.pdfbox.pdmodel.font.PDFont.readCMap(PDFont.java:197)
> > > at
> org.apache.pdfbox.pdmodel.font.PDFont.<init>(PDFont.java:137)
> > > at
> > > org.apache.pdfbox.pdmodel.font.PDType0Font.<init>(PDType0Font.java:176)
> > > at
> > >
> >
> org.apache.pdfbox.pdmodel.font.PDFontFactory.createFont(PDFontFactory.java:83)
> > > at
> > > org.apache.pdfbox.pdmodel.PDResources.getFont(PDResources.java:146)
> > > at
> > >
> >
> org.apache.pdfbox.contentstream.operator.text.SetFontAndSize.process(SetFontAndSize.java:60)
> > > at
> > >
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:848)
> > > at
> > >
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:503)
> > > at
> > >
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:477)
> > > at
> > >
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:150)
> > > at
> > >
> >
> org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
> > > at
> > >
> >
> org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
> > > at
> > > org.apache.tika.parser.pdf.PDF2XHTML.processPage(PDF2XHTML.java:147)
> > > at
> > >
> >
> org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
> > > at
> > >
> >
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
> > > at
> > org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:117)
> > > at
> org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:172)
> > >
> > >
> > > Please advise how can I detect that this can happen and skip such file
> > > from the parsing. Or this is a bug ?
> > >
> > > Thanks
> > >
> >
>
Re: Corrupted PDF file causing severe OOM
Posted by Tilman Hausherr <TH...@t-online.de>.
Am 15.05.2019 um 16:00 schrieb Slava G:
> Sure, I can share it privately.
Send it to
tilman (at) snafu (dot) de .
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
Re: Corrupted PDF file causing severe OOM
Posted by Slava G <sl...@gmail.com>.
Got you.
Thanks
On Thu, May 16, 2019 at 6:42 AM Tilman Hausherr <TH...@t-online.de>
wrote:
> Am 15.05.2019 um 21:57 schrieb Slava G:
> > But I tried to extract text using 2.0.15 and got immidiatelly exception
> and
> > didn't get OOM.
>
>
> I got slow response on the second page. I didn't wait until OOM.
>
> Tilman
>
>
>
> >
> > On Wed, May 15, 2019, 22:52 Tilman Hausherr <TH...@t-online.de>
> wrote:
> >
> >> Am 15.05.2019 um 16:00 schrieb Slava G:
> >>> But seems that in PDFBox 2.0.15 it's already fixed as, when I run
> >> tika-app
> >>
> >>
> >> No it's not fixed. The cause is a corrupt ToUnicode stream. Fixed in
> >>
> >> https://issues.apache.org/jira/browse/PDFBOX-4550
> >>
> >> Try a snapshot within a few hours
> >>
> >>
> >>
> https://repository.apache.org/content/groups/snapshots/org/apache/pdfbox/pdfbox-app/2.0.16-SNAPSHOT/
> >>
> >> Tilman
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> >> For additional commands, e-mail: users-help@pdfbox.apache.org
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
>
>
Re: Corrupted PDF file causing severe OOM
Posted by Tilman Hausherr <TH...@t-online.de>.
Am 15.05.2019 um 21:57 schrieb Slava G:
> But I tried to extract text using 2.0.15 and got immidiatelly exception and
> didn't get OOM.
I got slow response on the second page. I didn't wait until OOM.
Tilman
>
> On Wed, May 15, 2019, 22:52 Tilman Hausherr <TH...@t-online.de> wrote:
>
>> Am 15.05.2019 um 16:00 schrieb Slava G:
>>> But seems that in PDFBox 2.0.15 it's already fixed as, when I run
>> tika-app
>>
>>
>> No it's not fixed. The cause is a corrupt ToUnicode stream. Fixed in
>>
>> https://issues.apache.org/jira/browse/PDFBOX-4550
>>
>> Try a snapshot within a few hours
>>
>>
>> https://repository.apache.org/content/groups/snapshots/org/apache/pdfbox/pdfbox-app/2.0.16-SNAPSHOT/
>>
>> Tilman
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
>> For additional commands, e-mail: users-help@pdfbox.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
Re: Corrupted PDF file causing severe OOM
Posted by Slava G <sl...@gmail.com>.
But I tried to extract text using 2.0.15 and got immidiatelly exception and
didn't get OOM.
On Wed, May 15, 2019, 22:52 Tilman Hausherr <TH...@t-online.de> wrote:
> Am 15.05.2019 um 16:00 schrieb Slava G:
> > But seems that in PDFBox 2.0.15 it's already fixed as, when I run
> tika-app
>
>
> No it's not fixed. The cause is a corrupt ToUnicode stream. Fixed in
>
> https://issues.apache.org/jira/browse/PDFBOX-4550
>
> Try a snapshot within a few hours
>
>
> https://repository.apache.org/content/groups/snapshots/org/apache/pdfbox/pdfbox-app/2.0.16-SNAPSHOT/
>
> Tilman
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
>
>
Re: Corrupted PDF file causing severe OOM
Posted by Tilman Hausherr <TH...@t-online.de>.
Am 15.05.2019 um 16:00 schrieb Slava G:
> But seems that in PDFBox 2.0.15 it's already fixed as, when I run tika-app
No it's not fixed. The cause is a corrupt ToUnicode stream. Fixed in
https://issues.apache.org/jira/browse/PDFBOX-4550
Try a snapshot within a few hours
https://repository.apache.org/content/groups/snapshots/org/apache/pdfbox/pdfbox-app/2.0.16-SNAPSHOT/
Tilman
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
Re: Corrupted PDF file causing severe OOM
Posted by Slava G <sl...@gmail.com>.
Sure, I can share it privately.
But seems that in PDFBox 2.0.15 it's already fixed as, when I run tika-app
(1.20) it's caused same issue, but when I ran extractText in PDFBox 2.0.15
I got next :
May 15, 2019 4:59:11 PM org.apache.pdfbox.filter.FlateFilter decompress
WARNING: FlateFilter: premature end of stream due to a DataFormatException
May 15, 2019 4:59:11 PM org.apache.pdfbox.filter.FlateFilter decode
SEVERE: FlateFilter: stop reading corrupt stream due to a
DataFormatException
Exception in thread "main" java.io.IOException:
java.util.zip.DataFormatException: invalid literal/lengths set
at org.apache.pdfbox.filter.FlateFilter.decode(FlateFilter.java:58)
at org.apache.pdfbox.filter.Filter.decode(Filter.java:87)
at org.apache.pdfbox.cos.COSInputStream.create(COSInputStream.java:77)
at org.apache.pdfbox.cos.COSStream.createInputStream(COSStream.java:175)
at org.apache.pdfbox.cos.COSStream.createInputStream(COSStream.java:163)
at org.apache.pdfbox.pdmodel.PDPage.getContents(PDPage.java:170)
at
org.apache.pdfbox.pdfparser.PDFStreamParser.<init>(PDFStreamParser.java:91)
at
org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:495)
at
org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:479)
at
org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:152)
at
org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
at
org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
at
org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
at
org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
at org.apache.pdfbox.tools.ExtractText.extractPages(ExtractText.java:375)
at org.apache.pdfbox.tools.ExtractText.startExtraction(ExtractText.java:272)
at org.apache.pdfbox.tools.ExtractText.main(ExtractText.java:96)
at org.apache.pdfbox.tools.PDFBox.main(PDFBox.java:60)
Caused by: java.util.zip.DataFormatException: invalid literal/lengths set
at java.util.zip.Inflater.inflateBytes(Native Method)
at java.util.zip.Inflater.inflate(Inflater.java:259)
at java.util.zip.Inflater.inflate(Inflater.java:280)
at org.apache.pdfbox.filter.FlateFilter.decompress(FlateFilter.java:83)
at org.apache.pdfbox.filter.FlateFilter.decode(FlateFilter.java:50)
... 17 more
On Wed, May 15, 2019 at 4:54 PM Tim Allison <ta...@apache.org> wrote:
> Sounds like it might be a bug.
>
> PDFBox colleagues, any recs?
>
> Slava, if you’re able to share the file even if only privately, that’ll
> help.
>
> On Wed, May 15, 2019 at 9:49 AM Slava G <sl...@gmail.com> wrote:
>
> > I have small pdf file (142kb) while I'm trying to parse it with TIKA my
> > entire app is crashing on OOM with heap dump on 36gb (nothing else in the
> > code, hust parsing this PDF).
> > With possible error : FlateFilter: stop reading corrupt stream due to a
> > DataFormatException
> > And stack trace (at the moment of OOM):
> > "main" #1 prio=5 os_prio=0 tid=0x00007f6460009000 nid=0x4876 waiting for
> > monitor entry [0x00007f646680d000]
> > java.lang.Thread.State: BLOCKED (on object monitor)
> > at java.util.HashMap.newNode(HashMap.java:1734)
> > at java.util.HashMap.putVal(HashMap.java:630)
> > at java.util.HashMap.put(HashMap.java:611)
> > at org.apache.fontbox.cmap.CMap.addCharMapping(CMap.java:191)
> > at
> > org.apache.fontbox.cmap.CMapParser.parseBeginbfrange(CMapParser.java:398)
> > at org.apache.fontbox.cmap.CMapParser.parse(CMapParser.java:136)
> > at
> > org.apache.pdfbox.pdmodel.font.CMapManager.parseCMap(CMapManager.java:75)
> > at
> org.apache.pdfbox.pdmodel.font.PDFont.readCMap(PDFont.java:197)
> > at org.apache.pdfbox.pdmodel.font.PDFont.<init>(PDFont.java:137)
> > at
> > org.apache.pdfbox.pdmodel.font.PDType0Font.<init>(PDType0Font.java:176)
> > at
> >
> org.apache.pdfbox.pdmodel.font.PDFontFactory.createFont(PDFontFactory.java:83)
> > at
> > org.apache.pdfbox.pdmodel.PDResources.getFont(PDResources.java:146)
> > at
> >
> org.apache.pdfbox.contentstream.operator.text.SetFontAndSize.process(SetFontAndSize.java:60)
> > at
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:848)
> > at
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:503)
> > at
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:477)
> > at
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:150)
> > at
> >
> org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
> > at
> >
> org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
> > at
> > org.apache.tika.parser.pdf.PDF2XHTML.processPage(PDF2XHTML.java:147)
> > at
> >
> org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
> > at
> >
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
> > at
> org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:117)
> > at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:172)
> >
> >
> > Please advise how can I detect that this can happen and skip such file
> > from the parsing. Or this is a bug ?
> >
> > Thanks
> >
>
Re: Corrupted PDF file causing severe OOM
Posted by Slava G <sl...@gmail.com>.
Sure, I can share it privately.
But seems that in PDFBox 2.0.15 it's already fixed as, when I run tika-app
(1.20) it's caused same issue, but when I ran extractText in PDFBox 2.0.15
I got next :
May 15, 2019 4:59:11 PM org.apache.pdfbox.filter.FlateFilter decompress
WARNING: FlateFilter: premature end of stream due to a DataFormatException
May 15, 2019 4:59:11 PM org.apache.pdfbox.filter.FlateFilter decode
SEVERE: FlateFilter: stop reading corrupt stream due to a
DataFormatException
Exception in thread "main" java.io.IOException:
java.util.zip.DataFormatException: invalid literal/lengths set
at org.apache.pdfbox.filter.FlateFilter.decode(FlateFilter.java:58)
at org.apache.pdfbox.filter.Filter.decode(Filter.java:87)
at org.apache.pdfbox.cos.COSInputStream.create(COSInputStream.java:77)
at org.apache.pdfbox.cos.COSStream.createInputStream(COSStream.java:175)
at org.apache.pdfbox.cos.COSStream.createInputStream(COSStream.java:163)
at org.apache.pdfbox.pdmodel.PDPage.getContents(PDPage.java:170)
at
org.apache.pdfbox.pdfparser.PDFStreamParser.<init>(PDFStreamParser.java:91)
at
org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:495)
at
org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:479)
at
org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:152)
at
org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
at
org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
at
org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
at
org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
at org.apache.pdfbox.tools.ExtractText.extractPages(ExtractText.java:375)
at org.apache.pdfbox.tools.ExtractText.startExtraction(ExtractText.java:272)
at org.apache.pdfbox.tools.ExtractText.main(ExtractText.java:96)
at org.apache.pdfbox.tools.PDFBox.main(PDFBox.java:60)
Caused by: java.util.zip.DataFormatException: invalid literal/lengths set
at java.util.zip.Inflater.inflateBytes(Native Method)
at java.util.zip.Inflater.inflate(Inflater.java:259)
at java.util.zip.Inflater.inflate(Inflater.java:280)
at org.apache.pdfbox.filter.FlateFilter.decompress(FlateFilter.java:83)
at org.apache.pdfbox.filter.FlateFilter.decode(FlateFilter.java:50)
... 17 more
On Wed, May 15, 2019 at 4:54 PM Tim Allison <ta...@apache.org> wrote:
> Sounds like it might be a bug.
>
> PDFBox colleagues, any recs?
>
> Slava, if you’re able to share the file even if only privately, that’ll
> help.
>
> On Wed, May 15, 2019 at 9:49 AM Slava G <sl...@gmail.com> wrote:
>
> > I have small pdf file (142kb) while I'm trying to parse it with TIKA my
> > entire app is crashing on OOM with heap dump on 36gb (nothing else in the
> > code, hust parsing this PDF).
> > With possible error : FlateFilter: stop reading corrupt stream due to a
> > DataFormatException
> > And stack trace (at the moment of OOM):
> > "main" #1 prio=5 os_prio=0 tid=0x00007f6460009000 nid=0x4876 waiting for
> > monitor entry [0x00007f646680d000]
> > java.lang.Thread.State: BLOCKED (on object monitor)
> > at java.util.HashMap.newNode(HashMap.java:1734)
> > at java.util.HashMap.putVal(HashMap.java:630)
> > at java.util.HashMap.put(HashMap.java:611)
> > at org.apache.fontbox.cmap.CMap.addCharMapping(CMap.java:191)
> > at
> > org.apache.fontbox.cmap.CMapParser.parseBeginbfrange(CMapParser.java:398)
> > at org.apache.fontbox.cmap.CMapParser.parse(CMapParser.java:136)
> > at
> > org.apache.pdfbox.pdmodel.font.CMapManager.parseCMap(CMapManager.java:75)
> > at
> org.apache.pdfbox.pdmodel.font.PDFont.readCMap(PDFont.java:197)
> > at org.apache.pdfbox.pdmodel.font.PDFont.<init>(PDFont.java:137)
> > at
> > org.apache.pdfbox.pdmodel.font.PDType0Font.<init>(PDType0Font.java:176)
> > at
> >
> org.apache.pdfbox.pdmodel.font.PDFontFactory.createFont(PDFontFactory.java:83)
> > at
> > org.apache.pdfbox.pdmodel.PDResources.getFont(PDResources.java:146)
> > at
> >
> org.apache.pdfbox.contentstream.operator.text.SetFontAndSize.process(SetFontAndSize.java:60)
> > at
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:848)
> > at
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:503)
> > at
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:477)
> > at
> >
> org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:150)
> > at
> >
> org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
> > at
> >
> org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
> > at
> > org.apache.tika.parser.pdf.PDF2XHTML.processPage(PDF2XHTML.java:147)
> > at
> >
> org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
> > at
> >
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
> > at
> org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:117)
> > at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:172)
> >
> >
> > Please advise how can I detect that this can happen and skip such file
> > from the parsing. Or this is a bug ?
> >
> > Thanks
> >
>
Re: Corrupted PDF file causing severe OOM
Posted by Tim Allison <ta...@apache.org>.
Sounds like it might be a bug.
PDFBox colleagues, any recs?
Slava, if you’re able to share the file even if only privately, that’ll
help.
On Wed, May 15, 2019 at 9:49 AM Slava G <sl...@gmail.com> wrote:
> I have small pdf file (142kb) while I'm trying to parse it with TIKA my
> entire app is crashing on OOM with heap dump on 36gb (nothing else in the
> code, hust parsing this PDF).
> With possible error : FlateFilter: stop reading corrupt stream due to a
> DataFormatException
> And stack trace (at the moment of OOM):
> "main" #1 prio=5 os_prio=0 tid=0x00007f6460009000 nid=0x4876 waiting for
> monitor entry [0x00007f646680d000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at java.util.HashMap.newNode(HashMap.java:1734)
> at java.util.HashMap.putVal(HashMap.java:630)
> at java.util.HashMap.put(HashMap.java:611)
> at org.apache.fontbox.cmap.CMap.addCharMapping(CMap.java:191)
> at
> org.apache.fontbox.cmap.CMapParser.parseBeginbfrange(CMapParser.java:398)
> at org.apache.fontbox.cmap.CMapParser.parse(CMapParser.java:136)
> at
> org.apache.pdfbox.pdmodel.font.CMapManager.parseCMap(CMapManager.java:75)
> at org.apache.pdfbox.pdmodel.font.PDFont.readCMap(PDFont.java:197)
> at org.apache.pdfbox.pdmodel.font.PDFont.<init>(PDFont.java:137)
> at
> org.apache.pdfbox.pdmodel.font.PDType0Font.<init>(PDType0Font.java:176)
> at
> org.apache.pdfbox.pdmodel.font.PDFontFactory.createFont(PDFontFactory.java:83)
> at
> org.apache.pdfbox.pdmodel.PDResources.getFont(PDResources.java:146)
> at
> org.apache.pdfbox.contentstream.operator.text.SetFontAndSize.process(SetFontAndSize.java:60)
> at
> org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:848)
> at
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:503)
> at
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:477)
> at
> org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:150)
> at
> org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
> at
> org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
> at
> org.apache.tika.parser.pdf.PDF2XHTML.processPage(PDF2XHTML.java:147)
> at
> org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
> at
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:117)
> at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:172)
>
>
> Please advise how can I detect that this can happen and skip such file
> from the parsing. Or this is a bug ?
>
> Thanks
>
Re: Corrupted PDF file causing severe OOM
Posted by Tim Allison <ta...@apache.org>.
Sounds like it might be a bug.
PDFBox colleagues, any recs?
Slava, if you’re able to share the file even if only privately, that’ll
help.
On Wed, May 15, 2019 at 9:49 AM Slava G <sl...@gmail.com> wrote:
> I have small pdf file (142kb) while I'm trying to parse it with TIKA my
> entire app is crashing on OOM with heap dump on 36gb (nothing else in the
> code, hust parsing this PDF).
> With possible error : FlateFilter: stop reading corrupt stream due to a
> DataFormatException
> And stack trace (at the moment of OOM):
> "main" #1 prio=5 os_prio=0 tid=0x00007f6460009000 nid=0x4876 waiting for
> monitor entry [0x00007f646680d000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at java.util.HashMap.newNode(HashMap.java:1734)
> at java.util.HashMap.putVal(HashMap.java:630)
> at java.util.HashMap.put(HashMap.java:611)
> at org.apache.fontbox.cmap.CMap.addCharMapping(CMap.java:191)
> at
> org.apache.fontbox.cmap.CMapParser.parseBeginbfrange(CMapParser.java:398)
> at org.apache.fontbox.cmap.CMapParser.parse(CMapParser.java:136)
> at
> org.apache.pdfbox.pdmodel.font.CMapManager.parseCMap(CMapManager.java:75)
> at org.apache.pdfbox.pdmodel.font.PDFont.readCMap(PDFont.java:197)
> at org.apache.pdfbox.pdmodel.font.PDFont.<init>(PDFont.java:137)
> at
> org.apache.pdfbox.pdmodel.font.PDType0Font.<init>(PDType0Font.java:176)
> at
> org.apache.pdfbox.pdmodel.font.PDFontFactory.createFont(PDFontFactory.java:83)
> at
> org.apache.pdfbox.pdmodel.PDResources.getFont(PDResources.java:146)
> at
> org.apache.pdfbox.contentstream.operator.text.SetFontAndSize.process(SetFontAndSize.java:60)
> at
> org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:848)
> at
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:503)
> at
> org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:477)
> at
> org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:150)
> at
> org.apache.pdfbox.text.LegacyPDFStreamEngine.processPage(LegacyPDFStreamEngine.java:139)
> at
> org.apache.pdfbox.text.PDFTextStripper.processPage(PDFTextStripper.java:391)
> at
> org.apache.tika.parser.pdf.PDF2XHTML.processPage(PDF2XHTML.java:147)
> at
> org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
> at
> org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:117)
> at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:172)
>
>
> Please advise how can I detect that this can happen and skip such file
> from the parsing. Or this is a bug ?
>
> Thanks
>