You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Saphira <sp...@deustosistemas.net> on 2010/11/26 09:23:20 UTC
Unable to extract PDF content
Hello,
I've been searching but havent' found any solution for this. I'm testing
nutch with a lot of webpages (1 by 1 to see the results) and in all of this
websites that have pdfs I get "Unable to extract PDF content" error when
nutch tries to parse the document.
I've checked that the pdf are not truncated because I've set nutch property
for file size to 3Mb.
Any help please?
--
View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-extract-PDF-content-tp1971600p1971600.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: Unable to extract PDF content
Posted by Saphira <sp...@deustosistemas.net>.
Wow, that's a usefull class. Thanks!
And you're right, it parses ok.. what a weird thing... I'll check all my
configurations, but if I can't use tika for pdf I will use parser-pdf plugin
that seems to work ok.
Thanks for the help and If I find something new I'll let you know :)
--
View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-extract-PDF-content-tp1971600p1972309.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: Unable to extract PDF content
Posted by Julien Nioche <li...@gmail.com>.
Note that you can also test the parsing using : bin/nutch
org.apache.nutch.parse.ParserChecker
http://www.egamaster.com/datos/politica_fr.pdf
On 26 November 2010 11:35, Saphira <sp...@deustosistemas.net> wrote:
>
> I'll try to call directly from tika, but if you say that it gets parsed
> ok...
> may be the problem the integration of tika with nutch?
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Unable-to-extract-PDF-content-tp1971600p1972270.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>
--
*
*Open Source Solutions for Text Engineering
http://digitalpebble.blogspot.com/
http://www.digitalpebble.com
Re: Unable to extract PDF content
Posted by Saphira <sp...@deustosistemas.net>.
I'll try to call directly from tika, but if you say that it gets parsed ok...
may be the problem the integration of tika with nutch?
--
View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-extract-PDF-content-tp1971600p1972270.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: Unable to extract PDF content
Posted by Saphira <sp...@deustosistemas.net>.
Nope, that's not the problem. I changed the default value for 3000000 to be
sure that all those pdfs woul be fetched and not truncated.
--
View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-extract-PDF-content-tp1971600p1972264.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: Unable to extract PDF content
Posted by Julien Nioche <li...@gmail.com>.
Could be due to the fetch size limit. The URL you gave as an example is
163395 bytes long, whereas the default value in Nutch is
<property>
<name>http.content.limit</name>
<value>65536</value>
<description>The length limit for downloaded content using the http
protocol, in bytes. If this value is nonnegative (>=0), content longer
than it will be truncated; otherwise, no truncation at all. Do not
confuse this setting with the file.content.limit setting.
</description>
</property>
Calling Tika directly on a URL is a good way of checking where the problem
comes from e.g.
/usr/local/bin/tika-0.7/tika-app/target/tika-app-0.7.jar
http://www.egamaster.com/datos/politica_fr.pdf
works fine
J.
On 26 November 2010 10:57, Saphira <sp...@deustosistemas.net> wrote:
>
> that's what it says, I think is always the same error
>
>
> org.apache.tika.exception.TikaException: Unable to extract PDF content
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:61)
> at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:79)
> at
> org.apache.nutch.parse.tika.TikaParser.getParse(TikaParser.java:95)
> at org.apache.nutch.parse.ParseCallable.call(ParseCallable.java:18)
> at org.apache.nutch.parse.ParseCallable.call(ParseCallable.java:1)
> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: org.apache.pdfbox.exceptions.WrappedIOException:
> OperatorProcessor class org.pdfbox.util.operator.ShowTextGlyph could not be
> instantiated
> at
> org.apache.pdfbox.util.PDFStreamEngine.<init>(PDFStreamEngine.java:152)
> at
> org.apache.pdfbox.util.PDFTextStripper.<init>(PDFTextStripper.java:129)
> at org.apache.tika.parser.pdf.PDF2XHTML.<init>(PDF2XHTML.java:69)
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:56)
> ... 7 more
> Caused by: java.lang.ClassCastException:
> org.pdfbox.util.operator.ShowTextGlyph cannot be cast to
> org.apache.pdfbox.util.operator.OperatorProcessor
> at
> org.apache.pdfbox.util.PDFStreamEngine.<init>(PDFStreamEngine.java:146)
> ... 10 more
> 2010-11-26 08:27:42,113 WARN fetcher.Fetcher - Error parsing:
> http://www.egamaster.com/datos/politica_fr.pdf: failed(2,0): Unable to
> extract PDF content
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Unable-to-extract-PDF-content-tp1971600p1972145.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>
--
*
*Open Source Solutions for Text Engineering
http://digitalpebble.blogspot.com/
http://www.digitalpebble.com
Re: Unable to extract PDF content
Posted by Saphira <sp...@deustosistemas.net>.
Nope,
Whe I had just tika it didn't work. I then added pdf-parser and associated
pdf with that plugin and worked..
It's strange..
--
View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-extract-PDF-content-tp1971600p1987436.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: Unable to extract PDF content
Posted by Ken Krugler <kk...@transpac.com>.
I think this error can happen when you've got multiple versions of
fontbox on your classpath.
Perhaps you're including both Tika-based parsing and the older Nutch
PDF parser?
See http://thottingal.in/blog/2009/06/24/pdfbox-extract-text-from-pdf/
for more information.
-- Ken
On Nov 26, 2010, at 2:57am, Saphira wrote:
>
> that's what it says, I think is always the same error
>
>
> org.apache.tika.exception.TikaException: Unable to extract PDF content
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:61)
> at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:79)
> at org.apache.nutch.parse.tika.TikaParser.getParse(TikaParser.java:
> 95)
> at org.apache.nutch.parse.ParseCallable.call(ParseCallable.java:18)
> at org.apache.nutch.parse.ParseCallable.call(ParseCallable.java:1)
> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: org.apache.pdfbox.exceptions.WrappedIOException:
> OperatorProcessor class org.pdfbox.util.operator.ShowTextGlyph could
> not be
> instantiated
> at
> org.apache.pdfbox.util.PDFStreamEngine.<init>(PDFStreamEngine.java:
> 152)
> at
> org.apache.pdfbox.util.PDFTextStripper.<init>(PDFTextStripper.java:
> 129)
> at org.apache.tika.parser.pdf.PDF2XHTML.<init>(PDF2XHTML.java:69)
> at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:56)
> ... 7 more
> Caused by: java.lang.ClassCastException:
> org.pdfbox.util.operator.ShowTextGlyph cannot be cast to
> org.apache.pdfbox.util.operator.OperatorProcessor
> at
> org.apache.pdfbox.util.PDFStreamEngine.<init>(PDFStreamEngine.java:
> 146)
> ... 10 more
> 2010-11-26 08:27:42,113 WARN fetcher.Fetcher - Error parsing:
> http://www.egamaster.com/datos/politica_fr.pdf: failed(2,0): Unable to
> extract PDF content
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-extract-PDF-content-tp1971600p1972145.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
--------------------------
Ken Krugler
+1 530-210-6378
http://bixolabs.com
e l a s t i c w e b m i n i n g
Re: Unable to extract PDF content
Posted by Saphira <sp...@deustosistemas.net>.
that's what it says, I think is always the same error
org.apache.tika.exception.TikaException: Unable to extract PDF content
at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:61)
at org.apache.tika.parser.pdf.PDFParser.parse(PDFParser.java:79)
at org.apache.nutch.parse.tika.TikaParser.getParse(TikaParser.java:95)
at org.apache.nutch.parse.ParseCallable.call(ParseCallable.java:18)
at org.apache.nutch.parse.ParseCallable.call(ParseCallable.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.pdfbox.exceptions.WrappedIOException:
OperatorProcessor class org.pdfbox.util.operator.ShowTextGlyph could not be
instantiated
at org.apache.pdfbox.util.PDFStreamEngine.<init>(PDFStreamEngine.java:152)
at org.apache.pdfbox.util.PDFTextStripper.<init>(PDFTextStripper.java:129)
at org.apache.tika.parser.pdf.PDF2XHTML.<init>(PDF2XHTML.java:69)
at org.apache.tika.parser.pdf.PDF2XHTML.process(PDF2XHTML.java:56)
... 7 more
Caused by: java.lang.ClassCastException:
org.pdfbox.util.operator.ShowTextGlyph cannot be cast to
org.apache.pdfbox.util.operator.OperatorProcessor
at org.apache.pdfbox.util.PDFStreamEngine.<init>(PDFStreamEngine.java:146)
... 10 more
2010-11-26 08:27:42,113 WARN fetcher.Fetcher - Error parsing:
http://www.egamaster.com/datos/politica_fr.pdf: failed(2,0): Unable to
extract PDF content
--
View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-extract-PDF-content-tp1971600p1972145.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: Unable to extract PDF content
Posted by Claudio Martella <cl...@tis.bz.it>.
Could you post more details from the logs?
On 11/26/10 10:15 AM, Saphira wrote:
> I've changed to using the parser-pdf instead the tika-parser and now it
> works. Anyone knows why the tika-parser doesn't work for pdfs?
--
Claudio Martella
Digital Technologies
Unit Research & Development - Analyst
TIS innovation park
Via Siemens 19 | Siemensstr. 19
39100 Bolzano | 39100 Bozen
Tel. +39 0471 068 123
Fax +39 0471 068 129
claudio.martella@tis.bz.it http://www.tis.bz.it
Short information regarding use of personal data. According to Section 13 of Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we process your personal data in order to fulfil contractual and fiscal obligations and also to send you information regarding our services and events. Your personal data are processed with and without electronic means and by respecting data subjects' rights, fundamental freedoms and dignity, particularly with regard to confidentiality, personal identity and the right to personal data protection. At any time and without formalities you can write an e-mail to privacy@tis.bz.it in order to object the processing of your personal data for the purpose of sending advertising materials and also to exercise the right to access personal data and other rights referred to in Section 7 of Decree 196/2003. The data controller is TIS Techno Innovation Alto Adige, Siemens Street n. 19, Bolzano. You can find the complete information on the web site www.tis.bz.it.
Re: Unable to extract PDF content
Posted by Saphira <sp...@deustosistemas.net>.
I've changed to using the parser-pdf instead the tika-parser and now it
works. Anyone knows why the tika-parser doesn't work for pdfs?
--
View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-extract-PDF-content-tp1971600p1971784.html
Sent from the Nutch - User mailing list archive at Nabble.com.