You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Naveen Gupta <nk...@gmail.com> on 2011/06/08 05:12:23 UTC
tika integration exception and other related queries
Hi Can somebody answer this ...
3. can somebody tell me an idea how to do indexing for a zip file ?
1. while sending docx, we are getting following error.
java.lang.
>
> NumberFormatException: For input string: "2011-01-27T07:18:00Z"
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Long.parseLong(Long.java:412)
> at java.lang.Long.parseLong(Long.java:461)
> at org.apache.solr.schema.TrieField.createField(TrieField.java:434)
> at
> org.apache.solr.schema.SchemaField.createField(SchemaField.java:98)
> at
> org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:204)
> at
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:277)
> at
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:60)
> at
> org.apache.solr.handler.extraction.ExtractingDocumentLoader.doAdd(ExtractingDocumentLoader.java:121)
> at
> org.apache.solr.handler.extraction.ExtractingDocumentLoader.addDoc(ExtractingDocumentLoader.java:126)
> at
> org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:198)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:55)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
> at
> org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:238)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1360)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
> at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
> at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> at java.lang.Thread.run(Thread.java:619)
>
Thanks
Naveen
On Tue, Jun 7, 2011 at 3:33 PM, Naveen Gupta <nk...@gmail.com> wrote:
> Hi
>
> We are using requestextractinghandler and we are getting following error.
> we are giving microsoft docx file for indexing.
>
> I think that this is something to do with field date definition .. but now
> very sure ...what field type should we use?
>
> 2. we are trying to index jpg (when we search over the name of the jpg, it
> is not coming .. though in id i am passing one)
>
> 3. what about zip files or rar files.. does tika with solr handle this one
> ?
>
>
> java.lang.NumberFormatException: For input string:
> "2011-01-27T07:18:00Z"
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Long.parseLong(Long.java:412)
> at java.lang.Long.parseLong(Long.java:461)
> at org.apache.solr.schema.TrieField.createField(TrieField.java:434)
> at
> org.apache.solr.schema.SchemaField.createField(SchemaField.java:98)
> at
> org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:204)
> at
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:277)
> at
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:60)
> at
> org.apache.solr.handler.extraction.ExtractingDocumentLoader.doAdd(ExtractingDocumentLoader.java:121)
> at
> org.apache.solr.handler.extraction.ExtractingDocumentLoader.addDoc(ExtractingDocumentLoader.java:126)
> at
> org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:198)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:55)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
> at
> org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:238)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1360)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
> at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
> at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> at java.lang.Thread.run(Thread.java:619)
>
> Thanks
> Naveen
>
Re: tika integration exception and other related queries
Posted by Naveen Gupta <nk...@gmail.com>.
Hi Gary,
Similar thing we are doing, but we are not creating an XML doc, rather we
are leaving TIKA to extract the content and depends on dynamic fields. We
are not storing the text as well. But not sure if in future that would be
the case.
What about microsoft 7 and later related attachments. Is this working for
you, because we are always getting number format exception. I posted as well
in the community, but till now no response has some.
Thanks
Naveen
On Thu, Jun 9, 2011 at 6:43 PM, Gary Taylor <gt...@inovem.com> wrote:
> Naveen,
>
> Not sure our requirement matches yours, but one of the things we index is a
> "comment" item that can have one or more files attached to it. To index the
> whole thing as a single Solr document we create a zipfile containing a file
> with the comment details in it and any additional attached files. This is
> submitted to Solr as a TEXT field in an XML doc, along with other meta-data
> fields from the comment. In our schema the TEXT field is indexed but not
> stored, so when we search and get a match back it doesn't contain all of the
> contents from the attached files etc., only the stored fields in our schema.
> Admittedly, the user can therefore get back a "comment" match with no
> indication as to WHERE the match occurred (ie. was it in the meta-data or
> the contents of the attached files), but at the moment we're only interested
> in getting appropriate matches, not explaining where the match is.
>
> Hope that helps.
>
> Kind regards,
> Gary.
>
>
>
>
> On 09/06/2011 03:00, Naveen Gupta wrote:
>
>> Hi Gary
>>
>> It started working .. though i did not test for Zip files, but for rar
>> files, it is working fine ..
>>
>> only thing what i wanted to do is to index the metadata (text mapped to
>> content) not store the data .... Also in search result, i want to filter
>> the
>> stuffs ... and it started working fine .. i don't want to show the content
>> stuffs to the end user, since the way it extracts the information is not
>> very helpful to the user .. although we can apply few of the analyzers and
>> filters to remove the unnecessary tags ..still the information would not
>> be
>> of much help .. looking for your opinion ... what you did in order to
>> filter
>> out the content or are you showing the content extracted to the end user?
>>
>> Even in case, we are showing the text part to the end user, how can i
>> limit
>> the number of characters while querying the search results ... is there
>> any
>> feature where we can achieve this ... the concept of snippet kind of thing
>> ...
>>
>> Thanks
>> Naveen
>>
>> On Wed, Jun 8, 2011 at 1:45 PM, Gary Taylor<gt...@inovem.com> wrote:
>>
>> Naveen,
>>>
>>> For indexing Zip files with Tika, take a look at the following thread :
>>>
>>>
>>>
>>> http://lucene.472066.n3.nabble.com/Extracting-contents-of-zipped-files-with-Tika-and-Solr-1-4-1-td2327933.html
>>>
>>> I got it to work with the 3.1 source and a couple of patches.
>>>
>>> Hope this helps.
>>>
>>> Regards,
>>> Gary.
>>>
>>>
>>>
>>> On 08/06/2011 04:12, Naveen Gupta wrote:
>>>
>>> Hi Can somebody answer this ...
>>>>
>>>> 3. can somebody tell me an idea how to do indexing for a zip file ?
>>>>
>>>> 1. while sending docx, we are getting following error.
>>>>
>>>>
>
Re: tika integration exception and other related queries
Posted by Gary Taylor <gt...@inovem.com>.
Naveen,
Not sure our requirement matches yours, but one of the things we index
is a "comment" item that can have one or more files attached to it. To
index the whole thing as a single Solr document we create a zipfile
containing a file with the comment details in it and any additional
attached files. This is submitted to Solr as a TEXT field in an XML
doc, along with other meta-data fields from the comment. In our schema
the TEXT field is indexed but not stored, so when we search and get a
match back it doesn't contain all of the contents from the attached
files etc., only the stored fields in our schema. Admittedly, the user
can therefore get back a "comment" match with no indication as to WHERE
the match occurred (ie. was it in the meta-data or the contents of the
attached files), but at the moment we're only interested in getting
appropriate matches, not explaining where the match is.
Hope that helps.
Kind regards,
Gary.
On 09/06/2011 03:00, Naveen Gupta wrote:
> Hi Gary
>
> It started working .. though i did not test for Zip files, but for rar
> files, it is working fine ..
>
> only thing what i wanted to do is to index the metadata (text mapped to
> content) not store the data .... Also in search result, i want to filter the
> stuffs ... and it started working fine .. i don't want to show the content
> stuffs to the end user, since the way it extracts the information is not
> very helpful to the user .. although we can apply few of the analyzers and
> filters to remove the unnecessary tags ..still the information would not be
> of much help .. looking for your opinion ... what you did in order to filter
> out the content or are you showing the content extracted to the end user?
>
> Even in case, we are showing the text part to the end user, how can i limit
> the number of characters while querying the search results ... is there any
> feature where we can achieve this ... the concept of snippet kind of thing
> ...
>
> Thanks
> Naveen
>
> On Wed, Jun 8, 2011 at 1:45 PM, Gary Taylor<gt...@inovem.com> wrote:
>
>> Naveen,
>>
>> For indexing Zip files with Tika, take a look at the following thread :
>>
>>
>> http://lucene.472066.n3.nabble.com/Extracting-contents-of-zipped-files-with-Tika-and-Solr-1-4-1-td2327933.html
>>
>> I got it to work with the 3.1 source and a couple of patches.
>>
>> Hope this helps.
>>
>> Regards,
>> Gary.
>>
>>
>>
>> On 08/06/2011 04:12, Naveen Gupta wrote:
>>
>>> Hi Can somebody answer this ...
>>>
>>> 3. can somebody tell me an idea how to do indexing for a zip file ?
>>>
>>> 1. while sending docx, we are getting following error.
>>>
Re: tika integration exception and other related queries
Posted by Naveen Gupta <nk...@gmail.com>.
Hi Gary
It started working .. though i did not test for Zip files, but for rar
files, it is working fine ..
only thing what i wanted to do is to index the metadata (text mapped to
content) not store the data .... Also in search result, i want to filter the
stuffs ... and it started working fine .. i don't want to show the content
stuffs to the end user, since the way it extracts the information is not
very helpful to the user .. although we can apply few of the analyzers and
filters to remove the unnecessary tags ..still the information would not be
of much help .. looking for your opinion ... what you did in order to filter
out the content or are you showing the content extracted to the end user?
Even in case, we are showing the text part to the end user, how can i limit
the number of characters while querying the search results ... is there any
feature where we can achieve this ... the concept of snippet kind of thing
...
Thanks
Naveen
On Wed, Jun 8, 2011 at 1:45 PM, Gary Taylor <gt...@inovem.com> wrote:
> Naveen,
>
> For indexing Zip files with Tika, take a look at the following thread :
>
>
> http://lucene.472066.n3.nabble.com/Extracting-contents-of-zipped-files-with-Tika-and-Solr-1-4-1-td2327933.html
>
> I got it to work with the 3.1 source and a couple of patches.
>
> Hope this helps.
>
> Regards,
> Gary.
>
>
>
> On 08/06/2011 04:12, Naveen Gupta wrote:
>
>> Hi Can somebody answer this ...
>>
>> 3. can somebody tell me an idea how to do indexing for a zip file ?
>>
>> 1. while sending docx, we are getting following error.
>>
>
>
Re: tika integration exception and other related queries
Posted by Gary Taylor <gt...@inovem.com>.
Naveen,
For indexing Zip files with Tika, take a look at the following thread :
http://lucene.472066.n3.nabble.com/Extracting-contents-of-zipped-files-with-Tika-and-Solr-1-4-1-td2327933.html
I got it to work with the 3.1 source and a couple of patches.
Hope this helps.
Regards,
Gary.
On 08/06/2011 04:12, Naveen Gupta wrote:
> Hi Can somebody answer this ...
>
> 3. can somebody tell me an idea how to do indexing for a zip file ?
>
> 1. while sending docx, we are getting following error.