You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by coolmals <co...@gmail.com> on 2015/08/13 21:59:36 UTC
Adding additional file types for indexing
I am a new user to Solr. I am trying to index images using solr. Currently
the jpg, png etc image files are skipped in auto mode
Entering auto mode. File endings considered are
xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,
htm,html,txt,log
only these files are indexed. How to index additional file types. Is there a
configuration to add new file types for indexing?
Regards,
Malathy
--
View this message in context: http://lucene.472066.n3.nabble.com/Adding-additional-file-types-for-indexing-tp4222845.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Adding additional file types for indexing
Posted by Erik Hatcher <er...@gmail.com>.
I opened https://issues.apache.org/jira/browse/SOLR-7929 <https://issues.apache.org/jira/browse/SOLR-7929> for the issue below.
—
Erik Hatcher, Senior Solutions Architect
http://www.lucidworks.com <http://www.lucidworks.com/>
> On Aug 14, 2015, at 11:28 AM, Erik Hatcher <er...@gmail.com> wrote:
>
> Not sure how exactly you’re trying it, so let’s talk concretely with examples here:
>
> ----
> $ bin/solr create -c tmp
> $ bin/post -c tmp ~/Desktop/bin_post_featured_image.png -filetypes “*"
>
> $ curl "http://localhost:8983/solr/tmp/select?q=*:*&wt=csv&fl=id"
> id
> "/Users/erikhatcher/Desktop/bin_post_featured_image.png”
> ----
>
> There’s probably an issue in there somewhere, but it does work with the above.
>
> I wasn’t able to get *this* to work though:
>
> $ bin/post -c tmp http://lucene.apache.org/solr/assets/images/book_sia.png -filetypes “*”
> ...
> Entering auto mode. Indexing pages with content-types corresponding to file endings *
> Entering crawl at level 0 (1 links total, 1 new)
> SimplePostTool: WARNING: Skipping URL with unsupported type image/png
>
> So something needs fixing in web crawl mode it looks like.
>
> —
> Erik Hatcher, Senior Solutions Architect
> http://www.lucidworks.com
>
>
>
>
>> On Aug 14, 2015, at 11:10 AM, coolmals <co...@gmail.com> wrote:
>>
>> Ok. Quoting also doesnt seem to fix the png image issue. Any idea on indexing
>> png images?
>>
>>
>>
>> --
>> View this message in context: http://lucene.472066.n3.nabble.com/Adding-additional-file-types-for-indexing-tp4222845p4222987.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Adding additional file types for indexing
Posted by Erik Hatcher <er...@gmail.com>.
Not sure how exactly you’re trying it, so let’s talk concretely with examples here:
----
$ bin/solr create -c tmp
$ bin/post -c tmp ~/Desktop/bin_post_featured_image.png -filetypes “*"
$ curl "http://localhost:8983/solr/tmp/select?q=*:*&wt=csv&fl=id"
id
"/Users/erikhatcher/Desktop/bin_post_featured_image.png”
----
There’s probably an issue in there somewhere, but it does work with the above.
I wasn’t able to get *this* to work though:
$ bin/post -c tmp http://lucene.apache.org/solr/assets/images/book_sia.png -filetypes “*”
...
Entering auto mode. Indexing pages with content-types corresponding to file endings *
Entering crawl at level 0 (1 links total, 1 new)
SimplePostTool: WARNING: Skipping URL with unsupported type image/png
So something needs fixing in web crawl mode it looks like.
—
Erik Hatcher, Senior Solutions Architect
http://www.lucidworks.com
> On Aug 14, 2015, at 11:10 AM, coolmals <co...@gmail.com> wrote:
>
> Ok. Quoting also doesnt seem to fix the png image issue. Any idea on indexing
> png images?
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Adding-additional-file-types-for-indexing-tp4222845p4222987.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Adding additional file types for indexing
Posted by coolmals <co...@gmail.com>.
Ok. Quoting also doesnt seem to fix the png image issue. Any idea on indexing
png images?
--
View this message in context: http://lucene.472066.n3.nabble.com/Adding-additional-file-types-for-indexing-tp4222845p4222987.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Adding additional file types for indexing
Posted by Erik Hatcher <er...@gmail.com>.
Be careful with an unquoted * - try quoting it. (I was trying to be precise in my reply by quoting it but should have called that out as a potential issue)
—
Erik Hatcher, Senior Solutions Architect
http://www.lucidworks.com <http://www.lucidworks.com/>
> On Aug 14, 2015, at 10:40 AM, coolmals <co...@gmail.com> wrote:
>
> Thanks for your response. I used the -Dfiletypes=* when executing post.jar
> but png image files alone are not getting indexed. It gives 400 error
>
>
> <response>
> <lst name="responseHeader">
> <int name="status">400</int>
> <int name="QTime">1</int>
> </lst>
> <lst name="error">
> <str name="msg">missing content stream</str>
> <int name="code">400</int>
> </lst>
> </response>
>
> Can you let me know if there is any seperate configuration for png images?
> other images like jpg seems to be indexed well.
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Adding-additional-file-types-for-indexing-tp4222845p4222977.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Adding additional file types for indexing
Posted by coolmals <co...@gmail.com>.
Thanks for your response. I used the -Dfiletypes=* when executing post.jar
but png image files alone are not getting indexed. It gives 400 error
<response>
<lst name="responseHeader">
<int name="status">400</int>
<int name="QTime">1</int>
</lst>
<lst name="error">
<str name="msg">missing content stream</str>
<int name="code">400</int>
</lst>
</response>
Can you let me know if there is any seperate configuration for png images?
other images like jpg seems to be indexed well.
--
View this message in context: http://lucene.472066.n3.nabble.com/Adding-additional-file-types-for-indexing-tp4222845p4222977.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Adding additional file types for indexing
Posted by Erik Hatcher <er...@gmail.com>.
You can set the -filetypes parameter (either to post.jar or to bin/post).
-filetypes <type>[,<type>,...] (default: xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log)
Try -filetypes “*” to attempt indexing every file.
—
Erik Hatcher, Senior Solutions Architect
http://www.lucidworks.com
> On Aug 13, 2015, at 3:59 PM, coolmals <co...@gmail.com> wrote:
>
> I am a new user to Solr. I am trying to index images using solr. Currently
> the jpg, png etc image files are skipped in auto mode
>
> Entering auto mode. File endings considered are
> xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,
> htm,html,txt,log
>
> only these files are indexed. How to index additional file types. Is there a
> configuration to add new file types for indexing?
>
> Regards,
> Malathy
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Adding-additional-file-types-for-indexing-tp4222845.html
> Sent from the Solr - User mailing list archive at Nabble.com.