You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by sy...@web.de on 2012/09/07 19:31:39 UTC
Schema model to store additional field metadata
Hi,
I want to create a Solr index of articles. Each article should have a title, content, published date and an arbitrary number of images attached to. An article could look like this:
title: An article about Foo and Bar
content: This is some text about Foo and Bar.
published: 2012.09.07T19:23
image: 2012/09/foo.png
image: 2012/04/images/bar.png
image: 2012/02/abc.png
I want to display the images with html <img>-tags. But despite src I also want to include an alt attribute to describe each image with additional metadata. For example I want to display the article like this:
<h3>An article about Foo and Bar</h3>
<p>This is some text about Foo and Bar.</p>
<img src="2012/09/foo.png" alt="Foo. Waiting for the bus." />
<img src="2012/04/images/bar.png" alt="Bar again" />
<img src="2012/02/abc.png" alt="Foo and Bar at the beach" />
I know that I can use a multiValued field to store the images. But how should or I can store the additional src information? I have a problem finding the right schema for my index.
Re: Re: Schema model to store additional field metadata
Posted by Chris Hostetter <ho...@fucit.org>.
: So, let's assume you want to provide links to the images.
: Having a field (multiValued?) with index="false" and stored="true"
: would allow you to store all the img urls in a single field.
:
: All that said, now it's up to your application layer that
: constructs the pages for presentation to the user to
: "do the right thing" with the returned fields to allow
: images (or whatever) to be displayed.
Not yet mentioned in this thread: you are garunteed that the order you put
the values into a multiValued="true" stored="true" field is the order
those values will be returned when you ask for the field later.
so if you have a multiValued "img_src" field and a multiValued "img_alt"
field, and you add data to those fields i na loop over your image objects,
you can later know that the Nth value in img_src corrisponds to the Nth
value in img_alt
-Hoss
Re: Re: Schema model to store additional field metadata
Posted by Erick Erickson <er...@gmail.com>.
You might be confusing indexing and storing. When you
specify index="true" in your field definition, the input
is tokenized, transformed, etc and the results of this
(see the admin/analysis) page is what is searched.
But when you specify stored="true", a literal, verbatim
copy of the text is put in a distinct file, and when you
return data (e.g. fl=field1, field2...) then the verbatim
copy is returned.
If you specify both index="true" and stored="true" both
things happen, but they're entirely separate operations
even though they're on the same field....
So, let's assume you want to provide links to the images.
Having a field (multiValued?) with index="false" and stored="true"
would allow you to store all the img urls in a single field.
All that said, now it's up to your application layer that
constructs the pages for presentation to the user to
"do the right thing" with the returned fields to allow
images (or whatever) to be displayed.
Best
Erick
On Fri, Sep 7, 2012 at 12:03 PM, <sy...@web.de> wrote:
>> Why would you store the actual images in SOLR?
>
> No, the images are files on the filesystem. Only the path to the image should be stored in Solr.
>
>> And you are most likely looking at dynamic fields as the solution
>>
>> 1) Define *_Path, *_Size, *_Alt as a dynamic field with appropriate types
>> 2) During indexing, write those properties as Image_1_Path,
>> Image_1_Size, Image_1_Alt or some such
>> 3) Make sure that whatever search algorithm you have looks at those or
>> do a copyField to aggregate them into AllImage_Alt, etc.
>
> I was also thinking of a solution with dynamic fields, but I am very new to Solr and I am not sure if it is a good solution to solve this modelling issue. For example I thought about introducing two multiValued dynamic fields (image_src_*, image_alt_*) and store image data like file path on disc and alt-attribute like this:
>
> title: An article about Foo and Bar
> content: This is some text about Foo and Bar.
> published: 2012.09.07T19:23
> image_src_1: 2012/09/foo.png
> image_alt_1: Foo. Waiting for the bus.
> image_src_2: 2012/04/images/bar.png
> image_src_3: 2012/02/abc.png
> image_alt_3: Foo and Bar at the beach
>
> Of course a alt attribute for some images could be missing. I don't know if this is a good or better solution for this. It feels clumsy to me, like a workaround. But maybe this is the way to model this data, I don't know?
Re: Re: Schema model to store additional field metadata
Posted by sy...@web.de.
> Why would you store the actual images in SOLR?
No, the images are files on the filesystem. Only the path to the image should be stored in Solr.
> And you are most likely looking at dynamic fields as the solution
>
> 1) Define *_Path, *_Size, *_Alt as a dynamic field with appropriate types
> 2) During indexing, write those properties as Image_1_Path,
> Image_1_Size, Image_1_Alt or some such
> 3) Make sure that whatever search algorithm you have looks at those or
> do a copyField to aggregate them into AllImage_Alt, etc.
I was also thinking of a solution with dynamic fields, but I am very new to Solr and I am not sure if it is a good solution to solve this modelling issue. For example I thought about introducing two multiValued dynamic fields (image_src_*, image_alt_*) and store image data like file path on disc and alt-attribute like this:
title: An article about Foo and Bar
content: This is some text about Foo and Bar.
published: 2012.09.07T19:23
image_src_1: 2012/09/foo.png
image_alt_1: Foo. Waiting for the bus.
image_src_2: 2012/04/images/bar.png
image_src_3: 2012/02/abc.png
image_alt_3: Foo and Bar at the beach
Of course a alt attribute for some images could be missing. I don't know if this is a good or better solution for this. It feels clumsy to me, like a workaround. But maybe this is the way to model this data, I don't know?
Re: Schema model to store additional field metadata
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
Why would you store the actual images in SOLR? There is no way to
really search the bytes of image, is there? What you probably want to
do is extract all searchable metadata out of that image, name, alt,
EXIF, etc.
And you are most likely looking at dynamic fields as the solution
1) Define *_Path, *_Size, *_Alt as a dynamic field with appropriate types
2) During indexing, write those properties as Image_1_Path,
Image_1_Size, Image_1_Alt or some such
3) Make sure that whatever search algorithm you have looks at those or
do a copyField to aggregate them into AllImage_Alt, etc.
I do something similar by extracting metadata from .DOC files with
Tika and indexing it all regardless of the actual names.
Regards,
Alex.
Personal blog: http://blog.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all
at once. Lately, it doesn't seem to be working. (Anonymous - via GTD
book)
On Fri, Sep 7, 2012 at 1:31 PM, <sy...@web.de> wrote:
> Hi,
>
> I want to create a Solr index of articles. Each article should have a title, content, published date and an arbitrary number of images attached to. An article could look like this:
>
> title: An article about Foo and Bar
> content: This is some text about Foo and Bar.
> published: 2012.09.07T19:23
> image: 2012/09/foo.png
> image: 2012/04/images/bar.png
> image: 2012/02/abc.png
>
> I want to display the images with html <img>-tags. But despite src I also want to include an alt attribute to describe each image with additional metadata. For example I want to display the article like this:
>
> <h3>An article about Foo and Bar</h3>
> <p>This is some text about Foo and Bar.</p>
> <img src="2012/09/foo.png" alt="Foo. Waiting for the bus." />
> <img src="2012/04/images/bar.png" alt="Bar again" />
> <img src="2012/02/abc.png" alt="Foo and Bar at the beach" />
>
> I know that I can use a multiValued field to store the images. But how should or I can store the additional src information? I have a problem finding the right schema for my index.