Multipart documents with different update cycles

[Aurélien MAZOYER <au...@francelabs.com>]

Hello,

I have to index a dataset containing multipart documents. The "main" 
part and the "user metadata" part have different update cycles : we want 
to update the "user metadata part" frequently without having to refetch 
the main part from the datasource nor storing every fields in order to 
use atomic update. As there is no true field level update in Solr yet, I 
am afraid that I have to build an index for both parts and to perform a 
query time join, with all the well-known performance limitation. I have 
also heard of side car index. Is it a solution that can meet my 
requirements? Is it stable enough to be usable in production? Does the 
community plan to make it part of the trunk code?

Thanks,

Aurelien

Re: Multipart documents with different update cycles

[Alexandre Rafalovitch <ar...@gmail.com>]

Do you search the frequently changing user-metadata? If not, maybe the
external file field is helpful.
https://cwiki.apache.org/confluence/display/solr/Working+with+External+Files+and+Processes

Regards,
   Alex.
Personal: http://www.outerthoughts.com/ and @arafalov
Solr resources and newsletter: http://www.solr-start.com/ and @solrstart
Solr popularizers community: https://www.linkedin.com/groups?gid=6713853


On Fri, Jul 25, 2014 at 12:04 AM, Aurélien MAZOYER
<au...@francelabs.com> wrote:
> Hello,
>
> I have to index a dataset containing multipart documents. The "main" part
> and the "user metadata" part have different update cycles : we want to
> update the "user metadata part" frequently without having to refetch the
> main part from the datasource nor storing every fields in order to use
> atomic update. As there is no true field level update in Solr yet, I am
> afraid that I have to build an index for both parts and to perform a query
> time join, with all the well-known performance limitation. I have also heard
> of side car index. Is it a solution that can meet my requirements? Is it
> stable enough to be usable in production? Does the community plan to make it
> part of the trunk code?
>
> Thanks,
>
> Aurelien
>