storing a lot of values

[Yan Georget <ya...@sap.com>]

Hi,
 
I am facing the following problem: I want to store and update a lot (say 500 000) of values eg Longs.
Is it better to:
- store each value as a property of a node, in which case I'll have a lot of nodes and Lucene will spend a lot of time indexing.
or to:
- use a multi valued property, in which case I don't know how to update a single value without updating the entire array of values.
 
This brings me to the following question: why isn't there a method on multi valued properties to update one of the values of the array?
 
Thanks in advance,
Yan

Re: storing a lot of values

[Marcel Reutegger <ma...@gmx.net>]

Yan Georget wrote:
>> how often are those values updated? An initial import will certainly
>> take time,
>> but if you update the values only occasionally that shouldn't be a
>> problem.
> 
> Is such a number of nodes acceptable?

sure, that shouldn't be a problem.

> Updates won't probably happen frequently but need to be quick.

I understand you will only update some of the values, right? This is fairly quick.

regards
  marcel

RE: storing a lot of values

[Yan Georget <ya...@sap.com>]

> how often are those values updated? An initial import will certainly
> take time,
> but if you update the values only occasionally that shouldn't be a
> problem.

Is such a number of nodes acceptable?
Updates won't probably happen frequently but need to be quick.
 
> that's how it is specified and implemented in Jackrabbit. Updating
just
> a single
> value is not possible.

Too bad ...
 
> How about breaking the values up and distribute them across a small
> node
> hierarchy? Is there any inherent structure in the value series that
you
> can take
> advantage of?

I was considering this option even if in this case there is no inherent
structure.

Thanks for your comments.

Yan

Re: storing a lot of values

[Marcel Reutegger <ma...@gmx.net>]

Hi,

Yan Georget wrote:
> I am facing the following problem: I want to store and update a lot (say 500
> 000) of values eg Longs. Is it better to: - store each value as a property of
> a node, in which case I'll have a lot of nodes and Lucene will spend a lot of
> time indexing.

how often are those values updated? An initial import will certainly take time, 
but if you update the values only occasionally that shouldn't be a problem.

> or to: - use a multi valued property, in which case I don't
> know how to update a single value without updating the entire array of
> values.

that's how it is specified and implemented in Jackrabbit. Updating just a single 
value is not possible.

> This brings me to the following question: why isn't there a method on multi
> valued properties to update one of the values of the array?

I guess the EG did not consider such a use case with thousands of values for a 
multi-valued property.

How about breaking the values up and distribute them across a small node 
hierarchy? Is there any inherent structure in the value series that you can take 
advantage of?

regards
  marcel