OakDirectory with cluster

[Shinichiro Abe <sh...@gmail.com>]

Hello,

If I understand correctly, by OakDirectory, a Lucene index is placed as Blob on a Storage.
If there is an Oak cluster condition, Oak instances on the cluster will have a LuceneIndexEditor individually in which IndexWriter is working.
In this case when multiple IndexWriters update documents against one OakDirectory on the same timing, 
I think it will lead to an index corruption even though OakDirectory controls a lock with NoLockFactory. 
How are you avoiding this? Oak has a OakDirectory per cluster id? Oak impl locks other writer behaviors during one update?
I'd like to oak-lucene design. Where the code should I see?
Background: I'd like to get a hint from Oak team for CONNECTORS-1219 where I'm working on,
I have cluster(multiprocess) condition problem. Currently I hit index corruption when multiple IndexWriters write to an index.

Thanks in advance.
Shinichiro Abe

Re: OakDirectory with cluster

[Shinichiro Abe <sh...@gmail.com>]

Hi Chetan,

Thanks. That means a direct use of RAMDirectory for NRT functionality instead of NRTCachingDirectory, 
and using OakDirectory as persistent directory in parallel? It looks like tricky a little bit for me but an interesting idea.

Thank you very much,
Shinichiro Abe

> 2015/08/05 14:56、Chetan Mehrotra <ch...@gmail.com> のメール：
> 
> Hi Abe-san,
> 
> On Wed, Aug 5, 2015 at 7:48 AM, Shinichiro Abe
> <sh...@gmail.com> wrote:
>> I saw the discussion that you proposed about NRT searching in Oak a few weeks ago.
>> NRT search requires an IndexWriter which has uncommitted documents in-memory.
>> If users do a NRT searching, do you intend to get the writer from that singleton writer service?
> 
> My reference to NRT in that thread was bit overloaded usage. What I
> meant was to enable search for data added on same cluster node if the
> user has a session affinity with that node. For e.g. if a cluster
> consist of 2 nodes N1 and N2 and user U1 is always interacting with N1
> i.e. adding new images etc. Then to enable search for images added by
> him on N1 faster the proposal was to keep recently added data in an
> index backed by RAMDirectory. Any query would then consult both
> persistent index and the in mem index.
> 
> User U2 interacting with N2 server would still not be able see changes
> done by U1 on N1.
> 
>> (I don't know how to get the writer from another process.)
> Yup thats not possible.
> 
>> I'm interested in Oak from 1 year ago but didn't take Sling into account until today.
> 
> Sling is just a host application which embeds Oak and Oak relies on
> host application for certain aspects like coordinating the jobs across
> cluster.
> 
> Chetan Mehrotra

Re: OakDirectory with cluster

[Chetan Mehrotra <ch...@gmail.com>]

Hi Abe-san,

On Wed, Aug 5, 2015 at 7:48 AM, Shinichiro Abe
<sh...@gmail.com> wrote:
> I saw the discussion that you proposed about NRT searching in Oak a few weeks ago.
> NRT search requires an IndexWriter which has uncommitted documents in-memory.
> If users do a NRT searching, do you intend to get the writer from that singleton writer service?

My reference to NRT in that thread was bit overloaded usage. What I
meant was to enable search for data added on same cluster node if the
user has a session affinity with that node. For e.g. if a cluster
consist of 2 nodes N1 and N2 and user U1 is always interacting with N1
i.e. adding new images etc. Then to enable search for images added by
him on N1 faster the proposal was to keep recently added data in an
index backed by RAMDirectory. Any query would then consult both
persistent index and the in mem index.

User U2 interacting with N2 server would still not be able see changes
done by U1 on N1.

> (I don't know how to get the writer from another process.)
Yup thats not possible.

> I'm interested in Oak from 1 year ago but didn't take Sling into account until today.

Sling is just a host application which embeds Oak and Oak relies on
host application for certain aspects like coordinating the jobs across
cluster.

Chetan Mehrotra

Re: OakDirectory with cluster

[Shinichiro Abe <sh...@gmail.com>]

Hi Mehrotra,

Thank you for the reply. It is very helpful for me.
I didn't know Sling Scheduler Service writes documents to an index by a single thread.
I saw the discussion that you proposed about NRT searching in Oak a few weeks ago.
NRT search requires an IndexWriter which has uncommitted documents in-memory.
If users do a NRT searching, do you intend to get the writer from that singleton writer service?
Or to get from a writer on each node in a cluster?
(I don't know how to get the writer from another process.)
Do you assume Oak can do that by OSGi/Sling implementation?
I'm interested in Oak from 1 year ago but didn't take Sling into account until today.
If my questions are incorrect, I'm sorry for that.

Regards,
Shinichiro Abe

> 2015/08/04 22:47、Chetan Mehrotra <ch...@gmail.com> のメール：
> 
> Lucene indexing in Oak is done via a single thread in a cluster. The
> AsyncIndex job which is responsible for performing the Lucene indexing
> is scheduled to run as a singleton in the cluster [1]. For this Oak
> depends on the embedding application. For e.g. when running in Apache
> Sling the Job is registered with following properties
> 
> * scheduler.period=5
> * scheduler.runOn=SINGLE
> 
> And Sling scheduler component then ensures that this job (i.e.
> indexing) is only performed on a one node in a cluster. Oak internally
> also ensures that at a time only one index is running via some state
> management in :async hidden node. So essentially by design we enforce
> that no concurrent indexing should happen and hence the use of
> NoLockFactory
> Chetan Mehrotra
> 
> 
> On Tue, Aug 4, 2015 at 5:40 PM, Shinichiro Abe
> <sh...@gmail.com> wrote:
>> Hello,
>> 
>> If I understand correctly, by OakDirectory, a Lucene index is placed as Blob on a Storage.
>> If there is an Oak cluster condition, Oak instances on the cluster will have a LuceneIndexEditor individually in which IndexWriter is working.
>> In this case when multiple IndexWriters update documents against one OakDirectory on the same timing,
>> I think it will lead to an index corruption even though OakDirectory controls a lock with NoLockFactory.
>> How are you avoiding this? Oak has a OakDirectory per cluster id? Oak impl locks other writer behaviors during one update?
>> I'd like to oak-lucene design. Where the code should I see?
>> Background: I'd like to get a hint from Oak team for CONNECTORS-1219 where I'm working on,
>> I have cluster(multiprocess) condition problem. Currently I hit index corruption when multiple IndexWriters write to an index.
>> 
>> Thanks in advance.
>> Shinichiro Abe

Re: OakDirectory with cluster

[Chetan Mehrotra <ch...@gmail.com>]

Lucene indexing in Oak is done via a single thread in a cluster. The
AsyncIndex job which is responsible for performing the Lucene indexing
is scheduled to run as a singleton in the cluster [1]. For this Oak
depends on the embedding application. For e.g. when running in Apache
Sling the Job is registered with following properties

* scheduler.period=5
* scheduler.runOn=SINGLE

And Sling scheduler component then ensures that this job (i.e.
indexing) is only performed on a one node in a cluster. Oak internally
also ensures that at a time only one index is running via some state
management in :async hidden node. So essentially by design we enforce
that no concurrent indexing should happen and hence the use of
NoLockFactory
Chetan Mehrotra


On Tue, Aug 4, 2015 at 5:40 PM, Shinichiro Abe
<sh...@gmail.com> wrote:
> Hello,
>
> If I understand correctly, by OakDirectory, a Lucene index is placed as Blob on a Storage.
> If there is an Oak cluster condition, Oak instances on the cluster will have a LuceneIndexEditor individually in which IndexWriter is working.
> In this case when multiple IndexWriters update documents against one OakDirectory on the same timing,
> I think it will lead to an index corruption even though OakDirectory controls a lock with NoLockFactory.
> How are you avoiding this? Oak has a OakDirectory per cluster id? Oak impl locks other writer behaviors during one update?
> I'd like to oak-lucene design. Where the code should I see?
> Background: I'd like to get a hint from Oak team for CONNECTORS-1219 where I'm working on,
> I have cluster(multiprocess) condition problem. Currently I hit index corruption when multiple IndexWriters write to an index.
>
> Thanks in advance.
> Shinichiro Abe