Write-behind store behavior on node stop.

[Vladimir Ozerov <vo...@gridgain.com>]

Igniters,

Can someone explain me how write-behind store should behave in case of node
stop when some changes has not been flushed to the underlying store yet?
Are we guarantee that all pending changes will be flushed to the underlying
store?

Vladimir.

Re: Write-behind store behavior on node stop.

[Atri Sharma <at...@gmail.com>]

Done.

https://issues.apache.org/jira/browse/IGNITE-1096

On Mon, Jul 6, 2015 at 1:49 PM, Atri Sharma <at...@gmail.com> wrote:

> Let me do that
>
> On Mon, Jul 6, 2015 at 1:47 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
>> On Mon, Jul 6, 2015 at 1:13 AM, Atri Sharma <at...@gmail.com> wrote:
>>
>> > Sounds good.
>> >
>> > I also propose controlling this behaviour through user settable option
>> > (since this might potentially have some performance implications). So if
>> > the user is doing ad hoc analytics, he/she might not care much about
>> bit of
>> > data loss and might prefer the performance gain instead.
>> >
>>
>> I agree. Atri, can you please create a ticket describing this behavior? We
>> can move this discussion there.
>>
>>
>> >
>> > On Mon, Jul 6, 2015 at 1:40 PM, Dmitriy Setrakyan <
>> dsetrakyan@apache.org>
>> > wrote:
>> >
>> > > On Mon, Jul 6, 2015 at 1:00 AM, Atri Sharma <at...@gmail.com>
>> wrote:
>> > >
>> > > > Can some sort of persistent Write Ahead Logging help here?
>> > > >
>> > >
>> > > I think so. Perhaps we can have the same type of queue on the backup
>> > nodes
>> > > which will only get flushed in case of primary node failure?
>> > >
>> > >
>> > > >
>> > > > On Mon, Jul 6, 2015 at 11:32 AM, Dmitriy Setrakyan <
>> > > dsetrakyan@apache.org>
>> > > > wrote:
>> > > >
>> > > > > On Sun, Jul 5, 2015 at 10:58 PM, Vladimir Ozerov <
>> > vozerov@gridgain.com
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Igniters,
>> > > > > >
>> > > > > > Can someone explain me how write-behind store should behave in
>> case
>> > > of
>> > > > > node
>> > > > > > stop when some changes has not been flushed to the underlying
>> store
>> > > > yet?
>> > > > > > Are we guarantee that all pending changes will be flushed to the
>> > > > > underlying
>> > > > > > store?
>> > > > > >
>> > > > > >
>> > > > > I think we should make the best effort to persist the un-flushed
>> > > updates
>> > > > in
>> > > > > case of graceful exit. Of course, not guarantees can be provided,
>> and
>> > > if
>> > > > a
>> > > > > server stop abruptly, then some updates might be lost.
>> > > > >
>> > > > >
>> > > > > > Vladimir.
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Regards,
>> > > >
>> > > > Atri
>> > > > *l'apprenant*
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Regards,
>> >
>> > Atri
>> > *l'apprenant*
>> >
>>
>
>
>
> --
> Regards,
>
> Atri
> *l'apprenant*
>



-- 
Regards,

Atri
*l'apprenant*

Re: Write-behind store behavior on node stop.

[Atri Sharma <at...@gmail.com>]

Let me do that

On Mon, Jul 6, 2015 at 1:47 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> On Mon, Jul 6, 2015 at 1:13 AM, Atri Sharma <at...@gmail.com> wrote:
>
> > Sounds good.
> >
> > I also propose controlling this behaviour through user settable option
> > (since this might potentially have some performance implications). So if
> > the user is doing ad hoc analytics, he/she might not care much about bit
> of
> > data loss and might prefer the performance gain instead.
> >
>
> I agree. Atri, can you please create a ticket describing this behavior? We
> can move this discussion there.
>
>
> >
> > On Mon, Jul 6, 2015 at 1:40 PM, Dmitriy Setrakyan <dsetrakyan@apache.org
> >
> > wrote:
> >
> > > On Mon, Jul 6, 2015 at 1:00 AM, Atri Sharma <at...@gmail.com>
> wrote:
> > >
> > > > Can some sort of persistent Write Ahead Logging help here?
> > > >
> > >
> > > I think so. Perhaps we can have the same type of queue on the backup
> > nodes
> > > which will only get flushed in case of primary node failure?
> > >
> > >
> > > >
> > > > On Mon, Jul 6, 2015 at 11:32 AM, Dmitriy Setrakyan <
> > > dsetrakyan@apache.org>
> > > > wrote:
> > > >
> > > > > On Sun, Jul 5, 2015 at 10:58 PM, Vladimir Ozerov <
> > vozerov@gridgain.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > Can someone explain me how write-behind store should behave in
> case
> > > of
> > > > > node
> > > > > > stop when some changes has not been flushed to the underlying
> store
> > > > yet?
> > > > > > Are we guarantee that all pending changes will be flushed to the
> > > > > underlying
> > > > > > store?
> > > > > >
> > > > > >
> > > > > I think we should make the best effort to persist the un-flushed
> > > updates
> > > > in
> > > > > case of graceful exit. Of course, not guarantees can be provided,
> and
> > > if
> > > > a
> > > > > server stop abruptly, then some updates might be lost.
> > > > >
> > > > >
> > > > > > Vladimir.
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Atri
> > > > *l'apprenant*
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > Atri
> > *l'apprenant*
> >
>



-- 
Regards,

Atri
*l'apprenant*

Re: Write-behind store behavior on node stop.

[Atri Sharma <at...@gmail.com>]

Folks,

I thought more about this and came up with a WAL page formal that we can
maintain in a separate queue and flush to persistent store. This allows to
recover any lost queue data and also allows to rebuild a cluster from
scratch.

Dmitriy, I understand that you were inclined to have redundancy of write
behind store buffers for primary node on backup nodes as well and flush
them in case of failure. Does the WAL page idea fit in your mind? Else we
could go with pure redundancy based solution.

Thoughts?
On 6 Jul 2015 16:15, "Atri Sharma" <at...@gmail.com> wrote:

> Dmitriy,
>
> I created the ticket. Please let me know your thoughts around this.
>
> Once we have something in place, I will create page on wiki.
>
> On Mon, Jul 6, 2015 at 1:47 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
>> On Mon, Jul 6, 2015 at 1:13 AM, Atri Sharma <at...@gmail.com> wrote:
>>
>> > Sounds good.
>> >
>> > I also propose controlling this behaviour through user settable option
>> > (since this might potentially have some performance implications). So if
>> > the user is doing ad hoc analytics, he/she might not care much about
>> bit of
>> > data loss and might prefer the performance gain instead.
>> >
>>
>> I agree. Atri, can you please create a ticket describing this behavior? We
>> can move this discussion there.
>>
>>
>> >
>> > On Mon, Jul 6, 2015 at 1:40 PM, Dmitriy Setrakyan <
>> dsetrakyan@apache.org>
>> > wrote:
>> >
>> > > On Mon, Jul 6, 2015 at 1:00 AM, Atri Sharma <at...@gmail.com>
>> wrote:
>> > >
>> > > > Can some sort of persistent Write Ahead Logging help here?
>> > > >
>> > >
>> > > I think so. Perhaps we can have the same type of queue on the backup
>> > nodes
>> > > which will only get flushed in case of primary node failure?
>> > >
>> > >
>> > > >
>> > > > On Mon, Jul 6, 2015 at 11:32 AM, Dmitriy Setrakyan <
>> > > dsetrakyan@apache.org>
>> > > > wrote:
>> > > >
>> > > > > On Sun, Jul 5, 2015 at 10:58 PM, Vladimir Ozerov <
>> > vozerov@gridgain.com
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Igniters,
>> > > > > >
>> > > > > > Can someone explain me how write-behind store should behave in
>> case
>> > > of
>> > > > > node
>> > > > > > stop when some changes has not been flushed to the underlying
>> store
>> > > > yet?
>> > > > > > Are we guarantee that all pending changes will be flushed to the
>> > > > > underlying
>> > > > > > store?
>> > > > > >
>> > > > > >
>> > > > > I think we should make the best effort to persist the un-flushed
>> > > updates
>> > > > in
>> > > > > case of graceful exit. Of course, not guarantees can be provided,
>> and
>> > > if
>> > > > a
>> > > > > server stop abruptly, then some updates might be lost.
>> > > > >
>> > > > >
>> > > > > > Vladimir.
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Regards,
>> > > >
>> > > > Atri
>> > > > *l'apprenant*
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Regards,
>> >
>> > Atri
>> > *l'apprenant*
>> >
>>
>
>
>
> --
> Regards,
>
> Atri
> *l'apprenant*
>

Re: Write-behind store behavior on node stop.

[Atri Sharma <at...@gmail.com>]

Dmitriy,

I created the ticket. Please let me know your thoughts around this.

Once we have something in place, I will create page on wiki.

On Mon, Jul 6, 2015 at 1:47 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> On Mon, Jul 6, 2015 at 1:13 AM, Atri Sharma <at...@gmail.com> wrote:
>
> > Sounds good.
> >
> > I also propose controlling this behaviour through user settable option
> > (since this might potentially have some performance implications). So if
> > the user is doing ad hoc analytics, he/she might not care much about bit
> of
> > data loss and might prefer the performance gain instead.
> >
>
> I agree. Atri, can you please create a ticket describing this behavior? We
> can move this discussion there.
>
>
> >
> > On Mon, Jul 6, 2015 at 1:40 PM, Dmitriy Setrakyan <dsetrakyan@apache.org
> >
> > wrote:
> >
> > > On Mon, Jul 6, 2015 at 1:00 AM, Atri Sharma <at...@gmail.com>
> wrote:
> > >
> > > > Can some sort of persistent Write Ahead Logging help here?
> > > >
> > >
> > > I think so. Perhaps we can have the same type of queue on the backup
> > nodes
> > > which will only get flushed in case of primary node failure?
> > >
> > >
> > > >
> > > > On Mon, Jul 6, 2015 at 11:32 AM, Dmitriy Setrakyan <
> > > dsetrakyan@apache.org>
> > > > wrote:
> > > >
> > > > > On Sun, Jul 5, 2015 at 10:58 PM, Vladimir Ozerov <
> > vozerov@gridgain.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > Can someone explain me how write-behind store should behave in
> case
> > > of
> > > > > node
> > > > > > stop when some changes has not been flushed to the underlying
> store
> > > > yet?
> > > > > > Are we guarantee that all pending changes will be flushed to the
> > > > > underlying
> > > > > > store?
> > > > > >
> > > > > >
> > > > > I think we should make the best effort to persist the un-flushed
> > > updates
> > > > in
> > > > > case of graceful exit. Of course, not guarantees can be provided,
> and
> > > if
> > > > a
> > > > > server stop abruptly, then some updates might be lost.
> > > > >
> > > > >
> > > > > > Vladimir.
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Atri
> > > > *l'apprenant*
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > Atri
> > *l'apprenant*
> >
>



-- 
Regards,

Atri
*l'apprenant*

Re: Write-behind store behavior on node stop.

[Dmitriy Setrakyan <ds...@apache.org>]

On Mon, Jul 6, 2015 at 1:13 AM, Atri Sharma <at...@gmail.com> wrote:

> Sounds good.
>
> I also propose controlling this behaviour through user settable option
> (since this might potentially have some performance implications). So if
> the user is doing ad hoc analytics, he/she might not care much about bit of
> data loss and might prefer the performance gain instead.
>

I agree. Atri, can you please create a ticket describing this behavior? We
can move this discussion there.


>
> On Mon, Jul 6, 2015 at 1:40 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > On Mon, Jul 6, 2015 at 1:00 AM, Atri Sharma <at...@gmail.com> wrote:
> >
> > > Can some sort of persistent Write Ahead Logging help here?
> > >
> >
> > I think so. Perhaps we can have the same type of queue on the backup
> nodes
> > which will only get flushed in case of primary node failure?
> >
> >
> > >
> > > On Mon, Jul 6, 2015 at 11:32 AM, Dmitriy Setrakyan <
> > dsetrakyan@apache.org>
> > > wrote:
> > >
> > > > On Sun, Jul 5, 2015 at 10:58 PM, Vladimir Ozerov <
> vozerov@gridgain.com
> > >
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > Can someone explain me how write-behind store should behave in case
> > of
> > > > node
> > > > > stop when some changes has not been flushed to the underlying store
> > > yet?
> > > > > Are we guarantee that all pending changes will be flushed to the
> > > > underlying
> > > > > store?
> > > > >
> > > > >
> > > > I think we should make the best effort to persist the un-flushed
> > updates
> > > in
> > > > case of graceful exit. Of course, not guarantees can be provided, and
> > if
> > > a
> > > > server stop abruptly, then some updates might be lost.
> > > >
> > > >
> > > > > Vladimir.
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > Atri
> > > *l'apprenant*
> > >
> >
>
>
>
> --
> Regards,
>
> Atri
> *l'apprenant*
>

Re: Write-behind store behavior on node stop.

[Atri Sharma <at...@gmail.com>]

Sounds good.

I also propose controlling this behaviour through user settable option
(since this might potentially have some performance implications). So if
the user is doing ad hoc analytics, he/she might not care much about bit of
data loss and might prefer the performance gain instead.

On Mon, Jul 6, 2015 at 1:40 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> On Mon, Jul 6, 2015 at 1:00 AM, Atri Sharma <at...@gmail.com> wrote:
>
> > Can some sort of persistent Write Ahead Logging help here?
> >
>
> I think so. Perhaps we can have the same type of queue on the backup nodes
> which will only get flushed in case of primary node failure?
>
>
> >
> > On Mon, Jul 6, 2015 at 11:32 AM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> > wrote:
> >
> > > On Sun, Jul 5, 2015 at 10:58 PM, Vladimir Ozerov <vozerov@gridgain.com
> >
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > Can someone explain me how write-behind store should behave in case
> of
> > > node
> > > > stop when some changes has not been flushed to the underlying store
> > yet?
> > > > Are we guarantee that all pending changes will be flushed to the
> > > underlying
> > > > store?
> > > >
> > > >
> > > I think we should make the best effort to persist the un-flushed
> updates
> > in
> > > case of graceful exit. Of course, not guarantees can be provided, and
> if
> > a
> > > server stop abruptly, then some updates might be lost.
> > >
> > >
> > > > Vladimir.
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > Atri
> > *l'apprenant*
> >
>



-- 
Regards,

Atri
*l'apprenant*

Re: Write-behind store behavior on node stop.

[Dmitriy Setrakyan <ds...@apache.org>]

On Mon, Jul 6, 2015 at 1:00 AM, Atri Sharma <at...@gmail.com> wrote:

> Can some sort of persistent Write Ahead Logging help here?
>

I think so. Perhaps we can have the same type of queue on the backup nodes
which will only get flushed in case of primary node failure?


>
> On Mon, Jul 6, 2015 at 11:32 AM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > On Sun, Jul 5, 2015 at 10:58 PM, Vladimir Ozerov <vo...@gridgain.com>
> > wrote:
> >
> > > Igniters,
> > >
> > > Can someone explain me how write-behind store should behave in case of
> > node
> > > stop when some changes has not been flushed to the underlying store
> yet?
> > > Are we guarantee that all pending changes will be flushed to the
> > underlying
> > > store?
> > >
> > >
> > I think we should make the best effort to persist the un-flushed updates
> in
> > case of graceful exit. Of course, not guarantees can be provided, and if
> a
> > server stop abruptly, then some updates might be lost.
> >
> >
> > > Vladimir.
> > >
> >
>
>
>
> --
> Regards,
>
> Atri
> *l'apprenant*
>

Re: Write-behind store behavior on node stop.

[Atri Sharma <at...@gmail.com>]

Can some sort of persistent Write Ahead Logging help here?

On Mon, Jul 6, 2015 at 11:32 AM, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> On Sun, Jul 5, 2015 at 10:58 PM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
>
> > Igniters,
> >
> > Can someone explain me how write-behind store should behave in case of
> node
> > stop when some changes has not been flushed to the underlying store yet?
> > Are we guarantee that all pending changes will be flushed to the
> underlying
> > store?
> >
> >
> I think we should make the best effort to persist the un-flushed updates in
> case of graceful exit. Of course, not guarantees can be provided, and if a
> server stop abruptly, then some updates might be lost.
>
>
> > Vladimir.
> >
>



-- 
Regards,

Atri
*l'apprenant*

Re: Write-behind store behavior on node stop.

[Dmitriy Setrakyan <ds...@apache.org>]

On Sun, Jul 5, 2015 at 10:58 PM, Vladimir Ozerov <vo...@gridgain.com>
wrote:

> Igniters,
>
> Can someone explain me how write-behind store should behave in case of node
> stop when some changes has not been flushed to the underlying store yet?
> Are we guarantee that all pending changes will be flushed to the underlying
> store?
>
>
I think we should make the best effort to persist the un-flushed updates in
case of graceful exit. Of course, not guarantees can be provided, and if a
server stop abruptly, then some updates might be lost.


> Vladimir.
>