Disambiguate cell time for 1.0

[Nick Dimiduk <nd...@gmail.com>]

Reading through the write path, it seems to me that
RSRpcServices#doBatchOp(RegionActionResult.Builder,HRegion,List<Action>
mutations,CellScanner) should be honoring a nonce if present. The reason
being: if a client sends some Puts without specifying a TS, it relies on
the RS to provide one. Should such an operation succeed on the server but
the ACK not reach the client, client may resend the operation, silently
inserting more cells than intended. Deletes may well be a more sinister
issue, removing more cells than intended.

I've not yet written a test to confirm this.

There was conversation around the implementation of nonces discussing
options for removing the coupling of TS to clock-on-the-wall time. Sergey
describes the current situation quite eloquently: "server-generated TS
provide illusion of consistency guarantees which is not there by any
means". A fix for this will likely subtly break the semantics of our data
coordinates, and so should be addressed with 1.0, perhaps along side a
revamped client-side API.

-n

Re: Disambiguate cell time for 1.0

[Nick Dimiduk <nd...@gmail.com>]

Yes, 10247 is precisely where we should have this discussion. Thanks for
pointing it out.


On Thu, Apr 10, 2014 at 8:22 PM, lars hofhansl <la...@apache.org> wrote:

> Should we discuss this here:
> https://issues.apache.org/jira/browse/HBASE-10247?
>
>
>
> ________________________________
>  From: Sergey Shelukhin <se...@hortonworks.com>
> To: dev@hbase.apache.org
> Sent: Thursday, April 10, 2014 6:16 PM
> Subject: Re: Disambiguate cell time for 1.0
>
>
> Adding nonces to deletes and puts is possible, but it is overhead...
>
>
>
> On Thu, Apr 10, 2014 at 6:16 PM, Sergey Shelukhin <sergey@hortonworks.com
> >wrote:
>
> > Just to clarify what I was saying, there was a JIRA where we were
> > discussing this.
> > Server clocks are unreliable and when region moves (or if we have
> writable
> > replicas) between-server clocks are even more sor.
> > So we cannot really promise consistency wrt order even for requests from
> > the same client now; even if we manage TS 100%.
> > My suggestion was that we have 3 modes: (1) we manage TS on server => TS
> > gets written as seqNum which has guarantees, no client-supplied TS; (2)
> > client manages TS and supplies it, fail put if no TS; (3) legacy compat
> > mode, but TS is generated in the client library instead of RS, so the
> onus
> > is on the client app/client's server to manage clocks and there's no
> > expectation that HBase will guarantee order.
> >
> >
> >
> > On Thu, Apr 10, 2014 at 5:09 PM, Nick Dimiduk <nd...@gmail.com>
> wrote:
> >
> >> Reading through the write path, it seems to me that
> >> RSRpcServices#doBatchOp(RegionActionResult.Builder,HRegion,List<Action>
> >> mutations,CellScanner) should be honoring a nonce if present. The reason
> >> being: if a client sends some Puts without specifying a TS, it relies on
> >> the RS to provide one. Should such an operation succeed on the server
> but
> >> the ACK not reach the client, client may resend the operation, silently
> >> inserting more cells than intended. Deletes may well be a more sinister
> >> issue, removing more cells than intended.
> >>
> >> I've not yet written a test to confirm this.
> >>
> >> There was conversation around the implementation of nonces discussing
> >> options for removing the coupling of TS to clock-on-the-wall time.
> Sergey
> >> describes the current situation quite eloquently: "server-generated TS
> >> provide illusion of consistency guarantees which is not there by any
> >> means". A fix for this will likely subtly break the semantics of our
> data
> >> coordinates, and so should be addressed with 1.0, perhaps along side a
> >> revamped client-side API.
> >>
> >> -n
> >>
> >
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: Disambiguate cell time for 1.0

[lars hofhansl <la...@apache.org>]

Should we discuss this here: https://issues.apache.org/jira/browse/HBASE-10247?



________________________________
 From: Sergey Shelukhin <se...@hortonworks.com>
To: dev@hbase.apache.org 
Sent: Thursday, April 10, 2014 6:16 PM
Subject: Re: Disambiguate cell time for 1.0
 

Adding nonces to deletes and puts is possible, but it is overhead...



On Thu, Apr 10, 2014 at 6:16 PM, Sergey Shelukhin <se...@hortonworks.com>wrote:

> Just to clarify what I was saying, there was a JIRA where we were
> discussing this.
> Server clocks are unreliable and when region moves (or if we have writable
> replicas) between-server clocks are even more sor.
> So we cannot really promise consistency wrt order even for requests from
> the same client now; even if we manage TS 100%.
> My suggestion was that we have 3 modes: (1) we manage TS on server => TS
> gets written as seqNum which has guarantees, no client-supplied TS; (2)
> client manages TS and supplies it, fail put if no TS; (3) legacy compat
> mode, but TS is generated in the client library instead of RS, so the onus
> is on the client app/client's server to manage clocks and there's no
> expectation that HBase will guarantee order.
>
>
>
> On Thu, Apr 10, 2014 at 5:09 PM, Nick Dimiduk <nd...@gmail.com> wrote:
>
>> Reading through the write path, it seems to me that
>> RSRpcServices#doBatchOp(RegionActionResult.Builder,HRegion,List<Action>
>> mutations,CellScanner) should be honoring a nonce if present. The reason
>> being: if a client sends some Puts without specifying a TS, it relies on
>> the RS to provide one. Should such an operation succeed on the server but
>> the ACK not reach the client, client may resend the operation, silently
>> inserting more cells than intended. Deletes may well be a more sinister
>> issue, removing more cells than intended.
>>
>> I've not yet written a test to confirm this.
>>
>> There was conversation around the implementation of nonces discussing
>> options for removing the coupling of TS to clock-on-the-wall time. Sergey
>> describes the current situation quite eloquently: "server-generated TS
>> provide illusion of consistency guarantees which is not there by any
>> means". A fix for this will likely subtly break the semantics of our data
>> coordinates, and so should be addressed with 1.0, perhaps along side a
>> revamped client-side API.
>>
>> -n
>>
>
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Disambiguate cell time for 1.0

[Sergey Shelukhin <se...@hortonworks.com>]

Adding nonces to deletes and puts is possible, but it is overhead...


On Thu, Apr 10, 2014 at 6:16 PM, Sergey Shelukhin <se...@hortonworks.com>wrote:

> Just to clarify what I was saying, there was a JIRA where we were
> discussing this.
> Server clocks are unreliable and when region moves (or if we have writable
> replicas) between-server clocks are even more sor.
> So we cannot really promise consistency wrt order even for requests from
> the same client now; even if we manage TS 100%.
> My suggestion was that we have 3 modes: (1) we manage TS on server => TS
> gets written as seqNum which has guarantees, no client-supplied TS; (2)
> client manages TS and supplies it, fail put if no TS; (3) legacy compat
> mode, but TS is generated in the client library instead of RS, so the onus
> is on the client app/client's server to manage clocks and there's no
> expectation that HBase will guarantee order.
>
>
>
> On Thu, Apr 10, 2014 at 5:09 PM, Nick Dimiduk <nd...@gmail.com> wrote:
>
>> Reading through the write path, it seems to me that
>> RSRpcServices#doBatchOp(RegionActionResult.Builder,HRegion,List<Action>
>> mutations,CellScanner) should be honoring a nonce if present. The reason
>> being: if a client sends some Puts without specifying a TS, it relies on
>> the RS to provide one. Should such an operation succeed on the server but
>> the ACK not reach the client, client may resend the operation, silently
>> inserting more cells than intended. Deletes may well be a more sinister
>> issue, removing more cells than intended.
>>
>> I've not yet written a test to confirm this.
>>
>> There was conversation around the implementation of nonces discussing
>> options for removing the coupling of TS to clock-on-the-wall time. Sergey
>> describes the current situation quite eloquently: "server-generated TS
>> provide illusion of consistency guarantees which is not there by any
>> means". A fix for this will likely subtly break the semantics of our data
>> coordinates, and so should be addressed with 1.0, perhaps along side a
>> revamped client-side API.
>>
>> -n
>>
>
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Disambiguate cell time for 1.0

[Sergey Shelukhin <se...@hortonworks.com>]

Just to clarify what I was saying, there was a JIRA where we were
discussing this.
Server clocks are unreliable and when region moves (or if we have writable
replicas) between-server clocks are even more sor.
So we cannot really promise consistency wrt order even for requests from
the same client now; even if we manage TS 100%.
My suggestion was that we have 3 modes: (1) we manage TS on server => TS
gets written as seqNum which has guarantees, no client-supplied TS; (2)
client manages TS and supplies it, fail put if no TS; (3) legacy compat
mode, but TS is generated in the client library instead of RS, so the onus
is on the client app/client's server to manage clocks and there's no
expectation that HBase will guarantee order.



On Thu, Apr 10, 2014 at 5:09 PM, Nick Dimiduk <nd...@gmail.com> wrote:

> Reading through the write path, it seems to me that
> RSRpcServices#doBatchOp(RegionActionResult.Builder,HRegion,List<Action>
> mutations,CellScanner) should be honoring a nonce if present. The reason
> being: if a client sends some Puts without specifying a TS, it relies on
> the RS to provide one. Should such an operation succeed on the server but
> the ACK not reach the client, client may resend the operation, silently
> inserting more cells than intended. Deletes may well be a more sinister
> issue, removing more cells than intended.
>
> I've not yet written a test to confirm this.
>
> There was conversation around the implementation of nonces discussing
> options for removing the coupling of TS to clock-on-the-wall time. Sergey
> describes the current situation quite eloquently: "server-generated TS
> provide illusion of consistency guarantees which is not there by any
> means". A fix for this will likely subtly break the semantics of our data
> coordinates, and so should be addressed with 1.0, perhaps along side a
> revamped client-side API.
>
> -n
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.