You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by A J <s5...@gmail.com> on 2011/06/28 00:03:49 UTC
Clock skew
During writes, the timestamp field in the column is the system-time of
that node (correct me if that is not the case and the system-time of
the co-ordinator is what gets applied to all the replicas).
During reads, the latest write wins.
What if there is a clock skew ? It could lead to a stale write
over-riding the actual latest write, just because the clock of that
node is ahead of the other node. Right ?
Re: Clock skew
Posted by Eric Evans <ee...@rackspace.com>.
On Tue, 2011-06-28 at 11:54 +1200, aaron morton wrote:
> Without exception the timestamp is set by the client, not the server.
> The one exception to the without exception rule is CounterColumnType
> operations.
And CQL...
--
Eric Evans
eevans@rackspace.com
Re: Clock skew
Posted by Yang <te...@gmail.com>.
oftentimes people use time actually subconsciously to express causal
relations ("before/after"), as long as you have some other means to
establish causal relations, you don't really need to have an exactly clock.
On Mon, Jun 27, 2011 at 4:54 PM, aaron morton <aa...@thelastpickle.com>wrote:
> Without exception the timestamp is set by the client, not the server. The
> one exception to the without exception rule is CounterColumnType operations.
>
> If you are in a situation where you need better timing than you can get
> with ntp you should try to design around it.
>
> Hope that helps.
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 28 Jun 2011, at 10:03, A J wrote:
>
> > During writes, the timestamp field in the column is the system-time of
> > that node (correct me if that is not the case and the system-time of
> > the co-ordinator is what gets applied to all the replicas).
> > During reads, the latest write wins.
> >
> > What if there is a clock skew ? It could lead to a stale write
> > over-riding the actual latest write, just because the clock of that
> > node is ahead of the other node. Right ?
>
>
Re: Clock skew
Posted by aaron morton <aa...@thelastpickle.com>.
Without exception the timestamp is set by the client, not the server. The one exception to the without exception rule is CounterColumnType operations.
If you are in a situation where you need better timing than you can get with ntp you should try to design around it.
Hope that helps.
-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 28 Jun 2011, at 10:03, A J wrote:
> During writes, the timestamp field in the column is the system-time of
> that node (correct me if that is not the case and the system-time of
> the co-ordinator is what gets applied to all the replicas).
> During reads, the latest write wins.
>
> What if there is a clock skew ? It could lead to a stale write
> over-riding the actual latest write, just because the clock of that
> node is ahead of the other node. Right ?
Re: Clock skew
Posted by A J <s5...@gmail.com>.
Thanks.
On Tue, Jun 28, 2011 at 1:31 PM, Dominic Williams
<dw...@system7.co.uk> wrote:
> Hi, yes you are correct, and this is a potential problem.
> IMPORTANT: If you need to serialize writes from your application servers,
> for example using distributed locking, then before releasing locks you must
> sleep for a period equal to the maximum variance between the clocks on your
> application server nodes.
> I had a problem with the clocks on my nodes which led to all kinds of
> problems. There is a slightly out of date post, which may not mentioned the
> above point, on my experiences
> here http://ria101.wordpress.com/2011/02/08/cassandra-the-importance-of-system-clocks-avoiding-oom-and-how-to-escape-oom-meltdown/
> Hope this helps
> Dominic
> On 27 June 2011 23:03, A J <s5...@gmail.com> wrote:
>>
>> During writes, the timestamp field in the column is the system-time of
>> that node (correct me if that is not the case and the system-time of
>> the co-ordinator is what gets applied to all the replicas).
>> During reads, the latest write wins.
>>
>> What if there is a clock skew ? It could lead to a stale write
>> over-riding the actual latest write, just because the clock of that
>> node is ahead of the other node. Right ?
>
>
Re: Clock skew
Posted by AJ <aj...@dude.podzone.net>.
Yikes! I just read your blog Dominic. Now I'm worried since my app was
going to be mostly cloud-based. But, you didn't mention anything about
sleeping for 'max clock variance' after making the ntp-related config
changes (maybe you haven't had the time to blog it).
I'm curious, do you think the sleep is required even in a
non-virtualized environment? Is it only needed when implementing some
kind of lock? Does the type of lock make a difference?
Thanks!
aj (the other one)
On 6/28/2011 11:31 AM, Dominic Williams wrote:
> Hi, yes you are correct, and this is a potential problem.
>
> IMPORTANT: If you need to serialize writes from your application
> servers, for example using distributed locking, then before releasing
> locks you must sleep for a period equal to the maximum variance
> between the clocks on your application server nodes.
>
> I had a problem with the clocks on my nodes which led to all kinds of
> problems. There is a slightly out of date post, which may not
> mentioned the above point, on my experiences here
> http://ria101.wordpress.com/2011/02/08/cassandra-the-importance-of-system-clocks-avoiding-oom-and-how-to-escape-oom-meltdown/
>
> Hope this helps
> Dominic
>
> On 27 June 2011 23:03, A J <s5alye@gmail.com
> <ma...@gmail.com>> wrote:
>
> During writes, the timestamp field in the column is the system-time of
> that node (correct me if that is not the case and the system-time of
> the co-ordinator is what gets applied to all the replicas).
> During reads, the latest write wins.
>
> What if there is a clock skew ? It could lead to a stale write
> over-riding the actual latest write, just because the clock of that
> node is ahead of the other node. Right ?
>
>
Re: Clock skew
Posted by Dominic Williams <dw...@system7.co.uk>.
Hi, yes you are correct, and this is a potential problem.
IMPORTANT: If you need to serialize writes from your application servers,
for example using distributed locking, then before releasing locks you must
sleep for a period equal to the maximum variance between the clocks on your
application server nodes.
I had a problem with the clocks on my nodes which led to all kinds of
problems. There is a slightly out of date post, which may not mentioned the
above point, on my experiences here
http://ria101.wordpress.com/2011/02/08/cassandra-the-importance-of-system-clocks-avoiding-oom-and-how-to-escape-oom-meltdown/
Hope this helps
Dominic
On 27 June 2011 23:03, A J <s5...@gmail.com> wrote:
> During writes, the timestamp field in the column is the system-time of
> that node (correct me if that is not the case and the system-time of
> the co-ordinator is what gets applied to all the replicas).
> During reads, the latest write wins.
>
> What if there is a clock skew ? It could lead to a stale write
> over-riding the actual latest write, just because the clock of that
> node is ahead of the other node. Right ?
>