You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by Rakesh R <ra...@huawei.com> on 2012/07/05 11:11:21 UTC

Reg System.currentTimeMillis usage in BookKeeper

Hi,



In our test env, we have got the following problem. After analyzing found the problem occured due system time changes. Actually in our cluster we are running a periodic syncup scripts just to unify the SystemTime in all the machines. This is causing the problem and resulting ArrayIndexOutOfBoundException.



I agree it is not good to change the system time on a running process, still I'm thinking we can use System.nanoTime(), this will be giving the JVM startup time and will helpful to withstand this kind of minor accidents(as this came in the stat calculation logic).



Exception in thread "BookieJournal-3181" java.lang.ArrayIndexOutOfBoundsException: -423
at org.apache.bookkeeper.proto.BKStats$OpStats.updateLatency(BKStats.java:126)
at org.apache.bookkeeper.proto.BookieServer.writeComplete(BookieServer.java:655)
at org.apache.bookkeeper.bookie.Journal.run(Journal.java:507)

           // compute the latency
            if (0 == rc) {
                // for add operations, we compute latency in writeComplete callbacks.
                long elapsedTime = System.currentTimeMillis() - startTime;
                bkStats.getOpStats(BKStats.STATS_ADD).updateLatency(elapsedTime);
            }



Thanks & Regards,

Rakesh

RE: Reg System.currentTimeMillis usage in BookKeeper

Posted by Mridul Muralidharan <mr...@yahoo-inc.com>.
There is no guarantee that it needs to be monotonic - particularly in MP systems - both numa and smp systems : unless you tie each process to a particular core via affinity.
nanotime() attempts to do a bestcase effort based on ticks (usually), while currentTimeInMillis is based on clock.
Both can drift in unpredictable ways (former due to core/proc changes, latter due to ntp updates usually, etc).


Relying on monotonicity might leads to unpredictable errors :-)


Regards,
Mridul


> -----Original Message-----
> From: Ivan Kelly [mailto:ivank@apache.org]
> Sent: Thursday, July 05, 2012 8:15 PM
> To: bookkeeper-dev@zookeeper.apache.org
> Subject: Re: Reg System.currentTimeMillis usage in BookKeeper
> 
> > I agree it is not good to change the system time on a running
> process,
> > still I'm thinking we can use System.nanoTime(), this will be giving
> > the JVM startup time and will helpful to withstand this kind of minor
> > accidents(as this came in the stat calculation logic).
> 
> Good idea. Could you open a JIRA for this.
> 
> System.nanoTime() should be monotonic on most systems, whereas
> currentTimeMillis() is not guarenteed to be. Apparently,
> System.nanoTime() can be non monotonic if the OS doesn't support it,
> but in that case it falls back to currentTimeMillis anyhow.
> 
> Performance wise, they seem very similar on linux. currentTimeMillis is
> a little quicker, but it's the difference between .66 microseconds and
> .7 microseconds. Not worth losing sleep over.
> 
> 100m invokations of nanoTime : 71085070000ns 71085ms 100m invokations
> of currentTimeMillis : 66250899000ns 66250ms
> 
> There's an interesting bug report that codes through the monotonicness
> issue.
> 
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6458294
> 
> 
> -Ivan

RE: Reg System.currentTimeMillis usage in BookKeeper

Posted by Uma Maheswara Rao G <ma...@huawei.com>.
Yep, Agreed. Better to go with System.nanoTime(). Atleast we will avoid the problems in the machines where it supports monotonicness.

Regards,
Uma

________________________________________
From: Ivan Kelly [ivank@apache.org]
Sent: Thursday, July 05, 2012 8:14 PM
To: bookkeeper-dev@zookeeper.apache.org
Subject: Re: Reg System.currentTimeMillis usage in BookKeeper

> I agree it is not good to change the system time on a running
> process, still I'm thinking we can use System.nanoTime(), this will
> be giving the JVM startup time and will helpful to withstand this
> kind of minor accidents(as this came in the stat calculation
> logic).

Good idea. Could you open a JIRA for this.

System.nanoTime() should be monotonic on most systems, whereas
currentTimeMillis() is not guarenteed to be. Apparently,
System.nanoTime() can be non monotonic if the OS doesn't support it,
but in that case it falls back to currentTimeMillis anyhow.

Performance wise, they seem very similar on linux. currentTimeMillis
is a little quicker, but it's the difference between .66 microseconds
and .7 microseconds. Not worth losing sleep over.

100m invokations of nanoTime : 71085070000ns 71085ms
100m invokations of currentTimeMillis : 66250899000ns 66250ms

There's an interesting bug report that codes through the monotonicness
issue.

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6458294


-Ivan

Re: Reg System.currentTimeMillis usage in BookKeeper

Posted by Ivan Kelly <iv...@apache.org>.
> I agree it is not good to change the system time on a running
> process, still I'm thinking we can use System.nanoTime(), this will
> be giving the JVM startup time and will helpful to withstand this
> kind of minor accidents(as this came in the stat calculation
> logic). 

Good idea. Could you open a JIRA for this.

System.nanoTime() should be monotonic on most systems, whereas
currentTimeMillis() is not guarenteed to be. Apparently,
System.nanoTime() can be non monotonic if the OS doesn't support it,
but in that case it falls back to currentTimeMillis anyhow.

Performance wise, they seem very similar on linux. currentTimeMillis
is a little quicker, but it's the difference between .66 microseconds
and .7 microseconds. Not worth losing sleep over.

100m invokations of nanoTime : 71085070000ns 71085ms
100m invokations of currentTimeMillis : 66250899000ns 66250ms

There's an interesting bug report that codes through the monotonicness
issue.

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6458294


-Ivan