You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Maxim Gekk (JIRA)" <ji...@apache.org> on 2019/02/02 14:31:00 UTC
[jira] [Updated] (SPARK-26817) Use System.nanoTime to measure time
intervals
[ https://issues.apache.org/jira/browse/SPARK-26817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maxim Gekk updated SPARK-26817:
-------------------------------
Description:
Replace System.currentTimeMillis() by System.nanoTime in time intervals measurements. System.currentTimeMillis() returns current wallclock time and will follow changes to the system clock. Thus, negative wallclock adjustments can cause timeouts to "hang" for a long time (until wallclock time has caught up to its previous value again). This can happen when ntpd does a "step" after the network has been disconnected for some time. The most canonical example is during system bootup when DHCP takes longer than usual. This can lead to failures that are really hard to understand/reproduce. System.nanoTime() is guaranteed to be monotonically increasing irrespective of wallclock changes.
https://github.com/databricks/scala-style-guide#prefer-nanotime-over-currenttimemillis
was:Replace System.currentTimeMillis() by System.nanoTime in time intervals measurements. System.currentTimeMillis() returns current wallclock time and will follow changes to the system clock. Thus, negative wallclock adjustments can cause timeouts to "hang" for a long time (until wallclock time has caught up to its previous value again). This can happen when ntpd does a "step" after the network has been disconnected for some time. The most canonical example is during system bootup when DHCP takes longer than usual. This can lead to failures that are really hard to understand/reproduce. System.nanoTime() is guaranteed to be monotonically increasing irrespective of wallclock changes.
> Use System.nanoTime to measure time intervals
> ---------------------------------------------
>
> Key: SPARK-26817
> URL: https://issues.apache.org/jira/browse/SPARK-26817
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.4.0
> Reporter: Maxim Gekk
> Priority: Minor
>
> Replace System.currentTimeMillis() by System.nanoTime in time intervals measurements. System.currentTimeMillis() returns current wallclock time and will follow changes to the system clock. Thus, negative wallclock adjustments can cause timeouts to "hang" for a long time (until wallclock time has caught up to its previous value again). This can happen when ntpd does a "step" after the network has been disconnected for some time. The most canonical example is during system bootup when DHCP takes longer than usual. This can lead to failures that are really hard to understand/reproduce. System.nanoTime() is guaranteed to be monotonically increasing irrespective of wallclock changes.
>
> https://github.com/databricks/scala-style-guide#prefer-nanotime-over-currenttimemillis
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org