You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@heron.apache.org by GitBox <gi...@apache.org> on 2018/06/28 09:59:36 UTC
[GitHub] huijunw commented on issue #2927: [dhalion] null exception for
previousCheckpoint
huijunw commented on issue #2927: [dhalion] null exception for previousCheckpoint
URL: https://github.com/apache/incubator-heron/issues/2927#issuecomment-400982016
Hi @avflor
The healthmgr failed and quited in the first round policy execution, then restarted. There were no last 5 policy execution.
The issue is `previousCheckpoint ` is not initialized (null) in the first round policy execution. https://github.com/Microsoft/Dhalion/blob/0.2.1/src/main/java/com/microsoft/dhalion/policy/PoliciesExecutor.java#L137
```
[2018-06-27 23:15:11 +0000] [INFO] org.apache.heron.healthmgr.HealthManager: Logging setup done.
[2018-06-27 23:15:11 +0000] [INFO] org.apache.heron.healthmgr.HealthManager: Static Heron config loaded successfully
[2018-06-27 23:15:11 +0000] [INFO] org.apache.heron.healthmgr.HealthManager: Initializing health manager
[2018-06-27 23:15:13 +0000] [INFO] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Starting Curator client connecting to: xxxxxxx_zk
[2018-06-27 23:15:13 +0000] [INFO] org.apache.curator.framework.imps.CuratorFrameworkImpl: Starting
[2018-06-27 23:15:13 +0000] [INFO] org.apache.curator.framework.state.ConnectionStateManager: State change: CONNECTED
[2018-06-27 23:15:13 +0000] [INFO] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Directory tree initialized.
[2018-06-27 23:15:14 +0000] [INFO] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Starting Curator client connecting to: xxxxxxx_zk
[2018-06-27 23:15:14 +0000] [INFO] org.apache.curator.framework.imps.CuratorFrameworkImpl: Starting
[2018-06-27 23:15:14 +0000] [INFO] org.apache.curator.framework.state.ConnectionStateManager: State change: CONNECTED
[2018-06-27 23:15:14 +0000] [INFO] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Directory tree initialized.
[2018-06-27 23:15:14 +0000] [INFO] org.apache.heron.healthmgr.HealthPolicyConfig: Health Policy Configuration:{health.policy.class=org.apache.heron.healthmgr.policy.AutoRestartBackpressureContainerPolicy, health.policy.interval.ms=120000, BackPressureDetector.noiseFilterMillis=20}
[2018-06-27 23:15:14 +0000] [INFO] org.apache.heron.healthmgr.HealthManager: Initializing auto-restart-backpressure-container with class org.apache.heron.healthmgr.policy.AutoRestartBackpressureContainerPolicy
[2018-06-27 23:15:14 +0000] [INFO] org.apache.heron.healthmgr.sensors.MetricsCacheMetricsProvider: Metrics will be provided by MetricsCache at http://yyyyyyyyyy/stats
[2018-06-27 23:15:14 +0000] [INFO] org.apache.heron.healthmgr.HealthManager: Starting Health Manager
[2018-06-27 23:15:14 +0000] [INFO] org.apache.heron.healthmgr.HealthManager: Starting Health Manager metric posting thread
[2018-06-27 23:15:14 +0000] [INFO] org.apache.heron.common.network.HeronClient: Connecting to endpoint: /127.0.0.1:31671
[2018-06-27 23:15:14 +0000] [INFO] org.apache.heron.healthmgr.HealthManagerMetrics: Connected to Metrics Manager. Ready to send register request
[2018-06-27 23:15:14 +0000] [INFO] com.microsoft.dhalion.policy.PoliciesExecutor: Executing Policy: AutoRestartBackpressureContainerPolicy, checkpoint: 2018-06-27T23:15:14.462Z
[2018-06-27 23:15:14 +0000] [INFO] org.apache.heron.healthmgr.common.TopologyProvider: Fetching topology from state manager: zzzzzzzzzz
[2018-06-27 23:15:14 +0000] [INFO] org.apache.heron.healthmgr.HealthManagerMetrics: We registered ourselves to the Metrics Manager
[2018-06-27 23:15:15 +0000] [INFO] org.apache.heron.common.network.SocketChannelHelper: Forcing to flush data to socket with best effort.
[2018-06-27 23:15:15 +0000] [INFO] org.apache.heron.common.network.HeronClient: To stop the HeronClient.
[2018-06-27 23:15:15 +0000] [INFO] org.apache.heron.healthmgr.HealthManagerMetrics: SimpleMetricsManagerClient exits
[2018-06-27 23:15:15 +0000] [STDERR] stderr: Exception in thread "main"
[2018-06-27 23:15:15 +0000] [STDERR] stderr: java.util.concurrent.ExecutionException: java.lang.NullPointerException
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.concurrent.FutureTask.report(FutureTask.java:122)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.concurrent.FutureTask.get(FutureTask.java:192)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at org.apache.heron.healthmgr.HealthManager.main(HealthManager.java:241)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: Caused by: java.lang.NullPointerException
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.time.Instant.compareTo(Instant.java:1255)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.time.Instant.isBefore(Instant.java:1285)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at com.microsoft.dhalion.policy.PoliciesExecutor.lambda$null$0(PoliciesExecutor.java:83)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at com.microsoft.dhalion.policy.PoliciesExecutor.lambda$start$2(PoliciesExecutor.java:84)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[2018-06-27 23:15:15 +0000] [STDERR] stderr: at java.lang.Thread.run(Thread.java:748)
[2018-06-27 23:15:26 +0000] [INFO] org.apache.heron.healthmgr.HealthManager: Logging setup done.
[2018-06-27 23:15:26 +0000] [INFO] org.apache.heron.healthmgr.HealthManager: Static Heron config loaded successfully
[2018-06-27 23:15:26 +0000] [INFO] org.apache.heron.healthmgr.HealthManager: Initializing health manager
[2018-06-27 23:15:27 +0000] [INFO] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Starting Curator client connecting to: xxxxxxx_zk
[2018-06-27 23:15:27 +0000] [INFO] org.apache.curator.framework.imps.CuratorFrameworkImpl: Starting
[2018-06-27 23:15:27 +0000] [INFO] org.apache.curator.framework.state.ConnectionStateManager: State change: CONNECTED
[2018-06-27 23:15:27 +0000] [INFO] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Directory tree initialized.
[2018-06-27 23:15:28 +0000] [INFO] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Starting Curator client connecting to: xxxxxxx_zk
[2018-06-27 23:15:28 +0000] [INFO] org.apache.curator.framework.imps.CuratorFrameworkImpl: Starting
[2018-06-27 23:15:28 +0000] [INFO] org.apache.curator.framework.state.ConnectionStateManager: State change: CONNECTED
[2018-06-27 23:15:28 +0000] [INFO] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Directory tree initialized.
[2018-06-27 23:15:28 +0000] [INFO] org.apache.heron.healthmgr.HealthPolicyConfig: Health Policy Configuration:{health.policy.class=org.apache.heron.healthmgr.policy.AutoRestartBackpressureContainerPolicy, health.policy.interval.ms=120000, BackPressureDetector.noiseFilterMillis=20}
[2018-06-27 23:15:28 +0000] [INFO] org.apache.heron.healthmgr.HealthManager: Initializing auto-restart-backpressure-container with class org.apache.heron.healthmgr.policy.AutoRestartBackpressureContainerPolicy
[2018-06-27 23:15:28 +0000] [INFO] org.apache.heron.healthmgr.sensors.MetricsCacheMetricsProvider: Metrics will be provided by MetricsCache at http://yyyyyyyyyy/stats
[2018-06-27 23:15:28 +0000] [INFO] org.apache.heron.healthmgr.HealthManager: Starting Health Manager
[2018-06-27 23:15:29 +0000] [INFO] org.apache.heron.healthmgr.HealthManager: Starting Health Manager metric posting thread
[2018-06-27 23:15:29 +0000] [INFO] org.apache.heron.common.network.HeronClient: Connecting to endpoint: /127.0.0.1:31671
[2018-06-27 23:15:29 +0000] [INFO] org.apache.heron.healthmgr.HealthManagerMetrics: Connected to Metrics Manager. Ready to send register request
[2018-06-27 23:15:29 +0000] [INFO] com.microsoft.dhalion.policy.PoliciesExecutor: Executing Policy: AutoRestartBackpressureContainerPolicy, checkpoint: 2018-06-27T23:15:29.036Z
[2018-06-27 23:15:29 +0000] [INFO] org.apache.heron.healthmgr.common.TopologyProvider: Fetching topology from state manager: zzzzzzzzzz
[2018-06-27 23:15:29 +0000] [INFO] org.apache.heron.healthmgr.HealthManagerMetrics: We registered ourselves to the Metrics Manager
[2018-06-27 23:15:30 +0000] [INFO] org.apache.heron.common.network.SocketChannelHelper: Forcing to flush data to socket with best effort.
[2018-06-27 23:15:30 +0000] [INFO] org.apache.heron.common.network.HeronClient: To stop the HeronClient.
[2018-06-27 23:15:30 +0000] [INFO] org.apache.heron.healthmgr.HealthManagerMetrics: SimpleMetricsManagerClient exits
[2018-06-27 23:15:30 +0000] [STDERR] stderr: Exception in thread "main"
[2018-06-27 23:15:30 +0000] [STDERR] stderr: java.util.concurrent.ExecutionException: java.lang.NullPointerException
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.concurrent.FutureTask.report(FutureTask.java:122)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.concurrent.FutureTask.get(FutureTask.java:192)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at org.apache.heron.healthmgr.HealthManager.main(HealthManager.java:241)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: Caused by: java.lang.NullPointerException
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.time.Instant.compareTo(Instant.java:1255)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.time.Instant.isBefore(Instant.java:1285)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at com.microsoft.dhalion.policy.PoliciesExecutor.lambda$null$0(PoliciesExecutor.java:83)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at com.microsoft.dhalion.policy.PoliciesExecutor.lambda$start$2(PoliciesExecutor.java:84)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[2018-06-27 23:15:30 +0000] [STDERR] stderr: at java.lang.Thread.run(Thread.java:748)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services