You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2019/11/15 10:24:57 UTC
[GitHub] [pulsar] ArtemShaban opened a new issue #5673: pulsar-broker] fix
ConcurrentModificationException while load-report
serialization/updateLocalBrokerData
ArtemShaban opened a new issue #5673: pulsar-broker] fix ConcurrentModificationException while load-report serialization/updateLocalBrokerData
URL: https://github.com/apache/pulsar/issues/5673
**Issue desctiption:**
Pulsar version = 2.4.1
Sometimes I see 2 ConcurrentModificationException in broker logs.
```
06:24:51.990 [pulsar-load-manager-4-1] WARN org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Error when attempting to update local broker data: {}
java.util.ConcurrentModificationException: null
#011at java.util.HashMap$HashIterator.nextNode(HashMap.java:1445) ~[?:1.8.0_231]
#011at java.util.HashMap$EntryIterator.next(HashMap.java:1479) ~[?:1.8.0_231]
#011at java.util.HashMap$EntryIterator.next(HashMap.java:1477) ~[?:1.8.0_231]
#011at org.apache.pulsar.policies.data.loadbalancer.LocalBrokerData.updateBundleData(LocalBrokerData.java:175) ~[org.apache.pulsar-pulsar-common-2.4.1.jar:2.4.1]
#011at org.apache.pulsar.policies.data.loadbalancer.LocalBrokerData.update(LocalBrokerData.java:121) ~[org.apache.pulsar-pulsar-common-2.4.1.jar:2.4.1]
#011at org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl.updateLocalBrokerData(ModularLoadManagerImpl.java:847) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1]
#011at org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl.writeBrokerDataOnZooKeeper(ModularLoadManagerImpl.java:860) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1]
#011at org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerWrapper.writeLoadReportOnZookeeper(ModularLoadManagerWrapper.java:120) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1]
#011at org.apache.pulsar.broker.loadbalance.LoadReportUpdaterTask.run(LoadReportUpdaterTask.java:41) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1]
#011at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_231]
#011at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_231]
#011at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_231]
#011at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_231]
#011at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_231]
#011at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_231]
#011at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
#011at java.lang.Thread.run(Thread.java:748) [?:1.8.0_231]
```
```
06:24:51.990 [pulsar-load-manager-4-1] WARN org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Error writing broker data on ZooKeeper: {}
com.fasterxml.jackson.databind.JsonMappingException: (was java.util.ConcurrentModificationException) (through reference chain: org.apache.pulsar.policies.data.loadbalancer.LocalBrokerData["lastStats"])
#011at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:394) ~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:353) ~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:316) ~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:727) ~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at com.fasterxml.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3906) ~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsBytes(ObjectMapper.java:3244) ~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at org.apache.pulsar.policies.data.loadbalancer.JSONWritable.getJsonBytes(JSONWritable.java:39) ~[org.apache.pulsar-pulsar-common-2.4.1.jar:2.4.1]
#011at org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl.writeBrokerDataOnZooKeeper(ModularLoadManagerImpl.java:863) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1]
#011at org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerWrapper.writeLoadReportOnZookeeper(ModularLoadManagerWrapper.java:120) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1]
#011at org.apache.pulsar.broker.loadbalance.LoadReportUpdaterTask.run(LoadReportUpdaterTask.java:41) ~[org.apache.pulsar-pulsar-broker-2.4.1.jar:2.4.1]
#011at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_231]
#011at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_231]
#011at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_231]
#011at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_231]
#011at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_231]
#011at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_231]
#011at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
#011at java.lang.Thread.run(Thread.java:748) [?:1.8.0_231]
Caused by: java.util.ConcurrentModificationException
#011at java.util.HashMap$HashIterator.nextNode(HashMap.java:1445) ~[?:1.8.0_231]
#011at java.util.HashMap$EntryIterator.next(HashMap.java:1479) ~[?:1.8.0_231]
#011at java.util.HashMap$EntryIterator.next(HashMap.java:1477) ~[?:1.8.0_231]
#011at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeOptionalFields(MapSerializer.java:744) ~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639) ~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33) ~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[com.fasterxml.jackson.core-jackson-databind-2.9.9.3.jar:2.9.9.3]
#011... 17 more
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services