You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by userx <ga...@gmail.com> on 2020/01/20 13:40:11 UTC
java.lang.NullPointerException while preloadEntry
hi team,
I recently observed an error bringing down one of the ignite server nodes.
The log mentions a NullPointerExceptionException on preload of an entry.
Subsequently 2 other nodes went down. Here is the stack trace of first
failure
2019-12-14 06:47:51,030 ERROR [sys-#61006%ABCD4%] {}
org.apache.ignite.internal.processors.cache.GridCacheIoManager -
Failed processing message
[
senderId=647bc0f2-67d8-469b-9a49-af4231463887,
msg=GridDhtPartitionSupplyMessage
[
rebalanceId=117004,
topVer=AffinityTopologyVersion
[
topVer=67,
minorTopVer=395
],
missed=null,
clean=null,
msgSize=524366,
estimatedKeysCnt=-1,
size=1,
parts=[449],
super=GridCacheGroupIdMessage [grpId=1152451571]
]
]
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander.preloadEntry(GridDhtPartitionDemander.java:893)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander.handleSupplyMessage(GridDhtPartitionDemander.java:772)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.handleSupplyMessage(GridDhtPreloader.java:387)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:419)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:409)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$700(GridCacheIoManager.java:101)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$OrderedMessageListener.onMessage(GridCacheIoManager.java:1613)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:127)
at
org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2768)
at
org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:1529)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4400(GridIoManager.java:127)
at
org.apache.ignite.internal.managers.communication.GridIoManager$10.run(GridIoManager.java:1498)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
1) The question is that in spite of the fixed number of Ignite server nodes,
why is the rebalance triggered ?
2) What is causing the Null pointer ?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: java.lang.NullPointerException while preloadEntry
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I see nothing suspicious in the logs.
I think it's just https://issues.apache.org/jira/browse/IGNITE-10242
Please try working around it or wait for 2.8 to be released. You can also
try builds from 2.8 release branch.
Regards,
--
Ilya Kasnacheev
пт, 31 янв. 2020 г. в 07:53, userx <ga...@gmail.com>:
> Hi Ilya,
>
> Did you get a chance to see the logs ?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: java.lang.NullPointerException while preloadEntry
Posted by userx <ga...@gmail.com>.
Hi Ilya,
Did you get a chance to see the logs ?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: java.lang.NullPointerException while preloadEntry
Posted by userx <ga...@gmail.com>.
Hi Ilya,
Apologies for delay. PFA the logs. Ignite.zip
<http://apache-ignite-users.70518.x6.nabble.com/file/t1165/Ignite.zip>
Let me know if any other information is required.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: java.lang.NullPointerException while preloadEntry
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I don't see why you can't upload large logs on e.g. Google Drive.
I'm not sure about the last 500 lines because the root cause can be
decoupled in time with the manifestation of problem.
Regards,
--
Ilya Kasnacheev
вт, 21 янв. 2020 г. в 14:55, userx <ga...@gmail.com>:
> hi Ilya,
>
> The logs are pretty big. Do you need a specific portion on the log like the
> last 500 lines or so ?
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: java.lang.NullPointerException while preloadEntry
Posted by userx <ga...@gmail.com>.
hi Ilya,
The logs are pretty big. Do you need a specific portion on the log like the
last 500 lines or so ?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: java.lang.NullPointerException while preloadEntry
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Can you provide logs from all server nodes? I think there's something in
them, besides this error.
Regards,
--
Ilya Kasnacheev
пн, 20 янв. 2020 г. в 20:03, userx <ga...@gmail.com>:
> Hi,
>
> Topology version is 67. Ignite version is 2.7.6.
>
> I went through the thread mentioned by you but in our case, we do not have
> clients connecting and disconnecting time and again. If a client is
> connected to the cluster, the topololgy version is increased by 1. The
> client doesn't disconnect ever. There are around 11 ignite servers and 56
> clients.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: java.lang.NullPointerException while preloadEntry
Posted by userx <ga...@gmail.com>.
Hi,
Topology version is 67. Ignite version is 2.7.6.
I went through the thread mentioned by you but in our case, we do not have
clients connecting and disconnecting time and again. If a client is
connected to the cluster, the topololgy version is increased by 1. The
client doesn't disconnect ever. There are around 11 ignite servers and 56
clients.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: java.lang.NullPointerException while preloadEntry
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I can see there is a previous occurrence of this issue:
http://apache-ignite-users.70518.x6.nabble.com/NullPointerException-in-GridDhtPartitionDemander-td19010.html
Can you elaborate what is your topology version and actual Ignite version?
Maybe you can find anything else in server logs?
Regards,
--
Ilya Kasnacheev
пн, 20 янв. 2020 г. в 16:40, userx <ga...@gmail.com>:
> hi team,
>
> I recently observed an error bringing down one of the ignite server nodes.
> The log mentions a NullPointerExceptionException on preload of an entry.
> Subsequently 2 other nodes went down. Here is the stack trace of first
> failure
>
> 2019-12-14 06:47:51,030 ERROR [sys-#61006%ABCD4%] {}
> org.apache.ignite.internal.processors.cache.GridCacheIoManager -
> Failed processing message
> [
> senderId=647bc0f2-67d8-469b-9a49-af4231463887,
> msg=GridDhtPartitionSupplyMessage
> [
> rebalanceId=117004,
> topVer=AffinityTopologyVersion
> [
> topVer=67,
> minorTopVer=395
> ],
> missed=null,
> clean=null,
> msgSize=524366,
> estimatedKeysCnt=-1,
> size=1,
> parts=[449],
> super=GridCacheGroupIdMessage [grpId=1152451571]
> ]
> ]
> java.lang.NullPointerException
> at
>
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander.preloadEntry(GridDhtPartitionDemander.java:893)
> at
>
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander.handleSupplyMessage(GridDhtPartitionDemander.java:772)
> at
>
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.handleSupplyMessage(GridDhtPreloader.java:387)
> at
>
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:419)
> at
>
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:409)
> at
>
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056)
> at
>
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
> at
>
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$700(GridCacheIoManager.java:101)
> at
>
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$OrderedMessageListener.onMessage(GridCacheIoManager.java:1613)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:127)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2768)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:1529)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4400(GridIoManager.java:127)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager$10.run(GridIoManager.java:1498)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
>
> 1) The question is that in spite of the fixed number of Ignite server
> nodes,
> why is the rebalance triggered ?
> 2) What is causing the Null pointer ?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>