You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@geode.apache.org by Mark Secrist <ms...@pivotal.io> on 2018/02/19 14:59:28 UTC
Re: IOExcpetion with Part length () and number of parts ()
inconsistent during registerInterest
I'm not sure exactly what your registerInterest call looks like, but you
may want to use the method that takes an additional argument of enum type
InterestResultPolicy as that specifies the behavior when making the call.
The default is to immediately return all keys and values to the caller. You
may consider instead an enum value of NONE or KEYS.
On Feb 19, 2018 7:48 AM, "Vahram Aharonyan" <va...@vmware.com> wrote:
> Hi All,
>
>
>
> We are getting following exception while trying to register interest on
> some Region on client boot.
>
>
>
>
>
> 2018-02-12 19:12:22,975 INFO [Heartbeat] com.integrien.alive.collector.HeartbeatThread.doHeartBeat
> - Heartbeat failed: Pool unexpected IOException connection=
> SubscriptionConnectionImpl[10.194.13.202:10000:closed]). Server
> unreachable: could not connect after 1 attempts
> 2018-02-12 19:12:22,975 DEBUG [Heartbeat] com.integrien.alive.collector.HeartbeatThread.doHeartBeat
> - The reason was:
> org.apache.geode.cache.client.ServerConnectivityException: Pool
> unexpected IOException connection=SubscriptionConnectionImpl[10.194.13.202:10000:closed]).
> Server unreachable: could not connect after 1 attempts
> at org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(
> OpExecutorImpl.java:798)
> at org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(
> OpExecutorImpl.java:623)
> at org.apache.geode.cache.client.internal.OpExecutorImpl.
> executeOnQueuesAndReturnPrimaryResult(OpExecutorImpl.java:569)
> at org.apache.geode.cache.client.internal.PoolImpl.
> executeOnQueuesAndReturnPrimaryResult(PoolImpl.java:805)
> at org.apache.geode.cache.client.internal.RegisterInterestOp.
> execute(RegisterInterestOp.java:58)
> at org.apache.geode.cache.client.internal.ServerRegionProxy.
> registerInterest(ServerRegionProxy.java:362)
> at org.apache.geode.internal.cache.LocalRegion.processSingleInterest(
> LocalRegion.java:3917)
> at org.apache.geode.internal.cache.LocalRegion.registerInterestRegex(
> LocalRegion.java:3999)
> at org.apache.geode.internal.cache.LocalRegion.registerInterestRegex(
> LocalRegion.java:3982)
> at org.apache.geode.internal.cache.LocalRegion.registerInterestRegex(
> LocalRegion.java:3978)
> at com.vmware.vcops.platform.common.collector.GemfireCommunicator.
> registerInterest(GemfireCommunicator.java:336)
> at com.vmware.vcops.platform.common.collector.
> GemfireCommunicator.heartbeat(GemfireCommunicator.java:100)
> at com.integrien.alive.collector.HeartbeatThread.doHeartBeat(
> HeartbeatThread.java:77)
> at com.integrien.alive.collector.HeartbeatThread.run(
> HeartbeatThread.java:113)
> at com.integrien.alive.common.util.BaseThread$BaseThreadRunnable.run(
> BaseThread.java:176)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Part length ( -1,791,928,534 ) and number
> of parts ( 1 ) inconsistent
> at org.apache.geode.internal.cache.tier.sockets.Message.
> readPayloadFields(Message.java:826)
> at org.apache.geode.internal.cache.tier.sockets.ChunkedMessage.readChunk(
> ChunkedMessage.java:275)
> at org.apache.geode.internal.cache.tier.sockets.
> ChunkedMessage.receiveChunk(ChunkedMessage.java:227)
> at org.apache.geode.cache.client.internal.RegisterInterestOp$
> RegisterInterestOpImpl.processResponse(RegisterInterestOp.java:184)
> at org.apache.geode.cache.client.internal.AbstractOp.attemptReadResponse(
> AbstractOp.java:159)
> at org.apache.geode.cache.client.internal.AbstractOp.attempt(
> AbstractOp.java:382)
> at org.apache.geode.cache.client.internal.ConnectionImpl.
> execute(ConnectionImpl.java:266)
> at org.apache.geode.cache.client.internal.QueueConnectionImpl.
> execute(QueueConnectionImpl.java:165)
> at org.apache.geode.cache.client.internal.OpExecutorImpl.
> executeWithPossibleReAuthentication(OpExecutorImpl.java:900)
> at org.apache.geode.cache.client.internal.OpExecutorImpl.
> executeOnQueuesAndReturnPrimaryResult(OpExecutorImpl.java:562)
> ... 13 more
>
>
>
>
>
> 1. I’ve seen couple of tickets on Geode with similar stacktraces:
> GEODE-2517 <https://issues.apache.org/jira/browse/GEODE-2517>, GEODE-478
> <https://issues.apache.org/jira/browse/GEODE-478> and they all refer to
> the fact that huge amount of data is being transferred between client and
> server. But for me it is strange what can cause large data transfer during
> generic registerInterest call from client side. Could someone have info
> what kind of response client is receiving from Server during
> RegisterInterest that is so huge?
>
>
>
> And is there any workaround (parameter value tune) we can try to get out
> of this situation?
>
>
>
> Thanks,
>
> Vahram.
>
>
>
RE: IOExcpetion with Part length () and number of parts ()
inconsistent during registerInterest
Posted by Mark Secrist <ms...@pivotal.io>.
Yes, the default is to send keys and values.
On Feb 20, 2018 6:15 AM, "Vahram Aharonyan" <va...@vmware.com> wrote:
> Hi Mark,
>
>
>
> We use org.apache.geode.cache.Region#registerInterestRegex(java.lang.String)
> that is without specifying return policy.
>
> Does it mean that all keys and values matching this regexp will be
> returned to caller immediately during registerInterest() call even
> irrespective to the Region type that is set to PROXY in our case?
>
>
>
> Thanks,
>
> Vahram.
>
>
>
> *From:* Mark Secrist [mailto:msecrist@pivotal.io]
> *Sent:* Monday, February 19, 2018 6:59 PM
> *To:* user@geode.apache.org
> *Subject:* Re: IOExcpetion with Part length () and number of parts ()
> inconsistent during registerInterest
>
>
>
> I'm not sure exactly what your registerInterest call looks like, but you
> may want to use the method that takes an additional argument of enum type
> InterestResultPolicy as that specifies the behavior when making the call.
> The default is to immediately return all keys and values to the caller. You
> may consider instead an enum value of NONE or KEYS.
>
>
>
>
>
>
>
> On Feb 19, 2018 7:48 AM, "Vahram Aharonyan" <va...@vmware.com> wrote:
>
> Hi All,
>
>
>
> We are getting following exception while trying to register interest on
> some Region on client boot.
>
>
>
>
>
> 2018-02-12 19:12:22,975 INFO [Heartbeat] com.integrien.alive.collector.HeartbeatThread.doHeartBeat
> - Heartbeat failed: Pool unexpected IOException connection=
> SubscriptionConnectionImpl[10.194.13.202:10000:closed]). Server
> unreachable: could not connect after 1 attempts
> 2018-02-12 19:12:22,975 DEBUG [Heartbeat] com.integrien.alive.collector.HeartbeatThread.doHeartBeat
> - The reason was:
> org.apache.geode.cache.client.ServerConnectivityException: Pool
> unexpected IOException connection=SubscriptionConnectionImpl[10.194.13.202:10000:closed]).
> Server unreachable: could not connect after 1 attempts
> at org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(
> OpExecutorImpl.java:798)
> at org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(
> OpExecutorImpl.java:623)
> at org.apache.geode.cache.client.internal.OpExecutorImpl.
> executeOnQueuesAndReturnPrimaryResult(OpExecutorImpl.java:569)
> at org.apache.geode.cache.client.internal.PoolImpl.
> executeOnQueuesAndReturnPrimaryResult(PoolImpl.java:805)
> at org.apache.geode.cache.client.internal.RegisterInterestOp.
> execute(RegisterInterestOp.java:58)
> at org.apache.geode.cache.client.internal.ServerRegionProxy.
> registerInterest(ServerRegionProxy.java:362)
> at org.apache.geode.internal.cache.LocalRegion.processSingleInterest(
> LocalRegion.java:3917)
> at org.apache.geode.internal.cache.LocalRegion.registerInterestRegex(
> LocalRegion.java:3999)
> at org.apache.geode.internal.cache.LocalRegion.registerInterestRegex(
> LocalRegion.java:3982)
> at org.apache.geode.internal.cache.LocalRegion.registerInterestRegex(
> LocalRegion.java:3978)
> at com.vmware.vcops.platform.common.collector.GemfireCommunicator.
> registerInterest(GemfireCommunicator.java:336)
> at com.vmware.vcops.platform.common.collector.
> GemfireCommunicator.heartbeat(GemfireCommunicator.java:100)
> at com.integrien.alive.collector.HeartbeatThread.doHeartBeat(
> HeartbeatThread.java:77)
> at com.integrien.alive.collector.HeartbeatThread.run(
> HeartbeatThread.java:113)
> at com.integrien.alive.common.util.BaseThread$BaseThreadRunnable.run(
> BaseThread.java:176)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Part length ( -1,791,928,534 ) and number
> of parts ( 1 ) inconsistent
> at org.apache.geode.internal.cache.tier.sockets.Message.
> readPayloadFields(Message.java:826)
> at org.apache.geode.internal.cache.tier.sockets.ChunkedMessage.readChunk(
> ChunkedMessage.java:275)
> at org.apache.geode.internal.cache.tier.sockets.
> ChunkedMessage.receiveChunk(ChunkedMessage.java:227)
> at org.apache.geode.cache.client.internal.RegisterInterestOp$
> RegisterInterestOpImpl.processResponse(RegisterInterestOp.java:184)
> at org.apache.geode.cache.client.internal.AbstractOp.attemptReadResponse(
> AbstractOp.java:159)
> at org.apache.geode.cache.client.internal.AbstractOp.attempt(
> AbstractOp.java:382)
> at org.apache.geode.cache.client.internal.ConnectionImpl.
> execute(ConnectionImpl.java:266)
> at org.apache.geode.cache.client.internal.QueueConnectionImpl.
> execute(QueueConnectionImpl.java:165)
> at org.apache.geode.cache.client.internal.OpExecutorImpl.
> executeWithPossibleReAuthentication(OpExecutorImpl.java:900)
> at org.apache.geode.cache.client.internal.OpExecutorImpl.
> executeOnQueuesAndReturnPrimaryResult(OpExecutorImpl.java:562)
> ... 13 more
>
>
>
>
>
> 1. I’ve seen couple of tickets on Geode with similar stacktraces:
> GEODE-2517
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_GEODE-2D2517&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpTWSXVvcGFCkFEMePbOecdHHTbyiIj9aWq7oqKb0J8&m=MuQ2hlU1zYFVwrnTUeHb1gchZX6fgQMMkBVUP-5MtJg&s=NpR9Caf3G7MXsQxBAAxec-XrUlgT4Oog-1vnnFPTFIk&e=>,
> GEODE-478
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_GEODE-2D478&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpTWSXVvcGFCkFEMePbOecdHHTbyiIj9aWq7oqKb0J8&m=MuQ2hlU1zYFVwrnTUeHb1gchZX6fgQMMkBVUP-5MtJg&s=Zd1jc0HIe19Na0SyZFmJwT5rMKg8CHv1NfwFZjm_z2E&e=>
> and they all refer to the fact that huge amount of data is being
> transferred between client and server. But for me it is strange what can
> cause large data transfer during generic registerInterest call from client
> side. Could someone have info what kind of response client is receiving
> from Server during RegisterInterest that is so huge?
>
>
>
> And is there any workaround (parameter value tune) we can try to get out
> of this situation?
>
>
>
> Thanks,
>
> Vahram.
>
>
>
>
RE: IOExcpetion with Part length () and number of parts ()
inconsistent during registerInterest
Posted by Vahram Aharonyan <va...@vmware.com>.
Hi Mark,
We use org.apache.geode.cache.Region#registerInterestRegex(java.lang.String) that is without specifying return policy.
Does it mean that all keys and values matching this regexp will be returned to caller immediately during registerInterest() call even irrespective to the Region type that is set to PROXY in our case?
Thanks,
Vahram.
From: Mark Secrist [mailto:msecrist@pivotal.io]
Sent: Monday, February 19, 2018 6:59 PM
To: user@geode.apache.org
Subject: Re: IOExcpetion with Part length () and number of parts () inconsistent during registerInterest
I'm not sure exactly what your registerInterest call looks like, but you may want to use the method that takes an additional argument of enum type InterestResultPolicy as that specifies the behavior when making the call. The default is to immediately return all keys and values to the caller. You may consider instead an enum value of NONE or KEYS.
On Feb 19, 2018 7:48 AM, "Vahram Aharonyan" <va...@vmware.com>> wrote:
Hi All,
We are getting following exception while trying to register interest on some Region on client boot.
2018-02-12 19:12:22,975 INFO [Heartbeat] com.integrien.alive.collector.HeartbeatThread.doHeartBeat - Heartbeat failed: Pool unexpected IOException connection=SubscriptionConnectionImpl[10.194.13.202:10000:closed]). Server unreachable: could not connect after 1 attempts
2018-02-12 19:12:22,975 DEBUG [Heartbeat] com.integrien.alive.collector.HeartbeatThread.doHeartBeat - The reason was:
org.apache.geode.cache.client.ServerConnectivityException: Pool unexpected IOException connection=SubscriptionConnectionImpl[10.194.13.202:10000:closed]). Server unreachable: could not connect after 1 attempts
at org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(OpExecutorImpl.java:798)
at org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(OpExecutorImpl.java:623)
at org.apache.geode.cache.client.internal.OpExecutorImpl.executeOnQueuesAndReturnPrimaryResult(OpExecutorImpl.java:569)
at org.apache.geode.cache.client.internal.PoolImpl.executeOnQueuesAndReturnPrimaryResult(PoolImpl.java:805)
at org.apache.geode.cache.client.internal.RegisterInterestOp.execute(RegisterInterestOp.java:58)
at org.apache.geode.cache.client.internal.ServerRegionProxy.registerInterest(ServerRegionProxy.java:362)
at org.apache.geode.internal.cache.LocalRegion.processSingleInterest(LocalRegion.java:3917)
at org.apache.geode.internal.cache.LocalRegion.registerInterestRegex(LocalRegion.java:3999)
at org.apache.geode.internal.cache.LocalRegion.registerInterestRegex(LocalRegion.java:3982)
at org.apache.geode.internal.cache.LocalRegion.registerInterestRegex(LocalRegion.java:3978)
at com.vmware.vcops.platform.common.collector.GemfireCommunicator.registerInterest(GemfireCommunicator.java:336)
at com.vmware.vcops.platform.common.collector.GemfireCommunicator.heartbeat(GemfireCommunicator.java:100)
at com.integrien.alive.collector.HeartbeatThread.doHeartBeat(HeartbeatThread.java:77)
at com.integrien.alive.collector.HeartbeatThread.run(HeartbeatThread.java:113)
at com.integrien.alive.common.util.BaseThread$BaseThreadRunnable.run(BaseThread.java:176)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Part length ( -1,791,928,534 ) and number of parts ( 1 ) inconsistent
at org.apache.geode.internal.cache.tier.sockets.Message.readPayloadFields(Message.java:826)
at org.apache.geode.internal.cache.tier.sockets.ChunkedMessage.readChunk(ChunkedMessage.java:275)
at org.apache.geode.internal.cache.tier.sockets.ChunkedMessage.receiveChunk(ChunkedMessage.java:227)
at org.apache.geode.cache.client.internal.RegisterInterestOp$RegisterInterestOpImpl.processResponse(RegisterInterestOp.java:184)
at org.apache.geode.cache.client.internal.AbstractOp.attemptReadResponse(AbstractOp.java:159)
at org.apache.geode.cache.client.internal.AbstractOp.attempt(AbstractOp.java:382)
at org.apache.geode.cache.client.internal.ConnectionImpl.execute(ConnectionImpl.java:266)
at org.apache.geode.cache.client.internal.QueueConnectionImpl.execute(QueueConnectionImpl.java:165)
at org.apache.geode.cache.client.internal.OpExecutorImpl.executeWithPossibleReAuthentication(OpExecutorImpl.java:900)
at org.apache.geode.cache.client.internal.OpExecutorImpl.executeOnQueuesAndReturnPrimaryResult(OpExecutorImpl.java:562)
... 13 more
1. I’ve seen couple of tickets on Geode with similar stacktraces: GEODE-2517<https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_GEODE-2D2517&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpTWSXVvcGFCkFEMePbOecdHHTbyiIj9aWq7oqKb0J8&m=MuQ2hlU1zYFVwrnTUeHb1gchZX6fgQMMkBVUP-5MtJg&s=NpR9Caf3G7MXsQxBAAxec-XrUlgT4Oog-1vnnFPTFIk&e=>, GEODE-478<https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_GEODE-2D478&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpTWSXVvcGFCkFEMePbOecdHHTbyiIj9aWq7oqKb0J8&m=MuQ2hlU1zYFVwrnTUeHb1gchZX6fgQMMkBVUP-5MtJg&s=Zd1jc0HIe19Na0SyZFmJwT5rMKg8CHv1NfwFZjm_z2E&e=> and they all refer to the fact that huge amount of data is being transferred between client and server. But for me it is strange what can cause large data transfer during generic registerInterest call from client side. Could someone have info what kind of response client is receiving from Server during RegisterInterest that is so huge?
And is there any workaround (parameter value tune) we can try to get out of this situation?
Thanks,
Vahram.