You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@geode.apache.org by Pieter van Zyl <pi...@lautus.net> on 2018/07/10 11:08:30 UTC
Debug Functions on server via Idea...timeout
Good day.
This might be a weird question and I hope I am asking the right forum.
I am trying to debug a Geode function. I am running the server via Spring
from Idea and have a few breakpoints. The problem is that as soon as I hit
a breakpoint and investigate a few fields the server seems to timeout and I
cannot continue.
See this in the logs:
* Membership received a request to remove 192.xxx(23019)<v3>:1025 from
192.xxxx(GemFireLocator:31342:locator)<ec><v0>:1024 reason=Member isn't
responding to heartbeat requests*
*[fatal 2018/07/10 13:05:24.880 SAST <unicast receiver,pvz-dell-40480>
tid=0x1e] Membership service failure: Member isn't responding to heartbeat
requests*
*org.apache.geode.ForcedDisconnectException: Member isn't responding to
heartbeat requests*
*[info 2018/07/10 13:05:27.987 SAST <DisconnectThread> tid=0x7b] Stopping
membership services*
*[info 2018/07/10 13:05:28.038 SAST <DisconnectThread> tid=0x7b]
GMSHealthMonitor server socket is closed in stopServices().*
*[info 2018/07/10 13:05:28.040 SAST <Geode Failure Detection Server thread
0> tid=0x22] GMSHealthMonitor server thread exiting*
*[info 2018/07/10 13:05:28.041 SAST <DisconnectThread> tid=0x7b]
GMSHealthMonitor serverSocketExecutor is terminated*
I was wondering if there is some setting on the Geode server or client that
I can set that won't timeout so quickly.
I have one locator and my server connects to that locator.
Not sure if anyone has tried to debug functions running on a Geode server
via Intellij Idea.....
Kindly
Pieter
Re: Debug Functions on server via Idea...timeout
Posted by Pieter van Zyl <pi...@lautus.net>.
Thanks guys. Both options are working great :)
On Wed, Jul 11, 2018 at 7:21 AM, Charlie Black <cb...@pivotal.io> wrote:
> I use the technique Jens mentions... pause current thread. Works like
> champ no mater what you are debugging.
>
>
> Charlie
>
>
> On Tue, Jul 10, 2018 at 10:26 AM John Blum <jb...@pivotal.io> wrote:
>
>> Hi Pieter - Yes, set the member-timeout Geode property when debugging,
>> and then set breakpoints in whatever user-defined code you have deployed on
>> the server (e.g. *Functions*, CacheListeners/Loaders/Writers, etc) and
>> you will be all set. For example [1].
>>
>> -j
>>
>> [1] https://github.com/spring-projects/spring-data-geode/
>> blob/issue/DATAGEODE-120/src/test/java/org/springframework/
>> data/gemfire/function/support/SpringDefinedFunctionAwareRegi
>> strarIntegrationTests.java#L257
>>
>>
>> On Tue, Jul 10, 2018 at 5:23 AM, Pieter van Zyl <
>> pieter.van.zyl@lautus.net> wrote:
>>
>>> Hi guys.
>>>
>>> Thanks for the suggestions.
>>>
>>> I am currently trying member-timeout and it seems to be working
>>>
>>> Kindly
>>> Pieter
>>>
>>> On Tue, Jul 10, 2018 at 1:57 PM, Jens Deppe <jd...@pivotal.io> wrote:
>>>
>>>> Hi Pieter,
>>>>
>>>> Make sure that you are suspending only the current thread and not all
>>>> threads in your breakpoints. Otherwise the member will not be able to
>>>> respond to heartbeats and will get kicked out of the cluster pretty quickly.
>>>>
>>>> --Jens
>>>>
>>>> On Tue, Jul 10, 2018 at 4:08 AM Pieter van Zyl <
>>>> pieter.van.zyl@lautus.net> wrote:
>>>>
>>>>> Good day.
>>>>>
>>>>> This might be a weird question and I hope I am asking the right forum.
>>>>>
>>>>> I am trying to debug a Geode function. I am running the server via
>>>>> Spring from Idea and have a few breakpoints. The problem is that as soon as
>>>>> I hit a breakpoint and investigate a few fields the server seems to timeout
>>>>> and I cannot continue.
>>>>>
>>>>> See this in the logs:
>>>>>
>>>>> * Membership received a request to remove 192.xxx(23019)<v3>:1025 from
>>>>> 192.xxxx(GemFireLocator:31342:locator)<ec><v0>:1024 reason=Member isn't
>>>>> responding to heartbeat requests*
>>>>>
>>>>> *[fatal 2018/07/10 13:05:24.880 SAST <unicast receiver,pvz-dell-40480>
>>>>> tid=0x1e] Membership service failure: Member isn't responding to heartbeat
>>>>> requests*
>>>>> *org.apache.geode.ForcedDisconnectException: Member isn't responding
>>>>> to heartbeat requests*
>>>>> *[info 2018/07/10 13:05:27.987 SAST <DisconnectThread> tid=0x7b]
>>>>> Stopping membership services*
>>>>>
>>>>> *[info 2018/07/10 13:05:28.038 SAST <DisconnectThread> tid=0x7b]
>>>>> GMSHealthMonitor server socket is closed in stopServices().*
>>>>>
>>>>> *[info 2018/07/10 13:05:28.040 SAST <Geode Failure Detection Server
>>>>> thread 0> tid=0x22] GMSHealthMonitor server thread exiting*
>>>>>
>>>>> *[info 2018/07/10 13:05:28.041 SAST <DisconnectThread> tid=0x7b]
>>>>> GMSHealthMonitor serverSocketExecutor is terminated*
>>>>>
>>>>> I was wondering if there is some setting on the Geode server or client
>>>>> that I can set that won't timeout so quickly.
>>>>>
>>>>> I have one locator and my server connects to that locator.
>>>>>
>>>>> Not sure if anyone has tried to debug functions running on a Geode
>>>>> server via Intellij Idea.....
>>>>> Kindly
>>>>> Pieter
>>>>>
>>>>
>>>
>>
>>
>> --
>> -John
>> john.blum10101 (skype)
>>
> --
> cblack@pivotal.io | +1.858.480.9722
> Principal Realtime Data Engineer
>
Re: Debug Functions on server via Idea...timeout
Posted by Charlie Black <cb...@pivotal.io>.
I use the technique Jens mentions... pause current thread. Works like champ
no mater what you are debugging.
Charlie
On Tue, Jul 10, 2018 at 10:26 AM John Blum <jb...@pivotal.io> wrote:
> Hi Pieter - Yes, set the member-timeout Geode property when debugging,
> and then set breakpoints in whatever user-defined code you have deployed on
> the server (e.g. *Functions*, CacheListeners/Loaders/Writers, etc) and
> you will be all set. For example [1].
>
> -j
>
> [1]
> https://github.com/spring-projects/spring-data-geode/blob/issue/DATAGEODE-120/src/test/java/org/springframework/data/gemfire/function/support/SpringDefinedFunctionAwareRegistrarIntegrationTests.java#L257
>
>
> On Tue, Jul 10, 2018 at 5:23 AM, Pieter van Zyl <pieter.van.zyl@lautus.net
> > wrote:
>
>> Hi guys.
>>
>> Thanks for the suggestions.
>>
>> I am currently trying member-timeout and it seems to be working
>>
>> Kindly
>> Pieter
>>
>> On Tue, Jul 10, 2018 at 1:57 PM, Jens Deppe <jd...@pivotal.io> wrote:
>>
>>> Hi Pieter,
>>>
>>> Make sure that you are suspending only the current thread and not all
>>> threads in your breakpoints. Otherwise the member will not be able to
>>> respond to heartbeats and will get kicked out of the cluster pretty quickly.
>>>
>>> --Jens
>>>
>>> On Tue, Jul 10, 2018 at 4:08 AM Pieter van Zyl <
>>> pieter.van.zyl@lautus.net> wrote:
>>>
>>>> Good day.
>>>>
>>>> This might be a weird question and I hope I am asking the right forum.
>>>>
>>>> I am trying to debug a Geode function. I am running the server via
>>>> Spring from Idea and have a few breakpoints. The problem is that as soon as
>>>> I hit a breakpoint and investigate a few fields the server seems to timeout
>>>> and I cannot continue.
>>>>
>>>> See this in the logs:
>>>>
>>>> * Membership received a request to remove 192.xxx(23019)<v3>:1025 from
>>>> 192.xxxx(GemFireLocator:31342:locator)<ec><v0>:1024 reason=Member isn't
>>>> responding to heartbeat requests*
>>>>
>>>> *[fatal 2018/07/10 13:05:24.880 SAST <unicast receiver,pvz-dell-40480>
>>>> tid=0x1e] Membership service failure: Member isn't responding to heartbeat
>>>> requests*
>>>> *org.apache.geode.ForcedDisconnectException: Member isn't responding to
>>>> heartbeat requests*
>>>> *[info 2018/07/10 13:05:27.987 SAST <DisconnectThread> tid=0x7b]
>>>> Stopping membership services*
>>>>
>>>> *[info 2018/07/10 13:05:28.038 SAST <DisconnectThread> tid=0x7b]
>>>> GMSHealthMonitor server socket is closed in stopServices().*
>>>>
>>>> *[info 2018/07/10 13:05:28.040 SAST <Geode Failure Detection Server
>>>> thread 0> tid=0x22] GMSHealthMonitor server thread exiting*
>>>>
>>>> *[info 2018/07/10 13:05:28.041 SAST <DisconnectThread> tid=0x7b]
>>>> GMSHealthMonitor serverSocketExecutor is terminated*
>>>>
>>>> I was wondering if there is some setting on the Geode server or client
>>>> that I can set that won't timeout so quickly.
>>>>
>>>> I have one locator and my server connects to that locator.
>>>>
>>>> Not sure if anyone has tried to debug functions running on a Geode
>>>> server via Intellij Idea.....
>>>> Kindly
>>>> Pieter
>>>>
>>>
>>
>
>
> --
> -John
> john.blum10101 (skype)
>
--
cblack@pivotal.io | +1.858.480.9722
Principal Realtime Data Engineer
Re: Debug Functions on server via Idea...timeout
Posted by John Blum <jb...@pivotal.io>.
Hi Pieter - Yes, set the member-timeout Geode property when debugging, and
then set breakpoints in whatever user-defined code you have deployed on the
server (e.g. *Functions*, CacheListeners/Loaders/Writers, etc) and you will
be all set. For example [1].
-j
[1]
https://github.com/spring-projects/spring-data-geode/blob/issue/DATAGEODE-120/src/test/java/org/springframework/data/gemfire/function/support/SpringDefinedFunctionAwareRegistrarIntegrationTests.java#L257
On Tue, Jul 10, 2018 at 5:23 AM, Pieter van Zyl <pi...@lautus.net>
wrote:
> Hi guys.
>
> Thanks for the suggestions.
>
> I am currently trying member-timeout and it seems to be working
>
> Kindly
> Pieter
>
> On Tue, Jul 10, 2018 at 1:57 PM, Jens Deppe <jd...@pivotal.io> wrote:
>
>> Hi Pieter,
>>
>> Make sure that you are suspending only the current thread and not all
>> threads in your breakpoints. Otherwise the member will not be able to
>> respond to heartbeats and will get kicked out of the cluster pretty quickly.
>>
>> --Jens
>>
>> On Tue, Jul 10, 2018 at 4:08 AM Pieter van Zyl <pi...@lautus.net>
>> wrote:
>>
>>> Good day.
>>>
>>> This might be a weird question and I hope I am asking the right forum.
>>>
>>> I am trying to debug a Geode function. I am running the server via
>>> Spring from Idea and have a few breakpoints. The problem is that as soon as
>>> I hit a breakpoint and investigate a few fields the server seems to timeout
>>> and I cannot continue.
>>>
>>> See this in the logs:
>>>
>>> * Membership received a request to remove 192.xxx(23019)<v3>:1025 from
>>> 192.xxxx(GemFireLocator:31342:locator)<ec><v0>:1024 reason=Member isn't
>>> responding to heartbeat requests*
>>>
>>> *[fatal 2018/07/10 13:05:24.880 SAST <unicast receiver,pvz-dell-40480>
>>> tid=0x1e] Membership service failure: Member isn't responding to heartbeat
>>> requests*
>>> *org.apache.geode.ForcedDisconnectException: Member isn't responding to
>>> heartbeat requests*
>>> *[info 2018/07/10 13:05:27.987 SAST <DisconnectThread> tid=0x7b]
>>> Stopping membership services*
>>>
>>> *[info 2018/07/10 13:05:28.038 SAST <DisconnectThread> tid=0x7b]
>>> GMSHealthMonitor server socket is closed in stopServices().*
>>>
>>> *[info 2018/07/10 13:05:28.040 SAST <Geode Failure Detection Server
>>> thread 0> tid=0x22] GMSHealthMonitor server thread exiting*
>>>
>>> *[info 2018/07/10 13:05:28.041 SAST <DisconnectThread> tid=0x7b]
>>> GMSHealthMonitor serverSocketExecutor is terminated*
>>>
>>> I was wondering if there is some setting on the Geode server or client
>>> that I can set that won't timeout so quickly.
>>>
>>> I have one locator and my server connects to that locator.
>>>
>>> Not sure if anyone has tried to debug functions running on a Geode
>>> server via Intellij Idea.....
>>> Kindly
>>> Pieter
>>>
>>
>
--
-John
john.blum10101 (skype)
Re: Debug Functions on server via Idea...timeout
Posted by Pieter van Zyl <pi...@lautus.net>.
Hi guys.
Thanks for the suggestions.
I am currently trying member-timeout and it seems to be working
Kindly
Pieter
On Tue, Jul 10, 2018 at 1:57 PM, Jens Deppe <jd...@pivotal.io> wrote:
> Hi Pieter,
>
> Make sure that you are suspending only the current thread and not all
> threads in your breakpoints. Otherwise the member will not be able to
> respond to heartbeats and will get kicked out of the cluster pretty quickly.
>
> --Jens
>
> On Tue, Jul 10, 2018 at 4:08 AM Pieter van Zyl <pi...@lautus.net>
> wrote:
>
>> Good day.
>>
>> This might be a weird question and I hope I am asking the right forum.
>>
>> I am trying to debug a Geode function. I am running the server via Spring
>> from Idea and have a few breakpoints. The problem is that as soon as I hit
>> a breakpoint and investigate a few fields the server seems to timeout and I
>> cannot continue.
>>
>> See this in the logs:
>>
>> * Membership received a request to remove 192.xxx(23019)<v3>:1025 from
>> 192.xxxx(GemFireLocator:31342:locator)<ec><v0>:1024 reason=Member isn't
>> responding to heartbeat requests*
>>
>> *[fatal 2018/07/10 13:05:24.880 SAST <unicast receiver,pvz-dell-40480>
>> tid=0x1e] Membership service failure: Member isn't responding to heartbeat
>> requests*
>> *org.apache.geode.ForcedDisconnectException: Member isn't responding to
>> heartbeat requests*
>> *[info 2018/07/10 13:05:27.987 SAST <DisconnectThread> tid=0x7b] Stopping
>> membership services*
>>
>> *[info 2018/07/10 13:05:28.038 SAST <DisconnectThread> tid=0x7b]
>> GMSHealthMonitor server socket is closed in stopServices().*
>>
>> *[info 2018/07/10 13:05:28.040 SAST <Geode Failure Detection Server
>> thread 0> tid=0x22] GMSHealthMonitor server thread exiting*
>>
>> *[info 2018/07/10 13:05:28.041 SAST <DisconnectThread> tid=0x7b]
>> GMSHealthMonitor serverSocketExecutor is terminated*
>>
>> I was wondering if there is some setting on the Geode server or client
>> that I can set that won't timeout so quickly.
>>
>> I have one locator and my server connects to that locator.
>>
>> Not sure if anyone has tried to debug functions running on a Geode server
>> via Intellij Idea.....
>> Kindly
>> Pieter
>>
>
Re: Debug Functions on server via Idea...timeout
Posted by Jens Deppe <jd...@pivotal.io>.
Hi Pieter,
Make sure that you are suspending only the current thread and not all
threads in your breakpoints. Otherwise the member will not be able to
respond to heartbeats and will get kicked out of the cluster pretty quickly.
--Jens
On Tue, Jul 10, 2018 at 4:08 AM Pieter van Zyl <pi...@lautus.net>
wrote:
> Good day.
>
> This might be a weird question and I hope I am asking the right forum.
>
> I am trying to debug a Geode function. I am running the server via Spring
> from Idea and have a few breakpoints. The problem is that as soon as I hit
> a breakpoint and investigate a few fields the server seems to timeout and I
> cannot continue.
>
> See this in the logs:
>
> * Membership received a request to remove 192.xxx(23019)<v3>:1025 from
> 192.xxxx(GemFireLocator:31342:locator)<ec><v0>:1024 reason=Member isn't
> responding to heartbeat requests*
>
> *[fatal 2018/07/10 13:05:24.880 SAST <unicast receiver,pvz-dell-40480>
> tid=0x1e] Membership service failure: Member isn't responding to heartbeat
> requests*
> *org.apache.geode.ForcedDisconnectException: Member isn't responding to
> heartbeat requests*
> *[info 2018/07/10 13:05:27.987 SAST <DisconnectThread> tid=0x7b] Stopping
> membership services*
>
> *[info 2018/07/10 13:05:28.038 SAST <DisconnectThread> tid=0x7b]
> GMSHealthMonitor server socket is closed in stopServices().*
>
> *[info 2018/07/10 13:05:28.040 SAST <Geode Failure Detection Server thread
> 0> tid=0x22] GMSHealthMonitor server thread exiting*
>
> *[info 2018/07/10 13:05:28.041 SAST <DisconnectThread> tid=0x7b]
> GMSHealthMonitor serverSocketExecutor is terminated*
>
> I was wondering if there is some setting on the Geode server or client
> that I can set that won't timeout so quickly.
>
> I have one locator and my server connects to that locator.
>
> Not sure if anyone has tried to debug functions running on a Geode server
> via Intellij Idea.....
> Kindly
> Pieter
>
RE: Debug Functions on server via Idea...timeout
Posted by Rupert St John Webster <ru...@impress-solutions.com>.
Hi Pieter,
Looks like it could be “member-timeout”. From https://geode.apache.org/docs/guide/10/reference/topics/gemfire_properties.html
member-timeout
Geode uses the member-timeout server configuration, specified in milliseconds, to detect the abnormal termination of members. The configuration setting is used in two ways: 1) First it is used during the UDP heartbeat detection process. When a member detects that a heartbeat datagram is missing from the member that it is monitoring after the time interval of 2 * the value of member-timeout, the detecting member attempts to form a TCP/IP stream-socket connection with the monitored member as described in the next case. 2) The property is then used again during the TCP/IP stream-socket connection. If the suspected process does not respond to the are you alive datagram within the time period specified in member-timeout, the membership coordinator sends out a new membership view that notes the member’s failure.
Valid values are in the range 1000..600000.
5000
From: Pieter van Zyl <pi...@lautus.net>
Sent: 10 July 2018 12:09
To: user@geode.apache.org
Subject: Debug Functions on server via Idea...timeout
Good day.
This might be a weird question and I hope I am asking the right forum.
I am trying to debug a Geode function. I am running the server via Spring from Idea and have a few breakpoints. The problem is that as soon as I hit a breakpoint and investigate a few fields the server seems to timeout and I cannot continue.
See this in the logs:
Membership received a request to remove 192.xxx(23019)<v3>:1025 from 192.xxxx(GemFireLocator:31342:locator)<ec><v0>:1024 reason=Member isn't responding to heartbeat requests
[fatal 2018/07/10 13:05:24.880 SAST <unicast receiver,pvz-dell-40480> tid=0x1e] Membership service failure: Member isn't responding to heartbeat requests
org.apache.geode.ForcedDisconnectException: Member isn't responding to heartbeat requests
[info 2018/07/10 13:05:27.987 SAST <DisconnectThread> tid=0x7b] Stopping membership services
[info 2018/07/10 13:05:28.038 SAST <DisconnectThread> tid=0x7b] GMSHealthMonitor server socket is closed in stopServices().
[info 2018/07/10 13:05:28.040 SAST <Geode Failure Detection Server thread 0> tid=0x22] GMSHealthMonitor server thread exiting
[info 2018/07/10 13:05:28.041 SAST <DisconnectThread> tid=0x7b] GMSHealthMonitor serverSocketExecutor is terminated
I was wondering if there is some setting on the Geode server or client that I can set that won't timeout so quickly.
I have one locator and my server connects to that locator.
Not sure if anyone has tried to debug functions running on a Geode server via Intellij Idea.....
Kindly
Pieter