You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@geode.apache.org by aashish choudhary <aa...@gmail.com> on 2020/07/09 16:48:51 UTC
internal geode calls taking too long in function
Hi,
We are seeing lot of Thread Contention issue via invoking geode functions.
We have enabled Appdynamics transaction monitoring which points to below
geode internal classes with method which are taking too long.
Thread Contention on this call
org.apache.geode.internal.cache.execute.PartitionedRegionFunctionResultSender:lastResult
Blocked on
org.apache.geode.internal.cache.execute.PartitionedRegionFunctionResultSender@3501a829
Class org.apache.geode.internal.cache.execute.PartitionedRegionFunctionResul
tSender
Method
lastResult
Time taken:- 1,664ms
Class org.apache.geode.internal.HeapDataOutputStream
Method sendTo
Time taken:- 27,525ms
Then in this case there are repeated calls like this while put operation
Class org.apache.geode.internal.cache.PartitionedRegion
Method putInBucket
Class
java.lang.Thread
sleep
java.lang.Thread
sleep
java.lang.Thread
sleep
Is this an environmental issue or some configuration tuning is required
over here. Why would geode internal method calls take so much time?
Thanks,
Ashish
Re: internal geode calls taking too long in function
Posted by Kirk Lund <kl...@apache.org>.
I've seen some geode tests take a very long time if DNS is timing out. In
general, this sounds like environment, but it's hard to guess without some
more details about your config and what you're trying to do.
On Fri, Jul 10, 2020 at 7:34 AM aashish choudhary <
aashish.choudhary1@gmail.com> wrote:
> We tried to check if there is any data rebalancing required but rebalance
> command gives no such indication. Another gfsh command we tried is show
> dead-locks just in case if there are any BLOCKED threads but that command
> returned “No dead-locks found”. In some cases below mentioned geode
> internal method calls are taking more time(>60 seconds).
>
> Any suggestions?
>
> On Thu, 9 Jul 2020 at 11:21 PM, aashish choudhary <
> aashish.choudhary1@gmail.com> wrote:
>
>> We are using geode 1.8.0.
>>
>> With best regards,
>> Ashish
>>
>> On Thu, Jul 9, 2020, 10:18 PM aashish choudhary <
>> aashish.choudhary1@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> We are seeing lot of Thread Contention issue via invoking geode
>>> functions. We have enabled Appdynamics transaction monitoring which points
>>> to below geode internal classes with method which are taking too long.
>>>
>>>
>>>
>>> Thread Contention on this call
>>>
>>>
>>> org.apache.geode.internal.cache.execute.PartitionedRegionFunctionResultSender:lastResult
>>> Blocked on
>>> org.apache.geode.internal.cache.execute.PartitionedRegionFunctionResultSender@3501a829
>>>
>>> Class org.apache.geode.internal.cache.execute.
>>> PartitionedRegionFunctionResultSender
>>>
>>> Method
>>>
>>> lastResult
>>>
>>> Time taken:- 1,664ms
>>>
>>>
>>> Class org.apache.geode.internal.HeapDataOutputStream
>>>
>>> Method sendTo
>>>
>>> Time taken:- 27,525ms
>>>
>>>
>>> Then in this case there are repeated calls like this while put operation
>>>
>>> Class org.apache.geode.internal.cache.PartitionedRegion
>>>
>>> Method putInBucket
>>>
>>> Class
>>>
>>> java.lang.Thread
>>>
>>> sleep
>>>
>>> java.lang.Thread
>>>
>>> sleep
>>>
>>> java.lang.Thread
>>>
>>> sleep
>>>
>>> Is this an environmental issue or some configuration tuning is required
>>> over here. Why would geode internal method calls take so much time?
>>>
>>> Thanks,
>>> Ashish
>>>
>> --
> With Best Regards,
> Ashish
>
Re: internal geode calls taking too long in function
Posted by aashish choudhary <aa...@gmail.com>.
We tried to check if there is any data rebalancing required but rebalance
command gives no such indication. Another gfsh command we tried is show
dead-locks just in case if there are any BLOCKED threads but that command
returned “No dead-locks found”. In some cases below mentioned geode
internal method calls are taking more time(>60 seconds).
Any suggestions?
On Thu, 9 Jul 2020 at 11:21 PM, aashish choudhary <
aashish.choudhary1@gmail.com> wrote:
> We are using geode 1.8.0.
>
> With best regards,
> Ashish
>
> On Thu, Jul 9, 2020, 10:18 PM aashish choudhary <
> aashish.choudhary1@gmail.com> wrote:
>
>> Hi,
>>
>> We are seeing lot of Thread Contention issue via invoking geode
>> functions. We have enabled Appdynamics transaction monitoring which points
>> to below geode internal classes with method which are taking too long.
>>
>>
>>
>> Thread Contention on this call
>>
>>
>> org.apache.geode.internal.cache.execute.PartitionedRegionFunctionResultSender:lastResult
>> Blocked on
>> org.apache.geode.internal.cache.execute.PartitionedRegionFunctionResultSender@3501a829
>>
>> Class org.apache.geode.internal.cache.execute.
>> PartitionedRegionFunctionResultSender
>>
>> Method
>>
>> lastResult
>>
>> Time taken:- 1,664ms
>>
>>
>> Class org.apache.geode.internal.HeapDataOutputStream
>>
>> Method sendTo
>>
>> Time taken:- 27,525ms
>>
>>
>> Then in this case there are repeated calls like this while put operation
>>
>> Class org.apache.geode.internal.cache.PartitionedRegion
>>
>> Method putInBucket
>>
>> Class
>>
>> java.lang.Thread
>>
>> sleep
>>
>> java.lang.Thread
>>
>> sleep
>>
>> java.lang.Thread
>>
>> sleep
>>
>> Is this an environmental issue or some configuration tuning is required
>> over here. Why would geode internal method calls take so much time?
>>
>> Thanks,
>> Ashish
>>
> --
With Best Regards,
Ashish
Re: internal geode calls taking too long in function
Posted by aashish choudhary <aa...@gmail.com>.
We are using geode 1.8.0.
With best regards,
Ashish
On Thu, Jul 9, 2020, 10:18 PM aashish choudhary <
aashish.choudhary1@gmail.com> wrote:
> Hi,
>
> We are seeing lot of Thread Contention issue via invoking geode
> functions. We have enabled Appdynamics transaction monitoring which points
> to below geode internal classes with method which are taking too long.
>
>
>
> Thread Contention on this call
>
>
> org.apache.geode.internal.cache.execute.PartitionedRegionFunctionResultSender:lastResult
> Blocked on
> org.apache.geode.internal.cache.execute.PartitionedRegionFunctionResultSender@3501a829
>
> Class org.apache.geode.internal.cache.execute.
> PartitionedRegionFunctionResultSender
>
> Method
>
> lastResult
>
> Time taken:- 1,664ms
>
>
> Class org.apache.geode.internal.HeapDataOutputStream
>
> Method sendTo
>
> Time taken:- 27,525ms
>
>
> Then in this case there are repeated calls like this while put operation
>
> Class org.apache.geode.internal.cache.PartitionedRegion
>
> Method putInBucket
>
> Class
>
> java.lang.Thread
>
> sleep
>
> java.lang.Thread
>
> sleep
>
> java.lang.Thread
>
> sleep
>
> Is this an environmental issue or some configuration tuning is required
> over here. Why would geode internal method calls take so much time?
>
> Thanks,
> Ashish
>