You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by rui chen <ru...@gmail.com> on 2023/09/25 02:15:15 UTC
After using the jemalloc memory allocator for a period of time, checkpoint timeout occurs and tasks are stuck
After using the jemalloc memory allocator for a period of time, checkpoint
timeout occurs and tasks are stuck. Who has encountered this? flink
version:1.13.2, jiemalloc version: 5.3.0
Re: After using the jemalloc memory allocator for a period of time, checkpoint timeout occurs and tasks are stuck
Posted by Feng Jin <ji...@gmail.com>.
hi Rui,
We are using open jdk , `openjdk version "1.8.0_265"`
Best,
Feng
On Thu, Sep 28, 2023 at 2:15 PM rui chen <ru...@gmail.com> wrote:
> hi Feng
>
> Are you using the open jdk or the oracle jdk?
>
> Best，
> rui
>
> rui chen <ru...@gmail.com> 于2023年9月27日周三 20:22写道：
>
>> hi Feng，
>>
>> Thanks for your reply, we are 8 u192 JDK, may be is the question, I found
>> a JDK issue:https://bugs.openjdk.org/browse/JDK-8215355.
>>
>> Best,
>> rui
>>
>> Feng Jin <ji...@gmail.com> 于2023年9月27日周三 20:09写道：
>>
>>> hi Rui,
>>>
>>> Which version of JDK are you using?
>>>
>>> This issue could potentially be a bug in the JDK version.
>>>
>>> If you are using JDK 8, you can try using OpenJDK 8u265 as a possible
>>> solution.
>>>
>>>
>>> Best,
>>> Feng
>>>
>>>
>>> On Wed, Sep 27, 2023 at 8:08 PM rui chen <ru...@gmail.com> wrote:
>>>
>>>>
>>>>
>>>> rui chen <ru...@gmail.com> 于2023年9月27日周三 19:32写道：
>>>>
>>>>> hi Feng,
>>>>>
>>>>> Thank you for your reply，We observed the GC situation, there is no
>>>>> change before and after replacement, several tasks on our line using
>>>>> jemalloc have appeared stuck, after removing jemalloc, no stuck situation
>>>>> has been found.
>>>>>
>>>>> Best,
>>>>> rui
>>>>>
>>>>> Feng Jin <ji...@gmail.com> 于2023年9月27日周三 19:19写道：
>>>>>
>>>>>>
>>>>>> hi rui,
>>>>>>
>>>>>> In general, checkpoint timeouts are typically associated with the
>>>>>> job's processing performance. When using jemalloc, performance degradation
>>>>>> is generally not observed.
>>>>>>
>>>>>> It is advisable to analyze whether the job's garbage collection (GC)
>>>>>> has become more frequent.
>>>>>>
>>>>>>
>>>>>> Best,
>>>>>> Feng
>>>>>>
>>>>>>
>>>>>> On Mon, Sep 25, 2023 at 1:21 PM rui chen <ru...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> After using the jemalloc memory allocator for a period of time,
>>>>>>> checkpoint timeout occurs and tasks are stuck. Who has encountered this?
>>>>>>> flink version:1.13.2, jiemalloc version: 5.3.0
>>>>>>>
>>>>>>
Re: After using the jemalloc memory allocator for a period of time, checkpoint timeout occurs and tasks are stuck
Posted by rui chen <ru...@gmail.com>.
hi Feng
Are you using the open jdk or the oracle jdk?
Best，
rui
rui chen <ru...@gmail.com> 于2023年9月27日周三 20:22写道：
> hi Feng，
>
> Thanks for your reply, we are 8 u192 JDK, may be is the question, I found
> a JDK issue:https://bugs.openjdk.org/browse/JDK-8215355.
>
> Best,
> rui
>
> Feng Jin <ji...@gmail.com> 于2023年9月27日周三 20:09写道：
>
>> hi Rui,
>>
>> Which version of JDK are you using?
>>
>> This issue could potentially be a bug in the JDK version.
>>
>> If you are using JDK 8, you can try using OpenJDK 8u265 as a possible
>> solution.
>>
>>
>> Best,
>> Feng
>>
>>
>> On Wed, Sep 27, 2023 at 8:08 PM rui chen <ru...@gmail.com> wrote:
>>
>>>
>>>
>>> rui chen <ru...@gmail.com> 于2023年9月27日周三 19:32写道：
>>>
>>>> hi Feng,
>>>>
>>>> Thank you for your reply，We observed the GC situation, there is no
>>>> change before and after replacement, several tasks on our line using
>>>> jemalloc have appeared stuck, after removing jemalloc, no stuck situation
>>>> has been found.
>>>>
>>>> Best,
>>>> rui
>>>>
>>>> Feng Jin <ji...@gmail.com> 于2023年9月27日周三 19:19写道：
>>>>
>>>>>
>>>>> hi rui,
>>>>>
>>>>> In general, checkpoint timeouts are typically associated with the
>>>>> job's processing performance. When using jemalloc, performance degradation
>>>>> is generally not observed.
>>>>>
>>>>> It is advisable to analyze whether the job's garbage collection (GC)
>>>>> has become more frequent.
>>>>>
>>>>>
>>>>> Best,
>>>>> Feng
>>>>>
>>>>>
>>>>> On Mon, Sep 25, 2023 at 1:21 PM rui chen <ru...@gmail.com> wrote:
>>>>>
>>>>>> After using the jemalloc memory allocator for a period of time,
>>>>>> checkpoint timeout occurs and tasks are stuck. Who has encountered this?
>>>>>> flink version:1.13.2, jiemalloc version: 5.3.0
>>>>>>
>>>>>
Re: After using the jemalloc memory allocator for a period of time, checkpoint timeout occurs and tasks are stuck
Posted by rui chen <ru...@gmail.com>.
hi Feng，
Thanks for your reply, we are 8 u192 JDK, may be is the question, I found a
JDK issue:https://bugs.openjdk.org/browse/JDK-8215355.
Best,
rui
Feng Jin <ji...@gmail.com> 于2023年9月27日周三 20:09写道：
> hi Rui,
>
> Which version of JDK are you using?
>
> This issue could potentially be a bug in the JDK version.
>
> If you are using JDK 8, you can try using OpenJDK 8u265 as a possible
> solution.
>
>
> Best,
> Feng
>
>
> On Wed, Sep 27, 2023 at 8:08 PM rui chen <ru...@gmail.com> wrote:
>
>>
>>
>> rui chen <ru...@gmail.com> 于2023年9月27日周三 19:32写道：
>>
>>> hi Feng,
>>>
>>> Thank you for your reply，We observed the GC situation, there is no
>>> change before and after replacement, several tasks on our line using
>>> jemalloc have appeared stuck, after removing jemalloc, no stuck situation
>>> has been found.
>>>
>>> Best,
>>> rui
>>>
>>> Feng Jin <ji...@gmail.com> 于2023年9月27日周三 19:19写道：
>>>
>>>>
>>>> hi rui,
>>>>
>>>> In general, checkpoint timeouts are typically associated with the job's
>>>> processing performance. When using jemalloc, performance degradation is
>>>> generally not observed.
>>>>
>>>> It is advisable to analyze whether the job's garbage collection (GC)
>>>> has become more frequent.
>>>>
>>>>
>>>> Best,
>>>> Feng
>>>>
>>>>
>>>> On Mon, Sep 25, 2023 at 1:21 PM rui chen <ru...@gmail.com> wrote:
>>>>
>>>>> After using the jemalloc memory allocator for a period of time,
>>>>> checkpoint timeout occurs and tasks are stuck. Who has encountered this?
>>>>> flink version:1.13.2, jiemalloc version: 5.3.0
>>>>>
>>>>
Re: After using the jemalloc memory allocator for a period of time, checkpoint timeout occurs and tasks are stuck
Posted by Feng Jin <ji...@gmail.com>.
hi Rui,
Which version of JDK are you using?
This issue could potentially be a bug in the JDK version.
If you are using JDK 8, you can try using OpenJDK 8u265 as a possible
solution.
Best,
Feng
On Wed, Sep 27, 2023 at 8:08 PM rui chen <ru...@gmail.com> wrote:
>
>
> rui chen <ru...@gmail.com> 于2023年9月27日周三 19:32写道：
>
>> hi Feng,
>>
>> Thank you for your reply，We observed the GC situation, there is no change
>> before and after replacement, several tasks on our line using jemalloc have
>> appeared stuck, after removing jemalloc, no stuck situation has been found.
>>
>> Best,
>> rui
>>
>> Feng Jin <ji...@gmail.com> 于2023年9月27日周三 19:19写道：
>>
>>>
>>> hi rui,
>>>
>>> In general, checkpoint timeouts are typically associated with the job's
>>> processing performance. When using jemalloc, performance degradation is
>>> generally not observed.
>>>
>>> It is advisable to analyze whether the job's garbage collection (GC) has
>>> become more frequent.
>>>
>>>
>>> Best,
>>> Feng
>>>
>>>
>>> On Mon, Sep 25, 2023 at 1:21 PM rui chen <ru...@gmail.com> wrote:
>>>
>>>> After using the jemalloc memory allocator for a period of time,
>>>> checkpoint timeout occurs and tasks are stuck. Who has encountered this?
>>>> flink version:1.13.2, jiemalloc version: 5.3.0
>>>>
>>>
Re: After using the jemalloc memory allocator for a period of time, checkpoint timeout occurs and tasks are stuck
Posted by rui chen <ru...@gmail.com>.
rui chen <ru...@gmail.com> 于2023年9月27日周三 19:32写道：
> hi Feng,
>
> Thank you for your reply，We observed the GC situation, there is no change
> before and after replacement, several tasks on our line using jemalloc have
> appeared stuck, after removing jemalloc, no stuck situation has been found.
>
> Best,
> rui
>
> Feng Jin <ji...@gmail.com> 于2023年9月27日周三 19:19写道：
>
>>
>> hi rui,
>>
>> In general, checkpoint timeouts are typically associated with the job's
>> processing performance. When using jemalloc, performance degradation is
>> generally not observed.
>>
>> It is advisable to analyze whether the job's garbage collection (GC) has
>> become more frequent.
>>
>>
>> Best,
>> Feng
>>
>>
>> On Mon, Sep 25, 2023 at 1:21 PM rui chen <ru...@gmail.com> wrote:
>>
>>> After using the jemalloc memory allocator for a period of time,
>>> checkpoint timeout occurs and tasks are stuck. Who has encountered this?
>>> flink version:1.13.2, jiemalloc version: 5.3.0
>>>
>>
Re: After using the jemalloc memory allocator for a period of time, checkpoint timeout occurs and tasks are stuck
Posted by Feng Jin <ji...@gmail.com>.
hi rui,
In general, checkpoint timeouts are typically associated with the job's
processing performance. When using jemalloc, performance degradation is
generally not observed.
It is advisable to analyze whether the job's garbage collection (GC) has
become more frequent.
Best,
Feng
On Mon, Sep 25, 2023 at 1:21 PM rui chen <ru...@gmail.com> wrote:
> After using the jemalloc memory allocator for a period of time, checkpoint
> timeout occurs and tasks are stuck. Who has encountered this? flink
> version:1.13.2, jiemalloc version: 5.3.0
>