You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomee.apache.org by Matthew Broadhead <ma...@nbmlaw.co.uk> on 2017/10/10 08:52:25 UTC
freezing on CentOS 7
i am currently stuck on Java release
java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64
on CentOS 7 with TomEE 7.0.3
updating CentOS to
Version : 1.8.0.144
Release : 0.b01.el7_4
results in random freezes with no error in catalina.out
back story: the system automatically updated to 1.8.0.144. TomEE
started crashing. looked like it was still running. no output in
catalina.out. kept restarting and it would run for a while but same
result. rebooted whole server. no change. rolled back to 1.8.0.131
and blocked yum from updating. everything works ok now.
Re: freezing on CentOS 7
Posted by Jonathan Gallimore <jo...@gmail.com>.
Fantastic news. Thanks for following up, and I'm glad things are looking ok.
I'd still be interested in figuring out what was going on if you're able to
tell us a bit more about your app so we can try and reproduce.
Jon
On 5 Nov 2017 09:39, "Matthew Broadhead" <ma...@nbmlaw.co.uk>
wrote:
> to follow up i have not had any more freezing since updating to 1.8.0.151
> (skipping 144)
>
> On 25/10/2017 10:05, Matthew Broadhead wrote:
>
>> i just allowed CentOS 7 to update to
>> Name : java-1.8.0-openjdk
>> Arch : x86_64
>> Epoch : 1
>> Version : 1.8.0.151
>> Release : 1.b12.el7_4
>> Size : 241 k
>> last night. everything seems to be running ok with that version so far.
>> i will let you know if there are any problems
>>
>> On 10/10/2017 12:52, Matthew Broadhead wrote:
>>
>>> this was only on 7.0.3 in production. only happens if i upgrade from 131
>>> to 144
>>>
>>> On 10/10/2017 12:20, Jonathan Gallimore wrote:
>>>
>>>> Thanks Matthew - if we're able to reproduce in a safe environment, that
>>>> will be a big help to solve your problem. One other question - is this
>>>> specific to the JDK version or were there any other changes? I note you
>>>> posted on the 7.0.4 vote on dev@ - does this only happen on 7.0.4 or
>>>> are
>>>> you seeing it with 7.0.3 as well?
>>>>
>>>> Cheers
>>>>
>>>> Jon
>>>>
>>>> On Tue, Oct 10, 2017 at 10:06 AM, Matthew Broadhead <
>>>> matthew.broadhead@nbmlaw.co.uk> wrote:
>>>>
>>>> I am not sure if it recovers because in production I was forced to
>>>>> restart
>>>>> the system.
>>>>> I can set it up on my local virtual box but without much load it might
>>>>> be
>>>>> hard to get it to crash. I will try it and let you know
>>>>>
>>>>>
>>>>> On 10/10/2017 11:02, Jonathan Gallimore wrote:
>>>>>
>>>>> Do you have a test system you can run it on, and grab the output from
>>>>>> jstack and jmap when it happens? How long do the freezes last - does
>>>>>> the
>>>>>> system ever recover? Do you see increases in active threads or CPU?
>>>>>>
>>>>>> Many thanks
>>>>>>
>>>>>> Jon
>>>>>>
>>>>>> On Tue, Oct 10, 2017 at 9:52 AM, Matthew Broadhead <
>>>>>> matthew.broadhead@nbmlaw.co.uk> wrote:
>>>>>>
>>>>>> i am currently stuck on Java release
>>>>>>
>>>>>>> java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64
>>>>>>> on CentOS 7 with TomEE 7.0.3
>>>>>>>
>>>>>>> updating CentOS to
>>>>>>> Version : 1.8.0.144
>>>>>>> Release : 0.b01.el7_4
>>>>>>> results in random freezes with no error in catalina.out
>>>>>>>
>>>>>>> back story: the system automatically updated to 1.8.0.144. TomEE
>>>>>>> started
>>>>>>> crashing. looked like it was still running. no output in
>>>>>>> catalina.out.
>>>>>>> kept restarting and it would run for a while but same result.
>>>>>>> rebooted
>>>>>>> whole server. no change. rolled back to 1.8.0.131 and blocked yum
>>>>>>> from
>>>>>>> updating. everything works ok now.
>>>>>>>
>>>>>>>
>>>>>>>
>>>
>>
>
Re: freezing on CentOS 7
Posted by Matthew Broadhead <ma...@nbmlaw.co.uk>.
to follow up i have not had any more freezing since updating to
1.8.0.151 (skipping 144)
On 25/10/2017 10:05, Matthew Broadhead wrote:
> i just allowed CentOS 7 to update to
> Name : java-1.8.0-openjdk
> Arch : x86_64
> Epoch : 1
> Version : 1.8.0.151
> Release : 1.b12.el7_4
> Size : 241 k
> last night. everything seems to be running ok with that version so
> far. i will let you know if there are any problems
>
> On 10/10/2017 12:52, Matthew Broadhead wrote:
>> this was only on 7.0.3 in production. only happens if i upgrade from
>> 131 to 144
>>
>> On 10/10/2017 12:20, Jonathan Gallimore wrote:
>>> Thanks Matthew - if we're able to reproduce in a safe environment, that
>>> will be a big help to solve your problem. One other question - is this
>>> specific to the JDK version or were there any other changes? I note you
>>> posted on the 7.0.4 vote on dev@ - does this only happen on 7.0.4 or
>>> are
>>> you seeing it with 7.0.3 as well?
>>>
>>> Cheers
>>>
>>> Jon
>>>
>>> On Tue, Oct 10, 2017 at 10:06 AM, Matthew Broadhead <
>>> matthew.broadhead@nbmlaw.co.uk> wrote:
>>>
>>>> I am not sure if it recovers because in production I was forced to
>>>> restart
>>>> the system.
>>>> I can set it up on my local virtual box but without much load it
>>>> might be
>>>> hard to get it to crash. I will try it and let you know
>>>>
>>>>
>>>> On 10/10/2017 11:02, Jonathan Gallimore wrote:
>>>>
>>>>> Do you have a test system you can run it on, and grab the output from
>>>>> jstack and jmap when it happens? How long do the freezes last -
>>>>> does the
>>>>> system ever recover? Do you see increases in active threads or CPU?
>>>>>
>>>>> Many thanks
>>>>>
>>>>> Jon
>>>>>
>>>>> On Tue, Oct 10, 2017 at 9:52 AM, Matthew Broadhead <
>>>>> matthew.broadhead@nbmlaw.co.uk> wrote:
>>>>>
>>>>> i am currently stuck on Java release
>>>>>> java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64
>>>>>> on CentOS 7 with TomEE 7.0.3
>>>>>>
>>>>>> updating CentOS to
>>>>>> Version : 1.8.0.144
>>>>>> Release : 0.b01.el7_4
>>>>>> results in random freezes with no error in catalina.out
>>>>>>
>>>>>> back story: the system automatically updated to 1.8.0.144. TomEE
>>>>>> started
>>>>>> crashing. looked like it was still running. no output in
>>>>>> catalina.out.
>>>>>> kept restarting and it would run for a while but same result.
>>>>>> rebooted
>>>>>> whole server. no change. rolled back to 1.8.0.131 and blocked
>>>>>> yum from
>>>>>> updating. everything works ok now.
>>>>>>
>>>>>>
>>
>
Re: freezing on CentOS 7
Posted by Matthew Broadhead <ma...@nbmlaw.co.uk>.
i just allowed CentOS 7 to update to
Name : java-1.8.0-openjdk
Arch : x86_64
Epoch : 1
Version : 1.8.0.151
Release : 1.b12.el7_4
Size : 241 k
last night. everything seems to be running ok with that version so
far. i will let you know if there are any problems
On 10/10/2017 12:52, Matthew Broadhead wrote:
> this was only on 7.0.3 in production. only happens if i upgrade from
> 131 to 144
>
> On 10/10/2017 12:20, Jonathan Gallimore wrote:
>> Thanks Matthew - if we're able to reproduce in a safe environment, that
>> will be a big help to solve your problem. One other question - is this
>> specific to the JDK version or were there any other changes? I note you
>> posted on the 7.0.4 vote on dev@ - does this only happen on 7.0.4 or are
>> you seeing it with 7.0.3 as well?
>>
>> Cheers
>>
>> Jon
>>
>> On Tue, Oct 10, 2017 at 10:06 AM, Matthew Broadhead <
>> matthew.broadhead@nbmlaw.co.uk> wrote:
>>
>>> I am not sure if it recovers because in production I was forced to
>>> restart
>>> the system.
>>> I can set it up on my local virtual box but without much load it
>>> might be
>>> hard to get it to crash. I will try it and let you know
>>>
>>>
>>> On 10/10/2017 11:02, Jonathan Gallimore wrote:
>>>
>>>> Do you have a test system you can run it on, and grab the output from
>>>> jstack and jmap when it happens? How long do the freezes last -
>>>> does the
>>>> system ever recover? Do you see increases in active threads or CPU?
>>>>
>>>> Many thanks
>>>>
>>>> Jon
>>>>
>>>> On Tue, Oct 10, 2017 at 9:52 AM, Matthew Broadhead <
>>>> matthew.broadhead@nbmlaw.co.uk> wrote:
>>>>
>>>> i am currently stuck on Java release
>>>>> java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64
>>>>> on CentOS 7 with TomEE 7.0.3
>>>>>
>>>>> updating CentOS to
>>>>> Version : 1.8.0.144
>>>>> Release : 0.b01.el7_4
>>>>> results in random freezes with no error in catalina.out
>>>>>
>>>>> back story: the system automatically updated to 1.8.0.144. TomEE
>>>>> started
>>>>> crashing. looked like it was still running. no output in
>>>>> catalina.out.
>>>>> kept restarting and it would run for a while but same result.
>>>>> rebooted
>>>>> whole server. no change. rolled back to 1.8.0.131 and blocked
>>>>> yum from
>>>>> updating. everything works ok now.
>>>>>
>>>>>
>
Re: freezing on CentOS 7
Posted by Matthew Broadhead <ma...@nbmlaw.co.uk>.
this was only on 7.0.3 in production. only happens if i upgrade from
131 to 144
On 10/10/2017 12:20, Jonathan Gallimore wrote:
> Thanks Matthew - if we're able to reproduce in a safe environment, that
> will be a big help to solve your problem. One other question - is this
> specific to the JDK version or were there any other changes? I note you
> posted on the 7.0.4 vote on dev@ - does this only happen on 7.0.4 or are
> you seeing it with 7.0.3 as well?
>
> Cheers
>
> Jon
>
> On Tue, Oct 10, 2017 at 10:06 AM, Matthew Broadhead <
> matthew.broadhead@nbmlaw.co.uk> wrote:
>
>> I am not sure if it recovers because in production I was forced to restart
>> the system.
>> I can set it up on my local virtual box but without much load it might be
>> hard to get it to crash. I will try it and let you know
>>
>>
>> On 10/10/2017 11:02, Jonathan Gallimore wrote:
>>
>>> Do you have a test system you can run it on, and grab the output from
>>> jstack and jmap when it happens? How long do the freezes last - does the
>>> system ever recover? Do you see increases in active threads or CPU?
>>>
>>> Many thanks
>>>
>>> Jon
>>>
>>> On Tue, Oct 10, 2017 at 9:52 AM, Matthew Broadhead <
>>> matthew.broadhead@nbmlaw.co.uk> wrote:
>>>
>>> i am currently stuck on Java release
>>>> java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64
>>>> on CentOS 7 with TomEE 7.0.3
>>>>
>>>> updating CentOS to
>>>> Version : 1.8.0.144
>>>> Release : 0.b01.el7_4
>>>> results in random freezes with no error in catalina.out
>>>>
>>>> back story: the system automatically updated to 1.8.0.144. TomEE started
>>>> crashing. looked like it was still running. no output in catalina.out.
>>>> kept restarting and it would run for a while but same result. rebooted
>>>> whole server. no change. rolled back to 1.8.0.131 and blocked yum from
>>>> updating. everything works ok now.
>>>>
>>>>
Re: freezing on CentOS 7
Posted by Jonathan Gallimore <jo...@gmail.com>.
Thanks Matthew - if we're able to reproduce in a safe environment, that
will be a big help to solve your problem. One other question - is this
specific to the JDK version or were there any other changes? I note you
posted on the 7.0.4 vote on dev@ - does this only happen on 7.0.4 or are
you seeing it with 7.0.3 as well?
Cheers
Jon
On Tue, Oct 10, 2017 at 10:06 AM, Matthew Broadhead <
matthew.broadhead@nbmlaw.co.uk> wrote:
> I am not sure if it recovers because in production I was forced to restart
> the system.
> I can set it up on my local virtual box but without much load it might be
> hard to get it to crash. I will try it and let you know
>
>
> On 10/10/2017 11:02, Jonathan Gallimore wrote:
>
>> Do you have a test system you can run it on, and grab the output from
>> jstack and jmap when it happens? How long do the freezes last - does the
>> system ever recover? Do you see increases in active threads or CPU?
>>
>> Many thanks
>>
>> Jon
>>
>> On Tue, Oct 10, 2017 at 9:52 AM, Matthew Broadhead <
>> matthew.broadhead@nbmlaw.co.uk> wrote:
>>
>> i am currently stuck on Java release
>>> java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64
>>> on CentOS 7 with TomEE 7.0.3
>>>
>>> updating CentOS to
>>> Version : 1.8.0.144
>>> Release : 0.b01.el7_4
>>> results in random freezes with no error in catalina.out
>>>
>>> back story: the system automatically updated to 1.8.0.144. TomEE started
>>> crashing. looked like it was still running. no output in catalina.out.
>>> kept restarting and it would run for a while but same result. rebooted
>>> whole server. no change. rolled back to 1.8.0.131 and blocked yum from
>>> updating. everything works ok now.
>>>
>>>
>
Re: freezing on CentOS 7
Posted by Matthew Broadhead <ma...@nbmlaw.co.uk>.
I am not sure if it recovers because in production I was forced to
restart the system.
I can set it up on my local virtual box but without much load it might
be hard to get it to crash. I will try it and let you know
On 10/10/2017 11:02, Jonathan Gallimore wrote:
> Do you have a test system you can run it on, and grab the output from
> jstack and jmap when it happens? How long do the freezes last - does the
> system ever recover? Do you see increases in active threads or CPU?
>
> Many thanks
>
> Jon
>
> On Tue, Oct 10, 2017 at 9:52 AM, Matthew Broadhead <
> matthew.broadhead@nbmlaw.co.uk> wrote:
>
>> i am currently stuck on Java release
>> java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64
>> on CentOS 7 with TomEE 7.0.3
>>
>> updating CentOS to
>> Version : 1.8.0.144
>> Release : 0.b01.el7_4
>> results in random freezes with no error in catalina.out
>>
>> back story: the system automatically updated to 1.8.0.144. TomEE started
>> crashing. looked like it was still running. no output in catalina.out.
>> kept restarting and it would run for a while but same result. rebooted
>> whole server. no change. rolled back to 1.8.0.131 and blocked yum from
>> updating. everything works ok now.
>>
Re: freezing on CentOS 7
Posted by Jonathan Gallimore <jo...@gmail.com>.
Do you have a test system you can run it on, and grab the output from
jstack and jmap when it happens? How long do the freezes last - does the
system ever recover? Do you see increases in active threads or CPU?
Many thanks
Jon
On Tue, Oct 10, 2017 at 9:52 AM, Matthew Broadhead <
matthew.broadhead@nbmlaw.co.uk> wrote:
> i am currently stuck on Java release
> java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64
> on CentOS 7 with TomEE 7.0.3
>
> updating CentOS to
> Version : 1.8.0.144
> Release : 0.b01.el7_4
> results in random freezes with no error in catalina.out
>
> back story: the system automatically updated to 1.8.0.144. TomEE started
> crashing. looked like it was still running. no output in catalina.out.
> kept restarting and it would run for a while but same result. rebooted
> whole server. no change. rolled back to 1.8.0.131 and blocked yum from
> updating. everything works ok now.
>