You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Ekta Awasthi <Ek...@officedepot.com.INVALID> on 2022/09/19 18:36:58 UTC
Artemis server getting killed randomly by OS Kernel
Hello,
My apache artemis server running on 2.22.0 is getting randomly, upon checking it turns out that the OS kernel is killing it with below error.
To my understanding the process with highest consuming memory should be getting killed but this is NOT the case when I am checking metrics from a metric tool. The memory and all are well under where it should be and there are no broker logs to check on anything.
dmesg | grep "21457"
Out of memory: Kill process 21457 (java) score 852 or sacrifice child
I am following up with my OS team to see if they can point me in the right direction or something is off with the version itself...?
Thanks
Ekta
CONFIDENTIALITY NOTICE: The information contained in this email and attached document(s) may contain confidential information that is intended only for the addressee(s). If you are not the intended recipient, you are hereby advised that any disclosure, copying, distribution or the taking of any action in reliance upon the information is prohibited. If you have received this email in error, please immediately notify the sender and delete it from your system.
Re: Artemis server getting killed randomly by OS Kernel
Posted by Ekta Awasthi <Ek...@officedepot.com.INVALID>.
Sad part is I am not seeing any logs with paging or when the broken is getting killed.
There are simply no logs, the only logs I see are before the process got killed.
Last log I see in Artemis.log is console is up and available. That’s it.
Thanks
Ekta Awasthi,
Engineer, EAI Operations & Support | Office Depot, Inc.
6600 North Military Trail | Boca Raton, FL 33496-2434
Office: 561-438-3552 | Mobile: 206-966-5577 | ekta.awasthi@officedepot.com
-- Tips for EAI Support Engagement --
-EAI Pre-Prod Support: Create requests on the following JIRA board EAI Operations Support
-EAI Production Support: Create requests via IT Service Desk self-service portal, instructions click here: EAI Support queue --> ODP - Enterprise Apps Integration Support
-As a reminder, the Service Availability Managers should be engaged for any service impacting issues, with a ***Page*** to NAITAVAILABILITYMGRS@OFFICEDEPOT.COM or by initiating a MIRT
________________________________
From: Clebert Suconic <cl...@gmail.com>
Sent: Monday, September 19, 2022 6:52:39 PM
To: users@activemq.apache.org <us...@activemq.apache.org>
Subject: Re: Artemis server getting killed randomly by OS Kernel
[CAUTION: EXTERNAL SENDER]
I would recommend check if your system is paging.. and how many
destinations are paging... 2.25.0 offers a significant improvement in
paging.
On Mon, Sep 19, 2022 at 3:31 PM Ekta Awasthi
<Ek...@officedepot.com.invalid> wrote:
>
> Hello,
>
> Thanks for your response on this issue.
>
> Artemis is the significant process sitting on my system with -Xms2G -Xmx6G. At the time instance was running it was hardly hitting 3gb is what I could see from monitoring tool so not sure why artemis is getting killed. If any thoughts, please share, I'd appreciate it. Thanks
>
> Thanks
> Ekta
>
> ________________________________
> From: slx@cecurity.com.INVALID <sl...@cecurity.com.INVALID>
> Sent: Monday, September 19, 2022 2:49 PM
> To: users@activemq.apache.org <us...@activemq.apache.org>
> Subject: Re: Artemis server getting killed randomly by OS Kernel
>
> [CAUTION: EXTERNAL SENDER]
>
>
> By default the oom killer will always target first the biggest process.
> You'll need to monitor your global memory consumption and check if it is
> not being eaten by a swarm of little hungry processes.
>
> If artemis is the only significant process sitting on your system, check
> first the -Xmx value passed to your java runtime and if the running
> instance have a consumption significantly higher than the configured value.
>
> Regards,
>
> SL
>
> Le 19/09/2022 à 20:36, Ekta Awasthi a écrit :
> > Hello,
> >
> > My apache artemis server running on 2.22.0 is getting randomly, upon checking it turns out that the OS kernel is killing it with below error.
> > To my understanding the process with highest consuming memory should be getting killed but this is NOT the case when I am checking metrics from a metric tool. The memory and all are well under where it should be and there are no broker logs to check on anything.
> >
> > dmesg | grep "21457"
> > Out of memory: Kill process 21457 (java) score 852 or sacrifice child
> >
> > I am following up with my OS team to see if they can point me in the right direction or something is off with the version itself...?
> >
> > Thanks
> > Ekta
> >
> > CONFIDENTIALITY NOTICE: The information contained in this email and attached document(s) may contain confidential information that is intended only for the addressee(s). If you are not the intended recipient, you are hereby advised that any disclosure, copying, distribution or the taking of any action in reliance upon the information is prohibited. If you have received this email in error, please immediately notify the sender and delete it from your system.
> >
--
Clebert Suconic
Re: Artemis server getting killed randomly by OS Kernel
Posted by Clebert Suconic <cl...@gmail.com>.
I would recommend check if your system is paging.. and how many
destinations are paging... 2.25.0 offers a significant improvement in
paging.
On Mon, Sep 19, 2022 at 3:31 PM Ekta Awasthi
<Ek...@officedepot.com.invalid> wrote:
>
> Hello,
>
> Thanks for your response on this issue.
>
> Artemis is the significant process sitting on my system with -Xms2G -Xmx6G. At the time instance was running it was hardly hitting 3gb is what I could see from monitoring tool so not sure why artemis is getting killed. If any thoughts, please share, I'd appreciate it. Thanks
>
> Thanks
> Ekta
>
> ________________________________
> From: slx@cecurity.com.INVALID <sl...@cecurity.com.INVALID>
> Sent: Monday, September 19, 2022 2:49 PM
> To: users@activemq.apache.org <us...@activemq.apache.org>
> Subject: Re: Artemis server getting killed randomly by OS Kernel
>
> [CAUTION: EXTERNAL SENDER]
>
>
> By default the oom killer will always target first the biggest process.
> You'll need to monitor your global memory consumption and check if it is
> not being eaten by a swarm of little hungry processes.
>
> If artemis is the only significant process sitting on your system, check
> first the -Xmx value passed to your java runtime and if the running
> instance have a consumption significantly higher than the configured value.
>
> Regards,
>
> SL
>
> Le 19/09/2022 à 20:36, Ekta Awasthi a écrit :
> > Hello,
> >
> > My apache artemis server running on 2.22.0 is getting randomly, upon checking it turns out that the OS kernel is killing it with below error.
> > To my understanding the process with highest consuming memory should be getting killed but this is NOT the case when I am checking metrics from a metric tool. The memory and all are well under where it should be and there are no broker logs to check on anything.
> >
> > dmesg | grep "21457"
> > Out of memory: Kill process 21457 (java) score 852 or sacrifice child
> >
> > I am following up with my OS team to see if they can point me in the right direction or something is off with the version itself...?
> >
> > Thanks
> > Ekta
> >
> > CONFIDENTIALITY NOTICE: The information contained in this email and attached document(s) may contain confidential information that is intended only for the addressee(s). If you are not the intended recipient, you are hereby advised that any disclosure, copying, distribution or the taking of any action in reliance upon the information is prohibited. If you have received this email in error, please immediately notify the sender and delete it from your system.
> >
--
Clebert Suconic
Re: Artemis server getting killed randomly by OS Kernel
Posted by Ekta Awasthi <Ek...@officedepot.com.INVALID>.
Hello,
Thanks for your response on this issue.
Artemis is the significant process sitting on my system with -Xms2G -Xmx6G. At the time instance was running it was hardly hitting 3gb is what I could see from monitoring tool so not sure why artemis is getting killed. If any thoughts, please share, I'd appreciate it. Thanks
Thanks
Ekta
________________________________
From: slx@cecurity.com.INVALID <sl...@cecurity.com.INVALID>
Sent: Monday, September 19, 2022 2:49 PM
To: users@activemq.apache.org <us...@activemq.apache.org>
Subject: Re: Artemis server getting killed randomly by OS Kernel
[CAUTION: EXTERNAL SENDER]
By default the oom killer will always target first the biggest process.
You'll need to monitor your global memory consumption and check if it is
not being eaten by a swarm of little hungry processes.
If artemis is the only significant process sitting on your system, check
first the -Xmx value passed to your java runtime and if the running
instance have a consumption significantly higher than the configured value.
Regards,
SL
Le 19/09/2022 à 20:36, Ekta Awasthi a écrit :
> Hello,
>
> My apache artemis server running on 2.22.0 is getting randomly, upon checking it turns out that the OS kernel is killing it with below error.
> To my understanding the process with highest consuming memory should be getting killed but this is NOT the case when I am checking metrics from a metric tool. The memory and all are well under where it should be and there are no broker logs to check on anything.
>
> dmesg | grep "21457"
> Out of memory: Kill process 21457 (java) score 852 or sacrifice child
>
> I am following up with my OS team to see if they can point me in the right direction or something is off with the version itself...?
>
> Thanks
> Ekta
>
> CONFIDENTIALITY NOTICE: The information contained in this email and attached document(s) may contain confidential information that is intended only for the addressee(s). If you are not the intended recipient, you are hereby advised that any disclosure, copying, distribution or the taking of any action in reliance upon the information is prohibited. If you have received this email in error, please immediately notify the sender and delete it from your system.
>
Re: Artemis server getting killed randomly by OS Kernel
Posted by sl...@cecurity.com.INVALID.
By default the oom killer will always target first the biggest process.
You'll need to monitor your global memory consumption and check if it is
not being eaten by a swarm of little hungry processes.
If artemis is the only significant process sitting on your system, check
first the -Xmx value passed to your java runtime and if the running
instance have a consumption significantly higher than the configured value.
Regards,
SL
Le 19/09/2022 à 20:36, Ekta Awasthi a écrit :
> Hello,
>
> My apache artemis server running on 2.22.0 is getting randomly, upon checking it turns out that the OS kernel is killing it with below error.
> To my understanding the process with highest consuming memory should be getting killed but this is NOT the case when I am checking metrics from a metric tool. The memory and all are well under where it should be and there are no broker logs to check on anything.
>
> dmesg | grep "21457"
> Out of memory: Kill process 21457 (java) score 852 or sacrifice child
>
> I am following up with my OS team to see if they can point me in the right direction or something is off with the version itself...?
>
> Thanks
> Ekta
>
> CONFIDENTIALITY NOTICE: The information contained in this email and attached document(s) may contain confidential information that is intended only for the addressee(s). If you are not the intended recipient, you are hereby advised that any disclosure, copying, distribution or the taking of any action in reliance upon the information is prohibited. If you have received this email in error, please immediately notify the sender and delete it from your system.
>