You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Patil, Pratibha" <pr...@siemens.com> on 2021/01/05 17:58:55 UTC
RE: [jira] [Work logged] (ARTEMIS-3021) OOM due to wrong CORE message
memory estimation
Please delete my account. I am getting too much mails everyday.
I Am not getting option in use profile.
Username - pratibha.patil@siemens.com
-----Original Message-----
From: ASF GitHub Bot (Jira) <ji...@apache.org>
Sent: Tuesday, January 5, 2021 10:51 PM
To: issues@activemq.apache.org
Subject: [jira] [Work logged] (ARTEMIS-3021) OOM due to wrong CORE message memory estimation
[ https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FARTEMIS-3021%3FfocusedWorklogId%3D531359%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Aworklog-tabpanel%23worklog-531359&data=04%7C01%7Cpratibha.patil%40siemens.com%7C70c604a6390c4dd8d80b08d8b19e47a2%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637454640700572684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ntHKSO8ulEITS1W847MMvNwhHLlH1tRz0dswo4TpDtQ%3D&reserved=0 ]
ASF GitHub Bot logged work on ARTEMIS-3021:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 05/Jan/21 17:20
Start Date: 05/Jan/21 17:20
Worklog Time Spent: 10m
Work Description: franz1981 commented on a change in pull request #3370:
URL: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Factivemq-artemis%2Fpull%2F3370%23discussion_r552075767&data=04%7C01%7Cpratibha.patil%40siemens.com%7C70c604a6390c4dd8d80b08d8b19e47a2%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637454640700572684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=xC4amBCRyzCBX9hfAEnLnYlF88dBNIFYqBHsw0W33es%3D&reserved=0
##########
File path: artemis-core-client/src/main/java/org/apache/activemq/artemis/core/message/impl/CoreMessage.java
##########
@@ -622,6 +619,12 @@ private RuntimeException onCheckPropertiesError(Throwable e) {
@Override
public int getMemoryEstimate() {
if (memoryEstimate == -1) {
+ // durable messages can enlarge buffer::capacity while persisted:
+ // better to encode it to properly account its max capacity
+ if (isDurable()) {
+ checkEncode();
Review comment:
That means that a durable message would get encoded some time before then expected: @clebertsuconic it's something we can tolerate?
if not the ideal thing would be to estimate it based on its state, but I'm not sure is feasable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 531359)
Time Spent: 1h (was: 50m)
> OOM due to wrong CORE message memory estimation
> -----------------------------------------------
>
> Key: ARTEMIS-3021
> URL: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FARTEMIS-3021&data=04%7C01%7Cpratibha.patil%40siemens.com%7C70c604a6390c4dd8d80b08d8b19e47a2%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637454640700572684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=R4HvrBc9VqNIit2GjqgfbStwO0xsyUK9BtA4CJ72uFc%3D&reserved=0
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Reporter: Francesco Nigro
> Assignee: Francesco Nigro
> Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Durable CORE messages can get their internal buffer enlarged by encodeHeadersAndProperties while being persisted on the journal, but the address size memory estimation using the estimated memory of a message is performed before that, making it less precise.
> This bad timing estimation, together with Netty ByteBuf auto-sizing mechanism can cause the broker to underestimate the message footprint, causing it to go OOM.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [jira] [Work logged] (ARTEMIS-3021) OOM due to wrong CORE message
memory estimation
Posted by Gavin McDonald <gm...@apache.org>.
I have unsubscribed you from the activemq issues mailing list
HTH
On Tue, Jan 5, 2021 at 6:59 PM Patil, Pratibha <pr...@siemens.com>
wrote:
> Please delete my account. I am getting too much mails everyday.
>
> I Am not getting option in use profile.
>
> Username - pratibha.patil@siemens.com
>
> -----Original Message-----
> From: ASF GitHub Bot (Jira) <ji...@apache.org>
> Sent: Tuesday, January 5, 2021 10:51 PM
> To: issues@activemq.apache.org
> Subject: [jira] [Work logged] (ARTEMIS-3021) OOM due to wrong CORE message
> memory estimation
>
>
> [
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FARTEMIS-3021%3FfocusedWorklogId%3D531359%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Aworklog-tabpanel%23worklog-531359&data=04%7C01%7Cpratibha.patil%40siemens.com%7C70c604a6390c4dd8d80b08d8b19e47a2%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637454640700572684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ntHKSO8ulEITS1W847MMvNwhHLlH1tRz0dswo4TpDtQ%3D&reserved=0
> ]
>
> ASF GitHub Bot logged work on ARTEMIS-3021:
> -------------------------------------------
>
> Author: ASF GitHub Bot
> Created on: 05/Jan/21 17:20
> Start Date: 05/Jan/21 17:20
> Worklog Time Spent: 10m
> Work Description: franz1981 commented on a change in pull request
> #3370:
> URL:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Factivemq-artemis%2Fpull%2F3370%23discussion_r552075767&data=04%7C01%7Cpratibha.patil%40siemens.com%7C70c604a6390c4dd8d80b08d8b19e47a2%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637454640700572684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=xC4amBCRyzCBX9hfAEnLnYlF88dBNIFYqBHsw0W33es%3D&reserved=0
>
>
>
> ##########
> File path:
> artemis-core-client/src/main/java/org/apache/activemq/artemis/core/message/impl/CoreMessage.java
> ##########
> @@ -622,6 +619,12 @@ private RuntimeException
> onCheckPropertiesError(Throwable e) {
> @Override
> public int getMemoryEstimate() {
> if (memoryEstimate == -1) {
> + // durable messages can enlarge buffer::capacity while persisted:
> + // better to encode it to properly account its max capacity
> + if (isDurable()) {
> + checkEncode();
>
> Review comment:
> That means that a durable message would get encoded some time
> before then expected: @clebertsuconic it's something we can tolerate?
> if not the ideal thing would be to estimate it based on its state, but
> I'm not sure is feasable.
>
>
>
>
> ----------------------------------------------------------------
> This is an automated message from the Apache Git Service.
> To respond to the message, please log on to GitHub and use the URL above
> to go to the specific comment.
>
> For queries about this service, please contact Infrastructure at:
> users@infra.apache.org
>
>
> Issue Time Tracking
> -------------------
>
> Worklog Id: (was: 531359)
> Time Spent: 1h (was: 50m)
>
> > OOM due to wrong CORE message memory estimation
> > -----------------------------------------------
> >
> > Key: ARTEMIS-3021
> > URL:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FARTEMIS-3021&data=04%7C01%7Cpratibha.patil%40siemens.com%7C70c604a6390c4dd8d80b08d8b19e47a2%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637454640700572684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=R4HvrBc9VqNIit2GjqgfbStwO0xsyUK9BtA4CJ72uFc%3D&reserved=0
> > Project: ActiveMQ Artemis
> > Issue Type: Bug
> > Reporter: Francesco Nigro
> > Assignee: Francesco Nigro
> > Priority: Major
> > Time Spent: 1h
> > Remaining Estimate: 0h
> >
> > Durable CORE messages can get their internal buffer enlarged by
> encodeHeadersAndProperties while being persisted on the journal, but the
> address size memory estimation using the estimated memory of a message is
> performed before that, making it less precise.
> > This bad timing estimation, together with Netty ByteBuf auto-sizing
> mechanism can cause the broker to underestimate the message footprint,
> causing it to go OOM.
>
>
>
> --
> This message was sent by Atlassian Jira
> (v8.3.4#803005)
>
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [jira] [Work logged] (ARTEMIS-3021) OOM due to wrong CORE message
memory estimation
Posted by Justin Bertram <jb...@apache.org>.
If you want to unsubscribe from dev@activemq.apache.org then please send an
email to dev-unsubscribe@activemq.apache.org. The automated mailing list
manager will send an email back to you confirming you want to unsubscribe.
Once you respond to that confirmation you should be successfully
unsubscribed from dev@activemq.apache.org.
I recently worked with another user who claimed that he couldn't
unsubscribe, but he simply didn't follow the process I just outlined. Once
he followed the process he was unsubscribed successfully.
Justin
On Tue, Jan 5, 2021 at 12:33 PM sagar bamashetti <sa...@gmail.com>
wrote:
> Hi same here. Please delete my account too. I tried removing my self from
> list, but it's not working.
>
> user:. sagarmb55@gmail.com
>
> Thanks,
> Sagar
>
> On Tue, Jan 5, 2021, 11:44 PM Patil, Pratibha <pr...@siemens.com>
> wrote:
>
> > Please delete my account. I am getting too much mails everyday.
> >
> > I Am not getting option in use profile.
> >
> > Username - pratibha.patil@siemens.com
> >
> > -----Original Message-----
> > From: ASF GitHub Bot (Jira) <ji...@apache.org>
> > Sent: Tuesday, January 5, 2021 10:51 PM
> > To: issues@activemq.apache.org
> > Subject: [jira] [Work logged] (ARTEMIS-3021) OOM due to wrong CORE
> message
> > memory estimation
> >
> >
> > [
> >
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FARTEMIS-3021%3FfocusedWorklogId%3D531359%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Aworklog-tabpanel%23worklog-531359&data=04%7C01%7Cpratibha.patil%40siemens.com%7C70c604a6390c4dd8d80b08d8b19e47a2%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637454640700572684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ntHKSO8ulEITS1W847MMvNwhHLlH1tRz0dswo4TpDtQ%3D&reserved=0
> > ]
> >
> > ASF GitHub Bot logged work on ARTEMIS-3021:
> > -------------------------------------------
> >
> > Author: ASF GitHub Bot
> > Created on: 05/Jan/21 17:20
> > Start Date: 05/Jan/21 17:20
> > Worklog Time Spent: 10m
> > Work Description: franz1981 commented on a change in pull request
> > #3370:
> > URL:
> >
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Factivemq-artemis%2Fpull%2F3370%23discussion_r552075767&data=04%7C01%7Cpratibha.patil%40siemens.com%7C70c604a6390c4dd8d80b08d8b19e47a2%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637454640700572684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=xC4amBCRyzCBX9hfAEnLnYlF88dBNIFYqBHsw0W33es%3D&reserved=0
> >
> >
> >
> > ##########
> > File path:
> >
> artemis-core-client/src/main/java/org/apache/activemq/artemis/core/message/impl/CoreMessage.java
> > ##########
> > @@ -622,6 +619,12 @@ private RuntimeException
> > onCheckPropertiesError(Throwable e) {
> > @Override
> > public int getMemoryEstimate() {
> > if (memoryEstimate == -1) {
> > + // durable messages can enlarge buffer::capacity while
> persisted:
> > + // better to encode it to properly account its max capacity
> > + if (isDurable()) {
> > + checkEncode();
> >
> > Review comment:
> > That means that a durable message would get encoded some time
> > before then expected: @clebertsuconic it's something we can tolerate?
> > if not the ideal thing would be to estimate it based on its state, but
> > I'm not sure is feasable.
> >
> >
> >
> >
> > ----------------------------------------------------------------
> > This is an automated message from the Apache Git Service.
> > To respond to the message, please log on to GitHub and use the URL above
> > to go to the specific comment.
> >
> > For queries about this service, please contact Infrastructure at:
> > users@infra.apache.org
> >
> >
> > Issue Time Tracking
> > -------------------
> >
> > Worklog Id: (was: 531359)
> > Time Spent: 1h (was: 50m)
> >
> > > OOM due to wrong CORE message memory estimation
> > > -----------------------------------------------
> > >
> > > Key: ARTEMIS-3021
> > > URL:
> >
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FARTEMIS-3021&data=04%7C01%7Cpratibha.patil%40siemens.com%7C70c604a6390c4dd8d80b08d8b19e47a2%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637454640700572684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=R4HvrBc9VqNIit2GjqgfbStwO0xsyUK9BtA4CJ72uFc%3D&reserved=0
> > > Project: ActiveMQ Artemis
> > > Issue Type: Bug
> > > Reporter: Francesco Nigro
> > > Assignee: Francesco Nigro
> > > Priority: Major
> > > Time Spent: 1h
> > > Remaining Estimate: 0h
> > >
> > > Durable CORE messages can get their internal buffer enlarged by
> > encodeHeadersAndProperties while being persisted on the journal, but the
> > address size memory estimation using the estimated memory of a message is
> > performed before that, making it less precise.
> > > This bad timing estimation, together with Netty ByteBuf auto-sizing
> > mechanism can cause the broker to underestimate the message footprint,
> > causing it to go OOM.
> >
> >
> >
> > --
> > This message was sent by Atlassian Jira
> > (v8.3.4#803005)
> >
>