You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Roskvist Anton <an...@volvo.com> on 2021/04/30 09:39:39 UTC
Seeing some AIO issues in artemis-2.16.0 and artemis-2.17.0
Hi,
I have a problem with Artemis that I'd like some insight on... starting with artemis-2.16.0 I was getting these WARN messages, followed by a broker shutdown:
WARN [org.apache.activemq.artemis.core.server] AMQ222010: Critical IO Error, shutting down the server. file=AIOSequentialFile:/path/to/artemis-2.16.0/data/journal/activemq-data-2665706.amq, message=Timeout on close: java.io.IOException: Timeout on close
They were not that frequent, maybe once per week or so. I read that this was a known issue that was supposed to be resolved in artemis-2.17.0.
After upgrading I see this WARN message instead, followed by a thread dump:
WARN [org.apache.activemq.artemis.journal] File activemq-data-49.amq still has pending IO before closing it
This does not appear to cause any issues but it is printed several times a day so the log is completely cluttered (from the thread dump).Do I have some issue in my setup or need to worry about loosing data, or is the aforementioned bug not quite fixed in artemis-2.17.0 but safe? Should I write a problem ticket for it?
Br,
Anton
This email message (including its attachments) is confidential and may contain privileged information and is intended solely for the use of the individual and/or entity to whom it is addressed. If you are not the intended recipient of this e-mail you may not disseminate, distribute or copy this e-mail (including its attachments), or any part thereof. If this e-mail is received in error, please notify the sender immediately by return e-mail and make sure that this e-mail (including its attachments), and all copies thereof, are immediately deleted from your system. Please further note that when you communicate with us via email or visit our website we process your personal data. See our privacy policy for more information about how we process it: https://www.volvogroup.com/en-en/privacy.html
RE: Seeing some AIO issues in artemis-2.16.0 and artemis-2.17.0
Posted by Roskvist Anton <an...@volvo.com>.
Okay gtully, thanks for your input! I have created this issue:
https://issues.apache.org/jira/browse/ARTEMIS-3272
Let me know if I can provide any more information or should try something out.
-----Original Message-----
From: Gary Tully <ga...@gmail.com>
Sent: den 30 april 2021 13:46
To: users@activemq.apache.org
Subject: Re: Seeing some AIO issues in artemis-2.16.0 and artemis-2.17.0
CAUTION: This email originated from outside of the organization.
From peeking at the fix: we wait 60 seconds, warn and give up if a close seems to be stalled/pending. However the stack trace will be for the waiter, rather for the initial close call. before the change the broker would exit.
please open an issue with the stack trace, there is no point printing that out many times. But it does suggest some problem, either in the logic that tracks duplicate close or in the AOI subsystem that blocks on close.
from a broker perspective, if this is intermittent behaviour from the OS that we don't expect then we issue a warn, but there is little point flooding the logs. Doing it once or once a day may be sufficient.
On Fri, 30 Apr 2021 at 10:39, Roskvist Anton <an...@volvo.com> wrote:
>
> Hi,
>
> I have a problem with Artemis that I'd like some insight on... starting with artemis-2.16.0 I was getting these WARN messages, followed by a broker shutdown:
> WARN [org.apache.activemq.artemis.core.server] AMQ222010: Critical IO
> Error, shutting down the server.
> file=AIOSequentialFile:/path/to/artemis-2.16.0/data/journal/activemq-d
> ata-2665706.amq, message=Timeout on close: java.io.IOException:
> Timeout on close
>
> They were not that frequent, maybe once per week or so. I read that this was a known issue that was supposed to be resolved in artemis-2.17.0.
>
> After upgrading I see this WARN message instead, followed by a thread dump:
> WARN [org.apache.activemq.artemis.journal] File activemq-data-49.amq
> still has pending IO before closing it
>
> This does not appear to cause any issues but it is printed several times a day so the log is completely cluttered (from the thread dump).Do I have some issue in my setup or need to worry about loosing data, or is the aforementioned bug not quite fixed in artemis-2.17.0 but safe? Should I write a problem ticket for it?
>
> Br,
> Anton
>
> This email message (including its attachments) is confidential and may
> contain privileged information and is intended solely for the use of
> the individual and/or entity to whom it is addressed. If you are not
> the intended recipient of this e-mail you may not disseminate,
> distribute or copy this e-mail (including its attachments), or any
> part thereof. If this e-mail is received in error, please notify the
> sender immediately by return e-mail and make sure that this e-mail
> (including its attachments), and all copies thereof, are immediately
> deleted from your system. Please further note that when you
> communicate with us via email or visit our website we process your
> personal data. See our privacy policy for more information about how
> we process it: https://www.volvogroup.com/en-en/privacy.html
This email message (including its attachments) is confidential and may contain privileged information and is intended solely for the use of the individual and/or entity to whom it is addressed. If you are not the intended recipient of this e-mail you may not disseminate, distribute or copy this e-mail (including its attachments), or any part thereof. If this e-mail is received in error, please notify the sender immediately by return e-mail and make sure that this e-mail (including its attachments), and all copies thereof, are immediately deleted from your system. Please further note that when you communicate with us via email or visit our website we process your personal data. See our privacy policy for more information about how we process it: https://www.volvogroup.com/en-en/privacy.html
Re: Seeing some AIO issues in artemis-2.16.0 and artemis-2.17.0
Posted by Gary Tully <ga...@gmail.com>.
From peeking at the fix: we wait 60 seconds, warn and give up if a
close seems to be stalled/pending. However the stack trace will be for
the waiter, rather for the initial close call. before the change the
broker would exit.
please open an issue with the stack trace, there is no point printing
that out many times. But it does suggest some problem, either in the
logic that tracks duplicate close or in the AOI subsystem that blocks
on close.
from a broker perspective, if this is intermittent behaviour from the
OS that we don't expect then we issue a warn, but there is little
point flooding the logs. Doing it once or once a day may be
sufficient.
On Fri, 30 Apr 2021 at 10:39, Roskvist Anton <an...@volvo.com> wrote:
>
> Hi,
>
> I have a problem with Artemis that I'd like some insight on... starting with artemis-2.16.0 I was getting these WARN messages, followed by a broker shutdown:
> WARN [org.apache.activemq.artemis.core.server] AMQ222010: Critical IO Error, shutting down the server. file=AIOSequentialFile:/path/to/artemis-2.16.0/data/journal/activemq-data-2665706.amq, message=Timeout on close: java.io.IOException: Timeout on close
>
> They were not that frequent, maybe once per week or so. I read that this was a known issue that was supposed to be resolved in artemis-2.17.0.
>
> After upgrading I see this WARN message instead, followed by a thread dump:
> WARN [org.apache.activemq.artemis.journal] File activemq-data-49.amq still has pending IO before closing it
>
> This does not appear to cause any issues but it is printed several times a day so the log is completely cluttered (from the thread dump).Do I have some issue in my setup or need to worry about loosing data, or is the aforementioned bug not quite fixed in artemis-2.17.0 but safe? Should I write a problem ticket for it?
>
> Br,
> Anton
>
> This email message (including its attachments) is confidential and may contain privileged information and is intended solely for the use of the individual and/or entity to whom it is addressed. If you are not the intended recipient of this e-mail you may not disseminate, distribute or copy this e-mail (including its attachments), or any part thereof. If this e-mail is received in error, please notify the sender immediately by return e-mail and make sure that this e-mail (including its attachments), and all copies thereof, are immediately deleted from your system. Please further note that when you communicate with us via email or visit our website we process your personal data. See our privacy policy for more information about how we process it: https://www.volvogroup.com/en-en/privacy.html