You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "Justin Bertram (Jira)" <ji...@apache.org> on 2021/06/21 19:19:00 UTC
[jira] [Commented] (ARTEMIS-3358) AMQ221055: There were too many
old replicated folders upon startup,
[ https://issues.apache.org/jira/browse/ARTEMIS-3358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17366800#comment-17366800 ]
Justin Bertram commented on ARTEMIS-3358:
-----------------------------------------
bq. When ever our brokers upon restart we observed weird behavior as that backlog messages added to expiry queue. let's say Before broker restart we have 50 messages in expiry queue after restart broker it got pilled up with 12000 messages.
This isn't necessarily abnormal. The expiration time of a message can elapse while the broker is stopped. When the broker is started these messages will be detected and sent to the expiry address.
bq. AMQ221055: There were too many old replicated folders upon startup...
This also isn't abnormal. The a replicated master broker keeps journal backups in case of problems, but it only keeps so many (configured with {{max-saved-replicated-journals-size}}). Once the limit is reached it starts deleting the oldest backups.
bq. ... INFO [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841013: expired message...
This also is not abnormal. This is coming from the LoggingActiveMQServerPlugin which ostensibly you have configured. It simply means that a message has expired.
> AMQ221055: There were too many old replicated folders upon startup,
> --------------------------------------------------------------------
>
> Key: ARTEMIS-3358
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3358
> Project: ActiveMQ Artemis
> Issue Type: Task
> Components: AMQP
> Affects Versions: 2.17.0
> Reporter: Jetti
> Priority: Major
> Fix For: 2.17.0
>
>
> We have following below Architecture. We need your help to find out the solution for the subjected issue.
> {noformat}
> MicroServices<-->Qpid <--> Artemis(Master/Slave Pair with Replicated)
> {noformat}
>
> When ever our brokers upon restart we observed weird behavior as that backlog messages added to expiry queue. let's say Before broker restart we have 50 messages in expiry queue after restart broker it got pilled up with 12000 messages.
> Below was the error log:
> {noformat}
> AMQ221055: There were too many old replicated folders upon startup, removing /app/apache/amq/brokerdata/activemq-prod-master1a/paging/oldreplica.7
> {noformat}
>
> And also we could see the below error as well
> {noformat}
> 2021-06-19 02:30:05,910 INFO [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841013: expired message: Reference[23726540863]:RELIABLE:AMQPStandardMessage( [durable=true, messageID=23726540, address=abc.inbound.error.unicast, size=1585, applicationProperties=\{X_HYPHEN_Forwarded_HYPHEN_Port=7000, CamelJmsDeliveryMode=2, X_HYPHEN_Forwarded_HYPHEN_For=205.157.103.242, HEADER_REQUEST_CARRIER=FEDEX, newrelic=eyJkIjp7ImFjIjoiMjY4Nzg1NSIsInByIjoxLjUzNjU0OCwidHgiOiJiMmZiMjIzNTkyYWEwOGY1IiwidGkiOjE2MjEwNTg2NDM0NTcsInR5IjoiQXBwIiwidGsiOiIyNDU3NTY1IiwiaWQiOiIxMTJkN2Q3NGUxZDgwNDU0IiwidHIiOiJiZGNkMmMyZmQzNzhjOTAxYWE2Yjg2YTc2NDI3OGExMiIsInNhIjp0cnVlLCJhcCI6Ijg0MTYxNzM2MyJ9LCJ2IjpbMCwxXX0=, Authorization=Basic c3ZjLXdpc21vLWIyYjpzdmN3aXNtb2IyYjRwcmQ=, Content_HYPHEN_Length=2653, X_HYPHEN_Original_HYPHEN_Forwarded_HYPHEN_For=205.157.103.242, 10.94.84.21, X_HYPHEN_Request_HYPHEN_ID=e9efafc3df0e6c8883d5f3c07803cff2, tracestate=2457565@nr=0-0-2687855-841617363-112d7d74e1d80454-b2fb223592aa08f5-1-1.536548-1621058643457,, Content_HYPHEN_Type=text/plain, CamelHttpMethod=POST, CamelServletContextPath=/fedexEventTracking, HEADER_REQUEST_SOURCE=FEDEX, CamelHttpUrl=http://api.officedepot.io:7000/eaiapi/fedexEventTracking, CamelHttpCharacterEncoding=UTF-8, User_HYPHEN_Agent=Mozilla/4.0 [en] (WinNT; I), X_HYPHEN_Forwarded_HYPHEN_Proto=http, Host=api.officedepot.io:7000, X_HYPHEN_Real_HYPHEN_IP=205.157.103.242, X_HYPHEN_Scheme=http, traceparent=00-bdcd2c2fd378c901aa6b86a764278a12-112d7d74e1d80454-01, CamelHttpUri=/eaiapi/fedexEventTracking, X_HYPHEN_Forwarded_HYPHEN_Host=api.officeio:7000, CamelHttpPath=}, properties=Properties\{messageId=ID:85686cbb-c5e6-4782-a55c-4661f0f106e3:22009:1:1-2, userId=null, to='abc.tracking.inbound.error.unicast', subject='null', replyTo='null', correlationId=null, contentType=null, contentEncoding=null, absoluteExpiryTime=null, creationTime=Sat May 15 02:04:03 EDT 2021, groupId='null', groupSequence=null, replyToGroupId='null'}, extraProperties = null], messageExpiryAddress: ExpiryQueue
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)