Artemis clusters and large messages

["Dondorp, Erwin" <er...@cgi.com>]

Hello,

I'm testing the handling of large messages in an Artemis cluster, while using CORE protocol everywhere.
My messages are up to 1Mb, but I want to keep them in memory, not as large messages, so I'm using the option 'minLargeMessageSize=1000000' in the client connection-url.
And also in the callback connector that is used to connect the cluster members. So that connector is now "tcp://hostname:61616?minLargeMessageSize=1000000".
When not using a cluster, the clients behave well and the message behaviour is as expected, only beyond that size limit will they go through the "large-messages" directory.

But when using a cluster, mayhem occurs, as just after startup the following error occurs on all cluster nodes:
ERROR [org.apache.activemq.artemis.core.client] AMQ214013: Failed to decode packet: java.lang.IndexOutOfBoundsException: Error reading in simpleString, length=1850499442 is greater than readableBytes=34
Note that 1850499442 (dec) = 6E4C 6172 (hex) = "nLar" (txt), which are 4 characters that also appear in the relevant parameter name "minLargeMessageSize".

In other scenario's, I've used other url-parameter on that url (for ssl) without any problems.
But nevertheless, I'm still in doubt whether this is a bug, or just me using the broker in the wrong way...

Any hints/tips?

Thx,
Erwin

RE: Artemis clusters and large messages

["Dondorp, Erwin" <er...@cgi.com>]

Justin,

> FWIW, the cluster-connection does have its own min-large-message-size configuration element.
It is actually worth a lot. I did not look at these details because the URL parameter already seemed like the correct choice.

> Do you still see this problem when you use that?
No, there is no problem visible when min-large-message-size is set.
I still have to do my end-to-end testing but this looks very promising.

> In any event, this does seem like a bug. Please open a Jira with a minimal reproducible example.
I'll do that.

> Thanks!
Thank you very much for the very adequate answer!

Erwin

-----Oorspronkelijk bericht-----
Van: Justin Bertram <jb...@apache.org> 
Verzonden: maandag 21 juni 2021 21:24
Aan: users@activemq.apache.org
Onderwerp: Re: Artemis clusters and large messages


EXTERNAL SENDER:   Do not click any links or open any attachments unless you trust the sender and know the content is safe.
EXPÉDITEUR EXTERNE:    Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe à moins qu’ils ne proviennent d’un expéditeur fiable, ou que vous ayez l'assurance que le contenu provient d'une source sûre.

FWIW, the cluster-connection does have its own min-large-message-size configuration element. Do you still see this problem when you use that?

In any event, this does seem like a bug. Please open a Jira with a minimal reproducible example. Thanks!


Justin

On Wed, Jun 16, 2021 at 7:46 PM Dondorp, Erwin <er...@cgi.com>
wrote:

> Hello,
>
> I'm testing the handling of large messages in an Artemis cluster, 
> while using CORE protocol everywhere.
> My messages are up to 1Mb, but I want to keep them in memory, not as 
> large messages, so I'm using the option 'minLargeMessageSize=1000000' 
> in the client connection-url.
> And also in the callback connector that is used to connect the cluster 
> members. So that connector is now 
> "tcp://hostname:61616?minLargeMessageSize=1000000".
> When not using a cluster, the clients behave well and the message 
> behaviour is as expected, only beyond that size limit will they go 
> through the "large-messages" directory.
>
> But when using a cluster, mayhem occurs, as just after startup the 
> following error occurs on all cluster nodes:
> ERROR [org.apache.activemq.artemis.core.client] AMQ214013: Failed to 
> decode packet: java.lang.IndexOutOfBoundsException: Error reading in 
> simpleString, length=1850499442 is greater than readableBytes=34 Note 
> that 1850499442 (dec) = 6E4C 6172 (hex) = "nLar" (txt), which are 4 
> characters that also appear in the relevant parameter name 
> "minLargeMessageSize".
>
> In other scenario's, I've used other url-parameter on that url (for 
> ssl) without any problems.
> But nevertheless, I'm still in doubt whether this is a bug, or just me 
> using the broker in the wrong way...
>
> Any hints/tips?
>
> Thx,
> Erwin
>

Re: Artemis clusters and large messages

[Justin Bertram <jb...@apache.org>]

FWIW, the cluster-connection does have its own min-large-message-size
configuration element. Do you still see this problem when you use that?

In any event, this does seem like a bug. Please open a Jira with a minimal
reproducible example. Thanks!


Justin

On Wed, Jun 16, 2021 at 7:46 PM Dondorp, Erwin <er...@cgi.com>
wrote:

> Hello,
>
> I'm testing the handling of large messages in an Artemis cluster, while
> using CORE protocol everywhere.
> My messages are up to 1Mb, but I want to keep them in memory, not as large
> messages, so I'm using the option 'minLargeMessageSize=1000000' in the
> client connection-url.
> And also in the callback connector that is used to connect the cluster
> members. So that connector is now
> "tcp://hostname:61616?minLargeMessageSize=1000000".
> When not using a cluster, the clients behave well and the message
> behaviour is as expected, only beyond that size limit will they go through
> the "large-messages" directory.
>
> But when using a cluster, mayhem occurs, as just after startup the
> following error occurs on all cluster nodes:
> ERROR [org.apache.activemq.artemis.core.client] AMQ214013: Failed to
> decode packet: java.lang.IndexOutOfBoundsException: Error reading in
> simpleString, length=1850499442 is greater than readableBytes=34
> Note that 1850499442 (dec) = 6E4C 6172 (hex) = "nLar" (txt), which are 4
> characters that also appear in the relevant parameter name
> "minLargeMessageSize".
>
> In other scenario's, I've used other url-parameter on that url (for ssl)
> without any problems.
> But nevertheless, I'm still in doubt whether this is a bug, or just me
> using the broker in the wrong way...
>
> Any hints/tips?
>
> Thx,
> Erwin
>