You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Bindu Wavell <bw...@google.com.INVALID> on 2022/03/01 21:54:57 UTC
Re: configuring failover and prefetch via URI
I'm wondering if anyone has advice on how to configure a Spring Boot
application that uses DefaultJmsListenerContainerFactory() to set prefetch
to 1. I am not able to change the code, only configure the broker URL. I
can place config into the application.yml, but I suspect that will only be
helpful if the person who wrote the app specifically exposed configuration
which I don't see... So I guess if the DefaultJmsListener logic registers
some config, there might be something we could set?
Thanks in advance,
-- Bindu
On Fri, Feb 25, 2022 at 4:29 PM Bindu Wavell <bw...@google.com> wrote:
> Hi Tim,
>
> Thank you for your reply and the reference to the SO article. I had not
> found that one.
>
> I had not tried just using a single tcp:// connection, I just did (
> tcp://broker1:61616?jms.prefetchPolicy.queuePrefetch=1) and the client
> fails to connect with the following WARNing logged:
>
> 2022-02-25 22:42:52.727 WARN 1 --- [ntContainer#0-1]
> o.s.j.l.DefaultMessageListenerContainer : Setup of JMS message listener
> invoker failed for destination 'xx.yy.zz' - trying to recover. Cause: Could
> not create JMS transaction; nested exception is javax.jms.JMSException:
> Could not create Transport. Reason: java.lang.IllegalArgumentException:
> Invalid connect parameters: {jms.prefetchPolicy.queuePrefetch=1}
>
> I guess because this is an ActiveMQ specific argument and the vendor code
> is almost completely pure JMS without any ActiveMQ specific
> classes/factories, etc?
>
>
>
> On Fri, Feb 25, 2022 at 6:35 AM Tim Bain <tb...@alumni.duke.edu> wrote:
>
>> Your first and third URIs should work, so maybe this is something specific
>> to DefaultJmsListenerContainerFactory()? I don't have experience using it,
>> but an answer on
>>
>> https://stackoverflow.com/questions/33339224/spring-jms-listener-container-concurrency-attribute-not-working
>> lists 3 ways to set prefetch under Spring Boot, including via
>> application.properties which looks like what you're doing. Are either of
>> the other two possible without the vendor change you're waiting for?
>>
>> Does it work if you try tcp to the active broker (no failover), just to
>> test?
>>
>> Tim
>>
>> On Thu, Feb 24, 2022, 11:03 AM Bindu Wavell <bw...@google.com.invalid>
>> wrote:
>>
>> > Hi folks,
>> >
>> > We are using ActiveMQ Classic (5.16.1). I have 3 consumer processes and
>> 2
>> > ActiveMQ brokers setup with the shared storage active/passive mode.
>> >
>> > I setup the connection URI like this:
>> >
>> >
>> >
>> failover:(tcp://broker1:61616,tcp://broker2:61616)?timeout=3000&nested.jms.prefetchPolicy.queuePrefetch=1
>> >
>> > Also tried this:
>> >
>> >
>> >
>> failover:(tcp://broker1:61616,tcp://broker2:61616)?timeout=3000&jms.prefetchPolicy.queuePrefetch=1
>> >
>> > And this:
>> >
>> >
>> >
>> failover:(tcp://broker1:61616?jms.prefetchPolicy.queuePrefetch=1,tcp://broker2:61616?jms.prefetchPolicy.queuePrefetch=1)?timeout=3000
>> >
>> > Looking through the web console at active consumers for a queue I care
>> > about I see:
>> >
>> > Prefetch
>> > Max Pending
>> > 1000
>> > 0
>> >
>> > For each of the consumers. I'm also seeing Enqueues > 10. We have an
>> > application where there are not a ton of messages, but the consumers
>> take a
>> > while to process messages. In the current environment, when we start our
>> > clients, we see most messages going to one client. Over time this does
>> seem
>> > to even out, but I think it's indicative of prefetch not being applied.
>> >
>> > Am I doing this right?
>> >
>> > FWIW, the client is a Spring Boot application built by a vendor and I'm
>> > trying to get the vendor to actually allow us to configure prefetch and
>> > they will set it from code. That is taking a long time, so I was
>> excited it
>> > would be possible to set this via the connection URI.
>> >
>> > The vendor sets up the jmsListenerConnectionFactory in
>> > their JmsListenerConfigurer implementation like this:
>> >
>> >
>> > @Bean
>> > public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(
>> > final ConnectionFactory connectionFactory,
>> > final TransformMessageConverter transformMessageConverter)
>> > {
>> > final DefaultJmsListenerContainerFactory factory = new
>> > DefaultJmsListenerContainerFactory();
>> > factory.setConnectionFactory(connectionFactory);
>> > factory.setMessageConverter(transformMessageConverter);
>> > factory.setErrorHandler(t -> logger.error("JMS error: " +
>> t.getMessage(),
>> > t));
>> > factory.setTransactionManager(transactionManager(connectionFactory));
>> > return factory;
>> >
>> >
>> > }
>> >
>>
>
Re: configuring failover and prefetch via URI
Posted by Tim Bain <tb...@alumni.duke.edu>.
Thanks for reporting back with that info, and I'm glad you figured it out.
Tim
On Fri, Mar 4, 2022, 4:25 PM Bindu Wavell <bw...@google.com.invalid>
wrote:
> Reporting back, the magic incantation was to pass a JVM arg `-
>
> Dspring.activemq.broker-url=tcp://broker1:61616?jms.prefetchPolicy.queuePrefetch=1`
> in addition to the application specific application.yml configuration we
> were setting before.
>
> On Tue, Mar 1, 2022 at 2:54 PM Bindu Wavell <bw...@google.com> wrote:
>
> > I'm wondering if anyone has advice on how to configure a Spring Boot
> > application that uses DefaultJmsListenerContainerFactory() to set
> prefetch
> > to 1. I am not able to change the code, only configure the broker URL. I
> > can place config into the application.yml, but I suspect that will only
> be
> > helpful if the person who wrote the app specifically exposed
> configuration
> > which I don't see... So I guess if the DefaultJmsListener logic registers
> > some config, there might be something we could set?
> >
> > Thanks in advance,
> > -- Bindu
> >
> > On Fri, Feb 25, 2022 at 4:29 PM Bindu Wavell <bw...@google.com> wrote:
> >
> >> Hi Tim,
> >>
> >> Thank you for your reply and the reference to the SO article. I had not
> >> found that one.
> >>
> >> I had not tried just using a single tcp:// connection, I just did (
> >> tcp://broker1:61616?jms.prefetchPolicy.queuePrefetch=1) and the client
> >> fails to connect with the following WARNing logged:
> >>
> >> 2022-02-25 22:42:52.727 WARN 1 --- [ntContainer#0-1]
> >> o.s.j.l.DefaultMessageListenerContainer : Setup of JMS message listener
> >> invoker failed for destination 'xx.yy.zz' - trying to recover. Cause:
> Could
> >> not create JMS transaction; nested exception is javax.jms.JMSException:
> >> Could not create Transport. Reason: java.lang.IllegalArgumentException:
> >> Invalid connect parameters: {jms.prefetchPolicy.queuePrefetch=1}
> >>
> >> I guess because this is an ActiveMQ specific argument and the vendor
> code
> >> is almost completely pure JMS without any ActiveMQ specific
> >> classes/factories, etc?
> >>
> >>
> >>
> >> On Fri, Feb 25, 2022 at 6:35 AM Tim Bain <tb...@alumni.duke.edu> wrote:
> >>
> >>> Your first and third URIs should work, so maybe this is something
> >>> specific
> >>> to DefaultJmsListenerContainerFactory()? I don't have experience using
> >>> it,
> >>> but an answer on
> >>>
> >>>
> https://stackoverflow.com/questions/33339224/spring-jms-listener-container-concurrency-attribute-not-working
> >>> lists 3 ways to set prefetch under Spring Boot, including via
> >>> application.properties which looks like what you're doing. Are either
> of
> >>> the other two possible without the vendor change you're waiting for?
> >>>
> >>> Does it work if you try tcp to the active broker (no failover), just to
> >>> test?
> >>>
> >>> Tim
> >>>
> >>> On Thu, Feb 24, 2022, 11:03 AM Bindu Wavell <bwavell@google.com.invalid
> >
> >>> wrote:
> >>>
> >>> > Hi folks,
> >>> >
> >>> > We are using ActiveMQ Classic (5.16.1). I have 3 consumer processes
> >>> and 2
> >>> > ActiveMQ brokers setup with the shared storage active/passive mode.
> >>> >
> >>> > I setup the connection URI like this:
> >>> >
> >>> >
> >>> >
> >>>
> failover:(tcp://broker1:61616,tcp://broker2:61616)?timeout=3000&nested.jms.prefetchPolicy.queuePrefetch=1
> >>> >
> >>> > Also tried this:
> >>> >
> >>> >
> >>> >
> >>>
> failover:(tcp://broker1:61616,tcp://broker2:61616)?timeout=3000&jms.prefetchPolicy.queuePrefetch=1
> >>> >
> >>> > And this:
> >>> >
> >>> >
> >>> >
> >>>
> failover:(tcp://broker1:61616?jms.prefetchPolicy.queuePrefetch=1,tcp://broker2:61616?jms.prefetchPolicy.queuePrefetch=1)?timeout=3000
> >>> >
> >>> > Looking through the web console at active consumers for a queue I
> care
> >>> > about I see:
> >>> >
> >>> > Prefetch
> >>> > Max Pending
> >>> > 1000
> >>> > 0
> >>> >
> >>> > For each of the consumers. I'm also seeing Enqueues > 10. We have an
> >>> > application where there are not a ton of messages, but the consumers
> >>> take a
> >>> > while to process messages. In the current environment, when we start
> >>> our
> >>> > clients, we see most messages going to one client. Over time this
> does
> >>> seem
> >>> > to even out, but I think it's indicative of prefetch not being
> applied.
> >>> >
> >>> > Am I doing this right?
> >>> >
> >>> > FWIW, the client is a Spring Boot application built by a vendor and
> I'm
> >>> > trying to get the vendor to actually allow us to configure prefetch
> and
> >>> > they will set it from code. That is taking a long time, so I was
> >>> excited it
> >>> > would be possible to set this via the connection URI.
> >>> >
> >>> > The vendor sets up the jmsListenerConnectionFactory in
> >>> > their JmsListenerConfigurer implementation like this:
> >>> >
> >>> >
> >>> > @Bean
> >>> > public DefaultJmsListenerContainerFactory
> jmsListenerContainerFactory(
> >>> > final ConnectionFactory connectionFactory,
> >>> > final TransformMessageConverter transformMessageConverter)
> >>> > {
> >>> > final DefaultJmsListenerContainerFactory factory = new
> >>> > DefaultJmsListenerContainerFactory();
> >>> > factory.setConnectionFactory(connectionFactory);
> >>> > factory.setMessageConverter(transformMessageConverter);
> >>> > factory.setErrorHandler(t -> logger.error("JMS error: " +
> >>> t.getMessage(),
> >>> > t));
> >>> > factory.setTransactionManager(transactionManager(connectionFactory));
> >>> > return factory;
> >>> >
> >>> >
> >>> > }
> >>> >
> >>>
> >>
>
Re: configuring failover and prefetch via URI
Posted by Bindu Wavell <bw...@google.com.INVALID>.
Reporting back, the magic incantation was to pass a JVM arg `-
Dspring.activemq.broker-url=tcp://broker1:61616?jms.prefetchPolicy.queuePrefetch=1`
in addition to the application specific application.yml configuration we
were setting before.
On Tue, Mar 1, 2022 at 2:54 PM Bindu Wavell <bw...@google.com> wrote:
> I'm wondering if anyone has advice on how to configure a Spring Boot
> application that uses DefaultJmsListenerContainerFactory() to set prefetch
> to 1. I am not able to change the code, only configure the broker URL. I
> can place config into the application.yml, but I suspect that will only be
> helpful if the person who wrote the app specifically exposed configuration
> which I don't see... So I guess if the DefaultJmsListener logic registers
> some config, there might be something we could set?
>
> Thanks in advance,
> -- Bindu
>
> On Fri, Feb 25, 2022 at 4:29 PM Bindu Wavell <bw...@google.com> wrote:
>
>> Hi Tim,
>>
>> Thank you for your reply and the reference to the SO article. I had not
>> found that one.
>>
>> I had not tried just using a single tcp:// connection, I just did (
>> tcp://broker1:61616?jms.prefetchPolicy.queuePrefetch=1) and the client
>> fails to connect with the following WARNing logged:
>>
>> 2022-02-25 22:42:52.727 WARN 1 --- [ntContainer#0-1]
>> o.s.j.l.DefaultMessageListenerContainer : Setup of JMS message listener
>> invoker failed for destination 'xx.yy.zz' - trying to recover. Cause: Could
>> not create JMS transaction; nested exception is javax.jms.JMSException:
>> Could not create Transport. Reason: java.lang.IllegalArgumentException:
>> Invalid connect parameters: {jms.prefetchPolicy.queuePrefetch=1}
>>
>> I guess because this is an ActiveMQ specific argument and the vendor code
>> is almost completely pure JMS without any ActiveMQ specific
>> classes/factories, etc?
>>
>>
>>
>> On Fri, Feb 25, 2022 at 6:35 AM Tim Bain <tb...@alumni.duke.edu> wrote:
>>
>>> Your first and third URIs should work, so maybe this is something
>>> specific
>>> to DefaultJmsListenerContainerFactory()? I don't have experience using
>>> it,
>>> but an answer on
>>>
>>> https://stackoverflow.com/questions/33339224/spring-jms-listener-container-concurrency-attribute-not-working
>>> lists 3 ways to set prefetch under Spring Boot, including via
>>> application.properties which looks like what you're doing. Are either of
>>> the other two possible without the vendor change you're waiting for?
>>>
>>> Does it work if you try tcp to the active broker (no failover), just to
>>> test?
>>>
>>> Tim
>>>
>>> On Thu, Feb 24, 2022, 11:03 AM Bindu Wavell <bw...@google.com.invalid>
>>> wrote:
>>>
>>> > Hi folks,
>>> >
>>> > We are using ActiveMQ Classic (5.16.1). I have 3 consumer processes
>>> and 2
>>> > ActiveMQ brokers setup with the shared storage active/passive mode.
>>> >
>>> > I setup the connection URI like this:
>>> >
>>> >
>>> >
>>> failover:(tcp://broker1:61616,tcp://broker2:61616)?timeout=3000&nested.jms.prefetchPolicy.queuePrefetch=1
>>> >
>>> > Also tried this:
>>> >
>>> >
>>> >
>>> failover:(tcp://broker1:61616,tcp://broker2:61616)?timeout=3000&jms.prefetchPolicy.queuePrefetch=1
>>> >
>>> > And this:
>>> >
>>> >
>>> >
>>> failover:(tcp://broker1:61616?jms.prefetchPolicy.queuePrefetch=1,tcp://broker2:61616?jms.prefetchPolicy.queuePrefetch=1)?timeout=3000
>>> >
>>> > Looking through the web console at active consumers for a queue I care
>>> > about I see:
>>> >
>>> > Prefetch
>>> > Max Pending
>>> > 1000
>>> > 0
>>> >
>>> > For each of the consumers. I'm also seeing Enqueues > 10. We have an
>>> > application where there are not a ton of messages, but the consumers
>>> take a
>>> > while to process messages. In the current environment, when we start
>>> our
>>> > clients, we see most messages going to one client. Over time this does
>>> seem
>>> > to even out, but I think it's indicative of prefetch not being applied.
>>> >
>>> > Am I doing this right?
>>> >
>>> > FWIW, the client is a Spring Boot application built by a vendor and I'm
>>> > trying to get the vendor to actually allow us to configure prefetch and
>>> > they will set it from code. That is taking a long time, so I was
>>> excited it
>>> > would be possible to set this via the connection URI.
>>> >
>>> > The vendor sets up the jmsListenerConnectionFactory in
>>> > their JmsListenerConfigurer implementation like this:
>>> >
>>> >
>>> > @Bean
>>> > public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(
>>> > final ConnectionFactory connectionFactory,
>>> > final TransformMessageConverter transformMessageConverter)
>>> > {
>>> > final DefaultJmsListenerContainerFactory factory = new
>>> > DefaultJmsListenerContainerFactory();
>>> > factory.setConnectionFactory(connectionFactory);
>>> > factory.setMessageConverter(transformMessageConverter);
>>> > factory.setErrorHandler(t -> logger.error("JMS error: " +
>>> t.getMessage(),
>>> > t));
>>> > factory.setTransactionManager(transactionManager(connectionFactory));
>>> > return factory;
>>> >
>>> >
>>> > }
>>> >
>>>
>>