ActiveMQ not able to handle message load

[ma258110 <ma...@rediffmail.com>]

Hi I am using Apache ActiveMQ 5.12. In my project the requirement is 1000s of
clients will connect and send messages all at a time. My server application
is able to accept 1000s of connection and store in database but while
dumping those to ActiveMQ its showing error. Only after 10 to 15 messages
the server app is throwing JMSException which is saying JMSConnection is
down and after that it is not able to send any message to queue. Below is
the message coming in the ActiveMQ console.

"*jvm 1    |  WARN | Async error occurred: javax.jms.JMSException:
Transaction 'TX:ID:L-501800318-53077-1447330462832-1:1:2' has not been
started.*"

The producer has implemented ExceptionListener which override onException()
and in the onException() i am reseting the JMSConnection. But not able to
send data to queue.

It's bit urgent and critical. Please guide me what can i do?


Thanks In Advance.



--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-not-able-to-handle-message-load-tp4703928.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ not able to handle message load

[ma258110 <ma...@rediffmail.com>]

I tried with Fail Over Transport but its the same error and out of 500
messages only 140 messages went to  Queue. Below is the high level design of
app.

<http://activemq.2283324.n4.nabble.com/file/n4704089/CCG_Design.png> 



--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-not-able-to-handle-message-load-tp4703928p4704089.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ not able to handle message load

[ma258110 <ma...@rediffmail.com>]

my clients are not disconnecting frequently bcoz one client will connect only
once and stay connected for 10 mins without any disconnection. Similarly all
1000s clients will connect only once and stay connected for 10mins. So there
is no chance of clients diconnecting and reconnecting frequently.Even i also
checked the same thing by extending the client disconnection time from 10
mins to 30 mins and i didn't find a single trace of Client disconnection. If
i will give a delay of 1000 ms (1 sec) between two clients to send data then
activemq will never fail and never disconnect which was happenging since
last two years. Now i have a requirement where all the 1000 clients will
connect and send data at one time means all 1000 data will reach ActiveMQ at
same time.

Below is my design pattern

i m maintaining only one JMSConnection for all the 1000s clients. When a
client connection reached my server application after accetping that client
i create a new Thread to handle that particular client including Connection,
Data, Acknowledgment and disconnection in that particular Thread only. So
for 1000s clients i am creating 1000s Thread and manage their interaction
and all 1000s client's data go to ActiveMQ through only one JMS Connection
and one Producer. Here only ActiveMQ fails. If i give delay in sending the
data then ActiveMQ find enough time to send the data to Queue and go for
next client's data. Here all 1000s client's data comes at same time bcoz of
which ActiveMQ or Producer is not finding time to put the data in Queue and
go for next client's data.

And only One JMSCOnnection is maintained through the application life cycle
and when JMSBroker connection fails it reconnect to the JMSBroker and after
that only application send the data.
Here one JMS Connection and One Producer is maintained.

And i am not maintaining the fail over yet. Will that help me and make sure
that Activemq is not diconnecting??

And how to enable the log ?



--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-not-able-to-handle-message-load-tp4703928p4704077.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ not able to handle message load

[artnaseef <ar...@artnaseef.com>]

There is no set messaging rates, and ActiveMQ doesn't lose messages in the
normal case, so something else must be wrong.

It's not entirely clear, but it sounds like your clients are disconnecting
from the broker and reconnecting frequently.  Is that true?  If so, that's
an anti-pattern that will lead to problems.  JMS is designed for long-lived
connections, unlike REST and SOAP.  Short lived connections lead to
significant extra load on the broker, which is greatly compounded in a
network of brokers.

In the initial problem description, it also sounded like clients lose the
connection to the broker and fail to send messages when that happens.  Are
clients using the failover transport?  The failover transport will eliminate
problems created by lost connections to the broker.  One caveat - make sure
to have logging enabled because clients won't really know when they are
connected or not, which can lead to confusion.  The logging makes it clear
when the client is connected or disconnected.



--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-not-able-to-handle-message-load-tp4703928p4704064.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ not able to handle message load

[ma258110 <ma...@rediffmail.com>]

Hi my project requirement is for 1000s of clients but it is failing even it
is failing for 100 clients. I think the problem is not for number of clients
but for the *frequency* of client connection. Here the clients are getting
connected and sending messages with in fractions of milliseconds means 100
messages sent to activeMQ within *7 seconds* and activeMQ not able to
respond so quickly??. So if 100 messages sent to queue only 60 messages
actually sent to queue and 40 are lost.

Can you please guide me how can i configure activeMQ so that it can accept
messages without fail. Is there any limit for active MQ on frequency of
messages?



--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-not-able-to-handle-message-load-tp4703928p4704025.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ not able to handle message load

[artnaseef <ar...@artnaseef.com>]

The problem with helping on this one is it's very complex.  1000s of clients
and it sounds like there are issues with losing connections, although that's
not clear.

Do you have a minimal use-case that demonstrates the problem?



--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-not-able-to-handle-message-load-tp4703928p4703971.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ not able to handle message load

[ma258110 <ma...@rediffmail.com>]

Hi All,

Can anybody answer?? Or is it a duplicate question? Sorry if it is a
duplicate. If so please provide me the link where i can i see the answer to
this? I searched a lot before putting this question.

Please help




--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-not-able-to-handle-message-load-tp4703928p4703938.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.