traffic on the activemq with 300 Producer

[muthana <mu...@gmx.de>]

Dear Sir,

Some information about the our system: we installed the activemq on Ubuntu
Server, a Java Processor works with multithreading (300 threads), all of
them send messages to the activemq, there is no messages loss, but writting
into the activemq is very slow. we built for each thread a connection to the
AMQ, that means we have 300 connections writting parallel to the AMQ. We
tried also one connection for all the 300 threads and and the average in
writting into AMQ between 3-4 sec. per message

I want to make the writting faster. can somebody help me on this? how can i
solve this problem, therefore iwould like to ask which of the following
scenario can help me better: 

1. using 2 Broker
2. using connections pool steed of the 300 connections
3. or any better idea 

our currently average in writting into AMQ between 3-4 sec. per message. -->
very slow

any help will be great

Muthana
Email: altemimi@i-digital-m.com



--
View this message in context: http://activemq.2283324.n4.nabble.com/traffic-on-the-activemq-with-300-Producer-tp4674889.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: traffic on the activemq with 300 Producer

[James Carman <ja...@carmanconsulting.com>]

Are you seeing any interesting logging messages in your AMQ log file?

On Wed, Nov 27, 2013 at 11:44 AM, muthana <mu...@gmx.de> wrote:
> Dear Sir,
>
> Some information about the our system: we installed the activemq on Ubuntu
> Server, a Java Processor works with multithreading (300 threads), all of
> them send messages to the activemq, there is no messages loss, but writting
> into the activemq is very slow. we built for each thread a connection to the
> AMQ, that means we have 300 connections writting parallel to the AMQ. We
> tried also one connection for all the 300 threads and and the average in
> writting into AMQ between 3-4 sec. per message
>
> I want to make the writting faster. can somebody help me on this? how can i
> solve this problem, therefore iwould like to ask which of the following
> scenario can help me better:
>
> 1. using 2 Broker
> 2. using connections pool steed of the 300 connections
> 3. or any better idea
>
> our currently average in writting into AMQ between 3-4 sec. per message. -->
> very slow
>
> any help will be great
>
> Muthana
> Email: altemimi@i-digital-m.com
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/traffic-on-the-activemq-with-300-Producer-tp4674889.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: traffic on the activemq with 300 Producer

[Arjen van der Meijden <ac...@tweakers.net>]

Hello Muthana,

By switching to asynchronous connections, you don't solve the underlying 
problem. You just stop waiting for the final delivery of the messages. 
But they'll most likely still take 4 seconds to deliver.

You can't have both ways; asynchronous message delivery and a easy and 
reliable way to actually get the result of the delivery.

So I'd advice to go back to the original problem and don't add 
work-arounds unless that turns out to be the only solution. Try and find 
the actual problem: Why is your ActiveMQ slow?

Ask yourself all these kinds of questions: What is it doing (cpu? disk? 
network? memory? garbage collection?) ? How large are the messages you 
actually send (which is something else than how large you think they 
ought to be)? Did you add (or not remove) limits on the queue sizes, 
forcing blocking waits until there is some space available? Do you run 
into limitations of the OS like running out of file-pointers? Have you 
actually looked into the log-messages of ActiveMQ?
Is your hardware actually up to the task (but don't blindly replace the 
hardware with a more expensive box)?

If you just keep working around the actual problem, you'll never 
actually solve it. Of course, when you've asked and answered all of the 
above questions, it may turn out that you need connection pools, 
asynchronous connections and more... but those kinds of changes may 
solve some problems, but will likely introduce new problems (like you 
ran into).

Best regards,

Arjen

On 28-11-2013 14:59 muthana wrote:
> Hello Arjen,
>
> thank you for your answer. It was helpful. I change the connection
> parameters into: connection.AsyncSend=true. This option made the producer
> faster because i'm sending the message async. into the AMQ. But i have loss
> some of the message if the broker is die for some reason.
>
> Actually i want ask in order to work around the sending in async. way. Which
> kind of response i got from AMQ, when i use the AsyncSend set to true by
> sending OR in another way: can i get some confirmation from AMQ that our
> message has been saved.
>
> Regards
> Muthana
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/traffic-on-the-activemq-with-300-Producer-tp4674889p4674915.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Re: traffic on the activemq with 300 Producer

[muthana <mu...@gmx.de>]

Hello Arjen,

thank you for your answer. It was helpful. I change the connection
parameters into: connection.AsyncSend=true. This option made the producer
faster because i'm sending the message async. into the AMQ. But i have loss
some of the message if the broker is die for some reason. 

Actually i want ask in order to work around the sending in async. way. Which
kind of response i got from AMQ, when i use the AsyncSend set to true by
sending OR in another way: can i get some confirmation from AMQ that our
message has been saved.

Regards
Muthana



--
View this message in context: http://activemq.2283324.n4.nabble.com/traffic-on-the-activemq-with-300-Producer-tp4674889p4674915.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: traffic on the activemq with 300 Producer

[Arjen van der Meijden <ac...@tweakers.net>]

Hi Muthana,

While connect, send, disconnect is not super efficient, I wouldn't look 
into replacing that right now... You should first identify the problem, 
rather than jumping to solutions that may not work at all.

The overhead of repeated connect/disconnect should not be ignored. But 
they don't add multi-second delays, at least not until some 
resource-starvation (like file-pointers on Linux).

Even 0.1 second would be long for this kind of set-up. Unless you send 
very large messages.

Could you also try and see how fast, for instance, your set-up is with 
just 1 producer. Or only 10-30?

Apart from the suggesting to look at the log, also check the cpu-usage 
of your ActiveMQ (i.e. via 'top') and other characteristics of the 
system (like disk and memory usage, for instance via 'vmstat' or 'iostat').

Best regards,

Arjen

On 27-11-2013 17:44 muthana wrote:
> Dear Sir,
>
> Some information about the our system: we installed the activemq on Ubuntu
> Server, a Java Processor works with multithreading (300 threads), all of
> them send messages to the activemq, there is no messages loss, but writting
> into the activemq is very slow. we built for each thread a connection to the
> AMQ, that means we have 300 connections writting parallel to the AMQ. We
> tried also one connection for all the 300 threads and and the average in
> writting into AMQ between 3-4 sec. per message
>
> I want to make the writting faster. can somebody help me on this? how can i
> solve this problem, therefore iwould like to ask which of the following
> scenario can help me better:
>
> 1. using 2 Broker
> 2. using connections pool steed of the 300 connections
> 3. or any better idea
>
> our currently average in writting into AMQ between 3-4 sec. per message. -->
> very slow
>
> any help will be great
>
> Muthana
> Email: altemimi@i-digital-m.com
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/traffic-on-the-activemq-with-300-Producer-tp4674889.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>