You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by Ceriac <se...@gmail.com> on 2008/12/24 04:46:55 UTC
ActiveMQ loading and persistence to MySQL
My problem is as follows:
I need to pull comma separated text using TCP sockets from a web site and
persist it to a log and a MySQL database. Each message will be at most 500
bytes and there can be between 0 and 160 messages per second. However the
system will send heartbeats every 60 seconds if no messages are sent.
I read on this forum that ActiveMQ errors out if no messages are received in
30 seconds.
My question is as follows: What tools do I need to solve the problem above?
I am guessing I need to use ActiveMQ, Spring and MySQL with Java. Do I need
other tools like Camel, Jencks etc?
What steps do I need to take in order to solve the problem above?
Thanks for all your help.
Regards,
Ceriac
--
View this message in context: http://www.nabble.com/ActiveMQ-loading-and-persistence-to-MySQL-tp21154986p21154986.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: ActiveMQ loading and persistence to MySQL
Posted by Dejan Bosanac <de...@nighttale.net>.
Hi,
you can try the splitter pattern
http://activemq.apache.org/camel/splitter.html and see if it works for you.
You'd get the better support for Camel related questions on Camel mailing
lists (http://activemq.apache.org/camel/mailing-lists.html)
Cheers
--
Dejan Bosanac
Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net
On Thu, Dec 25, 2008 at 6:47 PM, Ceriac <se...@gmail.com> wrote:
>
> Hi Dejan,
> I am leaning towards Camel. Owing to the nature of my problem, I will need
> to write to two queues (one for a file based destination and the other for
> a
> MySQL db destination).
>
> 1. Do I need to use Mina for my TCP socket connection to load the data into
> the two queues?
>
> 2. Also, how do I go about writing to two queues in Camel for the
> destinations above?
>
Re: ActiveMQ loading and persistence to MySQL
Posted by Ceriac <se...@gmail.com>.
Further to Dejan's feedback, I would say that I am leaning towards Camel.
Owing to the nature of my problem, I will need to write to two queues (one
for a file based destination and the other for a MySQL db destination).
I have a couple of follow-up questions:
1. Do I need to use Apache Mina for my TCP socket connection to load the
data into the two queues?
2. Also, how do I go about writing to two queues in Camel simultaneously for
the destinations above?
Thanks and regards,
Ceriac
Ceriac wrote:
>
> Hi Dejan,
> I am leaning towards Camel. Owing to the nature of my problem, I will need
> to write to two queues (one for a file based destination and the other for
> a MySQL db destination).
>
> 1. Do I need to use Mina for my TCP socket connection to load the data
> into the two queues?
>
> 2. Also, how do I go about writing to two queues in Camel for the
> destinations above?
>
> Thanks and regards,
> Ceriac
> _________________________________________________________
>
> Dejan Bosanac wrote:
>>
>> Hi Ceriac,
>>
>> there is no problem with ActiveMQ if there are no messages. The default
>> tcp
>> transport sends so called keep-alive packets to keep the connection
>> alive.
>> The only problem could be that your clients loose the network connection
>> to
>> the broker so that these keep-alive packets could not be exchanged.
>>
>> As for your other questions, it all depends on how you want to build your
>> solution. You can use pure JMS, or you can use Jencks and Spring to add
>> some
>> layer on top of it (for easier programming). Finally, you can try using
>> Camel to achive all of this. For example, look at the following
>> components:
>>
>> http://activemq.apache.org/camel/http.html
>> http://activemq.apache.org/camel/sql-component.html
>> http://activemq.apache.org/camel/file.html
>>
>>
>>
>> Cheers
>> --
>> Dejan Bosanac
>>
>> Open Source Integration - http://fusesource.com/
>> ActiveMQ in Action - http://www.manning.com/snyder/
>> Blog - http://www.nighttale.net
>>
>>
>> On Wed, Dec 24, 2008 at 4:46 AM, Ceriac <se...@gmail.com> wrote:
>>
>>>
>>> My problem is as follows:
>>> I need to pull comma separated text using TCP sockets from a web site
>>> and
>>> persist it to a log and a MySQL database. Each message will be at most
>>> 500
>>> bytes and there can be between 0 and 160 messages per second. However
>>> the
>>> system will send heartbeats every 60 seconds if no messages are sent.
>>> I read on this forum that ActiveMQ errors out if no messages are
>>> received
>>> in
>>> 30 seconds.
>>> My question is as follows: What tools do I need to solve the problem
>>> above?
>>> I am guessing I need to use ActiveMQ, Spring and MySQL with Java. Do I
>>> need
>>> other tools like Camel, Jencks etc?
>>> What steps do I need to take in order to solve the problem above?
>>> Thanks for all your help.
>>> Regards,
>>> Ceriac
>>> --
>>> View this message in context:
>>> http://www.nabble.com/ActiveMQ-loading-and-persistence-to-MySQL-tp21154986p21154986.html
>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>>>
>>
>>
>> -----
>> Dejan Bosanac
>>
>>
>> http://www.ttmsolutions.com - get a free ActiveMQ user guide
>>
>> ActiveMQ in Action - http://www.manning.com/snyder/
>> Scripting in Java - http://www.scriptinginjava.net
>>
>
>
--
View this message in context: http://www.nabble.com/Question-about-ActiveMQ-queuing-and-persistence-to-MySQL-tp21154986p21175509.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: ActiveMQ loading and persistence to MySQL
Posted by Ceriac <se...@gmail.com>.
Hi Dejan,
I am leaning towards Camel. Owing to the nature of my problem, I will need
to write to two queues (one for a file based destination and the other for a
MySQL db destination).
1. Do I need to use Mina for my TCP socket connection to load the data into
the two queues?
2. Also, how do I go about writing to two queues in Camel for the
destinations above?
Thanks and regards,
Ceriac
_________________________________________________________
Dejan Bosanac wrote:
>
> Hi Ceriac,
>
> there is no problem with ActiveMQ if there are no messages. The default
> tcp
> transport sends so called keep-alive packets to keep the connection alive.
> The only problem could be that your clients loose the network connection
> to
> the broker so that these keep-alive packets could not be exchanged.
>
> As for your other questions, it all depends on how you want to build your
> solution. You can use pure JMS, or you can use Jencks and Spring to add
> some
> layer on top of it (for easier programming). Finally, you can try using
> Camel to achive all of this. For example, look at the following
> components:
>
> http://activemq.apache.org/camel/http.html
> http://activemq.apache.org/camel/sql-component.html
> http://activemq.apache.org/camel/file.html
>
>
>
> Cheers
> --
> Dejan Bosanac
>
> Open Source Integration - http://fusesource.com/
> ActiveMQ in Action - http://www.manning.com/snyder/
> Blog - http://www.nighttale.net
>
>
> On Wed, Dec 24, 2008 at 4:46 AM, Ceriac <se...@gmail.com> wrote:
>
>>
>> My problem is as follows:
>> I need to pull comma separated text using TCP sockets from a web site and
>> persist it to a log and a MySQL database. Each message will be at most
>> 500
>> bytes and there can be between 0 and 160 messages per second. However the
>> system will send heartbeats every 60 seconds if no messages are sent.
>> I read on this forum that ActiveMQ errors out if no messages are received
>> in
>> 30 seconds.
>> My question is as follows: What tools do I need to solve the problem
>> above?
>> I am guessing I need to use ActiveMQ, Spring and MySQL with Java. Do I
>> need
>> other tools like Camel, Jencks etc?
>> What steps do I need to take in order to solve the problem above?
>> Thanks for all your help.
>> Regards,
>> Ceriac
>> --
>> View this message in context:
>> http://www.nabble.com/ActiveMQ-loading-and-persistence-to-MySQL-tp21154986p21154986.html
>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>>
>
>
> -----
> Dejan Bosanac
>
>
> http://www.ttmsolutions.com - get a free ActiveMQ user guide
>
> ActiveMQ in Action - http://www.manning.com/snyder/
> Scripting in Java - http://www.scriptinginjava.net
>
--
View this message in context: http://www.nabble.com/Question-about-ActiveMQ-queuing-and-persistence-to-MySQL-tp21154986p21168963.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: ActiveMQ loading and persistence to MySQL
Posted by Dejan Bosanac <de...@nighttale.net>.
Hi Ceriac,
there is no problem with ActiveMQ if there are no messages. The default tcp
transport sends so called keep-alive packets to keep the connection alive.
The only problem could be that your clients loose the network connection to
the broker so that these keep-alive packets could not be exchanged.
As for your other questions, it all depends on how you want to build your
solution. You can use pure JMS, or you can use Jencks and Spring to add some
layer on top of it (for easier programming). Finally, you can try using
Camel to achive all of this. For example, look at the following components:
http://activemq.apache.org/camel/http.html
http://activemq.apache.org/camel/sql-component.html
http://activemq.apache.org/camel/file.html
Cheers
--
Dejan Bosanac
Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net
On Wed, Dec 24, 2008 at 4:46 AM, Ceriac <se...@gmail.com> wrote:
>
> My problem is as follows:
> I need to pull comma separated text using TCP sockets from a web site and
> persist it to a log and a MySQL database. Each message will be at most 500
> bytes and there can be between 0 and 160 messages per second. However the
> system will send heartbeats every 60 seconds if no messages are sent.
> I read on this forum that ActiveMQ errors out if no messages are received
> in
> 30 seconds.
> My question is as follows: What tools do I need to solve the problem above?
> I am guessing I need to use ActiveMQ, Spring and MySQL with Java. Do I need
> other tools like Camel, Jencks etc?
> What steps do I need to take in order to solve the problem above?
> Thanks for all your help.
> Regards,
> Ceriac
> --
> View this message in context:
> http://www.nabble.com/ActiveMQ-loading-and-persistence-to-MySQL-tp21154986p21154986.html
> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>