You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Sean Beck <se...@gmail.com> on 2013/06/20 21:37:32 UTC
ActiveMQ functionality to detect if certain messages are in a queue
So I am sending files over to a server and there are two files with the
same name but of a different "type" that are merged together on the server
side. As soon as a file is received I send a message containing the type
and the filename to a queue. Once both files are over on the server I want
to then start another program that combines the two.
Does ActiveMQ support this kind of functionality? I don't really understand
Camel but I have been told that it might. If so could you point me in the
right direction?
Thanks
Re: ActiveMQ functionality to detect if certain messages are in a queue
Posted by Christian Posta <ch...@gmail.com>.
There is no C version of camel AFAIK.
Yep, you could have the second part done by Camel and leave the first part
as it is (especially if it's something that's currently implemented and
working.... )
On Thu, Jun 20, 2013 at 4:21 PM, Sean Beck <se...@gmail.com> wrote:
> I subscribed to the Camel mailing list and asked there as well after your
> first response. Thanks for the second one. Camel makes that process super
> easy it seems!
>
> So there is no C version of Camel? Could I still send messages the way they
> are currently then have Camel listening in on that queue from a Java
> program?
>
>
> On Thu, Jun 20, 2013 at 2:14 PM, Christian Posta
> <ch...@gmail.com>wrote:
>
> > Well Camel is a java library.
> > Seems like you could split your use case into two steps:
> >
> > 1) wait for files to arrive, and send messages via STOMP/activemq to
> alert
> > a downstream process that files have arrived
> >
> > 2) wait for messages, and when a certain set of messages has arrived, you
> > could pick up the files and begin processing.
> >
> > You could use camel for part 1 + part 2, or just part 1, or just part
> 2...
> > or neither :) Camel just makes all of that stuff above pretty easy.
> >
> > Camel itself basically abstracts the details by leveraging existing
> > libraries or APIs and makes it easier to read and write "integration
> > routes"
> >
> > So a route for step 1 could look like this:
> >
> >
> >
> from("file:/path/incoming").to("jms:newFileQueue").to("file:/path/incomingProcessed")
> >
> > this would send a message to a 'newFileQueue' when a new file was
> detected
> > in /path/incoming and it will also move to /path/incomingProcessed folder
> > so another process can pick it up... note we use the "jms" component,
> but
> > you could use STOMP under the covers.
> >
> > then in a route for step 2:
> >
> >
> >
> from("jms:newFileQueue").aggregate(header("CamelFileName")).completionSize(2).to("trigger
> > service to process and merge files")
> >
> > this would read messages from the queue, and aggregate based on
> headers...
> > if two files with the same filename came through, we would kick off the
> > processing/merge process.
> >
> > Take a closer look at the camel docs for more details on building the
> > routes as I just popped this off the top of my head... Basically you can
> > save a lot of custom code around functionality that is commoditized and
> > leverage existing code.
> >
> > Also consider asking specific questions at the Camel mailing list.
> >
> >
> > On Thu, Jun 20, 2013 at 3:47 PM, Sean Beck <se...@gmail.com>
> > wrote:
> >
> > > So currently I'm using the libstomp library to send messages to my
> queue.
> > > Would I have to switch to some special Camel API to send messages or
> does
> > > my message get sent to Camel then to ActiveMQ?
> > >
> > >
> > > On Thu, Jun 20, 2013 at 1:43 PM, Christian Posta
> > > <ch...@gmail.com>wrote:
> > >
> > > > Yep, you would use Camel to send the message to ActiveMQ, and also
> use
> > > > Camel to read from the file system and kickoff whatever the merge
> > process
> > > > is. ActiveMQ would act as the "queue" in your scenario.
> > > >
> > > > Take a look at
> > > > http://camel.apache.org/file2.html
> > > > http://camel.apache.org/jms.html
> > > >
> > > >
> > > > On Thu, Jun 20, 2013 at 3:37 PM, Sean Beck <se...@gmail.com>
> > > > wrote:
> > > >
> > > > > So I am sending files over to a server and there are two files with
> > the
> > > > > same name but of a different "type" that are merged together on the
> > > > server
> > > > > side. As soon as a file is received I send a message containing the
> > > type
> > > > > and the filename to a queue. Once both files are over on the
> server I
> > > > want
> > > > > to then start another program that combines the two.
> > > > >
> > > > > Does ActiveMQ support this kind of functionality? I don't really
> > > > understand
> > > > > Camel but I have been told that it might. If so could you point me
> in
> > > the
> > > > > right direction?
> > > > >
> > > > > Thanks
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > *Christian Posta*
> > > > http://www.christianposta.com/blog
> > > > twitter: @christianposta
> > > >
> > >
> >
> >
> >
> > --
> > *Christian Posta*
> > http://www.christianposta.com/blog
> > twitter: @christianposta
> >
>
--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
Re: ActiveMQ functionality to detect if certain messages are in a queue
Posted by Sean Beck <se...@gmail.com>.
I subscribed to the Camel mailing list and asked there as well after your
first response. Thanks for the second one. Camel makes that process super
easy it seems!
So there is no C version of Camel? Could I still send messages the way they
are currently then have Camel listening in on that queue from a Java
program?
On Thu, Jun 20, 2013 at 2:14 PM, Christian Posta
<ch...@gmail.com>wrote:
> Well Camel is a java library.
> Seems like you could split your use case into two steps:
>
> 1) wait for files to arrive, and send messages via STOMP/activemq to alert
> a downstream process that files have arrived
>
> 2) wait for messages, and when a certain set of messages has arrived, you
> could pick up the files and begin processing.
>
> You could use camel for part 1 + part 2, or just part 1, or just part 2...
> or neither :) Camel just makes all of that stuff above pretty easy.
>
> Camel itself basically abstracts the details by leveraging existing
> libraries or APIs and makes it easier to read and write "integration
> routes"
>
> So a route for step 1 could look like this:
>
>
> from("file:/path/incoming").to("jms:newFileQueue").to("file:/path/incomingProcessed")
>
> this would send a message to a 'newFileQueue' when a new file was detected
> in /path/incoming and it will also move to /path/incomingProcessed folder
> so another process can pick it up... note we use the "jms" component, but
> you could use STOMP under the covers.
>
> then in a route for step 2:
>
>
> from("jms:newFileQueue").aggregate(header("CamelFileName")).completionSize(2).to("trigger
> service to process and merge files")
>
> this would read messages from the queue, and aggregate based on headers...
> if two files with the same filename came through, we would kick off the
> processing/merge process.
>
> Take a closer look at the camel docs for more details on building the
> routes as I just popped this off the top of my head... Basically you can
> save a lot of custom code around functionality that is commoditized and
> leverage existing code.
>
> Also consider asking specific questions at the Camel mailing list.
>
>
> On Thu, Jun 20, 2013 at 3:47 PM, Sean Beck <se...@gmail.com>
> wrote:
>
> > So currently I'm using the libstomp library to send messages to my queue.
> > Would I have to switch to some special Camel API to send messages or does
> > my message get sent to Camel then to ActiveMQ?
> >
> >
> > On Thu, Jun 20, 2013 at 1:43 PM, Christian Posta
> > <ch...@gmail.com>wrote:
> >
> > > Yep, you would use Camel to send the message to ActiveMQ, and also use
> > > Camel to read from the file system and kickoff whatever the merge
> process
> > > is. ActiveMQ would act as the "queue" in your scenario.
> > >
> > > Take a look at
> > > http://camel.apache.org/file2.html
> > > http://camel.apache.org/jms.html
> > >
> > >
> > > On Thu, Jun 20, 2013 at 3:37 PM, Sean Beck <se...@gmail.com>
> > > wrote:
> > >
> > > > So I am sending files over to a server and there are two files with
> the
> > > > same name but of a different "type" that are merged together on the
> > > server
> > > > side. As soon as a file is received I send a message containing the
> > type
> > > > and the filename to a queue. Once both files are over on the server I
> > > want
> > > > to then start another program that combines the two.
> > > >
> > > > Does ActiveMQ support this kind of functionality? I don't really
> > > understand
> > > > Camel but I have been told that it might. If so could you point me in
> > the
> > > > right direction?
> > > >
> > > > Thanks
> > > >
> > >
> > >
> > >
> > > --
> > > *Christian Posta*
> > > http://www.christianposta.com/blog
> > > twitter: @christianposta
> > >
> >
>
>
>
> --
> *Christian Posta*
> http://www.christianposta.com/blog
> twitter: @christianposta
>
Re: ActiveMQ functionality to detect if certain messages are in a queue
Posted by Christian Posta <ch...@gmail.com>.
Well Camel is a java library.
Seems like you could split your use case into two steps:
1) wait for files to arrive, and send messages via STOMP/activemq to alert
a downstream process that files have arrived
2) wait for messages, and when a certain set of messages has arrived, you
could pick up the files and begin processing.
You could use camel for part 1 + part 2, or just part 1, or just part 2...
or neither :) Camel just makes all of that stuff above pretty easy.
Camel itself basically abstracts the details by leveraging existing
libraries or APIs and makes it easier to read and write "integration routes"
So a route for step 1 could look like this:
from("file:/path/incoming").to("jms:newFileQueue").to("file:/path/incomingProcessed")
this would send a message to a 'newFileQueue' when a new file was detected
in /path/incoming and it will also move to /path/incomingProcessed folder
so another process can pick it up... note we use the "jms" component, but
you could use STOMP under the covers.
then in a route for step 2:
from("jms:newFileQueue").aggregate(header("CamelFileName")).completionSize(2).to("trigger
service to process and merge files")
this would read messages from the queue, and aggregate based on headers...
if two files with the same filename came through, we would kick off the
processing/merge process.
Take a closer look at the camel docs for more details on building the
routes as I just popped this off the top of my head... Basically you can
save a lot of custom code around functionality that is commoditized and
leverage existing code.
Also consider asking specific questions at the Camel mailing list.
On Thu, Jun 20, 2013 at 3:47 PM, Sean Beck <se...@gmail.com> wrote:
> So currently I'm using the libstomp library to send messages to my queue.
> Would I have to switch to some special Camel API to send messages or does
> my message get sent to Camel then to ActiveMQ?
>
>
> On Thu, Jun 20, 2013 at 1:43 PM, Christian Posta
> <ch...@gmail.com>wrote:
>
> > Yep, you would use Camel to send the message to ActiveMQ, and also use
> > Camel to read from the file system and kickoff whatever the merge process
> > is. ActiveMQ would act as the "queue" in your scenario.
> >
> > Take a look at
> > http://camel.apache.org/file2.html
> > http://camel.apache.org/jms.html
> >
> >
> > On Thu, Jun 20, 2013 at 3:37 PM, Sean Beck <se...@gmail.com>
> > wrote:
> >
> > > So I am sending files over to a server and there are two files with the
> > > same name but of a different "type" that are merged together on the
> > server
> > > side. As soon as a file is received I send a message containing the
> type
> > > and the filename to a queue. Once both files are over on the server I
> > want
> > > to then start another program that combines the two.
> > >
> > > Does ActiveMQ support this kind of functionality? I don't really
> > understand
> > > Camel but I have been told that it might. If so could you point me in
> the
> > > right direction?
> > >
> > > Thanks
> > >
> >
> >
> >
> > --
> > *Christian Posta*
> > http://www.christianposta.com/blog
> > twitter: @christianposta
> >
>
--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
Re: ActiveMQ functionality to detect if certain messages are in a queue
Posted by Sean Beck <se...@gmail.com>.
So currently I'm using the libstomp library to send messages to my queue.
Would I have to switch to some special Camel API to send messages or does
my message get sent to Camel then to ActiveMQ?
On Thu, Jun 20, 2013 at 1:43 PM, Christian Posta
<ch...@gmail.com>wrote:
> Yep, you would use Camel to send the message to ActiveMQ, and also use
> Camel to read from the file system and kickoff whatever the merge process
> is. ActiveMQ would act as the "queue" in your scenario.
>
> Take a look at
> http://camel.apache.org/file2.html
> http://camel.apache.org/jms.html
>
>
> On Thu, Jun 20, 2013 at 3:37 PM, Sean Beck <se...@gmail.com>
> wrote:
>
> > So I am sending files over to a server and there are two files with the
> > same name but of a different "type" that are merged together on the
> server
> > side. As soon as a file is received I send a message containing the type
> > and the filename to a queue. Once both files are over on the server I
> want
> > to then start another program that combines the two.
> >
> > Does ActiveMQ support this kind of functionality? I don't really
> understand
> > Camel but I have been told that it might. If so could you point me in the
> > right direction?
> >
> > Thanks
> >
>
>
>
> --
> *Christian Posta*
> http://www.christianposta.com/blog
> twitter: @christianposta
>
Re: ActiveMQ functionality to detect if certain messages are in a queue
Posted by Christian Posta <ch...@gmail.com>.
Yep, you would use Camel to send the message to ActiveMQ, and also use
Camel to read from the file system and kickoff whatever the merge process
is. ActiveMQ would act as the "queue" in your scenario.
Take a look at
http://camel.apache.org/file2.html
http://camel.apache.org/jms.html
On Thu, Jun 20, 2013 at 3:37 PM, Sean Beck <se...@gmail.com> wrote:
> So I am sending files over to a server and there are two files with the
> same name but of a different "type" that are merged together on the server
> side. As soon as a file is received I send a message containing the type
> and the filename to a queue. Once both files are over on the server I want
> to then start another program that combines the two.
>
> Does ActiveMQ support this kind of functionality? I don't really understand
> Camel but I have been told that it might. If so could you point me in the
> right direction?
>
> Thanks
>
--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta