You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by spasco <st...@gmail.com> on 2008/03/23 08:10:17 UTC
extract xml from URL and save to file
How would you go about extracting XML document from a URL then saving that
file to disk? I'd like this setup to poll the URL about once per day.
Thanks in advance!
--
View this message in context: http://www.nabble.com/extract-xml-from-URL-and-save-to-file-tp16232121s22882p16232121.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: extract xml from URL and save to file
Posted by James Strachan <ja...@gmail.com>.
On 26/03/2008, davsclaus <ci...@yahoo.dk> wrote:
>
> Shouldn't we create a JIRA ticket for Aarons great idea of having somekind of
> DSL for specifying how some pooling consumers/producers can be activated?
>
> At least to not to lose the idea? Forums tend to only be focused on new
> topics.
Definitely, let do it! Here we go...
https://issues.apache.org/activemq/browse/CAMEL-402
--
James
-------
http://macstrac.blogspot.com/
Open Source Integration
http://open.iona.com
Re: extract xml from URL and save to file
Posted by James Strachan <ja...@gmail.com>.
On 26/03/2008, Brad Fritz <br...@fritzfam.com> wrote:
> On Wed, Mar 26, 2008 at 10:39:14AM -0700, davsclaus wrote:
>
> > Shouldn't we create a JIRA ticket for Aarons great idea of having somekind of
> > DSL for specifying how some pooling consumers/producers can be activated?
>
>
> For what it's worth, I think that is a good idea.
>
> I ran across this thread while looking for a way to kick off an SFTP
> consumer on a schedule. Seems like a reasonably typical integration
> scenario. If there were a JIRA issue and some direction from
> developers about what type of solution would be accepted, I would
> definitely consider working on a patch.
Awesome! I love patches :)
> (I am in the process of
> migrating some integration utilities from Mule to Camel at work.)
Glad to hear it :)
--
James
-------
http://macstrac.blogspot.com/
Open Source Integration
http://open.iona.com
Re: extract xml from URL and save to file
Posted by Brad Fritz <br...@fritzfam.com>.
On Wed, Mar 26, 2008 at 10:39:14AM -0700, davsclaus wrote:
> Shouldn't we create a JIRA ticket for Aarons great idea of having somekind of
> DSL for specifying how some pooling consumers/producers can be activated?
For what it's worth, I think that is a good idea.
I ran across this thread while looking for a way to kick off an SFTP
consumer on a schedule. Seems like a reasonably typical integration
scenario. If there were a JIRA issue and some direction from
developers about what type of solution would be accepted, I would
definitely consider working on a patch. (I am in the process of
migrating some integration utilities from Mule to Camel at work.)
--Brad
Re: extract xml from URL and save to file
Posted by davsclaus <ci...@yahoo.dk>.
Shouldn't we create a JIRA ticket for Aarons great idea of having somekind of
DSL for specifying how some pooling consumers/producers can be activated?
At least to not to lose the idea? Forums tend to only be focused on new
topics.
--
View this message in context: http://www.nabble.com/extract-xml-from-URL-and-save-to-file-tp16232121s22882p16308137.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: extract xml from URL and save to file
Posted by Aaron Crickenberger <aa...@intalgent.com>.
On Mar 25, 2008, at 10:31 AM, James Strachan wrote:
> I guess the biggest issue right now is if we did...
>
> from("quartz://myGroup/myTimerName/0/0/12/*/*/$").to("http://foo.com/bar.xml
> ").to("file://myDirectory/someName.xml");
>
> then the HTTP endpoint would tend to do a POST not a GET as it would
> be receiving a payload of the quartz event. I guess thats just a
> limitation of the HTTP endpoint; we maybe need some way to enforce the
> GET operation or something?
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://open.iona.com
This seems like a great idea that could be applied to the file, ftp,
and even jms endpoints (basically anything with a
ScheduledPollConsumer.) Maybe another DSL construct or a polling
component could generalize this further? Not terribly familiar with
what could work within the DSL, but some ideas...
from("quartz://myGroup/myTimerName/0/0/12/*/*/$").to("poll:http://foo.com/bar.xml
").to("file://myDirectory/someName.xml");
from("quartz://myGroup/myTimerName/0/0/12/*/*/$").from("http://foo.com/bar.xml
").to("file://myDirectory/someName.xml");
trigger(from("jms:queue:pollRequests")).poll("http://foo.com/
bar.xml").to("file://myDirectory/someName.xml");
No clue how any of this could be achieved, just throwing it out there :)
- aaron
Re: extract xml from URL and save to file
Posted by James Strachan <ja...@gmail.com>.
On 26/03/2008, Roman Kalukiewicz <ro...@gmail.com> wrote:
> 2008/3/25, James Strachan <ja...@gmail.com>:
>
> > I guess the biggest issue right now is if we did...
> >
> > from("quartz://myGroup/myTimerName/0/0/12/*/*/$").to("http://foo.com/bar.xml").to("file://myDirectory/someName.xml");
> >
> > then the HTTP endpoint would tend to do a POST not a GET as it would
> > be receiving a payload of the quartz event. I guess thats just a
> > limitation of the HTTP endpoint; we maybe need some way to enforce the
> > GET operation or something?
>
>
> but this should work OK:
>
>
> from("quartz://myGroup/myTimerName/0/0/12/*/*/$")
>
> .setBody(constant(null))
>
> .to("http://foo.com/bar.xml")
> .to("file://myDirectory/someName.xml");
Great idea :)
I have been wondering if we should add verbs to the DSL to
differentiate between folks wanting to do InOnly (one way messaging)
versus InOut (request/reply). I guess polling (OutOnly?) is another
variation on the theme.
I wonder if it might be cleaner to do something like.. (using 'get' as
the OutOnly).
from("quartz://myGroup/myTimerName/0/0/12/*/*/$")
.get("http://foo.com/bar.xml")
.to("file://myDirectory/someName.xml");
Or if we wanted to consume from A then invoke B and send the result to
C we could be more explicit about the one-way / request/response
exchange pattern by
from(a).
request(b).
oneway(c)
using the verbs "request" to mean invoke with an InOut and "oneway" to
mean use an explicit InOnly.
Thoughts?
--
James
-------
http://macstrac.blogspot.com/
Open Source Integration
http://open.iona.com
Re: extract xml from URL and save to file
Posted by Roman Kalukiewicz <ro...@gmail.com>.
2008/3/25, James Strachan <ja...@gmail.com>:
> I guess the biggest issue right now is if we did...
>
> from("quartz://myGroup/myTimerName/0/0/12/*/*/$").to("http://foo.com/bar.xml").to("file://myDirectory/someName.xml");
>
> then the HTTP endpoint would tend to do a POST not a GET as it would
> be receiving a payload of the quartz event. I guess thats just a
> limitation of the HTTP endpoint; we maybe need some way to enforce the
> GET operation or something?
but this should work OK:
from("quartz://myGroup/myTimerName/0/0/12/*/*/$")
.setBody(constant(null))
.to("http://foo.com/bar.xml")
.to("file://myDirectory/someName.xml");
Romek
Re: extract xml from URL and save to file
Posted by James Strachan <ja...@gmail.com>.
On 25/03/2008, davsclaus <ci...@yahoo.dk> wrote:
>
> Would be awesome to let quartz handle the polling interval logic, then we
> could use CRON expressions, to set the interval to run at 4:15am in the
> night, so it wont be active during backup runs from 1am-3am etc.
>
> As now the polling is based on when your application/host server was started
> etc. In my experience being able to run jobs where you can determine the
> interval to be fixed at hours:mins is much better.
Agreed!
I guess the biggest issue right now is if we did...
from("quartz://myGroup/myTimerName/0/0/12/*/*/$").to("http://foo.com/bar.xml").to("file://myDirectory/someName.xml");
then the HTTP endpoint would tend to do a POST not a GET as it would
be receiving a payload of the quartz event. I guess thats just a
limitation of the HTTP endpoint; we maybe need some way to enforce the
GET operation or something?
--
James
-------
http://macstrac.blogspot.com/
Open Source Integration
http://open.iona.com
Re: extract xml from URL and save to file
Posted by davsclaus <ci...@yahoo.dk>.
Would be awesome to let quartz handle the polling interval logic, then we
could use CRON expressions, to set the interval to run at 4:15am in the
night, so it wont be active during backup runs from 1am-3am etc.
As now the polling is based on when your application/host server was started
etc. In my experience being able to run jobs where you can determine the
interval to be fixed at hours:mins is much better.
/Claus
James.Strachan wrote:
>
> On 23/03/2008, spasco <st...@gmail.com> wrote:
>>
>> How would you go about extracting XML document from a URL then saving
>> that
>> file to disk? I'd like this setup to poll the URL about once per day.
>
> something like this should do the trick...
>
> from("http://foo.bar/something.xml?delay=1000").to("file://mydir/something.xml");
>
> where the delay is the polling delay in milliseconds.
>
> I guess we could do with an easier mechanism to configure the polling
> interval; maybe including in the DSL some kind of mechanism to specify
> polling times.
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://open.iona.com
>
>
--
View this message in context: http://www.nabble.com/extract-xml-from-URL-and-save-to-file-tp16232121s22882p16274777.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: extract xml from URL and save to file
Posted by James Strachan <ja...@gmail.com>.
On 23/03/2008, spasco <st...@gmail.com> wrote:
>
> How would you go about extracting XML document from a URL then saving that
> file to disk? I'd like this setup to poll the URL about once per day.
something like this should do the trick...
from("http://foo.bar/something.xml?delay=1000").to("file://mydir/something.xml");
where the delay is the polling delay in milliseconds.
I guess we could do with an easier mechanism to configure the polling
interval; maybe including in the DSL some kind of mechanism to specify
polling times.
--
James
-------
http://macstrac.blogspot.com/
Open Source Integration
http://open.iona.com