You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by brbornia <br...@gmail.com> on 2012/03/07 23:02:23 UTC
Need help to load csv into database
Hello guys,
I'm just starting with camel, so sorry to bother you with a must likely
dummy question.
I am trying to load a csv file into database with something like this:
<camel:camelContext xmlns="http://camel.apache.org/schema/spring">
<camel:package>com.test.integration.camel.spring.poc</camel:package>
<camel:route>
<camel:from uri="file:/media/arq/work/workspace/servicemix/datafeed/from/>
<camel:split streaming="true" parallelProcessing="true">
<camel:tokenize token="\n" />
<camel:split>
<camel:tokenize token=","/>
<camel:to
uri="sql:INSERT INTO mytable (id, name, event, time)
VALUES (#, #, #, #)?dataSourceRef=myDataSource"/>
</camel:split>
</camel:split>
</camel:route>
</camel:camelContext>
And I know that will not work because I separated the csv line into several
messages when what I want is simple convert that line to an array or
something like that, but I just do not know how to do it.
A friend told me how to do it converting the csv lines into beans and
inserting them with JPA, but since I will be loading huge csv files very
often, I am concerned about unnecessary memory and processing use, so I
would like to do it without creating several objects.
Can you please help me?
Thank you in advance,
Bruno
--
View this message in context: http://camel.465427.n5.nabble.com/Need-help-to-load-csv-into-database-tp5545754p5545754.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Need help to load csv into database
Posted by brbornia <br...@gmail.com>.
Ok guys,I just found what I wanted.
Here is the code:
<camel:camelContext xmlns="http://camel.apache.org/schema/spring">
<camel:package>com.test.integration.camel.spring.poc</camel:package>
<camel:route>
<camel:from
uri="file:/media/arq/work/workspace/servicemix/datafeed/from?delete=true" />
<camel:split streaming="true" parallelProcessing="true">
<camel:tokenize token="\n" />
<camel:unmarshal>
<camel:csv/>
</camel:unmarshal>
<camel:to uri= "sql:INSERT INTO mytable (id, name, event, time)
VALUES (#, #, #, #)?dataSourceRef=feedDS"/>
</camel:split>
</camel:route>
</camel:camelContext>
Thanks anyway for your tips,
Bruno
--
View this message in context: http://camel.465427.n5.nabble.com/Need-help-to-load-csv-into-database-tp5545754p5550700.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Need help to load csv into database
Posted by Jason Chaffee <Ja...@betfair.com>.
You might want to look at Smooks camel component for this. It supports streaming too. Snooks and Camel are a powerful combination.
http://www.smooks.org/mediawiki/index.php?title=Main_Page
http://blog.smooks.org/2010/09/23/smooks-camel-integration-revisited/
Jason
On Mar 7, 2012, at 11:23 PM, "Willem Jiang" <wi...@gmail.com> wrote:
> I think you can user the bean to do the below job yourself. <camel:split>
> <camel:tokenize token=","/>
> <camel:to
> uri="sql:INSERT INTO mytable (id, name, event, time)
> VALUES (#, #, #, #)?dataSourceRef=myDataSource"/>
> </camel:split>
>
> You just need to read the message body an split the message, put it into sql parameter and call the jdbc yourself.
>
> On Thu Mar 8 06:02:23 2012, brbornia wrote:
>> Hello guys,
>>
>> I'm just starting with camel, so sorry to bother you with a must likely
>> dummy question.
>>
>> I am trying to load a csv file into database with something like this:
>>
>> <camel:camelContext xmlns="http://camel.apache.org/schema/spring">
>> <camel:package>com.test.integration.camel.spring.poc</camel:package>
>> <camel:route>
>> <camel:from uri="file:/media/arq/work/workspace/servicemix/datafeed/from/>
>> <camel:split streaming="true" parallelProcessing="true">
>> <camel:tokenize token="\n" />
>> <camel:split>
>> <camel:tokenize token=","/>
>> <camel:to
>> uri="sql:INSERT INTO mytable (id, name, event, time)
>> VALUES (#, #, #, #)?dataSourceRef=myDataSource"/>
>> </camel:split>
>> </camel:split>
>> </camel:route>
>> </camel:camelContext>
>>
>> And I know that will not work because I separated the csv line into several
>> messages when what I want is simple convert that line to an array or
>> something like that, but I just do not know how to do it.
>>
>> A friend told me how to do it converting the csv lines into beans and
>> inserting them with JPA, but since I will be loading huge csv files very
>> often, I am concerned about unnecessary memory and processing use, so I
>> would like to do it without creating several objects.
>>
>> Can you please help me?
>>
>> Thank you in advance,
>> Bruno
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.com/Need-help-to-load-csv-into-database-tp5545754p5545754.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>
>
>
> --
> Willem
> ----------------------------------
> FuseSource
> Web: http://www.fusesource.com
> Blog: http://willemjiang.blogspot.com (English)
> http://jnn.javaeye.com (Chinese)
> Twitter: willemjiang Weibo: willemjiang
________________________________________________________________________
In order to protect our email recipients, Betfair Group use SkyScan from
MessageLabs to scan all Incoming and Outgoing mail for viruses.
________________________________________________________________________
Re: Need help to load csv into database
Posted by Willem Jiang <wi...@gmail.com>.
I think you can user the bean to do the below job yourself.
<camel:split>
<camel:tokenize token=","/>
<camel:to
uri="sql:INSERT INTO mytable (id, name, event, time)
VALUES (#, #, #, #)?dataSourceRef=myDataSource"/>
</camel:split>
You just need to read the message body an split the message, put it
into sql parameter and call the jdbc yourself.
On Thu Mar 8 06:02:23 2012, brbornia wrote:
> Hello guys,
>
> I'm just starting with camel, so sorry to bother you with a must likely
> dummy question.
>
> I am trying to load a csv file into database with something like this:
>
> <camel:camelContext xmlns="http://camel.apache.org/schema/spring">
> <camel:package>com.test.integration.camel.spring.poc</camel:package>
> <camel:route>
> <camel:from uri="file:/media/arq/work/workspace/servicemix/datafeed/from/>
> <camel:split streaming="true" parallelProcessing="true">
> <camel:tokenize token="\n" />
> <camel:split>
> <camel:tokenize token=","/>
> <camel:to
> uri="sql:INSERT INTO mytable (id, name, event, time)
> VALUES (#, #, #, #)?dataSourceRef=myDataSource"/>
> </camel:split>
> </camel:split>
> </camel:route>
> </camel:camelContext>
>
> And I know that will not work because I separated the csv line into several
> messages when what I want is simple convert that line to an array or
> something like that, but I just do not know how to do it.
>
> A friend told me how to do it converting the csv lines into beans and
> inserting them with JPA, but since I will be loading huge csv files very
> often, I am concerned about unnecessary memory and processing use, so I
> would like to do it without creating several objects.
>
> Can you please help me?
>
> Thank you in advance,
> Bruno
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Need-help-to-load-csv-into-database-tp5545754p5545754.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.javaeye.com (Chinese)
Twitter: willemjiang
Weibo: willemjiang