can flume get incremental data from database(such as DB2)?

[lulynn_2008 <lu...@163.com>]

  Hello,
If I want to transfer incremental data from database to HDFS. Can flume be the middle layer? I mean flume get incremental data from database(such as DB2), and then transfer the data to HDFS. Please give your suggestion. Thank you.

Re: can flume get incremental data from database(such as DB2)?

[Arvind Prabhakar <ar...@apache.org>]

You may want to explore using Sqoop for this particular use-case. More
information on Sqoop is available at the following links:

Sqoop Wiki: https://cwiki.apache.org/confluence/display/SQOOP/Home
User Guide: http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html

Thanks,
Arvind


2011/9/27 lulynn_2008 <lu...@163.com>:
>   Hello,
> If I want to transfer incremental data from database to HDFS. Can flume be
> the middle layer? I mean flume get incremental data from database(such as
> DB2), and then transfer the data to HDFS. Please give your suggestion. Thank
> you.
>
>

RE: can flume get incremental data from database(such as DB2)?

["Huang, Zijian(Victor)" <zi...@etrade.com>]

I am not aware of any SQL source in Flume. it only has  "tailing files" as a source right now. you probably have to write your own. I don't think it is difficult to implement a simple one, which read in records from a DB table and then serialized as Avro event and send it over to the collectorSource

Vic


________________________________
From: lulynn_2008 [mailto:lulynn_2008@163.com]
Sent: Tuesday, September 27, 2011 12:03 AM
To: flume-user@incubator.apache.org
Subject: can flume get incremental data from database(such as DB2)?

  Hello,
If I want to transfer incremental data from database to HDFS. Can flume be the middle layer? I mean flume get incremental data from database(such as DB2), and then transfer the data to HDFS. Please give your suggestion. Thank you.