You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Weishung Chung <we...@gmail.com> on 2011/02/03 22:18:02 UTC

is there any tool that facilitate the import of data to hbase

I am looking for tool that allows me to import data from mysql to hbase. Any
suggestion?
Thank you :)

Re: is there any tool that facilitate the import of data to hbase

Posted by Weishung Chung <we...@gmail.com>.
I am looking forward to it :)
btw, I found this informative video about Sqoop.
http://www.cloudera.com/blog/2009/12/hadoop-world-sqoop-database-import-for-hadoop/

On Thu, Feb 3, 2011 at 4:44 PM, arvind@cloudera.com <ar...@cloudera.com>wrote:

> On Thu, Feb 3, 2011 at 2:17 PM, Weishung Chung <we...@gmail.com> wrote:
>
> > that's awesome !!!
> > could I go from HBase to Mysql too since I might have to maintain
> > backward compatibility between the two systems in the process of switch
> > over
> > to HBase?
> >
>
> At this time Sqoop supports importing data from external databases into
> Hive
> and HBase. However, exporting is not directly supported. You could extract
> data into HDFS and then move it back to the external system possibly.
>
> We are working on providing export capability from HBase and Hive
> to external databases in Sqoop. This will likely be implemented in the next
> few releases.
>
> Arvind
>
>
> >
> > On Thu, Feb 3, 2011 at 4:11 PM, arvind@cloudera.com <arvind@cloudera.com
> > >wrote:
> >
> > > On Thu, Feb 3, 2011 at 1:56 PM, Weishung Chung <we...@gmail.com>
> > wrote:
> > >
> > > > About Sqoop, could I import the data specifically into HBase ?
> > > >
> > >
> > > Yes - Sqoop supports direct imports from external databases into HBase.
> > > More
> > > details can be found in Sqoop documentation available
> > > here<
> > >
> >
> http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html#_importing_data_into_hbase
> > > >
> > > .
> > >
> > > Arvind
> > >
> > >
> > > > I know I could write a program to read from mysql and use HBase API
> to
> > > > write
> > > > to HBase.
> > > >
> > > > On Thu, Feb 3, 2011 at 3:49 PM, Weishung Chung <we...@gmail.com>
> > > wrote:
> > > >
> > > > > thank you for the clarification :) I am reading about sqoop now...
> > > > >
> > > > >
> > > > > On Thu, Feb 3, 2011 at 3:41 PM, Mark Kerzner <
> markkerzner@gmail.com
> > > > >wrote:
> > > > >
> > > > >> scribe is a tool for log aggregation, at face value, not mysql
> > > > >>
> > > > >> On Thu, Feb 3, 2011 at 3:37 PM, Weishung Chung <
> weishung@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Thank you for all the quick response. This really helps me
> along.
> > > How
> > > > >> about
> > > > >> > facebook's scribe?
> > > > >> >
> > > > >> > On Thu, Feb 3, 2011 at 3:35 PM, Mark Kerzner <
> > markkerzner@gmail.com
> > > >
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Overview:
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> http://hadoopinpractice.blogspot.com/2011/01/loading-data-from-mysql-to-hadoop.html
> > > > >> > >
> > > > >> > > On Thu, Feb 3, 2011 at 3:28 PM, Ryan Rawson <
> ryanobjc@gmail.com
> > >
> > > > >> wrote:
> > > > >> > >
> > > > >> > > > ImportTSV?
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/ImportTsv.html
> > > > >> > > >
> > > > >> > > > Also writing a job to read from JDBC and write to hbase isnt
> > too
> > > > bad
> > > > >> > > > if your schema isnt too insanely complex.
> > > > >> > > >
> > > > >> > > > -ryan
> > > > >> > > >
> > > > >> > > > On Thu, Feb 3, 2011 at 1:23 PM, Buttler, David <
> > > buttler1@llnl.gov
> > > > >
> > > > >> > > wrote:
> > > > >> > > > > Sqoop?
> > > > >> > > > >
> http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html
> > > > >> > > > >
> > > > >> > > > > -----Original Message-----
> > > > >> > > > > From: Weishung Chung [mailto:weishung@gmail.com]
> > > > >> > > > > Sent: Thursday, February 03, 2011 1:18 PM
> > > > >> > > > > To: user@hbase.apache.org
> > > > >> > > > > Subject: is there any tool that facilitate the import of
> > data
> > > to
> > > > >> > hbase
> > > > >> > > > >
> > > > >> > > > > I am looking for tool that allows me to import data from
> > mysql
> > > > to
> > > > >> > > hbase.
> > > > >> > > > Any
> > > > >> > > > > suggestion?
> > > > >> > > > > Thank you :)
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: is there any tool that facilitate the import of data to hbase

Posted by "arvind@cloudera.com" <ar...@cloudera.com>.
On Thu, Feb 3, 2011 at 2:17 PM, Weishung Chung <we...@gmail.com> wrote:

> that's awesome !!!
> could I go from HBase to Mysql too since I might have to maintain
> backward compatibility between the two systems in the process of switch
> over
> to HBase?
>

At this time Sqoop supports importing data from external databases into Hive
and HBase. However, exporting is not directly supported. You could extract
data into HDFS and then move it back to the external system possibly.

We are working on providing export capability from HBase and Hive
to external databases in Sqoop. This will likely be implemented in the next
few releases.

Arvind


>
> On Thu, Feb 3, 2011 at 4:11 PM, arvind@cloudera.com <arvind@cloudera.com
> >wrote:
>
> > On Thu, Feb 3, 2011 at 1:56 PM, Weishung Chung <we...@gmail.com>
> wrote:
> >
> > > About Sqoop, could I import the data specifically into HBase ?
> > >
> >
> > Yes - Sqoop supports direct imports from external databases into HBase.
> > More
> > details can be found in Sqoop documentation available
> > here<
> >
> http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html#_importing_data_into_hbase
> > >
> > .
> >
> > Arvind
> >
> >
> > > I know I could write a program to read from mysql and use HBase API to
> > > write
> > > to HBase.
> > >
> > > On Thu, Feb 3, 2011 at 3:49 PM, Weishung Chung <we...@gmail.com>
> > wrote:
> > >
> > > > thank you for the clarification :) I am reading about sqoop now...
> > > >
> > > >
> > > > On Thu, Feb 3, 2011 at 3:41 PM, Mark Kerzner <markkerzner@gmail.com
> > > >wrote:
> > > >
> > > >> scribe is a tool for log aggregation, at face value, not mysql
> > > >>
> > > >> On Thu, Feb 3, 2011 at 3:37 PM, Weishung Chung <we...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Thank you for all the quick response. This really helps me along.
> > How
> > > >> about
> > > >> > facebook's scribe?
> > > >> >
> > > >> > On Thu, Feb 3, 2011 at 3:35 PM, Mark Kerzner <
> markkerzner@gmail.com
> > >
> > > >> > wrote:
> > > >> >
> > > >> > > Overview:
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> http://hadoopinpractice.blogspot.com/2011/01/loading-data-from-mysql-to-hadoop.html
> > > >> > >
> > > >> > > On Thu, Feb 3, 2011 at 3:28 PM, Ryan Rawson <ryanobjc@gmail.com
> >
> > > >> wrote:
> > > >> > >
> > > >> > > > ImportTSV?
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/ImportTsv.html
> > > >> > > >
> > > >> > > > Also writing a job to read from JDBC and write to hbase isnt
> too
> > > bad
> > > >> > > > if your schema isnt too insanely complex.
> > > >> > > >
> > > >> > > > -ryan
> > > >> > > >
> > > >> > > > On Thu, Feb 3, 2011 at 1:23 PM, Buttler, David <
> > buttler1@llnl.gov
> > > >
> > > >> > > wrote:
> > > >> > > > > Sqoop?
> > > >> > > > > http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html
> > > >> > > > >
> > > >> > > > > -----Original Message-----
> > > >> > > > > From: Weishung Chung [mailto:weishung@gmail.com]
> > > >> > > > > Sent: Thursday, February 03, 2011 1:18 PM
> > > >> > > > > To: user@hbase.apache.org
> > > >> > > > > Subject: is there any tool that facilitate the import of
> data
> > to
> > > >> > hbase
> > > >> > > > >
> > > >> > > > > I am looking for tool that allows me to import data from
> mysql
> > > to
> > > >> > > hbase.
> > > >> > > > Any
> > > >> > > > > suggestion?
> > > >> > > > > Thank you :)
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>

Re: is there any tool that facilitate the import of data to hbase

Posted by Weishung Chung <we...@gmail.com>.
that's awesome !!!
could I go from HBase to Mysql too since I might have to maintain
backward compatibility between the two systems in the process of switch over
to HBase?

On Thu, Feb 3, 2011 at 4:11 PM, arvind@cloudera.com <ar...@cloudera.com>wrote:

> On Thu, Feb 3, 2011 at 1:56 PM, Weishung Chung <we...@gmail.com> wrote:
>
> > About Sqoop, could I import the data specifically into HBase ?
> >
>
> Yes - Sqoop supports direct imports from external databases into HBase.
> More
> details can be found in Sqoop documentation available
> here<
> http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html#_importing_data_into_hbase
> >
> .
>
> Arvind
>
>
> > I know I could write a program to read from mysql and use HBase API to
> > write
> > to HBase.
> >
> > On Thu, Feb 3, 2011 at 3:49 PM, Weishung Chung <we...@gmail.com>
> wrote:
> >
> > > thank you for the clarification :) I am reading about sqoop now...
> > >
> > >
> > > On Thu, Feb 3, 2011 at 3:41 PM, Mark Kerzner <markkerzner@gmail.com
> > >wrote:
> > >
> > >> scribe is a tool for log aggregation, at face value, not mysql
> > >>
> > >> On Thu, Feb 3, 2011 at 3:37 PM, Weishung Chung <we...@gmail.com>
> > >> wrote:
> > >>
> > >> > Thank you for all the quick response. This really helps me along.
> How
> > >> about
> > >> > facebook's scribe?
> > >> >
> > >> > On Thu, Feb 3, 2011 at 3:35 PM, Mark Kerzner <markkerzner@gmail.com
> >
> > >> > wrote:
> > >> >
> > >> > > Overview:
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> http://hadoopinpractice.blogspot.com/2011/01/loading-data-from-mysql-to-hadoop.html
> > >> > >
> > >> > > On Thu, Feb 3, 2011 at 3:28 PM, Ryan Rawson <ry...@gmail.com>
> > >> wrote:
> > >> > >
> > >> > > > ImportTSV?
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/ImportTsv.html
> > >> > > >
> > >> > > > Also writing a job to read from JDBC and write to hbase isnt too
> > bad
> > >> > > > if your schema isnt too insanely complex.
> > >> > > >
> > >> > > > -ryan
> > >> > > >
> > >> > > > On Thu, Feb 3, 2011 at 1:23 PM, Buttler, David <
> buttler1@llnl.gov
> > >
> > >> > > wrote:
> > >> > > > > Sqoop?
> > >> > > > > http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html
> > >> > > > >
> > >> > > > > -----Original Message-----
> > >> > > > > From: Weishung Chung [mailto:weishung@gmail.com]
> > >> > > > > Sent: Thursday, February 03, 2011 1:18 PM
> > >> > > > > To: user@hbase.apache.org
> > >> > > > > Subject: is there any tool that facilitate the import of data
> to
> > >> > hbase
> > >> > > > >
> > >> > > > > I am looking for tool that allows me to import data from mysql
> > to
> > >> > > hbase.
> > >> > > > Any
> > >> > > > > suggestion?
> > >> > > > > Thank you :)
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Re: is there any tool that facilitate the import of data to hbase

Posted by Weishung Chung <we...@gmail.com>.
Got lotsa reading to do tonite :D

On Thu, Feb 3, 2011 at 4:17 PM, Dani Rayan <da...@gmail.com> wrote:

> Hey,
>
> Flume: http://archive.cloudera.com/cdh/3/flume/UserGuide.html has a HBase
> sink.
> Flume is similar to Scribe, but has more to it.
>
> Original Jira: https://issues.cloudera.org/browse/FLUME-6
> Branch: https://github.com/cloudera/flume/tree/hbase
>
> -Thanks,
> Dani.
> http://www.cc.gatech.edu/~iar3/
>
> On Thu, Feb 3, 2011 at 5:11 PM, arvind@cloudera.com <arvind@cloudera.com
> >wrote:
>
> > On Thu, Feb 3, 2011 at 1:56 PM, Weishung Chung <we...@gmail.com>
> wrote:
> >
> > > About Sqoop, could I import the data specifically into HBase ?
> > >
> >
> > Yes - Sqoop supports direct imports from external databases into HBase.
> > More
> > details can be found in Sqoop documentation available
> > here<
> >
> http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html#_importing_data_into_hbase
> > >
> > .
> >
> > Arvind
> >
> >
> > > I know I could write a program to read from mysql and use HBase API to
> > > write
> > > to HBase.
> > >
> > > On Thu, Feb 3, 2011 at 3:49 PM, Weishung Chung <we...@gmail.com>
> > wrote:
> > >
> > > > thank you for the clarification :) I am reading about sqoop now...
> > > >
> > > >
> > > > On Thu, Feb 3, 2011 at 3:41 PM, Mark Kerzner <markkerzner@gmail.com
> > > >wrote:
> > > >
> > > >> scribe is a tool for log aggregation, at face value, not mysql
> > > >>
> > > >> On Thu, Feb 3, 2011 at 3:37 PM, Weishung Chung <we...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Thank you for all the quick response. This really helps me along.
> > How
> > > >> about
> > > >> > facebook's scribe?
> > > >> >
> > > >> > On Thu, Feb 3, 2011 at 3:35 PM, Mark Kerzner <
> markkerzner@gmail.com
> > >
> > > >> > wrote:
> > > >> >
> > > >> > > Overview:
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> http://hadoopinpractice.blogspot.com/2011/01/loading-data-from-mysql-to-hadoop.html
> > > >> > >
> > > >> > > On Thu, Feb 3, 2011 at 3:28 PM, Ryan Rawson <ryanobjc@gmail.com
> >
> > > >> wrote:
> > > >> > >
> > > >> > > > ImportTSV?
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/ImportTsv.html
> > > >> > > >
> > > >> > > > Also writing a job to read from JDBC and write to hbase isnt
> too
> > > bad
> > > >> > > > if your schema isnt too insanely complex.
> > > >> > > >
> > > >> > > > -ryan
> > > >> > > >
> > > >> > > > On Thu, Feb 3, 2011 at 1:23 PM, Buttler, David <
> > buttler1@llnl.gov
> > > >
> > > >> > > wrote:
> > > >> > > > > Sqoop?
> > > >> > > > > http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html
> > > >> > > > >
> > > >> > > > > -----Original Message-----
> > > >> > > > > From: Weishung Chung [mailto:weishung@gmail.com]
> > > >> > > > > Sent: Thursday, February 03, 2011 1:18 PM
> > > >> > > > > To: user@hbase.apache.org
> > > >> > > > > Subject: is there any tool that facilitate the import of
> data
> > to
> > > >> > hbase
> > > >> > > > >
> > > >> > > > > I am looking for tool that allows me to import data from
> mysql
> > > to
> > > >> > > hbase.
> > > >> > > > Any
> > > >> > > > > suggestion?
> > > >> > > > > Thank you :)
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>

Re: is there any tool that facilitate the import of data to hbase

Posted by Dani Rayan <da...@gmail.com>.
Hey,

Flume: http://archive.cloudera.com/cdh/3/flume/UserGuide.html has a HBase
sink.
Flume is similar to Scribe, but has more to it.

Original Jira: https://issues.cloudera.org/browse/FLUME-6
Branch: https://github.com/cloudera/flume/tree/hbase

-Thanks,
Dani.
http://www.cc.gatech.edu/~iar3/

On Thu, Feb 3, 2011 at 5:11 PM, arvind@cloudera.com <ar...@cloudera.com>wrote:

> On Thu, Feb 3, 2011 at 1:56 PM, Weishung Chung <we...@gmail.com> wrote:
>
> > About Sqoop, could I import the data specifically into HBase ?
> >
>
> Yes - Sqoop supports direct imports from external databases into HBase.
> More
> details can be found in Sqoop documentation available
> here<
> http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html#_importing_data_into_hbase
> >
> .
>
> Arvind
>
>
> > I know I could write a program to read from mysql and use HBase API to
> > write
> > to HBase.
> >
> > On Thu, Feb 3, 2011 at 3:49 PM, Weishung Chung <we...@gmail.com>
> wrote:
> >
> > > thank you for the clarification :) I am reading about sqoop now...
> > >
> > >
> > > On Thu, Feb 3, 2011 at 3:41 PM, Mark Kerzner <markkerzner@gmail.com
> > >wrote:
> > >
> > >> scribe is a tool for log aggregation, at face value, not mysql
> > >>
> > >> On Thu, Feb 3, 2011 at 3:37 PM, Weishung Chung <we...@gmail.com>
> > >> wrote:
> > >>
> > >> > Thank you for all the quick response. This really helps me along.
> How
> > >> about
> > >> > facebook's scribe?
> > >> >
> > >> > On Thu, Feb 3, 2011 at 3:35 PM, Mark Kerzner <markkerzner@gmail.com
> >
> > >> > wrote:
> > >> >
> > >> > > Overview:
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> http://hadoopinpractice.blogspot.com/2011/01/loading-data-from-mysql-to-hadoop.html
> > >> > >
> > >> > > On Thu, Feb 3, 2011 at 3:28 PM, Ryan Rawson <ry...@gmail.com>
> > >> wrote:
> > >> > >
> > >> > > > ImportTSV?
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/ImportTsv.html
> > >> > > >
> > >> > > > Also writing a job to read from JDBC and write to hbase isnt too
> > bad
> > >> > > > if your schema isnt too insanely complex.
> > >> > > >
> > >> > > > -ryan
> > >> > > >
> > >> > > > On Thu, Feb 3, 2011 at 1:23 PM, Buttler, David <
> buttler1@llnl.gov
> > >
> > >> > > wrote:
> > >> > > > > Sqoop?
> > >> > > > > http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html
> > >> > > > >
> > >> > > > > -----Original Message-----
> > >> > > > > From: Weishung Chung [mailto:weishung@gmail.com]
> > >> > > > > Sent: Thursday, February 03, 2011 1:18 PM
> > >> > > > > To: user@hbase.apache.org
> > >> > > > > Subject: is there any tool that facilitate the import of data
> to
> > >> > hbase
> > >> > > > >
> > >> > > > > I am looking for tool that allows me to import data from mysql
> > to
> > >> > > hbase.
> > >> > > > Any
> > >> > > > > suggestion?
> > >> > > > > Thank you :)
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Re: is there any tool that facilitate the import of data to hbase

Posted by "arvind@cloudera.com" <ar...@cloudera.com>.
On Thu, Feb 3, 2011 at 1:56 PM, Weishung Chung <we...@gmail.com> wrote:

> About Sqoop, could I import the data specifically into HBase ?
>

Yes - Sqoop supports direct imports from external databases into HBase. More
details can be found in Sqoop documentation available
here<http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html#_importing_data_into_hbase>
.

Arvind


> I know I could write a program to read from mysql and use HBase API to
> write
> to HBase.
>
> On Thu, Feb 3, 2011 at 3:49 PM, Weishung Chung <we...@gmail.com> wrote:
>
> > thank you for the clarification :) I am reading about sqoop now...
> >
> >
> > On Thu, Feb 3, 2011 at 3:41 PM, Mark Kerzner <markkerzner@gmail.com
> >wrote:
> >
> >> scribe is a tool for log aggregation, at face value, not mysql
> >>
> >> On Thu, Feb 3, 2011 at 3:37 PM, Weishung Chung <we...@gmail.com>
> >> wrote:
> >>
> >> > Thank you for all the quick response. This really helps me along. How
> >> about
> >> > facebook's scribe?
> >> >
> >> > On Thu, Feb 3, 2011 at 3:35 PM, Mark Kerzner <ma...@gmail.com>
> >> > wrote:
> >> >
> >> > > Overview:
> >> > >
> >> > >
> >> >
> >>
> http://hadoopinpractice.blogspot.com/2011/01/loading-data-from-mysql-to-hadoop.html
> >> > >
> >> > > On Thu, Feb 3, 2011 at 3:28 PM, Ryan Rawson <ry...@gmail.com>
> >> wrote:
> >> > >
> >> > > > ImportTSV?
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/ImportTsv.html
> >> > > >
> >> > > > Also writing a job to read from JDBC and write to hbase isnt too
> bad
> >> > > > if your schema isnt too insanely complex.
> >> > > >
> >> > > > -ryan
> >> > > >
> >> > > > On Thu, Feb 3, 2011 at 1:23 PM, Buttler, David <buttler1@llnl.gov
> >
> >> > > wrote:
> >> > > > > Sqoop?
> >> > > > > http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html
> >> > > > >
> >> > > > > -----Original Message-----
> >> > > > > From: Weishung Chung [mailto:weishung@gmail.com]
> >> > > > > Sent: Thursday, February 03, 2011 1:18 PM
> >> > > > > To: user@hbase.apache.org
> >> > > > > Subject: is there any tool that facilitate the import of data to
> >> > hbase
> >> > > > >
> >> > > > > I am looking for tool that allows me to import data from mysql
> to
> >> > > hbase.
> >> > > > Any
> >> > > > > suggestion?
> >> > > > > Thank you :)
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Re: is there any tool that facilitate the import of data to hbase

Posted by Weishung Chung <we...@gmail.com>.
About Sqoop, could I import the data specifically into HBase ?
I know I could write a program to read from mysql and use HBase API to write
to HBase.

On Thu, Feb 3, 2011 at 3:49 PM, Weishung Chung <we...@gmail.com> wrote:

> thank you for the clarification :) I am reading about sqoop now...
>
>
> On Thu, Feb 3, 2011 at 3:41 PM, Mark Kerzner <ma...@gmail.com>wrote:
>
>> scribe is a tool for log aggregation, at face value, not mysql
>>
>> On Thu, Feb 3, 2011 at 3:37 PM, Weishung Chung <we...@gmail.com>
>> wrote:
>>
>> > Thank you for all the quick response. This really helps me along. How
>> about
>> > facebook's scribe?
>> >
>> > On Thu, Feb 3, 2011 at 3:35 PM, Mark Kerzner <ma...@gmail.com>
>> > wrote:
>> >
>> > > Overview:
>> > >
>> > >
>> >
>> http://hadoopinpractice.blogspot.com/2011/01/loading-data-from-mysql-to-hadoop.html
>> > >
>> > > On Thu, Feb 3, 2011 at 3:28 PM, Ryan Rawson <ry...@gmail.com>
>> wrote:
>> > >
>> > > > ImportTSV?
>> > > >
>> > > >
>> > > >
>> > >
>> >
>> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/ImportTsv.html
>> > > >
>> > > > Also writing a job to read from JDBC and write to hbase isnt too bad
>> > > > if your schema isnt too insanely complex.
>> > > >
>> > > > -ryan
>> > > >
>> > > > On Thu, Feb 3, 2011 at 1:23 PM, Buttler, David <bu...@llnl.gov>
>> > > wrote:
>> > > > > Sqoop?
>> > > > > http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html
>> > > > >
>> > > > > -----Original Message-----
>> > > > > From: Weishung Chung [mailto:weishung@gmail.com]
>> > > > > Sent: Thursday, February 03, 2011 1:18 PM
>> > > > > To: user@hbase.apache.org
>> > > > > Subject: is there any tool that facilitate the import of data to
>> > hbase
>> > > > >
>> > > > > I am looking for tool that allows me to import data from mysql to
>> > > hbase.
>> > > > Any
>> > > > > suggestion?
>> > > > > Thank you :)
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Re: is there any tool that facilitate the import of data to hbase

Posted by Weishung Chung <we...@gmail.com>.
thank you for the clarification :) I am reading about sqoop now...

On Thu, Feb 3, 2011 at 3:41 PM, Mark Kerzner <ma...@gmail.com> wrote:

> scribe is a tool for log aggregation, at face value, not mysql
>
> On Thu, Feb 3, 2011 at 3:37 PM, Weishung Chung <we...@gmail.com> wrote:
>
> > Thank you for all the quick response. This really helps me along. How
> about
> > facebook's scribe?
> >
> > On Thu, Feb 3, 2011 at 3:35 PM, Mark Kerzner <ma...@gmail.com>
> > wrote:
> >
> > > Overview:
> > >
> > >
> >
> http://hadoopinpractice.blogspot.com/2011/01/loading-data-from-mysql-to-hadoop.html
> > >
> > > On Thu, Feb 3, 2011 at 3:28 PM, Ryan Rawson <ry...@gmail.com>
> wrote:
> > >
> > > > ImportTSV?
> > > >
> > > >
> > > >
> > >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/ImportTsv.html
> > > >
> > > > Also writing a job to read from JDBC and write to hbase isnt too bad
> > > > if your schema isnt too insanely complex.
> > > >
> > > > -ryan
> > > >
> > > > On Thu, Feb 3, 2011 at 1:23 PM, Buttler, David <bu...@llnl.gov>
> > > wrote:
> > > > > Sqoop?
> > > > > http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html
> > > > >
> > > > > -----Original Message-----
> > > > > From: Weishung Chung [mailto:weishung@gmail.com]
> > > > > Sent: Thursday, February 03, 2011 1:18 PM
> > > > > To: user@hbase.apache.org
> > > > > Subject: is there any tool that facilitate the import of data to
> > hbase
> > > > >
> > > > > I am looking for tool that allows me to import data from mysql to
> > > hbase.
> > > > Any
> > > > > suggestion?
> > > > > Thank you :)
> > > > >
> > > >
> > >
> >
>

Re: is there any tool that facilitate the import of data to hbase

Posted by Mark Kerzner <ma...@gmail.com>.
scribe is a tool for log aggregation, at face value, not mysql

On Thu, Feb 3, 2011 at 3:37 PM, Weishung Chung <we...@gmail.com> wrote:

> Thank you for all the quick response. This really helps me along. How about
> facebook's scribe?
>
> On Thu, Feb 3, 2011 at 3:35 PM, Mark Kerzner <ma...@gmail.com>
> wrote:
>
> > Overview:
> >
> >
> http://hadoopinpractice.blogspot.com/2011/01/loading-data-from-mysql-to-hadoop.html
> >
> > On Thu, Feb 3, 2011 at 3:28 PM, Ryan Rawson <ry...@gmail.com> wrote:
> >
> > > ImportTSV?
> > >
> > >
> > >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/ImportTsv.html
> > >
> > > Also writing a job to read from JDBC and write to hbase isnt too bad
> > > if your schema isnt too insanely complex.
> > >
> > > -ryan
> > >
> > > On Thu, Feb 3, 2011 at 1:23 PM, Buttler, David <bu...@llnl.gov>
> > wrote:
> > > > Sqoop?
> > > > http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html
> > > >
> > > > -----Original Message-----
> > > > From: Weishung Chung [mailto:weishung@gmail.com]
> > > > Sent: Thursday, February 03, 2011 1:18 PM
> > > > To: user@hbase.apache.org
> > > > Subject: is there any tool that facilitate the import of data to
> hbase
> > > >
> > > > I am looking for tool that allows me to import data from mysql to
> > hbase.
> > > Any
> > > > suggestion?
> > > > Thank you :)
> > > >
> > >
> >
>

Re: is there any tool that facilitate the import of data to hbase

Posted by Weishung Chung <we...@gmail.com>.
Thank you for all the quick response. This really helps me along. How about
facebook's scribe?

On Thu, Feb 3, 2011 at 3:35 PM, Mark Kerzner <ma...@gmail.com> wrote:

> Overview:
>
> http://hadoopinpractice.blogspot.com/2011/01/loading-data-from-mysql-to-hadoop.html
>
> On Thu, Feb 3, 2011 at 3:28 PM, Ryan Rawson <ry...@gmail.com> wrote:
>
> > ImportTSV?
> >
> >
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/ImportTsv.html
> >
> > Also writing a job to read from JDBC and write to hbase isnt too bad
> > if your schema isnt too insanely complex.
> >
> > -ryan
> >
> > On Thu, Feb 3, 2011 at 1:23 PM, Buttler, David <bu...@llnl.gov>
> wrote:
> > > Sqoop?
> > > http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html
> > >
> > > -----Original Message-----
> > > From: Weishung Chung [mailto:weishung@gmail.com]
> > > Sent: Thursday, February 03, 2011 1:18 PM
> > > To: user@hbase.apache.org
> > > Subject: is there any tool that facilitate the import of data to hbase
> > >
> > > I am looking for tool that allows me to import data from mysql to
> hbase.
> > Any
> > > suggestion?
> > > Thank you :)
> > >
> >
>

Re: is there any tool that facilitate the import of data to hbase

Posted by Mark Kerzner <ma...@gmail.com>.
Overview:
http://hadoopinpractice.blogspot.com/2011/01/loading-data-from-mysql-to-hadoop.html

On Thu, Feb 3, 2011 at 3:28 PM, Ryan Rawson <ry...@gmail.com> wrote:

> ImportTSV?
>
>
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/ImportTsv.html
>
> Also writing a job to read from JDBC and write to hbase isnt too bad
> if your schema isnt too insanely complex.
>
> -ryan
>
> On Thu, Feb 3, 2011 at 1:23 PM, Buttler, David <bu...@llnl.gov> wrote:
> > Sqoop?
> > http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html
> >
> > -----Original Message-----
> > From: Weishung Chung [mailto:weishung@gmail.com]
> > Sent: Thursday, February 03, 2011 1:18 PM
> > To: user@hbase.apache.org
> > Subject: is there any tool that facilitate the import of data to hbase
> >
> > I am looking for tool that allows me to import data from mysql to hbase.
> Any
> > suggestion?
> > Thank you :)
> >
>

Re: is there any tool that facilitate the import of data to hbase

Posted by Ryan Rawson <ry...@gmail.com>.
ImportTSV?

http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/ImportTsv.html

Also writing a job to read from JDBC and write to hbase isnt too bad
if your schema isnt too insanely complex.

-ryan

On Thu, Feb 3, 2011 at 1:23 PM, Buttler, David <bu...@llnl.gov> wrote:
> Sqoop?
> http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html
>
> -----Original Message-----
> From: Weishung Chung [mailto:weishung@gmail.com]
> Sent: Thursday, February 03, 2011 1:18 PM
> To: user@hbase.apache.org
> Subject: is there any tool that facilitate the import of data to hbase
>
> I am looking for tool that allows me to import data from mysql to hbase. Any
> suggestion?
> Thank you :)
>

RE: is there any tool that facilitate the import of data to hbase

Posted by "Buttler, David" <bu...@llnl.gov>.
Sqoop?
http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html

-----Original Message-----
From: Weishung Chung [mailto:weishung@gmail.com] 
Sent: Thursday, February 03, 2011 1:18 PM
To: user@hbase.apache.org
Subject: is there any tool that facilitate the import of data to hbase

I am looking for tool that allows me to import data from mysql to hbase. Any
suggestion?
Thank you :)