You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Fatih Haltas <fa...@nyu.edu> on 2013/07/10 09:59:48 UTC
Sqoop and Hadoop
Hi Everyone,
I am trying to import data from postgresql to hdfs via sqoop, however, all
examples, i got on internet is talking about hive,hbase etc. kind of
system,running within hadoop.
I am not using, any of these systems, isnt it possible to import data
without having those kind of systems,running on hadoop via sqoop?
In other words, I am using hadoop and mapreduce systems itself alone, is it
possible to import data from postgresql to that basic hadoop system via
sqoop?
Re: Sqoop and Hadoop
Posted by Fatih Haltas <fa...@nyu.edu>.
Thanks you all so much.
On Wed, Jul 10, 2013 at 12:09 PM, Alexander Alten-Lorenz <
wget.null@gmail.com> wrote:
> Moving to user@sqoop.apache.org
>
> For the original question - ONE! google search (sqoop postgres hdfs):
>
> http://alexkehayias.tumblr.com/post/44153307024/importing-postgres-data-into-hadoop-hdfs
>
> Regards
>
> On Jul 10, 2013, at 9:59 AM, Fatih Haltas <fa...@nyu.edu> wrote:
>
> > Hi Everyone,
> >
> > I am trying to import data from postgresql to hdfs via sqoop, however,
> all examples, i got on internet is talking about hive,hbase etc. kind of
> system,running within hadoop.
> >
> > I am not using, any of these systems, isnt it possible to import data
> without having those kind of systems,running on hadoop via sqoop?
> >
> > In other words, I am using hadoop and mapreduce systems itself alone, is
> it possible to import data from postgresql to that basic hadoop system via
> sqoop?
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>
Re: Sqoop and Hadoop
Posted by Fatih Haltas <fa...@nyu.edu>.
Thanks you all so much.
On Wed, Jul 10, 2013 at 12:09 PM, Alexander Alten-Lorenz <
wget.null@gmail.com> wrote:
> Moving to user@sqoop.apache.org
>
> For the original question - ONE! google search (sqoop postgres hdfs):
>
> http://alexkehayias.tumblr.com/post/44153307024/importing-postgres-data-into-hadoop-hdfs
>
> Regards
>
> On Jul 10, 2013, at 9:59 AM, Fatih Haltas <fa...@nyu.edu> wrote:
>
> > Hi Everyone,
> >
> > I am trying to import data from postgresql to hdfs via sqoop, however,
> all examples, i got on internet is talking about hive,hbase etc. kind of
> system,running within hadoop.
> >
> > I am not using, any of these systems, isnt it possible to import data
> without having those kind of systems,running on hadoop via sqoop?
> >
> > In other words, I am using hadoop and mapreduce systems itself alone, is
> it possible to import data from postgresql to that basic hadoop system via
> sqoop?
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>
Re: Sqoop and Hadoop
Posted by Fatih Haltas <fa...@nyu.edu>.
Thanks you all so much.
On Wed, Jul 10, 2013 at 12:09 PM, Alexander Alten-Lorenz <
wget.null@gmail.com> wrote:
> Moving to user@sqoop.apache.org
>
> For the original question - ONE! google search (sqoop postgres hdfs):
>
> http://alexkehayias.tumblr.com/post/44153307024/importing-postgres-data-into-hadoop-hdfs
>
> Regards
>
> On Jul 10, 2013, at 9:59 AM, Fatih Haltas <fa...@nyu.edu> wrote:
>
> > Hi Everyone,
> >
> > I am trying to import data from postgresql to hdfs via sqoop, however,
> all examples, i got on internet is talking about hive,hbase etc. kind of
> system,running within hadoop.
> >
> > I am not using, any of these systems, isnt it possible to import data
> without having those kind of systems,running on hadoop via sqoop?
> >
> > In other words, I am using hadoop and mapreduce systems itself alone, is
> it possible to import data from postgresql to that basic hadoop system via
> sqoop?
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>
Re: Sqoop and Hadoop
Posted by Fatih Haltas <fa...@nyu.edu>.
Thanks you all so much.
On Wed, Jul 10, 2013 at 12:09 PM, Alexander Alten-Lorenz <
wget.null@gmail.com> wrote:
> Moving to user@sqoop.apache.org
>
> For the original question - ONE! google search (sqoop postgres hdfs):
>
> http://alexkehayias.tumblr.com/post/44153307024/importing-postgres-data-into-hadoop-hdfs
>
> Regards
>
> On Jul 10, 2013, at 9:59 AM, Fatih Haltas <fa...@nyu.edu> wrote:
>
> > Hi Everyone,
> >
> > I am trying to import data from postgresql to hdfs via sqoop, however,
> all examples, i got on internet is talking about hive,hbase etc. kind of
> system,running within hadoop.
> >
> > I am not using, any of these systems, isnt it possible to import data
> without having those kind of systems,running on hadoop via sqoop?
> >
> > In other words, I am using hadoop and mapreduce systems itself alone, is
> it possible to import data from postgresql to that basic hadoop system via
> sqoop?
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>
Re: Sqoop and Hadoop
Posted by Alexander Alten-Lorenz <wg...@gmail.com>.
Moving to user@sqoop.apache.org
For the original question - ONE! google search (sqoop postgres hdfs):
http://alexkehayias.tumblr.com/post/44153307024/importing-postgres-data-into-hadoop-hdfs
Regards
On Jul 10, 2013, at 9:59 AM, Fatih Haltas <fa...@nyu.edu> wrote:
> Hi Everyone,
>
> I am trying to import data from postgresql to hdfs via sqoop, however, all examples, i got on internet is talking about hive,hbase etc. kind of system,running within hadoop.
>
> I am not using, any of these systems, isnt it possible to import data without having those kind of systems,running on hadoop via sqoop?
>
> In other words, I am using hadoop and mapreduce systems itself alone, is it possible to import data from postgresql to that basic hadoop system via sqoop?
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: Sqoop and Hadoop
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi,
You don't need hive nor hbase. A basic hadoop system (hdfs + mapreduce) is
enough.
I believe the documentation is well done.
If you have further questions, you should ask the correct mailing list.
http://sqoop.apache.org/mail-lists.html
Bertrand
On Wed, Jul 10, 2013 at 10:05 AM, Nitin Pawar <ni...@gmail.com>wrote:
> why not?
> you can use sqoop to import to plain text files or avrofiles or squence
> files
> here is one example
>
> sqoop import
> --connect <conn>
> --username <user>
> -P
> --table <table>
> --columns "column1,column2,column3,.."
> --as-textfile
> --target-dir <hdfs dir>
> -m 1
>
>
>
> On Wed, Jul 10, 2013 at 1:29 PM, Fatih Haltas <fa...@nyu.edu>wrote:
>
>> Hi Everyone,
>>
>> I am trying to import data from postgresql to hdfs via sqoop, however,
>> all examples, i got on internet is talking about hive,hbase etc. kind of
>> system,running within hadoop.
>>
>> I am not using, any of these systems, isnt it possible to import data
>> without having those kind of systems,running on hadoop via sqoop?
>>
>> In other words, I am using hadoop and mapreduce systems itself alone, is
>> it possible to import data from postgresql to that basic hadoop system via
>> sqoop?
>>
>
>
>
> --
> Nitin Pawar
>
--
Bertrand Dechoux
Re: Sqoop and Hadoop
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi,
You don't need hive nor hbase. A basic hadoop system (hdfs + mapreduce) is
enough.
I believe the documentation is well done.
If you have further questions, you should ask the correct mailing list.
http://sqoop.apache.org/mail-lists.html
Bertrand
On Wed, Jul 10, 2013 at 10:05 AM, Nitin Pawar <ni...@gmail.com>wrote:
> why not?
> you can use sqoop to import to plain text files or avrofiles or squence
> files
> here is one example
>
> sqoop import
> --connect <conn>
> --username <user>
> -P
> --table <table>
> --columns "column1,column2,column3,.."
> --as-textfile
> --target-dir <hdfs dir>
> -m 1
>
>
>
> On Wed, Jul 10, 2013 at 1:29 PM, Fatih Haltas <fa...@nyu.edu>wrote:
>
>> Hi Everyone,
>>
>> I am trying to import data from postgresql to hdfs via sqoop, however,
>> all examples, i got on internet is talking about hive,hbase etc. kind of
>> system,running within hadoop.
>>
>> I am not using, any of these systems, isnt it possible to import data
>> without having those kind of systems,running on hadoop via sqoop?
>>
>> In other words, I am using hadoop and mapreduce systems itself alone, is
>> it possible to import data from postgresql to that basic hadoop system via
>> sqoop?
>>
>
>
>
> --
> Nitin Pawar
>
--
Bertrand Dechoux
Re: Sqoop and Hadoop
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi,
You don't need hive nor hbase. A basic hadoop system (hdfs + mapreduce) is
enough.
I believe the documentation is well done.
If you have further questions, you should ask the correct mailing list.
http://sqoop.apache.org/mail-lists.html
Bertrand
On Wed, Jul 10, 2013 at 10:05 AM, Nitin Pawar <ni...@gmail.com>wrote:
> why not?
> you can use sqoop to import to plain text files or avrofiles or squence
> files
> here is one example
>
> sqoop import
> --connect <conn>
> --username <user>
> -P
> --table <table>
> --columns "column1,column2,column3,.."
> --as-textfile
> --target-dir <hdfs dir>
> -m 1
>
>
>
> On Wed, Jul 10, 2013 at 1:29 PM, Fatih Haltas <fa...@nyu.edu>wrote:
>
>> Hi Everyone,
>>
>> I am trying to import data from postgresql to hdfs via sqoop, however,
>> all examples, i got on internet is talking about hive,hbase etc. kind of
>> system,running within hadoop.
>>
>> I am not using, any of these systems, isnt it possible to import data
>> without having those kind of systems,running on hadoop via sqoop?
>>
>> In other words, I am using hadoop and mapreduce systems itself alone, is
>> it possible to import data from postgresql to that basic hadoop system via
>> sqoop?
>>
>
>
>
> --
> Nitin Pawar
>
--
Bertrand Dechoux
Re: Sqoop and Hadoop
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi,
You don't need hive nor hbase. A basic hadoop system (hdfs + mapreduce) is
enough.
I believe the documentation is well done.
If you have further questions, you should ask the correct mailing list.
http://sqoop.apache.org/mail-lists.html
Bertrand
On Wed, Jul 10, 2013 at 10:05 AM, Nitin Pawar <ni...@gmail.com>wrote:
> why not?
> you can use sqoop to import to plain text files or avrofiles or squence
> files
> here is one example
>
> sqoop import
> --connect <conn>
> --username <user>
> -P
> --table <table>
> --columns "column1,column2,column3,.."
> --as-textfile
> --target-dir <hdfs dir>
> -m 1
>
>
>
> On Wed, Jul 10, 2013 at 1:29 PM, Fatih Haltas <fa...@nyu.edu>wrote:
>
>> Hi Everyone,
>>
>> I am trying to import data from postgresql to hdfs via sqoop, however,
>> all examples, i got on internet is talking about hive,hbase etc. kind of
>> system,running within hadoop.
>>
>> I am not using, any of these systems, isnt it possible to import data
>> without having those kind of systems,running on hadoop via sqoop?
>>
>> In other words, I am using hadoop and mapreduce systems itself alone, is
>> it possible to import data from postgresql to that basic hadoop system via
>> sqoop?
>>
>
>
>
> --
> Nitin Pawar
>
--
Bertrand Dechoux
Re: Sqoop and Hadoop
Posted by Nitin Pawar <ni...@gmail.com>.
why not?
you can use sqoop to import to plain text files or avrofiles or squence
files
here is one example
sqoop import
--connect <conn>
--username <user>
-P
--table <table>
--columns "column1,column2,column3,.."
--as-textfile
--target-dir <hdfs dir>
-m 1
On Wed, Jul 10, 2013 at 1:29 PM, Fatih Haltas <fa...@nyu.edu> wrote:
> Hi Everyone,
>
> I am trying to import data from postgresql to hdfs via sqoop, however, all
> examples, i got on internet is talking about hive,hbase etc. kind of
> system,running within hadoop.
>
> I am not using, any of these systems, isnt it possible to import data
> without having those kind of systems,running on hadoop via sqoop?
>
> In other words, I am using hadoop and mapreduce systems itself alone, is
> it possible to import data from postgresql to that basic hadoop system via
> sqoop?
>
--
Nitin Pawar
Re: Sqoop and Hadoop
Posted by Alexander Alten-Lorenz <wg...@gmail.com>.
Moving to user@sqoop.apache.org
For the original question - ONE! google search (sqoop postgres hdfs):
http://alexkehayias.tumblr.com/post/44153307024/importing-postgres-data-into-hadoop-hdfs
Regards
On Jul 10, 2013, at 9:59 AM, Fatih Haltas <fa...@nyu.edu> wrote:
> Hi Everyone,
>
> I am trying to import data from postgresql to hdfs via sqoop, however, all examples, i got on internet is talking about hive,hbase etc. kind of system,running within hadoop.
>
> I am not using, any of these systems, isnt it possible to import data without having those kind of systems,running on hadoop via sqoop?
>
> In other words, I am using hadoop and mapreduce systems itself alone, is it possible to import data from postgresql to that basic hadoop system via sqoop?
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: Sqoop and Hadoop
Posted by Nitin Pawar <ni...@gmail.com>.
why not?
you can use sqoop to import to plain text files or avrofiles or squence
files
here is one example
sqoop import
--connect <conn>
--username <user>
-P
--table <table>
--columns "column1,column2,column3,.."
--as-textfile
--target-dir <hdfs dir>
-m 1
On Wed, Jul 10, 2013 at 1:29 PM, Fatih Haltas <fa...@nyu.edu> wrote:
> Hi Everyone,
>
> I am trying to import data from postgresql to hdfs via sqoop, however, all
> examples, i got on internet is talking about hive,hbase etc. kind of
> system,running within hadoop.
>
> I am not using, any of these systems, isnt it possible to import data
> without having those kind of systems,running on hadoop via sqoop?
>
> In other words, I am using hadoop and mapreduce systems itself alone, is
> it possible to import data from postgresql to that basic hadoop system via
> sqoop?
>
--
Nitin Pawar
Re: Sqoop and Hadoop
Posted by Alexander Alten-Lorenz <wg...@gmail.com>.
Moving to user@sqoop.apache.org
For the original question - ONE! google search (sqoop postgres hdfs):
http://alexkehayias.tumblr.com/post/44153307024/importing-postgres-data-into-hadoop-hdfs
Regards
On Jul 10, 2013, at 9:59 AM, Fatih Haltas <fa...@nyu.edu> wrote:
> Hi Everyone,
>
> I am trying to import data from postgresql to hdfs via sqoop, however, all examples, i got on internet is talking about hive,hbase etc. kind of system,running within hadoop.
>
> I am not using, any of these systems, isnt it possible to import data without having those kind of systems,running on hadoop via sqoop?
>
> In other words, I am using hadoop and mapreduce systems itself alone, is it possible to import data from postgresql to that basic hadoop system via sqoop?
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: Sqoop and Hadoop
Posted by Nitin Pawar <ni...@gmail.com>.
why not?
you can use sqoop to import to plain text files or avrofiles or squence
files
here is one example
sqoop import
--connect <conn>
--username <user>
-P
--table <table>
--columns "column1,column2,column3,.."
--as-textfile
--target-dir <hdfs dir>
-m 1
On Wed, Jul 10, 2013 at 1:29 PM, Fatih Haltas <fa...@nyu.edu> wrote:
> Hi Everyone,
>
> I am trying to import data from postgresql to hdfs via sqoop, however, all
> examples, i got on internet is talking about hive,hbase etc. kind of
> system,running within hadoop.
>
> I am not using, any of these systems, isnt it possible to import data
> without having those kind of systems,running on hadoop via sqoop?
>
> In other words, I am using hadoop and mapreduce systems itself alone, is
> it possible to import data from postgresql to that basic hadoop system via
> sqoop?
>
--
Nitin Pawar
Re: Sqoop and Hadoop
Posted by Alexander Alten-Lorenz <wg...@gmail.com>.
Moving to user@sqoop.apache.org
For the original question - ONE! google search (sqoop postgres hdfs):
http://alexkehayias.tumblr.com/post/44153307024/importing-postgres-data-into-hadoop-hdfs
Regards
On Jul 10, 2013, at 9:59 AM, Fatih Haltas <fa...@nyu.edu> wrote:
> Hi Everyone,
>
> I am trying to import data from postgresql to hdfs via sqoop, however, all examples, i got on internet is talking about hive,hbase etc. kind of system,running within hadoop.
>
> I am not using, any of these systems, isnt it possible to import data without having those kind of systems,running on hadoop via sqoop?
>
> In other words, I am using hadoop and mapreduce systems itself alone, is it possible to import data from postgresql to that basic hadoop system via sqoop?
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: Sqoop and Hadoop
Posted by Nitin Pawar <ni...@gmail.com>.
why not?
you can use sqoop to import to plain text files or avrofiles or squence
files
here is one example
sqoop import
--connect <conn>
--username <user>
-P
--table <table>
--columns "column1,column2,column3,.."
--as-textfile
--target-dir <hdfs dir>
-m 1
On Wed, Jul 10, 2013 at 1:29 PM, Fatih Haltas <fa...@nyu.edu> wrote:
> Hi Everyone,
>
> I am trying to import data from postgresql to hdfs via sqoop, however, all
> examples, i got on internet is talking about hive,hbase etc. kind of
> system,running within hadoop.
>
> I am not using, any of these systems, isnt it possible to import data
> without having those kind of systems,running on hadoop via sqoop?
>
> In other words, I am using hadoop and mapreduce systems itself alone, is
> it possible to import data from postgresql to that basic hadoop system via
> sqoop?
>
--
Nitin Pawar
Re: Sqoop and Hadoop
Posted by Alexander Alten-Lorenz <wg...@gmail.com>.
Moving to user@sqoop.apache.org
For the original question - ONE! google search (sqoop postgres hdfs):
http://alexkehayias.tumblr.com/post/44153307024/importing-postgres-data-into-hadoop-hdfs
Regards
On Jul 10, 2013, at 9:59 AM, Fatih Haltas <fa...@nyu.edu> wrote:
> Hi Everyone,
>
> I am trying to import data from postgresql to hdfs via sqoop, however, all examples, i got on internet is talking about hive,hbase etc. kind of system,running within hadoop.
>
> I am not using, any of these systems, isnt it possible to import data without having those kind of systems,running on hadoop via sqoop?
>
> In other words, I am using hadoop and mapreduce systems itself alone, is it possible to import data from postgresql to that basic hadoop system via sqoop?
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF