You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Anil Jagtap <an...@gmail.com> on 2014/12/17 23:29:34 UTC
Copying files to hadoop.
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling
even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to
connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto
Hadoop. As per my knowledge i can copy those files first to Linux and then
put to Hadoop. But is there a way in which just in one command it will copy
to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
Re: Copying files to hadoop.
Posted by Wilm Schumacher <wi...@gmail.com>.
Hi,
but I want to point out: my solution works, but is not very smart. I
think Rich has the better answer.
If your file is REALLY huge, then his answer Nr. 1 is the way to go. If
you want to dynamically add and remove files, test this and test that,
than his second answer is a good fit. You just "mount" a shared folder
in linux, and than it's "local" in linux/unix.
My answer was more of an academic possibility ;).
Good luck
Wilm
Am 18.12.2014 um 00:16 schrieb Anil Jagtap:
> Oh Thanks a lot Wilm.. You understood my problem accurately.. I
> executed it and it worked.
>
> I understand i can always copy it to Linux and then put it to hadoop,
> but i was just trying to find out if this is possible.
>
> Thanks again.
>
> Rgds, Anil
>
> On Thu, Dec 18, 2014 at 9:56 AM, Wilm Schumacher
> <wilm.schumacher@gmail.com <ma...@gmail.com>> wrote:
>
> Am 17.12.2014 um 23:29 schrieb Anil Jagtap:
> > Dear All,
> >
> > I'm pretty new to Hadoop technology and Linux environment hence
> > struggling even to find solutions for the basic stuff.
> >
> > For now, Hortonworks Sandbox is working fine for me and i managed to
> > connect to it thru SSH.
> >
> > Now i have some csv files in my mac os folders which i want to copy
> > onto Hadoop. As per my knowledge i can copy those files first to
> Linux
> > and then put to Hadoop. But is there a way in which just in one
> > command it will copy to Hadoop directly from mac os folder?
> yes, there is.
>
> cat /path/to/your/local/file.csv | ssh hadoopuser@namenode
> "/remote/server/path/to/hadoop fs -put - /hadoop/folder/name/file.csv"
>
> As you wrote, that you are also new to linux/unix, this above means:
>
> * cat => concanate the files (only one file given) and print to
> standard
> output
>
> * pipe | => means, write the standard output from the left hand to the
> standard input of the right hand side
>
> * ssh reads from standard input and writes its to the standard
> input on
> the remote server command, which is hadoop fs put command, which
> is told
> to read from stdin
>
> Thus you are actually piping the content of the file through 3
> services.
> And that's actually a little bit of a hack and in my opinion there
> is no
> reason to do this if your file is reasonable small to fit on the
> remote
> server. It's like asking "is it possible to reach my destination only
> using left turns". Well ... it's possible, but not always a good
> idea ;).
>
> Best
>
> Wilm
>
Re: Copying files to hadoop.
Posted by Wilm Schumacher <wi...@gmail.com>.
Hi,
but I want to point out: my solution works, but is not very smart. I
think Rich has the better answer.
If your file is REALLY huge, then his answer Nr. 1 is the way to go. If
you want to dynamically add and remove files, test this and test that,
than his second answer is a good fit. You just "mount" a shared folder
in linux, and than it's "local" in linux/unix.
My answer was more of an academic possibility ;).
Good luck
Wilm
Am 18.12.2014 um 00:16 schrieb Anil Jagtap:
> Oh Thanks a lot Wilm.. You understood my problem accurately.. I
> executed it and it worked.
>
> I understand i can always copy it to Linux and then put it to hadoop,
> but i was just trying to find out if this is possible.
>
> Thanks again.
>
> Rgds, Anil
>
> On Thu, Dec 18, 2014 at 9:56 AM, Wilm Schumacher
> <wilm.schumacher@gmail.com <ma...@gmail.com>> wrote:
>
> Am 17.12.2014 um 23:29 schrieb Anil Jagtap:
> > Dear All,
> >
> > I'm pretty new to Hadoop technology and Linux environment hence
> > struggling even to find solutions for the basic stuff.
> >
> > For now, Hortonworks Sandbox is working fine for me and i managed to
> > connect to it thru SSH.
> >
> > Now i have some csv files in my mac os folders which i want to copy
> > onto Hadoop. As per my knowledge i can copy those files first to
> Linux
> > and then put to Hadoop. But is there a way in which just in one
> > command it will copy to Hadoop directly from mac os folder?
> yes, there is.
>
> cat /path/to/your/local/file.csv | ssh hadoopuser@namenode
> "/remote/server/path/to/hadoop fs -put - /hadoop/folder/name/file.csv"
>
> As you wrote, that you are also new to linux/unix, this above means:
>
> * cat => concanate the files (only one file given) and print to
> standard
> output
>
> * pipe | => means, write the standard output from the left hand to the
> standard input of the right hand side
>
> * ssh reads from standard input and writes its to the standard
> input on
> the remote server command, which is hadoop fs put command, which
> is told
> to read from stdin
>
> Thus you are actually piping the content of the file through 3
> services.
> And that's actually a little bit of a hack and in my opinion there
> is no
> reason to do this if your file is reasonable small to fit on the
> remote
> server. It's like asking "is it possible to reach my destination only
> using left turns". Well ... it's possible, but not always a good
> idea ;).
>
> Best
>
> Wilm
>
Re: Copying files to hadoop.
Posted by Wilm Schumacher <wi...@gmail.com>.
Hi,
but I want to point out: my solution works, but is not very smart. I
think Rich has the better answer.
If your file is REALLY huge, then his answer Nr. 1 is the way to go. If
you want to dynamically add and remove files, test this and test that,
than his second answer is a good fit. You just "mount" a shared folder
in linux, and than it's "local" in linux/unix.
My answer was more of an academic possibility ;).
Good luck
Wilm
Am 18.12.2014 um 00:16 schrieb Anil Jagtap:
> Oh Thanks a lot Wilm.. You understood my problem accurately.. I
> executed it and it worked.
>
> I understand i can always copy it to Linux and then put it to hadoop,
> but i was just trying to find out if this is possible.
>
> Thanks again.
>
> Rgds, Anil
>
> On Thu, Dec 18, 2014 at 9:56 AM, Wilm Schumacher
> <wilm.schumacher@gmail.com <ma...@gmail.com>> wrote:
>
> Am 17.12.2014 um 23:29 schrieb Anil Jagtap:
> > Dear All,
> >
> > I'm pretty new to Hadoop technology and Linux environment hence
> > struggling even to find solutions for the basic stuff.
> >
> > For now, Hortonworks Sandbox is working fine for me and i managed to
> > connect to it thru SSH.
> >
> > Now i have some csv files in my mac os folders which i want to copy
> > onto Hadoop. As per my knowledge i can copy those files first to
> Linux
> > and then put to Hadoop. But is there a way in which just in one
> > command it will copy to Hadoop directly from mac os folder?
> yes, there is.
>
> cat /path/to/your/local/file.csv | ssh hadoopuser@namenode
> "/remote/server/path/to/hadoop fs -put - /hadoop/folder/name/file.csv"
>
> As you wrote, that you are also new to linux/unix, this above means:
>
> * cat => concanate the files (only one file given) and print to
> standard
> output
>
> * pipe | => means, write the standard output from the left hand to the
> standard input of the right hand side
>
> * ssh reads from standard input and writes its to the standard
> input on
> the remote server command, which is hadoop fs put command, which
> is told
> to read from stdin
>
> Thus you are actually piping the content of the file through 3
> services.
> And that's actually a little bit of a hack and in my opinion there
> is no
> reason to do this if your file is reasonable small to fit on the
> remote
> server. It's like asking "is it possible to reach my destination only
> using left turns". Well ... it's possible, but not always a good
> idea ;).
>
> Best
>
> Wilm
>
Re: Copying files to hadoop.
Posted by Wilm Schumacher <wi...@gmail.com>.
Hi,
but I want to point out: my solution works, but is not very smart. I
think Rich has the better answer.
If your file is REALLY huge, then his answer Nr. 1 is the way to go. If
you want to dynamically add and remove files, test this and test that,
than his second answer is a good fit. You just "mount" a shared folder
in linux, and than it's "local" in linux/unix.
My answer was more of an academic possibility ;).
Good luck
Wilm
Am 18.12.2014 um 00:16 schrieb Anil Jagtap:
> Oh Thanks a lot Wilm.. You understood my problem accurately.. I
> executed it and it worked.
>
> I understand i can always copy it to Linux and then put it to hadoop,
> but i was just trying to find out if this is possible.
>
> Thanks again.
>
> Rgds, Anil
>
> On Thu, Dec 18, 2014 at 9:56 AM, Wilm Schumacher
> <wilm.schumacher@gmail.com <ma...@gmail.com>> wrote:
>
> Am 17.12.2014 um 23:29 schrieb Anil Jagtap:
> > Dear All,
> >
> > I'm pretty new to Hadoop technology and Linux environment hence
> > struggling even to find solutions for the basic stuff.
> >
> > For now, Hortonworks Sandbox is working fine for me and i managed to
> > connect to it thru SSH.
> >
> > Now i have some csv files in my mac os folders which i want to copy
> > onto Hadoop. As per my knowledge i can copy those files first to
> Linux
> > and then put to Hadoop. But is there a way in which just in one
> > command it will copy to Hadoop directly from mac os folder?
> yes, there is.
>
> cat /path/to/your/local/file.csv | ssh hadoopuser@namenode
> "/remote/server/path/to/hadoop fs -put - /hadoop/folder/name/file.csv"
>
> As you wrote, that you are also new to linux/unix, this above means:
>
> * cat => concanate the files (only one file given) and print to
> standard
> output
>
> * pipe | => means, write the standard output from the left hand to the
> standard input of the right hand side
>
> * ssh reads from standard input and writes its to the standard
> input on
> the remote server command, which is hadoop fs put command, which
> is told
> to read from stdin
>
> Thus you are actually piping the content of the file through 3
> services.
> And that's actually a little bit of a hack and in my opinion there
> is no
> reason to do this if your file is reasonable small to fit on the
> remote
> server. It's like asking "is it possible to reach my destination only
> using left turns". Well ... it's possible, but not always a good
> idea ;).
>
> Best
>
> Wilm
>
Re: Copying files to hadoop.
Posted by Anil Jagtap <an...@gmail.com>.
Oh Thanks a lot Wilm.. You understood my problem accurately.. I executed it
and it worked.
I understand i can always copy it to Linux and then put it to hadoop, but i
was just trying to find out if this is possible.
Thanks again.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:56 AM, Wilm Schumacher <wi...@gmail.com>
wrote:
>
> Am 17.12.2014 um 23:29 schrieb Anil Jagtap:
> > Dear All,
> >
> > I'm pretty new to Hadoop technology and Linux environment hence
> > struggling even to find solutions for the basic stuff.
> >
> > For now, Hortonworks Sandbox is working fine for me and i managed to
> > connect to it thru SSH.
> >
> > Now i have some csv files in my mac os folders which i want to copy
> > onto Hadoop. As per my knowledge i can copy those files first to Linux
> > and then put to Hadoop. But is there a way in which just in one
> > command it will copy to Hadoop directly from mac os folder?
> yes, there is.
>
> cat /path/to/your/local/file.csv | ssh hadoopuser@namenode
> "/remote/server/path/to/hadoop fs -put - /hadoop/folder/name/file.csv"
>
> As you wrote, that you are also new to linux/unix, this above means:
>
> * cat => concanate the files (only one file given) and print to standard
> output
>
> * pipe | => means, write the standard output from the left hand to the
> standard input of the right hand side
>
> * ssh reads from standard input and writes its to the standard input on
> the remote server command, which is hadoop fs put command, which is told
> to read from stdin
>
> Thus you are actually piping the content of the file through 3 services.
> And that's actually a little bit of a hack and in my opinion there is no
> reason to do this if your file is reasonable small to fit on the remote
> server. It's like asking "is it possible to reach my destination only
> using left turns". Well ... it's possible, but not always a good idea ;).
>
> Best
>
> Wilm
>
Re: Copying files to hadoop.
Posted by Anil Jagtap <an...@gmail.com>.
Oh Thanks a lot Wilm.. You understood my problem accurately.. I executed it
and it worked.
I understand i can always copy it to Linux and then put it to hadoop, but i
was just trying to find out if this is possible.
Thanks again.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:56 AM, Wilm Schumacher <wi...@gmail.com>
wrote:
>
> Am 17.12.2014 um 23:29 schrieb Anil Jagtap:
> > Dear All,
> >
> > I'm pretty new to Hadoop technology and Linux environment hence
> > struggling even to find solutions for the basic stuff.
> >
> > For now, Hortonworks Sandbox is working fine for me and i managed to
> > connect to it thru SSH.
> >
> > Now i have some csv files in my mac os folders which i want to copy
> > onto Hadoop. As per my knowledge i can copy those files first to Linux
> > and then put to Hadoop. But is there a way in which just in one
> > command it will copy to Hadoop directly from mac os folder?
> yes, there is.
>
> cat /path/to/your/local/file.csv | ssh hadoopuser@namenode
> "/remote/server/path/to/hadoop fs -put - /hadoop/folder/name/file.csv"
>
> As you wrote, that you are also new to linux/unix, this above means:
>
> * cat => concanate the files (only one file given) and print to standard
> output
>
> * pipe | => means, write the standard output from the left hand to the
> standard input of the right hand side
>
> * ssh reads from standard input and writes its to the standard input on
> the remote server command, which is hadoop fs put command, which is told
> to read from stdin
>
> Thus you are actually piping the content of the file through 3 services.
> And that's actually a little bit of a hack and in my opinion there is no
> reason to do this if your file is reasonable small to fit on the remote
> server. It's like asking "is it possible to reach my destination only
> using left turns". Well ... it's possible, but not always a good idea ;).
>
> Best
>
> Wilm
>
Re: Copying files to hadoop.
Posted by Anil Jagtap <an...@gmail.com>.
Oh Thanks a lot Wilm.. You understood my problem accurately.. I executed it
and it worked.
I understand i can always copy it to Linux and then put it to hadoop, but i
was just trying to find out if this is possible.
Thanks again.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:56 AM, Wilm Schumacher <wi...@gmail.com>
wrote:
>
> Am 17.12.2014 um 23:29 schrieb Anil Jagtap:
> > Dear All,
> >
> > I'm pretty new to Hadoop technology and Linux environment hence
> > struggling even to find solutions for the basic stuff.
> >
> > For now, Hortonworks Sandbox is working fine for me and i managed to
> > connect to it thru SSH.
> >
> > Now i have some csv files in my mac os folders which i want to copy
> > onto Hadoop. As per my knowledge i can copy those files first to Linux
> > and then put to Hadoop. But is there a way in which just in one
> > command it will copy to Hadoop directly from mac os folder?
> yes, there is.
>
> cat /path/to/your/local/file.csv | ssh hadoopuser@namenode
> "/remote/server/path/to/hadoop fs -put - /hadoop/folder/name/file.csv"
>
> As you wrote, that you are also new to linux/unix, this above means:
>
> * cat => concanate the files (only one file given) and print to standard
> output
>
> * pipe | => means, write the standard output from the left hand to the
> standard input of the right hand side
>
> * ssh reads from standard input and writes its to the standard input on
> the remote server command, which is hadoop fs put command, which is told
> to read from stdin
>
> Thus you are actually piping the content of the file through 3 services.
> And that's actually a little bit of a hack and in my opinion there is no
> reason to do this if your file is reasonable small to fit on the remote
> server. It's like asking "is it possible to reach my destination only
> using left turns". Well ... it's possible, but not always a good idea ;).
>
> Best
>
> Wilm
>
Re: Copying files to hadoop.
Posted by Anil Jagtap <an...@gmail.com>.
Oh Thanks a lot Wilm.. You understood my problem accurately.. I executed it
and it worked.
I understand i can always copy it to Linux and then put it to hadoop, but i
was just trying to find out if this is possible.
Thanks again.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:56 AM, Wilm Schumacher <wi...@gmail.com>
wrote:
>
> Am 17.12.2014 um 23:29 schrieb Anil Jagtap:
> > Dear All,
> >
> > I'm pretty new to Hadoop technology and Linux environment hence
> > struggling even to find solutions for the basic stuff.
> >
> > For now, Hortonworks Sandbox is working fine for me and i managed to
> > connect to it thru SSH.
> >
> > Now i have some csv files in my mac os folders which i want to copy
> > onto Hadoop. As per my knowledge i can copy those files first to Linux
> > and then put to Hadoop. But is there a way in which just in one
> > command it will copy to Hadoop directly from mac os folder?
> yes, there is.
>
> cat /path/to/your/local/file.csv | ssh hadoopuser@namenode
> "/remote/server/path/to/hadoop fs -put - /hadoop/folder/name/file.csv"
>
> As you wrote, that you are also new to linux/unix, this above means:
>
> * cat => concanate the files (only one file given) and print to standard
> output
>
> * pipe | => means, write the standard output from the left hand to the
> standard input of the right hand side
>
> * ssh reads from standard input and writes its to the standard input on
> the remote server command, which is hadoop fs put command, which is told
> to read from stdin
>
> Thus you are actually piping the content of the file through 3 services.
> And that's actually a little bit of a hack and in my opinion there is no
> reason to do this if your file is reasonable small to fit on the remote
> server. It's like asking "is it possible to reach my destination only
> using left turns". Well ... it's possible, but not always a good idea ;).
>
> Best
>
> Wilm
>
Re: Copying files to hadoop.
Posted by Wilm Schumacher <wi...@gmail.com>.
Am 17.12.2014 um 23:29 schrieb Anil Jagtap:
> Dear All,
>
> I'm pretty new to Hadoop technology and Linux environment hence
> struggling even to find solutions for the basic stuff.
>
> For now, Hortonworks Sandbox is working fine for me and i managed to
> connect to it thru SSH.
>
> Now i have some csv files in my mac os folders which i want to copy
> onto Hadoop. As per my knowledge i can copy those files first to Linux
> and then put to Hadoop. But is there a way in which just in one
> command it will copy to Hadoop directly from mac os folder?
yes, there is.
cat /path/to/your/local/file.csv | ssh hadoopuser@namenode
"/remote/server/path/to/hadoop fs -put - /hadoop/folder/name/file.csv"
As you wrote, that you are also new to linux/unix, this above means:
* cat => concanate the files (only one file given) and print to standard
output
* pipe | => means, write the standard output from the left hand to the
standard input of the right hand side
* ssh reads from standard input and writes its to the standard input on
the remote server command, which is hadoop fs put command, which is told
to read from stdin
Thus you are actually piping the content of the file through 3 services.
And that's actually a little bit of a hack and in my opinion there is no
reason to do this if your file is reasonable small to fit on the remote
server. It's like asking "is it possible to reach my destination only
using left turns". Well ... it's possible, but not always a good idea ;).
Best
Wilm
Re: Copying files to hadoop.
Posted by Wilm Schumacher <wi...@gmail.com>.
Am 17.12.2014 um 23:29 schrieb Anil Jagtap:
> Dear All,
>
> I'm pretty new to Hadoop technology and Linux environment hence
> struggling even to find solutions for the basic stuff.
>
> For now, Hortonworks Sandbox is working fine for me and i managed to
> connect to it thru SSH.
>
> Now i have some csv files in my mac os folders which i want to copy
> onto Hadoop. As per my knowledge i can copy those files first to Linux
> and then put to Hadoop. But is there a way in which just in one
> command it will copy to Hadoop directly from mac os folder?
yes, there is.
cat /path/to/your/local/file.csv | ssh hadoopuser@namenode
"/remote/server/path/to/hadoop fs -put - /hadoop/folder/name/file.csv"
As you wrote, that you are also new to linux/unix, this above means:
* cat => concanate the files (only one file given) and print to standard
output
* pipe | => means, write the standard output from the left hand to the
standard input of the right hand side
* ssh reads from standard input and writes its to the standard input on
the remote server command, which is hadoop fs put command, which is told
to read from stdin
Thus you are actually piping the content of the file through 3 services.
And that's actually a little bit of a hack and in my opinion there is no
reason to do this if your file is reasonable small to fit on the remote
server. It's like asking "is it possible to reach my destination only
using left turns". Well ... it's possible, but not always a good idea ;).
Best
Wilm
Re: Copying files to hadoop.
Posted by Wilm Schumacher <wi...@gmail.com>.
Am 17.12.2014 um 23:29 schrieb Anil Jagtap:
> Dear All,
>
> I'm pretty new to Hadoop technology and Linux environment hence
> struggling even to find solutions for the basic stuff.
>
> For now, Hortonworks Sandbox is working fine for me and i managed to
> connect to it thru SSH.
>
> Now i have some csv files in my mac os folders which i want to copy
> onto Hadoop. As per my knowledge i can copy those files first to Linux
> and then put to Hadoop. But is there a way in which just in one
> command it will copy to Hadoop directly from mac os folder?
yes, there is.
cat /path/to/your/local/file.csv | ssh hadoopuser@namenode
"/remote/server/path/to/hadoop fs -put - /hadoop/folder/name/file.csv"
As you wrote, that you are also new to linux/unix, this above means:
* cat => concanate the files (only one file given) and print to standard
output
* pipe | => means, write the standard output from the left hand to the
standard input of the right hand side
* ssh reads from standard input and writes its to the standard input on
the remote server command, which is hadoop fs put command, which is told
to read from stdin
Thus you are actually piping the content of the file through 3 services.
And that's actually a little bit of a hack and in my opinion there is no
reason to do this if your file is reasonable small to fit on the remote
server. It's like asking "is it possible to reach my destination only
using left turns". Well ... it's possible, but not always a good idea ;).
Best
Wilm
Re: Copying files to hadoop.
Posted by Rich Haase <rh...@pandora.com>.
Anil,
Happy to help!
Cheers,
Rich
Rich Haase | Sr. Software Engineer | Pandora
m 303.887.1146 | rhaase@pandora.com
From: Anil Jagtap <an...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wednesday, December 17, 2014 at 4:35 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Re: Copying files to hadoop.
Hi Rich,
Yes infact i was too thinking the same but then somehow slipped of my mind. I guess the second option would be really great so i don't even need to build the complex and length commands. The shared folder will be anyways appear as local in vm.
Thanks a lot Rich.
Rgds, Anil
On Thu, Dec 18, 2014 at 10:03 AM, Rich Haase <rh...@pandora.com>> wrote:
Anil,
You have two main options:
1. install the hadoop software on OSX and add the configuration files appropriate for your sandbox, then do use hdfs dfs –put <local> <remote>
2. Setup your sandbox VM to share a directory between OS X and Linux. All virtual machines that I know of support sharing a file system between the VM and host. This is probably the easiest solution since it will allow you to see the files you have on OS X in your Linux VM and then you can use the hdfs/hadoop/yarn commands on linux (which you already have configured).
Cheers,
Rich
Rich Haase | Sr. Software Engineer | Pandora
m 303.887.1146 | rhaase@pandora.com<ma...@pandora.com>
From: Anil Jagtap <an...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wednesday, December 17, 2014 at 3:58 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Re: Copying files to hadoop.
Yes i can do that but I have connected from my mac os terminal to linux using ssh.
Now when I run LS command it shows me list of files & folders from Linux and not from Mac OS.
I have files which I need to put onto Hadoop directly from Mac OS.
So something like below.
>From Mac OS Terminal:
[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>> wrote:
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
________________________________
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com<ma...@gmail.com>
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
Re: Copying files to hadoop.
Posted by Rich Haase <rh...@pandora.com>.
Anil,
Happy to help!
Cheers,
Rich
Rich Haase | Sr. Software Engineer | Pandora
m 303.887.1146 | rhaase@pandora.com
From: Anil Jagtap <an...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wednesday, December 17, 2014 at 4:35 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Re: Copying files to hadoop.
Hi Rich,
Yes infact i was too thinking the same but then somehow slipped of my mind. I guess the second option would be really great so i don't even need to build the complex and length commands. The shared folder will be anyways appear as local in vm.
Thanks a lot Rich.
Rgds, Anil
On Thu, Dec 18, 2014 at 10:03 AM, Rich Haase <rh...@pandora.com>> wrote:
Anil,
You have two main options:
1. install the hadoop software on OSX and add the configuration files appropriate for your sandbox, then do use hdfs dfs –put <local> <remote>
2. Setup your sandbox VM to share a directory between OS X and Linux. All virtual machines that I know of support sharing a file system between the VM and host. This is probably the easiest solution since it will allow you to see the files you have on OS X in your Linux VM and then you can use the hdfs/hadoop/yarn commands on linux (which you already have configured).
Cheers,
Rich
Rich Haase | Sr. Software Engineer | Pandora
m 303.887.1146 | rhaase@pandora.com<ma...@pandora.com>
From: Anil Jagtap <an...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wednesday, December 17, 2014 at 3:58 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Re: Copying files to hadoop.
Yes i can do that but I have connected from my mac os terminal to linux using ssh.
Now when I run LS command it shows me list of files & folders from Linux and not from Mac OS.
I have files which I need to put onto Hadoop directly from Mac OS.
So something like below.
>From Mac OS Terminal:
[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>> wrote:
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
________________________________
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com<ma...@gmail.com>
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
Re: Copying files to hadoop.
Posted by Rich Haase <rh...@pandora.com>.
Anil,
Happy to help!
Cheers,
Rich
Rich Haase | Sr. Software Engineer | Pandora
m 303.887.1146 | rhaase@pandora.com
From: Anil Jagtap <an...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wednesday, December 17, 2014 at 4:35 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Re: Copying files to hadoop.
Hi Rich,
Yes infact i was too thinking the same but then somehow slipped of my mind. I guess the second option would be really great so i don't even need to build the complex and length commands. The shared folder will be anyways appear as local in vm.
Thanks a lot Rich.
Rgds, Anil
On Thu, Dec 18, 2014 at 10:03 AM, Rich Haase <rh...@pandora.com>> wrote:
Anil,
You have two main options:
1. install the hadoop software on OSX and add the configuration files appropriate for your sandbox, then do use hdfs dfs –put <local> <remote>
2. Setup your sandbox VM to share a directory between OS X and Linux. All virtual machines that I know of support sharing a file system between the VM and host. This is probably the easiest solution since it will allow you to see the files you have on OS X in your Linux VM and then you can use the hdfs/hadoop/yarn commands on linux (which you already have configured).
Cheers,
Rich
Rich Haase | Sr. Software Engineer | Pandora
m 303.887.1146 | rhaase@pandora.com<ma...@pandora.com>
From: Anil Jagtap <an...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wednesday, December 17, 2014 at 3:58 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Re: Copying files to hadoop.
Yes i can do that but I have connected from my mac os terminal to linux using ssh.
Now when I run LS command it shows me list of files & folders from Linux and not from Mac OS.
I have files which I need to put onto Hadoop directly from Mac OS.
So something like below.
>From Mac OS Terminal:
[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>> wrote:
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
________________________________
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com<ma...@gmail.com>
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
Re: Copying files to hadoop.
Posted by Rich Haase <rh...@pandora.com>.
Anil,
Happy to help!
Cheers,
Rich
Rich Haase | Sr. Software Engineer | Pandora
m 303.887.1146 | rhaase@pandora.com
From: Anil Jagtap <an...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wednesday, December 17, 2014 at 4:35 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Re: Copying files to hadoop.
Hi Rich,
Yes infact i was too thinking the same but then somehow slipped of my mind. I guess the second option would be really great so i don't even need to build the complex and length commands. The shared folder will be anyways appear as local in vm.
Thanks a lot Rich.
Rgds, Anil
On Thu, Dec 18, 2014 at 10:03 AM, Rich Haase <rh...@pandora.com>> wrote:
Anil,
You have two main options:
1. install the hadoop software on OSX and add the configuration files appropriate for your sandbox, then do use hdfs dfs –put <local> <remote>
2. Setup your sandbox VM to share a directory between OS X and Linux. All virtual machines that I know of support sharing a file system between the VM and host. This is probably the easiest solution since it will allow you to see the files you have on OS X in your Linux VM and then you can use the hdfs/hadoop/yarn commands on linux (which you already have configured).
Cheers,
Rich
Rich Haase | Sr. Software Engineer | Pandora
m 303.887.1146 | rhaase@pandora.com<ma...@pandora.com>
From: Anil Jagtap <an...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wednesday, December 17, 2014 at 3:58 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Re: Copying files to hadoop.
Yes i can do that but I have connected from my mac os terminal to linux using ssh.
Now when I run LS command it shows me list of files & folders from Linux and not from Mac OS.
I have files which I need to put onto Hadoop directly from Mac OS.
So something like below.
>From Mac OS Terminal:
[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>> wrote:
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
________________________________
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com<ma...@gmail.com>
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
Re: Copying files to hadoop.
Posted by Anil Jagtap <an...@gmail.com>.
Hi Rich,
Yes infact i was too thinking the same but then somehow slipped of my mind.
I guess the second option would be really great so i don't even need to
build the complex and length commands. The shared folder will be anyways
appear as local in vm.
Thanks a lot Rich.
Rgds, Anil
On Thu, Dec 18, 2014 at 10:03 AM, Rich Haase <rh...@pandora.com> wrote:
>
> Anil,
>
> You have two main options:
>
> 1. install the hadoop software on OSX and add the configuration files
> appropriate for your sandbox, then do use hdfs dfs –put <local> <remote>
> 2. Setup your sandbox VM to share a directory between OS X and Linux.
> All virtual machines that I know of support sharing a file system between
> the VM and host. This is probably the easiest solution since it will allow
> you to see the files you have on OS X in your Linux VM and then you can use
> the hdfs/hadoop/yarn commands on linux (which you already have configured).
>
> Cheers,
>
> Rich
>
> *Rich Haase* | Sr. Software Engineer | Pandora
> m 303.887.1146 | rhaase@pandora.com
>
> From: Anil Jagtap <an...@gmail.com>
> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Date: Wednesday, December 17, 2014 at 3:58 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: Re: Copying files to hadoop.
>
> Yes i can do that but I have connected from my mac os terminal to linux
> using ssh.
> Now when I run LS command it shows me list of files & folders from Linux
> and not from Mac OS.
> I have files which I need to put onto Hadoop directly from Mac OS.
> So something like below.
>
> From Mac OS Terminal:
>
> [root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
>
> Hope my requirement is clear.
>
> Rgds, Anil
>
>
>
>
> On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>
> wrote:
>>
>> Hi Anil,
>>
>> you can use the hadoop fs put "file" or directory and that should add it
>> to your hdfs
>>
>> ------------------------------
>> Date: Thu, 18 Dec 2014 09:29:34 +1100
>> Subject: Copying files to hadoop.
>> From: anil.jagtap@gmail.com
>> To: user@hadoop.apache.org
>>
>> Dear All,
>>
>> I'm pretty new to Hadoop technology and Linux environment hence
>> struggling even to find solutions for the basic stuff.
>>
>> For now, Hortonworks Sandbox is working fine for me and i managed to
>> connect to it thru SSH.
>>
>> Now i have some csv files in my mac os folders which i want to copy
>> onto Hadoop. As per my knowledge i can copy those files first to Linux and
>> then put to Hadoop. But is there a way in which just in one command it will
>> copy to Hadoop directly from mac os folder?
>>
>> Appreciate your advices.
>>
>> Thank you guys...
>>
>> Rgds, Anil
>>
>>
Re: Copying files to hadoop.
Posted by Anil Jagtap <an...@gmail.com>.
Hi Rich,
Yes infact i was too thinking the same but then somehow slipped of my mind.
I guess the second option would be really great so i don't even need to
build the complex and length commands. The shared folder will be anyways
appear as local in vm.
Thanks a lot Rich.
Rgds, Anil
On Thu, Dec 18, 2014 at 10:03 AM, Rich Haase <rh...@pandora.com> wrote:
>
> Anil,
>
> You have two main options:
>
> 1. install the hadoop software on OSX and add the configuration files
> appropriate for your sandbox, then do use hdfs dfs –put <local> <remote>
> 2. Setup your sandbox VM to share a directory between OS X and Linux.
> All virtual machines that I know of support sharing a file system between
> the VM and host. This is probably the easiest solution since it will allow
> you to see the files you have on OS X in your Linux VM and then you can use
> the hdfs/hadoop/yarn commands on linux (which you already have configured).
>
> Cheers,
>
> Rich
>
> *Rich Haase* | Sr. Software Engineer | Pandora
> m 303.887.1146 | rhaase@pandora.com
>
> From: Anil Jagtap <an...@gmail.com>
> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Date: Wednesday, December 17, 2014 at 3:58 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: Re: Copying files to hadoop.
>
> Yes i can do that but I have connected from my mac os terminal to linux
> using ssh.
> Now when I run LS command it shows me list of files & folders from Linux
> and not from Mac OS.
> I have files which I need to put onto Hadoop directly from Mac OS.
> So something like below.
>
> From Mac OS Terminal:
>
> [root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
>
> Hope my requirement is clear.
>
> Rgds, Anil
>
>
>
>
> On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>
> wrote:
>>
>> Hi Anil,
>>
>> you can use the hadoop fs put "file" or directory and that should add it
>> to your hdfs
>>
>> ------------------------------
>> Date: Thu, 18 Dec 2014 09:29:34 +1100
>> Subject: Copying files to hadoop.
>> From: anil.jagtap@gmail.com
>> To: user@hadoop.apache.org
>>
>> Dear All,
>>
>> I'm pretty new to Hadoop technology and Linux environment hence
>> struggling even to find solutions for the basic stuff.
>>
>> For now, Hortonworks Sandbox is working fine for me and i managed to
>> connect to it thru SSH.
>>
>> Now i have some csv files in my mac os folders which i want to copy
>> onto Hadoop. As per my knowledge i can copy those files first to Linux and
>> then put to Hadoop. But is there a way in which just in one command it will
>> copy to Hadoop directly from mac os folder?
>>
>> Appreciate your advices.
>>
>> Thank you guys...
>>
>> Rgds, Anil
>>
>>
Re: Copying files to hadoop.
Posted by Anil Jagtap <an...@gmail.com>.
Hi Rich,
Yes infact i was too thinking the same but then somehow slipped of my mind.
I guess the second option would be really great so i don't even need to
build the complex and length commands. The shared folder will be anyways
appear as local in vm.
Thanks a lot Rich.
Rgds, Anil
On Thu, Dec 18, 2014 at 10:03 AM, Rich Haase <rh...@pandora.com> wrote:
>
> Anil,
>
> You have two main options:
>
> 1. install the hadoop software on OSX and add the configuration files
> appropriate for your sandbox, then do use hdfs dfs –put <local> <remote>
> 2. Setup your sandbox VM to share a directory between OS X and Linux.
> All virtual machines that I know of support sharing a file system between
> the VM and host. This is probably the easiest solution since it will allow
> you to see the files you have on OS X in your Linux VM and then you can use
> the hdfs/hadoop/yarn commands on linux (which you already have configured).
>
> Cheers,
>
> Rich
>
> *Rich Haase* | Sr. Software Engineer | Pandora
> m 303.887.1146 | rhaase@pandora.com
>
> From: Anil Jagtap <an...@gmail.com>
> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Date: Wednesday, December 17, 2014 at 3:58 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: Re: Copying files to hadoop.
>
> Yes i can do that but I have connected from my mac os terminal to linux
> using ssh.
> Now when I run LS command it shows me list of files & folders from Linux
> and not from Mac OS.
> I have files which I need to put onto Hadoop directly from Mac OS.
> So something like below.
>
> From Mac OS Terminal:
>
> [root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
>
> Hope my requirement is clear.
>
> Rgds, Anil
>
>
>
>
> On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>
> wrote:
>>
>> Hi Anil,
>>
>> you can use the hadoop fs put "file" or directory and that should add it
>> to your hdfs
>>
>> ------------------------------
>> Date: Thu, 18 Dec 2014 09:29:34 +1100
>> Subject: Copying files to hadoop.
>> From: anil.jagtap@gmail.com
>> To: user@hadoop.apache.org
>>
>> Dear All,
>>
>> I'm pretty new to Hadoop technology and Linux environment hence
>> struggling even to find solutions for the basic stuff.
>>
>> For now, Hortonworks Sandbox is working fine for me and i managed to
>> connect to it thru SSH.
>>
>> Now i have some csv files in my mac os folders which i want to copy
>> onto Hadoop. As per my knowledge i can copy those files first to Linux and
>> then put to Hadoop. But is there a way in which just in one command it will
>> copy to Hadoop directly from mac os folder?
>>
>> Appreciate your advices.
>>
>> Thank you guys...
>>
>> Rgds, Anil
>>
>>
Re: Copying files to hadoop.
Posted by Anil Jagtap <an...@gmail.com>.
Hi Rich,
Yes infact i was too thinking the same but then somehow slipped of my mind.
I guess the second option would be really great so i don't even need to
build the complex and length commands. The shared folder will be anyways
appear as local in vm.
Thanks a lot Rich.
Rgds, Anil
On Thu, Dec 18, 2014 at 10:03 AM, Rich Haase <rh...@pandora.com> wrote:
>
> Anil,
>
> You have two main options:
>
> 1. install the hadoop software on OSX and add the configuration files
> appropriate for your sandbox, then do use hdfs dfs –put <local> <remote>
> 2. Setup your sandbox VM to share a directory between OS X and Linux.
> All virtual machines that I know of support sharing a file system between
> the VM and host. This is probably the easiest solution since it will allow
> you to see the files you have on OS X in your Linux VM and then you can use
> the hdfs/hadoop/yarn commands on linux (which you already have configured).
>
> Cheers,
>
> Rich
>
> *Rich Haase* | Sr. Software Engineer | Pandora
> m 303.887.1146 | rhaase@pandora.com
>
> From: Anil Jagtap <an...@gmail.com>
> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Date: Wednesday, December 17, 2014 at 3:58 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: Re: Copying files to hadoop.
>
> Yes i can do that but I have connected from my mac os terminal to linux
> using ssh.
> Now when I run LS command it shows me list of files & folders from Linux
> and not from Mac OS.
> I have files which I need to put onto Hadoop directly from Mac OS.
> So something like below.
>
> From Mac OS Terminal:
>
> [root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
>
> Hope my requirement is clear.
>
> Rgds, Anil
>
>
>
>
> On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>
> wrote:
>>
>> Hi Anil,
>>
>> you can use the hadoop fs put "file" or directory and that should add it
>> to your hdfs
>>
>> ------------------------------
>> Date: Thu, 18 Dec 2014 09:29:34 +1100
>> Subject: Copying files to hadoop.
>> From: anil.jagtap@gmail.com
>> To: user@hadoop.apache.org
>>
>> Dear All,
>>
>> I'm pretty new to Hadoop technology and Linux environment hence
>> struggling even to find solutions for the basic stuff.
>>
>> For now, Hortonworks Sandbox is working fine for me and i managed to
>> connect to it thru SSH.
>>
>> Now i have some csv files in my mac os folders which i want to copy
>> onto Hadoop. As per my knowledge i can copy those files first to Linux and
>> then put to Hadoop. But is there a way in which just in one command it will
>> copy to Hadoop directly from mac os folder?
>>
>> Appreciate your advices.
>>
>> Thank you guys...
>>
>> Rgds, Anil
>>
>>
Re: Copying files to hadoop.
Posted by Rich Haase <rh...@pandora.com>.
Anil,
You have two main options:
1. install the hadoop software on OSX and add the configuration files appropriate for your sandbox, then do use hdfs dfs –put <local> <remote>
2. Setup your sandbox VM to share a directory between OS X and Linux. All virtual machines that I know of support sharing a file system between the VM and host. This is probably the easiest solution since it will allow you to see the files you have on OS X in your Linux VM and then you can use the hdfs/hadoop/yarn commands on linux (which you already have configured).
Cheers,
Rich
Rich Haase | Sr. Software Engineer | Pandora
m 303.887.1146 | rhaase@pandora.com
From: Anil Jagtap <an...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wednesday, December 17, 2014 at 3:58 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Re: Copying files to hadoop.
Yes i can do that but I have connected from my mac os terminal to linux using ssh.
Now when I run LS command it shows me list of files & folders from Linux and not from Mac OS.
I have files which I need to put onto Hadoop directly from Mac OS.
So something like below.
>From Mac OS Terminal:
[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>> wrote:
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
________________________________
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com<ma...@gmail.com>
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
RE: Copying files to hadoop.
Posted by johny casanova <pc...@outlook.com>.
what you can do is copy the files to the linux box then use the hadoop fs put. You can do this like "scp /directory/i/want or "file.name" "username"@"hostname":/directorytoputfiles/"
for example : scp dude.txt dude@main-hadoop:/opt/
Date: Thu, 18 Dec 2014 09:58:43 +1100
Subject: Re: Copying files to hadoop.
From: anil.jagtap@gmail.com
To: user@hadoop.apache.org
Yes i can do that but I have connected from my mac os terminal to linux using ssh.Now when I run LS command it shows me list of files & folders from Linux and not from Mac OS.I have files which I need to put onto Hadoop directly from Mac OS.So something like below.
>From Mac OS Terminal:[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com> wrote:
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com
To: user@hadoop.apache.org
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
RE: Copying files to hadoop.
Posted by johny casanova <pc...@outlook.com>.
what you can do is copy the files to the linux box then use the hadoop fs put. You can do this like "scp /directory/i/want or "file.name" "username"@"hostname":/directorytoputfiles/"
for example : scp dude.txt dude@main-hadoop:/opt/
Date: Thu, 18 Dec 2014 09:58:43 +1100
Subject: Re: Copying files to hadoop.
From: anil.jagtap@gmail.com
To: user@hadoop.apache.org
Yes i can do that but I have connected from my mac os terminal to linux using ssh.Now when I run LS command it shows me list of files & folders from Linux and not from Mac OS.I have files which I need to put onto Hadoop directly from Mac OS.So something like below.
>From Mac OS Terminal:[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com> wrote:
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com
To: user@hadoop.apache.org
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
Re: Copying files to hadoop.
Posted by Rich Haase <rh...@pandora.com>.
Anil,
You have two main options:
1. install the hadoop software on OSX and add the configuration files appropriate for your sandbox, then do use hdfs dfs –put <local> <remote>
2. Setup your sandbox VM to share a directory between OS X and Linux. All virtual machines that I know of support sharing a file system between the VM and host. This is probably the easiest solution since it will allow you to see the files you have on OS X in your Linux VM and then you can use the hdfs/hadoop/yarn commands on linux (which you already have configured).
Cheers,
Rich
Rich Haase | Sr. Software Engineer | Pandora
m 303.887.1146 | rhaase@pandora.com
From: Anil Jagtap <an...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wednesday, December 17, 2014 at 3:58 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Re: Copying files to hadoop.
Yes i can do that but I have connected from my mac os terminal to linux using ssh.
Now when I run LS command it shows me list of files & folders from Linux and not from Mac OS.
I have files which I need to put onto Hadoop directly from Mac OS.
So something like below.
>From Mac OS Terminal:
[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>> wrote:
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
________________________________
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com<ma...@gmail.com>
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
RE: Copying files to hadoop.
Posted by johny casanova <pc...@outlook.com>.
what you can do is copy the files to the linux box then use the hadoop fs put. You can do this like "scp /directory/i/want or "file.name" "username"@"hostname":/directorytoputfiles/"
for example : scp dude.txt dude@main-hadoop:/opt/
Date: Thu, 18 Dec 2014 09:58:43 +1100
Subject: Re: Copying files to hadoop.
From: anil.jagtap@gmail.com
To: user@hadoop.apache.org
Yes i can do that but I have connected from my mac os terminal to linux using ssh.Now when I run LS command it shows me list of files & folders from Linux and not from Mac OS.I have files which I need to put onto Hadoop directly from Mac OS.So something like below.
>From Mac OS Terminal:[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com> wrote:
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com
To: user@hadoop.apache.org
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
RE: Copying files to hadoop.
Posted by johny casanova <pc...@outlook.com>.
what you can do is copy the files to the linux box then use the hadoop fs put. You can do this like "scp /directory/i/want or "file.name" "username"@"hostname":/directorytoputfiles/"
for example : scp dude.txt dude@main-hadoop:/opt/
Date: Thu, 18 Dec 2014 09:58:43 +1100
Subject: Re: Copying files to hadoop.
From: anil.jagtap@gmail.com
To: user@hadoop.apache.org
Yes i can do that but I have connected from my mac os terminal to linux using ssh.Now when I run LS command it shows me list of files & folders from Linux and not from Mac OS.I have files which I need to put onto Hadoop directly from Mac OS.So something like below.
>From Mac OS Terminal:[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com> wrote:
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com
To: user@hadoop.apache.org
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
Re: Copying files to hadoop.
Posted by Rich Haase <rh...@pandora.com>.
Anil,
You have two main options:
1. install the hadoop software on OSX and add the configuration files appropriate for your sandbox, then do use hdfs dfs –put <local> <remote>
2. Setup your sandbox VM to share a directory between OS X and Linux. All virtual machines that I know of support sharing a file system between the VM and host. This is probably the easiest solution since it will allow you to see the files you have on OS X in your Linux VM and then you can use the hdfs/hadoop/yarn commands on linux (which you already have configured).
Cheers,
Rich
Rich Haase | Sr. Software Engineer | Pandora
m 303.887.1146 | rhaase@pandora.com
From: Anil Jagtap <an...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wednesday, December 17, 2014 at 3:58 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Re: Copying files to hadoop.
Yes i can do that but I have connected from my mac os terminal to linux using ssh.
Now when I run LS command it shows me list of files & folders from Linux and not from Mac OS.
I have files which I need to put onto Hadoop directly from Mac OS.
So something like below.
>From Mac OS Terminal:
[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>> wrote:
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
________________________________
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com<ma...@gmail.com>
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
Re: Copying files to hadoop.
Posted by Rich Haase <rh...@pandora.com>.
Anil,
You have two main options:
1. install the hadoop software on OSX and add the configuration files appropriate for your sandbox, then do use hdfs dfs –put <local> <remote>
2. Setup your sandbox VM to share a directory between OS X and Linux. All virtual machines that I know of support sharing a file system between the VM and host. This is probably the easiest solution since it will allow you to see the files you have on OS X in your Linux VM and then you can use the hdfs/hadoop/yarn commands on linux (which you already have configured).
Cheers,
Rich
Rich Haase | Sr. Software Engineer | Pandora
m 303.887.1146 | rhaase@pandora.com
From: Anil Jagtap <an...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wednesday, December 17, 2014 at 3:58 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Re: Copying files to hadoop.
Yes i can do that but I have connected from my mac os terminal to linux using ssh.
Now when I run LS command it shows me list of files & folders from Linux and not from Mac OS.
I have files which I need to put onto Hadoop directly from Mac OS.
So something like below.
>From Mac OS Terminal:
[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>> wrote:
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
________________________________
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com<ma...@gmail.com>
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
Re: Copying files to hadoop.
Posted by Anil Jagtap <an...@gmail.com>.
Yes i can do that but I have connected from my mac os terminal to linux
using ssh.
Now when I run LS command it shows me list of files & folders from Linux
and not from Mac OS.
I have files which I need to put onto Hadoop directly from Mac OS.
So something like below.
>From Mac OS Terminal:
[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>
wrote:
>
> Hi Anil,
>
> you can use the hadoop fs put "file" or directory and that should add it
> to your hdfs
>
> ------------------------------
> Date: Thu, 18 Dec 2014 09:29:34 +1100
> Subject: Copying files to hadoop.
> From: anil.jagtap@gmail.com
> To: user@hadoop.apache.org
>
> Dear All,
>
> I'm pretty new to Hadoop technology and Linux environment hence struggling
> even to find solutions for the basic stuff.
>
> For now, Hortonworks Sandbox is working fine for me and i managed to
> connect to it thru SSH.
>
> Now i have some csv files in my mac os folders which i want to copy onto
> Hadoop. As per my knowledge i can copy those files first to Linux and then
> put to Hadoop. But is there a way in which just in one command it will copy
> to Hadoop directly from mac os folder?
>
> Appreciate your advices.
>
> Thank you guys...
>
> Rgds, Anil
>
>
Re: Copying files to hadoop.
Posted by Anil Jagtap <an...@gmail.com>.
Yes i can do that but I have connected from my mac os terminal to linux
using ssh.
Now when I run LS command it shows me list of files & folders from Linux
and not from Mac OS.
I have files which I need to put onto Hadoop directly from Mac OS.
So something like below.
>From Mac OS Terminal:
[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>
wrote:
>
> Hi Anil,
>
> you can use the hadoop fs put "file" or directory and that should add it
> to your hdfs
>
> ------------------------------
> Date: Thu, 18 Dec 2014 09:29:34 +1100
> Subject: Copying files to hadoop.
> From: anil.jagtap@gmail.com
> To: user@hadoop.apache.org
>
> Dear All,
>
> I'm pretty new to Hadoop technology and Linux environment hence struggling
> even to find solutions for the basic stuff.
>
> For now, Hortonworks Sandbox is working fine for me and i managed to
> connect to it thru SSH.
>
> Now i have some csv files in my mac os folders which i want to copy onto
> Hadoop. As per my knowledge i can copy those files first to Linux and then
> put to Hadoop. But is there a way in which just in one command it will copy
> to Hadoop directly from mac os folder?
>
> Appreciate your advices.
>
> Thank you guys...
>
> Rgds, Anil
>
>
Re: Copying files to hadoop.
Posted by Anil Jagtap <an...@gmail.com>.
Yes i can do that but I have connected from my mac os terminal to linux
using ssh.
Now when I run LS command it shows me list of files & folders from Linux
and not from Mac OS.
I have files which I need to put onto Hadoop directly from Mac OS.
So something like below.
>From Mac OS Terminal:
[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>
wrote:
>
> Hi Anil,
>
> you can use the hadoop fs put "file" or directory and that should add it
> to your hdfs
>
> ------------------------------
> Date: Thu, 18 Dec 2014 09:29:34 +1100
> Subject: Copying files to hadoop.
> From: anil.jagtap@gmail.com
> To: user@hadoop.apache.org
>
> Dear All,
>
> I'm pretty new to Hadoop technology and Linux environment hence struggling
> even to find solutions for the basic stuff.
>
> For now, Hortonworks Sandbox is working fine for me and i managed to
> connect to it thru SSH.
>
> Now i have some csv files in my mac os folders which i want to copy onto
> Hadoop. As per my knowledge i can copy those files first to Linux and then
> put to Hadoop. But is there a way in which just in one command it will copy
> to Hadoop directly from mac os folder?
>
> Appreciate your advices.
>
> Thank you guys...
>
> Rgds, Anil
>
>
Re: Copying files to hadoop.
Posted by Anil Jagtap <an...@gmail.com>.
Yes i can do that but I have connected from my mac os terminal to linux
using ssh.
Now when I run LS command it shows me list of files & folders from Linux
and not from Mac OS.
I have files which I need to put onto Hadoop directly from Mac OS.
So something like below.
>From Mac OS Terminal:
[root@sandbox ~]#hadoop fs -put <MAC OS FOLDER PATH/FILE> <HADOOP PATH>
Hope my requirement is clear.
Rgds, Anil
On Thu, Dec 18, 2014 at 9:39 AM, johny casanova <pc...@outlook.com>
wrote:
>
> Hi Anil,
>
> you can use the hadoop fs put "file" or directory and that should add it
> to your hdfs
>
> ------------------------------
> Date: Thu, 18 Dec 2014 09:29:34 +1100
> Subject: Copying files to hadoop.
> From: anil.jagtap@gmail.com
> To: user@hadoop.apache.org
>
> Dear All,
>
> I'm pretty new to Hadoop technology and Linux environment hence struggling
> even to find solutions for the basic stuff.
>
> For now, Hortonworks Sandbox is working fine for me and i managed to
> connect to it thru SSH.
>
> Now i have some csv files in my mac os folders which i want to copy onto
> Hadoop. As per my knowledge i can copy those files first to Linux and then
> put to Hadoop. But is there a way in which just in one command it will copy
> to Hadoop directly from mac os folder?
>
> Appreciate your advices.
>
> Thank you guys...
>
> Rgds, Anil
>
>
RE: Copying files to hadoop.
Posted by johny casanova <pc...@outlook.com>.
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com
To: user@hadoop.apache.org
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
RE: Copying files to hadoop.
Posted by johny casanova <pc...@outlook.com>.
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com
To: user@hadoop.apache.org
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
RE: Copying files to hadoop.
Posted by johny casanova <pc...@outlook.com>.
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com
To: user@hadoop.apache.org
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
RE: Copying files to hadoop.
Posted by johny casanova <pc...@outlook.com>.
Hi Anil,
you can use the hadoop fs put "file" or directory and that should add it to your hdfs
Date: Thu, 18 Dec 2014 09:29:34 +1100
Subject: Copying files to hadoop.
From: anil.jagtap@gmail.com
To: user@hadoop.apache.org
Dear All,
I'm pretty new to Hadoop technology and Linux environment hence struggling even to find solutions for the basic stuff.
For now, Hortonworks Sandbox is working fine for me and i managed to connect to it thru SSH.
Now i have some csv files in my mac os folders which i want to copy onto Hadoop. As per my knowledge i can copy those files first to Linux and then put to Hadoop. But is there a way in which just in one command it will copy to Hadoop directly from mac os folder?
Appreciate your advices.
Thank you guys...
Rgds, Anil
Re: Copying files to hadoop.
Posted by Wilm Schumacher <wi...@gmail.com>.
Am 17.12.2014 um 23:29 schrieb Anil Jagtap:
> Dear All,
>
> I'm pretty new to Hadoop technology and Linux environment hence
> struggling even to find solutions for the basic stuff.
>
> For now, Hortonworks Sandbox is working fine for me and i managed to
> connect to it thru SSH.
>
> Now i have some csv files in my mac os folders which i want to copy
> onto Hadoop. As per my knowledge i can copy those files first to Linux
> and then put to Hadoop. But is there a way in which just in one
> command it will copy to Hadoop directly from mac os folder?
yes, there is.
cat /path/to/your/local/file.csv | ssh hadoopuser@namenode
"/remote/server/path/to/hadoop fs -put - /hadoop/folder/name/file.csv"
As you wrote, that you are also new to linux/unix, this above means:
* cat => concanate the files (only one file given) and print to standard
output
* pipe | => means, write the standard output from the left hand to the
standard input of the right hand side
* ssh reads from standard input and writes its to the standard input on
the remote server command, which is hadoop fs put command, which is told
to read from stdin
Thus you are actually piping the content of the file through 3 services.
And that's actually a little bit of a hack and in my opinion there is no
reason to do this if your file is reasonable small to fit on the remote
server. It's like asking "is it possible to reach my destination only
using left turns". Well ... it's possible, but not always a good idea ;).
Best
Wilm