You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by xeonmailinglist <xe...@gmail.com> on 2015/08/17 22:46:00 UTC
can't copy files between HDFS to a specific dir
Hi,
I am trying to understand why I can't copy files between HDFS filesytems
to a directory "/input1", but I can copy files to the root dir "/".
Eg, I have 2 hosts (hadoop-coc-2, and hadoop-coc3), and I can't copy
files between the dir "input1" [1], but I can copy the file to the root
dir [2].
Initially only hadoop-coc-2 (192.168.56.102) have the data. hadoop-coc-3
is empty. After I tried to copy the file to hadoop-coc-3, only the dir
"input1 was created in the destination.
I don't understand why this is happening. Any help?
[1]
`hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
hdfs://192.168.56.103:9000/input1/`
[2]
`hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
hdfs://192.168.56.103:9000/`
Thanks,
Re: can't copy files between HDFS to a specific dir
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
Have you created the destination directory "/input1"?
I'm thinking you need to create the directory before distcp.
Regards,
Akira
On 8/18/15 17:29, xeonmailinglist wrote:
> In my case it has nothing to do with directory permissions. I think
> |distcp| cannot copy a file into a directory. It just can copy
> directories, or copy a file into |/| dir.
>
>
> On 08/17/2015 11:30 PM, Vernon Funkhouser wrote:
>
>> I would first check the directory permissions. If they are set to
>> read only thenn you cannot write to it. They must be set to
>> read/write to copy files.
>>
>> Cheers
>> Vern
>>
>>
>> On Mon, 2015-08-17 at 21:46 +0100, xeonmailinglist wrote:
>>> Hi,
>>>
>>> I am trying to understand why I can't copy files between HDFS filesytems
>>> to a directory "/input1", but I can copy files to the root dir "/".
>>>
>>> Eg, I have 2 hosts (hadoop-coc-2, and hadoop-coc3), and I can't copy
>>> files between the dir "input1" [1], but I can copy the file to the root
>>> dir [2].
>>>
>>> Initially only hadoop-coc-2 (192.168.56.102) have the data. hadoop-coc-3
>>> is empty. After I tried to copy the file to hadoop-coc-3, only the dir
>>> "input1 was created in the destination.
>>>
>>> I don't understand why this is happening. Any help?
>>>
>>> [1]
>>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>>> hdfs://192.168.56.103:9000/input1/`
>>>
>>> [2]
>>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>>> hdfs://192.168.56.103:9000/`
>>>
>>> Thanks,
>>>
>
Re: can't copy files between HDFS to a specific dir
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
Have you created the destination directory "/input1"?
I'm thinking you need to create the directory before distcp.
Regards,
Akira
On 8/18/15 17:29, xeonmailinglist wrote:
> In my case it has nothing to do with directory permissions. I think
> |distcp| cannot copy a file into a directory. It just can copy
> directories, or copy a file into |/| dir.
>
>
> On 08/17/2015 11:30 PM, Vernon Funkhouser wrote:
>
>> I would first check the directory permissions. If they are set to
>> read only thenn you cannot write to it. They must be set to
>> read/write to copy files.
>>
>> Cheers
>> Vern
>>
>>
>> On Mon, 2015-08-17 at 21:46 +0100, xeonmailinglist wrote:
>>> Hi,
>>>
>>> I am trying to understand why I can't copy files between HDFS filesytems
>>> to a directory "/input1", but I can copy files to the root dir "/".
>>>
>>> Eg, I have 2 hosts (hadoop-coc-2, and hadoop-coc3), and I can't copy
>>> files between the dir "input1" [1], but I can copy the file to the root
>>> dir [2].
>>>
>>> Initially only hadoop-coc-2 (192.168.56.102) have the data. hadoop-coc-3
>>> is empty. After I tried to copy the file to hadoop-coc-3, only the dir
>>> "input1 was created in the destination.
>>>
>>> I don't understand why this is happening. Any help?
>>>
>>> [1]
>>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>>> hdfs://192.168.56.103:9000/input1/`
>>>
>>> [2]
>>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>>> hdfs://192.168.56.103:9000/`
>>>
>>> Thanks,
>>>
>
Re: can't copy files between HDFS to a specific dir
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
Have you created the destination directory "/input1"?
I'm thinking you need to create the directory before distcp.
Regards,
Akira
On 8/18/15 17:29, xeonmailinglist wrote:
> In my case it has nothing to do with directory permissions. I think
> |distcp| cannot copy a file into a directory. It just can copy
> directories, or copy a file into |/| dir.
>
>
> On 08/17/2015 11:30 PM, Vernon Funkhouser wrote:
>
>> I would first check the directory permissions. If they are set to
>> read only thenn you cannot write to it. They must be set to
>> read/write to copy files.
>>
>> Cheers
>> Vern
>>
>>
>> On Mon, 2015-08-17 at 21:46 +0100, xeonmailinglist wrote:
>>> Hi,
>>>
>>> I am trying to understand why I can't copy files between HDFS filesytems
>>> to a directory "/input1", but I can copy files to the root dir "/".
>>>
>>> Eg, I have 2 hosts (hadoop-coc-2, and hadoop-coc3), and I can't copy
>>> files between the dir "input1" [1], but I can copy the file to the root
>>> dir [2].
>>>
>>> Initially only hadoop-coc-2 (192.168.56.102) have the data. hadoop-coc-3
>>> is empty. After I tried to copy the file to hadoop-coc-3, only the dir
>>> "input1 was created in the destination.
>>>
>>> I don't understand why this is happening. Any help?
>>>
>>> [1]
>>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>>> hdfs://192.168.56.103:9000/input1/`
>>>
>>> [2]
>>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>>> hdfs://192.168.56.103:9000/`
>>>
>>> Thanks,
>>>
>
Re: can't copy files between HDFS to a specific dir
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
Have you created the destination directory "/input1"?
I'm thinking you need to create the directory before distcp.
Regards,
Akira
On 8/18/15 17:29, xeonmailinglist wrote:
> In my case it has nothing to do with directory permissions. I think
> |distcp| cannot copy a file into a directory. It just can copy
> directories, or copy a file into |/| dir.
>
>
> On 08/17/2015 11:30 PM, Vernon Funkhouser wrote:
>
>> I would first check the directory permissions. If they are set to
>> read only thenn you cannot write to it. They must be set to
>> read/write to copy files.
>>
>> Cheers
>> Vern
>>
>>
>> On Mon, 2015-08-17 at 21:46 +0100, xeonmailinglist wrote:
>>> Hi,
>>>
>>> I am trying to understand why I can't copy files between HDFS filesytems
>>> to a directory "/input1", but I can copy files to the root dir "/".
>>>
>>> Eg, I have 2 hosts (hadoop-coc-2, and hadoop-coc3), and I can't copy
>>> files between the dir "input1" [1], but I can copy the file to the root
>>> dir [2].
>>>
>>> Initially only hadoop-coc-2 (192.168.56.102) have the data. hadoop-coc-3
>>> is empty. After I tried to copy the file to hadoop-coc-3, only the dir
>>> "input1 was created in the destination.
>>>
>>> I don't understand why this is happening. Any help?
>>>
>>> [1]
>>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>>> hdfs://192.168.56.103:9000/input1/`
>>>
>>> [2]
>>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>>> hdfs://192.168.56.103:9000/`
>>>
>>> Thanks,
>>>
>
Re: can't copy files between HDFS to a specific dir
Posted by xeonmailinglist <xe...@gmail.com>.
In my case it has nothing to do with directory permissions. I think
|distcp| cannot copy a file into a directory. It just can copy
directories, or copy a file into |/| dir.
On 08/17/2015 11:30 PM, Vernon Funkhouser wrote:
> I would first check the directory permissions. If they are set to
> read only thenn you cannot write to it. They must be set to
> read/write to copy files.
>
> Cheers
> Vern
>
>
> On Mon, 2015-08-17 at 21:46 +0100, xeonmailinglist wrote:
>> Hi,
>>
>> I am trying to understand why I can't copy files between HDFS filesytems
>> to a directory "/input1", but I can copy files to the root dir "/".
>>
>> Eg, I have 2 hosts (hadoop-coc-2, and hadoop-coc3), and I can't copy
>> files between the dir "input1" [1], but I can copy the file to the root
>> dir [2].
>>
>> Initially only hadoop-coc-2 (192.168.56.102) have the data. hadoop-coc-3
>> is empty. After I tried to copy the file to hadoop-coc-3, only the dir
>> "input1 was created in the destination.
>>
>> I don't understand why this is happening. Any help?
>>
>> [1]
>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>> hdfs://192.168.56.103:9000/input1/`
>>
>> [2]
>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>> hdfs://192.168.56.103:9000/`
>>
>> Thanks,
>>
Re: can't copy files between HDFS to a specific dir
Posted by xeonmailinglist <xe...@gmail.com>.
In my case it has nothing to do with directory permissions. I think
|distcp| cannot copy a file into a directory. It just can copy
directories, or copy a file into |/| dir.
On 08/17/2015 11:30 PM, Vernon Funkhouser wrote:
> I would first check the directory permissions. If they are set to
> read only thenn you cannot write to it. They must be set to
> read/write to copy files.
>
> Cheers
> Vern
>
>
> On Mon, 2015-08-17 at 21:46 +0100, xeonmailinglist wrote:
>> Hi,
>>
>> I am trying to understand why I can't copy files between HDFS filesytems
>> to a directory "/input1", but I can copy files to the root dir "/".
>>
>> Eg, I have 2 hosts (hadoop-coc-2, and hadoop-coc3), and I can't copy
>> files between the dir "input1" [1], but I can copy the file to the root
>> dir [2].
>>
>> Initially only hadoop-coc-2 (192.168.56.102) have the data. hadoop-coc-3
>> is empty. After I tried to copy the file to hadoop-coc-3, only the dir
>> "input1 was created in the destination.
>>
>> I don't understand why this is happening. Any help?
>>
>> [1]
>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>> hdfs://192.168.56.103:9000/input1/`
>>
>> [2]
>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>> hdfs://192.168.56.103:9000/`
>>
>> Thanks,
>>
Re: can't copy files between HDFS to a specific dir
Posted by xeonmailinglist <xe...@gmail.com>.
In my case it has nothing to do with directory permissions. I think
|distcp| cannot copy a file into a directory. It just can copy
directories, or copy a file into |/| dir.
On 08/17/2015 11:30 PM, Vernon Funkhouser wrote:
> I would first check the directory permissions. If they are set to
> read only thenn you cannot write to it. They must be set to
> read/write to copy files.
>
> Cheers
> Vern
>
>
> On Mon, 2015-08-17 at 21:46 +0100, xeonmailinglist wrote:
>> Hi,
>>
>> I am trying to understand why I can't copy files between HDFS filesytems
>> to a directory "/input1", but I can copy files to the root dir "/".
>>
>> Eg, I have 2 hosts (hadoop-coc-2, and hadoop-coc3), and I can't copy
>> files between the dir "input1" [1], but I can copy the file to the root
>> dir [2].
>>
>> Initially only hadoop-coc-2 (192.168.56.102) have the data. hadoop-coc-3
>> is empty. After I tried to copy the file to hadoop-coc-3, only the dir
>> "input1 was created in the destination.
>>
>> I don't understand why this is happening. Any help?
>>
>> [1]
>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>> hdfs://192.168.56.103:9000/input1/`
>>
>> [2]
>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>> hdfs://192.168.56.103:9000/`
>>
>> Thanks,
>>
Re: can't copy files between HDFS to a specific dir
Posted by xeonmailinglist <xe...@gmail.com>.
In my case it has nothing to do with directory permissions. I think
|distcp| cannot copy a file into a directory. It just can copy
directories, or copy a file into |/| dir.
On 08/17/2015 11:30 PM, Vernon Funkhouser wrote:
> I would first check the directory permissions. If they are set to
> read only thenn you cannot write to it. They must be set to
> read/write to copy files.
>
> Cheers
> Vern
>
>
> On Mon, 2015-08-17 at 21:46 +0100, xeonmailinglist wrote:
>> Hi,
>>
>> I am trying to understand why I can't copy files between HDFS filesytems
>> to a directory "/input1", but I can copy files to the root dir "/".
>>
>> Eg, I have 2 hosts (hadoop-coc-2, and hadoop-coc3), and I can't copy
>> files between the dir "input1" [1], but I can copy the file to the root
>> dir [2].
>>
>> Initially only hadoop-coc-2 (192.168.56.102) have the data. hadoop-coc-3
>> is empty. After I tried to copy the file to hadoop-coc-3, only the dir
>> "input1 was created in the destination.
>>
>> I don't understand why this is happening. Any help?
>>
>> [1]
>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>> hdfs://192.168.56.103:9000/input1/`
>>
>> [2]
>> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
>> hdfs://192.168.56.103:9000/`
>>
>> Thanks,
>>
Re: can't copy files between HDFS to a specific dir
Posted by Vernon Funkhouser <vf...@earthlink.net>.
I would first check the directory permissions. If they are set to read
only thenn you cannot write to it. They must be set to read/write to
copy files.
Cheers
Vern
On Mon, 2015-08-17 at 21:46 +0100, xeonmailinglist wrote:
> Hi,
>
> I am trying to understand why I can't copy files between HDFS filesytems
> to a directory "/input1", but I can copy files to the root dir "/".
>
> Eg, I have 2 hosts (hadoop-coc-2, and hadoop-coc3), and I can't copy
> files between the dir "input1" [1], but I can copy the file to the root
> dir [2].
>
> Initially only hadoop-coc-2 (192.168.56.102) have the data. hadoop-coc-3
> is empty. After I tried to copy the file to hadoop-coc-3, only the dir
> "input1 was created in the destination.
>
> I don't understand why this is happening. Any help?
>
> [1]
> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
> hdfs://192.168.56.103:9000/input1/`
>
> [2]
> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
> hdfs://192.168.56.103:9000/`
>
> Thanks,
>
Re: can't copy files between HDFS to a specific dir
Posted by Vernon Funkhouser <vf...@earthlink.net>.
I would first check the directory permissions. If they are set to read
only thenn you cannot write to it. They must be set to read/write to
copy files.
Cheers
Vern
On Mon, 2015-08-17 at 21:46 +0100, xeonmailinglist wrote:
> Hi,
>
> I am trying to understand why I can't copy files between HDFS filesytems
> to a directory "/input1", but I can copy files to the root dir "/".
>
> Eg, I have 2 hosts (hadoop-coc-2, and hadoop-coc3), and I can't copy
> files between the dir "input1" [1], but I can copy the file to the root
> dir [2].
>
> Initially only hadoop-coc-2 (192.168.56.102) have the data. hadoop-coc-3
> is empty. After I tried to copy the file to hadoop-coc-3, only the dir
> "input1 was created in the destination.
>
> I don't understand why this is happening. Any help?
>
> [1]
> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
> hdfs://192.168.56.103:9000/input1/`
>
> [2]
> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
> hdfs://192.168.56.103:9000/`
>
> Thanks,
>
Re: can't copy files between HDFS to a specific dir
Posted by Vernon Funkhouser <vf...@earthlink.net>.
I would first check the directory permissions. If they are set to read
only thenn you cannot write to it. They must be set to read/write to
copy files.
Cheers
Vern
On Mon, 2015-08-17 at 21:46 +0100, xeonmailinglist wrote:
> Hi,
>
> I am trying to understand why I can't copy files between HDFS filesytems
> to a directory "/input1", but I can copy files to the root dir "/".
>
> Eg, I have 2 hosts (hadoop-coc-2, and hadoop-coc3), and I can't copy
> files between the dir "input1" [1], but I can copy the file to the root
> dir [2].
>
> Initially only hadoop-coc-2 (192.168.56.102) have the data. hadoop-coc-3
> is empty. After I tried to copy the file to hadoop-coc-3, only the dir
> "input1 was created in the destination.
>
> I don't understand why this is happening. Any help?
>
> [1]
> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
> hdfs://192.168.56.103:9000/input1/`
>
> [2]
> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
> hdfs://192.168.56.103:9000/`
>
> Thanks,
>
Re: can't copy files between HDFS to a specific dir
Posted by Vernon Funkhouser <vf...@earthlink.net>.
I would first check the directory permissions. If they are set to read
only thenn you cannot write to it. They must be set to read/write to
copy files.
Cheers
Vern
On Mon, 2015-08-17 at 21:46 +0100, xeonmailinglist wrote:
> Hi,
>
> I am trying to understand why I can't copy files between HDFS filesytems
> to a directory "/input1", but I can copy files to the root dir "/".
>
> Eg, I have 2 hosts (hadoop-coc-2, and hadoop-coc3), and I can't copy
> files between the dir "input1" [1], but I can copy the file to the root
> dir [2].
>
> Initially only hadoop-coc-2 (192.168.56.102) have the data. hadoop-coc-3
> is empty. After I tried to copy the file to hadoop-coc-3, only the dir
> "input1 was created in the destination.
>
> I don't understand why this is happening. Any help?
>
> [1]
> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
> hdfs://192.168.56.103:9000/input1/`
>
> [2]
> `hadoop distcp hdfs://192.168.56.102:9000/input1/a.txt
> hdfs://192.168.56.103:9000/`
>
> Thanks,
>