You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Dhanasekaran Anbalagan <bu...@gmail.com> on 2013/02/15 17:22:01 UTC
Hadoop Distcp [ Incomplete HDFS URI ]
HI Guys,
we have two cluster running with CDH4.0.1 I am trying data one cluster to
another cluster.
It's says Incomplete HDFS URI
tech@dvcliftonhera227:~$ hadoop distcp
hdfs://172.16.30.122:8020:/user/thirumal/test2/part-00000
hdfs://172.16.30.227:/user/tech
13/02/15 11:13:15 INFO tools.DistCp: srcPaths=[hdfs://172.16.30.122:8020
:/user/thirumal/test2/part-00000]
13/02/15 11:13:15 INFO tools.DistCp: destPath=hdfs://172.16.30.227:
/user/tech
With failures, global counters are inaccurate; consider running with -i
Copy failed: java.io.IOException: Incomplete HDFS URI, no host:
hdfs://172.16.30.122:8020:/user/thirumal/test2/part-00000
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:118)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2150)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2184)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2166)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:302)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:194)
at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:635)
at org.apache.hadoop.tools.DistCp.copy(DistCp.java:656)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:881)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.tools.DistCp.main(DistCp.java:908)
I try to debug, The port listening
How to resolve this. Please guide me.
$telnet 172.16.30.122 8020
Trying 172.16.30.122...
Connected to 172.16.30.122.
Escape character is '^]'.
^]
telnet> quit
$ telnet 172.16.30.227 8020
Trying 172.16.30.227...
Connected to 172.16.30.227.
Escape character is '^]'.
^]
telnet> quit
Connection closed.
#####################core-site.xml
<property>
<name>fs.defaultFS</name>
<value>hdfs://dvcliftonhera227:*8020*</value>
</property>
-Dhanasekaran
Did I learn something today? If not, I wasted it.
Re: Hadoop Distcp [ Incomplete HDFS URI ]
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
it seems you are having an extra : before the first / in your uris.
thx
Alejandro
(phone typing)
On Feb 15, 2013, at 8:22 AM, Dhanasekaran Anbalagan <bu...@gmail.com> wrote:
> HI Guys,
>
> we have two cluster running with CDH4.0.1 I am trying data one cluster to another cluster.
> It's says Incomplete HDFS URI
>
> tech@dvcliftonhera227:~$ hadoop distcp hdfs://172.16.30.122:8020:/user/thirumal/test2/part-00000 hdfs://172.16.30.227:/user/tech
> 13/02/15 11:13:15 INFO tools.DistCp: srcPaths=[hdfs://172.16.30.122:8020:/user/thirumal/test2/part-00000]
> 13/02/15 11:13:15 INFO tools.DistCp: destPath=hdfs://172.16.30.227:/user/tech
> With failures, global counters are inaccurate; consider running with -i
> Copy failed: java.io.IOException: Incomplete HDFS URI, no host: hdfs://172.16.30.122:8020:/user/thirumal/test2/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:118)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2150)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2184)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2166)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:302)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:194)
> at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:635)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:656)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:881)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.tools.DistCp.main(DistCp.java:908)
>
>
> I try to debug, The port listening
> How to resolve this. Please guide me.
>
> $telnet 172.16.30.122 8020
> Trying 172.16.30.122...
> Connected to 172.16.30.122.
> Escape character is '^]'.
> ^]
> telnet> quit
>
> $ telnet 172.16.30.227 8020
> Trying 172.16.30.227...
> Connected to 172.16.30.227.
> Escape character is '^]'.
> ^]
> telnet> quit
> Connection closed.
>
> #####################core-site.xml
>
> <property>
> <name>fs.defaultFS</name>
> <value>hdfs://dvcliftonhera227:8020</value>
> </property>
>
>
> -Dhanasekaran
>
> Did I learn something today? If not, I wasted it.
> --
>
>
>
Re: Hadoop Distcp [ Incomplete HDFS URI ]
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
it seems you are having an extra : before the first / in your uris.
thx
Alejandro
(phone typing)
On Feb 15, 2013, at 8:22 AM, Dhanasekaran Anbalagan <bu...@gmail.com> wrote:
> HI Guys,
>
> we have two cluster running with CDH4.0.1 I am trying data one cluster to another cluster.
> It's says Incomplete HDFS URI
>
> tech@dvcliftonhera227:~$ hadoop distcp hdfs://172.16.30.122:8020:/user/thirumal/test2/part-00000 hdfs://172.16.30.227:/user/tech
> 13/02/15 11:13:15 INFO tools.DistCp: srcPaths=[hdfs://172.16.30.122:8020:/user/thirumal/test2/part-00000]
> 13/02/15 11:13:15 INFO tools.DistCp: destPath=hdfs://172.16.30.227:/user/tech
> With failures, global counters are inaccurate; consider running with -i
> Copy failed: java.io.IOException: Incomplete HDFS URI, no host: hdfs://172.16.30.122:8020:/user/thirumal/test2/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:118)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2150)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2184)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2166)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:302)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:194)
> at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:635)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:656)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:881)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.tools.DistCp.main(DistCp.java:908)
>
>
> I try to debug, The port listening
> How to resolve this. Please guide me.
>
> $telnet 172.16.30.122 8020
> Trying 172.16.30.122...
> Connected to 172.16.30.122.
> Escape character is '^]'.
> ^]
> telnet> quit
>
> $ telnet 172.16.30.227 8020
> Trying 172.16.30.227...
> Connected to 172.16.30.227.
> Escape character is '^]'.
> ^]
> telnet> quit
> Connection closed.
>
> #####################core-site.xml
>
> <property>
> <name>fs.defaultFS</name>
> <value>hdfs://dvcliftonhera227:8020</value>
> </property>
>
>
> -Dhanasekaran
>
> Did I learn something today? If not, I wasted it.
> --
>
>
>
Re: Hadoop Distcp [ Incomplete HDFS URI ]
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
it seems you are having an extra : before the first / in your uris.
thx
Alejandro
(phone typing)
On Feb 15, 2013, at 8:22 AM, Dhanasekaran Anbalagan <bu...@gmail.com> wrote:
> HI Guys,
>
> we have two cluster running with CDH4.0.1 I am trying data one cluster to another cluster.
> It's says Incomplete HDFS URI
>
> tech@dvcliftonhera227:~$ hadoop distcp hdfs://172.16.30.122:8020:/user/thirumal/test2/part-00000 hdfs://172.16.30.227:/user/tech
> 13/02/15 11:13:15 INFO tools.DistCp: srcPaths=[hdfs://172.16.30.122:8020:/user/thirumal/test2/part-00000]
> 13/02/15 11:13:15 INFO tools.DistCp: destPath=hdfs://172.16.30.227:/user/tech
> With failures, global counters are inaccurate; consider running with -i
> Copy failed: java.io.IOException: Incomplete HDFS URI, no host: hdfs://172.16.30.122:8020:/user/thirumal/test2/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:118)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2150)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2184)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2166)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:302)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:194)
> at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:635)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:656)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:881)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.tools.DistCp.main(DistCp.java:908)
>
>
> I try to debug, The port listening
> How to resolve this. Please guide me.
>
> $telnet 172.16.30.122 8020
> Trying 172.16.30.122...
> Connected to 172.16.30.122.
> Escape character is '^]'.
> ^]
> telnet> quit
>
> $ telnet 172.16.30.227 8020
> Trying 172.16.30.227...
> Connected to 172.16.30.227.
> Escape character is '^]'.
> ^]
> telnet> quit
> Connection closed.
>
> #####################core-site.xml
>
> <property>
> <name>fs.defaultFS</name>
> <value>hdfs://dvcliftonhera227:8020</value>
> </property>
>
>
> -Dhanasekaran
>
> Did I learn something today? If not, I wasted it.
> --
>
>
>
Re: Hadoop Distcp [ Incomplete HDFS URI ]
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
it seems you are having an extra : before the first / in your uris.
thx
Alejandro
(phone typing)
On Feb 15, 2013, at 8:22 AM, Dhanasekaran Anbalagan <bu...@gmail.com> wrote:
> HI Guys,
>
> we have two cluster running with CDH4.0.1 I am trying data one cluster to another cluster.
> It's says Incomplete HDFS URI
>
> tech@dvcliftonhera227:~$ hadoop distcp hdfs://172.16.30.122:8020:/user/thirumal/test2/part-00000 hdfs://172.16.30.227:/user/tech
> 13/02/15 11:13:15 INFO tools.DistCp: srcPaths=[hdfs://172.16.30.122:8020:/user/thirumal/test2/part-00000]
> 13/02/15 11:13:15 INFO tools.DistCp: destPath=hdfs://172.16.30.227:/user/tech
> With failures, global counters are inaccurate; consider running with -i
> Copy failed: java.io.IOException: Incomplete HDFS URI, no host: hdfs://172.16.30.122:8020:/user/thirumal/test2/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:118)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2150)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2184)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2166)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:302)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:194)
> at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:635)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:656)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:881)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.tools.DistCp.main(DistCp.java:908)
>
>
> I try to debug, The port listening
> How to resolve this. Please guide me.
>
> $telnet 172.16.30.122 8020
> Trying 172.16.30.122...
> Connected to 172.16.30.122.
> Escape character is '^]'.
> ^]
> telnet> quit
>
> $ telnet 172.16.30.227 8020
> Trying 172.16.30.227...
> Connected to 172.16.30.227.
> Escape character is '^]'.
> ^]
> telnet> quit
> Connection closed.
>
> #####################core-site.xml
>
> <property>
> <name>fs.defaultFS</name>
> <value>hdfs://dvcliftonhera227:8020</value>
> </property>
>
>
> -Dhanasekaran
>
> Did I learn something today? If not, I wasted it.
> --
>
>
>