You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Nataraj Rashmi - rnatar <Ra...@acxiom.com> on 2013/01/23 23:22:08 UTC
Error after upgrading from CDH3 to CDH4
Hi,
I am getting this error when I run the java application that uses HDFS API to transfer files to HDFS remotely. This used to work fine with CDH3 and now we are using CDH4.
Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2206)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2213)
I have this in my client config cml. Please help.
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://servername:8020</value>
</property>
</configuration>
Thanks.
***************************************************************************
The information contained in this communication is confidential, is
intended only for the use of the recipient named above, and may be legally
privileged.
If the reader of this message is not the intended recipient, you are
hereby notified that any dissemination, distribution or copying of this
communication is strictly prohibited.
If you have received this communication in error, please resend this
communication to the sender and delete the original message or any copy
of it from your computer system.
Thank You.
****************************************************************************
Re: Error after upgrading from CDH3 to CDH4
Posted by Harsh J <ha...@cloudera.com>.
Hi Rashmi,
Moving the post to cdh-user@cloudera.org
(https://groups.google.com/a/cloudera.org/forum/?fromgroups=#!forum/cdh-user)
as it is CDH4 you specifically are asking about. BCC'd user@hadoop
lists, lets carry forward the discussion on the CDH lists. My response
below.
With CDH4 (basically with 2.x+), the project now supplies 3 sets of
packages each for common libs, hdfs libs and yarn (and also mr) libs.
Your error would appear when you run the program without the HDFS
libs, since the HDFS implementation classes are now discovered
automatically from the available classpath at runtime.
If you can tell us what dependencies your project uses, I can help you
fix this further. But essentially you need to try to place HDFS libs
onto the application's runtime classpath.
General note: Your application should be using hadoop-client
dependencies as pointed out by
https://ccp.cloudera.com/display/CDH4DOC/Managing+Hadoop+API+Dependencies+in+CDH4
and https://ccp.cloudera.com/display/DOC/Hadoop+Tutorial, not the
single hadoop-core jar as was in CDH3.
On Thu, Jan 24, 2013 at 3:52 AM, Nataraj Rashmi - rnatar
<Ra...@acxiom.com> wrote:
> Hi,
>
>
>
> I am getting this error when I run the java application that uses HDFS API
> to transfer files to HDFS remotely. This used to work fine with CDH3 and
> now we are using CDH4.
>
>
>
> Exception in thread "main" java.io.IOException: No FileSystem for scheme:
> hdfs
>
> at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2206)
>
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2213)
>
>
>
> I have this in my client config cml. Please help.
>
>
>
> <configuration>
>
> <property>
>
> <name>fs.default.name</name>
>
> <value>hdfs://servername:8020</value>
>
> </property>
>
> </configuration>
>
>
>
>
>
> Thanks.
>
> ***************************************************************************
> The information contained in this communication is confidential, is
> intended only for the use of the recipient named above, and may be legally
> privileged.
>
> If the reader of this message is not the intended recipient, you are
> hereby notified that any dissemination, distribution or copying of this
> communication is strictly prohibited.
>
> If you have received this communication in error, please resend this
> communication to the sender and delete the original message or any copy
> of it from your computer system.
>
> Thank You.
> ****************************************************************************
--
Harsh J
Re: Error after upgrading from CDH3 to CDH4
Posted by Harsh J <ha...@cloudera.com>.
Hi Rashmi,
Moving the post to cdh-user@cloudera.org
(https://groups.google.com/a/cloudera.org/forum/?fromgroups=#!forum/cdh-user)
as it is CDH4 you specifically are asking about. BCC'd user@hadoop
lists, lets carry forward the discussion on the CDH lists. My response
below.
With CDH4 (basically with 2.x+), the project now supplies 3 sets of
packages each for common libs, hdfs libs and yarn (and also mr) libs.
Your error would appear when you run the program without the HDFS
libs, since the HDFS implementation classes are now discovered
automatically from the available classpath at runtime.
If you can tell us what dependencies your project uses, I can help you
fix this further. But essentially you need to try to place HDFS libs
onto the application's runtime classpath.
General note: Your application should be using hadoop-client
dependencies as pointed out by
https://ccp.cloudera.com/display/CDH4DOC/Managing+Hadoop+API+Dependencies+in+CDH4
and https://ccp.cloudera.com/display/DOC/Hadoop+Tutorial, not the
single hadoop-core jar as was in CDH3.
On Thu, Jan 24, 2013 at 3:52 AM, Nataraj Rashmi - rnatar
<Ra...@acxiom.com> wrote:
> Hi,
>
>
>
> I am getting this error when I run the java application that uses HDFS API
> to transfer files to HDFS remotely. This used to work fine with CDH3 and
> now we are using CDH4.
>
>
>
> Exception in thread "main" java.io.IOException: No FileSystem for scheme:
> hdfs
>
> at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2206)
>
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2213)
>
>
>
> I have this in my client config cml. Please help.
>
>
>
> <configuration>
>
> <property>
>
> <name>fs.default.name</name>
>
> <value>hdfs://servername:8020</value>
>
> </property>
>
> </configuration>
>
>
>
>
>
> Thanks.
>
> ***************************************************************************
> The information contained in this communication is confidential, is
> intended only for the use of the recipient named above, and may be legally
> privileged.
>
> If the reader of this message is not the intended recipient, you are
> hereby notified that any dissemination, distribution or copying of this
> communication is strictly prohibited.
>
> If you have received this communication in error, please resend this
> communication to the sender and delete the original message or any copy
> of it from your computer system.
>
> Thank You.
> ****************************************************************************
--
Harsh J
Re: Error after upgrading from CDH3 to CDH4
Posted by Harsh J <ha...@cloudera.com>.
Hi Rashmi,
Moving the post to cdh-user@cloudera.org
(https://groups.google.com/a/cloudera.org/forum/?fromgroups=#!forum/cdh-user)
as it is CDH4 you specifically are asking about. BCC'd user@hadoop
lists, lets carry forward the discussion on the CDH lists. My response
below.
With CDH4 (basically with 2.x+), the project now supplies 3 sets of
packages each for common libs, hdfs libs and yarn (and also mr) libs.
Your error would appear when you run the program without the HDFS
libs, since the HDFS implementation classes are now discovered
automatically from the available classpath at runtime.
If you can tell us what dependencies your project uses, I can help you
fix this further. But essentially you need to try to place HDFS libs
onto the application's runtime classpath.
General note: Your application should be using hadoop-client
dependencies as pointed out by
https://ccp.cloudera.com/display/CDH4DOC/Managing+Hadoop+API+Dependencies+in+CDH4
and https://ccp.cloudera.com/display/DOC/Hadoop+Tutorial, not the
single hadoop-core jar as was in CDH3.
On Thu, Jan 24, 2013 at 3:52 AM, Nataraj Rashmi - rnatar
<Ra...@acxiom.com> wrote:
> Hi,
>
>
>
> I am getting this error when I run the java application that uses HDFS API
> to transfer files to HDFS remotely. This used to work fine with CDH3 and
> now we are using CDH4.
>
>
>
> Exception in thread "main" java.io.IOException: No FileSystem for scheme:
> hdfs
>
> at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2206)
>
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2213)
>
>
>
> I have this in my client config cml. Please help.
>
>
>
> <configuration>
>
> <property>
>
> <name>fs.default.name</name>
>
> <value>hdfs://servername:8020</value>
>
> </property>
>
> </configuration>
>
>
>
>
>
> Thanks.
>
> ***************************************************************************
> The information contained in this communication is confidential, is
> intended only for the use of the recipient named above, and may be legally
> privileged.
>
> If the reader of this message is not the intended recipient, you are
> hereby notified that any dissemination, distribution or copying of this
> communication is strictly prohibited.
>
> If you have received this communication in error, please resend this
> communication to the sender and delete the original message or any copy
> of it from your computer system.
>
> Thank You.
> ****************************************************************************
--
Harsh J
Re: Error after upgrading from CDH3 to CDH4
Posted by Harsh J <ha...@cloudera.com>.
Hi Rashmi,
Moving the post to cdh-user@cloudera.org
(https://groups.google.com/a/cloudera.org/forum/?fromgroups=#!forum/cdh-user)
as it is CDH4 you specifically are asking about. BCC'd user@hadoop
lists, lets carry forward the discussion on the CDH lists. My response
below.
With CDH4 (basically with 2.x+), the project now supplies 3 sets of
packages each for common libs, hdfs libs and yarn (and also mr) libs.
Your error would appear when you run the program without the HDFS
libs, since the HDFS implementation classes are now discovered
automatically from the available classpath at runtime.
If you can tell us what dependencies your project uses, I can help you
fix this further. But essentially you need to try to place HDFS libs
onto the application's runtime classpath.
General note: Your application should be using hadoop-client
dependencies as pointed out by
https://ccp.cloudera.com/display/CDH4DOC/Managing+Hadoop+API+Dependencies+in+CDH4
and https://ccp.cloudera.com/display/DOC/Hadoop+Tutorial, not the
single hadoop-core jar as was in CDH3.
On Thu, Jan 24, 2013 at 3:52 AM, Nataraj Rashmi - rnatar
<Ra...@acxiom.com> wrote:
> Hi,
>
>
>
> I am getting this error when I run the java application that uses HDFS API
> to transfer files to HDFS remotely. This used to work fine with CDH3 and
> now we are using CDH4.
>
>
>
> Exception in thread "main" java.io.IOException: No FileSystem for scheme:
> hdfs
>
> at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2206)
>
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2213)
>
>
>
> I have this in my client config cml. Please help.
>
>
>
> <configuration>
>
> <property>
>
> <name>fs.default.name</name>
>
> <value>hdfs://servername:8020</value>
>
> </property>
>
> </configuration>
>
>
>
>
>
> Thanks.
>
> ***************************************************************************
> The information contained in this communication is confidential, is
> intended only for the use of the recipient named above, and may be legally
> privileged.
>
> If the reader of this message is not the intended recipient, you are
> hereby notified that any dissemination, distribution or copying of this
> communication is strictly prohibited.
>
> If you have received this communication in error, please resend this
> communication to the sender and delete the original message or any copy
> of it from your computer system.
>
> Thank You.
> ****************************************************************************
--
Harsh J