You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by elton sky <el...@gmail.com> on 2010/06/24 08:42:34 UTC
Could not get FileSystem obj, get java.lang.NullPointerException !!
Hi,
I am new to hadoop programming. I am trying to copy a local file to HDFS. My
code snippet is:
.
.
Configuration conf = new Configuration();
InputStream in=null;
OutputStream out = null;
try {
in = new BufferedInputStream(new FileInputStream(src));
FileSystem fs = FileSystem.get(URI.create(dest), conf); *//line:42.
Exception here*
if(fs == null)
System.out.println("fs is null");
else
System.out.println("fs is not null");
out = fs.create(new Path(dest), true);
IOUtils.copyBytes(in, out, conf, true);
System.out.println(dest +" is created successfully.");
} catch (IOException e) {
IOUtils.closeStream(out);
IOUtils.closeStream(in);
throw e;
}
.
.
and when I run it, I got exception:
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.conf.Configuration.get(Configuration.java:436)
at
org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:103)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
at HDFSFile.create(HDFSFile.java:42)
at HDFSFile.main(HDFSFile.java:130)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at
java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156
After dug into source code of hdfs, I found the problem happens at
"Configuration.getProps()". The value of getProps() is null and, obviously,
it shouldn't be.
I couldn't figure out why this happen. Please help me out here.
Appreciate!
Re: Could not get FileSystem obj, get java.lang.NullPointerException
!!
Posted by chaitanya krishna <ch...@gmail.com>.
Hi Elton,
Can you mention the hadoop version? Also, can you double-check if you set
"fs.default.name" property correctly in conf/hdfs-site.xml?
-Chaitanya.
On Thu, Jun 24, 2010 at 12:12 PM, elton sky <el...@gmail.com> wrote:
> Hi,
> I am new to hadoop programming. I am trying to copy a local file to HDFS.
> My
> code snippet is:
> .
> .
> Configuration conf = new Configuration();
>
> InputStream in=null;
> OutputStream out = null;
> try {
> in = new BufferedInputStream(new FileInputStream(src));
>
> FileSystem fs = FileSystem.get(URI.create(dest), conf);
> *//line:42.
> Exception here*
>
> if(fs == null)
> System.out.println("fs is null");
> else
> System.out.println("fs is not null");
>
> out = fs.create(new Path(dest), true);
> IOUtils.copyBytes(in, out, conf, true);
>
> System.out.println(dest +" is created successfully.");
>
> } catch (IOException e) {
> IOUtils.closeStream(out);
> IOUtils.closeStream(in);
> throw e;
> }
> .
> .
> and when I run it, I got exception:
> Exception in thread "main" java.lang.NullPointerException
>
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:436)
>
> at
> org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:103)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
>
> at HDFSFile.create(HDFSFile.java:42)
>
> at HDFSFile.main(HDFSFile.java:130)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at
> java.lang.reflect.Method.invoke(Method.java:616)
>
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156
>
> After dug into source code of hdfs, I found the problem happens at
> "Configuration.getProps()". The value of getProps() is null and, obviously,
> it shouldn't be.
>
> I couldn't figure out why this happen. Please help me out here.
> Appreciate!
>
Re: Could not get FileSystem obj, get java.lang.NullPointerException
!!
Posted by elton sky <el...@gmail.com>.
Thanks for reply,
On Thu, Jun 24, 2010 at 7:35 PM, Steve Loughran <st...@apache.org> wrote:
> elton sky wrote:
>
>> Hi,
>> I am new to hadoop programming.
>>
>
>
> OK. First hint of advice: don't spam the dev lists with user problems, you
> will only get ignored in both mailing lists.
>
>
Get ur advice :)
>
> I am trying to copy a local file to HDFS. My
>
> .
>> and when I run it, I got exception:
>> Exception in thread "main" java.lang.NullPointerException
>>
>> at org.apache.hadoop.conf.Configuration.get(Configuration.java:436)
>>
>> at
>> org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:103)
>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
>>
>> at HDFSFile.create(HDFSFile.java:42)
>>
>> at HDFSFile.main(HDFSFile.java:130)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>> at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at
>> java.lang.reflect.Method.invoke(Method.java:616)
>>
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156
>>
>> After dug into source code of hdfs, I found the problem happens at
>> "Configuration.getProps()". The value of getProps() is null and,
>> obviously,
>> it shouldn't be.
>>
>
> This is time to reach for your debugger. We don't know why it is null,
> probably something odd with how your configuration is booted up. Follow the
> constructor.
>
> The other tactic, and the one I do first, is search for that stack trace on
> the web and on https://issues.apache.org/jira/ That way, if someone has
> hit the same problem as you, you'll find the solution
>
I mistakenly put :
static {
URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());
}
on the top of the code, which makes the exception. I commented out and it
just works.
I have no idea why this matters?
Re: Could not get FileSystem obj, get java.lang.NullPointerException
!!
Posted by Steve Loughran <st...@apache.org>.
elton sky wrote:
> Hi,
> I am new to hadoop programming.
OK. First hint of advice: don't spam the dev lists with user problems,
you will only get ignored in both mailing lists.
I am trying to copy a local file to HDFS. My
> .
> and when I run it, I got exception:
> Exception in thread "main" java.lang.NullPointerException
>
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:436)
>
> at
> org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:103)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
>
> at HDFSFile.create(HDFSFile.java:42)
>
> at HDFSFile.main(HDFSFile.java:130)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at
> java.lang.reflect.Method.invoke(Method.java:616)
>
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156
>
> After dug into source code of hdfs, I found the problem happens at
> "Configuration.getProps()". The value of getProps() is null and, obviously,
> it shouldn't be.
This is time to reach for your debugger. We don't know why it is null,
probably something odd with how your configuration is booted up. Follow
the constructor.
The other tactic, and the one I do first, is search for that stack trace
on the web and on https://issues.apache.org/jira/ That way, if someone
has hit the same problem as you, you'll find the solution