You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Sandeep Reddy P <sa...@gmail.com> on 2012/07/18 17:33:09 UTC
Error:Hdfs Client for hadoop using native java api
Hi,
I'm trying to load data into hdfs from local linux file system using java
code from a windows machine.But i'm getting the error
java.lang.IllegalArgumentException: Wrong FS:
hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
at
org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
at
org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
at
org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
at Hdfs.main(Hdfs.java:18)
File not found
Can any one please help me with the issue.
--
Thanks,
sandeep
Re: Error:Hdfs Client for hadoop using native java api
Posted by shashwat shriparv <dw...@gmail.com>.
Or try something like this
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
HFile.Writer hwriter = new HFileWriterV2(conf, new CacheConfig(conf), fs, new
Path(fs.getWorkingDirectory() + "/foo"));
On Thu, Jul 19, 2012 at 9:35 PM, shashwat shriparv <
dwivedishashwat@gmail.com> wrote:
> By the way you have these two options
>
>
> 1.
>
> Put the configuration files in the classpath, so that the code picks
> it.
> 2.
>
> Use Configuration.set()<http://hadoop.apache.org/common/docs/r0.21.0/api/org/apache/hadoop/conf/Configuration.html#set%28java.lang.String,%20java.lang.String%29>to set the required parameters in the code.
>
> because the configuration may be directing to local file whice code is nt
> abl to find
>
>
> On Thu, Jul 19, 2012 at 9:32 PM, shashwat shriparv <
> dwivedishashwat@gmail.com> wrote:
>
>> Can you provide the code(some part of it if you don't want to throw the
>> full code here) and let us know which part of your code is throwing this
>> error.
>>
>> Regards
>>
>> ∞
>> Shashwat Shriparv
>>
>>
>>
>>
>> On Thu, Jul 19, 2012 at 6:46 PM, Sandeep Reddy P <
>> sandeepreddy.3647@gmail.com> wrote:
>>
>>> Hi John,
>>> We have applications in windows. So our dev's need to connect to HDFS
>>> from
>>> eclipse installed in windows. I'm trying to put data from <local-file> to
>>> <hdfs-file> using java code from windows.
>>>
>>> On Thu, Jul 19, 2012 at 5:41 AM, John Hancock <jhancock1975@gmail.com
>>> >wrote:
>>>
>>> > Sandeep,
>>> >
>>> > I don't understand your situation completely, but why not just use
>>> > bin/hadoop dfs -copyFromLocal <local-file-name> <hdfs-file-name> ?
>>> >
>>> > -John
>>> >
>>> > On Wed, Jul 18, 2012 at 11:33 AM, Sandeep Reddy P <
>>> > sandeepreddy.3647@gmail.com> wrote:
>>> >
>>> > > Hi,
>>> > > I'm trying to load data into hdfs from local linux file system using
>>> java
>>> > > code from a windows machine.But i'm getting the error
>>> > >
>>> > > java.lang.IllegalArgumentException: Wrong FS:
>>> > > hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected: file:///
>>> > > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
>>> > > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
>>> > > at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
>>> > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
>>> > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
>>> > > at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
>>> > > at
>>> > >
>>> org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
>>> > > at Hdfs.main(Hdfs.java:18)
>>> > > File not found
>>> > >
>>> > > Can any one please help me with the issue.
>>> > >
>>> > > --
>>> > > Thanks,
>>> > > sandeep
>>> > >
>>> >
>>>
>>>
>>>
>>> --
>>> Thanks,
>>> sandeep
>>>
>>
>>
>>
>> --
>>
>>
>> ∞
>> Shashwat Shriparv
>>
>>
>>
>
>
> --
>
>
> ∞
> Shashwat Shriparv
>
>
>
--
∞
Shashwat Shriparv
Re: Error:Hdfs Client for hadoop using native java api
Posted by shashwat shriparv <dw...@gmail.com>.
By the way you have these two options
1.
Put the configuration files in the classpath, so that the code picks it.
2.
Use Configuration.set()<http://hadoop.apache.org/common/docs/r0.21.0/api/org/apache/hadoop/conf/Configuration.html#set%28java.lang.String,%20java.lang.String%29>to
set the required parameters in the code.
because the configuration may be directing to local file whice code is nt
abl to find
On Thu, Jul 19, 2012 at 9:32 PM, shashwat shriparv <
dwivedishashwat@gmail.com> wrote:
> Can you provide the code(some part of it if you don't want to throw the
> full code here) and let us know which part of your code is throwing this
> error.
>
> Regards
>
> ∞
> Shashwat Shriparv
>
>
>
>
> On Thu, Jul 19, 2012 at 6:46 PM, Sandeep Reddy P <
> sandeepreddy.3647@gmail.com> wrote:
>
>> Hi John,
>> We have applications in windows. So our dev's need to connect to HDFS from
>> eclipse installed in windows. I'm trying to put data from <local-file> to
>> <hdfs-file> using java code from windows.
>>
>> On Thu, Jul 19, 2012 at 5:41 AM, John Hancock <jhancock1975@gmail.com
>> >wrote:
>>
>> > Sandeep,
>> >
>> > I don't understand your situation completely, but why not just use
>> > bin/hadoop dfs -copyFromLocal <local-file-name> <hdfs-file-name> ?
>> >
>> > -John
>> >
>> > On Wed, Jul 18, 2012 at 11:33 AM, Sandeep Reddy P <
>> > sandeepreddy.3647@gmail.com> wrote:
>> >
>> > > Hi,
>> > > I'm trying to load data into hdfs from local linux file system using
>> java
>> > > code from a windows machine.But i'm getting the error
>> > >
>> > > java.lang.IllegalArgumentException: Wrong FS:
>> > > hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected: file:///
>> > > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
>> > > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
>> > > at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
>> > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
>> > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
>> > > at
>> > >
>> > >
>> >
>> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
>> > > at
>> > >
>> org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
>> > > at Hdfs.main(Hdfs.java:18)
>> > > File not found
>> > >
>> > > Can any one please help me with the issue.
>> > >
>> > > --
>> > > Thanks,
>> > > sandeep
>> > >
>> >
>>
>>
>>
>> --
>> Thanks,
>> sandeep
>>
>
>
>
> --
>
>
> ∞
> Shashwat Shriparv
>
>
>
--
∞
Shashwat Shriparv
Re: Error:Hdfs Client for hadoop using native java api
Posted by Shailesh Samudrala <sh...@gmail.com>.
Hi Sandeep,
Can you tell us how you fixed it please.
On Sun, Jul 22, 2012 at 2:48 PM, Sandeep Reddy P <
sandeepreddy.3647@gmail.com> wrote:
> Thank you every one for the help. Issue is fixed.
>
> On Sun, Jul 22, 2012 at 4:31 PM, shashwat shriparv <
> dwivedishashwat@gmail.com> wrote:
>
> > there may be a problem with hostname resolution check if it is forward
> and
> > backword resolvable...and if you have any ip address anywhere please
> > replace it with hostname and the try..
> >
> > On Sun, Jul 22, 2012 at 9:56 PM, Minh Duc Nguyen <md...@gmail.com>
> > wrote:
> >
> > > As Shaswat mentioned previously, you're problem may be related to your
> > > configuration.
> > >
> > > Is core-site.xml on your classpath? For example, what is the value for
> > > conf.get("fs.default.name")?
> > >
> > > Alternatively, you can set this property directly in your code:
> > >
> > > conf.set("fs.default.name", "hdfs://hadoop1.devqa.local:8020");
> > >
> > > HTH,
> > > Minh
> > >
> > > On Thu, Jul 19, 2012 at 1:18 PM, Sandeep Reddy P <
> > > sandeepreddy.3647@gmail.com> wrote:
> > >
> > > > Hi Shashwat,
> > > > Here is the snippet of code which is throwing error
> > > > Path phdfs = new Path(
> > > >
> "hdfs://hadoop1.devqa.local:8020/user/hdfs/java/");
> > > > java.lang.IllegalArgumentException: Wrong FS:
> > > > hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected: file:///
> > > > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> > > > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
> > > > at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
> > > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
> > > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
> > > > at
> > > >
> org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
> > > > at Hdfs.main(Hdfs.java:18)
> > > >
> > > > On Thu, Jul 19, 2012 at 12:02 PM, shashwat shriparv <
> > > > dwivedishashwat@gmail.com> wrote:
> > > >
> > > > > Can you provide the code(some part of it if you don't want to throw
> > the
> > > > > full code here) and let us know which part of your code is throwing
> > > this
> > > > > error.
> > > > >
> > > > > Regards
> > > > >
> > > > > ∞
> > > > > Shashwat Shriparv
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Jul 19, 2012 at 6:46 PM, Sandeep Reddy P <
> > > > > sandeepreddy.3647@gmail.com> wrote:
> > > > >
> > > > > > Hi John,
> > > > > > We have applications in windows. So our dev's need to connect to
> > HDFS
> > > > > from
> > > > > > eclipse installed in windows. I'm trying to put data from
> > > <local-file>
> > > > to
> > > > > > <hdfs-file> using java code from windows.
> > > > > >
> > > > > > On Thu, Jul 19, 2012 at 5:41 AM, John Hancock <
> > > jhancock1975@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > Sandeep,
> > > > > > >
> > > > > > > I don't understand your situation completely, but why not just
> > use
> > > > > > > bin/hadoop dfs -copyFromLocal <local-file-name>
> <hdfs-file-name>
> > ?
> > > > > > >
> > > > > > > -John
> > > > > > >
> > > > > > > On Wed, Jul 18, 2012 at 11:33 AM, Sandeep Reddy P <
> > > > > > > sandeepreddy.3647@gmail.com> wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > > I'm trying to load data into hdfs from local linux file
> system
> > > > using
> > > > > > java
> > > > > > > > code from a windows machine.But i'm getting the error
> > > > > > > >
> > > > > > > > java.lang.IllegalArgumentException: Wrong FS:
> > > > > > > > hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected:
> > > file:///
> > > > > > > > at
> > > > org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
> > > > > > > > at
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
> > > > > > > > at
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
> > > > > > > > at
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> > > > > > > > at
> > > org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
> > > > > > > > at
> > org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
> > > > > > > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
> > > > > > > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
> > > > > > > > at
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
> > > > > > > > at
> > > > > > > >
> > > > >
> > org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
> > > > > > > > at Hdfs.main(Hdfs.java:18)
> > > > > > > > File not found
> > > > > > > >
> > > > > > > > Can any one please help me with the issue.
> > > > > > > >
> > > > > > > > --
> > > > > > > > Thanks,
> > > > > > > > sandeep
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Thanks,
> > > > > > sandeep
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > >
> > > > > ∞
> > > > > Shashwat Shriparv
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks,
> > > > sandeep
> > > >
> > >
> >
> >
> >
> > --
> >
> >
> > ∞
> > Shashwat Shriparv
> >
>
>
>
> --
> Thanks,
> sandeep
>
Re: Error:Hdfs Client for hadoop using native java api
Posted by Sandeep Reddy P <sa...@gmail.com>.
Thank you every one for the help. Issue is fixed.
On Sun, Jul 22, 2012 at 4:31 PM, shashwat shriparv <
dwivedishashwat@gmail.com> wrote:
> there may be a problem with hostname resolution check if it is forward and
> backword resolvable...and if you have any ip address anywhere please
> replace it with hostname and the try..
>
> On Sun, Jul 22, 2012 at 9:56 PM, Minh Duc Nguyen <md...@gmail.com>
> wrote:
>
> > As Shaswat mentioned previously, you're problem may be related to your
> > configuration.
> >
> > Is core-site.xml on your classpath? For example, what is the value for
> > conf.get("fs.default.name")?
> >
> > Alternatively, you can set this property directly in your code:
> >
> > conf.set("fs.default.name", "hdfs://hadoop1.devqa.local:8020");
> >
> > HTH,
> > Minh
> >
> > On Thu, Jul 19, 2012 at 1:18 PM, Sandeep Reddy P <
> > sandeepreddy.3647@gmail.com> wrote:
> >
> > > Hi Shashwat,
> > > Here is the snippet of code which is throwing error
> > > Path phdfs = new Path(
> > > "hdfs://hadoop1.devqa.local:8020/user/hdfs/java/");
> > > java.lang.IllegalArgumentException: Wrong FS:
> > > hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected: file:///
> > > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
> > > at
> > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
> > > at
> > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
> > > at
> > >
> > >
> >
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> > > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
> > > at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
> > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
> > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
> > > at
> > >
> > >
> >
> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
> > > at
> > > org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
> > > at Hdfs.main(Hdfs.java:18)
> > >
> > > On Thu, Jul 19, 2012 at 12:02 PM, shashwat shriparv <
> > > dwivedishashwat@gmail.com> wrote:
> > >
> > > > Can you provide the code(some part of it if you don't want to throw
> the
> > > > full code here) and let us know which part of your code is throwing
> > this
> > > > error.
> > > >
> > > > Regards
> > > >
> > > > ∞
> > > > Shashwat Shriparv
> > > >
> > > >
> > > >
> > > > On Thu, Jul 19, 2012 at 6:46 PM, Sandeep Reddy P <
> > > > sandeepreddy.3647@gmail.com> wrote:
> > > >
> > > > > Hi John,
> > > > > We have applications in windows. So our dev's need to connect to
> HDFS
> > > > from
> > > > > eclipse installed in windows. I'm trying to put data from
> > <local-file>
> > > to
> > > > > <hdfs-file> using java code from windows.
> > > > >
> > > > > On Thu, Jul 19, 2012 at 5:41 AM, John Hancock <
> > jhancock1975@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Sandeep,
> > > > > >
> > > > > > I don't understand your situation completely, but why not just
> use
> > > > > > bin/hadoop dfs -copyFromLocal <local-file-name> <hdfs-file-name>
> ?
> > > > > >
> > > > > > -John
> > > > > >
> > > > > > On Wed, Jul 18, 2012 at 11:33 AM, Sandeep Reddy P <
> > > > > > sandeepreddy.3647@gmail.com> wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > > I'm trying to load data into hdfs from local linux file system
> > > using
> > > > > java
> > > > > > > code from a windows machine.But i'm getting the error
> > > > > > >
> > > > > > > java.lang.IllegalArgumentException: Wrong FS:
> > > > > > > hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected:
> > file:///
> > > > > > > at
> > > org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
> > > > > > > at
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
> > > > > > > at
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
> > > > > > > at
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> > > > > > > at
> > org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
> > > > > > > at
> org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
> > > > > > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
> > > > > > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
> > > > > > > at
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
> > > > > > > at
> > > > > > >
> > > >
> org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
> > > > > > > at Hdfs.main(Hdfs.java:18)
> > > > > > > File not found
> > > > > > >
> > > > > > > Can any one please help me with the issue.
> > > > > > >
> > > > > > > --
> > > > > > > Thanks,
> > > > > > > sandeep
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Thanks,
> > > > > sandeep
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > >
> > > > ∞
> > > > Shashwat Shriparv
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > sandeep
> > >
> >
>
>
>
> --
>
>
> ∞
> Shashwat Shriparv
>
--
Thanks,
sandeep
Re: Error:Hdfs Client for hadoop using native java api
Posted by shashwat shriparv <dw...@gmail.com>.
there may be a problem with hostname resolution check if it is forward and
backword resolvable...and if you have any ip address anywhere please
replace it with hostname and the try..
On Sun, Jul 22, 2012 at 9:56 PM, Minh Duc Nguyen <md...@gmail.com> wrote:
> As Shaswat mentioned previously, you're problem may be related to your
> configuration.
>
> Is core-site.xml on your classpath? For example, what is the value for
> conf.get("fs.default.name")?
>
> Alternatively, you can set this property directly in your code:
>
> conf.set("fs.default.name", "hdfs://hadoop1.devqa.local:8020");
>
> HTH,
> Minh
>
> On Thu, Jul 19, 2012 at 1:18 PM, Sandeep Reddy P <
> sandeepreddy.3647@gmail.com> wrote:
>
> > Hi Shashwat,
> > Here is the snippet of code which is throwing error
> > Path phdfs = new Path(
> > "hdfs://hadoop1.devqa.local:8020/user/hdfs/java/");
> > java.lang.IllegalArgumentException: Wrong FS:
> > hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected: file:///
> > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
> > at
> >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
> > at
> >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
> > at
> >
> >
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
> > at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
> > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
> > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
> > at
> >
> >
> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
> > at
> > org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
> > at Hdfs.main(Hdfs.java:18)
> >
> > On Thu, Jul 19, 2012 at 12:02 PM, shashwat shriparv <
> > dwivedishashwat@gmail.com> wrote:
> >
> > > Can you provide the code(some part of it if you don't want to throw the
> > > full code here) and let us know which part of your code is throwing
> this
> > > error.
> > >
> > > Regards
> > >
> > > ∞
> > > Shashwat Shriparv
> > >
> > >
> > >
> > > On Thu, Jul 19, 2012 at 6:46 PM, Sandeep Reddy P <
> > > sandeepreddy.3647@gmail.com> wrote:
> > >
> > > > Hi John,
> > > > We have applications in windows. So our dev's need to connect to HDFS
> > > from
> > > > eclipse installed in windows. I'm trying to put data from
> <local-file>
> > to
> > > > <hdfs-file> using java code from windows.
> > > >
> > > > On Thu, Jul 19, 2012 at 5:41 AM, John Hancock <
> jhancock1975@gmail.com
> > > > >wrote:
> > > >
> > > > > Sandeep,
> > > > >
> > > > > I don't understand your situation completely, but why not just use
> > > > > bin/hadoop dfs -copyFromLocal <local-file-name> <hdfs-file-name> ?
> > > > >
> > > > > -John
> > > > >
> > > > > On Wed, Jul 18, 2012 at 11:33 AM, Sandeep Reddy P <
> > > > > sandeepreddy.3647@gmail.com> wrote:
> > > > >
> > > > > > Hi,
> > > > > > I'm trying to load data into hdfs from local linux file system
> > using
> > > > java
> > > > > > code from a windows machine.But i'm getting the error
> > > > > >
> > > > > > java.lang.IllegalArgumentException: Wrong FS:
> > > > > > hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected:
> file:///
> > > > > > at
> > org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
> > > > > > at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
> > > > > > at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
> > > > > > at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> > > > > > at
> org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
> > > > > > at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
> > > > > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
> > > > > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
> > > > > > at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
> > > > > > at
> > > > > >
> > > org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
> > > > > > at Hdfs.main(Hdfs.java:18)
> > > > > > File not found
> > > > > >
> > > > > > Can any one please help me with the issue.
> > > > > >
> > > > > > --
> > > > > > Thanks,
> > > > > > sandeep
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks,
> > > > sandeep
> > > >
> > >
> > >
> > >
> > > --
> > >
> > >
> > > ∞
> > > Shashwat Shriparv
> > >
> >
> >
> >
> > --
> > Thanks,
> > sandeep
> >
>
--
∞
Shashwat Shriparv
Re: Error:Hdfs Client for hadoop using native java api
Posted by Minh Duc Nguyen <md...@gmail.com>.
As Shaswat mentioned previously, you're problem may be related to your
configuration.
Is core-site.xml on your classpath? For example, what is the value for
conf.get("fs.default.name")?
Alternatively, you can set this property directly in your code:
conf.set("fs.default.name", "hdfs://hadoop1.devqa.local:8020");
HTH,
Minh
On Thu, Jul 19, 2012 at 1:18 PM, Sandeep Reddy P <
sandeepreddy.3647@gmail.com> wrote:
> Hi Shashwat,
> Here is the snippet of code which is throwing error
> Path phdfs = new Path(
> "hdfs://hadoop1.devqa.local:8020/user/hdfs/java/");
> java.lang.IllegalArgumentException: Wrong FS:
> hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected: file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
> at
>
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
> at
>
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
> at
>
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
> at
>
> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
> at
> org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
> at Hdfs.main(Hdfs.java:18)
>
> On Thu, Jul 19, 2012 at 12:02 PM, shashwat shriparv <
> dwivedishashwat@gmail.com> wrote:
>
> > Can you provide the code(some part of it if you don't want to throw the
> > full code here) and let us know which part of your code is throwing this
> > error.
> >
> > Regards
> >
> > ∞
> > Shashwat Shriparv
> >
> >
> >
> > On Thu, Jul 19, 2012 at 6:46 PM, Sandeep Reddy P <
> > sandeepreddy.3647@gmail.com> wrote:
> >
> > > Hi John,
> > > We have applications in windows. So our dev's need to connect to HDFS
> > from
> > > eclipse installed in windows. I'm trying to put data from <local-file>
> to
> > > <hdfs-file> using java code from windows.
> > >
> > > On Thu, Jul 19, 2012 at 5:41 AM, John Hancock <jhancock1975@gmail.com
> > > >wrote:
> > >
> > > > Sandeep,
> > > >
> > > > I don't understand your situation completely, but why not just use
> > > > bin/hadoop dfs -copyFromLocal <local-file-name> <hdfs-file-name> ?
> > > >
> > > > -John
> > > >
> > > > On Wed, Jul 18, 2012 at 11:33 AM, Sandeep Reddy P <
> > > > sandeepreddy.3647@gmail.com> wrote:
> > > >
> > > > > Hi,
> > > > > I'm trying to load data into hdfs from local linux file system
> using
> > > java
> > > > > code from a windows machine.But i'm getting the error
> > > > >
> > > > > java.lang.IllegalArgumentException: Wrong FS:
> > > > > hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected: file:///
> > > > > at
> org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> > > > > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
> > > > > at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
> > > > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
> > > > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
> > > > > at
> > > > >
> > org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
> > > > > at Hdfs.main(Hdfs.java:18)
> > > > > File not found
> > > > >
> > > > > Can any one please help me with the issue.
> > > > >
> > > > > --
> > > > > Thanks,
> > > > > sandeep
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > sandeep
> > >
> >
> >
> >
> > --
> >
> >
> > ∞
> > Shashwat Shriparv
> >
>
>
>
> --
> Thanks,
> sandeep
>
Re: Error:Hdfs Client for hadoop using native java api
Posted by Sandeep Reddy P <sa...@gmail.com>.
Hi Shashwat,
Here is the snippet of code which is throwing error
Path phdfs = new Path(
"hdfs://hadoop1.devqa.local:8020/user/hdfs/java/");
java.lang.IllegalArgumentException: Wrong FS:
hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
at
org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
at
org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
at
org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
at Hdfs.main(Hdfs.java:18)
On Thu, Jul 19, 2012 at 12:02 PM, shashwat shriparv <
dwivedishashwat@gmail.com> wrote:
> Can you provide the code(some part of it if you don't want to throw the
> full code here) and let us know which part of your code is throwing this
> error.
>
> Regards
>
> ∞
> Shashwat Shriparv
>
>
>
> On Thu, Jul 19, 2012 at 6:46 PM, Sandeep Reddy P <
> sandeepreddy.3647@gmail.com> wrote:
>
> > Hi John,
> > We have applications in windows. So our dev's need to connect to HDFS
> from
> > eclipse installed in windows. I'm trying to put data from <local-file> to
> > <hdfs-file> using java code from windows.
> >
> > On Thu, Jul 19, 2012 at 5:41 AM, John Hancock <jhancock1975@gmail.com
> > >wrote:
> >
> > > Sandeep,
> > >
> > > I don't understand your situation completely, but why not just use
> > > bin/hadoop dfs -copyFromLocal <local-file-name> <hdfs-file-name> ?
> > >
> > > -John
> > >
> > > On Wed, Jul 18, 2012 at 11:33 AM, Sandeep Reddy P <
> > > sandeepreddy.3647@gmail.com> wrote:
> > >
> > > > Hi,
> > > > I'm trying to load data into hdfs from local linux file system using
> > java
> > > > code from a windows machine.But i'm getting the error
> > > >
> > > > java.lang.IllegalArgumentException: Wrong FS:
> > > > hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected: file:///
> > > > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> > > > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
> > > > at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
> > > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
> > > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
> > > > at
> > > >
> org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
> > > > at Hdfs.main(Hdfs.java:18)
> > > > File not found
> > > >
> > > > Can any one please help me with the issue.
> > > >
> > > > --
> > > > Thanks,
> > > > sandeep
> > > >
> > >
> >
> >
> >
> > --
> > Thanks,
> > sandeep
> >
>
>
>
> --
>
>
> ∞
> Shashwat Shriparv
>
--
Thanks,
sandeep
Re: Error:Hdfs Client for hadoop using native java api
Posted by shashwat shriparv <dw...@gmail.com>.
Can you provide the code(some part of it if you don't want to throw the
full code here) and let us know which part of your code is throwing this
error.
Regards
∞
Shashwat Shriparv
On Thu, Jul 19, 2012 at 6:46 PM, Sandeep Reddy P <
sandeepreddy.3647@gmail.com> wrote:
> Hi John,
> We have applications in windows. So our dev's need to connect to HDFS from
> eclipse installed in windows. I'm trying to put data from <local-file> to
> <hdfs-file> using java code from windows.
>
> On Thu, Jul 19, 2012 at 5:41 AM, John Hancock <jhancock1975@gmail.com
> >wrote:
>
> > Sandeep,
> >
> > I don't understand your situation completely, but why not just use
> > bin/hadoop dfs -copyFromLocal <local-file-name> <hdfs-file-name> ?
> >
> > -John
> >
> > On Wed, Jul 18, 2012 at 11:33 AM, Sandeep Reddy P <
> > sandeepreddy.3647@gmail.com> wrote:
> >
> > > Hi,
> > > I'm trying to load data into hdfs from local linux file system using
> java
> > > code from a windows machine.But i'm getting the error
> > >
> > > java.lang.IllegalArgumentException: Wrong FS:
> > > hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected: file:///
> > > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
> > > at
> > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
> > > at
> > >
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
> > > at
> > >
> > >
> >
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> > > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
> > > at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
> > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
> > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
> > > at
> > >
> > >
> >
> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
> > > at
> > > org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
> > > at Hdfs.main(Hdfs.java:18)
> > > File not found
> > >
> > > Can any one please help me with the issue.
> > >
> > > --
> > > Thanks,
> > > sandeep
> > >
> >
>
>
>
> --
> Thanks,
> sandeep
>
--
∞
Shashwat Shriparv
Re: Error:Hdfs Client for hadoop using native java api
Posted by Sandeep Reddy P <sa...@gmail.com>.
Hi John,
We have applications in windows. So our dev's need to connect to HDFS from
eclipse installed in windows. I'm trying to put data from <local-file> to
<hdfs-file> using java code from windows.
On Thu, Jul 19, 2012 at 5:41 AM, John Hancock <jh...@gmail.com>wrote:
> Sandeep,
>
> I don't understand your situation completely, but why not just use
> bin/hadoop dfs -copyFromLocal <local-file-name> <hdfs-file-name> ?
>
> -John
>
> On Wed, Jul 18, 2012 at 11:33 AM, Sandeep Reddy P <
> sandeepreddy.3647@gmail.com> wrote:
>
> > Hi,
> > I'm trying to load data into hdfs from local linux file system using java
> > code from a windows machine.But i'm getting the error
> >
> > java.lang.IllegalArgumentException: Wrong FS:
> > hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected: file:///
> > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
> > at
> >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
> > at
> >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
> > at
> >
> >
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
> > at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
> > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
> > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
> > at
> >
> >
> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
> > at
> > org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
> > at Hdfs.main(Hdfs.java:18)
> > File not found
> >
> > Can any one please help me with the issue.
> >
> > --
> > Thanks,
> > sandeep
> >
>
--
Thanks,
sandeep
Re: Error:Hdfs Client for hadoop using native java api
Posted by John Hancock <jh...@gmail.com>.
Sandeep,
I don't understand your situation completely, but why not just use
bin/hadoop dfs -copyFromLocal <local-file-name> <hdfs-file-name> ?
-John
On Wed, Jul 18, 2012 at 11:33 AM, Sandeep Reddy P <
sandeepreddy.3647@gmail.com> wrote:
> Hi,
> I'm trying to load data into hdfs from local linux file system using java
> code from a windows machine.But i'm getting the error
>
> java.lang.IllegalArgumentException: Wrong FS:
> hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected: file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
> at
>
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
> at
>
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
> at
>
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
> at
>
> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
> at
> org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
> at Hdfs.main(Hdfs.java:18)
> File not found
>
> Can any one please help me with the issue.
>
> --
> Thanks,
> sandeep
>