org.apache.hadoop.fs.DF vs Windows

[Albert Strasheim <fu...@gmail.com>]

Hello all

I'm trying to get Hadoop going on my Windows machine without Cygwin. So far, 
I've sorted out the Java Service Wrapper configurations to run the services 
necessary for DFS to work:

https://issues.apache.org/jira/browse/HADOOP-1525

The next problem seems to be that DataNode is trying to run the df command, 
which is rather system specific.

Java 6 added some new methods to handle this issue, but I'm assuming Hadoop 
wants to stay compatible with older versions of the JDK. In that case, could 
FileSystemUtils from Commons IO be used instead of the df hack?

http://jakarta.apache.org/commons/io/api-release/index.html?org/apache/commons/io/FileSystemUtils.html

If this looks like a viable solution, I could try to prepare a patch.

Cheers,

Albert 

RE: org.apache.hadoop.fs.DF vs Windows

[Andrew Newman <an...@itee.uq.edu.au>]

We did a similar thing recently (setting up JSW for Windows).  We had:
wrapper.java.library.path.1=bin

Where the bin directory (it contains the JSW libraries like
wrapper-windows-x86-32.dll and Cygwin files for df) is under the root (which
contains the configuration and JSW executables).

The files required by df are:
- df.exe
- cygiconv-2.dll
- cygintl-8.dll
- cygwin1.dll

> -----Original Message-----
> From: Albert Strasheim [mailto:fullung@gmail.com]
> Sent: Monday, 25 June 2007 8:49 AM
> To: hadoop-user@lucene.apache.org
> Subject: Re: org.apache.hadoop.fs.DF vs Windows
> 
> Turns out Commons IO too uses df on most platforms and dir /-c on
> Windows.
> As a result, it kan only report the free space.
> 
> It seems the GnuWin32 coreutils includes a df command which doesn't
> depend
> on Cygwin.
> 
> http://gnuwin32.sourceforge.net/packages/coreutils.htm
> 
> I'll see if I can convince JSW to set the environment so that DF can
> find
> this program.
> 
> Cheers,
> 
> Albert
> 
> ----- Original Message -----
> From: "Albert Strasheim" <fu...@gmail.com>
> To: <ha...@lucene.apache.org>
> Sent: Monday, June 25, 2007 12:18 AM
> Subject: org.apache.hadoop.fs.DF vs Windows
> 
> 
> > The next problem seems to be that DataNode is trying to run the df
> > command, which is rather system specific.
> >
> > Java 6 added some new methods to handle this issue, but I'm assuming
> > Hadoop wants to stay compatible with older versions of the JDK. In
> that
> > case, could FileSystemUtils from Commons IO be used instead of the df
> > hack?
> >
> > http://jakarta.apache.org/commons/io/api-
> release/index.html?org/apache/commons/io/FileSystemUtils.html

Re: org.apache.hadoop.fs.DF vs Windows

[Albert Strasheim <fu...@gmail.com>]

Turns out Commons IO too uses df on most platforms and dir /-c on Windows. 
As a result, it kan only report the free space.

It seems the GnuWin32 coreutils includes a df command which doesn't depend 
on Cygwin.

http://gnuwin32.sourceforge.net/packages/coreutils.htm

I'll see if I can convince JSW to set the environment so that DF can find 
this program.

Cheers,

Albert

----- Original Message ----- 
From: "Albert Strasheim" <fu...@gmail.com>
To: <ha...@lucene.apache.org>
Sent: Monday, June 25, 2007 12:18 AM
Subject: org.apache.hadoop.fs.DF vs Windows


> The next problem seems to be that DataNode is trying to run the df 
> command, which is rather system specific.
>
> Java 6 added some new methods to handle this issue, but I'm assuming 
> Hadoop wants to stay compatible with older versions of the JDK. In that 
> case, could FileSystemUtils from Commons IO be used instead of the df 
> hack?
>
> http://jakarta.apache.org/commons/io/api-release/index.html?org/apache/commons/io/FileSystemUtils.html