FUSE writes in 0.18.1

[Brian Karlak <ze...@metaweb.com>]

Hello --

I am testing fuse-dfs on a cluster running Hadoop 0.18.1.

I can get reads to work properly, but when I try write operations, I  
get the following:

% ./fuse_dfs_wrapper.sh --debug --server=hadoop.corp.metaweb.com -- 
port=3000 /mnt/hadoop-fuse -d -o-orw

(other window)
% ls /mnt/hadoop-fuse/user/hadoop
jython-wc-3.sh/  test/
% cp ~/hadoop-default.xml /mnt/hadoop-fuse/user/hadoop
cp: cannot create regular file `/mnt/hadoop-fuse/user/hadoop/hadoop- 
default.xml': Function not implemented

When I look through fuse_dfs.c, I see a comment:

// The remainder are write functionality and therefore not implemented  
right now

Are fuse-dfs writes enabled in 0.18.1?  If not, what version do I need  
to get write support?  Or am I just configuring / calling fuse-dfs  
wrong?  I've tried a variety of other -o options on the command line  
but have not found anything that works.

Thanks in advance,
Brian

Re: FUSE writes in 0.18.1

[Brian Karlak <ze...@metaweb.com>]

On Nov 7, 2008, at 1:07 PM, Pete Wyckoff wrote:

> You know what - writes do not not work at all in 18.1 - sorry my  
> confusion. - just switch to 18.2 fuse-dfs. It can run against 18.1  
> dfs.
>
> Just remember to compile it with -Dlibhdfs.noperms=1

Will do -- many thanks for the pointers!

Brian

Re: FUSE writes in 0.18.1

[Pete Wyckoff <pw...@facebook.com>]

You know what - writes do not not work at all in 18.1 - sorry my confusion. - just switch to 18.2 fuse-dfs. It can run against 18.1 dfs.

Just remember to compile it with -Dlibhdfs.noperms=1


On 11/7/08 12:20 PM, "Brian Karlak" <ze...@metaweb.com> wrote:



On Nov 7, 2008, at 12:09 PM, Pete Wyckoff wrote:

> For 0.18.1, "cp foo bar" where bar does not exist will work. But, if
> bar exists, it will not.copying over an existing file requires
> truncate to work and fuse-dfs 0.18.1 does not support it.

Pete --

Thanks for the help.  I am still having trouble with writes, even to
file paths that don't exist on HDFS.  For instance:

% ls /mnt/hadoop-fuse/user/hadoop
jython-wc-3.sh/  test/
% cp ~/hadoop-default.xml /mnt/hadoop-fuse/user/hadoop/test20081108
cp: cannot create regular file `/mnt/hadoop-fuse/user/hadoop/
test20081108': Function not implemented

test20081108 does not exist, yet I cannot create it.  Nor can I do
touch or other writes.

Since writes should work in 0.18.1, do you have any suggestions as to
where I should start with debugging?  Are there any particular options
I should make sure I'm using?

Thanks,
Brian

Re: FUSE writes in 0.18.1

[Brian Karlak <ze...@metaweb.com>]

On Nov 7, 2008, at 12:09 PM, Pete Wyckoff wrote:

> For 0.18.1, "cp foo bar" where bar does not exist will work. But, if  
> bar exists, it will not.copying over an existing file requires  
> truncate to work and fuse-dfs 0.18.1 does not support it.

Pete --

Thanks for the help.  I am still having trouble with writes, even to  
file paths that don't exist on HDFS.  For instance:

% ls /mnt/hadoop-fuse/user/hadoop
jython-wc-3.sh/  test/
% cp ~/hadoop-default.xml /mnt/hadoop-fuse/user/hadoop/test20081108
cp: cannot create regular file `/mnt/hadoop-fuse/user/hadoop/ 
test20081108': Function not implemented

test20081108 does not exist, yet I cannot create it.  Nor can I do  
touch or other writes.

Since writes should work in 0.18.1, do you have any suggestions as to  
where I should start with debugging?  Are there any particular options  
I should make sure I'm using?

Thanks,
Brian

Re: FUSE writes in 0.18.1

[Pete Wyckoff <pw...@facebook.com>]

Hi Brian,

For 0.18.1, "cp foo bar" where bar does not exist will work. But, if bar exists, it will not.copying over an existing file requires truncate to work and fuse-dfs 0.18.1 does not support it.

Try using the 0.18.2 version where it supports truncate(file, 0).  It supports it, but permissions and such will be those of the user doing the cp and not the ones that were already on the file as in POSIX and Hadoop semantics.

-- pete


On 11/7/08 7:27 AM, "Brian Karlak" <ze...@metaweb.com> wrote:

Hello --

I am testing fuse-dfs on a cluster running Hadoop 0.18.1.

I can get reads to work properly, but when I try write operations, I
get the following:

% ./fuse_dfs_wrapper.sh --debug --server=hadoop.corp.metaweb.com --
port=3000 /mnt/hadoop-fuse -d -o-orw

(other window)
% ls /mnt/hadoop-fuse/user/hadoop
jython-wc-3.sh/  test/
% cp ~/hadoop-default.xml /mnt/hadoop-fuse/user/hadoop
cp: cannot create regular file `/mnt/hadoop-fuse/user/hadoop/hadoop-
default.xml': Function not implemented

When I look through fuse_dfs.c, I see a comment:

// The remainder are write functionality and therefore not implemented
right now

Are fuse-dfs writes enabled in 0.18.1?  If not, what version do I need
to get write support?  Or am I just configuring / calling fuse-dfs
wrong?  I've tried a variety of other -o options on the command line
but have not found anything that works.

Thanks in advance,
Brian