You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "John Xing (JIRA)" <ji...@apache.org> on 2006/02/04 19:56:03 UTC
[jira] Created: (HADOOP-4) tool to mount dfs on linux
tool to mount dfs on linux
--------------------------
Key: HADOOP-4
URL: http://issues.apache.org/jira/browse/HADOOP-4
Project: Hadoop
Type: Improvement
Environment: linux only
Reporter: John Xing
tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
RE: [jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by Yoram Arnon <ya...@yahoo-inc.com>.
Let's wait for hadoop-496 to be complete (and commited) before removing
hadoop-4.
> -----Original Message-----
> From: Sami Siren (JIRA) [mailto:jira@apache.org]
> Sent: Friday, October 27, 2006 11:44 PM
> To: hadoop-dev@lucene.apache.org
> Subject: [jira] Commented: (HADOOP-4) tool to mount dfs on linux
>
> [
> http://issues.apache.org/jira/browse/HADOOP-4?page=comments#ac
tion_12445345 ]
>
> Sami Siren commented on HADOOP-4:
> ---------------------------------
>
> Has anyone made any comparison (performance or other) of
> these two different approaches. It lloks like the webdav
> version (were dfs client is in servlet) introduces one more
> network hop compared to fuse version (where the dfs client is
> in machine using the dfs).
>
> One way or the other I could easily satisfy my mounting needs
> with any one of these solutions - so hopefully one will be
> integrated (so it gets maintenance attention).
>
>
> > tool to mount dfs on linux
> > --------------------------
> >
> > Key: HADOOP-4
> > URL: http://issues.apache.org/jira/browse/HADOOP-4
> > Project: Hadoop
> > Issue Type: Improvement
> > Components: fs
> > Affects Versions: 0.5.0
> > Environment: linux only
> > Reporter: John Xing
> > Attachments:
> fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz,
> fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz,
> fuse-hadoop-0.1.1.tar.gz
> >
> >
> > tool to mount dfs on linux
>
> --
> This message is automatically generated by JIRA.
> -
> If you think it was sent incorrectly contact one of the
> administrators:
> http://issues.apache.org/jira/secure/Administrators.jspa
> -
> For more information on JIRA, see:
> http://www.atlassian.com/software/jira
>
>
>
>
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-4?page=comments#action_12445345 ]
Sami Siren commented on HADOOP-4:
---------------------------------
Has anyone made any comparison (performance or other) of these two different approaches. It lloks like the webdav version (were dfs client is in servlet) introduces one more network hop compared to fuse version (where the dfs client is in machine using the dfs).
One way or the other I could easily satisfy my mounting needs with any one of these solutions - so hopefully one will be integrated (so it gets maintenance attention).
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: http://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by "Anurag Sharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anurag Sharma updated HADOOP-4:
-------------------------------
Attachment: (was: fuse-j-hadoopfs-0.1.zip)
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Anurag Sharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12551102 ]
Anurag Sharma commented on HADOOP-4:
------------------------------------
hi,
We re-submitted the fuse-j-hadoopfs package with the following changes (as suggested above):
* we are hosting a patched FUSE-J on a separate server.
* The fuse-j-hadoopfs build downloads this patched version at compile time.
We restructured the fuse-j-hadoopfs build to be a contrib, and have tested it with the Hadoop source-tree build.
The fuse-j-hadoopfs build is a no-op when a standard "compile" target is specified.
To actually build fuse-j-hadoopfs, the user has to specify the following command line: "ant compile -Dbuild-fuse-j-hadoopfs=1".
We still have the following todo's remaining:
* Pick up some environment variables dynamically, so the user doesn't have to set them in our build.properties file (these do not affect the no-op build).
* Change the 'hadoopfs_fuse_mount.sh' script to use the 'hadoop/bin' scripts, so it can automatically pick up hadoop-specific conf, jar and class files.
The above tarball (fuse-j-hadoopfs-03.tar.gz) consists of a directory that can be placed inside "hadoop/src/contrib", please let us know if we should submit this as a patch-file instead, or if we need to make more changes...
-thanks
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by Sami Siren <ss...@gmail.com>.
Yoram Arnon wrote:
> Does it work on 64bit linux?
>
>
Haven't tested it on 64bit linux but cannot see why not.
--
Sami Siren
> -----Original Message-----
> From: Nguyen Quoc Mai (JIRA) [mailto:jira@apache.org]
> Sent: Thursday, August 17, 2006 10:36 AM
> To: hadoop-dev@lucene.apache.org
> Subject: [jira] Updated: (HADOOP-4) tool to mount dfs on linux
>
> [ http://issues.apache.org/jira/browse/HADOOP-4?page=all ]
>
> Nguyen Quoc Mai updated HADOOP-4:
> ---------------------------------
>
> Attachment: fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz
>
> I changed the code for fuse-hadoop. This is a working version for mounting
> DFS to linux file system. This version works fine with FUSE-J.2.4 and
> HADOOP.0.5.0
>
> Enjoy
>
>
>> tool to mount dfs on linux
>> --------------------------
>>
>> Key: HADOOP-4
>> URL: http://issues.apache.org/jira/browse/HADOOP-4
>> Project: Hadoop
>> Issue Type: Improvement
>> Components: fs
>> Environment: linux only
>> Reporter: John Xing
>> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz,
>>
> fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz
>
>> tool to mount dfs on linux
>>
>
>
RE: [jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by Yoram Arnon <ya...@yahoo-inc.com>.
Does it work on 64bit linux?
-----Original Message-----
From: Nguyen Quoc Mai (JIRA) [mailto:jira@apache.org]
Sent: Thursday, August 17, 2006 10:36 AM
To: hadoop-dev@lucene.apache.org
Subject: [jira] Updated: (HADOOP-4) tool to mount dfs on linux
[ http://issues.apache.org/jira/browse/HADOOP-4?page=all ]
Nguyen Quoc Mai updated HADOOP-4:
---------------------------------
Attachment: fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz
I changed the code for fuse-hadoop. This is a working version for mounting
DFS to linux file system. This version works fine with FUSE-J.2.4 and
HADOOP.0.5.0
Enjoy
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: http://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Environment: linux only
> Reporter: John Xing
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz,
fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by "Nguyen Quoc Mai (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-4?page=all ]
Nguyen Quoc Mai updated HADOOP-4:
---------------------------------
Attachment: fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz
I changed the code for fuse-hadoop. This is a working version for mounting DFS to linux file system. This version works fine with FUSE-J.2.4 and HADOOP.0.5.0
Enjoy
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: http://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Environment: linux only
> Reporter: John Xing
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561517#action_12561517 ]
Pete Wyckoff commented on HADOOP-4:
-----------------------------------
The hadoop API is not POSIX-compliant. Wouldn't it be better to protect people from what is catastrophic data loss?
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-dfs.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Anurag Sharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12548789 ]
Anurag Sharma commented on HADOOP-4:
------------------------------------
hi Doug,
Thanks for pointing out this issue. I will remove the FUSE-J patch and try one of the other routes you suggested (to have a patched FUSE-J available), and will come back with a resolution on this very soon.
-anurag
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-0.1.zip, fuse-j-patch.zip
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-4?page=comments#action_12428802 ]
Doug Cutting commented on HADOOP-4:
-----------------------------------
I'd like to be able to commit this to the contrib tree, but it could use a bit more polish. Unfortunately we cannot commit the fuse-j jar file, since it is released under the LGPL. What would be great is a build.xml that downloaded and built fuse-j.
The README could also include more info on installing fuse.
On Ubuntu I was able to do this with:
sudo apt-get install fuse-utils libfuse2 libfuse-devel
If you've never built things on your Ubuntu box then you might also need:
sudo apt-get install make gcc libc6-dev
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: http://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-4?page=all ]
Doug Cutting updated HADOOP-4:
------------------------------
Status: Open (was: Patch Available)
I'm cancelling this patch to remove it from the queue of patches that we intend to immenently apply to the current trunk. This is still useful stuff, and, depending on what happens with webdav, we may still decide to integrate it, so the issue should remain open for now.
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: http://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by "John Xing (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-4?page=all ]
John Xing updated HADOOP-4:
---------------------------
Attachment: fuse-hadoop-0.1.1.tar.gz
It works with new hadoop project now (tarball attached).
The intended commit location is http://svn.apache.org/repos/asf/lucene/hadoop/trunk/contrib/fuse
Please vote on this issue.
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: http://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Type: Improvement
> Environment: linux only
> Reporter: John Xing
> Attachments: fuse-hadoop-0.1.1.tar.gz
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by "Nguyen Quoc Mai (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-4?page=all ]
Nguyen Quoc Mai updated HADOOP-4:
---------------------------------
Status: Patch Available (was: Open)
Affects Version/s: 0.5.0
I changed the code for fuse-hadoop. This is a working version for mounting DFS to linux file system. This version works fine with HADOOP.0.5.0
Enjoy
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: http://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12548444 ]
Doug Cutting commented on HADOOP-4:
-----------------------------------
> I think I commented on another JIRA that I have implemented a dfs fuse module for the straight C fuse module.
Yes, you noted that in HADOOP-496. Can you post a patch that implements this?
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-0.1.zip, fuse-j-patch.zip
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-4?page=all ]
Doug Cutting updated HADOOP-4:
------------------------------
Component: fs
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: http://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Type: Improvement
> Components: fs
> Environment: linux only
> Reporter: John Xing
> Attachments: fuse-hadoop-0.1.1.tar.gz
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12563222#action_12563222 ]
Pete Wyckoff commented on HADOOP-4:
-----------------------------------
Sami - can you attach the Makefile?
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop Core
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-dfs.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12548807 ]
Doug Cutting commented on HADOOP-4:
-----------------------------------
> I went through the license for Fuse-J and it is distributed under LGPL,
Unfortunately, the ASF cannot host things published under LGPL either. Sorry!
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-0.1.zip, fuse-j-patch.zip
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pete Wyckoff updated HADOOP-4:
------------------------------
Attachment: fuse_dfs.c
Here's the source. I will attach the Makefiles and full tar tomorrow or Wed morning.
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-0.1.zip, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561516#action_12561516 ]
Doug Cutting commented on HADOOP-4:
-----------------------------------
> I'm just wondering why the programmatic API never used trash in the first place??
Because most other programmatic APIs don't. Unix 'rm' does not use the trash, nor does the unlink system call, nor does DOS command line, etc. Trash is usually only a feature of a user-interface, like a GUI and command shells designed for interactive use.
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-dfs.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pete Wyckoff updated HADOOP-4:
------------------------------
Attachment: fuse_dfs.c
newer version which supports mkdir, rmdir, mv, and rm. NOTE - this is still a work in progress. Any help appreciated :)
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop Core
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-dfs.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pete Wyckoff updated HADOOP-4:
------------------------------
Attachment: fuse-dfs.tar.gz
Here's the same code (w/o the closelog after the return in main :)) and the Makefile.am - sorry I haven't been able to get to this and making a nice autoconf. But, it is easy to build:
0. install hadoop 14.x
1. build and install latest fuse
2. compile fuse_dfs.c with 2 includes - one for fuse and one for hadoop's hdfs.h and their libraries as well.
3. run it ./fuse_dfs -debug -server <hadoopnn> -port <hadoopnnport> /mnt/hdfs -d -o allow_other
Obviously, hadoop needs to be in your class path and your ld library path needs the fuse and hdfs .sos
When ready for production remove the -debug (which is an option to fuse-dfs for deubgging) and the -d which is the fuse debugging option.
On my return from vacation, I will make better docs and autoconf.
Note again, this is read only, but it is really easy to implement writes.
-- pete
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-dfs.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12548436 ]
Pete Wyckoff commented on HADOOP-4:
-----------------------------------
I think I commented on another JIRA that I have implemented a dfs fuse module for the straight C fuse module.
thus far I have only implemented read-only and everything works fine. It's been up for about a month with low use, but no problems.
pete
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-0.1.zip, fuse-j-patch.zip
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-4) tool to mount dfs on linux
Posted by "Anurag Sharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12551102 ]
as106 edited comment on HADOOP-4 at 12/12/07 11:58 AM:
---------------------------------------------------------------
hi,
We re-submitted the fuse-j-hadoopfs package with the following changes (as suggested above):
-we are hosting a patched FUSE-J on a separate server.
-the fuse-j-hadoopfs build downloads this patched version at compile time.
We restructured the fuse-j-hadoopfs build to be a contrib, and have tested it with the Hadoop source-tree build.
The fuse-j-hadoopfs build is a no-op when a standard "compile" target is specified. To actually build fuse-j-hadoopfs, the user has to specify the following command line: "ant compile -Dbuild-fuse-j-hadoopfs=1".
We still have the following todo's remaining:
-Pick up some environment variables dynamically, so the user doesn't have to set them in our build.properties file (these do not affect the no-op build).
-Change the 'hadoopfs_fuse_mount.sh' script to use the 'hadoop/bin' scripts, so it can automatically pick up hadoop-specific conf, jar and class files.
The above tarball (fuse-j-hadoopfs-03.tar.gz) consists of a directory that can be placed inside "hadoop/src/contrib", please let us know if we should submit this as a patch-file instead, or if we need to make more changes...
-thanks
was (Author: as106):
hi,
We re-submitted the fuse-j-hadoopfs package with the following changes (as suggested above):
- we are hosting a patched FUSE-J on a separate server.
- The fuse-j-hadoopfs build downloads this patched version at compile time.
We restructured the fuse-j-hadoopfs build to be a contrib, and have tested it with the Hadoop source-tree build.
The fuse-j-hadoopfs build is a no-op when a standard "compile" target is specified. To actually build fuse-j-hadoopfs, the user has to specify the following command line: "ant compile -Dbuild-fuse-j-hadoopfs=1".
We still have the following todo's remaining:
- Pick up some environment variables dynamically, so the user doesn't have to set them in our build.properties file (these do not affect the no-op build).
- Change the 'hadoopfs_fuse_mount.sh' script to use the 'hadoop/bin' scripts, so it can automatically pick up hadoop-specific conf, jar and class files.
The above tarball (fuse-j-hadoopfs-03.tar.gz) consists of a directory that can be placed inside "hadoop/src/contrib", please let us know if we should submit this as a patch-file instead, or if we need to make more changes...
-thanks
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-4) tool to mount dfs on linux
Posted by "Anurag Sharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12551102 ]
as106 edited comment on HADOOP-4 at 12/12/07 11:56 AM:
---------------------------------------------------------------
hi,
We re-submitted the fuse-j-hadoopfs package with the following changes (as suggested above):
- we are hosting a patched FUSE-J on a separate server.
- The fuse-j-hadoopfs build downloads this patched version at compile time.
We restructured the fuse-j-hadoopfs build to be a contrib, and have tested it with the Hadoop source-tree build.
The fuse-j-hadoopfs build is a no-op when a standard "compile" target is specified. To actually build fuse-j-hadoopfs, the user has to specify the following command line: "ant compile -Dbuild-fuse-j-hadoopfs=1".
We still have the following todo's remaining:
- Pick up some environment variables dynamically, so the user doesn't have to set them in our build.properties file (these do not affect the no-op build).
- Change the 'hadoopfs_fuse_mount.sh' script to use the 'hadoop/bin' scripts, so it can automatically pick up hadoop-specific conf, jar and class files.
The above tarball (fuse-j-hadoopfs-03.tar.gz) consists of a directory that can be placed inside "hadoop/src/contrib", please let us know if we should submit this as a patch-file instead, or if we need to make more changes...
-thanks
was (Author: as106):
hi,
We re-submitted the fuse-j-hadoopfs package with the following changes (as suggested above):
- we are hosting a patched FUSE-J on a separate server.
- The fuse-j-hadoopfs build downloads this patched version at compile time.
We restructured the fuse-j-hadoopfs build to be a contrib, and have tested it with the Hadoop source-tree build.
The fuse-j-hadoopfs build is a no-op when a standard "compile" target is specified.
To actually build fuse-j-hadoopfs, the user has to specify the following command line: "ant compile -Dbuild-fuse-j-hadoopfs=1".
We still have the following todo's remaining:
- Pick up some environment variables dynamically, so the user doesn't have to set them in our build.properties file (these do not affect the no-op build).
- Change the 'hadoopfs_fuse_mount.sh' script to use the 'hadoop/bin' scripts, so it can automatically pick up hadoop-specific conf, jar and class files.
The above tarball (fuse-j-hadoopfs-03.tar.gz) consists of a directory that can be placed inside "hadoop/src/contrib", please let us know if we should submit this as a patch-file instead, or if we need to make more changes...
-thanks
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by "Anurag Sharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anurag Sharma updated HADOOP-4:
-------------------------------
Attachment: (was: fuse-j-patch.zip)
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-0.1.zip
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by "Anurag Sharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anurag Sharma updated HADOOP-4:
-------------------------------
Attachment: fuse-j-hadoopfs-03.tar.gz
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Anurag Sharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12548852 ]
Anurag Sharma commented on HADOOP-4:
------------------------------------
hi Doug. ok :- ), we will follow one of the alternate options you suggested of hosting either the patch or the jar file ourselves, and fixing the fuse-j-hadoop package build to work with this. Will re-submit our changes soon.
-thanks,
-anurag
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-0.1.zip, fuse-j-patch.zip
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12563223#action_12563223 ]
Pete Wyckoff commented on HADOOP-4:
-----------------------------------
should have mentioned the problem with mine is that it is generated with some custom auto make macros that we use here.
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop Core
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-dfs.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12548409 ]
Doug Cutting commented on HADOOP-4:
-----------------------------------
It is unfortunate that this requires patches to the fuse-j sources. The fuse-j project does not appear to be active, so I don't see much point in trying to submit these as a patch there.
We cannot host GPL'd code or even patches to GPL'd code at Apache. We can however have optional commands in our build scripts that download GPL'd code. So this could be structured as a contrib module whose build.xml, when a non-default build option is specified, downloads, patches and compiles fuse-j. But since the patch to fuse-j itself cannot be hosted at Apache, it might be simpler to just create a jar file for the patched version, host that somewhere, and bypass the patch and compile steps. It would be great to get this into a form that we can commit, so that it stays synchronized with the rest of Hadoop.
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-0.1.zip, fuse-j-patch.zip
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561469#action_12561469 ]
Pete Wyckoff commented on HADOOP-4:
-----------------------------------
I implemented mkdir, mv, rm and rmdir. But, since the programmatic API doesn't use the Trash feature, this is a pretty big problem.
2 solutions:
1. have fuse dfs rename things into /Trash
2. make the programmatic API use the Trash. Which is more general.
I'm just wondering why the programmatic API never used trash in the first place??
-- pete
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-dfs.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-4) tool to mount dfs on linux
Posted by "Anurag Sharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12551102 ]
as106 edited comment on HADOOP-4 at 12/12/07 11:55 AM:
---------------------------------------------------------------
hi,
We re-submitted the fuse-j-hadoopfs package with the following changes (as suggested above):
- we are hosting a patched FUSE-J on a separate server.
- The fuse-j-hadoopfs build downloads this patched version at compile time.
We restructured the fuse-j-hadoopfs build to be a contrib, and have tested it with the Hadoop source-tree build.
The fuse-j-hadoopfs build is a no-op when a standard "compile" target is specified.
To actually build fuse-j-hadoopfs, the user has to specify the following command line: "ant compile -Dbuild-fuse-j-hadoopfs=1".
We still have the following todo's remaining:
- Pick up some environment variables dynamically, so the user doesn't have to set them in our build.properties file (these do not affect the no-op build).
- Change the 'hadoopfs_fuse_mount.sh' script to use the 'hadoop/bin' scripts, so it can automatically pick up hadoop-specific conf, jar and class files.
The above tarball (fuse-j-hadoopfs-03.tar.gz) consists of a directory that can be placed inside "hadoop/src/contrib", please let us know if we should submit this as a patch-file instead, or if we need to make more changes...
-thanks
was (Author: as106):
hi,
We re-submitted the fuse-j-hadoopfs package with the following changes (as suggested above):
* we are hosting a patched FUSE-J on a separate server.
* The fuse-j-hadoopfs build downloads this patched version at compile time.
We restructured the fuse-j-hadoopfs build to be a contrib, and have tested it with the Hadoop source-tree build.
The fuse-j-hadoopfs build is a no-op when a standard "compile" target is specified.
To actually build fuse-j-hadoopfs, the user has to specify the following command line: "ant compile -Dbuild-fuse-j-hadoopfs=1".
We still have the following todo's remaining:
* Pick up some environment variables dynamically, so the user doesn't have to set them in our build.properties file (these do not affect the no-op build).
* Change the 'hadoopfs_fuse_mount.sh' script to use the 'hadoop/bin' scripts, so it can automatically pick up hadoop-specific conf, jar and class files.
The above tarball (fuse-j-hadoopfs-03.tar.gz) consists of a directory that can be placed inside "hadoop/src/contrib", please let us know if we should submit this as a patch-file instead, or if we need to make more changes...
-thanks
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12563182#action_12563182 ]
Sami Siren commented on HADOOP-4:
---------------------------------
I did some testing with the c version of this tool (and managed to write a working makefile :). For some reason it only worked for me when running it with fuse debug on, anyone else seen this? Other than that it worked great.
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop Core
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-dfs.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Anurag Sharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12548806 ]
Anurag Sharma commented on HADOOP-4:
------------------------------------
Hi Doug,
I went through the license for Fuse-J and it is distributed under LGPL, do you think that would allow the Fuse-J patches to be hosted on Apache?
(In the latter case we would still modify the submission above to be a contrib module that downloads Fuse-J, applies our patch, and builds it, except we won't have to find a place to host the patch).
-thanks
-anurag
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-0.1.zip, fuse-j-patch.zip
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12548452 ]
Pete Wyckoff commented on HADOOP-4:
-----------------------------------
Yes but need a couple of days to get our build guy to make the Makefile.am less facebook-centric.
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-0.1.zip, fuse-j-patch.zip
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by "Anurag Sharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anurag Sharma updated HADOOP-4:
-------------------------------
Attachment: fuse-j-patch.zip
fuse-j-hadoopfs-0.1.zip
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-0.1.zip, fuse-j-patch.zip
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4) tool to mount dfs on linux
Posted by "Nguyen Quoc Mai (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-4?page=all ]
Nguyen Quoc Mai updated HADOOP-4:
---------------------------------
Attachment: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz
I changed the code for fuse-hadoop. This is a working version for mounting DFS to linux file system. This version works fine with FUSE-J.2.2.3 and HADOOP.0.5.0
Enjoy
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: http://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Environment: linux only
> Reporter: John Xing
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-4?page=comments#action_12445323 ]
Doug Cutting commented on HADOOP-4:
-----------------------------------
There are already tools to mount WebDAV on Windows, MacOS and Linux. So HADOOP-496 will provide a more universal solution for this issue. I'd like to resolve this as a duplicate of that issue. Does anyone object?
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: http://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562214#action_12562214 ]
Sami Siren commented on HADOOP-4:
---------------------------------
Since a) the latest patches from Pete are not depending on fuse-j and b) fuse is now part of linux kernel (2.6.14 and later) It would be really nice to get this into hadoop proper.
Before that it would be nice to try this out so i am begging for some more information on how to compile this :)
I downloaded fuse-dfs.tgz and there were two files fuse_dfs.c and Makefile.am , with what commands do I generate the makefile, tried many starting with auto* but none were succesfull.
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop Core
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-dfs.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Anurag Sharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12547964 ]
Anurag Sharma commented on HADOOP-4:
------------------------------------
Hello,
We posted this on HADOOP-496 and were pointed to this jira entry as a better place to post this patch. Pasting our original submission message below...
--------------------------------------
Hi,
We revived the old fuse-hadoop project (a FUSE-J based plugin that lets you mount Hadoop-FS). We have tried this on a small cluster (10 nodes) and basic functionality works (mount, ls, cat,cp, mkdir, rm, mv, ...).
The main changes include some bug fixes to FUSE-J and changing the previous fuse-hadoop implementation to enforce write-once. We found the FUSE framework to be straightforward and simple.
We have seen several mentions of using FUSE with Hadoop, so if there is a better place to post these files, please let me know.
Attachments to follow...
-thanks
--------------------------------------
Attachments include the following:
* fuse-j-hadoop package
* fuse-j patch.
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-0.1.zip, fuse-j-patch.zip
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-4) tool to mount dfs on linux
Posted by "Anurag Sharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12551102 ]
as106 edited comment on HADOOP-4 at 12/12/07 11:58 AM:
---------------------------------------------------------------
hi,
We re-submitted the fuse-j-hadoopfs package with the following changes (as suggested above):
- we are hosting a patched FUSE-J on a separate server.
- the fuse-j-hadoopfs build downloads this patched version at compile time.
We restructured the fuse-j-hadoopfs build to be a contrib, and have tested it with the Hadoop source-tree build.
The fuse-j-hadoopfs build is a no-op when a standard "compile" target is specified. To actually build fuse-j-hadoopfs, the user has to specify the following command line: "ant compile -Dbuild-fuse-j-hadoopfs=1".
We still have the following todo's remaining:
- Pick up some environment variables dynamically, so the user doesn't have to set them in our build.properties file (these do not affect the no-op build).
- Change the 'hadoopfs_fuse_mount.sh' script to use the 'hadoop/bin' scripts, so it can automatically pick up hadoop-specific conf, jar and class files.
The above tarball (fuse-j-hadoopfs-03.tar.gz) consists of a directory that can be placed inside "hadoop/src/contrib", please let us know if we should submit this as a patch-file instead, or if we need to make more changes...
-thanks
was (Author: as106):
hi,
We re-submitted the fuse-j-hadoopfs package with the following changes (as suggested above):
-we are hosting a patched FUSE-J on a separate server.
-the fuse-j-hadoopfs build downloads this patched version at compile time.
We restructured the fuse-j-hadoopfs build to be a contrib, and have tested it with the Hadoop source-tree build.
The fuse-j-hadoopfs build is a no-op when a standard "compile" target is specified. To actually build fuse-j-hadoopfs, the user has to specify the following command line: "ant compile -Dbuild-fuse-j-hadoopfs=1".
We still have the following todo's remaining:
-Pick up some environment variables dynamically, so the user doesn't have to set them in our build.properties file (these do not affect the no-op build).
-Change the 'hadoopfs_fuse_mount.sh' script to use the 'hadoop/bin' scripts, so it can automatically pick up hadoop-specific conf, jar and class files.
The above tarball (fuse-j-hadoopfs-03.tar.gz) consists of a directory that can be placed inside "hadoop/src/contrib", please let us know if we should submit this as a patch-file instead, or if we need to make more changes...
-thanks
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4) tool to mount dfs on linux
Posted by "Jason (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12563216#action_12563216 ]
Jason commented on HADOOP-4:
----------------------------
I have seen the debug issue myself, I think it is related to fuse more
than the fuse_dfs.c, as i also see that with some of my ssh mounts to
some systems.
> tool to mount dfs on linux
> --------------------------
>
> Key: HADOOP-4
> URL: https://issues.apache.org/jira/browse/HADOOP-4
> Project: Hadoop Core
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.5.0
> Environment: linux only
> Reporter: John Xing
> Assignee: Doug Cutting
> Attachments: fuse-dfs.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c, fuse_dfs.c
>
>
> tool to mount dfs on linux
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.