You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "thilee (JIRA)" <ji...@apache.org> on 2012/10/05 19:20:02 UTC
[jira] [Created] (HADOOP-8885) Need to add fs shim to use QFS
thilee created HADOOP-8885:
------------------------------
Summary: Need to add fs shim to use QFS
Key: HADOOP-8885
URL: https://issues.apache.org/jira/browse/HADOOP-8885
Project: Hadoop Common
Issue Type: New Feature
Components: fs
Affects Versions: 2.0.1-alpha, 2.0.0-alpha, 0.23.3, 1.0.3, 1.0.2
Reporter: thilee
Quantcast has released QFS 1.0 (http://quantcast.github.com/qfs), a C++ distributed filesystem based on Kosmos File System(KFS). QFS comes with various feature, performance, and stability improvements over KFS.
A hadoop 'fs' shim needs be added to support QFS through 'qfs://' URIs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work stopped] (HADOOP-8885) Need to add fs shim to use QFS
Posted by "thilee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HADOOP-8885 stopped by thilee.
> Need to add fs shim to use QFS
> ------------------------------
>
> Key: HADOOP-8885
> URL: https://issues.apache.org/jira/browse/HADOOP-8885
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Reporter: thilee
> Assignee: thilee
> Fix For: 1.0.2, 1.0.3, 0.23.3, 2.0.0-alpha, 3.0.0, 2.0.1-alpha
>
> Attachments: HADOOP-8885-branch-1.patch, HADOOP-8885-trunk.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Quantcast has released QFS 1.0 (http://quantcast.github.com/qfs), a C++ distributed filesystem based on Kosmos File System(KFS). QFS comes with various feature, performance, and stability improvements over KFS.
> A hadoop 'fs' shim needs be added to support QFS through 'qfs://' URIs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8885) Need to add fs shim to use QFS
Posted by "thilee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
thilee updated HADOOP-8885:
---------------------------
Status: Open (was: Patch Available)
> Need to add fs shim to use QFS
> ------------------------------
>
> Key: HADOOP-8885
> URL: https://issues.apache.org/jira/browse/HADOOP-8885
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Reporter: thilee
> Assignee: thilee
> Fix For: 3.0.0, 2.0.1-alpha, 2.0.0-alpha, 0.23.3, 1.0.3, 1.0.2
>
> Attachments: HADOOP-8885-branch-1.patch, HADOOP-8885-trunk.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Quantcast has released QFS 1.0 (http://quantcast.github.com/qfs), a C++ distributed filesystem based on Kosmos File System(KFS). QFS comes with various feature, performance, and stability improvements over KFS.
> A hadoop 'fs' shim needs be added to support QFS through 'qfs://' URIs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8885) Need to add fs shim to use QFS
Posted by "thilee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484601#comment-13484601 ]
thilee commented on HADOOP-8885:
--------------------------------
Based on the comments in the mailing list, I have pulled out the Quantcast File System implementation from Apache Hadoop to the QFS project. QFS wil be maintained by Quantcast. Please refer to http://quantcast.github.com/qfs/ and https://github.com/quantcast/qfs/wiki/Migration-Guide.
> Need to add fs shim to use QFS
> ------------------------------
>
> Key: HADOOP-8885
> URL: https://issues.apache.org/jira/browse/HADOOP-8885
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Reporter: thilee
> Assignee: thilee
> Fix For: 1.0.2, 1.0.3, 0.23.3, 2.0.0-alpha, 3.0.0, 2.0.1-alpha
>
> Attachments: HADOOP-8885-branch-1.patch, HADOOP-8885-trunk.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Quantcast has released QFS 1.0 (http://quantcast.github.com/qfs), a C++ distributed filesystem based on Kosmos File System(KFS). QFS comes with various feature, performance, and stability improvements over KFS.
> A hadoop 'fs' shim needs be added to support QFS through 'qfs://' URIs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8885) Need to add fs shim to use QFS
Posted by "thilee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
thilee updated HADOOP-8885:
---------------------------
Attachment: HADOOP-8885-trunk.patch
Patch for trunk is attached as well now.
> Need to add fs shim to use QFS
> ------------------------------
>
> Key: HADOOP-8885
> URL: https://issues.apache.org/jira/browse/HADOOP-8885
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Affects Versions: 1.0.2, 1.0.3, 0.23.3, 2.0.0-alpha, 2.0.1-alpha
> Reporter: thilee
> Assignee: thilee
> Attachments: HADOOP-8885-branch-1.patch, HADOOP-8885-trunk.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Quantcast has released QFS 1.0 (http://quantcast.github.com/qfs), a C++ distributed filesystem based on Kosmos File System(KFS). QFS comes with various feature, performance, and stability improvements over KFS.
> A hadoop 'fs' shim needs be added to support QFS through 'qfs://' URIs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8885) Need to add fs shim to use QFS
Posted by "thilee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
thilee updated HADOOP-8885:
---------------------------
Attachment: HADOOP-8885-branch-1.patch
Attached is the patch for *branch-1*.
In addition to applying the patch, you would need to do two things:
(1) To compile: you need to copy the qfs access JAR (qfs-<version>.jar) to the branch-1 _lib_ directory.
(2) To run: you need to have the directory of libqfs*.so in your {{LD_LIBRARY_PATH}}.
The tarball containing the JAR file and the shared libraries can be downloaded here: (please download the one that matches your platform, untar and look inside _lib_ directory)
[https://github.com/quantcast/qfs/wiki/Binary-Distributions]
The JAR file should eventually be integrated to Apache Hadoop, under _lib_ directory, as qfs-1.0.0.jar.
> Need to add fs shim to use QFS
> ------------------------------
>
> Key: HADOOP-8885
> URL: https://issues.apache.org/jira/browse/HADOOP-8885
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Affects Versions: 1.0.2, 1.0.3, 0.23.3, 2.0.0-alpha, 2.0.1-alpha
> Reporter: thilee
> Attachments: HADOOP-8885-branch-1.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Quantcast has released QFS 1.0 (http://quantcast.github.com/qfs), a C++ distributed filesystem based on Kosmos File System(KFS). QFS comes with various feature, performance, and stability improvements over KFS.
> A hadoop 'fs' shim needs be added to support QFS through 'qfs://' URIs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HADOOP-8885) Need to add fs shim to use QFS
Posted by "thilee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HADOOP-8885 started by thilee.
> Need to add fs shim to use QFS
> ------------------------------
>
> Key: HADOOP-8885
> URL: https://issues.apache.org/jira/browse/HADOOP-8885
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Reporter: thilee
> Assignee: thilee
> Fix For: 1.0.2, 1.0.3, 0.23.3, 2.0.0-alpha, 3.0.0, 2.0.1-alpha
>
> Attachments: HADOOP-8885-branch-1.patch, HADOOP-8885-trunk.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Quantcast has released QFS 1.0 (http://quantcast.github.com/qfs), a C++ distributed filesystem based on Kosmos File System(KFS). QFS comes with various feature, performance, and stability improvements over KFS.
> A hadoop 'fs' shim needs be added to support QFS through 'qfs://' URIs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8885) Need to add fs shim to use QFS
Posted by "thilee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
thilee updated HADOOP-8885:
---------------------------
Fix Version/s: 1.0.2
1.0.3
0.23.3
2.0.0-alpha
2.0.1-alpha
3.0.0
Affects Version/s: (was: 2.0.1-alpha)
(was: 2.0.0-alpha)
(was: 1.0.3)
(was: 1.0.2)
(was: 0.23.3)
Status: Patch Available (was: Open)
> Need to add fs shim to use QFS
> ------------------------------
>
> Key: HADOOP-8885
> URL: https://issues.apache.org/jira/browse/HADOOP-8885
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Reporter: thilee
> Assignee: thilee
> Fix For: 3.0.0, 2.0.1-alpha, 2.0.0-alpha, 0.23.3, 1.0.3, 1.0.2
>
> Attachments: HADOOP-8885-branch-1.patch, HADOOP-8885-trunk.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Quantcast has released QFS 1.0 (http://quantcast.github.com/qfs), a C++ distributed filesystem based on Kosmos File System(KFS). QFS comes with various feature, performance, and stability improvements over KFS.
> A hadoop 'fs' shim needs be added to support QFS through 'qfs://' URIs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-8885) Need to add fs shim to use QFS
Posted by "thilee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
thilee resolved HADOOP-8885.
----------------------------
Resolution: Fixed
Release Note:
Made Hadoop QFS plugin as part of Quantcast File System open-source project.
Users can download the Hadoop QFS jar + libs and dynamically use it with Hadoop.
Based on suggestions in the hadoop-common mailing list, we decided to take the QFS implementation out of Hadoop Common/core.
QFS Hadoop plugin is now maintained at http://quantcast.github.com/qfs/
Instructions for obtaining and using the QFS Hadoop plugin are at
https://github.com/quantcast/qfs/wiki/Migration-Guide
> Need to add fs shim to use QFS
> ------------------------------
>
> Key: HADOOP-8885
> URL: https://issues.apache.org/jira/browse/HADOOP-8885
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Reporter: thilee
> Assignee: thilee
> Fix For: 3.0.0, 2.0.1-alpha, 2.0.0-alpha, 0.23.3, 1.0.3, 1.0.2
>
> Attachments: HADOOP-8885-branch-1.patch, HADOOP-8885-trunk.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Quantcast has released QFS 1.0 (http://quantcast.github.com/qfs), a C++ distributed filesystem based on Kosmos File System(KFS). QFS comes with various feature, performance, and stability improvements over KFS.
> A hadoop 'fs' shim needs be added to support QFS through 'qfs://' URIs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-8885) Need to add fs shim to use QFS
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers reassigned HADOOP-8885:
--------------------------------------
Assignee: thilee
> Need to add fs shim to use QFS
> ------------------------------
>
> Key: HADOOP-8885
> URL: https://issues.apache.org/jira/browse/HADOOP-8885
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Affects Versions: 1.0.2, 1.0.3, 0.23.3, 2.0.0-alpha, 2.0.1-alpha
> Reporter: thilee
> Assignee: thilee
> Attachments: HADOOP-8885-branch-1.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Quantcast has released QFS 1.0 (http://quantcast.github.com/qfs), a C++ distributed filesystem based on Kosmos File System(KFS). QFS comes with various feature, performance, and stability improvements over KFS.
> A hadoop 'fs' shim needs be added to support QFS through 'qfs://' URIs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8885) Need to add fs shim to use QFS
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472563#comment-13472563 ]
Hadoop QA commented on HADOOP-8885:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12548323/HADOOP-8885-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to fail.
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1589//console
This message is automatically generated.
> Need to add fs shim to use QFS
> ------------------------------
>
> Key: HADOOP-8885
> URL: https://issues.apache.org/jira/browse/HADOOP-8885
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Reporter: thilee
> Assignee: thilee
> Fix For: 1.0.2, 1.0.3, 0.23.3, 2.0.0-alpha, 3.0.0, 2.0.1-alpha
>
> Attachments: HADOOP-8885-branch-1.patch, HADOOP-8885-trunk.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Quantcast has released QFS 1.0 (http://quantcast.github.com/qfs), a C++ distributed filesystem based on Kosmos File System(KFS). QFS comes with various feature, performance, and stability improvements over KFS.
> A hadoop 'fs' shim needs be added to support QFS through 'qfs://' URIs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira