You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Jeff Hammerbacher (JIRA)" <ji...@apache.org> on 2010/08/10 18:41:17 UTC
[jira] Created: (HADOOP-6909) Umbrella ticket for reducing the
number of libraries in Common
Umbrella ticket for reducing the number of libraries in Common
--------------------------------------------------------------
Key: HADOOP-6909
URL: https://issues.apache.org/jira/browse/HADOOP-6909
Project: Hadoop Common
Issue Type: Improvement
Reporter: Jeff Hammerbacher
There seems to be some consensus that moving libraries out of Common, with the eventual goal of getting rid of the Common package all together, is a good idea: http://hadoop.markmail.org/thread/jhiactdekhywznac. Hopefully we can collect the incremental tasks needed to achieve that goal here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6909) Umbrella ticket for reducing the
number of libraries in Common
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896941#action_12896941 ]
Arun C Murthy commented on HADOOP-6909:
---------------------------------------
bq. There seems to be some consensus that moving libraries out of Common
I'm sure I'd call it 'consensus' yet, we need a discussion before we reach consensus.
> Umbrella ticket for reducing the number of libraries in Common
> --------------------------------------------------------------
>
> Key: HADOOP-6909
> URL: https://issues.apache.org/jira/browse/HADOOP-6909
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Jeff Hammerbacher
>
> There seems to be some consensus that moving libraries out of Common, with the eventual goal of getting rid of the Common package all together, is a good idea: http://hadoop.markmail.org/thread/jhiactdekhywznac. Hopefully we can collect the incremental tasks needed to achieve that goal here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6909) Umbrella ticket for reducing the
number of libraries in Common
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896949#action_12896949 ]
Jeff Hammerbacher commented on HADOOP-6909:
-------------------------------------------
bq. I'm sure I'd call it 'consensus' yet, we need a discussion before we reach consensus.
Sure, and hopefully that discussion can take place here, to avoid derailing the thread on combining the committer lists.
> Umbrella ticket for reducing the number of libraries in Common
> --------------------------------------------------------------
>
> Key: HADOOP-6909
> URL: https://issues.apache.org/jira/browse/HADOOP-6909
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Jeff Hammerbacher
>
> There seems to be some consensus that moving libraries out of Common, with the eventual goal of getting rid of the Common package all together, is a good idea: http://hadoop.markmail.org/thread/jhiactdekhywznac. Hopefully we can collect the incremental tasks needed to achieve that goal here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6909) Umbrella ticket for reducing the
number of libraries in Common
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896960#action_12896960 ]
Jeff Hammerbacher commented on HADOOP-6909:
-------------------------------------------
Also, I apologize if I jumped the gun a little early, I became irrationally exuberant when I thought about moving to a generic configuration system.
> Umbrella ticket for reducing the number of libraries in Common
> --------------------------------------------------------------
>
> Key: HADOOP-6909
> URL: https://issues.apache.org/jira/browse/HADOOP-6909
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Jeff Hammerbacher
>
> There seems to be some consensus that moving libraries out of Common, with the eventual goal of getting rid of the Common package all together, is a good idea: http://hadoop.markmail.org/thread/jhiactdekhywznac. Hopefully we can collect the incremental tasks needed to achieve that goal here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6909) Umbrella ticket for reducing the
number of libraries in Common
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896940#action_12896940 ]
Arun C Murthy commented on HADOOP-6909:
---------------------------------------
Moving fs to HDFS might be reasonable, but moving metrics to Commons means we will have another dependency, plus it isn't clear metrics is usable in a wider context.
> Umbrella ticket for reducing the number of libraries in Common
> --------------------------------------------------------------
>
> Key: HADOOP-6909
> URL: https://issues.apache.org/jira/browse/HADOOP-6909
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Jeff Hammerbacher
>
> There seems to be some consensus that moving libraries out of Common, with the eventual goal of getting rid of the Common package all together, is a good idea: http://hadoop.markmail.org/thread/jhiactdekhywznac. Hopefully we can collect the incremental tasks needed to achieve that goal here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6909) Umbrella ticket for reducing the
number of libraries in Common
Posted by "Jay Booth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896962#action_12896962 ]
Jay Booth commented on HADOOP-6909:
-----------------------------------
Well, if we're encourage to derail this thread with config-related stuff, what do people think about a variable-substituted properties file instead of XML?
#comment
key=value
#other comment
key2=${key}
> Umbrella ticket for reducing the number of libraries in Common
> --------------------------------------------------------------
>
> Key: HADOOP-6909
> URL: https://issues.apache.org/jira/browse/HADOOP-6909
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Jeff Hammerbacher
>
> There seems to be some consensus that moving libraries out of Common, with the eventual goal of getting rid of the Common package all together, is a good idea: http://hadoop.markmail.org/thread/jhiactdekhywznac. Hopefully we can collect the incremental tasks needed to achieve that goal here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6909) Umbrella ticket for reducing the
number of libraries in Common
Posted by "Ranjit Mathew (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897129#action_12897129 ]
Ranjit Mathew commented on HADOOP-6909:
---------------------------------------
??I think it makes sense to move fs APIs like FileSystem and FileContext into HDFS. The same way as mapreduce APIs like Mapper and Reducer are in mapreduce project.??
I believe HDFS is _a_ file-system supported by Hadoop and it could potentially end up supporting others (e.g Ceph via HADOOP-6253).
So it makes sense to have a common FS API that then has a concrete implementation in HDFS.
> Umbrella ticket for reducing the number of libraries in Common
> --------------------------------------------------------------
>
> Key: HADOOP-6909
> URL: https://issues.apache.org/jira/browse/HADOOP-6909
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Jeff Hammerbacher
>
> There seems to be some consensus that moving libraries out of Common, with the eventual goal of getting rid of the Common package all together, is a good idea: http://hadoop.markmail.org/thread/jhiactdekhywznac. Hopefully we can collect the incremental tasks needed to achieve that goal here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6909) Umbrella ticket for reducing the
number of libraries in Common
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896964#action_12896964 ]
Arun C Murthy commented on HADOOP-6909:
---------------------------------------
Jay, can we please use HADOOP-6910 for discussions on configuration? Thanks.
> Umbrella ticket for reducing the number of libraries in Common
> --------------------------------------------------------------
>
> Key: HADOOP-6909
> URL: https://issues.apache.org/jira/browse/HADOOP-6909
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Jeff Hammerbacher
>
> There seems to be some consensus that moving libraries out of Common, with the eventual goal of getting rid of the Common package all together, is a good idea: http://hadoop.markmail.org/thread/jhiactdekhywznac. Hopefully we can collect the incremental tasks needed to achieve that goal here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6909) Umbrella ticket for reducing the
number of libraries in Common
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896968#action_12896968 ]
Owen O'Malley commented on HADOOP-6909:
---------------------------------------
Moving FileSystem and FileContext to Jakarta is a non-starter. Having it in Common where we are all committers is hard enough.
We've discussed moving FileSystem to HDFS, but that is both problematic (it moves a lot of non-HDFS stuff into HDFS) and breaks the abstraction that HDFS implements the contract required by FileSystem.
I don't think this is a good direction.
> Umbrella ticket for reducing the number of libraries in Common
> --------------------------------------------------------------
>
> Key: HADOOP-6909
> URL: https://issues.apache.org/jira/browse/HADOOP-6909
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Jeff Hammerbacher
>
> There seems to be some consensus that moving libraries out of Common, with the eventual goal of getting rid of the Common package all together, is a good idea: http://hadoop.markmail.org/thread/jhiactdekhywznac. Hopefully we can collect the incremental tasks needed to achieve that goal here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6909) Umbrella ticket for reducing the
number of libraries in Common
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896938#action_12896938 ]
Jeff Hammerbacher commented on HADOOP-6909:
-------------------------------------------
Doug also mentions on that thread that the metrics and fs packages might also be moved to Jakarta Commons.
> Umbrella ticket for reducing the number of libraries in Common
> --------------------------------------------------------------
>
> Key: HADOOP-6909
> URL: https://issues.apache.org/jira/browse/HADOOP-6909
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Jeff Hammerbacher
>
> There seems to be some consensus that moving libraries out of Common, with the eventual goal of getting rid of the Common package all together, is a good idea: http://hadoop.markmail.org/thread/jhiactdekhywznac. Hopefully we can collect the incremental tasks needed to achieve that goal here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6909) Umbrella ticket for reducing the
number of libraries in Common
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896998#action_12896998 ]
Konstantin Shvachko commented on HADOOP-6909:
---------------------------------------------
- Getting rid of common, as Dough proposed, is a good idea, imo.
- I think it makes sense to move fs APIs like FileSystem and FileContext into HDFS. The same way as mapreduce APIs like Mapper and Reducer are in mapreduce project.
- I looked at [commons.vfs|http://commons.apache.org/vfs/apidocs/index.html] packages. There are clearly some similarities with fs.FileSystem, but the differences are substantial. I believe one can implement {{commons.vfs.HDFS}} using {{hadoop.fs.FileSystem}}. But using commons.vfs as a new API for HDFS would be a big, incompatible, hard to justify change, which also lacks intrinsic for HDFS apis like getBlockLocations().
So at this point I don't understand what the proposal of moving fs packages into Jakarta Commons means.
> Umbrella ticket for reducing the number of libraries in Common
> --------------------------------------------------------------
>
> Key: HADOOP-6909
> URL: https://issues.apache.org/jira/browse/HADOOP-6909
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Jeff Hammerbacher
>
> There seems to be some consensus that moving libraries out of Common, with the eventual goal of getting rid of the Common package all together, is a good idea: http://hadoop.markmail.org/thread/jhiactdekhywznac. Hopefully we can collect the incremental tasks needed to achieve that goal here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6909) Umbrella ticket for reducing the
number of libraries in Common
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896952#action_12896952 ]
Arun C Murthy commented on HADOOP-6909:
---------------------------------------
Yep.
> Umbrella ticket for reducing the number of libraries in Common
> --------------------------------------------------------------
>
> Key: HADOOP-6909
> URL: https://issues.apache.org/jira/browse/HADOOP-6909
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Jeff Hammerbacher
>
> There seems to be some consensus that moving libraries out of Common, with the eventual goal of getting rid of the Common package all together, is a good idea: http://hadoop.markmail.org/thread/jhiactdekhywznac. Hopefully we can collect the incremental tasks needed to achieve that goal here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.