You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Feng Jiang (JIRA)" <ji...@apache.org> on 2006/11/08 04:16:50 UTC
[jira] Created: (HADOOP-689) hadoop should provide a common way to
wrap instances with different types into one type
hadoop should provide a common way to wrap instances with different types into one type
---------------------------------------------------------------------------------------
Key: HADOOP-689
URL: http://issues.apache.org/jira/browse/HADOOP-689
Project: Hadoop
Issue Type: Improvement
Components: io
Environment: All environment
Reporter: Feng Jiang
When two sequence files, which have same Key type but different Value types, are mapped out to reduce, multiple Value types is not allowed. In this case, we need a way to wrap instances with different types into one class type to reduce.
In current code, ObjectWritable is a sole choice. but it costs too many space, because the class declaration will be appended into output file as a string for every Key-value pair.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-689) hadoop should provide a common way
to wrap instances with different types into one type
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-689?page=comments#action_12454060 ]
Hadoop QA commented on HADOOP-689:
----------------------------------
+1, http://issues.apache.org/jira/secure/attachment/12345896/HADOOP-689.patch.new applied and successfully tested against trunk revision 480142
> hadoop should provide a common way to wrap instances with different types into one type
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-689
> URL: http://issues.apache.org/jira/browse/HADOOP-689
> Project: Hadoop
> Issue Type: Improvement
> Components: io
> Environment: All environment
> Reporter: Feng Jiang
> Attachments: HADOOP-689.patch, HADOOP-689.patch.new
>
>
> When two sequence files, which have same Key type but different Value types, are mapped out to reduce, multiple Value types is not allowed. In this case, we need a way to wrap instances with different types into one class type to reduce.
> In current code, ObjectWritable is a sole choice. but it costs too many space, because the class declaration will be appended into output file as a string for every Key-value pair.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-689) hadoop should provide a common way
to wrap instances with different types into one type
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-689?page=comments#action_12448829 ]
Doug Cutting commented on HADOOP-689:
-------------------------------------
Overall this looks good. A few issues.
1. A new array of classes should not be allocated for each call to readFields(). Perhaps instead introspection should be used to call a static getTypes() method and somehow cache this per subclass. At the least, the example code should create the array as a static field, and then have the getTypes() implementation return a reference to this.
2. set(), get(), and getTypes() need javadoc.
> hadoop should provide a common way to wrap instances with different types into one type
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-689
> URL: http://issues.apache.org/jira/browse/HADOOP-689
> Project: Hadoop
> Issue Type: Improvement
> Components: io
> Environment: All environment
> Reporter: Feng Jiang
> Attachments: HADOOP-689.patch
>
>
> When two sequence files, which have same Key type but different Value types, are mapped out to reduce, multiple Value types is not allowed. In this case, we need a way to wrap instances with different types into one class type to reduce.
> In current code, ObjectWritable is a sole choice. but it costs too many space, because the class declaration will be appended into output file as a string for every Key-value pair.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-689) hadoop should provide a common way to
wrap instances with different types into one type
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-689?page=all ]
Doug Cutting updated HADOOP-689:
--------------------------------
Status: Patch Available (was: Open)
> hadoop should provide a common way to wrap instances with different types into one type
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-689
> URL: http://issues.apache.org/jira/browse/HADOOP-689
> Project: Hadoop
> Issue Type: Improvement
> Components: io
> Environment: All environment
> Reporter: Feng Jiang
> Attachments: HADOOP-689.patch
>
>
> When two sequence files, which have same Key type but different Value types, are mapped out to reduce, multiple Value types is not allowed. In this case, we need a way to wrap instances with different types into one class type to reduce.
> In current code, ObjectWritable is a sole choice. but it costs too many space, because the class declaration will be appended into output file as a string for every Key-value pair.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-689) hadoop should provide a common way to
wrap instances with different types into one type
Posted by "Feng Jiang (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-689?page=all ]
Feng Jiang updated HADOOP-689:
------------------------------
Attachment: HADOOP-689.patch.new
Update the patch, just refined the javadoc.
> hadoop should provide a common way to wrap instances with different types into one type
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-689
> URL: http://issues.apache.org/jira/browse/HADOOP-689
> Project: Hadoop
> Issue Type: Improvement
> Components: io
> Environment: All environment
> Reporter: Feng Jiang
> Attachments: HADOOP-689.patch, HADOOP-689.patch.new
>
>
> When two sequence files, which have same Key type but different Value types, are mapped out to reduce, multiple Value types is not allowed. In this case, we need a way to wrap instances with different types into one class type to reduce.
> In current code, ObjectWritable is a sole choice. but it costs too many space, because the class declaration will be appended into output file as a string for every Key-value pair.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-689) hadoop should provide a common way to
wrap instances with different types into one type
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-689?page=all ]
Doug Cutting updated HADOOP-689:
--------------------------------
Status: Open (was: Patch Available)
Please submit a new patch containing all the changes. Thanks!
> hadoop should provide a common way to wrap instances with different types into one type
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-689
> URL: http://issues.apache.org/jira/browse/HADOOP-689
> Project: Hadoop
> Issue Type: Improvement
> Components: io
> Environment: All environment
> Reporter: Feng Jiang
> Attachments: HADOOP-689.patch
>
>
> When two sequence files, which have same Key type but different Value types, are mapped out to reduce, multiple Value types is not allowed. In this case, we need a way to wrap instances with different types into one class type to reduce.
> In current code, ObjectWritable is a sole choice. but it costs too many space, because the class declaration will be appended into output file as a string for every Key-value pair.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-689) hadoop should provide a common way
to wrap instances with different types into one type
Posted by "Feng Jiang (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-689?page=comments#action_12449550 ]
Feng Jiang commented on HADOOP-689:
-----------------------------------
how should I change the patch I submited a couple of days ago?
submit a new one? or make a patch for that patch?
Thanks.
> hadoop should provide a common way to wrap instances with different types into one type
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-689
> URL: http://issues.apache.org/jira/browse/HADOOP-689
> Project: Hadoop
> Issue Type: Improvement
> Components: io
> Environment: All environment
> Reporter: Feng Jiang
> Attachments: HADOOP-689.patch
>
>
> When two sequence files, which have same Key type but different Value types, are mapped out to reduce, multiple Value types is not allowed. In this case, we need a way to wrap instances with different types into one class type to reduce.
> In current code, ObjectWritable is a sole choice. but it costs too many space, because the class declaration will be appended into output file as a string for every Key-value pair.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-689) hadoop should provide a common way to
wrap instances with different types into one type
Posted by "Feng Jiang (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-689?page=all ]
Feng Jiang updated HADOOP-689:
------------------------------
Attachment: HADOOP-689.patch
Fix this issue.
> hadoop should provide a common way to wrap instances with different types into one type
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-689
> URL: http://issues.apache.org/jira/browse/HADOOP-689
> Project: Hadoop
> Issue Type: Improvement
> Components: io
> Environment: All environment
> Reporter: Feng Jiang
> Attachments: HADOOP-689.patch
>
>
> When two sequence files, which have same Key type but different Value types, are mapped out to reduce, multiple Value types is not allowed. In this case, we need a way to wrap instances with different types into one class type to reduce.
> In current code, ObjectWritable is a sole choice. but it costs too many space, because the class declaration will be appended into output file as a string for every Key-value pair.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-689) hadoop should provide a common way to
wrap instances with different types into one type
Posted by "Feng Jiang (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-689?page=all ]
Feng Jiang updated HADOOP-689:
------------------------------
Status: Patch Available (was: Open)
Submit a new patch to replace the old patch. (just refine the javadoc.)
> hadoop should provide a common way to wrap instances with different types into one type
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-689
> URL: http://issues.apache.org/jira/browse/HADOOP-689
> Project: Hadoop
> Issue Type: Improvement
> Components: io
> Environment: All environment
> Reporter: Feng Jiang
> Attachments: HADOOP-689.patch
>
>
> When two sequence files, which have same Key type but different Value types, are mapped out to reduce, multiple Value types is not allowed. In this case, we need a way to wrap instances with different types into one class type to reduce.
> In current code, ObjectWritable is a sole choice. but it costs too many space, because the class declaration will be appended into output file as a string for every Key-value pair.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-689) hadoop should provide a common way to
wrap instances with different types into one type
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-689?page=all ]
Doug Cutting updated HADOOP-689:
--------------------------------
Status: Open (was: Patch Available)
> hadoop should provide a common way to wrap instances with different types into one type
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-689
> URL: http://issues.apache.org/jira/browse/HADOOP-689
> Project: Hadoop
> Issue Type: Improvement
> Components: io
> Environment: All environment
> Reporter: Feng Jiang
> Attachments: HADOOP-689.patch
>
>
> When two sequence files, which have same Key type but different Value types, are mapped out to reduce, multiple Value types is not allowed. In this case, we need a way to wrap instances with different types into one class type to reduce.
> In current code, ObjectWritable is a sole choice. but it costs too many space, because the class declaration will be appended into output file as a string for every Key-value pair.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-689) hadoop should provide a common way to
wrap instances with different types into one type
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-689?page=all ]
Doug Cutting updated HADOOP-689:
--------------------------------
Status: Patch Available (was: Open)
Adding this to the patch-review queue...
> hadoop should provide a common way to wrap instances with different types into one type
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-689
> URL: http://issues.apache.org/jira/browse/HADOOP-689
> Project: Hadoop
> Issue Type: Improvement
> Components: io
> Environment: All environment
> Reporter: Feng Jiang
> Attachments: HADOOP-689.patch
>
>
> When two sequence files, which have same Key type but different Value types, are mapped out to reduce, multiple Value types is not allowed. In this case, we need a way to wrap instances with different types into one class type to reduce.
> In current code, ObjectWritable is a sole choice. but it costs too many space, because the class declaration will be appended into output file as a string for every Key-value pair.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-689) hadoop should provide a common way to
wrap instances with different types into one type
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-689?page=all ]
Doug Cutting updated HADOOP-689:
--------------------------------
Status: Resolved (was: Patch Available)
Fix Version/s: 0.9.0
Resolution: Fixed
I just committed this. Thanks, Feng Jiang!
> hadoop should provide a common way to wrap instances with different types into one type
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-689
> URL: http://issues.apache.org/jira/browse/HADOOP-689
> Project: Hadoop
> Issue Type: Improvement
> Components: io
> Environment: All environment
> Reporter: Feng Jiang
> Fix For: 0.9.0
>
> Attachments: HADOOP-689.patch, HADOOP-689.patch.new
>
>
> When two sequence files, which have same Key type but different Value types, are mapped out to reduce, multiple Value types is not allowed. In this case, we need a way to wrap instances with different types into one class type to reduce.
> In current code, ObjectWritable is a sole choice. but it costs too many space, because the class declaration will be appended into output file as a string for every Key-value pair.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira