You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Konstantin Shvachko (JIRA)" <ji...@apache.org> on 2009/05/16 02:23:45 UTC
[jira] Created: (HADOOP-5858) Eliminate UTF8 and fix warnings in
test/hdfs-with-mr package
Eliminate UTF8 and fix warnings in test/hdfs-with-mr package
------------------------------------------------------------
Key: HADOOP-5858
URL: https://issues.apache.org/jira/browse/HADOOP-5858
Project: Hadoop Core
Issue Type: Improvement
Affects Versions: 0.19.0
Reporter: Konstantin Shvachko
Assignee: Konstantin Shvachko
Fix For: 0.21.0
Replace UTF8 with Text and fix java warnings not related to deprecated mapred api.
Warnings related to the deprecated map reduce api should probably be targeted in a separate unified approach.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5858) Eliminate UTF8 and fix warnings in
test/hdfs-with-mr package
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-5858:
----------------------------------------
Status: Patch Available (was: Open)
> Eliminate UTF8 and fix warnings in test/hdfs-with-mr package
> ------------------------------------------------------------
>
> Key: HADOOP-5858
> URL: https://issues.apache.org/jira/browse/HADOOP-5858
> Project: Hadoop Core
> Issue Type: Improvement
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: hdfsWithMR.patch
>
>
> Replace UTF8 with Text and fix java warnings not related to deprecated mapred api.
> Warnings related to the deprecated map reduce api should probably be targeted in a separate unified approach.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5858) Eliminate UTF8 and fix warnings in
test/hdfs-with-mr package
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-5858:
----------------------------------------
Attachment: hdfsWithMR.patch
Fixed all three suggestions.
# Using {{VALUE_TYPE_*}} constants instead of "l:" etc in all related tests.
# {{VALUE_TYPE_LONG}} is not public.
# The return values of {{read()}} are verified.
> Eliminate UTF8 and fix warnings in test/hdfs-with-mr package
> ------------------------------------------------------------
>
> Key: HADOOP-5858
> URL: https://issues.apache.org/jira/browse/HADOOP-5858
> Project: Hadoop Core
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: hdfsWithMR.patch, hdfsWithMR.patch
>
>
> Replace UTF8 with Text and fix java warnings not related to deprecated mapred api.
> Warnings related to the deprecated map reduce api should probably be targeted in a separate unified approach.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5858) Eliminate UTF8 and fix warnings in
test/hdfs-with-mr package
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710049#action_12710049 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-5858:
------------------------------------------------
Changes look good to me. Thanks, Konstantin.
Want to mention that the following codes in the AccumulatingReducer is inefficient: The sSum string should be constructed by StringBuilder for reducing unnecessary object creations. Of course, this is an existing problem and has nothing to do with the patch or this issue.
{code}
String sSum = "";
while (values.hasNext())
sSum += values.next().toString() + ";";
{code}
> Eliminate UTF8 and fix warnings in test/hdfs-with-mr package
> ------------------------------------------------------------
>
> Key: HADOOP-5858
> URL: https://issues.apache.org/jira/browse/HADOOP-5858
> Project: Hadoop Core
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: hdfsWithMR.patch, hdfsWithMR.patch
>
>
> Replace UTF8 with Text and fix java warnings not related to deprecated mapred api.
> Warnings related to the deprecated map reduce api should probably be targeted in a separate unified approach.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5858) Eliminate UTF8 and fix warnings in
test/hdfs-with-mr package
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-5858:
----------------------------------------
Attachment: hdfsWithMR.patch
> Eliminate UTF8 and fix warnings in test/hdfs-with-mr package
> ------------------------------------------------------------
>
> Key: HADOOP-5858
> URL: https://issues.apache.org/jira/browse/HADOOP-5858
> Project: Hadoop Core
> Issue Type: Improvement
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: hdfsWithMR.patch
>
>
> Replace UTF8 with Text and fix java warnings not related to deprecated mapred api.
> Warnings related to the deprecated map reduce api should probably be targeted in a separate unified approach.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5858) Eliminate UTF8 and fix warnings in
test/hdfs-with-mr package
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-5858:
----------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this.
> Eliminate UTF8 and fix warnings in test/hdfs-with-mr package
> ------------------------------------------------------------
>
> Key: HADOOP-5858
> URL: https://issues.apache.org/jira/browse/HADOOP-5858
> Project: Hadoop Core
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: hdfsWithMR.patch, hdfsWithMR.patch
>
>
> Replace UTF8 with Text and fix java warnings not related to deprecated mapred api.
> Warnings related to the deprecated map reduce api should probably be targeted in a separate unified approach.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5858) Eliminate UTF8 and fix warnings in
test/hdfs-with-mr package
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710424#action_12710424 ]
Konstantin Shvachko commented on HADOOP-5858:
---------------------------------------------
As I said in the beginning converting to the new "mapreduce" api from "mapred" is not the goal of this issue.
> Eliminate UTF8 and fix warnings in test/hdfs-with-mr package
> ------------------------------------------------------------
>
> Key: HADOOP-5858
> URL: https://issues.apache.org/jira/browse/HADOOP-5858
> Project: Hadoop Core
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: hdfsWithMR.patch, hdfsWithMR.patch
>
>
> Replace UTF8 with Text and fix java warnings not related to deprecated mapred api.
> Warnings related to the deprecated map reduce api should probably be targeted in a separate unified approach.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5858) Eliminate UTF8 and fix warnings in
test/hdfs-with-mr package
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-5858:
-------------------------------------------
Component/s: test
Should we use the constants VALUE_TYPE_LONG in TestDFSIO?
> Eliminate UTF8 and fix warnings in test/hdfs-with-mr package
> ------------------------------------------------------------
>
> Key: HADOOP-5858
> URL: https://issues.apache.org/jira/browse/HADOOP-5858
> Project: Hadoop Core
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: hdfsWithMR.patch
>
>
> Replace UTF8 with Text and fix java warnings not related to deprecated mapred api.
> Warnings related to the deprecated map reduce api should probably be targeted in a separate unified approach.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5858) Eliminate UTF8 and fix warnings in
test/hdfs-with-mr package
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710041#action_12710041 ]
Suresh Srinivas commented on HADOOP-5858:
-----------------------------------------
# TestHarFileSystem.java - Not using {{fin.read(b)}} return value might result in findbugs warnings
# nit - {{AccumulatingReducer.VALUE_TYPE_LONG}} is public but other {{VALUE_TYPE_*}} are not
> Eliminate UTF8 and fix warnings in test/hdfs-with-mr package
> ------------------------------------------------------------
>
> Key: HADOOP-5858
> URL: https://issues.apache.org/jira/browse/HADOOP-5858
> Project: Hadoop Core
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: hdfsWithMR.patch
>
>
> Replace UTF8 with Text and fix java warnings not related to deprecated mapred api.
> Warnings related to the deprecated map reduce api should probably be targeted in a separate unified approach.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5858) Eliminate UTF8 and fix warnings in
test/hdfs-with-mr package
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710136#action_12710136 ]
Konstantin Boudnik commented on HADOOP-5858:
--------------------------------------------
It seems like AccumulatingReducer shouldn't extend MapReducerBase not implement org.apache.hadoop.mapred.Reducer
Instead it needs to extend org.apache.hadoop.mapreduce.Reducer
public class AccumulatingReducer extends Reducer<Text, Text, Text, Text>
> Eliminate UTF8 and fix warnings in test/hdfs-with-mr package
> ------------------------------------------------------------
>
> Key: HADOOP-5858
> URL: https://issues.apache.org/jira/browse/HADOOP-5858
> Project: Hadoop Core
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: 0.21.0
>
> Attachments: hdfsWithMR.patch, hdfsWithMR.patch
>
>
> Replace UTF8 with Text and fix java warnings not related to deprecated mapred api.
> Warnings related to the deprecated map reduce api should probably be targeted in a separate unified approach.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.