You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Jarek Jarcec Cecho (JIRA)" <ji...@apache.org> on 2012/08/16 08:48:38 UTC
[jira] [Created] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Jarek Jarcec Cecho created MAPREDUCE-4562:
---------------------------------------------
Summary: Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
Key: MAPREDUCE-4562
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.0.0-alpha, 0.23.1, 0.23.0
Reporter: Jarek Jarcec Cecho
Hi Guys,
I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
Included patch seems to be adding counter "FileSystemCounter":
{code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
{code}
But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
{code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
{code}
{code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
{code}
I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436733#comment-13436733 ]
Hudson commented on MAPREDUCE-4562:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #1169 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1169/])
MAPREDUCE-4562. Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name. Contributed by Jarek Jarcec Cecho. (Revision 1373823)
Result = FAILURE
tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373823
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestCounters.java
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.23.1, 2.0.0-alpha
> Reporter: Jarek Jarcec Cecho
> Assignee: Jarek Jarcec Cecho
> Fix For: 0.23.3, 2.1.0-alpha
>
> Attachments: MAPREDUCE-4562.patch, MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435829#comment-13435829 ]
Hadoop QA commented on MAPREDUCE-4562:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12541193/MAPREDUCE-4562.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2745//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2745//console
This message is automatically generated.
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.23.1, 2.0.0-alpha
> Reporter: Jarek Jarcec Cecho
> Attachments: MAPREDUCE-4562.patch, MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed MAPREDUCE-4562.
------------------------------------
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.23.1, 2.0.0-alpha
> Reporter: Jarek Jarcec Cecho
> Assignee: Jarek Jarcec Cecho
> Fix For: 0.23.3, 2.0.2-alpha
>
> Attachments: MAPREDUCE-4562.patch, MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jarek Jarcec Cecho updated MAPREDUCE-4562:
------------------------------------------
Status: Patch Available (was: Open)
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha, 0.23.1, 0.23.0
> Reporter: Jarek Jarcec Cecho
> Attachments: MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jarek Jarcec Cecho updated MAPREDUCE-4562:
------------------------------------------
Attachment: MAPREDUCE-4562.patch
Hudson's -1 is my fault. I've attach 3 days old patch and I did not notice that trunk has changed.
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.23.1, 2.0.0-alpha
> Reporter: Jarek Jarcec Cecho
> Attachments: MAPREDUCE-4562.patch, MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435810#comment-13435810 ]
Hadoop QA commented on MAPREDUCE-4562:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12541188/MAPREDUCE-4562.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core:
org.apache.hadoop.mapred.TestCounters
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2743//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2743//console
This message is automatically generated.
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.23.1, 2.0.0-alpha
> Reporter: Jarek Jarcec Cecho
> Attachments: MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435964#comment-13435964 ]
Hudson commented on MAPREDUCE-4562:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2651 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2651/])
MAPREDUCE-4562. Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name. Contributed by Jarek Jarcec Cecho. (Revision 1373823)
Result = SUCCESS
tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373823
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestCounters.java
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.23.1, 2.0.0-alpha
> Reporter: Jarek Jarcec Cecho
> Assignee: Jarek Jarcec Cecho
> Fix For: 2.1.0-alpha
>
> Attachments: MAPREDUCE-4562.patch, MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jarek Jarcec Cecho updated MAPREDUCE-4562:
------------------------------------------
Status: Open (was: Patch Available)
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha, 0.23.1, 0.23.0
> Reporter: Jarek Jarcec Cecho
> Attachments: MAPREDUCE-4562.patch, MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-4562:
-------------------------------------------
Fix Version/s: 0.23.3
Great Catch Jarek, I pulled this into 0.23.3 too.
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.23.1, 2.0.0-alpha
> Reporter: Jarek Jarcec Cecho
> Assignee: Jarek Jarcec Cecho
> Fix For: 0.23.3, 2.1.0-alpha
>
> Attachments: MAPREDUCE-4562.patch, MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jarek Jarcec Cecho updated MAPREDUCE-4562:
------------------------------------------
Status: Patch Available (was: Open)
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha, 0.23.1, 0.23.0
> Reporter: Jarek Jarcec Cecho
> Attachments: MAPREDUCE-4562.patch, MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jarek Jarcec Cecho updated MAPREDUCE-4562:
------------------------------------------
Attachment: MAPREDUCE-4562.patch
I've provided fix to rename the backward compatible name to the original name that is present in Hadoop 0.20 and Hadoop 1.0.3.
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.23.1, 2.0.0-alpha
> Reporter: Jarek Jarcec Cecho
> Attachments: MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435971#comment-13435971 ]
Hudson commented on MAPREDUCE-4562:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #2586 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2586/])
MAPREDUCE-4562. Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name. Contributed by Jarek Jarcec Cecho. (Revision 1373823)
Result = SUCCESS
tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373823
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestCounters.java
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.23.1, 2.0.0-alpha
> Reporter: Jarek Jarcec Cecho
> Assignee: Jarek Jarcec Cecho
> Fix For: 2.1.0-alpha
>
> Attachments: MAPREDUCE-4562.patch, MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436691#comment-13436691 ]
Hudson commented on MAPREDUCE-4562:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #1137 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1137/])
MAPREDUCE-4562. Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name. Contributed by Jarek Jarcec Cecho. (Revision 1373823)
Result = FAILURE
tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373823
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestCounters.java
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.23.1, 2.0.0-alpha
> Reporter: Jarek Jarcec Cecho
> Assignee: Jarek Jarcec Cecho
> Fix For: 0.23.3, 2.1.0-alpha
>
> Attachments: MAPREDUCE-4562.patch, MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435989#comment-13435989 ]
Hudson commented on MAPREDUCE-4562:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2616 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2616/])
MAPREDUCE-4562. Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name. Contributed by Jarek Jarcec Cecho. (Revision 1373823)
Result = FAILURE
tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373823
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestCounters.java
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.23.1, 2.0.0-alpha
> Reporter: Jarek Jarcec Cecho
> Assignee: Jarek Jarcec Cecho
> Fix For: 0.23.3, 2.1.0-alpha
>
> Attachments: MAPREDUCE-4562.patch, MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436683#comment-13436683 ]
Hudson commented on MAPREDUCE-4562:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #346 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/346/])
svn merge -c 1373823. FIXES: MAPREDUCE-4562. Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name. Contributed by Jarek Jarcec Cecho. (Revision 1373840)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373840
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestCounters.java
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.23.1, 2.0.0-alpha
> Reporter: Jarek Jarcec Cecho
> Assignee: Jarek Jarcec Cecho
> Fix For: 0.23.3, 2.1.0-alpha
>
> Attachments: MAPREDUCE-4562.patch, MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated MAPREDUCE-4562:
---------------------------------
Resolution: Fixed
Fix Version/s: 2.1.0-alpha
Assignee: Jarek Jarcec Cecho
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
+1 great catch Jarek. Thanks for fixing this. I just committed this to trunk and branch 2.
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.23.1, 2.0.0-alpha
> Reporter: Jarek Jarcec Cecho
> Assignee: Jarek Jarcec Cecho
> Fix For: 2.1.0-alpha
>
> Attachments: MAPREDUCE-4562.patch, MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4562) Support for "FileSystemCounter"
legacy counter group name for compatibility reasons is creating incorrect
counter name
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435828#comment-13435828 ]
Hadoop QA commented on MAPREDUCE-4562:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12541193/MAPREDUCE-4562.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2744//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2744//console
This message is automatically generated.
> Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4562
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4562
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.23.1, 2.0.0-alpha
> Reporter: Jarek Jarcec Cecho
> Attachments: MAPREDUCE-4562.patch, MAPREDUCE-4562.patch
>
>
> Hi Guys,
> I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.
> Included patch seems to be adding counter "FileSystemCounter":
> {code:title=hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84}
> legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
> {code}
> But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> {code:title=src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)}
> protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
> {code}
> I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira