You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Aditya Kishore (JIRA)" <ji...@apache.org> on 2012/07/26 02:32:33 UTC
[jira] [Created] (HBASE-6455)
org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output
path as a child of input path
Aditya Kishore created HBASE-6455:
-------------------------------------
Summary: org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
Key: HBASE-6455
URL: https://issues.apache.org/jira/browse/HBASE-6455
Project: HBase
Issue Type: Bug
Components: performance
Affects Versions: 0.94.0, 0.96.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
Priority: Minor
Attachments: HBASE-6455_trunk.patch
I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path.
The attached patch addresses both the issues.
# Creates separate input and output path rooted under a single folder e.g. "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/inputs" and "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/outputs", and
# The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6455)
org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output
path as a child of input path
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423137#comment-13423137 ]
Zhihong Ted Yu commented on HBASE-6455:
---------------------------------------
Integrated to trunk.
Thanks for the patch, Aditya.
Thanks for the review, Stack.
> org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
> ------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6455
> URL: https://issues.apache.org/jira/browse/HBASE-6455
> Project: HBase
> Issue Type: Bug
> Components: performance
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Priority: Minor
> Labels: benchmark
> Fix For: 0.96.0
>
> Attachments: HBASE-6455_trunk.patch
>
>
> I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path.
> The attached patch addresses both the issues.
> # Creates separate input and output path rooted under a single folder e.g. "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/inputs" and "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/outputs", and
> # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6455)
org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output
path as a child of input path
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-6455:
--------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
> ------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6455
> URL: https://issues.apache.org/jira/browse/HBASE-6455
> Project: HBase
> Issue Type: Bug
> Components: performance
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Priority: Minor
> Labels: benchmark
> Fix For: 0.96.0
>
> Attachments: HBASE-6455_trunk.patch
>
>
> I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path.
> The attached patch addresses both the issues.
> # Creates separate input and output path rooted under a single folder e.g. "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/inputs" and "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/outputs", and
> # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6455)
org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output
path as a child of input path
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423510#comment-13423510 ]
Hudson commented on HBASE-6455:
-------------------------------
Integrated in HBase-TRUNK #3173 (See [https://builds.apache.org/job/HBase-TRUNK/3173/])
HBASE-6455 org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path (Aditya) (Revision 1366034)
Result = FAILURE
tedyu :
Files :
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
> org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
> ------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6455
> URL: https://issues.apache.org/jira/browse/HBASE-6455
> Project: HBase
> Issue Type: Bug
> Components: performance
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Priority: Minor
> Labels: benchmark
> Fix For: 0.96.0
>
> Attachments: HBASE-6455_trunk.patch
>
>
> I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path.
> The attached patch addresses both the issues.
> # Creates separate input and output path rooted under a single folder e.g. "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/inputs" and "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/outputs", and
> # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6455)
org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output
path as a child of input path
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422855#comment-13422855 ]
Zhihong Ted Yu commented on HBASE-6455:
---------------------------------------
Patch looks good to me.
> org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
> ------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6455
> URL: https://issues.apache.org/jira/browse/HBASE-6455
> Project: HBase
> Issue Type: Bug
> Components: performance
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Priority: Minor
> Labels: benchmark
> Fix For: 0.96.0
>
> Attachments: HBASE-6455_trunk.patch
>
>
> I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path.
> The attached patch addresses both the issues.
> # Creates separate input and output path rooted under a single folder e.g. "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/inputs" and "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/outputs", and
> # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6455)
org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output
path as a child of input path
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aditya Kishore updated HBASE-6455:
----------------------------------
Attachment: HBASE-6455_trunk.patch
Patch attached for trunk
> org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
> ------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6455
> URL: https://issues.apache.org/jira/browse/HBASE-6455
> Project: HBase
> Issue Type: Bug
> Components: performance
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Priority: Minor
> Labels: benchmark
> Attachments: HBASE-6455_trunk.patch
>
>
> I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path.
> The attached patch addresses both the issues.
> # Creates separate input and output path rooted under a single folder e.g. "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/inputs" and "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/outputs", and
> # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6455)
org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output
path as a child of input path
Posted by "Michael Drzal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453002#comment-13453002 ]
Michael Drzal commented on HBASE-6455:
--------------------------------------
[~zhihyu@ebaysf.com] can we close this out?
> org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
> ------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6455
> URL: https://issues.apache.org/jira/browse/HBASE-6455
> Project: HBase
> Issue Type: Bug
> Components: performance
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Priority: Minor
> Labels: benchmark
> Fix For: 0.96.0
>
> Attachments: HBASE-6455_trunk.patch
>
>
> I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path.
> The attached patch addresses both the issues.
> # Creates separate input and output path rooted under a single folder e.g. "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/inputs" and "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/outputs", and
> # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6455)
org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output
path as a child of input path
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422815#comment-13422815 ]
Hadoop QA commented on HBASE-6455:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12537934/HBASE-6455_trunk.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).
-1 findbugs. The patch appears to introduce 14 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2437//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2437//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2437//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2437//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2437//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2437//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2437//console
This message is automatically generated.
> org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
> ------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6455
> URL: https://issues.apache.org/jira/browse/HBASE-6455
> Project: HBase
> Issue Type: Bug
> Components: performance
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Priority: Minor
> Labels: benchmark
> Fix For: 0.96.0
>
> Attachments: HBASE-6455_trunk.patch
>
>
> I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path.
> The attached patch addresses both the issues.
> # Creates separate input and output path rooted under a single folder e.g. "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/inputs" and "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/outputs", and
> # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6455)
org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output
path as a child of input path
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422953#comment-13422953 ]
stack commented on HBASE-6455:
------------------------------
+1 on patch
> org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
> ------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6455
> URL: https://issues.apache.org/jira/browse/HBASE-6455
> Project: HBase
> Issue Type: Bug
> Components: performance
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Priority: Minor
> Labels: benchmark
> Fix For: 0.96.0
>
> Attachments: HBASE-6455_trunk.patch
>
>
> I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path.
> The attached patch addresses both the issues.
> # Creates separate input and output path rooted under a single folder e.g. "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/inputs" and "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/outputs", and
> # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6455)
org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output
path as a child of input path
Posted by "Aditya Kishore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aditya Kishore updated HBASE-6455:
----------------------------------
Fix Version/s: 0.96.0
Status: Patch Available (was: Open)
> org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
> ------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6455
> URL: https://issues.apache.org/jira/browse/HBASE-6455
> Project: HBase
> Issue Type: Bug
> Components: performance
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Priority: Minor
> Labels: benchmark
> Fix For: 0.96.0
>
> Attachments: HBASE-6455_trunk.patch
>
>
> I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path.
> The attached patch addresses both the issues.
> # Creates separate input and output path rooted under a single folder e.g. "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/inputs" and "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/outputs", and
> # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6455)
org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output
path as a child of input path
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423444#comment-13423444 ]
Hudson commented on HBASE-6455:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #111 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/111/])
HBASE-6455 org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path (Aditya) (Revision 1366034)
Result = FAILURE
tedyu :
Files :
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
> org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
> ------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6455
> URL: https://issues.apache.org/jira/browse/HBASE-6455
> Project: HBase
> Issue Type: Bug
> Components: performance
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Priority: Minor
> Labels: benchmark
> Fix For: 0.96.0
>
> Attachments: HBASE-6455_trunk.patch
>
>
> I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path.
> The attached patch addresses both the issues.
> # Creates separate input and output path rooted under a single folder e.g. "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/inputs" and "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/outputs", and
> # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira