You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Takuya Ueshin (Created) (JIRA)" <ji...@apache.org> on 2012/03/26 18:34:26 UTC
[jira] [Created] (HBASE-5636) TestTableMapReduce doesn't work
properly.
TestTableMapReduce doesn't work properly.
-----------------------------------------
Key: HBASE-5636
URL: https://issues.apache.org/jira/browse/HBASE-5636
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.92.1
Reporter: Takuya Ueshin
No map function is called because there are no test data put before test starts.
The following three tests are in the same situation:
- org.apache.hadoop.hbase.mapred.TestTableMapReduce
- org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
- org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Enis Soztutar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239125#comment-13239125 ]
Enis Soztutar commented on HBASE-5636:
--------------------------------------
Very good find. It seems that HBASE-4503 might have caused it. Do you plan to work on this?
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1
> Reporter: Takuya Ueshin
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Zhihong Yu (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu reassigned HBASE-5636:
---------------------------------
Assignee: Takuya Ueshin
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Attachments: HBASE-5636-v2.patch, HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Nicolas Spiegelberg (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246762#comment-13246762 ]
Nicolas Spiegelberg commented on HBASE-5636:
--------------------------------------------
@Ted: It looks like this patch is introducing intermittent unit test failures in the trunk. Can we revert or fix? https://builds.apache.org/job/PreCommit-HBASE-Build/1369/
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.94.0, 0.96.0
>
> Attachments: HBASE-5636-v2.patch, HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Takuya Ueshin (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Takuya Ueshin updated HBASE-5636:
---------------------------------
Affects Version/s: 0.94.0
I'm sorry, TestMulitthreadedTableMapper was not included in 0.92.x.
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244244#comment-13244244 ]
Hudson commented on HBASE-5636:
-------------------------------
Integrated in HBase-0.94 #75 (See [https://builds.apache.org/job/HBase-0.94/75/])
HBASE-5663 HBASE-5636 MultithreadedTableMapper doesn't work (Takuya Ueshin) (Revision 1308354)
Result = FAILURE
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/MultithreadedTableMapper.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapred/TestTableMapReduce.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMulitthreadedTableMapper.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultithreadedTableMapper.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.94.0, 0.96.0
>
> Attachments: HBASE-5636-v2.patch, HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240517#comment-13240517 ]
Zhihong Yu commented on HBASE-5636:
-----------------------------------
Changes for mapred/TestTableMapReduce.java and mapreduce/TestTableMapReduce.java couldn't apply cleanly.
Can you upload a new patch for trunk ?
Thanks
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Attachments: HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Takuya Ueshin (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240514#comment-13240514 ]
Takuya Ueshin commented on HBASE-5636:
--------------------------------------
Oh, I'm very sorry.
Here is the diff of TestMulitthreadedTableMapper.java:
{noformat}
diff --git a/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMulitthreadedTableMapper.java b/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMulitthreadedTableMapper.java
index cc5b1df..ad34dd2 100644
--- a/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMulitthreadedTableMapper.java
+++ b/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMulitthreadedTableMapper.java
@@ -19,6 +19,7 @@ package org.apache.hadoop.hbase.mapreduce;
import java.io.File;
import java.io.IOException;
+import java.util.Iterator;
import java.util.Map;
import java.util.NavigableMap;
@@ -28,7 +29,6 @@ import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileUtil;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.*;
-import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
@@ -56,19 +56,17 @@ public class TestMulitthreadedTableMapper {
private static final Log LOG = LogFactory.getLog(TestMulitthreadedTableMapper.class);
private static final HBaseTestingUtility UTIL =
new HBaseTestingUtility();
- static final String MULTI_REGION_TABLE_NAME = "mrtest";
+ static final byte[] MULTI_REGION_TABLE_NAME = Bytes.toBytes("mrtest");
static final byte[] INPUT_FAMILY = Bytes.toBytes("contents");
static final byte[] OUTPUT_FAMILY = Bytes.toBytes("text");
static final int NUMBER_OF_THREADS = 10;
@BeforeClass
public static void beforeClass() throws Exception {
- HTableDescriptor desc = new HTableDescriptor(MULTI_REGION_TABLE_NAME);
- desc.addFamily(new HColumnDescriptor(INPUT_FAMILY));
- desc.addFamily(new HColumnDescriptor(OUTPUT_FAMILY));
UTIL.startMiniCluster();
- HBaseAdmin admin = new HBaseAdmin(UTIL.getConfiguration());
- admin.createTable(desc, HBaseTestingUtility.KEYS);
+ HTable table = UTIL.createTable(MULTI_REGION_TABLE_NAME, new byte[][] {INPUT_FAMILY, OUTPUT_FAMILY});
+ UTIL.createMultiRegions(table, INPUT_FAMILY);
+ UTIL.loadTable(table, INPUT_FAMILY);
UTIL.startMiniMapReduceCluster();
}
@@ -149,7 +147,7 @@ public class TestMulitthreadedTableMapper {
IdentityTableReducer.class, job);
FileOutputFormat.setOutputPath(job, new Path("test"));
LOG.info("Started " + Bytes.toString(table.getTableName()));
- job.waitForCompletion(true);
+ assertTrue(job.waitForCompletion(true));
LOG.info("After map/reduce completion");
// verify map-reduce results
verify(Bytes.toString(table.getTableName()));
@@ -203,7 +201,10 @@ public class TestMulitthreadedTableMapper {
scan.addFamily(OUTPUT_FAMILY);
ResultScanner scanner = table.getScanner(scan);
try {
- for (Result r : scanner) {
+ Iterator<Result> itr = scanner.iterator();
+ assertTrue(itr.hasNext());
+ while(itr.hasNext()) {
+ Result r = itr.next();
if (LOG.isDebugEnabled()) {
if (r.size() > 2 ) {
throw new IOException("Too many results, expected 2 got " +
{noformat}
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Attachments: HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240506#comment-13240506 ]
Zhihong Yu commented on HBASE-5636:
-----------------------------------
Since TestMultithreadedTableMapper.java is a new file, can you outline the changes you made on top of TestMulitthreadedTableMapper ?
I noticed this:
{code}
+ UTIL.loadTable(table, INPUT_FAMILY);
{code}
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Attachments: HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246899#comment-13246899 ]
Hudson commented on HBASE-5636:
-------------------------------
Integrated in HBase-0.94-security #7 (See [https://builds.apache.org/job/HBase-0.94-security/7/])
HBASE-5663 HBASE-5636 MultithreadedTableMapper doesn't work (Takuya Ueshin) (Revision 1308354)
Result = SUCCESS
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/MultithreadedTableMapper.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapred/TestTableMapReduce.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMulitthreadedTableMapper.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultithreadedTableMapper.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.94.0, 0.96.0
>
> Attachments: HBASE-5636-v2.patch, HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5636:
------------------------------
Status: Patch Available (was: Open)
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Attachments: HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5636:
------------------------------
Fix Version/s: 0.96.0
0.94.0
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.94.0, 0.96.0
>
> Attachments: HBASE-5636-v2.patch, HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243180#comment-13243180 ]
Zhihong Yu commented on HBASE-5636:
-----------------------------------
Without patch from HBASE-5663, I saw the following in test output:
{code}
2012-03-31 07:55:46,743 DEBUG [main] mapreduce.TableInputFormatBase(194): getSplits: split -> 24 -> 192.168.0.17:yyy,
java.io.IOException: java.lang.NoSuchMethodException: org.apache.hadoop.mapreduce.Mapper$Context.<init>(org.apache.hadoop.conf.Configuration, org.apache.hadoop.mapred.TaskAttemptID, org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordReader, org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordWriter, org.apache.hadoop.hbase.mapreduce.TableOutputCommitter, org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapStatusReporter, org.apache.hadoop.hbase.mapreduce.TableSplit)
at org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$MapRunner.<init>(MultithreadedTableMapper.java:260)
at org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper.run(MultithreadedTableMapper.java:133)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
{code}
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Attachments: HBASE-5636-v2.patch, HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Takuya Ueshin (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Takuya Ueshin updated HBASE-5636:
---------------------------------
Attachment: HBASE-5636-v2.patch
I attached a patch v2.
Maybe I forgot --no-prefix option of git-diff command.
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Attachments: HBASE-5636-v2.patch, HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244771#comment-13244771 ]
Hudson commented on HBASE-5636:
-------------------------------
Integrated in HBase-TRUNK #2703 (See [https://builds.apache.org/job/HBase-TRUNK/2703/])
HBASE-5663 HBASE-5636 MultithreadedTableMapper doesn't work (Takuya Ueshin) (Revision 1308353)
Result = SUCCESS
tedyu :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/MultithreadedTableMapper.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapred/TestTableMapReduce.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMulitthreadedTableMapper.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultithreadedTableMapper.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.94.0, 0.96.0
>
> Attachments: HBASE-5636-v2.patch, HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Takuya Ueshin (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Takuya Ueshin updated HBASE-5636:
---------------------------------
Attachment: HBASE-5636.patch
I attached a patch file for trunk.
This includes patches for TestMulitthreadedTableMapper.
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Attachments: HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Takuya Ueshin (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239154#comment-13239154 ]
Takuya Ueshin commented on HBASE-5636:
--------------------------------------
Yes, I'm going to work this week or weekend.
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1
> Reporter: Takuya Ueshin
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246777#comment-13246777 ]
Zhihong Yu commented on HBASE-5636:
-----------------------------------
You're suggesting we refine the logic of finding failed tests in dev-support/test-patch.sh ?
That sounds good.
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.94.0, 0.96.0
>
> Attachments: HBASE-5636-v2.patch, HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246765#comment-13246765 ]
Zhihong Yu commented on HBASE-5636:
-----------------------------------
A careful check of the failed test:
https://builds.apache.org/job/PreCommit-HBASE-Build/1369/testReport/junit/org.apache.hadoop.hbase.mapreduce/TestMultithreadedTableMapper/testMultithreadedTableMapper/
would reveal the following:
{code}
2012-04-02 14:54:57,385 DEBUG [pool-1-thread-1] mapreduce.TableInputFormatBase(194): getSplits: split -> 24 -> asf011.sp2.ygridcore.net.:yyy,
java.lang.NumberFormatException: For input string: "18446743988145906374"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Long.parseLong(Long.java:422)
at java.lang.Long.parseLong(Long.java:468)
at org.apache.hadoop.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:413)
at org.apache.hadoop.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:148)
at org.apache.hadoop.util.LinuxResourceCalculatorPlugin.getProcResourceValues(LinuxResourceCalculatorPlugin.java:401)
at org.apache.hadoop.mapred.Task.initialize(Task.java:536)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
{code}
This should have been fixed by MAPREDUCE-3583
Once hadoop 1.0.2 comes out, we won't see the above exceptions.
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.94.0, 0.96.0
>
> Attachments: HBASE-5636-v2.patch, HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Takuya Ueshin (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240393#comment-13240393 ]
Takuya Ueshin commented on HBASE-5636:
--------------------------------------
I filed the new issue of the bug.
By the way, the TestCase name 'TestMulitthreadedTableMapper' is typo, isn't it?
I will rename it to 'TestMultithreadedTableMapper'.
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1
> Reporter: Takuya Ueshin
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240513#comment-13240513 ]
Hadoop QA commented on HBASE-5636:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12520270/HBASE-5636.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 10 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1329//console
This message is automatically generated.
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Attachments: HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Nicolas Spiegelberg (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246770#comment-13246770 ]
Nicolas Spiegelberg commented on HBASE-5636:
--------------------------------------------
@Ted: we should list the tests as expected failures without Hadoop 1.0.2 then. Hadoop QA should be able to point out when a new patch introduces test failures & avoid forcing the user to understand the context of features that other people are working on.
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.94.0, 0.96.0
>
> Attachments: HBASE-5636-v2.patch, HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245005#comment-13245005 ]
Hudson commented on HBASE-5636:
-------------------------------
Integrated in HBase-TRUNK-security #156 (See [https://builds.apache.org/job/HBase-TRUNK-security/156/])
HBASE-5663 HBASE-5636 MultithreadedTableMapper doesn't work (Takuya Ueshin) (Revision 1308353)
Result = FAILURE
tedyu :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/MultithreadedTableMapper.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapred/TestTableMapReduce.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMulitthreadedTableMapper.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultithreadedTableMapper.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.94.0, 0.96.0
>
> Attachments: HBASE-5636-v2.patch, HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240540#comment-13240540 ]
Hadoop QA commented on HBASE-5636:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12520285/HBASE-5636-v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 10 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestColumnSeeking
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1331//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1331//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1331//console
This message is automatically generated.
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Attachments: HBASE-5636-v2.patch, HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5636) TestTableMapReduce doesn't work
properly.
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5636:
------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Integrated to trunk and 0.94.
Thanks for the patch, Takuya.
> TestTableMapReduce doesn't work properly.
> -----------------------------------------
>
> Key: HBASE-5636
> URL: https://issues.apache.org/jira/browse/HBASE-5636
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.1, 0.94.0
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Attachments: HBASE-5636-v2.patch, HBASE-5636.patch
>
>
> No map function is called because there are no test data put before test starts.
> The following three tests are in the same situation:
> - org.apache.hadoop.hbase.mapred.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira