You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Takuya Ueshin (JIRA)" <ji...@apache.org> on 2011/09/14 03:16:09 UTC
[jira] [Created] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
---------------------------------------------------------------------------------------------------------
Key: HBASE-4398
URL: https://issues.apache.org/jira/browse/HBASE-4398
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.90.4
Reporter: Takuya Ueshin
If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
We can reproduce the problem by the following instructions:
{noformat}
- Add HRegionPartitioner.class to the 4th argument of
TableMapReduceUtil#initTableReducerJob()
at line around 133
in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
- Change or remove "hbase.zookeeper.property.clientPort" property
in hbase-site.xml ( for example, changed to 12345 ).
- run testMultiRegionTable()
{noformat}
Then I got error messages as following:
{noformat}
2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
=====
2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
java.lang.NullPointerException
at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
{noformat}
I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241891#comment-13241891 ]
Hudson commented on HBASE-4398:
-------------------------------
Integrated in HBase-0.94 #67 (See [https://builds.apache.org/job/HBase-0.94/67/])
HBASE-4398 If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml (Revision 1307117)
Result = SUCCESS
stack :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.92.2, 0.94.1
>
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245381#comment-13245381 ]
stack commented on HBASE-4398:
------------------------------
@Takuya Sorry about that. A temporary disable while trying to figure why trunk build was broken got committed. I fixed trunk over in HBASE-5704. Thanks for noticing this.
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.92.2, 0.94.0
>
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "Takuya Ueshin (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245363#comment-13245363 ]
Takuya Ueshin commented on HBASE-4398:
--------------------------------------
Rolled back in trunk?
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.92.2, 0.94.0
>
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245750#comment-13245750 ]
Hudson commented on HBASE-4398:
-------------------------------
Integrated in HBase-TRUNK #2706 (See [https://builds.apache.org/job/HBase-TRUNK/2706/])
HBASE-5704 HBASE-4398 mistakenly rolled back on trunk (Revision 1308964)
Result = FAILURE
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.92.2, 0.94.0
>
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242062#comment-13242062 ]
Hudson commented on HBASE-4398:
-------------------------------
Integrated in HBase-TRUNK-security #154 (See [https://builds.apache.org/job/HBase-TRUNK-security/154/])
HBASE-4398 If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml (Revision 1307116)
Result = FAILURE
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.92.2, 0.94.1
>
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246954#comment-13246954 ]
Hudson commented on HBASE-4398:
-------------------------------
Integrated in HBase-0.92-security #104 (See [https://builds.apache.org/job/HBase-0.92-security/104/])
HBASE-4398 If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml (Revision 1307118)
Result = FAILURE
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.92.2, 0.94.0
>
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241792#comment-13241792 ]
stack commented on HBASE-4398:
------------------------------
Thanks for the patch Takuya. I applied it to 0.92, 0.94, and trunk.
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.92.2, 0.94.1
>
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242473#comment-13242473 ]
Hudson commented on HBASE-4398:
-------------------------------
Integrated in HBase-TRUNK #2698 (See [https://builds.apache.org/job/HBase-TRUNK/2698/])
HBASE-4398 If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml (Revision 1307116)
Result = FAILURE
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.92.2, 0.94.1
>
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-4398:
---------------------------------
Fix Version/s: (was: 0.94.1)
0.94.0
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.92.2, 0.94.0
>
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241994#comment-13241994 ]
Hudson commented on HBASE-4398:
-------------------------------
Integrated in HBase-0.92 #344 (See [https://builds.apache.org/job/HBase-0.92/344/])
HBASE-4398 If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml (Revision 1307118)
Result = SUCCESS
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.92.2, 0.94.1
>
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "Takuya Ueshin (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Takuya Ueshin updated HBASE-4398:
---------------------------------
Attachment: HBASE-4398.patch
I attatched a patch file.
Tnis includes a patch for HRegionPartitioner and a test case.
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "stack (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-4398.
--------------------------
Resolution: Fixed
Fix Version/s: 0.94.1
0.92.2
Assignee: Takuya Ueshin
Hadoop Flags: Reviewed
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.92.2, 0.94.1
>
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241787#comment-13241787 ]
stack commented on HBASE-4398:
------------------------------
Good one Takuya. Thats broke. I would suggest that your test case while comprehensive, it is way overkill for your fix. Its also mostly a copy of another heavy-weight test we already run. I can't believe I'm saying this but I'm going to commit this fix w/o including the test.
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247030#comment-13247030 ]
Hudson commented on HBASE-4398:
-------------------------------
Integrated in HBase-TRUNK-security #157 (See [https://builds.apache.org/job/HBase-TRUNK-security/157/])
HBASE-5704 HBASE-4398 mistakenly rolled back on trunk (Revision 1308964)
Result = FAILURE
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.92.2, 0.94.0
>
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in
MapReduce, client side configurations are overwritten by hbase-site.xml.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242447#comment-13242447 ]
Hudson commented on HBASE-4398:
-------------------------------
Integrated in HBase-0.94-security #6 (See [https://builds.apache.org/job/HBase-0.94-security/6/])
HBASE-4398 If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml (Revision 1307117)
Result = SUCCESS
stack :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4398
> URL: https://issues.apache.org/jira/browse/HBASE-4398
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Fix For: 0.92.2, 0.94.1
>
> Attachments: HBASE-4398.patch
>
>
> If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.
> We can reproduce the problem by the following instructions:
> {noformat}
> - Add HRegionPartitioner.class to the 4th argument of
> TableMapReduceUtil#initTableReducerJob()
> at line around 133
> in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
> - Change or remove "hbase.zookeeper.property.clientPort" property
> in hbase-site.xml ( for example, changed to 12345 ).
> - run testMultiRegionTable()
> {noformat}
> Then I got error messages as following:
> {noformat}
> 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection opening connection to ZooKeeper with ensemble (localhost:12345)
> 2011-09-12 22:28:51,022 INFO [Thread-832] zookeeper.RecoverableZooKeeper(89): The identifier of this process is 43200@imac.local
> 2011-09-12 22:28:51,123 WARN [Thread-832] zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
> 2011-09-12 22:28:51,123 INFO [Thread-832] zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after sleeping 1000 ms
> =====
> 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d closed
> 2011-09-12 22:29:02,422 WARN [Thread-832] mapred.LocalJobRunner$Job(256): job_local_0001
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> {noformat}
> I think HTable should connect to ZooKeeper at port 21818 configured at client side instead of 12345 in hbase-site.xml
> and It might be caused by "HBaseConfiguration.addHbaseResources(conf);" in HRegionPartitioner#setConf(Configuration).
> And this might mean that all of client side configurations, also configured in hbase-site.xml, are overwritten caused by this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira