You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org> on 2010/06/17 02:05:23 UTC

[jira] Created: (HBASE-2741) NPE in ServerManager when a region is closing

NPE in ServerManager when a region is closing
---------------------------------------------

                 Key: HBASE-2741
                 URL: https://issues.apache.org/jira/browse/HBASE-2741
             Project: HBase
          Issue Type: Bug
            Reporter: Jean-Daniel Cryans
            Assignee: Karthik Ranganathan
             Fix For: 0.21.0


While running TestReplication I bumped into:

{code}
2010-06-16 16:44:07,576 DEBUG [IPC Server handler 3 on 62423] master.RegionManager(357): Created UNASSIGNED zNode test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. in state M2ZK_REGION_OFFLINE
2010-06-16 16:44:07,577 INFO  [RegionServer:0] regionserver.HRegionServer(511): MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
2010-06-16 16:44:07,577 INFO  [RegionServer:0.worker] regionserver.HRegionServer$Worker(1358): Worker: MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
2010-06-16 16:44:07,578 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENING] expected version = 0
2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
2010-06-16 16:44:07,580 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
2010-06-16 16:44:07,581 DEBUG [RegionServer:0.worker] regionserver.HRegion(294): Creating region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENING, region = de5dcd3df0fbc58207ce6ccff9ff2870
2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(81): NO-OP call to handling region opening event
2010-06-16 16:44:07,589 INFO  [RegionServer:0.worker] regionserver.HRegion(369): region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. available; sequence id is 1
2010-06-16 16:44:07,590 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENED] expected version = 1
2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
2010-06-16 16:44:07,592 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
2010-06-16 16:44:07,591 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
2010-06-16 16:44:07,593 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENED, region = de5dcd3df0fbc58207ce6ccff9ff2870
2010-06-16 16:44:07,594 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(96): RS 10.10.1.130,62425,1276731832950 has opened region de5dcd3df0fbc58207ce6ccff9ff2870
2010-06-16 16:44:07,594 ERROR [MASTER_CLOSEREGION-master-1] server.NIOServerCnxn$Factory$1(81): Thread Thread[MASTER_CLOSEREGION-master-1,5,main] died
java.lang.NullPointerException
        at org.apache.hadoop.hbase.master.ServerManager.processRegionOpen(ServerManager.java:607)
        at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.handleRegionOpenedEvent(MasterOpenRegionHandler.java:99)
        at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.process(MasterOpenRegionHandler.java:75)
        at org.apache.hadoop.hbase.executor.HBaseEventHandler.run(HBaseEventHandler.java:215)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:637)
{code}

This looks very new. Assigning Karthik as he was there recently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Resolved: (HBASE-2741) HBaseExecutorService needs to be multi-cluster friendly

Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jean-Daniel Cryans resolved HBASE-2741.
---------------------------------------

    Hadoop Flags: [Reviewed]
      Resolution: Fixed

Committed to trunk, thanks for the patch Karthik!

> HBaseExecutorService needs to be multi-cluster friendly
> -------------------------------------------------------
>
>                 Key: HBASE-2741
>                 URL: https://issues.apache.org/jira/browse/HBASE-2741
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Jean-Daniel Cryans
>            Assignee: Karthik Ranganathan
>             Fix For: 0.21.0
>
>         Attachments: HBASE-2741-0.21.patch, HBASE-2741-test.patch
>
>
> While running TestReplication I bumped into:
> {code}
> 2010-06-16 16:44:07,576 DEBUG [IPC Server handler 3 on 62423] master.RegionManager(357): Created UNASSIGNED zNode test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. in state M2ZK_REGION_OFFLINE
> 2010-06-16 16:44:07,577 INFO  [RegionServer:0] regionserver.HRegionServer(511): MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,577 INFO  [RegionServer:0.worker] regionserver.HRegionServer$Worker(1358): Worker: MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,578 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENING] expected version = 0
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,581 DEBUG [RegionServer:0.worker] regionserver.HRegion(294): Creating region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENING, region = de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(81): NO-OP call to handling region opening event
> 2010-06-16 16:44:07,589 INFO  [RegionServer:0.worker] regionserver.HRegion(369): region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. available; sequence id is 1
> 2010-06-16 16:44:07,590 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENED] expected version = 1
> 2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,592 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,591 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,593 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENED, region = de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,594 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(96): RS 10.10.1.130,62425,1276731832950 has opened region de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,594 ERROR [MASTER_CLOSEREGION-master-1] server.NIOServerCnxn$Factory$1(81): Thread Thread[MASTER_CLOSEREGION-master-1,5,main] died
> java.lang.NullPointerException
>         at org.apache.hadoop.hbase.master.ServerManager.processRegionOpen(ServerManager.java:607)
>         at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.handleRegionOpenedEvent(MasterOpenRegionHandler.java:99)
>         at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.process(MasterOpenRegionHandler.java:75)
>         at org.apache.hadoop.hbase.executor.HBaseEventHandler.run(HBaseEventHandler.java:215)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:637)
> {code}
> This looks very new. Assigning Karthik as he was there recently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-2741) NPE in ServerManager when a region is closing

Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880367#action_12880367 ] 

Jean-Daniel Cryans commented on HBASE-2741:
-------------------------------------------

Also HBaseEventHandler has static members like serverManager, and since it's not static it means that when the second master boots up it replaces the instance with his own.

> NPE in ServerManager when a region is closing
> ---------------------------------------------
>
>                 Key: HBASE-2741
>                 URL: https://issues.apache.org/jira/browse/HBASE-2741
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Jean-Daniel Cryans
>            Assignee: Karthik Ranganathan
>             Fix For: 0.21.0
>
>
> While running TestReplication I bumped into:
> {code}
> 2010-06-16 16:44:07,576 DEBUG [IPC Server handler 3 on 62423] master.RegionManager(357): Created UNASSIGNED zNode test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. in state M2ZK_REGION_OFFLINE
> 2010-06-16 16:44:07,577 INFO  [RegionServer:0] regionserver.HRegionServer(511): MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,577 INFO  [RegionServer:0.worker] regionserver.HRegionServer$Worker(1358): Worker: MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,578 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENING] expected version = 0
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,581 DEBUG [RegionServer:0.worker] regionserver.HRegion(294): Creating region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENING, region = de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(81): NO-OP call to handling region opening event
> 2010-06-16 16:44:07,589 INFO  [RegionServer:0.worker] regionserver.HRegion(369): region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. available; sequence id is 1
> 2010-06-16 16:44:07,590 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENED] expected version = 1
> 2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,592 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,591 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,593 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENED, region = de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,594 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(96): RS 10.10.1.130,62425,1276731832950 has opened region de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,594 ERROR [MASTER_CLOSEREGION-master-1] server.NIOServerCnxn$Factory$1(81): Thread Thread[MASTER_CLOSEREGION-master-1,5,main] died
> java.lang.NullPointerException
>         at org.apache.hadoop.hbase.master.ServerManager.processRegionOpen(ServerManager.java:607)
>         at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.handleRegionOpenedEvent(MasterOpenRegionHandler.java:99)
>         at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.process(MasterOpenRegionHandler.java:75)
>         at org.apache.hadoop.hbase.executor.HBaseEventHandler.run(HBaseEventHandler.java:215)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:637)
> {code}
> This looks very new. Assigning Karthik as he was there recently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HBASE-2741) NPE in ServerManager when a region is closing

Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jean-Daniel Cryans updated HBASE-2741:
--------------------------------------

    Attachment: HBASE-2741-test.patch

The test I wrote in HBASE-2756, it has to be included in this jira now.

> NPE in ServerManager when a region is closing
> ---------------------------------------------
>
>                 Key: HBASE-2741
>                 URL: https://issues.apache.org/jira/browse/HBASE-2741
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Jean-Daniel Cryans
>            Assignee: Karthik Ranganathan
>             Fix For: 0.21.0
>
>         Attachments: HBASE-2741-test.patch
>
>
> While running TestReplication I bumped into:
> {code}
> 2010-06-16 16:44:07,576 DEBUG [IPC Server handler 3 on 62423] master.RegionManager(357): Created UNASSIGNED zNode test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. in state M2ZK_REGION_OFFLINE
> 2010-06-16 16:44:07,577 INFO  [RegionServer:0] regionserver.HRegionServer(511): MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,577 INFO  [RegionServer:0.worker] regionserver.HRegionServer$Worker(1358): Worker: MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,578 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENING] expected version = 0
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,581 DEBUG [RegionServer:0.worker] regionserver.HRegion(294): Creating region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENING, region = de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(81): NO-OP call to handling region opening event
> 2010-06-16 16:44:07,589 INFO  [RegionServer:0.worker] regionserver.HRegion(369): region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. available; sequence id is 1
> 2010-06-16 16:44:07,590 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENED] expected version = 1
> 2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,592 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,591 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,593 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENED, region = de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,594 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(96): RS 10.10.1.130,62425,1276731832950 has opened region de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,594 ERROR [MASTER_CLOSEREGION-master-1] server.NIOServerCnxn$Factory$1(81): Thread Thread[MASTER_CLOSEREGION-master-1,5,main] died
> java.lang.NullPointerException
>         at org.apache.hadoop.hbase.master.ServerManager.processRegionOpen(ServerManager.java:607)
>         at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.handleRegionOpenedEvent(MasterOpenRegionHandler.java:99)
>         at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.process(MasterOpenRegionHandler.java:75)
>         at org.apache.hadoop.hbase.executor.HBaseEventHandler.run(HBaseEventHandler.java:215)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:637)
> {code}
> This looks very new. Assigning Karthik as he was there recently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HBASE-2741) HBaseExecutorService needs to be multi-cluster friendly

Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jean-Daniel Cryans updated HBASE-2741:
--------------------------------------

    Summary: HBaseExecutorService needs to be multi-cluster friendly  (was: NPE in ServerManager when a region is closing)

Changing this jira's title to what it really is about.

> HBaseExecutorService needs to be multi-cluster friendly
> -------------------------------------------------------
>
>                 Key: HBASE-2741
>                 URL: https://issues.apache.org/jira/browse/HBASE-2741
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Jean-Daniel Cryans
>            Assignee: Karthik Ranganathan
>             Fix For: 0.21.0
>
>         Attachments: HBASE-2741-test.patch
>
>
> While running TestReplication I bumped into:
> {code}
> 2010-06-16 16:44:07,576 DEBUG [IPC Server handler 3 on 62423] master.RegionManager(357): Created UNASSIGNED zNode test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. in state M2ZK_REGION_OFFLINE
> 2010-06-16 16:44:07,577 INFO  [RegionServer:0] regionserver.HRegionServer(511): MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,577 INFO  [RegionServer:0.worker] regionserver.HRegionServer$Worker(1358): Worker: MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,578 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENING] expected version = 0
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,581 DEBUG [RegionServer:0.worker] regionserver.HRegion(294): Creating region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENING, region = de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(81): NO-OP call to handling region opening event
> 2010-06-16 16:44:07,589 INFO  [RegionServer:0.worker] regionserver.HRegion(369): region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. available; sequence id is 1
> 2010-06-16 16:44:07,590 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENED] expected version = 1
> 2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,592 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,591 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,593 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENED, region = de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,594 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(96): RS 10.10.1.130,62425,1276731832950 has opened region de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,594 ERROR [MASTER_CLOSEREGION-master-1] server.NIOServerCnxn$Factory$1(81): Thread Thread[MASTER_CLOSEREGION-master-1,5,main] died
> java.lang.NullPointerException
>         at org.apache.hadoop.hbase.master.ServerManager.processRegionOpen(ServerManager.java:607)
>         at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.handleRegionOpenedEvent(MasterOpenRegionHandler.java:99)
>         at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.process(MasterOpenRegionHandler.java:75)
>         at org.apache.hadoop.hbase.executor.HBaseEventHandler.run(HBaseEventHandler.java:215)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:637)
> {code}
> This looks very new. Assigning Karthik as he was there recently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HBASE-2741) HBaseExecutorService needs to be multi-cluster friendly

Posted by "Karthik Ranganathan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Karthik Ranganathan updated HBASE-2741:
---------------------------------------

    Attachment: HBASE-2741-0.21.patch

Attaching the patch file that JD already has tested...

> HBaseExecutorService needs to be multi-cluster friendly
> -------------------------------------------------------
>
>                 Key: HBASE-2741
>                 URL: https://issues.apache.org/jira/browse/HBASE-2741
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Jean-Daniel Cryans
>            Assignee: Karthik Ranganathan
>             Fix For: 0.21.0
>
>         Attachments: HBASE-2741-0.21.patch, HBASE-2741-test.patch
>
>
> While running TestReplication I bumped into:
> {code}
> 2010-06-16 16:44:07,576 DEBUG [IPC Server handler 3 on 62423] master.RegionManager(357): Created UNASSIGNED zNode test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. in state M2ZK_REGION_OFFLINE
> 2010-06-16 16:44:07,577 INFO  [RegionServer:0] regionserver.HRegionServer(511): MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,577 INFO  [RegionServer:0.worker] regionserver.HRegionServer$Worker(1358): Worker: MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,578 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENING] expected version = 0
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,581 DEBUG [RegionServer:0.worker] regionserver.HRegion(294): Creating region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENING, region = de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(81): NO-OP call to handling region opening event
> 2010-06-16 16:44:07,589 INFO  [RegionServer:0.worker] regionserver.HRegion(369): region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. available; sequence id is 1
> 2010-06-16 16:44:07,590 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENED] expected version = 1
> 2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,592 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,591 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,593 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENED, region = de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,594 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(96): RS 10.10.1.130,62425,1276731832950 has opened region de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,594 ERROR [MASTER_CLOSEREGION-master-1] server.NIOServerCnxn$Factory$1(81): Thread Thread[MASTER_CLOSEREGION-master-1,5,main] died
> java.lang.NullPointerException
>         at org.apache.hadoop.hbase.master.ServerManager.processRegionOpen(ServerManager.java:607)
>         at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.handleRegionOpenedEvent(MasterOpenRegionHandler.java:99)
>         at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.process(MasterOpenRegionHandler.java:75)
>         at org.apache.hadoop.hbase.executor.HBaseEventHandler.run(HBaseEventHandler.java:215)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:637)
> {code}
> This looks very new. Assigning Karthik as he was there recently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-2741) HBaseExecutorService needs to be multi-cluster friendly

Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881279#action_12881279 ] 

Jean-Daniel Cryans commented on HBASE-2741:
-------------------------------------------

+1 but some nits:
 - Don't set the class attributes to null.
 - Some lines are way over 80 chars, like in ZKUnassignedWatcher

But it passes the test I wrote, going to commit with the little fixes.

> HBaseExecutorService needs to be multi-cluster friendly
> -------------------------------------------------------
>
>                 Key: HBASE-2741
>                 URL: https://issues.apache.org/jira/browse/HBASE-2741
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Jean-Daniel Cryans
>            Assignee: Karthik Ranganathan
>             Fix For: 0.21.0
>
>         Attachments: HBASE-2741-0.21.patch, HBASE-2741-test.patch
>
>
> While running TestReplication I bumped into:
> {code}
> 2010-06-16 16:44:07,576 DEBUG [IPC Server handler 3 on 62423] master.RegionManager(357): Created UNASSIGNED zNode test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. in state M2ZK_REGION_OFFLINE
> 2010-06-16 16:44:07,577 INFO  [RegionServer:0] regionserver.HRegionServer(511): MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,577 INFO  [RegionServer:0.worker] regionserver.HRegionServer$Worker(1358): Worker: MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,578 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENING] expected version = 0
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,581 DEBUG [RegionServer:0.worker] regionserver.HRegion(294): Creating region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENING, region = de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(81): NO-OP call to handling region opening event
> 2010-06-16 16:44:07,589 INFO  [RegionServer:0.worker] regionserver.HRegion(369): region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. available; sequence id is 1
> 2010-06-16 16:44:07,590 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENED] expected version = 1
> 2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,592 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,591 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,593 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENED, region = de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,594 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(96): RS 10.10.1.130,62425,1276731832950 has opened region de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,594 ERROR [MASTER_CLOSEREGION-master-1] server.NIOServerCnxn$Factory$1(81): Thread Thread[MASTER_CLOSEREGION-master-1,5,main] died
> java.lang.NullPointerException
>         at org.apache.hadoop.hbase.master.ServerManager.processRegionOpen(ServerManager.java:607)
>         at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.handleRegionOpenedEvent(MasterOpenRegionHandler.java:99)
>         at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.process(MasterOpenRegionHandler.java:75)
>         at org.apache.hadoop.hbase.executor.HBaseEventHandler.run(HBaseEventHandler.java:215)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:637)
> {code}
> This looks very new. Assigning Karthik as he was there recently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HBASE-2741) NPE in ServerManager when a region is closing

Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880361#action_12880361 ] 

Jean-Daniel Cryans commented on HBASE-2741:
-------------------------------------------

Debugging this with Karthik's help, we found out that the new HBaseExecutorService wasn't multi-cluster friendly because it was named "master", instead of using something less static like host:port. As a matter of fact, in my log I can also see:

{code}
2010-06-18 15:35:08,205 DEBUG [main] executor.HBaseExecutorService$HBaseExecutorServiceType(88): Executor service MASTER_CLOSEREGION already running on master
{code}

This was in fact detecting the other master's service.

> NPE in ServerManager when a region is closing
> ---------------------------------------------
>
>                 Key: HBASE-2741
>                 URL: https://issues.apache.org/jira/browse/HBASE-2741
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Jean-Daniel Cryans
>            Assignee: Karthik Ranganathan
>             Fix For: 0.21.0
>
>
> While running TestReplication I bumped into:
> {code}
> 2010-06-16 16:44:07,576 DEBUG [IPC Server handler 3 on 62423] master.RegionManager(357): Created UNASSIGNED zNode test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. in state M2ZK_REGION_OFFLINE
> 2010-06-16 16:44:07,577 INFO  [RegionServer:0] regionserver.HRegionServer(511): MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,577 INFO  [RegionServer:0.worker] regionserver.HRegionServer$Worker(1358): Worker: MSG_REGION_OPEN: test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,578 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENING] expected version = 0
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,580 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,581 DEBUG [RegionServer:0.worker] regionserver.HRegion(294): Creating region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870.
> 2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENING, region = de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,582 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(81): NO-OP call to handling region opening event
> 2010-06-16 16:44:07,589 INFO  [RegionServer:0.worker] regionserver.HRegion(369): region test,,1276731846828.de5dcd3df0fbc58207ce6ccff9ff2870. available; sequence id is 1
> 2010-06-16 16:44:07,590 DEBUG [RegionServer:0.worker] regionserver.RSZookeeperUpdater(157): Updating ZNode /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870 with [RS2ZK_REGION_OPENED] expected version = 1
> 2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.HMaster(1142): Event NodeDataChanged with state SyncConnected with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,591 DEBUG [main-EventThread] master.ZKMasterAddressWatcher(64): Got event NodeDataChanged with path /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,592 DEBUG [main-EventThread] master.ZKUnassignedWatcher(71): ZK-EVENT-PROCESS: Got zkEvent NodeDataChanged state:SyncConnected path:/1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,591 INFO  [main-EventThread] regionserver.HRegionServer(379): Got ZooKeeper event, state: SyncConnected, type: NodeDataChanged, path: /1/UNASSIGNED/de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,593 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(70): Event = RS2ZK_REGION_OPENED, region = de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,594 DEBUG [MASTER_CLOSEREGION-master-1] handler.MasterOpenRegionHandler(96): RS 10.10.1.130,62425,1276731832950 has opened region de5dcd3df0fbc58207ce6ccff9ff2870
> 2010-06-16 16:44:07,594 ERROR [MASTER_CLOSEREGION-master-1] server.NIOServerCnxn$Factory$1(81): Thread Thread[MASTER_CLOSEREGION-master-1,5,main] died
> java.lang.NullPointerException
>         at org.apache.hadoop.hbase.master.ServerManager.processRegionOpen(ServerManager.java:607)
>         at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.handleRegionOpenedEvent(MasterOpenRegionHandler.java:99)
>         at org.apache.hadoop.hbase.master.handler.MasterOpenRegionHandler.process(MasterOpenRegionHandler.java:75)
>         at org.apache.hadoop.hbase.executor.HBaseEventHandler.run(HBaseEventHandler.java:215)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:637)
> {code}
> This looks very new. Assigning Karthik as he was there recently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.