You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Kevin Geiszler (Jira)" <ji...@apache.org> on 2026/03/20 23:31:00 UTC
[jira] [Updated] (HBASE-30014) refresh_meta not working due to regionNode lock
[ https://issues.apache.org/jira/browse/HBASE-30014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Geiszler updated HBASE-30014:
-----------------------------------
Description:
There is an issue in our new "HBASE-29081_rebased" branch after rebasing with the master branch. I am seeing the following being repeated in master log after running {{refresh_meta}} on the replica cluster:
{code:java}
2026-03-20T22:13:37,693 WARN [RpcServer.priority.RWQ.Fifo.write.handler=2,queue=0,port=16000] assignment.AssignmentManager: Unable to acquire lock for regionNode state=OPEN, location=hbase-docker-2,16020,1774044759623, table=test_table, region=4f345a940e95b6eba65d80f1ec34d71c. It is likely that another thread is currently holding the lock. To avoid deadlock, skip execution for now.{code}
I see this on the read-replica docker container setup. The steps to reproduce are:
# Start the hbase-docker container as the active cluster and hbase-docker-2 as the replica cluster
# On the active cluster, use the HBase shell to create a table
# On the replica cluster, run {{{}refresh_meta{}}}. You will then see the "Unable to acquire lock for regionNode..." log message repeated in {{{}/opt/hbase/logs/hbase-jboss-master-hbase-docker-2.log{}}}. Also, after running {{{}refresh_meta{}}}, you will not see the table that was created on the active cluster.
I have attached a log from both the working branch (HBASE-29081) and the branch where it does not work (HBASE-29081_rebased). Search the logs for "Submitting RefreshMetaProcedure" to see where the {{refresh_meta}} procedure began.
was:
There is an issue in our new HBASE-29081_rebased branch after rebasing with the master branch. I am seeing the following being repeated in master log after running {{refresh_meta}} on the replica cluster:
{code:java}
2026-03-20T22:13:37,693 WARN [RpcServer.priority.RWQ.Fifo.write.handler=2,queue=0,port=16000] assignment.AssignmentManager: Unable to acquire lock for regionNode state=OPEN, location=hbase-docker-2,16020,1774044759623, table=test_table, region=4f345a940e95b6eba65d80f1ec34d71c. It is likely that another thread is currently holding the lock. To avoid deadlock, skip execution for now.{code}
I see this on the read-replica docker container setup. The steps to reproduce are:
# Start the hbase-docker container as the active cluster and hbase-docker-2 as the replica cluster
# On the active cluster, use the HBase shell to create a table
# On the replica cluster, run {{{}refresh_meta{}}}. You will then see the "Unable to acquire lock for regionNode..." log message repeated in {{{}/opt/hbase/logs/hbase-jboss-master-hbase-docker-2.log{}}}. Also, after running {{{}refresh_meta{}}}, you will not see the table that was created on the active cluster.
I have attached a log from both the working branch (HBASE-29081) and the branch where it does not work (HBASE-29081_rebased). Search the logs for "Submitting RefreshMetaProcedure" to see where the {{refresh_meta}} procedure began.
> refresh_meta not working due to regionNode lock
> -----------------------------------------------
>
> Key: HBASE-30014
> URL: https://issues.apache.org/jira/browse/HBASE-30014
> Project: HBase
> Issue Type: Sub-task
> Components: read replicas, Replication
> Reporter: Kevin Geiszler
> Priority: Major
> Attachments: unable-to-acquire-lock-for-regionMode-replica-master.log, working-refresh-meta-master-replica.log
>
>
> There is an issue in our new "HBASE-29081_rebased" branch after rebasing with the master branch. I am seeing the following being repeated in master log after running {{refresh_meta}} on the replica cluster:
> {code:java}
> 2026-03-20T22:13:37,693 WARN [RpcServer.priority.RWQ.Fifo.write.handler=2,queue=0,port=16000] assignment.AssignmentManager: Unable to acquire lock for regionNode state=OPEN, location=hbase-docker-2,16020,1774044759623, table=test_table, region=4f345a940e95b6eba65d80f1ec34d71c. It is likely that another thread is currently holding the lock. To avoid deadlock, skip execution for now.{code}
> I see this on the read-replica docker container setup. The steps to reproduce are:
> # Start the hbase-docker container as the active cluster and hbase-docker-2 as the replica cluster
> # On the active cluster, use the HBase shell to create a table
> # On the replica cluster, run {{{}refresh_meta{}}}. You will then see the "Unable to acquire lock for regionNode..." log message repeated in {{{}/opt/hbase/logs/hbase-jboss-master-hbase-docker-2.log{}}}. Also, after running {{{}refresh_meta{}}}, you will not see the table that was created on the active cluster.
> I have attached a log from both the working branch (HBASE-29081) and the branch where it does not work (HBASE-29081_rebased). Search the logs for "Submitting RefreshMetaProcedure" to see where the {{refresh_meta}} procedure began.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)