You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Anoop Sam John (Jira)" <ji...@apache.org> on 2021/07/29 05:21:00 UTC
[jira] [Assigned] (HBASE-26114) when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster cannot start normally
[ https://issues.apache.org/jira/browse/HBASE-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anoop Sam John reassigned HBASE-26114:
--------------------------------------
Assignee: Anoop Sam John
> when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster cannot start normally
> ---------------------------------------------------------------------------------------------------
>
> Key: HBASE-26114
> URL: https://issues.apache.org/jira/browse/HBASE-26114
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 2.2.0, 2.4.4
> Environment: HBase 2.2.2
> os.name=Linux
> os.arch=amd64
> os.version=5.4.0-72-generic
> java.version=1.8.0_191
> java.vendor=Oracle Corporation
> Reporter: Jingxuan Fu
> Assignee: Anoop Sam John
> Priority: Minor
> Labels: patch
> Attachments: HBASE-26114.000.patch
>
> Original Estimate: 10m
> Remaining Estimate: 10m
>
> In hbase-default.xml：
>
> {code:java}
> <property>
> <name>hbase.mob.compaction.threads.max</name>
> <value>1</value>
> <description>
> The max number of threads used in MobCompactor.
> </description>
> </property>{code}
>
> When the value is set to a negative number, such as -1, Hmaster cannot start normally.
> The log file will output:
>
> {code:cpp}
> 2021-07-22 18:54:13,758 ERROR [master/JavaFuzz:16000:becomeActiveMaster] master.HMaster: Failed to become active master java.lang.IllegalArgumentException
> at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1314) at org.apache.hadoop.hbase.mob.MobUtils.createMobCompactorThreadPool(MobUtils.java:880)
> at org.apache.hadoop.hbase.master.MobCompactionChore.<init>
> (MobCompactionChore.java:51) at org.apache.hadoop.hbase.master.HMaster.initMobCleaner(HMaster.java:1278)
> at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1161)
> at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2112)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
> at java.lang.Thread.run(Thread.java:748)
> 2021-07-22 18:54:13,760 ERROR [master/JavaFuzz:16000:becomeActiveMaster] master.HMaster: Master server abort: loaded coprocessors are: [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
> 2021-07-22 18:54:13,760 ERROR [master/JavaFuzz:16000:becomeActiveMaster] master.HMaster: ***** ABORTING master javafuzz,16000,1626951243154: Unhandled exception. Starting shutdown. ***** java.lang.IllegalArgumentException
> at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1314) at org.apache.hadoop.hbase.mob.MobUtils.createMobCompactorThreadPool(MobUtils.java:880)
> at org.apache.hadoop.hbase.master.MobCompactionChore.<init>(MobCompactionChore.java:51)
> at org.apache.hadoop.hbase.master.HMaster.initMobCleaner(HMaster.java:1278)
> at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1161)
> at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2112)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
> at java.lang.Thread.run(Thread.java:748)
> 2021-07-
> 22 18:54:13,760 INFO [master/JavaFuzz:16000:becomeActiveMaster] regionserver.HRegionServer: ***** STOPPING region server 'javafuzz,16000,1626951243154' *****{code}
>
> In MobUtils.java(package org.apache.hadoop.hbase.mob)
> This method from version 2.2.0 to version 2.4.4 is the same
> {code:java}
> public static ExecutorService createMobCompactorThreadPool(Configuration conf) { int maxThreads = conf.getInt(MobConstants.MOB_COMPACTION_THREADS_MAX, MobConstants.DEFAULT_MOB_COMPACTION_THREADS_MAX);
> if (maxThreads == 0) {
> maxThreads = 1;
> }
> final SynchronousQueue<Runnable> queue = new SynchronousQueue<>();
> ThreadPoolExecutor pool = new ThreadPoolExecutor(1, maxThreads, 60, TimeUnit.SECONDS, queue, Threads.newDaemonThreadFactory("MobCompactor"), new RejectedExecutionHandler() {
> @Override
> public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
> try {
> // waiting for a thread to pick up instead of throwing exceptions.
> queue.put(r);
> } catch (InterruptedException e) {
> throw new RejectedExecutionException(e);
> }
> }
> });
> ((ThreadPoolExecutor) pool).allowCoreThreadTimeOut(true);
> return pool;
> }{code}
> When MOB_COMPACTION_THREADS_MAX is set to 0, mobUtil will set it to 1. But the program does not take into account that it is set to a negative number. When it is set to a negative number, the initialization of the ThreadPoolExecutor will fail and an IllegalArgumentException will be thrown, making HMaster fail to start.
> Sometimes users will use -1 as the value of the default item in the configuration file.
> Therefore, it is best to modify the source code, when the value is negative, also set its _maxThread_ to 1.
> Only need to modify
> if (maxThreads == 0) {
> to
> if (maxThreads <= 0) {
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)