[jira] [Created] (HBASE-19804) [hbase-indexer] Metrics source RegionServer,sub=Server already exists!

["stack (JIRA)" <ji...@apache.org>]

stack created HBASE-19804:
-----------------------------

             Summary: [hbase-indexer] Metrics source RegionServer,sub=Server already exists!
                 Key: HBASE-19804
                 URL: https://issues.apache.org/jira/browse/HBASE-19804
             Project: HBase
          Issue Type: Improvement
          Components: hbase-indexer
    Affects Versions: 2.0.0-beta-1
            Reporter: stack
            Assignee: stack
             Fix For: 2.0.0-beta-2


In the past, the hbase-indexer runs multiple RegionServers per JVM. In old days, they had their own cut-down "RegionServer". In 2.0.0, we made it so they could run an actual RegionServer but with services disabled. The latter has an issue if you run more than one instance per JVM and it is NOT a minihbasecluster instance. It fails with:
{code:java}
1:09:13.371 PM  ERROR  HRegionServer  
Failed init
org.apache.hadoop.metrics2.MetricsException: Metrics source RegionServer,sub=Server already exists!
  at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:152)
  at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:125)
  at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:229)
  at org.apache.hadoop.hbase.metrics.BaseSourceImpl.<init>(BaseSourceImpl.java:115)
  at org.apache.hadoop.hbase.regionserver.MetricsRegionServerSourceImpl.<init>(MetricsRegionServerSourceImpl.java:101)
  at org.apache.hadoop.hbase.regionserver.MetricsRegionServerSourceImpl.<init>(MetricsRegionServerSourceImpl.java:93)
  at org.apache.hadoop.hbase.regionserver.MetricsRegionServerSourceFactoryImpl.createServer(MetricsRegionServerSourceFactoryImpl.java:69)
  at org.apache.hadoop.hbase.regionserver.MetricsRegionServer.<init>(MetricsRegionServer.java:56)
  at org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1519)
  at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:954)
  at com.ngdata.sep.impl.SepConsumer$1.run(SepConsumer.java:203){code}
 

If you look in [10:26 AM] Wolfgang Hoschek: DefaultMetricsSystem code (found by [~whoschek]), you'll see this:
{code:java}
synchronized ObjectName newObjectName(String name) {
    try {
      if (mBeanNames.map.containsKey(name) && !miniClusterMode) {
        throw new MetricsException(name +" already exists!");
      }
      return new ObjectName(mBeanNames.uniqueName(name));
    } catch (Exception e) {
      throw new MetricsException(e);
    }
  }{code}
i.e. if we are in a mini cluster context, we will not fail registering the second bean instance.

 

If you look in master startup in HMasterCommandLine, you will see:

 
{code:java}
// If 'local', defer to LocalHBaseCluster instance.  Starts master
// and regionserver both in the one JVM.
if (LocalHBaseCluster.isLocal(conf)) {
  DefaultMetricsSystem.setMiniClusterMode(true);
....{code}
... will ensure we don't get the above exception in minihbasecluster context.

 

So, the idea here is to make it so being able to run more than one RS per JVM is cleaner than doing the above hack. It needs to be a config too.... a config. which says don't fail startup if second mbean registration just because two RS in the one context (A later issue will be the accounting of metrics per RS... If more than one RS, then we should make a unique mbean per RS in the JVM).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)