You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "Kevin Verhoeven (JIRA)" <ji...@apache.org> on 2014/08/26 20:19:58 UTC
[jira] [Commented] (PHOENIX-1148) Somewhat misleading error message
when SYSTEM.CATALOG is disabled
[ https://issues.apache.org/jira/browse/PHOENIX-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14111083#comment-14111083 ]
Kevin Verhoeven commented on PHOENIX-1148:
------------------------------------------
I received the same error after I lost a RegionServer unexpectedly. The SYSTEM.CATALOG table did not transfer to another RegionServer even after restarting the HBase Master. I fixed this problem by running hbck against a working node. For example: sudo -u hbase hbase hbck -fix
> Somewhat misleading error message when SYSTEM.CATALOG is disabled
> -----------------------------------------------------------------
>
> Key: PHOENIX-1148
> URL: https://issues.apache.org/jira/browse/PHOENIX-1148
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.0.0
> Reporter: Carter Shanklin
> Priority: Minor
>
> I was helping someone today whose HBase environment was a bit messed up. Somehow the system tables had gotten disabled. The error message threw us off enough that I think it should be improved. The main problem is that this message appears in red text, calling the most attention to itself:
> {code}
> Error: ERROR 2006 (INT08): Incompatible jars detected between client and server. Ensure that phoenix.jar is put on the classpath of HBase in every region server: SYSTEM.CATALOG is disabled. (state=INT08,code=2006)
> {code}
> Based on this message we started checking for duplicate JARs, etc. Now admittedly the real problem is stated there along with the misleading error message but it was enough to throw us off for a while.
> Reproduction is easy: disable your SYSTEM.CATALOG and connect using sqlline.
> Full traceback is attached, for the curious:
> {code}
> org.apache.hadoop.hbase.TableNotEnabledException: SYSTEM.CATALOG is disabled.
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1094)
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1084)
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:904)
> at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:78)
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:120)
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:97)
> at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:90)
> at org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:51)
> at org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService$Stub.getVersion(MetaDataProtos.java:6805)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl$3.call(ConnectionQueryServicesImpl.java:909)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl$3.call(ConnectionQueryServicesImpl.java:901)
> at org.apache.hadoop.hbase.client.HTable$16.call(HTable.java:1586)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 14/07/15 07:41:37 WARN client.HTable: Error calling coprocessor service org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService for row
> java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.TableNotEnabledException: SYSTEM.CATALOG is disabled.
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at org.apache.hadoop.hbase.client.HTable.coprocessorService(HTable.java:1598)
> at org.apache.hadoop.hbase.client.HTable.coprocessorService(HTable.java:1554)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl.checkClientServerCompatibility(ConnectionQueryServicesImpl.java:900)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:825)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl.createTable(ConnectionQueryServicesImpl.java:1058)
> at org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:1156)
> at org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:422)
> at org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:183)
> at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:226)
> at org.apache.phoenix.jdbc.PhoenixStatement.executeUpdate(PhoenixStatement.java:908)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1453)
> at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:131)
> at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:112)
> at sqlline.SqlLine$DatabaseConnection.connect(SqlLine.java:4650)
> at sqlline.SqlLine$DatabaseConnection.getConnection(SqlLine.java:4701)
> at sqlline.SqlLine$Commands.connect(SqlLine.java:3942)
> at sqlline.SqlLine$Commands.connect(SqlLine.java:3851)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sqlline.SqlLine$ReflectiveCommandHandler.execute(SqlLine.java:2810)
> at sqlline.SqlLine.dispatch(SqlLine.java:817)
> at sqlline.SqlLine.initArgs(SqlLine.java:633)
> at sqlline.SqlLine.begin(SqlLine.java:680)
> at sqlline.SqlLine.mainWithInputRedirection(SqlLine.java:441)
> at sqlline.SqlLine.main(SqlLine.java:424)
> Caused by: org.apache.hadoop.hbase.TableNotEnabledException: SYSTEM.CATALOG is disabled.
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1094)
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1084)
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:904)
> at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:78)
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:120)
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:97)
> at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:90)
> at org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:51)
> at org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService$Stub.getVersion(MetaDataProtos.java:6805)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl$3.call(ConnectionQueryServicesImpl.java:909)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl$3.call(ConnectionQueryServicesImpl.java:901)
> at org.apache.hadoop.hbase.client.HTable$16.call(HTable.java:1586)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Error: ERROR 2006 (INT08): Incompatible jars detected between client and server. Ensure that phoenix.jar is put on the classpath of HBase in every region server: SYSTEM.CATALOG is disabled. (state=INT08,code=2006)
> java.sql.SQLException: ERROR 2006 (INT08): Incompatible jars detected between client and server. Ensure that phoenix.jar is put on the classpath of HBase in every region server: SYSTEM.CATALOG is disabled.
> at org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:309)
> at org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:133)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl.checkClientServerCompatibility(ConnectionQueryServicesImpl.java:932)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:825)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl.createTable(ConnectionQueryServicesImpl.java:1058)
> at org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:1156)
> at org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:422)
> at org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:183)
> at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:226)
> at org.apache.phoenix.jdbc.PhoenixStatement.executeUpdate(PhoenixStatement.java:908)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1453)
> at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:131)
> at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:112)
> at sqlline.SqlLine$DatabaseConnection.connect(SqlLine.java:4650)
> at sqlline.SqlLine$DatabaseConnection.getConnection(SqlLine.java:4701)
> at sqlline.SqlLine$Commands.connect(SqlLine.java:3942)
> at sqlline.SqlLine$Commands.connect(SqlLine.java:3851)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sqlline.SqlLine$ReflectiveCommandHandler.execute(SqlLine.java:2810)
> at sqlline.SqlLine.dispatch(SqlLine.java:817)
> at sqlline.SqlLine.initArgs(SqlLine.java:633)
> at sqlline.SqlLine.begin(SqlLine.java:680)
> at sqlline.SqlLine.mainWithInputRedirection(SqlLine.java:441)
> at sqlline.SqlLine.main(SqlLine.java:424)
> Caused by: org.apache.hadoop.hbase.TableNotEnabledException: SYSTEM.CATALOG is disabled.
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1094)
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1084)
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:904)
> at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:78)
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:120)
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:97)
> at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:90)
> at org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:51)
> at org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService$Stub.getVersion(MetaDataProtos.java:6805)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl$3.call(ConnectionQueryServicesImpl.java:909)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl$3.call(ConnectionQueryServicesImpl.java:901)
> at org.apache.hadoop.hbase.client.HTable$16.call(HTable.java:1586)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)