You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Robbie Zhang (JIRA)" <ji...@apache.org> on 2019/06/05 06:17:00 UTC
[jira] [Commented] (IMPALA-4089) missing thrift function validation
checks
[ https://issues.apache.org/jira/browse/IMPALA-4089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16856388#comment-16856388 ]
Robbie Zhang commented on IMPALA-4089:
--------------------------------------
An empty catalog could cause this exception. Here is what I saw on a cluster:
{code:java}
E0604 07:59:21.678375 40945 MetaStoreUtils.java:1234] Got exception: org.apache.hadoop.hive.metastore.api.MetaException Could not retrieve transation read-only status server
Java exception follows:
MetaException(message:Could not retrieve transation read-only status server)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_all_databases_result$get_all_databases_resultStandardScheme.read(ThriftHiveMetastore.java:18313)
...
at org.apache.impala.service.JniCatalog.<init>(JniCatalog.java:108)
E0604 07:59:21.678470 40945 MetaStoreUtils.java:1235] Converting exception to MetaException
E0604 07:59:21.678840 40945 CatalogServiceCatalog.java:702] MetaException(message:Got exception: org.apache.hadoop.hive.metastore.api.MetaException Could not retrieve transation read-only status server)
E0604 07:59:21.679153 40945 JniCatalog.java:110] Error initializing Catalog. Please run 'invalidate metadata'
Java exception follows:
org.apache.impala.catalog.CatalogException: Error initializing Catalog. Catalog may be empty.
at org.apache.impala.catalog.CatalogServiceCatalog.reset(CatalogServiceCatalog.java:703)
at org.apache.impala.service.JniCatalog.<init>(JniCatalog.java:108)
Caused by: MetaException(message:Got exception: org.apache.hadoop.hive.metastore.api.MetaException Could not retrieve transation read-only status server)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:1236)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:1055)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:101)
at com.sun.proxy.$Proxy5.getAllDatabases(Unknown Source)
at org.apache.impala.catalog.CatalogServiceCatalog.reset(CatalogServiceCatalog.java:686)
... 1 more
...
I0604 07:59:21.718371 41215 jni-util.cc:169] org.apache.thrift.protocol.TProtocolException: Required field 'update_fn_symbol' was not present! Struct: TAggregateFunction(intermediate_type:TColumnType(types:[TTypeNode(type:SCALAR, scalar_type:TScalarType(type:STRING))]), update_fn_symbol:null, init_fn_symbol:null, ignores_distinct:false)
at org.apache.impala.thrift.TAggregateFunction.validate(TAggregateFunction.java:948)
at org.apache.impala.thrift.TFunction.validate(TFunction.java:1164)
at org.apache.impala.thrift.TCatalogObject.validate(TCatalogObject.java:1058)
at org.apache.impala.thrift.TCatalogObject$TCatalogObjectStandardScheme.write(TCatalogObject.java:1213)
at org.apache.impala.thrift.TCatalogObject$TCatalogObjectStandardScheme.write(TCatalogObject.java:1098)
at org.apache.impala.thrift.TCatalogObject.write(TCatalogObject.java:938)
at org.apache.impala.thrift.TGetAllCatalogObjectsResponse$TGetAllCatalogObjectsResponseStandardScheme.write(TGetAllCatalogObjectsResponse.java:487)
at org.apache.impala.thrift.TGetAllCatalogObjectsResponse$TGetAllCatalogObjectsResponseStandardScheme.write(TGetAllCatalogObjectsResponse.java:421)
at org.apache.impala.thrift.TGetAllCatalogObjectsResponse.write(TGetAllCatalogObjectsResponse.java:365)
at org.apache.thrift.TSerializer.serialize(TSerializer.java:79)
at org.apache.impala.service.JniCatalog.getCatalogObjects(JniCatalog.java:124)
{code}
Here is the error from HMS log file:
{code:java}
2019-06-04 07:59:21,670 WARN org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-6-thread-49]: Database initialization failed; direct SQL is disabled
javax.jdo.JDODataStoreException: Could not retrieve transation read-only status server
at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
...
java.sql.SQLException: Could not retrieve transation read-only status server
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:998)
...
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 561,221 milliseconds ago. The last packet sent successfully to the server was 561,221 milliseconds ago.
at sun.reflect.GeneratedConstructorAccessor29.newInstance(Unknown Source)
...
Caused by: java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:115)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3591)
... 50 more
{code}
> missing thrift function validation checks
> -----------------------------------------
>
> Key: IMPALA-4089
> URL: https://issues.apache.org/jira/browse/IMPALA-4089
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 2.7.0
> Reporter: Huaisi Xu
> Priority: Major
>
> org.apache.thrift.protocol.TProtocolException: Required field 'update_fn_symbol' was not present! Struct: TAggregateFunction(intermediate_type:TColumnType(types:[TTypeNode(type:SCALAR, scalar_type:TScalarType(type:BOOLEAN))]), update_fn_symbol:null, init_fn_symbol:null, ignores_distinct:false)
> at com.cloudera.impala.thrift.TAggregateFunction.validate(TAggregateFunction.java:948)
> at com.cloudera.impala.thrift.TFunction.validate(TFunction.java:1164)
> at com.cloudera.impala.thrift.TCatalogObject.validate(TCatalogObject.java:1058)
> at com.cloudera.impala.thrift.TCatalogObject$TCatalogObjectStandardScheme.write(TCatalogObject.java:1213)
> at com.cloudera.impala.thrift.TCatalogObject$TCatalogObjectStandardScheme.write(TCatalogObject.java:1098)
> at com.cloudera.impala.thrift.TCatalogObject.write(TCatalogObject.java:938)
> at com.cloudera.impala.thrift.TGetAllCatalogObjectsResponse$TGetAllCatalogObjectsResponseStandardScheme.write(TGetAllCatalogObjectsResponse.java:487)
> at com.cloudera.impala.thrift.TGetAllCatalogObjectsResponse$TGetAllCatalogObjectsResponseStandardScheme.write(TGetAllCatalogObjectsResponse.java:421)
> at com.cloudera.impala.thrift.TGetAllCatalogObjectsResponse.write(TGetAllCatalogObjectsResponse.java:365)
> at org.apache.thrift.TSerializer.serialize(TSerializer.java:79)
> at com.cloudera.impala.service.JniCatalog.getCatalogObjects(JniCatalog.java:113)
> not sure why we get a corrupted UDA from hms though, but we should probably do validate() in all fromThrift() and toThrift().
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org