You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Sai Hemanth Gantasala (Jira)" <ji...@apache.org> on 2023/06/13 19:30:00 UTC
[jira] [Updated] (IMPALA-12210) In FENG mode, catalogd.INFO shows get_table_req() forwarded to HMS and an exception
[ https://issues.apache.org/jira/browse/IMPALA-12210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sai Hemanth Gantasala updated IMPALA-12210:
-------------------------------------------
Description:
In FENG mode, running queries on external tables shows an exception in the catalogd.INFO log and catalogd forwards the get_table_req() requests to HMS (this is on the second execution of the query):
{noformat}
0: jdbc:hive2://localhost:11050/default> select * from ext_tbl_t1;
{noformat}
{noformat}
I0503 15:12:54.733690 15625 CatalogMetastoreServer.java:215] Invoking HMS API: open_txns
I0503 15:12:54.746543 15625 CatalogMetastoreServer.java:215] Invoking HMS API: get_open_txns_req
I0503 15:12:54.753791 15625 CatalogMetastoreServer.java:215] Invoking HMS API: get_valid_write_ids
I0503 15:12:54.757498 15625 CatalogMetastoreServer.java:215] Invoking HMS API: get_table_req
I0503 15:12:54.757787 15625 MetastoreServiceHandler.java:1506] Received exception while executing get_table_req
Java exception follows:
java.lang.IllegalStateException: Compaction id check cannot be done for non-transactional table foo.ext_tbl_t1 at com.google.common.base.Preconditions.checkState(Preconditions.java:508)
at org.apache.impala.catalog.CatalogServiceCatalog.getOrLoadTable(CatalogServiceCatalog.java:2247)
at org.apache.impala.catalog.CatalogServiceCatalog.doGetPartialCatalogObject(CatalogServiceCatalog.java:3537)
at org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:3463)
at org.apache.impala.catalog.CatalogHmsAPIHelper.getPartialCatalogObjResponse(CatalogHmsAPIHelper.java:216)
at org.apache.impala.catalog.CatalogHmsAPIHelper.getTableReq(CatalogHmsAPIHelper.java:152)
at org.apache.impala.catalog.metastore.CatalogMetastoreServiceHandler.get_table_req(CatalogMetastoreServiceHandler.java:114)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.impala.catalog.metastore.CatalogMetastoreServer$TimingInvocationHandler.invoke(CatalogMetastoreServer.java:216)
at com.sun.proxy.$Proxy13.get_table_req(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:20036)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:20015)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
I0503 15:12:54.757825 15625 MetastoreServiceHandler.java:897] Forwarding the request get_table_req for table db1.tt3 to the backing HiveMetastore service
{noformat}
For external tables, we don't need to do the compaction id check. Also, after the first execution, the expectation is that the table metadata is cached in the catalogd so the get_table_req() should ideally be satisfied from the cached entries instead of being forwarded to HMS.
was:
In Unified Analytics, running queries on external tables shows an exception in the catalogd.INFO log and catalogd forwards the get_table_req() requests to HMS (this is on the second execution of the query):
{noformat}
0: jdbc:hive2://localhost:11050/default> select * from ext_tbl_t1;
{noformat}
{noformat}
I0503 15:12:54.733690 15625 CatalogMetastoreServer.java:215] Invoking HMS API: open_txns
I0503 15:12:54.746543 15625 CatalogMetastoreServer.java:215] Invoking HMS API: get_open_txns_req
I0503 15:12:54.753791 15625 CatalogMetastoreServer.java:215] Invoking HMS API: get_valid_write_ids
I0503 15:12:54.757498 15625 CatalogMetastoreServer.java:215] Invoking HMS API: get_table_req
I0503 15:12:54.757787 15625 MetastoreServiceHandler.java:1506] Received exception while executing get_table_req
Java exception follows:
java.lang.IllegalStateException: Compaction id check cannot be done for non-transactional table foo.ext_tbl_t1 at com.google.common.base.Preconditions.checkState(Preconditions.java:508)
at org.apache.impala.catalog.CatalogServiceCatalog.getOrLoadTable(CatalogServiceCatalog.java:2247)
at org.apache.impala.catalog.CatalogServiceCatalog.doGetPartialCatalogObject(CatalogServiceCatalog.java:3537)
at org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:3463)
at org.apache.impala.catalog.CatalogHmsAPIHelper.getPartialCatalogObjResponse(CatalogHmsAPIHelper.java:216)
at org.apache.impala.catalog.CatalogHmsAPIHelper.getTableReq(CatalogHmsAPIHelper.java:152)
at org.apache.impala.catalog.metastore.CatalogMetastoreServiceHandler.get_table_req(CatalogMetastoreServiceHandler.java:114)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.impala.catalog.metastore.CatalogMetastoreServer$TimingInvocationHandler.invoke(CatalogMetastoreServer.java:216)
at com.sun.proxy.$Proxy13.get_table_req(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:20036)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:20015)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
I0503 15:12:54.757825 15625 MetastoreServiceHandler.java:897] Forwarding the request get_table_req for table db1.tt3 to the backing HiveMetastore service
{noformat}
For external tables, we don't need to do the compaction id check. Also, after the first execution, the expectation is that the table metadata is cached in the catalogd so the get_table_req() should ideally be satisfied from the cached entries instead of being forwarded to HMS.
> In FENG mode, catalogd.INFO shows get_table_req() forwarded to HMS and an exception
> -----------------------------------------------------------------------------------
>
> Key: IMPALA-12210
> URL: https://issues.apache.org/jira/browse/IMPALA-12210
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Sai Hemanth Gantasala
> Assignee: Sai Hemanth Gantasala
> Priority: Major
>
> In FENG mode, running queries on external tables shows an exception in the catalogd.INFO log and catalogd forwards the get_table_req() requests to HMS (this is on the second execution of the query):
> {noformat}
> 0: jdbc:hive2://localhost:11050/default> select * from ext_tbl_t1;
> {noformat}
> {noformat}
> I0503 15:12:54.733690 15625 CatalogMetastoreServer.java:215] Invoking HMS API: open_txns
> I0503 15:12:54.746543 15625 CatalogMetastoreServer.java:215] Invoking HMS API: get_open_txns_req
> I0503 15:12:54.753791 15625 CatalogMetastoreServer.java:215] Invoking HMS API: get_valid_write_ids
> I0503 15:12:54.757498 15625 CatalogMetastoreServer.java:215] Invoking HMS API: get_table_req
> I0503 15:12:54.757787 15625 MetastoreServiceHandler.java:1506] Received exception while executing get_table_req
> Java exception follows:
> java.lang.IllegalStateException: Compaction id check cannot be done for non-transactional table foo.ext_tbl_t1 at com.google.common.base.Preconditions.checkState(Preconditions.java:508)
> at org.apache.impala.catalog.CatalogServiceCatalog.getOrLoadTable(CatalogServiceCatalog.java:2247)
> at org.apache.impala.catalog.CatalogServiceCatalog.doGetPartialCatalogObject(CatalogServiceCatalog.java:3537)
> at org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:3463)
> at org.apache.impala.catalog.CatalogHmsAPIHelper.getPartialCatalogObjResponse(CatalogHmsAPIHelper.java:216)
> at org.apache.impala.catalog.CatalogHmsAPIHelper.getTableReq(CatalogHmsAPIHelper.java:152)
> at org.apache.impala.catalog.metastore.CatalogMetastoreServiceHandler.get_table_req(CatalogMetastoreServiceHandler.java:114)
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.impala.catalog.metastore.CatalogMetastoreServer$TimingInvocationHandler.invoke(CatalogMetastoreServer.java:216)
> at com.sun.proxy.$Proxy13.get_table_req(Unknown Source)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:20036)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:20015)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> I0503 15:12:54.757825 15625 MetastoreServiceHandler.java:897] Forwarding the request get_table_req for table db1.tt3 to the backing HiveMetastore service
> {noformat}
> For external tables, we don't need to do the compaction id check. Also, after the first execution, the expectation is that the table metadata is cached in the catalogd so the get_table_req() should ideally be satisfied from the cached entries instead of being forwarded to HMS.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org