You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2021/07/18 08:23:00 UTC
[jira] [Commented] (IMPALA-9136) IllegalStateException in calling
Table.toThrift() without holding the table lock
[ https://issues.apache.org/jira/browse/IMPALA-9136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17382769#comment-17382769 ]
Quanlong Huang commented on IMPALA-9136:
----------------------------------------
To be specifit, the fix is
{code}
diff --git a/fe/src/main/java/org/apache/impala/catalog/CatalogObjectImpl.java b/fe/src/main/java/org/apache/impala/catalog/CatalogObjectImpl.java
index 20351df..89723c0 100644
--- a/fe/src/main/java/org/apache/impala/catalog/CatalogObjectImpl.java
+++ b/fe/src/main/java/org/apache/impala/catalog/CatalogObjectImpl.java
@@ -41,7 +41,7 @@ abstract public class CatalogObjectImpl implements CatalogObject {
public String getName() { return ""; }
@Override
- public final String getUniqueName() {
+ public String getUniqueName() {
return Catalog.toCatalogObjectKey(toTCatalogObject());
}
diff --git a/fe/src/main/java/org/apache/impala/catalog/Table.java b/fe/src/main/java/org/apache/impala/catalog/Table.java
index 87a6077..ea20241 100644
--- a/fe/src/main/java/org/apache/impala/catalog/Table.java
+++ b/fe/src/main/java/org/apache/impala/catalog/Table.java
@@ -509,6 +509,17 @@ public abstract class Table extends CatalogObjectImpl implements FeTable {
return catalogObject;
}
+ /**
+ * Override parent implementation that will finally call toThrift() which requires
+ * holding the table lock. However, it's not guaranteed that caller holds the table
+ * lock (IMPALA-9136). Here we use toMinimalTCatalogObject() directly since only db
+ * name and table name are needed.
+ */
+ @Override
+ public final String getUniqueName() {
+ return Catalog.toCatalogObjectKey(toMinimalTCatalogObject());
+ }
+
@Override
protected void setTCatalogObject(TCatalogObject catalogObject) {
catalogObject.setTable(toThrift());
{code}
> IllegalStateException in calling Table.toThrift() without holding the table lock
> --------------------------------------------------------------------------------
>
> Key: IMPALA-9136
> URL: https://issues.apache.org/jira/browse/IMPALA-9136
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.2.0, Impala 3.3.0
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
> Fix For: Impala 3.4.0
>
>
> In CatalogServiceCatalog.addTableToCatalogDelta(), we are calling tbl.getUniqueName() without holding the table lock, which could finally results in an IllegalStateException:
> {code}
> I1107 17:34:27.058262 6381 CatalogServiceCatalog.java:1118] Table test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_14_part is skipping topic update 423
> I1107 17:34:27.058709 6381 catalog-server.cc:641] Collected update: 1:TABLE:test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_15_part, version=422, original size=2299, compressed size=1219
> I1107 17:34:27.061233 6381 jni-util.cc:288] java.lang.IllegalStateException: Table.toThrift() called without holding the table lock: test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_12_part org.apache.impala.catalog.HdfsTable
> at org.apache.impala.catalog.Table.toThrift(Table.java:448)
> at org.apache.impala.catalog.HdfsTable.toThrift(HdfsTable.java:1408)
> at org.apache.impala.catalog.Table.setTCatalogObject(Table.java:487)
> at org.apache.impala.catalog.CatalogObjectImpl.toTCatalogObject(CatalogObjectImpl.java:51)
> at org.apache.impala.catalog.CatalogObjectImpl.getUniqueName(CatalogObjectImpl.java:45)
> at org.apache.impala.catalog.CatalogServiceCatalog.addTableToCatalogDelta(CatalogServiceCatalog.java:1084)
> at org.apache.impala.catalog.CatalogServiceCatalog.addDatabaseToCatalogDelta(CatalogServiceCatalog.java:961)
> at org.apache.impala.catalog.CatalogServiceCatalog.getCatalogDelta(CatalogServiceCatalog.java:710)
> at org.apache.impala.service.JniCatalog.getCatalogDelta(JniCatalog.java:148)
> I1107 17:34:27.129712 7356 HdfsTable.java:934] Reloading metadata for all partition(s) of test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_16 (INSERT)
> I1107 17:34:27.132397 7356 HdfsTable.java:630] Loaded file and block metadata for test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_16 partitions:
> I1107 17:34:27.132424 7356 HdfsTable.java:965] Incrementally loaded table metadata for: test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_16
> I1107 17:34:27.157740 6381 status.cc:126] IllegalStateException: Table.toThrift() called without holding the table lock: test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_12_part org.apache.impala.catalog.HdfsTable
> @ 0x1c5ae50 impala::Status::Status()
> @ 0x24f7ad2 impala::JniUtil::GetJniExceptionMsg()
> @ 0x1c414af impala::JniCall::Call<>()
> @ 0x1c3fe27 impala::JniUtil::CallJniMethod<>()
> @ 0x1c3e06d impala::Catalog::GetCatalogDelta()
> @ 0x1c14344 impala::CatalogServer::GatherCatalogUpdatesThread()
> @ 0x1c2e14e boost::_mfi::mf0<>::operator()()
> @ 0x1c2da93 boost::_bi::list1<>::operator()<>()
> @ 0x1c2d4a6 boost::_bi::bind_t<>::operator()()
> @ 0x1c2cd91 boost::detail::function::void_function_obj_invoker0<>::invoke()
> @ 0x1ffb6e9 boost::function0<>::operator()()
> @ 0x2573dea impala::Thread::SuperviseThread()
> @ 0x257c16e boost::_bi::list5<>::operator()<>()
> @ 0x257c092 boost::_bi::bind_t<>::operator()()
> @ 0x257c055 boost::detail::thread_data<>::run()
> @ 0x3d61599 thread_proxy
> @ 0x7f1ce6ca46b9 start_thread
> @ 0x7f1ce343f41c clone
> E1107 17:34:27.157758 6381 catalog-server.cc:379] IllegalStateException: Table.toThrift() called without holding the table lock: test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_12_part org.apache.impala.catalog.HdfsTable
> I1107 17:34:27.158537 6389 catalog-server.cc:337] A catalog update with 4 entries is assembled. Catalog version: 419 Last sent catalog version: 419
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org