You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/10/17 17:04:00 UTC
[jira] [Commented] (IMPALA-7669) Concurrent invalidate with compute
(or drop) stats throws NPE.
[ https://issues.apache.org/jira/browse/IMPALA-7669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653855#comment-16653855 ]
ASF subversion and git services commented on IMPALA-7669:
---------------------------------------------------------
Commit 2b2cf8d96617320d184d070f9319c2463aa0d84f in impala's branch refs/heads/master from Bharath Vissapragada
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=2b2cf8d ]
IMPALA-7669: Gracefully handle concurrent invalidate/partial fetch RPCs
The bug here was that any partial RPC on an IncompleteTable was throwing
an NPE.
Ideally, we attempt to load the table (if we find that it is not loaded)
before making the partial info request, but a concurrent invalidate could
reset the table state and move it back to an uninitialized state.
This patch handles this case better by propagating a meaningful error to
the caller.
Testing:
-------
- Added a test that fails consistently with an NPE without this patch.
Change-Id: I8533f73f25ca42a20f146ddfd95d4213add9b705
Reviewed-on: http://gerrit.cloudera.org:8080/11638
Reviewed-by: Bharath Vissapragada <bh...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Concurrent invalidate with compute (or drop) stats throws NPE.
> --------------------------------------------------------------
>
> Key: IMPALA-7669
> URL: https://issues.apache.org/jira/browse/IMPALA-7669
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.1.0
> Reporter: bharath v
> Assignee: bharath v
> Priority: Critical
>
> *This is a Catalog V2 only bug*
> NPE is thrown when trying to getPartialInfo() from an IncompleteTable (result of ivalidate) and cause_ is null.
> {noformat}
> @Override
> public TGetPartialCatalogObjectResponse getPartialInfo(
> TGetPartialCatalogObjectRequest req) throws TableLoadingException {
> Throwables.propagateIfPossible(cause_, TableLoadingException.class);
> throw new TableLoadingException(cause_.getMessage()); <-----
> }
> {noformat}
> {noformat}
> I1004 16:51:28.845305 85380 jni-util.cc:308] java.lang.NullPointerException
> at org.apache.impala.catalog.IncompleteTable.getPartialInfo(IncompleteTable.java:140)
> at org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:2171)
> at org.apache.impala.service.JniCatalog.getPartialCatalogObject(JniCatalog.java:236)
> {noformat}
> Actual caller stack trace is this.
> {noformat}
> I1004 16:51:21.666422 67179 Frontend.java:1086] Analyzing query: compute stats ads
> I1004 16:51:28.850023 67179 jni-util.cc:308] org.apache.impala.catalog.local.LocalCatalogException: Could not load table parnal.ads from metastore
> at org.apache.impala.catalog.local.LocalTable.loadTableMetadata(LocalTable.java:128)
> at org.apache.impala.catalog.local.LocalTable.load(LocalTable.java:89)
> at org.apache.impala.catalog.local.LocalDb.getTable(LocalDb.java:119)
> at org.apache.impala.analysis.StmtMetadataLoader.getMissingTables(StmtMetadataLoader.java:251)
> at org.apache.impala.analysis.StmtMetadataLoader.loadTables(StmtMetadataLoader.java:140)
> at org.apache.impala.analysis.StmtMetadataLoader.loadTables(StmtMetadataLoader.java:116)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1118)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1092)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1064)
> at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:158)
> Caused by: org.apache.thrift.TException: TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL, error_msgs:[NullPointerException: null]), lookup_status:OK)
> at org.apache.impala.catalog.local.CatalogdMetaProvider.sendRequest(CatalogdMetaProvider.java:354)
> at org.apache.impala.catalog.local.CatalogdMetaProvider.access$100(CatalogdMetaProvider.java:163)
> at org.apache.impala.catalog.local.CatalogdMetaProvider$5.call(CatalogdMetaProvider.java:565)
> at org.apache.impala.catalog.local.CatalogdMetaProvider$5.call(CatalogdMetaProvider.java:560)
> at org.apache.impala.catalog.local.CatalogdMetaProvider$1.call(CatalogdMetaProvider.java:411)
> at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767)
> at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
> at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
> at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
> at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)
> at org.apache.impala.catalog.local.CatalogdMetaProvider.loadWithCaching(CatalogdMetaProvider.java:407)
> at org.apache.impala.catalog.local.CatalogdMetaProvider.loadTable(CatalogdMetaProvider.java:556)
> at org.apache.impala.catalog.local.LocalTable.loadTableMetadata(LocalTable.java:126)
> ... 9 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org