[jira] [Comment Edited] (HIVE-22410) CachedStore Prewarm Failure NPE

["fanshilun (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-22410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964567#comment-16964567 ] 

fanshilun edited comment on HIVE-22410 at 11/1/19 3:14 AM:
-----------------------------------------------------------

[~samuelan]
The problem does not appear in the Sql query, but appears in the HMS startup warm-up，I will give some detailed instructions.
1.The problem occurs in the hot load function prewarm (RawStore rawStore) of the CachedStore. After the DataBase is loaded, the table information loading starts.
2.sharedCache.populateTableInCache(table, tableColStats...) CachedStore 559 Lines Throw NPE.
3.Method populateTableInCache calculates the memory of the object, and throws NPE when performing OBJECT_ARRAY calculation.
 !image-2019-11-01-11-14-42-114.png! 



was (Author: slfan1989):
[~samuelan]
The problem does not appear in the Sql query, but appears in the HMS startup warm-up，I will give some detailed instructions.
1.The problem occurs in the hot load function prewarm (RawStore rawStore) of the CachedStore. After the DataBase is loaded, the table information loading starts.
2.sharedCache.populateTableInCache(table, tableColStats...) CachedStore 559 Lines Throw NPE.
3.Method populateTableInCache calculates the memory of the object, and throws NPE when performing OBJECT_ARRAY calculation.


> CachedStore Prewarm Failure NPE
> -------------------------------
>
>                 Key: HIVE-22410
>                 URL: https://issues.apache.org/jira/browse/HIVE-22410
>             Project: Hive
>          Issue Type: Bug
>          Components: Standalone Metastore
>    Affects Versions: 3.1.2, 2.3.7
>         Environment: JDK 1.7.0_40
>            Reporter: fanshilun
>            Assignee: fanshilun
>            Priority: Major
>              Labels: cachedstore
>             Fix For: 2.3.7, 3.1.3
>
>         Attachments: HIVE-22410.1.patch, image-2019-11-01-11-14-42-114.png
>
>
> java.lang.NullPointerException
> at org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimateArrayElements(IncrementalObjectSizeEstimator.java:501) ~[hive-metastore-2.3.5.jar:2.3.5]
> at org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:443) ~[hive-metastore-2.3.5.jar:2.3.5]
> at org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:433) ~[hive-metastore-2.3.5.jar:2.3.5]
> at org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:369) ~[hive-metastore-2.3.5.jar:2.3.5]
> at org.apache.hadoop.hive.metastore.cached.SharedCache.populateTableInCache(SharedCache.java:1028) ~[hive-metastore-2.3.5.jar:2.3.5]
> at org.apache.hadoop.hive.metastore.cached.CachedStore.prewarm(CachedStore.java:314) ~[hive-metastore-2.3.5.jar:2.3.5]
> at org.apache.hadoop.hive.metastore.cached.CachedStore$CacheUpdateMasterWork.run(CachedStore.java:492) [hive-metastore-2.3.5.jar:2.3.5]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [?:1.7.0_40]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [?:1.7.0_40]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [?:1.7.0_40]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.7.0_40]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_40]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_40]
> at java.lang.Thread.run(Thread.java:724) [?:1.7.0_40]
> *reason:*
> Methods estimateArrayElements throw NPE
> I found that the last class does not appear in the parent (HashMap<Class<?>, Object Estimator> ), and the parent does not contain the information of the ConcurrentHashMap Class.
> The solution is to initialize the class information before using it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)