You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Aihua Xu (JIRA)" <ji...@apache.org> on 2018/03/17 01:25:00 UTC
[jira] [Commented] (HIVE-18986) Table rename will run
java.lang.StackOverflowError in dataNucleus if the table contains large
number of columns
[ https://issues.apache.org/jira/browse/HIVE-18986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403188#comment-16403188 ]
Aihua Xu commented on HIVE-18986:
---------------------------------
patch-1: use directsql to get the column stats which can use in statement and can run in batch. Tested with large table and it works. I will not add such unit test for this case.
> Table rename will run java.lang.StackOverflowError in dataNucleus if the table contains large number of columns
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-18986
> URL: https://issues.apache.org/jira/browse/HIVE-18986
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
> Reporter: Aihua Xu
> Assignee: Aihua Xu
> Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18986.1.patch
>
>
> If the table contains a lot of columns e.g, 5k, simple table rename would fail with the following stack trace. The issue is datanucleus can't handle the query with lots of colName='c1' && colName='c2'.
>
> 2018-03-13 17:19:52,770 INFO org.apache.hadoop.hive.metastore.HiveMetaStore.audit: [pool-5-thread-200]: ugi=anonymous ip=10.17.100.135 cmd=source:10.17.100.135 alter_table: db=default tbl=fgv_full_var_pivoted02 newtbl=fgv_full_var_pivoted 2018-03-13 17:20:00,495 ERROR org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-5-thread-200]: java.lang.StackOverflowError at org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:330) at org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:339) at org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:339) at org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:339) at org.datanucleus.store.rdbms.sql.SQLText.toSQL(SQLText.java:339)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)