You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Thejas M Nair (JIRA)" <ji...@apache.org> on 2015/02/10 21:31:13 UTC
[jira] [Comment Edited] (HIVE-9500) Support nested structs over 24
levels.
[ https://issues.apache.org/jira/browse/HIVE-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314862#comment-14314862 ]
Thejas M Nair edited comment on HIVE-9500 at 2/10/15 8:31 PM:
--------------------------------------------------------------
If the theory is not convincing, attaching TClass.java that compares the performance. Use of HashMap is slower by around 4 times.
The results -
{noformat}
$ java -server testpkg.TClass
Array escape time(ms) 5919
Num chars escaped 200000000
Map escape time(ms) 23275
Num chars escaped 200000000
Perf diff factor = 3.9322520696063523
{noformat}
was (Author: thejas):
If the theory is not convincing, attaching TClass.java that compares the performance.
The results -
{noformat}
$ java -server testpkg.TClass
Array escape time(ms) 5919
Num chars escaped 200000000
Map escape time(ms) 23275
Num chars escaped 200000000
Perf diff factor = 3.9322520696063523
{noformat}
> Support nested structs over 24 levels.
> --------------------------------------
>
> Key: HIVE-9500
> URL: https://issues.apache.org/jira/browse/HIVE-9500
> Project: Hive
> Issue Type: Improvement
> Reporter: Aihua Xu
> Assignee: Aihua Xu
> Labels: SerDe
> Fix For: 1.2.0
>
> Attachments: HIVE-9500.1.patch, HIVE-9500.2.patch, HIVE-9500.3.patch, TClass.java
>
>
> Customer has deeply nested avro structure and is receiving the following error when performing queries.
> 15/01/09 20:59:29 ERROR ql.Driver: FAILED: SemanticException org.apache.hadoop.hive.serde2.SerDeException: Number of levels of nesting supported for LazySimpleSerde is 23 Unable to work with level 24
> Currently we support up to 24 levels of nested structs when hive.serialization.extend.nesting.levels is set to true, while the customers have the requirement to support more than that.
> It would be better to make the supported levels configurable or completely removed (i.e., we can support any number of levels).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)