You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Philip Tromans (JIRA)" <ji...@apache.org> on 2012/04/29 12:14:52 UTC
[jira] [Commented] (HIVE-2988) Use of XMLEncoder to serialize
MapredWork causes OOM in hive cli
[ https://issues.apache.org/jira/browse/HIVE-2988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264501#comment-13264501 ]
Philip Tromans commented on HIVE-2988:
--------------------------------------
This might not be related, but I've also seen an intermittent StackOverflowError (when Hive is serializing tasks at the beginning of a job) where most of the stack trace is within the XMLEncoder as well. Has anyone else had a problem with this?
> Use of XMLEncoder to serialize MapredWork causes OOM in hive cli
> ----------------------------------------------------------------
>
> Key: HIVE-2988
> URL: https://issues.apache.org/jira/browse/HIVE-2988
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Reporter: Rohini Palaniswamy
> Labels: Performance
>
> When running queries on tables with 6000 partitions, hive cli if configured with 128M runs into OOM. Heapdump showed 37MB occupied by one XMLEncoder object while the MapredWork was 500K which is highly inefficient. We should switch to using something more efficient like XStream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira