You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by chengxiang li <ch...@intel.com> on 2015/03/20 07:22:53 UTC
Review Request 32288: HIVE-10006 RSC has memory leak while execute
multi queries
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32288/
-----------------------------------------------------------
Review request for hive.
Bugs: HIVE-10006
https://issues.apache.org/jira/browse/HIVE-10006
Repository: hive
Description
-------
In RSC, while spark call CombineHiveInputFormat::getSplits to split the job into tasks in a thread called "dag-scheduler-event-loop", MapWork would be added to a ThreadLocal map of "dag-scheduler-event-loop", and never get removed. As the "dag-scheduler-event-loop" thread is a long live and daemon thread, so all the MapWorks would be hold in the ThreadLocal map until RSC jvm crash or exit.
Diffs
-----
branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1667894
branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1667894
branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/objectinspector/LazyBinaryObjectInspectorFactory.java 1667894
branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/MetadataListStructObjectInspector.java 1667894
branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1667894
branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1667894
branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.java 1667894
Diff: https://reviews.apache.org/r/32288/diff/
Testing
-------
Thanks,
chengxiang li
Re: Review Request 32288: HIVE-10006 RSC has memory leak while
execute multi queries
Posted by Xuefu Zhang <xz...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32288/#review77193
-----------------------------------------------------------
Not sure if the diff is for a different bug.
- Xuefu Zhang
On March 20, 2015, 6:22 a.m., chengxiang li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32288/
> -----------------------------------------------------------
>
> (Updated March 20, 2015, 6:22 a.m.)
>
>
> Review request for hive.
>
>
> Bugs: HIVE-10006
> https://issues.apache.org/jira/browse/HIVE-10006
>
>
> Repository: hive
>
>
> Description
> -------
>
> In RSC, while spark call CombineHiveInputFormat::getSplits to split the job into tasks in a thread called "dag-scheduler-event-loop", MapWork would be added to a ThreadLocal map of "dag-scheduler-event-loop", and never get removed. As the "dag-scheduler-event-loop" thread is a long live and daemon thread, so all the MapWorks would be hold in the ThreadLocal map until RSC jvm crash or exit.
>
>
> Diffs
> -----
>
> branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1667894
> branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1667894
> branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/objectinspector/LazyBinaryObjectInspectorFactory.java 1667894
> branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/MetadataListStructObjectInspector.java 1667894
> branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1667894
> branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1667894
> branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.java 1667894
>
> Diff: https://reviews.apache.org/r/32288/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> chengxiang li
>
>