You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Ning Zhang (JIRA)" <ji...@apache.org> on 2010/04/21 07:38:54 UTC
[jira] Created: (HIVE-1316) Increase the memory limit for CLI
client
Increase the memory limit for CLI client
----------------------------------------
Key: HIVE-1316
URL: https://issues.apache.org/jira/browse/HIVE-1316
Project: Hadoop Hive
Issue Type: Improvement
Affects Versions: 0.6.0
Reporter: Ning Zhang
Assignee: Ning Zhang
Fix For: 0.6.0
Hive CLI client stores the plan in memory and serialize it to XML file. the serialization takes a lot of memory. for large queries, we've seen OOM exception and GC overhead limit reached. It would be good to increase the max heap size of CLI client and disable GC overhead limit.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1316) Increase the memory limit for CLI
client
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12859519#action_12859519 ]
Namit Jain commented on HIVE-1316:
----------------------------------
+1
looks good - will commit if the tests pass
> Increase the memory limit for CLI client
> ----------------------------------------
>
> Key: HIVE-1316
> URL: https://issues.apache.org/jira/browse/HIVE-1316
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.6.0
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.6.0
>
> Attachments: HIVE-1316.patch
>
>
> Hive CLI client stores the plan in memory and serialize it to XML file. the serialization takes a lot of memory. for large queries, we've seen OOM exception and GC overhead limit reached. It would be good to increase the max heap size of CLI client and disable GC overhead limit.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1316) Increase the memory limit for CLI
client
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang updated HIVE-1316:
-----------------------------
Attachment: HIVE-1316.patch
> Increase the memory limit for CLI client
> ----------------------------------------
>
> Key: HIVE-1316
> URL: https://issues.apache.org/jira/browse/HIVE-1316
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.6.0
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.6.0
>
> Attachments: HIVE-1316.patch
>
>
> Hive CLI client stores the plan in memory and serialize it to XML file. the serialization takes a lot of memory. for large queries, we've seen OOM exception and GC overhead limit reached. It would be good to increase the max heap size of CLI client and disable GC overhead limit.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1316) Increase the memory limit for CLI
client
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang updated HIVE-1316:
-----------------------------
Status: Patch Available (was: Open)
> Increase the memory limit for CLI client
> ----------------------------------------
>
> Key: HIVE-1316
> URL: https://issues.apache.org/jira/browse/HIVE-1316
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.6.0
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.6.0
>
> Attachments: HIVE-1316.patch
>
>
> Hive CLI client stores the plan in memory and serialize it to XML file. the serialization takes a lot of memory. for large queries, we've seen OOM exception and GC overhead limit reached. It would be good to increase the max heap size of CLI client and disable GC overhead limit.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1316) Increase the memory limit for CLI
client
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12859509#action_12859509 ]
Ning Zhang commented on HIVE-1316:
----------------------------------
HADOOP_HEAPSIZE is used in -Xmx but not -Xms, so it won't reduce the amount of concurrency.
> Increase the memory limit for CLI client
> ----------------------------------------
>
> Key: HIVE-1316
> URL: https://issues.apache.org/jira/browse/HIVE-1316
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.6.0
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.6.0
>
> Attachments: HIVE-1316.patch
>
>
> Hive CLI client stores the plan in memory and serialize it to XML file. the serialization takes a lot of memory. for large queries, we've seen OOM exception and GC overhead limit reached. It would be good to increase the max heap size of CLI client and disable GC overhead limit.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1316) Increase the memory limit for CLI
client
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1316:
-----------------------------
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Resolution: Fixed
Committed. Thanks Ning
> Increase the memory limit for CLI client
> ----------------------------------------
>
> Key: HIVE-1316
> URL: https://issues.apache.org/jira/browse/HIVE-1316
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.6.0
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.6.0
>
> Attachments: HIVE-1316.patch
>
>
> Hive CLI client stores the plan in memory and serialize it to XML file. the serialization takes a lot of memory. for large queries, we've seen OOM exception and GC overhead limit reached. It would be good to increase the max heap size of CLI client and disable GC overhead limit.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1316) Increase the memory limit for CLI
client
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12859420#action_12859420 ]
Namit Jain commented on HIVE-1316:
----------------------------------
Will it reduce the amount of concurrent hive clients ?
> Increase the memory limit for CLI client
> ----------------------------------------
>
> Key: HIVE-1316
> URL: https://issues.apache.org/jira/browse/HIVE-1316
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.6.0
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.6.0
>
> Attachments: HIVE-1316.patch
>
>
> Hive CLI client stores the plan in memory and serialize it to XML file. the serialization takes a lot of memory. for large queries, we've seen OOM exception and GC overhead limit reached. It would be good to increase the max heap size of CLI client and disable GC overhead limit.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.