You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Zheng Shao (JIRA)" <ji...@apache.org> on 2010/02/25 00:07:27 UTC
[jira] Created: (HIVE-1195) Increase ObjectInspector[] length on
demand
Increase ObjectInspector[] length on demand
-------------------------------------------
Key: HIVE-1195
URL: https://issues.apache.org/jira/browse/HIVE-1195
Project: Hadoop Hive
Issue Type: Improvement
Affects Versions: 0.5.0, 0.6.0
Reporter: Zheng Shao
Assignee: Zheng Shao
{code}
Operator.java
protected transient ObjectInspector[] inputObjInspectors = new ObjectInspector[Short.MAX_VALUE];
{code}
An array of 32K elements takes 256KB memory under 64-bit Java.
We are seeing hive client going out of memory because of that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1195) Increase ObjectInspector[] length on
demand
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838114#action_12838114 ]
Ning Zhang commented on HIVE-1195:
----------------------------------
Cool. I'll take the new patches to test.
> Increase ObjectInspector[] length on demand
> -------------------------------------------
>
> Key: HIVE-1195
> URL: https://issues.apache.org/jira/browse/HIVE-1195
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.5.0, 0.6.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.5.1, 0.6.0
>
> Attachments: HIVE-1195-branch-0.5.patch, HIVE-1195.1.patch, HIVE-1195.2.branch-0.5.patch, HIVE-1195.2.patch
>
>
> {code}
> Operator.java
> protected transient ObjectInspector[] inputObjInspectors = new ObjectInspector[Short.MAX_VALUE];
> {code}
> An array of 32K elements takes 256KB memory under 64-bit Java.
> We are seeing hive client going out of memory because of that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1195) Increase ObjectInspector[] length on
demand
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang updated HIVE-1195:
-----------------------------
Attachment: HIVE-1195-branch-0.5.patch
Uploading a patch for branch 0.5. Zheng, can you double check?
> Increase ObjectInspector[] length on demand
> -------------------------------------------
>
> Key: HIVE-1195
> URL: https://issues.apache.org/jira/browse/HIVE-1195
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.5.0, 0.6.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.5.1, 0.6.0
>
> Attachments: HIVE-1195-branch-0.5.patch, HIVE-1195.1.patch
>
>
> {code}
> Operator.java
> protected transient ObjectInspector[] inputObjInspectors = new ObjectInspector[Short.MAX_VALUE];
> {code}
> An array of 32K elements takes 256KB memory under 64-bit Java.
> We are seeing hive client going out of memory because of that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1195) Increase ObjectInspector[] length on
demand
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang resolved HIVE-1195.
------------------------------
Resolution: Fixed
Committed to 0.5.1 and trunk. Thanks Zheng!
> Increase ObjectInspector[] length on demand
> -------------------------------------------
>
> Key: HIVE-1195
> URL: https://issues.apache.org/jira/browse/HIVE-1195
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.5.0, 0.6.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.5.1, 0.6.0
>
> Attachments: HIVE-1195-branch-0.5.patch, HIVE-1195.1.patch, HIVE-1195.2.branch-0.5.patch, HIVE-1195.2.patch
>
>
> {code}
> Operator.java
> protected transient ObjectInspector[] inputObjInspectors = new ObjectInspector[Short.MAX_VALUE];
> {code}
> An array of 32K elements takes 256KB memory under 64-bit Java.
> We are seeing hive client going out of memory because of that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1195) Increase ObjectInspector[] length on
demand
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838112#action_12838112 ]
Ning Zhang commented on HIVE-1195:
----------------------------------
Zheng, join26.q , join_map_ppr.q , union16.q, union9.q, failed on trunk. Can you take a look?
> Increase ObjectInspector[] length on demand
> -------------------------------------------
>
> Key: HIVE-1195
> URL: https://issues.apache.org/jira/browse/HIVE-1195
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.5.0, 0.6.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.5.1, 0.6.0
>
> Attachments: HIVE-1195-branch-0.5.patch, HIVE-1195.1.patch, HIVE-1195.2.branch-0.5.patch, HIVE-1195.2.patch
>
>
> {code}
> Operator.java
> protected transient ObjectInspector[] inputObjInspectors = new ObjectInspector[Short.MAX_VALUE];
> {code}
> An array of 32K elements takes 256KB memory under 64-bit Java.
> We are seeing hive client going out of memory because of that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1195) Increase ObjectInspector[] length on
demand
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-1195:
---------------------------------
Issue Type: Bug (was: Improvement)
Fix Version/s: (was: 0.5.1)
Component/s: Serializers/Deserializers
> Increase ObjectInspector[] length on demand
> -------------------------------------------
>
> Key: HIVE-1195
> URL: https://issues.apache.org/jira/browse/HIVE-1195
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.5.0, 0.6.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.6.0
>
> Attachments: HIVE-1195-branch-0.5.patch, HIVE-1195.1.patch, HIVE-1195.2.branch-0.5.patch, HIVE-1195.2.patch
>
>
> {code}
> Operator.java
> protected transient ObjectInspector[] inputObjInspectors = new ObjectInspector[Short.MAX_VALUE];
> {code}
> An array of 32K elements takes 256KB memory under 64-bit Java.
> We are seeing hive client going out of memory because of that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1195) Increase ObjectInspector[] length on
demand
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-1195:
-----------------------------
Attachment: HIVE-1195.1.patch
> Increase ObjectInspector[] length on demand
> -------------------------------------------
>
> Key: HIVE-1195
> URL: https://issues.apache.org/jira/browse/HIVE-1195
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.5.0, 0.6.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-1195.1.patch
>
>
> {code}
> Operator.java
> protected transient ObjectInspector[] inputObjInspectors = new ObjectInspector[Short.MAX_VALUE];
> {code}
> An array of 32K elements takes 256KB memory under 64-bit Java.
> We are seeing hive client going out of memory because of that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1195) Increase ObjectInspector[] length on
demand
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-1195:
-----------------------------
Attachment: HIVE-1195.2.patch
HIVE-1195.2.branch-0.5.patch
Fixed an obvious bug which caused unit test failures.
> Increase ObjectInspector[] length on demand
> -------------------------------------------
>
> Key: HIVE-1195
> URL: https://issues.apache.org/jira/browse/HIVE-1195
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.5.0, 0.6.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.5.1, 0.6.0
>
> Attachments: HIVE-1195-branch-0.5.patch, HIVE-1195.1.patch, HIVE-1195.2.branch-0.5.patch, HIVE-1195.2.patch
>
>
> {code}
> Operator.java
> protected transient ObjectInspector[] inputObjInspectors = new ObjectInspector[Short.MAX_VALUE];
> {code}
> An array of 32K elements takes 256KB memory under 64-bit Java.
> We are seeing hive client going out of memory because of that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1195) Increase ObjectInspector[] length on
demand
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang updated HIVE-1195:
-----------------------------
Status: Open (was: Patch Available)
> Increase ObjectInspector[] length on demand
> -------------------------------------------
>
> Key: HIVE-1195
> URL: https://issues.apache.org/jira/browse/HIVE-1195
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.5.0, 0.6.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.5.1, 0.6.0
>
> Attachments: HIVE-1195-branch-0.5.patch, HIVE-1195.1.patch, HIVE-1195.2.branch-0.5.patch, HIVE-1195.2.patch
>
>
> {code}
> Operator.java
> protected transient ObjectInspector[] inputObjInspectors = new ObjectInspector[Short.MAX_VALUE];
> {code}
> An array of 32K elements takes 256KB memory under 64-bit Java.
> We are seeing hive client going out of memory because of that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1195) Increase ObjectInspector[] length on
demand
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838080#action_12838080 ]
Ning Zhang commented on HIVE-1195:
----------------------------------
+1 Will commit after tests.
> Increase ObjectInspector[] length on demand
> -------------------------------------------
>
> Key: HIVE-1195
> URL: https://issues.apache.org/jira/browse/HIVE-1195
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.5.0, 0.6.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.5.1, 0.6.0
>
> Attachments: HIVE-1195.1.patch
>
>
> {code}
> Operator.java
> protected transient ObjectInspector[] inputObjInspectors = new ObjectInspector[Short.MAX_VALUE];
> {code}
> An array of 32K elements takes 256KB memory under 64-bit Java.
> We are seeing hive client going out of memory because of that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1195) Increase ObjectInspector[] length on
demand
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-1195:
-----------------------------
Fix Version/s: 0.6.0
0.5.1
Status: Patch Available (was: Open)
> Increase ObjectInspector[] length on demand
> -------------------------------------------
>
> Key: HIVE-1195
> URL: https://issues.apache.org/jira/browse/HIVE-1195
> Project: Hadoop Hive
> Issue Type: Improvement
> Affects Versions: 0.5.0, 0.6.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.5.1, 0.6.0
>
> Attachments: HIVE-1195.1.patch
>
>
> {code}
> Operator.java
> protected transient ObjectInspector[] inputObjInspectors = new ObjectInspector[Short.MAX_VALUE];
> {code}
> An array of 32K elements takes 256KB memory under 64-bit Java.
> We are seeing hive client going out of memory because of that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.