You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "He Yongqiang (JIRA)" <ji...@apache.org> on 2009/11/27 01:26:39 UTC
[jira] Created: (HIVE-955) Passing columns needed onto
ColumnarStruct
Passing columns needed onto ColumnarStruct
------------------------------------------
Key: HIVE-955
URL: https://issues.apache.org/jira/browse/HIVE-955
Project: Hadoop Hive
Issue Type: Improvement
Reporter: He Yongqiang
Assignee: He Yongqiang
Attachments: ColumnarStruct_columnids_2009-11-25.2.patch
This can avoid unneeded loop cycle in ColumnarStruct's init method. This can saves a lot when the table's column number is big.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-955) Passing columns needed onto
ColumnarStruct
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Yongqiang updated HIVE-955:
------------------------------
Attachment: ColumnarStruct_columnids_2009-11-25.2.patch
> Passing columns needed onto ColumnarStruct
> ------------------------------------------
>
> Key: HIVE-955
> URL: https://issues.apache.org/jira/browse/HIVE-955
> Project: Hadoop Hive
> Issue Type: Improvement
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Attachments: ColumnarStruct_columnids_2009-11-25.2.patch
>
>
> This can avoid unneeded loop cycle in ColumnarStruct's init method. This can saves a lot when the table's column number is big.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-955) Passing columns needed onto
ColumnarStruct
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Yongqiang updated HIVE-955:
------------------------------
Attachment: hive-955-2009-11-30.patch
The basic idea is to pass column needed into ColumnarStruct, and let the init method of ColumnarStruct only init those columns. The columns needed by the query is stored in TBS, so we need to get them out and set into the configuration. We did that when we call getRecordReader, where we clone a conf, and set column ids into the conf, and then pass the conf into native RecordReader. We need to do it over again when we init the operator tree. At this time, we need to pass a conf with these column information into the serde layer.
thanks, Namit!
> Passing columns needed onto ColumnarStruct
> ------------------------------------------
>
> Key: HIVE-955
> URL: https://issues.apache.org/jira/browse/HIVE-955
> Project: Hadoop Hive
> Issue Type: Improvement
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Attachments: ColumnarStruct_columnids_2009-11-25.2.patch, hive-955-2009-11-30.patch
>
>
> This can avoid unneeded loop cycle in ColumnarStruct's init method. This can saves a lot when the table's column number is big.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-955) Passing columns needed onto
ColumnarStruct
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783043#action_12783043 ]
Namit Jain commented on HIVE-955:
---------------------------------
Can you add more comments - I am not sure I understand this change
> Passing columns needed onto ColumnarStruct
> ------------------------------------------
>
> Key: HIVE-955
> URL: https://issues.apache.org/jira/browse/HIVE-955
> Project: Hadoop Hive
> Issue Type: Improvement
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Attachments: ColumnarStruct_columnids_2009-11-25.2.patch
>
>
> This can avoid unneeded loop cycle in ColumnarStruct's init method. This can saves a lot when the table's column number is big.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.