You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Namit Jain (JIRA)" <ji...@apache.org> on 2010/06/20 23:08:23 UTC
[jira] Created: (HIVE-1420) problem with sequence and rcfiles are
mixed for null partitions
problem with sequence and rcfiles are mixed for null partitions
---------------------------------------------------------------
Key: HIVE-1420
URL: https://issues.apache.org/jira/browse/HIVE-1420
Project: Hadoop Hive
Issue Type: Bug
Components: Query Processor
Reporter: Namit Jain
Assignee: He Yongqiang
Fix For: 0.6.0
drop table foo;
create table foo (src int, value string) partitioned by (ds string);
alter table foo set fileformat Sequencefile;
insert overwrite table foo partition (ds='1')
select key, value from src;
alter table foo add partition (ds='2');
alter table foo set fileformat rcfile;
select count(1) from foo;
The above testcase fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1420) problem with sequence and rcfiles are
mixed for null partitions
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Yongqiang updated HIVE-1420:
-------------------------------
Status: Patch Available (was: Open)
Affects Version/s: 0.6.0
Fix Version/s: 0.7.0
> problem with sequence and rcfiles are mixed for null partitions
> ---------------------------------------------------------------
>
> Key: HIVE-1420
> URL: https://issues.apache.org/jira/browse/HIVE-1420
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Namit Jain
> Assignee: He Yongqiang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: hive-1420.1.patch
>
>
> drop table foo;
> create table foo (src int, value string) partitioned by (ds string);
> alter table foo set fileformat Sequencefile;
> insert overwrite table foo partition (ds='1')
> select key, value from src;
> alter table foo add partition (ds='2');
> alter table foo set fileformat rcfile;
> select count(1) from foo;
> The above testcase fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1420) problem with sequence and rcfiles are
mixed for null partitions
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Yongqiang resolved HIVE-1420.
--------------------------------
Resolution: Duplicate
duplicate of 1421
> problem with sequence and rcfiles are mixed for null partitions
> ---------------------------------------------------------------
>
> Key: HIVE-1420
> URL: https://issues.apache.org/jira/browse/HIVE-1420
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Namit Jain
> Assignee: He Yongqiang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: hive-1420.1.patch
>
>
> drop table foo;
> create table foo (src int, value string) partitioned by (ds string);
> alter table foo set fileformat Sequencefile;
> insert overwrite table foo partition (ds='1')
> select key, value from src;
> alter table foo add partition (ds='2');
> alter table foo set fileformat rcfile;
> select count(1) from foo;
> The above testcase fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1420) problem with sequence and rcfiles are
mixed for null partitions
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Yongqiang updated HIVE-1420:
-------------------------------
Status: Open (was: Patch Available)
> problem with sequence and rcfiles are mixed for null partitions
> ---------------------------------------------------------------
>
> Key: HIVE-1420
> URL: https://issues.apache.org/jira/browse/HIVE-1420
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Namit Jain
> Assignee: He Yongqiang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: hive-1420.1.patch
>
>
> drop table foo;
> create table foo (src int, value string) partitioned by (ds string);
> alter table foo set fileformat Sequencefile;
> insert overwrite table foo partition (ds='1')
> select key, value from src;
> alter table foo add partition (ds='2');
> alter table foo set fileformat rcfile;
> select count(1) from foo;
> The above testcase fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1420) problem with sequence and rcfiles are
mixed for null partitions
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880904#action_12880904 ]
Namit Jain commented on HIVE-1420:
----------------------------------
+1
will commit if the tests pass
> problem with sequence and rcfiles are mixed for null partitions
> ---------------------------------------------------------------
>
> Key: HIVE-1420
> URL: https://issues.apache.org/jira/browse/HIVE-1420
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Namit Jain
> Assignee: He Yongqiang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: hive-1420.1.patch
>
>
> drop table foo;
> create table foo (src int, value string) partitioned by (ds string);
> alter table foo set fileformat Sequencefile;
> insert overwrite table foo partition (ds='1')
> select key, value from src;
> alter table foo add partition (ds='2');
> alter table foo set fileformat rcfile;
> select count(1) from foo;
> The above testcase fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1420) problem with sequence and rcfiles are
mixed for null partitions
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Yongqiang updated HIVE-1420:
-------------------------------
Attachment: hive-1420.1.patch
Two changes:
One is using fileformat from partition desc when needing to create an empty file.
this is to fix the bug for 'select src from fileformat_mix_test;'
The other is set oc = null when do closing in ExecMapper.
And in ExecMapper's close, when see no data came to the mapper (which means an empty file), just return and do nothing.
this is to fix the bug for 'select count(1) from fileformat_mix_test;' When encountering empty file, we are not closing MapOperator tree. Hope this is fine. If we do the close, we will get NullPointerException from deep Hadoop code because of non-valid output collector
> problem with sequence and rcfiles are mixed for null partitions
> ---------------------------------------------------------------
>
> Key: HIVE-1420
> URL: https://issues.apache.org/jira/browse/HIVE-1420
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Namit Jain
> Assignee: He Yongqiang
> Fix For: 0.6.0, 0.7.0
>
> Attachments: hive-1420.1.patch
>
>
> drop table foo;
> create table foo (src int, value string) partitioned by (ds string);
> alter table foo set fileformat Sequencefile;
> insert overwrite table foo partition (ds='1')
> select key, value from src;
> alter table foo add partition (ds='2');
> alter table foo set fileformat rcfile;
> select count(1) from foo;
> The above testcase fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1420) problem with sequence and rcfiles are
mixed for null partitions
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-1420:
---------------------------------
Affects Version/s: (was: 0.6.0)
Fix Version/s: (was: 0.7.0)
> problem with sequence and rcfiles are mixed for null partitions
> ---------------------------------------------------------------
>
> Key: HIVE-1420
> URL: https://issues.apache.org/jira/browse/HIVE-1420
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: He Yongqiang
> Fix For: 0.6.0
>
> Attachments: hive-1420.1.patch
>
>
> drop table foo;
> create table foo (src int, value string) partitioned by (ds string);
> alter table foo set fileformat Sequencefile;
> insert overwrite table foo partition (ds='1')
> select key, value from src;
> alter table foo add partition (ds='2');
> alter table foo set fileformat rcfile;
> select count(1) from foo;
> The above testcase fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.