You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Franklin Hu (JIRA)" <ji...@apache.org> on 2011/08/05 20:35:29 UTC
[jira] [Created] (HIVE-2352) selecting no rows and overwriting a
table creates empty file for nonbucketized table
selecting no rows and overwriting a table creates empty file for nonbucketized table
------------------------------------------------------------------------------------
Key: HIVE-2352
URL: https://issues.apache.org/jira/browse/HIVE-2352
Project: Hive
Issue Type: Bug
Reporter: Franklin Hu
Assignee: Franklin Hu
Priority: Minor
Attachments: hive-2352.1.patch
create table t1 (key int, value string) stored as rcfile;
insert overwrite table t1 select * from src where false;
Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) create empty files if and only if
table is bucketed and hive.enforce.bucketing=true
Posted by "Siying Dong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siying Dong updated HIVE-2352:
------------------------------
Assignee: (was: Franklin Hu)
> create empty files if and only if table is bucketed and hive.enforce.bucketing=true
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Improvement
> Reporter: Franklin Hu
> Fix For: 0.8.0
>
> Attachments: hive-2352.1.patch, hive-2352.2.patch, hive-2352.3.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) selecting no rows and overwriting a
table creates empty file for nonbucketized table
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2352:
------------------------------
Attachment: hive-2352.1.patch
> selecting no rows and overwriting a table creates empty file for nonbucketized table
> ------------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
> Attachments: hive-2352.1.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2352) create empty files if and only if
table is bucketed and hive.enforce.bucketing=true
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081044#comment-13081044 ]
Ning Zhang commented on HIVE-2352:
----------------------------------
Franklink, load_dyn_part*.q failed. Can you take a look?
> create empty files if and only if table is bucketed and hive.enforce.bucketing=true
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
> Attachments: hive-2352.1.patch, hive-2352.2.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) selecting no rows and overwriting a
table creates empty file for nonbucketized table
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2352:
------------------------------
Description:
create table t1 (key int, value string) stored as rcfile;
insert overwrite table t1 select * from src where false;
Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
Fix:
If bucketing IS NOT enforced:
Create empty files in neither bucketed nor non-bucketed tables
If bucketing IS enforced:
Do not create empty files in non-bucketed tables
Create BUCKET_COUNT number of empty files for bucketed tables
was:
create table t1 (key int, value string) stored as rcfile;
insert overwrite table t1 select * from src where false;
Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
> selecting no rows and overwriting a table creates empty file for nonbucketized table
> ------------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
> Fix:
> If bucketing IS NOT enforced:
> Create empty files in neither bucketed nor non-bucketed tables
> If bucketing IS enforced:
> Do not create empty files in non-bucketed tables
> Create BUCKET_COUNT number of empty files for bucketed tables
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) create empty files if and only if
table is bucketed and hive.enforce.bucketing=true
Posted by "Siying Dong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siying Dong updated HIVE-2352:
------------------------------
Priority: Major (was: Minor)
Issue Type: Improvement (was: Bug)
> create empty files if and only if table is bucketed and hive.enforce.bucketing=true
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Improvement
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Fix For: 0.8.0
>
> Attachments: hive-2352.1.patch, hive-2352.2.patch, hive-2352.3.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2352) create empty files if and only if
table is bucketed and hive.enforce.bucketing=true
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081809#comment-13081809 ]
Ning Zhang commented on HIVE-2352:
----------------------------------
Franklink there are 2 more diffs are the last change. input26.q. and bucketmapjoin2.q. It seems the changes are more involved than what we expect, can you run the whole unit tests before submitting the next fix?
> create empty files if and only if table is bucketed and hive.enforce.bucketing=true
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: hive-2352.1.patch, hive-2352.2.patch, hive-2352.3.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) selecting no rows and overwriting a
table creates empty file for nonbucketized table
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2352:
------------------------------
Attachment: hive-2352.1.patch
Patch abides by "Fix" conditions in JIRA description.
> selecting no rows and overwriting a table creates empty file for nonbucketized table
> ------------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
> Attachments: hive-2352.1.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
> Fix:
> If bucketing IS NOT enforced:
> Create empty files in neither bucketed nor non-bucketed tables
> If bucketing IS enforced:
> Do not create empty files in non-bucketed tables
> Create BUCKET_COUNT number of empty files for bucketed tables
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) create empty files if and only if
table is bucketed and hive.enforce.bucketing=true
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2352:
------------------------------
Attachment: hive-2352.2.patch
fix var initialization, unit test styling
> create empty files if and only if table is bucketed and hive.enforce.bucketing=true
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
> Attachments: hive-2352.1.patch, hive-2352.2.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) selecting no rows and overwriting a
table creates empty file for nonbucketized table
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2352:
------------------------------
Attachment: hive-2352.1.patch
> selecting no rows and overwriting a table creates empty file for nonbucketized table
> ------------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
> Attachments: hive-2352.1.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2352) selecting no rows and overwriting a
table creates empty file for nonbucketized table
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080248#comment-13080248 ]
Ning Zhang commented on HIVE-2352:
----------------------------------
Franklin, a simpler description of your fix would be: create empty files if and only if table is bucketed and hive.enforce.bucketing=true, right?
> selecting no rows and overwriting a table creates empty file for nonbucketized table
> ------------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
> Attachments: hive-2352.1.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
> Fix:
> If bucketing IS NOT enforced:
> Create empty files in neither bucketed nor non-bucketed tables
> If bucketing IS enforced:
> Do not create empty files in non-bucketed tables
> Create BUCKET_COUNT number of empty files for bucketed tables
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) selecting no rows and overwriting a
table creates empty file for nonbucketized table
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2352:
------------------------------
Attachment: (was: hive-2352.1.patch)
> selecting no rows and overwriting a table creates empty file for nonbucketized table
> ------------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
> Fix:
> If bucketing IS NOT enforced:
> Create empty files in neither bucketed nor non-bucketed tables
> If bucketing IS enforced:
> Do not create empty files in non-bucketed tables
> Create BUCKET_COUNT number of empty files for bucketed tables
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2352) create empty files if and only if
table is bucketed and hive.enforce.bucketing=true
Posted by "Siying Dong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090399#comment-13090399 ]
Siying Dong commented on HIVE-2352:
-----------------------------------
I ran tests twice. Both crashed. I think it is an important patch and will improve latency of some queries (like scanning a large dataset for one or two rows) dramatically (Currently I sometimes do a "ORDER BY LIMIT BY" to speed it up if I know the data set is small). We should raise the priority.
> create empty files if and only if table is bucketed and hive.enforce.bucketing=true
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: hive-2352.1.patch, hive-2352.2.patch, hive-2352.3.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) selecting no rows and overwriting a
table creates empty file for nonbucketized table
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2352:
------------------------------
Attachment: (was: hive-2352.1.patch)
> selecting no rows and overwriting a table creates empty file for nonbucketized table
> ------------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) create empty files if and only if
table is bucketed and hive.enforce.bucketing=true
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2352:
------------------------------
Fix Version/s: 0.8.0
Status: Patch Available (was: Open)
> create empty files if and only if table is bucketed and hive.enforce.bucketing=true
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: hive-2352.1.patch, hive-2352.2.patch, hive-2352.3.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) create empty files if and only if
table is bucketed and hive.enforce.bucketing=true
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-2352:
-----------------------------
Status: Open (was: Patch Available)
> create empty files if and only if table is bucketed and hive.enforce.bucketing=true
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Improvement
> Reporter: Franklin Hu
> Fix For: 0.8.0
>
> Attachments: hive-2352.1.patch, hive-2352.2.patch, hive-2352.3.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) create empty files if and only if
table is bucketed and hive.enforce.bucketing=true
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2352:
------------------------------
Attachment: hive-2352.3.patch
fixed null ptr, passes load_dyn_part*.q now
> create empty files if and only if table is bucketed and hive.enforce.bucketing=true
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
> Attachments: hive-2352.1.patch, hive-2352.2.patch, hive-2352.3.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) create empty files if and only if
table is bucketed and hive.enforce.bucketing=true
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2352:
---------------------------------
Fix Version/s: (was: 0.8.0)
> create empty files if and only if table is bucketed and hive.enforce.bucketing=true
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Improvement
> Reporter: Franklin Hu
> Attachments: hive-2352.1.patch, hive-2352.2.patch, hive-2352.3.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2352) create empty files if and only if
table is bucketed and hive.enforce.bucketing=true
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080276#comment-13080276 ]
Ning Zhang commented on HIVE-2352:
----------------------------------
+1. Will commit if tests pass.
> create empty files if and only if table is bucketed and hive.enforce.bucketing=true
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
> Attachments: hive-2352.1.patch, hive-2352.2.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) create empty files if and only if
table is bucketed and hive.enforce.bucketing=true
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2352:
------------------------------
Description:
create table t1 (key int, value string) stored as rcfile;
insert overwrite table t1 select * from src where false;
Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
was:
create table t1 (key int, value string) stored as rcfile;
insert overwrite table t1 select * from src where false;
Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
Fix:
If bucketing IS NOT enforced:
Create empty files in neither bucketed nor non-bucketed tables
If bucketing IS enforced:
Do not create empty files in non-bucketed tables
Create BUCKET_COUNT number of empty files for bucketed tables
> create empty files if and only if table is bucketed and hive.enforce.bucketing=true
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
> Attachments: hive-2352.1.patch, hive-2352.2.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2352) create empty files if and only if
table is bucketed and hive.enforce.bucketing=true
Posted by "Siying Dong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089575#comment-13089575 ]
Siying Dong commented on HIVE-2352:
-----------------------------------
Franklin's internship ended. Let me apply his patch and see whether there is any failed tests.
> create empty files if and only if table is bucketed and hive.enforce.bucketing=true
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: hive-2352.1.patch, hive-2352.2.patch, hive-2352.3.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2352) create empty files if and only if
table is bucketed and hive.enforce.bucketing=true
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2352:
------------------------------
Summary: create empty files if and only if table is bucketed and hive.enforce.bucketing=true (was: selecting no rows and overwriting a table creates empty file for nonbucketized table)
> create empty files if and only if table is bucketed and hive.enforce.bucketing=true
> -----------------------------------------------------------------------------------
>
> Key: HIVE-2352
> URL: https://issues.apache.org/jira/browse/HIVE-2352
> Project: Hive
> Issue Type: Bug
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Priority: Minor
> Attachments: hive-2352.1.patch
>
>
> create table t1 (key int, value string) stored as rcfile;
> insert overwrite table t1 select * from src where false;
> Creates an empty RCFile with no rows and size 151B. The file not should be created since there are no rows.
> Fix:
> If bucketing IS NOT enforced:
> Create empty files in neither bucketed nor non-bucketed tables
> If bucketing IS enforced:
> Do not create empty files in non-bucketed tables
> Create BUCKET_COUNT number of empty files for bucketed tables
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira