You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by "446463844@qq.com" <44...@qq.com> on 2016/12/07 02:08:17 UTC
I delete my table in hive,but the file in HDFS not be deleted
I meet a problem in hive.
I drop a table in hive and the table name ' user_info_20161206'
---------------------------------------
hive> show tables;
OK
kylin_cal_dt
kylin_category_groupings
kylin_intermediate_dmp_cube_fb5904cf_a4d3_4815_802d_c31afe9119e9
kylin_intermediate_test_cube_08677652_0f84_4322_a2a5_0a963723579e
kylin_intermediate_test_cube_a37beebf_d7da_4956_8e25_d563dd834364
kylin_intermediate_test_cube_aa9ee162_0d45_4ea6_853d_6df127799edf
kylin_sales
Time taken: 0.045 seconds, Fetched: 7 row(s)
hive>
--------------------------------------
but I find the user_info_20161206 file is exist in HDFS file
------------------------------------------------
drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/dm.db
drwxrwxrwt - hadoop hadoop 0 2016-11-21 15:39 /user/hive/warehouse/dw.db
drwxrwxrwt - hadoop hadoop 0 2016-11-03 12:44 /user/hive/warehouse/kylin_cal_dt
drwxrwxrwt - hadoop hadoop 0 2016-11-03 12:44 /user/hive/warehouse/kylin_category_groupings
drwxrwxrwt - hadoop hadoop 0 2016-11-03 12:44 /user/hive/warehouse/kylin_sales
drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/ods.db
drwxrwxrwt - hadoop hadoop 0 2016-11-30 17:53 /user/hive/warehouse/raw.db
drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/rpt.db
drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/temp.db
drwxrwxrwt - hadoop hadoop 0 2016-11-24 13:17 /user/hive/warehouse/test.db
drwxrwxrwt - hive hadoop 0 2016-12-06 21:15 /user/hive/warehouse/user_info_20161206
--------------------------------------------------
I don't kown why I drop table in hive but the same file not delete in HDFS . I test in mine test environment,it's work
the owner of file is 'hive' not 'hadoop'?
446463844@qq.com
Re: I delete my table in hive,but the file in HDFS not be deleted
Posted by abhishek <ec...@gmail.com>.
How did you define your table - external or hive managed ? If it's defined as external then table and when you drop the table , metadata will be delete but the data will still be in hdfs.
Sent from my iPhone
> On Dec 6, 2016, at 6:08 PM, "446463844@qq.com" <44...@qq.com> wrote:
>
> I meet a problem in hive.
>
> I drop a table in hive and the table name ' user_info_20161206'
> ---------------------------------------
> hive> show tables;
> OK
> kylin_cal_dt
> kylin_category_groupings
> kylin_intermediate_dmp_cube_fb5904cf_a4d3_4815_802d_c31afe9119e9
> kylin_intermediate_test_cube_08677652_0f84_4322_a2a5_0a963723579e
> kylin_intermediate_test_cube_a37beebf_d7da_4956_8e25_d563dd834364
> kylin_intermediate_test_cube_aa9ee162_0d45_4ea6_853d_6df127799edf
> kylin_sales
> Time taken: 0.045 seconds, Fetched: 7 row(s)
> hive>
> --------------------------------------
> but I find the user_info_20161206 file is exist in HDFS file
> ------------------------------------------------
> drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/dm.db
> drwxrwxrwt - hadoop hadoop 0 2016-11-21 15:39 /user/hive/warehouse/dw.db
> drwxrwxrwt - hadoop hadoop 0 2016-11-03 12:44 /user/hive/warehouse/kylin_cal_dt
> drwxrwxrwt - hadoop hadoop 0 2016-11-03 12:44 /user/hive/warehouse/kylin_category_groupings
> drwxrwxrwt - hadoop hadoop 0 2016-11-03 12:44 /user/hive/warehouse/kylin_sales
> drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/ods.db
> drwxrwxrwt - hadoop hadoop 0 2016-11-30 17:53 /user/hive/warehouse/raw.db
> drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/rpt.db
> drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/temp.db
> drwxrwxrwt - hadoop hadoop 0 2016-11-24 13:17 /user/hive/warehouse/test.db
> drwxrwxrwt - hive hadoop 0 2016-12-06 21:15 /user/hive/warehouse/user_info_20161206
> --------------------------------------------------
> I don't kown why I drop table in hive but the same file not delete in HDFS . I test in mine test environment,it's work
> the owner of file is 'hive' not 'hadoop'?
> 446463844@qq.com
Re: Re: I delete my table in hive,but the file in HDFS not be deleted
Posted by "446463844@qq.com" <44...@qq.com>.
not external
this is create table statement:
-----------------------------
create table if not exists user_info_20161207 (rowKey string,timestamp string,sex string ,birthday string ,native_place string ) row format delimited fields terminated by ','
------------------------------
I kown the owner is hive not hadoop
I specified the hive2 jdbc username 'hive' not hadoop
I just use 'hadoop' to create table and I drop table in hive .I find it's not delete in HDFS file system.sad :(
446463844@qq.com
From: Alan Gates
Date: 2016-12-07 10:34
To: user
Subject: Re: I delete my table in hive,but the file in HDFS not be deleted
Is the table external or managed? External tables do not remove their data when dropped, managed tables do.
Alan.
> On Dec 6, 2016, at 18:08, 446463844@qq.com wrote:
>
> I meet a problem in hive.
>
> I drop a table in hive and the table name ' user_info_20161206'
> ---------------------------------------
> hive> show tables;
> OK
> kylin_cal_dt
> kylin_category_groupings
> kylin_intermediate_dmp_cube_fb5904cf_a4d3_4815_802d_c31afe9119e9
> kylin_intermediate_test_cube_08677652_0f84_4322_a2a5_0a963723579e
> kylin_intermediate_test_cube_a37beebf_d7da_4956_8e25_d563dd834364
> kylin_intermediate_test_cube_aa9ee162_0d45_4ea6_853d_6df127799edf
> kylin_sales
> Time taken: 0.045 seconds, Fetched: 7 row(s)
> hive>
> --------------------------------------
> but I find the user_info_20161206 file is exist in HDFS file
> ------------------------------------------------
> drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/dm.db
> drwxrwxrwt - hadoop hadoop 0 2016-11-21 15:39 /user/hive/warehouse/dw.db
> drwxrwxrwt - hadoop hadoop 0 2016-11-03 12:44 /user/hive/warehouse/kylin_cal_dt
> drwxrwxrwt - hadoop hadoop 0 2016-11-03 12:44 /user/hive/warehouse/kylin_category_groupings
> drwxrwxrwt - hadoop hadoop 0 2016-11-03 12:44 /user/hive/warehouse/kylin_sales
> drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/ods.db
> drwxrwxrwt - hadoop hadoop 0 2016-11-30 17:53 /user/hive/warehouse/raw.db
> drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/rpt.db
> drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/temp.db
> drwxrwxrwt - hadoop hadoop 0 2016-11-24 13:17 /user/hive/warehouse/test.db
> drwxrwxrwt - hive hadoop 0 2016-12-06 21:15 /user/hive/warehouse/user_info_20161206
> --------------------------------------------------
> I don't kown why I drop table in hive but the same file not delete in HDFS . I test in mine test environment,it's work
> the owner of file is 'hive' not 'hadoop'?
> 446463844@qq.com
Re: I delete my table in hive,but the file in HDFS not be deleted
Posted by Alan Gates <al...@gmail.com>.
Is the table external or managed? External tables do not remove their data when dropped, managed tables do.
Alan.
> On Dec 6, 2016, at 18:08, 446463844@qq.com wrote:
>
> I meet a problem in hive.
>
> I drop a table in hive and the table name ' user_info_20161206'
> ---------------------------------------
> hive> show tables;
> OK
> kylin_cal_dt
> kylin_category_groupings
> kylin_intermediate_dmp_cube_fb5904cf_a4d3_4815_802d_c31afe9119e9
> kylin_intermediate_test_cube_08677652_0f84_4322_a2a5_0a963723579e
> kylin_intermediate_test_cube_a37beebf_d7da_4956_8e25_d563dd834364
> kylin_intermediate_test_cube_aa9ee162_0d45_4ea6_853d_6df127799edf
> kylin_sales
> Time taken: 0.045 seconds, Fetched: 7 row(s)
> hive>
> --------------------------------------
> but I find the user_info_20161206 file is exist in HDFS file
> ------------------------------------------------
> drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/dm.db
> drwxrwxrwt - hadoop hadoop 0 2016-11-21 15:39 /user/hive/warehouse/dw.db
> drwxrwxrwt - hadoop hadoop 0 2016-11-03 12:44 /user/hive/warehouse/kylin_cal_dt
> drwxrwxrwt - hadoop hadoop 0 2016-11-03 12:44 /user/hive/warehouse/kylin_category_groupings
> drwxrwxrwt - hadoop hadoop 0 2016-11-03 12:44 /user/hive/warehouse/kylin_sales
> drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/ods.db
> drwxrwxrwt - hadoop hadoop 0 2016-11-30 17:53 /user/hive/warehouse/raw.db
> drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/rpt.db
> drwxrwxrwt - hadoop hadoop 0 2016-09-26 11:50 /user/hive/warehouse/temp.db
> drwxrwxrwt - hadoop hadoop 0 2016-11-24 13:17 /user/hive/warehouse/test.db
> drwxrwxrwt - hive hadoop 0 2016-12-06 21:15 /user/hive/warehouse/user_info_20161206
> --------------------------------------------------
> I don't kown why I drop table in hive but the same file not delete in HDFS . I test in mine test environment,it's work
> the owner of file is 'hive' not 'hadoop'?
> 446463844@qq.com