You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by zh_wu <zh...@163.com> on 2016/04/15 09:28:06 UTC
tricky issue when insert table using dynamic paritition
Hi Hive experts,
I encounter one issue about insert table using dynamice partition. My partition name contains the char ':', like '2016-01-01 00:00:00'. but when doing the dynamic partition insert, it seems the partition names in the target table will randomly contain '%253A' instead of '%3A'. do you know why this happens and how to deal with it?
Thanks very much,
Jonathan
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-11-24 00%253A00%253A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-11-27 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-11-30 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-02 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-04 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-11 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-12 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-24 00%253A00%253A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-25 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-27 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-30 00%253A00%253A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-31 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-05 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-06 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-19 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-20 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-21 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-25 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-28 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-29 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-06 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-15 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-16 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-17 00%253A00%253A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-19 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-20 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-23 00%253A00%253A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-26 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-29 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-03-01 00%3A00%3A00
Re:tricky issue when insert table using dynamic paritition
Posted by zh_wu <zh...@163.com>.
here attached the hive log below:
INFO : MapReduce Total cumulative CPU time: 2 minutes 13 seconds 870 msec
INFO : Ended Job = job_1460187993973_1898
INFO : Starting task [Stage-6:MOVE] in serial mode
INFO : Moving data to: hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/.hive-staging_hive_2016-04-15_15-46-52_685_7519348183848403964-9/-ext-10000/dw_latest_flag=N/dw_effect_end_date=2015-11-24%2000%253A00%253A00 from hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/.hive-staging_hive_2016-04-15_15-46-52_685_7519348183848403964-9/-ext-10002/dw_latest_flag=N/dw_effect_end_date=2015-11-24 00%3A00%3A00
INFO : Moving data to: hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/.hive-staging_hive_2016-04-15_15-46-52_685_7519348183848403964-9/-ext-10000/dw_latest_flag=N/dw_effect_end_date=2015-12-24%2000%253A00%253A00 from hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/.hive-staging_hive_2016-04-15_15-46-52_685_7519348183848403964-9/-ext-10002/dw_latest_flag=N/dw_effect_end_date=2015-12-24 00%3A00%3A00
INFO : Moving data to: hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/.hive-staging_hive_2016-04-15_15-46-52_685_7519348183848403964-9/-ext-10000/dw_latest_flag=N/dw_effect_end_date=2015-12-30%2000%253A00%253A00 from hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/.hive-staging_hive_2016-04-15_15-46-52_685_7519348183848403964-9/-ext-10002/dw_latest_flag=N/dw_effect_end_date=2015-12-30 00%3A00%3A00
INFO : Moving data to: hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/.hive-staging_hive_2016-04-15_15-46-52_685_7519348183848403964-9/-ext-10000/dw_latest_flag=N/dw_effect_end_date=2016-02-17%2000%253A00%253A00 from hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/.hive-staging_hive_2016-04-15_15-46-52_685_7519348183848403964-9/-ext-10002/dw_latest_flag=N/dw_effect_end_date=2016-02-17 00%3A00%3A00
INFO : Moving data to: hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/.hive-staging_hive_2016-04-15_15-46-52_685_7519348183848403964-9/-ext-10000/dw_latest_flag=N/dw_effect_end_date=2016-02-23%2000%253A00%253A00 from hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/.hive-staging_hive_2016-04-15_15-46-52_685_7519348183848403964-9/-ext-10002/dw_latest_flag=N/dw_effect_end_date=2016-02-23 00%3A00%3A00
INFO : Moving data to: hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/.hive-staging_hive_2016-04-15_15-46-52_685_7519348183848403964-9/-ext-10000/dw_latest_flag=Y/dw_effect_end_date=3000-01-01%2000%253A00%253A00 from hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/.hive-staging_hive_2016-04-15_15-46-52_685_7519348183848403964-9/-ext-10002/dw_latest_flag=Y/dw_effect_end_date=3000-01-01 00%3A00%3A00
INFO : Starting task [Stage-0:MOVE] in serial mode
INFO : Loading data to table idw.dim_account_hist2 partition (dw_latest_flag=null, dw_effect_end_date=null) from hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/.hive-staging_hive_2016-04-15_15-46-52_685_7519348183848403964-9/-ext-10000
INFO : Time taken for load dynamic partitions : 23111
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-05 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-10-28 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-27 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-02-26 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-02-15 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-02-16 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-10-27 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-02-20 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-14 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-12-02 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-09 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-09-23 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-10-23 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-06 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-03 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-01-29 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-02-17 00%3A00%3A00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-10-26 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-09-14 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-13 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-09-24 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2013-08-23 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-06-25 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-09-17 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-10-12 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-16 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-01-25 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-09-29 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-02 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-02-23 00%3A00%3A00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-12-24 00%3A00%3A00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-09-15 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-09-28 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-10-16 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-02-29 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-09-22 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-03-01 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-07-23 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-09-21 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-12-27 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-12-12 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-07-09 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-12-31 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-01-28 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-24 00%3A00%3A00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-09-30 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-07-29 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-10-21 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-12 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-12-11 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-10-13 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-10-15 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-10-22 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-12-30 00%3A00%3A00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-01-05 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-09-25 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-01-06 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-19 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-02-19 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-20 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-12-04 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-30 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-01-20 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-09-16 00:00:00}
INFO : Loading partition {dw_latest_flag=Y, dw_effect_end_date=3000-01-01 00%3A00%3A00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-10-29 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-10 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-02-06 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-11-04 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-01-19 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2016-01-21 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-10-30 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-12-25 00:00:00}
INFO : Loading partition {dw_latest_flag=N, dw_effect_end_date=2015-09-18 00:00:00}
INFO : Time taken for adding to write entity : 9
INFO : Starting task [Stage-2:STATS] in serial mode
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2013-08-23 00:00:00} stats: [numFiles=1, numRows=101, totalSize=30870, rawDataSize=30769]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-06-25 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2156, rawDataSize=2149]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-07-09 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2156, rawDataSize=2149]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-07-23 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2072, rawDataSize=2065]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-07-29 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2170, rawDataSize=2163]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-09-14 00:00:00} stats: [numFiles=1, numRows=140, totalSize=41202, rawDataSize=41062]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-09-15 00:00:00} stats: [numFiles=1, numRows=2029, totalSize=596243, rawDataSize=594214]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-09-16 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2065, rawDataSize=2058]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-09-17 00:00:00} stats: [numFiles=1, numRows=42, totalSize=12460, rawDataSize=12418]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-09-18 00:00:00} stats: [numFiles=1, numRows=168, totalSize=49364, rawDataSize=49196]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-09-21 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2177, rawDataSize=2170]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-09-22 00:00:00} stats: [numFiles=1, numRows=51, totalSize=15113, rawDataSize=15062]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-09-23 00:00:00} stats: [numFiles=1, numRows=35, totalSize=10311, rawDataSize=10276]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-09-24 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2044, rawDataSize=2037]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-09-25 00:00:00} stats: [numFiles=1, numRows=6, totalSize=1776, rawDataSize=1770]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-09-28 00:00:00} stats: [numFiles=1, numRows=21, totalSize=6181, rawDataSize=6160]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-09-29 00:00:00} stats: [numFiles=1, numRows=82, totalSize=24174, rawDataSize=24092]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-09-30 00:00:00} stats: [numFiles=1, numRows=14, totalSize=4123, rawDataSize=4109]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-10-12 00:00:00} stats: [numFiles=1, numRows=76, totalSize=22432, rawDataSize=22356]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-10-13 00:00:00} stats: [numFiles=1, numRows=1260, totalSize=370440, rawDataSize=369180]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-10-15 00:00:00} stats: [numFiles=1, numRows=90, totalSize=26712, rawDataSize=26622]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-10-16 00:00:00} stats: [numFiles=1, numRows=28, totalSize=8232, rawDataSize=8204]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-10-21 00:00:00} stats: [numFiles=1, numRows=21, totalSize=6160, rawDataSize=6139]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-10-22 00:00:00} stats: [numFiles=1, numRows=34, totalSize=9991, rawDataSize=9957]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-10-23 00:00:00} stats: [numFiles=1, numRows=19, totalSize=5646, rawDataSize=5627]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-10-26 00:00:00} stats: [numFiles=1, numRows=42, totalSize=12306, rawDataSize=12264]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-10-27 00:00:00} stats: [numFiles=1, numRows=168, totalSize=49399, rawDataSize=49231]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-10-28 00:00:00} stats: [numFiles=1, numRows=35, totalSize=10290, rawDataSize=10255]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-10-29 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2058, rawDataSize=2051]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-10-30 00:00:00} stats: [numFiles=1, numRows=140, totalSize=41160, rawDataSize=41020]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-02 00:00:00} stats: [numFiles=1, numRows=28, totalSize=8386, rawDataSize=8358]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-03 00:00:00} stats: [numFiles=1, numRows=48, totalSize=14148, rawDataSize=14100]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-04 00:00:00} stats: [numFiles=1, numRows=91, totalSize=26843, rawDataSize=26752]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-05 00:00:00} stats: [numFiles=1, numRows=222, totalSize=65361, rawDataSize=65139]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-06 00:00:00} stats: [numFiles=1, numRows=497, totalSize=146118, rawDataSize=145621]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-09 00:00:00} stats: [numFiles=1, numRows=6, totalSize=1884, rawDataSize=1878]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-10 00:00:00} stats: [numFiles=1, numRows=4851, totalSize=1426992, rawDataSize=1422141]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-12 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2051, rawDataSize=2044]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-13 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2044, rawDataSize=2037]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-14 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2051, rawDataSize=2044]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-16 00:00:00} stats: [numFiles=1, numRows=21, totalSize=6195, rawDataSize=6174]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-19 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2107, rawDataSize=2100]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-20 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2170, rawDataSize=2163]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-24 00%3A00%3A00} stats: [numFiles=1, numRows=0, totalSize=2044, rawDataSize=0]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-27 00:00:00} stats: [numFiles=1, numRows=21, totalSize=6209, rawDataSize=6188]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-11-30 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2191, rawDataSize=2184]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-12-02 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2058, rawDataSize=2051]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-12-04 00:00:00} stats: [numFiles=1, numRows=14, totalSize=4214, rawDataSize=4200]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-12-11 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2177, rawDataSize=2170]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-12-12 00:00:00} stats: [numFiles=1, numRows=56, totalSize=16373, rawDataSize=16317]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-12-24 00%3A00%3A00} stats: [numFiles=1, numRows=0, totalSize=2086, rawDataSize=0]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-12-25 00:00:00} stats: [numFiles=1, numRows=21, totalSize=6258, rawDataSize=6237]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-12-27 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2191, rawDataSize=2184]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-12-30 00%3A00%3A00} stats: [numFiles=1, numRows=0, totalSize=2086, rawDataSize=0]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2015-12-31 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2107, rawDataSize=2100]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-01-05 00:00:00} stats: [numFiles=1, numRows=14, totalSize=4116, rawDataSize=4102]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-01-06 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2177, rawDataSize=2170]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-01-19 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2093, rawDataSize=2086]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-01-20 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2037, rawDataSize=2030]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-01-21 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2037, rawDataSize=2030]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-01-25 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2177, rawDataSize=2170]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-01-28 00:00:00} stats: [numFiles=1, numRows=35, totalSize=10213, rawDataSize=10178]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-01-29 00:00:00} stats: [numFiles=1, numRows=7, totalSize=2114, rawDataSize=2107]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-02-06 00:00:00} stats: [numFiles=1, numRows=5, totalSize=1490, rawDataSize=1485]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-02-15 00:00:00} stats: [numFiles=1, numRows=15, totalSize=4415, rawDataSize=4400]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-02-16 00:00:00} stats: [numFiles=1, numRows=17, totalSize=5007, rawDataSize=4990]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-02-17 00%3A00%3A00} stats: [numFiles=1, numRows=0, totalSize=584, rawDataSize=0]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-02-19 00:00:00} stats: [numFiles=1, numRows=38, totalSize=11156, rawDataSize=11118]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-02-20 00:00:00} stats: [numFiles=1, numRows=6, totalSize=1756, rawDataSize=1750]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-02-23 00%3A00%3A00} stats: [numFiles=1, numRows=0, totalSize=630, rawDataSize=0]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-02-26 00:00:00} stats: [numFiles=1, numRows=34, totalSize=9916, rawDataSize=9882]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-02-29 00:00:00} stats: [numFiles=1, numRows=10, totalSize=3004, rawDataSize=2994]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=N, dw_effect_end_date=2016-03-01 00:00:00} stats: [numFiles=1, numRows=2, totalSize=628, rawDataSize=626]
INFO : Partition idw.dim_account_hist2{dw_latest_flag=Y, dw_effect_end_date=3000-01-01 00%3A00%3A00} stats: [numFiles=2, numRows=0, totalSize=137545106, rawDataSize=0]
INFO : MapReduce Jobs Launched:
INFO : Stage-Stage-1: Map: 3 Cumulative CPU: 52.51 sec HDFS Read: 96504138 HDFS Write: 140756178 SUCCESS
INFO : Stage-Stage-5: Map: 68 Cumulative CPU: 133.87 sec HDFS Read: 5487467 HDFS Write: 3185947 SUCCESS
INFO : Total MapReduce CPU Time Spent: 3 minutes 6 seconds 380 msec
INFO : Completed executing command(queryId=hive_20160415154646_25881de8-a91d-4aa4-a12c-8428c57d85ae); Time taken: 86.949 seconds
INFO : OK
在2016年04月15 15时28分, "zh_wu"<zh...@163.com>写道:
Hi Hive experts,
I encounter one issue about insert table using dynamice partition. My partition name contains the char ':', like '2016-01-01 00:00:00'. but when doing the dynamic partition insert, it seems the partition names in the target table will randomly contain '%253A' instead of '%3A'. do you know why this happens and how to deal with it?
Thanks very much,
Jonathan
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-11-24 00%253A00%253A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-11-27 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-11-30 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-02 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-04 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-11 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-12 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-24 00%253A00%253A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-25 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-27 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-30 00%253A00%253A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2015-12-31 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-05 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-06 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-19 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-20 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-21 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-25 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-28 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-01-29 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-06 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-15 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-16 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-17 00%253A00%253A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-19 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-20 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-23 00%253A00%253A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-26 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-02-29 00%3A00%3A00
drwxrwxrwt - hive hive 0 2016-04-15 02:43 hdfs://lufaxhd/user/hive/warehouse/idw.db/dim_account_hist2/dw_latest_flag=N/dw_effect_end_date=2016-03-01 00%3A00%3A00