You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Tao Li (JIRA)" <ji...@apache.org> on 2017/09/18 15:29:00 UTC
[jira] [Comment Edited] (HIVE-17496) Bootstrap repl is not cleaning
up staging dirs
[ https://issues.apache.org/jira/browse/HIVE-17496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170144#comment-16170144 ]
Tao Li edited comment on HIVE-17496 at 9/18/17 3:28 PM:
--------------------------------------------------------
According to the logs below, I did not see any error from HDFS deletion for the testDeleteStagingDir test, according to change in HIVE-17496.5.patch. Looks like the file should have been deleted successfully. So I don't know why the test fails.
{noformat}
2017-09-18T02:25:52,399 INFO [main] ql.Context: Deleting scratch dir: pfile:/home/hiveptest/104.198.168.233-hiveptest-0/apache-github-source-source/itests/hive-unit/target/warehouse/deletestagingdir_org_apache_hadoop_hive_ql_parse_testreplicationscenarios_1505726246743_dupe.db/unptned/.hive-staging_hive_2017-09-18_02-25-52_081_7805993483018006783-1
{noformat}
But in some other repl test logs, I do see some deletion that failed. It indicates some possibility there.
{noformat}
2017-09-18T02:25:49,746 INFO [main] ql.Context: Deleting scratch dir: file:/home/hiveptest/104.198.168.233-hiveptest-0/apache-github-source-source/itests/hive-unit/target/tmp/localscratchdir/6b15a44f-aec0-43f8-851d-81db07e4e329/hive_2017-09-18_02-25-49_687_551878613091072616-1/-mr-10001/.hive-staging_hive_2017-09-18_02-25-49_687_551878613091072616-1
2017-09-18T02:25:49,746 ERROR [main] ql.Context: File failes to be deleted when removing Scratch (HIVE-17496)
{noformat}
was (Author: taoli-hwx):
According to the logs below, I did not see any error from HDFS deletion for the testDeleteStagingDir test, according to change in HIVE-17496.5.patch. Looks like the file should have been deleted successfully. So I don't know why the test fails.
{noformat}
2017-09-18T02:25:52,399 INFO [main] ql.Context: Deleting scratch dir: pfile:/home/hiveptest/104.198.168.233-hiveptest-0/apache-github-source-source/itests/hive-unit/target/warehouse/deletestagingdir_org_apache_hadoop_hive_ql_parse_testreplicationscenarios_1505726246743_dupe.db/unptned/.hive-staging_hive_2017-09-18_02-25-52_081_7805993483018006783-1
{noformat}
But in some other repl test logs, I do see some deletion that failed:
{noformat}
2017-09-18T02:25:49,746 INFO [main] ql.Context: Deleting scratch dir: file:/home/hiveptest/104.198.168.233-hiveptest-0/apache-github-source-source/itests/hive-unit/target/tmp/localscratchdir/6b15a44f-aec0-43f8-851d-81db07e4e329/hive_2017-09-18_02-25-49_687_551878613091072616-1/-mr-10001/.hive-staging_hive_2017-09-18_02-25-49_687_551878613091072616-1
2017-09-18T02:25:49,746 ERROR [main] ql.Context: File failes to be deleted when removing Scratch (HIVE-17496)
{noformat}
> Bootstrap repl is not cleaning up staging dirs
> ----------------------------------------------
>
> Key: HIVE-17496
> URL: https://issues.apache.org/jira/browse/HIVE-17496
> Project: Hive
> Issue Type: Bug
> Components: repl
> Reporter: Tao Li
> Assignee: Tao Li
> Attachments: HIVE-17496.1.patch, HIVE-17496.2.patch, HIVE-17496.3.patch, HIVE-17496.4.patch, HIVE-17496.5.patch, HIVE-17496.6.patch
>
>
> This will put more pressure on the HDFS file limit.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)