You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/03/06 17:22:00 UTC
[jira] [Updated] (HADOOP-15880) WASB doesn't honor
fs.trash.interval and this fails to auto purge trash folder
[ https://issues.apache.org/jira/browse/HADOOP-15880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated HADOOP-15880:
------------------------------------
Labels: WASB pull-request-available (was: WASB)
> WASB doesn't honor fs.trash.interval and this fails to auto purge trash folder
> ------------------------------------------------------------------------------
>
> Key: HADOOP-15880
> URL: https://issues.apache.org/jira/browse/HADOOP-15880
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation, fs/azure
> Affects Versions: 2.7.3
> Environment: Any HDInsigth cluster pointing to WASB.
> Reporter: Sunil Kumar Chakrapani
> Priority: Minor
> Labels: WASB, pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> when "fs.trash.interval" is set to a value, trash for the local hdfs got cleared where as the trash folder on WASB doesn't get deleted and the files get piled up on WASB store..
> WASB doesn't pick up fs.trash.interval value and this fails to auto purge trash folder on WASB store.
>
> *Issue : WASB doesn't honor fs.trash.interval and this fails to auto purge trash folder*
> *Steps to reproduce Scenario:*
> *Delete any file stored on HDFS*
> hdfs dfs -D "fs.default.name=hdfs://mycluster/" -rm /hivestore.txt
> 18/10/23 06:18:05 INFO fs.TrashPolicyDefault: Moved: 'hdfs://mycluster/hivestore.txt' to trash at: hdfs://mycluster/user/sshuser/.Trash/Current/hivestore.txt
> *When deleted the file is moved to trash folder*
> hdfs dfs -rm wasb:///hivestore.txt
> 18/10/23 06:19:13 INFO fs.TrashPolicyDefault: Moved: 'wasb://kcspark-2018-10-18t17-07-40-524z@kcdnsproxy.blob.core.windows.net/hivestore.txt' to trash at: wasb://kcspark-2018-10-18t17-07-40-524z@kcdnsproxy.blob.core.windows.net/user/sshuser/.Trash/Current/hivestore.txt
> *Reduced the fs.trash.interval from 360 to 1 and restarted all related services.*
> *Trash for the local hdfs gets cleared honoring the "fs.trash.interval" value.*
> hdfs dfs -D "fs.default.name=hdfs://mycluster/" -ls hdfs://mycluster/user/sshuser/.Trash/Current/
> ls: File hdfs://mycluster/user/sshuser/.Trash/Current does not exist.
> *Where as the trash for WASB doesn't get cleared.*
> hdfs dfs -ls wasb://kcspark-2018-10-18t17-07-40-524z@kcdnsproxy.blob.core.windows.net/user/sshuser/.Trash/Current/
> Found 1 items
> -rw-r--r-- 1 sshuser supergroup 1084 2018-10-23 06:19 wasb://kcspark-2018-10-18t17-07-40-524z@kcdnsproxy.blob.core.windows.net/user/sshuser/.Trash/Current/hivestore.txt
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org