You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2018/10/25 16:30:00 UTC
[jira] [Updated] (HADOOP-15880) WASB doesn't honor
fs.trash.interval and this fails to auto purge trash folder
[ https://issues.apache.org/jira/browse/HADOOP-15880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-15880:
------------------------------------
Component/s: documentation
> WASB doesn't honor fs.trash.interval and this fails to auto purge trash folder
> ------------------------------------------------------------------------------
>
> Key: HADOOP-15880
> URL: https://issues.apache.org/jira/browse/HADOOP-15880
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation, fs/azure
> Affects Versions: 2.7.3
> Environment: Any HDInsigth cluster pointing to WASB.
> Reporter: Sunil Kumar Chakrapani
> Priority: Minor
> Labels: WASB
>
> when "fs.trash.interval" is set to a value, trash for the local hdfs got cleared where as the trash folder on WASB doesn't get deleted and the files get piled up on WASB store..
> WASB doesn't pick up fs.trash.interval value and this fails to auto purge trash folder on WASB store.
>
> *Issue : WASB doesn't honor fs.trash.interval and this fails to auto purge trash folder*
> *Steps to reproduce Scenario:*
> *Delete any file stored on HDFS*
> hdfs dfs -D "fs.default.name=hdfs://mycluster/" -rm /hivestore.txt
> 18/10/23 06:18:05 INFO fs.TrashPolicyDefault: Moved: 'hdfs://mycluster/hivestore.txt' to trash at: hdfs://mycluster/user/sshuser/.Trash/Current/hivestore.txt
> *When deleted the file is moved to trash folder*
> hdfs dfs -rm wasb:///hivestore.txt
> 18/10/23 06:19:13 INFO fs.TrashPolicyDefault: Moved: 'wasb://kcspark-2018-10-18t17-07-40-524z@kcdnsproxy.blob.core.windows.net/hivestore.txt' to trash at: wasb://kcspark-2018-10-18t17-07-40-524z@kcdnsproxy.blob.core.windows.net/user/sshuser/.Trash/Current/hivestore.txt
> *Reduced the fs.trash.interval from 360 to 1 and restarted all related services.*
> *Trash for the local hdfs gets cleared honoring the "fs.trash.interval" value.*
> hdfs dfs -D "fs.default.name=hdfs://mycluster/" -ls hdfs://mycluster/user/sshuser/.Trash/Current/
> ls: File hdfs://mycluster/user/sshuser/.Trash/Current does not exist.
> *Where as the trash for WASB doesn't get cleared.*
> hdfs dfs -ls wasb://kcspark-2018-10-18t17-07-40-524z@kcdnsproxy.blob.core.windows.net/user/sshuser/.Trash/Current/
> Found 1 items
> -rw-r--r-- 1 sshuser supergroup 1084 2018-10-23 06:19 wasb://kcspark-2018-10-18t17-07-40-524z@kcdnsproxy.blob.core.windows.net/user/sshuser/.Trash/Current/hivestore.txt
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org