You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2023/10/18 07:16:54 UTC
[spark] branch master updated: [SPARK-45542][CORE] Replace `setSafeMode(HdfsConstants.SafeModeAction, boolean)` with `setSafeMode(SafeModeAction, boolean)`
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 5d5a9dbfba18 [SPARK-45542][CORE] Replace `setSafeMode(HdfsConstants.SafeModeAction, boolean)` with `setSafeMode(SafeModeAction, boolean)`
5d5a9dbfba18 is described below
commit 5d5a9dbfba1831d1aafcc47a82a25df02e431749
Author: yangjie01 <ya...@baidu.com>
AuthorDate: Wed Oct 18 00:16:43 2023 -0700
[SPARK-45542][CORE] Replace `setSafeMode(HdfsConstants.SafeModeAction, boolean)` with `setSafeMode(SafeModeAction, boolean)`
### What changes were proposed in this pull request?
The function `FsHistoryProvider#isFsInSafeMode` is using the deprecated API
https://github.com/apache/hadoop/blob/1be78238728da9266a4f88195058f08fd012bf9c/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java#L1702-L1722
```
/**
* Enter, leave or get safe mode.
*
* param action
* One of SafeModeAction.ENTER, SafeModeAction.LEAVE and
* SafeModeAction.GET.
* param isChecked
* If true check only for Active NNs status, else check first NN's
* status.
*
* see org.apache.hadoop.hdfs.protocol.ClientProtocol#setSafeMode(HdfsConstants.SafeModeAction,
* boolean)
*
* deprecated please instead use
* {link DistributedFileSystem#setSafeMode(SafeModeAction, boolean)}.
*/
Deprecated
public boolean setSafeMode(HdfsConstants.SafeModeAction action,
boolean isChecked) throws IOException {
return dfs.setSafeMode(action, isChecked);
}
```
This pr change to use
https://github.com/apache/hadoop/blob/1be78238728da9266a4f88195058f08fd012bf9c/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java#L1646-L1661
```
/**
* Enter, leave or get safe mode.
*
* param action
* One of SafeModeAction.ENTER, SafeModeAction.LEAVE and
* SafeModeAction.GET.
* param isChecked
* If true check only for Active NNs status, else check first NN's
* status.
*/
Override
SuppressWarnings("deprecation")
public boolean setSafeMode(SafeModeAction action, boolean isChecked)
throws IOException {
return this.setSafeMode(convertToClientProtocolSafeModeAction(action), isChecked);
}
```
instead of it.
The conversion relationship from `HdfsConstants.SafeModeAction` to `SafeModeAction` refers to `convertToClientProtocolSafeModeAction` method.
https://github.com/apache/hadoop/blob/1be78238728da9266a4f88195058f08fd012bf9c/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java#L1663-L1686
```
/**
* Translating the {link SafeModeAction} into {link HdfsConstants.SafeModeAction}
* that is used by {link DFSClient#setSafeMode(HdfsConstants.SafeModeAction, boolean)}.
*
* param action any supported action listed in {link SafeModeAction}.
* return the converted {link HdfsConstants.SafeModeAction}.
* throws UnsupportedOperationException if the provided {link SafeModeAction} cannot be
* translated.
*/
private static HdfsConstants.SafeModeAction convertToClientProtocolSafeModeAction(
SafeModeAction action) {
switch (action) {
case ENTER:
return HdfsConstants.SafeModeAction.SAFEMODE_ENTER;
case LEAVE:
return HdfsConstants.SafeModeAction.SAFEMODE_LEAVE;
case FORCE_EXIT:
return HdfsConstants.SafeModeAction.SAFEMODE_FORCE_EXIT;
case GET:
return HdfsConstants.SafeModeAction.SAFEMODE_GET;
default:
throw new UnsupportedOperationException("Unsupported safe mode action " + action);
}
}
```
### Why are the changes needed?
Clean up deprecated API usage
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Pass GitHub Actions
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #43377 from LuciferYang/SPARK-45542.
Authored-by: yangjie01 <ya...@baidu.com>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
.../scala/org/apache/spark/deploy/history/FsHistoryProvider.scala | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala b/core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala
index b5afa86180b3..99b3184ac918 100644
--- a/core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala
@@ -31,9 +31,8 @@ import scala.xml.Node
import com.fasterxml.jackson.annotation.JsonIgnore
import com.fasterxml.jackson.databind.annotation.JsonDeserialize
-import org.apache.hadoop.fs.{FileStatus, FileSystem, Path}
+import org.apache.hadoop.fs.{FileStatus, FileSystem, Path, SafeModeAction}
import org.apache.hadoop.hdfs.DistributedFileSystem
-import org.apache.hadoop.hdfs.protocol.HdfsConstants
import org.apache.hadoop.security.AccessControlException
import org.apache.spark.{SecurityManager, SparkConf, SparkException}
@@ -1158,7 +1157,7 @@ private[history] class FsHistoryProvider(conf: SparkConf, clock: Clock)
private[history] def isFsInSafeMode(dfs: DistributedFileSystem): Boolean = {
/* true to check only for Active NNs status */
- dfs.setSafeMode(HdfsConstants.SafeModeAction.SAFEMODE_GET, true)
+ dfs.setSafeMode(SafeModeAction.GET, true)
}
/**
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org