You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2023/12/17 21:22:31 UTC
(spark) branch master updated: [SPARK-46389][CORE] Manually close the `RocksDB/LevelDB` instance when `checkVersion` throw Exception
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 5e1b904ca54f [SPARK-46389][CORE] Manually close the `RocksDB/LevelDB` instance when `checkVersion` throw Exception
5e1b904ca54f is described below
commit 5e1b904ca54f8eddc5315933e43edc8bdd0d2982
Author: yangjie01 <ya...@baidu.com>
AuthorDate: Sun Dec 17 13:22:13 2023 -0800
[SPARK-46389][CORE] Manually close the `RocksDB/LevelDB` instance when `checkVersion` throw Exception
### What changes were proposed in this pull request?
In the process of initializing the `DB` in `RocksDBProvider/LevelDBProvider`, there is a `checkVersion` step that may throw an exception. After the exception is thrown, the upper-level caller cannot hold the already opened `RockDB/LevelDB` instance, so it cannot perform resource cleanup, which poses a potential risk of handle leakage. So this PR manually closes the `RocksDB/LevelDB` instance when `checkVersion` throws an exception.
### Why are the changes needed?
Should close the `RocksDB/LevelDB` instance when `checkVersion` throw Exception
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Pass GitHub Actions
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #44327 from LuciferYang/SPARK-46389.
Authored-by: yangjie01 <ya...@baidu.com>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
.../main/java/org/apache/spark/network/util/LevelDBProvider.java | 7 ++++++-
.../main/java/org/apache/spark/network/util/RocksDBProvider.java | 4 ++++
2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/common/network-common/src/main/java/org/apache/spark/network/util/LevelDBProvider.java b/common/network-common/src/main/java/org/apache/spark/network/util/LevelDBProvider.java
index b27e3beb77ef..aa8be0c663bc 100644
--- a/common/network-common/src/main/java/org/apache/spark/network/util/LevelDBProvider.java
+++ b/common/network-common/src/main/java/org/apache/spark/network/util/LevelDBProvider.java
@@ -80,7 +80,12 @@ public class LevelDBProvider {
}
}
// if there is a version mismatch, we throw an exception, which means the service is unusable
- checkVersion(tmpDb, version, mapper);
+ try {
+ checkVersion(tmpDb, version, mapper);
+ } catch (IOException ioe) {
+ tmpDb.close();
+ throw ioe;
+ }
}
return tmpDb;
}
diff --git a/common/network-common/src/main/java/org/apache/spark/network/util/RocksDBProvider.java b/common/network-common/src/main/java/org/apache/spark/network/util/RocksDBProvider.java
index f1f702c44245..f3b7b48355a0 100644
--- a/common/network-common/src/main/java/org/apache/spark/network/util/RocksDBProvider.java
+++ b/common/network-common/src/main/java/org/apache/spark/network/util/RocksDBProvider.java
@@ -100,7 +100,11 @@ public class RocksDBProvider {
// is unusable
checkVersion(tmpDb, version, mapper);
} catch (RocksDBException e) {
+ tmpDb.close();
throw new IOException(e.getMessage(), e);
+ } catch (IOException ioe) {
+ tmpDb.close();
+ throw ioe;
}
}
return tmpDb;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org