You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by ya...@apache.org on 2023/10/18 10:53:12 UTC
[kyuubi] branch branch-1.8 updated: [KYUUBI #5451] Ignore NoSuchFileException during OperationLog.close()
This is an automated email from the ASF dual-hosted git repository.
yao pushed a commit to branch branch-1.8
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/branch-1.8 by this push:
new 160e8d7eb [KYUUBI #5451] Ignore NoSuchFileException during OperationLog.close()
160e8d7eb is described below
commit 160e8d7ebe0550eef5fcec69d0929dd35cd0a9df
Author: minyk <mi...@gmail.com>
AuthorDate: Wed Oct 18 18:52:31 2023 +0800
[KYUUBI #5451] Ignore NoSuchFileException during OperationLog.close()
### _Why are the changes needed?_
OperationLog.close() try to close BufferedReader, but it says file is missing(NoSuchFileException). This happens a lot after restarting kyuubi-server and PV is not set for `kyuubi.operation.log.dir.root`.
Logs are like this:
```
2023-10-18 04:16:34.296 ERROR KyuubiTBinaryFrontendHandler-Pool: Thread-113 org.apache.kyuubi.operation.LaunchEngine: Failed to remove corresponding log file of operation: /opt/kyuubi/work/server_operation_logs/26434238-5615-456a-aa7b-fad4dde8da27/43a9bfd4-6795-422c-8031-5409d7fe3732
java.io.IOException: Failed to remove corresponding log file of operation: /opt/kyuubi/work/server_operation_logs/26434238-5615-456a-aa7b-fad4dde8da27/43a9bfd4-6795-422c-8031-5409d7fe3732
at org.apache.kyuubi.operation.log.OperationLog.trySafely(OperationLog.scala:230) ~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at org.apache.kyuubi.operation.log.OperationLog.close(OperationLog.scala:201) ~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at org.apache.kyuubi.operation.KyuubiOperation.$anonfun$close$2(KyuubiOperation.scala:119) ~[kyuubi-server_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at org.apache.kyuubi.operation.KyuubiOperation.$anonfun$close$2$adapted(KyuubiOperation.scala:119) ~[kyuubi-server_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at scala.Option.foreach(Option.scala:407) ~[scala-library-2.12.15.jar:?]
at org.apache.kyuubi.operation.KyuubiOperation.liftedTree2$1(KyuubiOperation.scala:119) ~[kyuubi-server_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at org.apache.kyuubi.operation.KyuubiOperation.close(KyuubiOperation.scala:116) ~[kyuubi-server_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at org.apache.kyuubi.operation.OperationManager.closeOperation(OperationManager.scala:126) ~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at org.apache.kyuubi.session.AbstractSession.$anonfun$close$2(AbstractSession.scala:89) ~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at java.lang.Iterable.forEach(Iterable.java:75) ~[?:1.8.0_342]
...
Caused by: java.nio.file.NoSuchFileException: /opt/kyuubi/work/server_operation_logs/26434238-5615-456a-aa7b-fad4dde8da27/43a9bfd4-6795-422c-8031-5409d7fe3732
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) ~[?:1.8.0_342]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:1.8.0_342]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:1.8.0_342]
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) ~[?:1.8.0_342]
at java.nio.file.Files.newByteChannel(Files.java:361) ~[?:1.8.0_342]
at java.nio.file.Files.newByteChannel(Files.java:407) ~[?:1.8.0_342]
at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384) ~[?:1.8.0_342]
at java.nio.file.Files.newInputStream(Files.java:152) ~[?:1.8.0_342]
at java.nio.file.Files.newBufferedReader(Files.java:2784) ~[?:1.8.0_342]
at org.apache.kyuubi.operation.log.OperationLog.reader$lzycompute(OperationLog.scala:89) ~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at org.apache.kyuubi.operation.log.OperationLog.reader(OperationLog.scala:89) ~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at org.apache.kyuubi.operation.log.OperationLog.$anonfun$close$1(OperationLog.scala:201) ~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at org.apache.kyuubi.operation.log.OperationLog.trySafely(OperationLog.scala:221) ~[kyuubi-common_2.12-1.6.1-incubating.jar:1.6.1-incubating]
... 34 more
```
`OperationLog.trySafely()`is called by `OperationLog.close()` only, so we can ignore this exception.
This closes #5451
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [X] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes #5453 from minyk/kyuubi-5451.
Closes #5451
b95731499 [minyk] ignore NoSuchFileException during OperationLog.close()
Authored-by: minyk <mi...@gmail.com>
Signed-off-by: Kent Yao <ya...@apache.org>
(cherry picked from commit aab8144e0b88e9c7ad7a37e44a1b3d426bfe6b2e)
Signed-off-by: Kent Yao <ya...@apache.org>
---
.../src/main/scala/org/apache/kyuubi/operation/log/OperationLog.scala | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/kyuubi-common/src/main/scala/org/apache/kyuubi/operation/log/OperationLog.scala b/kyuubi-common/src/main/scala/org/apache/kyuubi/operation/log/OperationLog.scala
index 7ee803cb3..2e133df28 100644
--- a/kyuubi-common/src/main/scala/org/apache/kyuubi/operation/log/OperationLog.scala
+++ b/kyuubi-common/src/main/scala/org/apache/kyuubi/operation/log/OperationLog.scala
@@ -20,7 +20,7 @@ package org.apache.kyuubi.operation.log
import java.io.{BufferedReader, IOException}
import java.nio.ByteBuffer
import java.nio.charset.StandardCharsets
-import java.nio.file.{Files, Path, Paths}
+import java.nio.file.{Files, NoSuchFileException, Path, Paths}
import java.util.{ArrayList => JArrayList, List => JList}
import scala.collection.JavaConverters._
@@ -262,6 +262,7 @@ class OperationLog(path: Path) {
try {
f
} catch {
+ case _: NoSuchFileException =>
case e: IOException =>
// Printing log here may cause a deadlock. The lock order of OperationLog.write
// is RootLogger -> LogDivertAppender -> OperationLog. If printing log here, the