You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by "qinbo12 (via GitHub)" <gi...@apache.org> on 2023/02/11 07:42:31 UTC
[GitHub] [zeppelin] qinbo12 opened a new pull request, #4563: Solve the concurrency clone note
qinbo12 opened a new pull request, #4563:
URL: https://github.com/apache/zeppelin/pull/4563
This pull request migrates solve the concurrency problem caused by multiple concurrent calls to note clone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] youshaojun commented on pull request #4563: [ZEPPELIN-5885] Solve the concurrency clone note
Posted by "youshaojun (via GitHub)" <gi...@apache.org>.
youshaojun commented on PR #4563:
URL: https://github.com/apache/zeppelin/pull/4563#issuecomment-1580071482
hi~, I tried to use the above modifications, but now occasionally throw java.nio.file.NoSuchFileException
```
Caused by: java.nio.file.NoSuchFileException: /usr/local/zeppelin-0.10.1-bin-all/conf/notebook-authorization.json
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:447)
at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
at java.nio.file.Files.move(Files.java:1395)
at org.apache.zeppelin.util.FileUtils.atomicWriteToFile(FileUtils.java:60)
at org.apache.zeppelin.util.FileUtils.atomicWriteToFile(FileUtils.java:71)
at org.apache.zeppelin.storage.LocalConfigStorage.save(LocalConfigStorage.java:71)
at org.apache.zeppelin.notebook.AuthorizationService.saveNoteAuth(AuthorizationService.java:109)
at org.apache.zeppelin.notebook.Notebook.createNote(Notebook.java:258)
at org.apache.zeppelin.service.NotebookService.createNote(NotebookService.java:168)
```
whether this method requires a lock ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] youshaojun commented on pull request #4563: [ZEPPELIN-5885] Solve the concurrency clone note
Posted by "youshaojun (via GitHub)" <gi...@apache.org>.
youshaojun commented on PR #4563:
URL: https://github.com/apache/zeppelin/pull/4563#issuecomment-1580078356
hi~, I tried to use the above modifications, but now occasionally throw java.nio.file.NoSuchFileException
```
Caused by: java.nio.file.NoSuchFileException: /usr/local/zeppelin-0.10.1-bin-all/conf/notebook-authorization.json
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:447)
at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
at java.nio.file.Files.move(Files.java:1395)
at org.apache.zeppelin.util.FileUtils.atomicWriteToFile(FileUtils.java:60)
at org.apache.zeppelin.util.FileUtils.atomicWriteToFile(FileUtils.java:71)
at org.apache.zeppelin.storage.LocalConfigStorage.save(LocalConfigStorage.java:71)
at org.apache.zeppelin.notebook.AuthorizationService.saveNoteAuth(AuthorizationService.java:109)
at org.apache.zeppelin.notebook.Notebook.createNote(Notebook.java:258)
at org.apache.zeppelin.service.NotebookService.createNote(NotebookService.java:168)
```
whether this method requires a lock ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on pull request #4563: Solve the concurrency clone note
Posted by "Reamer (via GitHub)" <gi...@apache.org>.
Reamer commented on PR #4563:
URL: https://github.com/apache/zeppelin/pull/4563#issuecomment-1441617178
Please open a JIRA ticket. A stack trace of the error would be good in the ticket. It would also be good if you use the pull request template.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] youshaojun commented on pull request #4563: [ZEPPELIN-5885] Solve the concurrency clone note
Posted by "youshaojun (via GitHub)" <gi...@apache.org>.
youshaojun commented on PR #4563:
URL: https://github.com/apache/zeppelin/pull/4563#issuecomment-1581779073
Thanks for your reply.
This pull request migrates solve the concurrency problem caused by multiple concurrent calls to `org.apache.zeppelin.notebook.AuthorizationService#saveNoteAuth`, but this can result in concurrent modifications to `notebook authorization.json`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer merged pull request #4563: [ZEPPELIN-5885] Solve the concurrency clone note
Posted by "Reamer (via GitHub)" <gi...@apache.org>.
Reamer merged PR #4563:
URL: https://github.com/apache/zeppelin/pull/4563
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] qinbo12 commented on pull request #4563: [ZEPPELIN-5885] Solve the concurrency clone note
Posted by "qinbo12 (via GitHub)" <gi...@apache.org>.
qinbo12 commented on PR #4563:
URL: https://github.com/apache/zeppelin/pull/4563#issuecomment-1449392398
> Please open a JIRA ticket. A stack trace of the error would be good in the ticket. It would also be good if you use the pull request template.
Hello, I have JIRA sheet and provided test case and stack information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on pull request #4563: [ZEPPELIN-5885] Solve the concurrency clone note
Posted by "Reamer (via GitHub)" <gi...@apache.org>.
Reamer commented on PR #4563:
URL: https://github.com/apache/zeppelin/pull/4563#issuecomment-1580598263
Are you sure it is because of this change?
I think you have another concurrency problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] youshaojun commented on pull request #4563: [ZEPPELIN-5885] Solve the concurrency clone note
Posted by "youshaojun (via GitHub)" <gi...@apache.org>.
youshaojun commented on PR #4563:
URL: https://github.com/apache/zeppelin/pull/4563#issuecomment-1582052448
Mybe a synchronized at the `org.apache.zeppelin.util.FileUtils#atomicWriteToFile(java.lang.String, java.io.File, java.util.Set<java.nio.file.attribute.PosixFilePermission>)` is a better option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] youshaojun commented on pull request #4563: [ZEPPELIN-5885] Solve the concurrency clone note
Posted by "youshaojun (via GitHub)" <gi...@apache.org>.
youshaojun commented on PR #4563:
URL: https://github.com/apache/zeppelin/pull/4563#issuecomment-1582337500
Thanks for your reply. I understand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] youshaojun commented on pull request #4563: [ZEPPELIN-5885] Solve the concurrency clone note
Posted by "youshaojun (via GitHub)" <gi...@apache.org>.
youshaojun commented on PR #4563:
URL: https://github.com/apache/zeppelin/pull/4563#issuecomment-1580078089
hi~, I tried to use the above modifications, but now occasionally throw java.nio.file.NoSuchFileException
```
Caused by: java.nio.file.NoSuchFileException: /usr/local/zeppelin-0.10.1-bin-all/conf/notebook-authorization.json
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:447)
at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
at java.nio.file.Files.move(Files.java:1395)
at org.apache.zeppelin.util.FileUtils.atomicWriteToFile(FileUtils.java:60)
at org.apache.zeppelin.util.FileUtils.atomicWriteToFile(FileUtils.java:71)
at org.apache.zeppelin.storage.LocalConfigStorage.save(LocalConfigStorage.java:71)
at org.apache.zeppelin.notebook.AuthorizationService.saveNoteAuth(AuthorizationService.java:109)
at org.apache.zeppelin.notebook.Notebook.createNote(Notebook.java:258)
at org.apache.zeppelin.service.NotebookService.createNote(NotebookService.java:168)
```
whether this method requires a lock ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on pull request #4563: [ZEPPELIN-5885] Solve the concurrency clone note
Posted by "Reamer (via GitHub)" <gi...@apache.org>.
Reamer commented on PR #4563:
URL: https://github.com/apache/zeppelin/pull/4563#issuecomment-1582001696
Feel free to prepare a PullRequest with JIRA ticket. Your StackTrace should be included in the JIRA ticket.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on pull request #4563: [ZEPPELIN-5885] Solve the concurrency clone note
Posted by "Reamer (via GitHub)" <gi...@apache.org>.
Reamer commented on PR #4563:
URL: https://github.com/apache/zeppelin/pull/4563#issuecomment-1581919786
A `synchronized` at the `saveNoteAuth` method should fix the problem. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] youshaojun commented on pull request #4563: [ZEPPELIN-5885] Solve the concurrency clone note
Posted by "youshaojun (via GitHub)" <gi...@apache.org>.
youshaojun commented on PR #4563:
URL: https://github.com/apache/zeppelin/pull/4563#issuecomment-1581997675
Yes, i also think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on pull request #4563: [ZEPPELIN-5885] Solve the concurrency clone note
Posted by "Reamer (via GitHub)" <gi...@apache.org>.
Reamer commented on PR #4563:
URL: https://github.com/apache/zeppelin/pull/4563#issuecomment-1582193016
I don't think so, because this function is used by all save operations. e.g. when note files are created. These would then block each other.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org