You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@submarine.apache.org by GitBox <gi...@apache.org> on 2021/07/23 17:46:27 UTC
[GitHub] [submarine] aeioulisa opened a new pull request #689: SUBMARINE-911. Fix the bug about deleting notebook
aeioulisa opened a new pull request #689:
URL: https://github.com/apache/submarine/pull/689
### What is this PR for?
Store notebook data to database.
Those data used to be saved in cache. The notebook pod have persistent volume to retain them which make the pod still working even if the server restart for some reason. But in this case, we will lose control of the pod on workbench because we don't have the data of these notebooks.
### What type of PR is it?
[Bug Fix]
### Todos
* [ ] - Task
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-911
### How should this be tested?
<!--
* First time? Setup Travis CI as described on https://submarine.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed behavior
* Outline any manual steps to test the PR here.
-->
### Screenshots (if appropriate)
### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? No
* Does this need new documentation? No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] asfgit closed pull request #689: SUBMARINE-911. Fix the bug about deleting notebook
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #689:
URL: https://github.com/apache/submarine/pull/689
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] pingsutw commented on a change in pull request #689: SUBMARINE-911. Fix the bug about deleting notebook
Posted by GitBox <gi...@apache.org>.
pingsutw commented on a change in pull request #689:
URL: https://github.com/apache/submarine/pull/689#discussion_r683152389
##########
File path: submarine-server/server-core/src/main/java/org/apache/submarine/server/notebook/NotebookManager.java
##########
@@ -118,79 +118,89 @@ public Notebook createNotebook(NotebookSpec spec) throws SubmarineRuntimeExcepti
/**
* Get a list of notebook with user id
+ *
* @param id user id
* @return a list of notebook
*/
public List<Notebook> listNotebooksByUserId(String id) {
- List<Notebook> notebookList = submitter.listNotebook(id);
- for (Notebook notebook : notebookList) {
- for (Map.Entry<String, Notebook> entry : cachedNotebookMap.entrySet()) {
- Notebook cachedNotebook = entry.getValue();
- if (cachedNotebook.getUid().equals(notebook.getUid())) {
- notebook.setNotebookId(cachedNotebook.getNotebookId());
- notebook.setSpec(cachedNotebook.getSpec());
- }
+ List<Notebook> serviceNotebooks = notebookService.selectAll();
+ List<Notebook> notebookList = new ArrayList<>();
+ for (Notebook nb : serviceNotebooks) {
+ try {
+ Notebook notebook = submitter.findNotebook(nb.getSpec());
+ notebook.setNotebookId(nb.getNotebookId());
+ notebook.setSpec(nb.getSpec());
+ notebookList.add(notebook);
+ } catch (SubmarineRuntimeException e) {
+ LOG.warn("Submitter can not find notebook: {}, will delete it", nb.getNotebookId());
+ notebookService.delete(nb.getNotebookId().toString());
+ continue;
}
}
return notebookList;
}
/**
* Get a notebook instance
+ *
* @param id notebook id
* @return object
* @throws SubmarineRuntimeException the service error
*/
public Notebook getNotebook(String id) throws SubmarineRuntimeException {
checkNotebookId(id);
- Notebook notebook = cachedNotebookMap.get(id);
- NotebookSpec spec = notebook.getSpec();
- Notebook patchNotebook = submitter.findNotebook(spec);
- notebook.rebuild(patchNotebook);
- return notebook;
+
+ Notebook notebook = notebookService.select(id);
+ Notebook foundNotebook = submitter.findNotebook(notebook.getSpec());
+ foundNotebook.rebuild(notebook);
+
+ return foundNotebook;
}
/**
* Delete the notebook instance
+ *
* @param id notebook id
* @return object
* @throws SubmarineRuntimeException the service error
*/
public Notebook deleteNotebook(String id) throws SubmarineRuntimeException {
- checkNotebookId(id);
- Notebook notebook = cachedNotebookMap.remove(id);
- NotebookSpec spec = notebook.getSpec();
- Notebook patchNotebook = submitter.deleteNotebook(spec);
+ Notebook notebook = getNotebook(id);
+ Notebook patchNotebook = submitter.deleteNotebook(notebook.getSpec());
+ notebookService.delete(id);
notebook.rebuild(patchNotebook);
return notebook;
}
/**
* Generate a unique notebook id
+ *
* @return notebook id
*/
private NotebookId generateNotebookId() {
return NotebookId.newInstance(SubmarineServer.getServerTimeStamp(),
- notebookCounter.incrementAndGet());
+ notebookCounter.incrementAndGet());
}
/**
* Check if notebook spec is valid
+ *
* @param spec notebook spec
*/
private void checkNotebookSpec(NotebookSpec spec) {
//TODO(ryan): The method need to be improved
if (spec == null) {
throw new SubmarineRuntimeException(Response.Status.OK.getStatusCode(),
- "Invalid. Notebook Spec object is null.");
+ "Invalid. Notebook Spec object is null.");
}
}
private void checkNotebookId(String id) throws SubmarineRuntimeException {
NotebookId notebookId = NotebookId.fromString(id);
- if (notebookId == null || !cachedNotebookMap.containsKey(id)) {
+ if (notebookId == null) {
Review comment:
Don't we need to check if notebookId is in the database?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] pingsutw commented on a change in pull request #689: SUBMARINE-911. Fix the bug about deleting notebook
Posted by GitBox <gi...@apache.org>.
pingsutw commented on a change in pull request #689:
URL: https://github.com/apache/submarine/pull/689#discussion_r684791079
##########
File path: submarine-server/server-core/src/main/java/org/apache/submarine/server/notebook/NotebookManager.java
##########
@@ -118,79 +118,93 @@ public Notebook createNotebook(NotebookSpec spec) throws SubmarineRuntimeExcepti
/**
* Get a list of notebook with user id
+ *
* @param id user id
* @return a list of notebook
*/
public List<Notebook> listNotebooksByUserId(String id) {
- List<Notebook> notebookList = submitter.listNotebook(id);
- for (Notebook notebook : notebookList) {
- for (Map.Entry<String, Notebook> entry : cachedNotebookMap.entrySet()) {
- Notebook cachedNotebook = entry.getValue();
- if (cachedNotebook.getUid().equals(notebook.getUid())) {
- notebook.setNotebookId(cachedNotebook.getNotebookId());
- notebook.setSpec(cachedNotebook.getSpec());
- }
+ List<Notebook> serviceNotebooks = notebookService.selectAll();
+ List<Notebook> notebookList = new ArrayList<>();
+ for (Notebook nb : serviceNotebooks) {
+ try {
+ Notebook notebook = submitter.findNotebook(nb.getSpec());
+ notebook.setNotebookId(nb.getNotebookId());
+ notebook.setSpec(nb.getSpec());
+ notebookList.add(notebook);
+ } catch (SubmarineRuntimeException e) {
+ LOG.warn("Submitter can not find notebook: {}, will delete it", nb.getNotebookId());
+ notebookService.delete(nb.getNotebookId().toString());
+ continue;
}
}
return notebookList;
}
/**
* Get a notebook instance
+ *
* @param id notebook id
* @return object
* @throws SubmarineRuntimeException the service error
*/
public Notebook getNotebook(String id) throws SubmarineRuntimeException {
checkNotebookId(id);
- Notebook notebook = cachedNotebookMap.get(id);
- NotebookSpec spec = notebook.getSpec();
- Notebook patchNotebook = submitter.findNotebook(spec);
- notebook.rebuild(patchNotebook);
- return notebook;
+
+ Notebook notebook = notebookService.select(id);
+ if (notebook == null) {
+ throw new SubmarineRuntimeException(Response.Status.NOT_FOUND.getStatusCode(),
+ "Not found notebook");
Review comment:
```suggestion
"Notebook not found.");
```
##########
File path: submarine-server/server-core/src/main/java/org/apache/submarine/server/notebook/NotebookManager.java
##########
@@ -118,79 +118,93 @@ public Notebook createNotebook(NotebookSpec spec) throws SubmarineRuntimeExcepti
/**
* Get a list of notebook with user id
+ *
* @param id user id
* @return a list of notebook
*/
public List<Notebook> listNotebooksByUserId(String id) {
- List<Notebook> notebookList = submitter.listNotebook(id);
- for (Notebook notebook : notebookList) {
- for (Map.Entry<String, Notebook> entry : cachedNotebookMap.entrySet()) {
- Notebook cachedNotebook = entry.getValue();
- if (cachedNotebook.getUid().equals(notebook.getUid())) {
- notebook.setNotebookId(cachedNotebook.getNotebookId());
- notebook.setSpec(cachedNotebook.getSpec());
- }
+ List<Notebook> serviceNotebooks = notebookService.selectAll();
+ List<Notebook> notebookList = new ArrayList<>();
+ for (Notebook nb : serviceNotebooks) {
+ try {
+ Notebook notebook = submitter.findNotebook(nb.getSpec());
+ notebook.setNotebookId(nb.getNotebookId());
+ notebook.setSpec(nb.getSpec());
+ notebookList.add(notebook);
+ } catch (SubmarineRuntimeException e) {
+ LOG.warn("Submitter can not find notebook: {}, will delete it", nb.getNotebookId());
+ notebookService.delete(nb.getNotebookId().toString());
+ continue;
}
}
return notebookList;
}
/**
* Get a notebook instance
+ *
* @param id notebook id
* @return object
* @throws SubmarineRuntimeException the service error
*/
public Notebook getNotebook(String id) throws SubmarineRuntimeException {
checkNotebookId(id);
- Notebook notebook = cachedNotebookMap.get(id);
- NotebookSpec spec = notebook.getSpec();
- Notebook patchNotebook = submitter.findNotebook(spec);
- notebook.rebuild(patchNotebook);
- return notebook;
+
+ Notebook notebook = notebookService.select(id);
+ if (notebook == null) {
+ throw new SubmarineRuntimeException(Response.Status.NOT_FOUND.getStatusCode(),
+ "Not found notebook");
+ }
+ Notebook foundNotebook = submitter.findNotebook(notebook.getSpec());
+ foundNotebook.rebuild(notebook);
+
+ return foundNotebook;
}
/**
* Delete the notebook instance
+ *
* @param id notebook id
* @return object
* @throws SubmarineRuntimeException the service error
*/
public Notebook deleteNotebook(String id) throws SubmarineRuntimeException {
- checkNotebookId(id);
- Notebook notebook = cachedNotebookMap.remove(id);
- NotebookSpec spec = notebook.getSpec();
- Notebook patchNotebook = submitter.deleteNotebook(spec);
+ Notebook notebook = getNotebook(id);
+ Notebook patchNotebook = submitter.deleteNotebook(notebook.getSpec());
+ notebookService.delete(id);
notebook.rebuild(patchNotebook);
return notebook;
}
/**
* Generate a unique notebook id
+ *
* @return notebook id
*/
private NotebookId generateNotebookId() {
return NotebookId.newInstance(SubmarineServer.getServerTimeStamp(),
- notebookCounter.incrementAndGet());
+ notebookCounter.incrementAndGet());
}
/**
* Check if notebook spec is valid
+ *
* @param spec notebook spec
*/
private void checkNotebookSpec(NotebookSpec spec) {
//TODO(ryan): The method need to be improved
if (spec == null) {
throw new SubmarineRuntimeException(Response.Status.OK.getStatusCode(),
- "Invalid. Notebook Spec object is null.");
+ "Invalid. Notebook Spec object is null.");
}
}
private void checkNotebookId(String id) throws SubmarineRuntimeException {
NotebookId notebookId = NotebookId.fromString(id);
- if (notebookId == null || !cachedNotebookMap.containsKey(id)) {
+ if (notebookId == null) {
throw new SubmarineRuntimeException(Response.Status.NOT_FOUND.getStatusCode(),
- "Not found notebook server.");
+ "Not found notebook server.");
Review comment:
```suggestion
"Notebook not found.");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] pingsutw commented on a change in pull request #689: SUBMARINE-911. Fix the bug about deleting notebook
Posted by GitBox <gi...@apache.org>.
pingsutw commented on a change in pull request #689:
URL: https://github.com/apache/submarine/pull/689#discussion_r683152389
##########
File path: submarine-server/server-core/src/main/java/org/apache/submarine/server/notebook/NotebookManager.java
##########
@@ -118,79 +118,89 @@ public Notebook createNotebook(NotebookSpec spec) throws SubmarineRuntimeExcepti
/**
* Get a list of notebook with user id
+ *
* @param id user id
* @return a list of notebook
*/
public List<Notebook> listNotebooksByUserId(String id) {
- List<Notebook> notebookList = submitter.listNotebook(id);
- for (Notebook notebook : notebookList) {
- for (Map.Entry<String, Notebook> entry : cachedNotebookMap.entrySet()) {
- Notebook cachedNotebook = entry.getValue();
- if (cachedNotebook.getUid().equals(notebook.getUid())) {
- notebook.setNotebookId(cachedNotebook.getNotebookId());
- notebook.setSpec(cachedNotebook.getSpec());
- }
+ List<Notebook> serviceNotebooks = notebookService.selectAll();
+ List<Notebook> notebookList = new ArrayList<>();
+ for (Notebook nb : serviceNotebooks) {
+ try {
+ Notebook notebook = submitter.findNotebook(nb.getSpec());
+ notebook.setNotebookId(nb.getNotebookId());
+ notebook.setSpec(nb.getSpec());
+ notebookList.add(notebook);
+ } catch (SubmarineRuntimeException e) {
+ LOG.warn("Submitter can not find notebook: {}, will delete it", nb.getNotebookId());
+ notebookService.delete(nb.getNotebookId().toString());
+ continue;
}
}
return notebookList;
}
/**
* Get a notebook instance
+ *
* @param id notebook id
* @return object
* @throws SubmarineRuntimeException the service error
*/
public Notebook getNotebook(String id) throws SubmarineRuntimeException {
checkNotebookId(id);
- Notebook notebook = cachedNotebookMap.get(id);
- NotebookSpec spec = notebook.getSpec();
- Notebook patchNotebook = submitter.findNotebook(spec);
- notebook.rebuild(patchNotebook);
- return notebook;
+
+ Notebook notebook = notebookService.select(id);
+ Notebook foundNotebook = submitter.findNotebook(notebook.getSpec());
+ foundNotebook.rebuild(notebook);
+
+ return foundNotebook;
}
/**
* Delete the notebook instance
+ *
* @param id notebook id
* @return object
* @throws SubmarineRuntimeException the service error
*/
public Notebook deleteNotebook(String id) throws SubmarineRuntimeException {
- checkNotebookId(id);
- Notebook notebook = cachedNotebookMap.remove(id);
- NotebookSpec spec = notebook.getSpec();
- Notebook patchNotebook = submitter.deleteNotebook(spec);
+ Notebook notebook = getNotebook(id);
+ Notebook patchNotebook = submitter.deleteNotebook(notebook.getSpec());
+ notebookService.delete(id);
notebook.rebuild(patchNotebook);
return notebook;
}
/**
* Generate a unique notebook id
+ *
* @return notebook id
*/
private NotebookId generateNotebookId() {
return NotebookId.newInstance(SubmarineServer.getServerTimeStamp(),
- notebookCounter.incrementAndGet());
+ notebookCounter.incrementAndGet());
}
/**
* Check if notebook spec is valid
+ *
* @param spec notebook spec
*/
private void checkNotebookSpec(NotebookSpec spec) {
//TODO(ryan): The method need to be improved
if (spec == null) {
throw new SubmarineRuntimeException(Response.Status.OK.getStatusCode(),
- "Invalid. Notebook Spec object is null.");
+ "Invalid. Notebook Spec object is null.");
}
}
private void checkNotebookId(String id) throws SubmarineRuntimeException {
NotebookId notebookId = NotebookId.fromString(id);
- if (notebookId == null || !cachedNotebookMap.containsKey(id)) {
+ if (notebookId == null) {
Review comment:
Don't we need to check if notebookId is in the database?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org