You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by GitBox <gi...@apache.org> on 2020/02/17 19:50:19 UTC
[GitHub] [zeppelin] Leemoonsoo opened a new pull request #3654:
[ZEPPELIN-4619] Run a note from the commandline
Leemoonsoo opened a new pull request #3654: [ZEPPELIN-4619] Run a note from the commandline
URL: https://github.com/apache/zeppelin/pull/3654
### What is this PR for?
This PR adds a command-line option to run a single note.
The code is pickled from https://github.com/apache/zeppelin/pull/3356.
Usage is
```
bin/zeppelin.sh --run <noteId>
```
The command will launch a Zeppelin server and run a given note, and terminate on after all paragraph's successful run (with exit 0) or terminate on any paragraph error (with exit 1).
### What type of PR is it?
Feature
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4619
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] Leemoonsoo commented on a change in pull request #3654:
[ZEPPELIN-4619] Run a note from the commandline
Posted by GitBox <gi...@apache.org>.
Leemoonsoo commented on a change in pull request #3654: [ZEPPELIN-4619] Run a note from the commandline
URL: https://github.com/apache/zeppelin/pull/3654#discussion_r385456388
##########
File path: zeppelin-server/src/main/java/org/apache/zeppelin/service/NotebookService.java
##########
@@ -344,19 +349,42 @@ public boolean runParagraph(String noteId,
}
}
- public void runAllParagraphs(String noteId,
+ public boolean runAllParagraphs(String noteId,
+ ServiceContext context,
+ ServiceCallback<Paragraph> callback) throws IOException {
+ Note note = notebook.getNote(noteId);
+ if (note == null) {
+ callback.onFailure(new NoteNotFoundException(noteId), context);
+ return false;
+ }
+
+ List<Paragraph> paragraphs = note.getParagraphs();
+ List<Map<String, Object>> rawParagraphs = paragraphs.stream()
+ .map(p ->
+ (Map<String, Object>) gson.fromJson(gson.toJson(p), new TypeToken<Map<String, Object>>(){}.getType()))
+ .collect(Collectors.toList());
+
+ return runAllParagraphs(
+ noteId,
+ rawParagraphs,
+ context,
+ callback
+ );
+ }
+
+ public boolean runAllParagraphs(String noteId,
Review comment:
Addressed the comment. Please take a look updated code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] zjffdu commented on issue #3654: [ZEPPELIN-4619] Run a
note from the commandline
Posted by GitBox <gi...@apache.org>.
zjffdu commented on issue #3654: [ZEPPELIN-4619] Run a note from the commandline
URL: https://github.com/apache/zeppelin/pull/3654#issuecomment-592266051
Thanks @Leemoonsoo LGTM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] zjffdu commented on a change in pull request #3654:
[ZEPPELIN-4619] Run a note from the commandline
Posted by GitBox <gi...@apache.org>.
zjffdu commented on a change in pull request #3654: [ZEPPELIN-4619] Run a note from the commandline
URL: https://github.com/apache/zeppelin/pull/3654#discussion_r381711537
##########
File path: zeppelin-server/src/main/java/org/apache/zeppelin/service/NotebookService.java
##########
@@ -344,19 +349,42 @@ public boolean runParagraph(String noteId,
}
}
- public void runAllParagraphs(String noteId,
+ public boolean runAllParagraphs(String noteId,
+ ServiceContext context,
+ ServiceCallback<Paragraph> callback) throws IOException {
+ Note note = notebook.getNote(noteId);
+ if (note == null) {
+ callback.onFailure(new NoteNotFoundException(noteId), context);
+ return false;
+ }
+
+ List<Paragraph> paragraphs = note.getParagraphs();
+ List<Map<String, Object>> rawParagraphs = paragraphs.stream()
+ .map(p ->
+ (Map<String, Object>) gson.fromJson(gson.toJson(p), new TypeToken<Map<String, Object>>(){}.getType()))
+ .collect(Collectors.toList());
+
+ return runAllParagraphs(
+ noteId,
+ rawParagraphs,
+ context,
+ callback
+ );
+ }
+
+ public boolean runAllParagraphs(String noteId,
Review comment:
How about using the existing runAllParagraphs, and check whether `paragraphs`. If it is empty, then use the paragraph in backend, otherwise use the paragraphs from frontend.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] Leemoonsoo commented on issue #3654: [ZEPPELIN-4619] Run
a note from the commandline
Posted by GitBox <gi...@apache.org>.
Leemoonsoo commented on issue #3654: [ZEPPELIN-4619] Run a note from the commandline
URL: https://github.com/apache/zeppelin/pull/3654#issuecomment-592267309
I removed a useless test. Thanks, @zjffdu for the review. I'll merge to master.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] asfgit closed pull request #3654: [ZEPPELIN-4619] Run a
note from the commandline
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #3654: [ZEPPELIN-4619] Run a note from the commandline
URL: https://github.com/apache/zeppelin/pull/3654
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] zjffdu commented on a change in pull request #3654:
[ZEPPELIN-4619] Run a note from the commandline
Posted by GitBox <gi...@apache.org>.
zjffdu commented on a change in pull request #3654: [ZEPPELIN-4619] Run a note from the commandline
URL: https://github.com/apache/zeppelin/pull/3654#discussion_r380417760
##########
File path: zeppelin-server/src/main/java/org/apache/zeppelin/service/NotebookService.java
##########
@@ -344,6 +344,42 @@ public boolean runParagraph(String noteId,
}
}
+ public boolean runNote(String noteId,
Review comment:
You can use `runAllParagraphs` https://github.com/apache/zeppelin/blob/master/zeppelin-server/src/main/java/org/apache/zeppelin/service/NotebookService.java#L347
Or maybe rename it to `runNote`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] zjffdu commented on a change in pull request #3654:
[ZEPPELIN-4619] Run a note from the commandline
Posted by GitBox <gi...@apache.org>.
zjffdu commented on a change in pull request #3654: [ZEPPELIN-4619] Run a note from the commandline
URL: https://github.com/apache/zeppelin/pull/3654#discussion_r381711537
##########
File path: zeppelin-server/src/main/java/org/apache/zeppelin/service/NotebookService.java
##########
@@ -344,19 +349,42 @@ public boolean runParagraph(String noteId,
}
}
- public void runAllParagraphs(String noteId,
+ public boolean runAllParagraphs(String noteId,
+ ServiceContext context,
+ ServiceCallback<Paragraph> callback) throws IOException {
+ Note note = notebook.getNote(noteId);
+ if (note == null) {
+ callback.onFailure(new NoteNotFoundException(noteId), context);
+ return false;
+ }
+
+ List<Paragraph> paragraphs = note.getParagraphs();
+ List<Map<String, Object>> rawParagraphs = paragraphs.stream()
+ .map(p ->
+ (Map<String, Object>) gson.fromJson(gson.toJson(p), new TypeToken<Map<String, Object>>(){}.getType()))
+ .collect(Collectors.toList());
+
+ return runAllParagraphs(
+ noteId,
+ rawParagraphs,
+ context,
+ callback
+ );
+ }
+
+ public boolean runAllParagraphs(String noteId,
Review comment:
How about using the existing runAllParagraphs, and check whether `paragraphs` is empty, then use the paragraph in backend, otherwise use the paragraphs from frontend.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] Leemoonsoo commented on a change in pull request #3654:
[ZEPPELIN-4619] Run a note from the commandline
Posted by GitBox <gi...@apache.org>.
Leemoonsoo commented on a change in pull request #3654: [ZEPPELIN-4619] Run a note from the commandline
URL: https://github.com/apache/zeppelin/pull/3654#discussion_r380969384
##########
File path: zeppelin-server/src/main/java/org/apache/zeppelin/service/NotebookService.java
##########
@@ -344,6 +344,42 @@ public boolean runParagraph(String noteId,
}
}
+ public boolean runNote(String noteId,
Review comment:
I modified existing `runAllParagraphs` to return boolean value depends (that will be translated to exit value of the commandline). And added overloaded method to make it run without providing paragraph data. What do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] Leemoonsoo commented on a change in pull request #3654:
[ZEPPELIN-4619] Run a note from the commandline
Posted by GitBox <gi...@apache.org>.
Leemoonsoo commented on a change in pull request #3654: [ZEPPELIN-4619] Run a note from the commandline
URL: https://github.com/apache/zeppelin/pull/3654#discussion_r380944401
##########
File path: zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java
##########
@@ -326,6 +336,46 @@ private static void initServerConnector(Server server, int port, int sslPort) {
server.addConnector(connector);
}
+ private static void runNoteOnStart(ZeppelinConfiguration conf) throws IOException, InterruptedException {
+ String noteIdToRun = conf.getNotebookRunId();
Review comment:
Log message added.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] zjffdu commented on a change in pull request #3654:
[ZEPPELIN-4619] Run a note from the commandline
Posted by GitBox <gi...@apache.org>.
zjffdu commented on a change in pull request #3654: [ZEPPELIN-4619] Run a note from the commandline
URL: https://github.com/apache/zeppelin/pull/3654#discussion_r380417269
##########
File path: zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java
##########
@@ -326,6 +336,46 @@ private static void initServerConnector(Server server, int port, int sslPort) {
server.addConnector(connector);
}
+ private static void runNoteOnStart(ZeppelinConfiguration conf) throws IOException, InterruptedException {
+ String noteIdToRun = conf.getNotebookRunId();
Review comment:
It's better to add logging here to show zeppelin server is running note after started.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services