You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@zeppelin.apache.org by "HwangRock (via GitHub)" <gi...@apache.org> on 2026/09/05 13:16:02 UTC
[PR] [ZEPPELIN-5745] Add headless CLI to run a note without starting the server [zeppelin]
HwangRock opened a new pull request, #5461:
URL: https://github.com/apache/zeppelin/pull/5461
### What is this PR for?
Today a note can only be run through a live server — you start the web server
(Jetty/REST/WebSocket) and drive it from the browser, or call the REST API of an
already-running instance. That ties every execution to a long-lived server
process, which is awkward for cron jobs, CI checks, and batch pipelines.
This PR adds a small CLI, `bin/run-note.sh`, that runs a single note **headlessly**
— no web server at all — in the spirit of [papermill](https://papermill.readthedocs.io/)
for Jupyter. It assembles just the interpreter runtime, executes the note, saves
the result, and exits.
**Usage**
```
bin/run-note.sh -i <notePath> [-o <outputPath>] [-p <key> <value>]...
```
- `-i` — logical path of the note to run (e.g. `/Demo`, as shown in the note list). Required.
- `-o` — save the executed note to a new path instead of overwriting the input.
- `-p` — inject a parameter, substituted into `${key}` / `${key=default}` placeholders in the note. Repeatable.
Example:
```
bin/run-note.sh -i "/Python Tutorial/1. IPython Basic" -p date 2026-09-05 -o /out/report
```
**How it works**
`NotebookRunner` builds the notebook + interpreter runtime with manual dependency
injection (the same recipe the interpreter tests already use), so no Jetty/HK2 is
involved. The interpreter event server is still started — interpreters run as
separate processes and need it to call results back — but the web layer is not.
Paragraph output streams to stdout as the note runs, and the executed note is
saved through the normal `NotebookRepo`.
**Behavior worth calling out**
- **Exit code**: a paragraph left in a non-`FINISHED` state (error, abort, or
skipped after an earlier failure) fails the run with a non-zero exit, so CI/batch
callers can detect failures; a clean run exits 0.
- **Clean shutdown**: a one-shot CLI must not inherit the server's "run forever"
assumption. On exit the process stops the interpreter processes, each
`RemoteScheduler`'s own thread pool, the event server, and `ExecutorFactory`'s
pools, then calls `System.exit()` as a backstop so a stray non-daemon thread
cannot hang the JVM.
- **stdout vs stderr**: paragraph output goes to stdout (so `> file` captures only
the results); the "saved / finished" summary goes to stderr.
**Known limitation**
Notes that drive other paragraphs programmatically via `z.run()` / `z.runNote()`
are **not** supported yet — that path executes paragraphs asynchronously and can
race the run's completion check. Straight top-to-bottom notes (the common batch
case) are unaffected. This is noted in `HeadlessProcessListener` and can be a
follow-up.
### What type of PR is it?
Feature
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-5745
### How should this be tested?
**Automated** — 19 unit/integration tests under `notebook/cli/`, including two that
launch `bin/run-note.sh` in a real **separate JVM** and assert the process exits
within a timeout (guarding against the JVM-hang class of bug) with the correct exit
code (0 on success, 1 on a failed paragraph):
```
mvn -pl zeppelin-server -am test -Dtest='NotebookRunner*Test'
```
**Manual**:
```
bin/run-note.sh -i /Demo
```
runs a note end to end, prints its output, saves the result, and returns to the
shell prompt on its own.
### Screenshots (if appropriate)
### Questions:
* Does the license files need to update? No — no new dependencies are introduced.
* Is there breaking changes for older versions? No — only new files are added; no existing code is modified.
* Does this needs documentation? A short docs page for \`run-note.sh\` would help and can follow.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_015e2hvRzUM4YohAs9ibF6z6
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-5745] Add headless CLI to run a note without starting the server [zeppelin]
Posted by "HwangRock (via GitHub)" <gi...@apache.org>.
HwangRock commented on PR #5461:
URL: https://github.com/apache/zeppelin/pull/5461#issuecomment-5557919532
Thanks for good questions @jongyoul .
> why this feature is needed? E.g. specific usecases?
Right now a note can only be run on a live server. You either drive it from the browser or call the REST API of a running instance, so every run is tied to a long-lived server.
That's awkward for batch/CI. Think of a cron job, a CI check, or an Airflow task that starts one container,
runs a single note, and checks the exit code.
Standing up a full Jetty/REST/WebSocket server just to run one note is a lot of overhead. It's basically what papermill does for Jupyter.
Some earlier context:
- [ZEPPELIN-4619 ](https://issues.apache.org/jira/browse/ZEPPELIN-4619)(2020) added --run, but that boots the whole server before running the note, so it's still heavy for batch. [ZEPPELIN-5745 ](https://issues.apache.org/jira/browse/ZEPPELIN-5745)(2022) then asked to run a note without starting the server, which is what this PR does.
- [mozilla/python-zeppelin#13](https://github.com/mozilla/python-zeppelin/issues/13) asked for a CLI that runs a notebook, prints errors to stderr, saves the executed note, and returns an exit code. That's pretty much this PR, but it never got picked up.
- [nteract/papermill#206](https://github.com/nteract/papermill/issues/206) is still open, asking to run Zeppelin notes through papermill.
So the goal is a lightweight way to run a single note without a server, for batch/CI, using the exit code to tell success from failure.
> Without Zeppelin server, we cannot do it. Am I understanding correctly?
The interpreter lifecycle isn't handled by Jetty (the web server) itself, but by the InterpreterSettingManager object.
So when the CLI runs a note, it builds that object just for that single run and manages the lifecycle through it, as shown in the sequence diagram below.
<img width="1202" height="1150" alt="zeppelin2" src="https://github.com/user-attachments/assets/68ce4a5f-e59b-4254-9cea-60a03c33d8ce" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-5745] Add headless CLI to run a note without starting the server [zeppelin]
Posted by "jongyoul (via GitHub)" <gi...@apache.org>.
jongyoul commented on PR #5461:
URL: https://github.com/apache/zeppelin/pull/5461#issuecomment-5557320901
@HwangRock Could you please explain why this feature is needed? E.g. specific usecases? No server exeuction looks good, but Zeppelin focuses on manaing lifecycle for interpreters. Without Zeppelin server, we cannot do it. Am I understanding correctly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org