You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@zeppelin.apache.org by "miinhho (via GitHub)" <gi...@apache.org> on 2026/09/05 08:01:44 UTC
[PR] [ZEPPELIN-6659] Render streaming interpreter output in Angular New UI [zeppelin]
miinhho opened a new pull request, #5460:
URL: https://github.com/apache/zeppelin/pull/5460
### What is this PR for?
The Angular New UI declares `PARAGRAPH_APPEND_OUTPUT` and `PARAGRAPH_UPDATE_OUTPUT` in its WebSocket operation enum but does not consume either message. Interpreter output emitted while a paragraph is running
is therefore discarded until the terminal `PARAGRAPH` snapshot arrives.
This PR adds the missing consumer based on the server's actual wire contract. It declares the receive payloads in the SDK, handles both operations in `ParagraphBase`, and introduces `ParagraphOutputState` to fold UPDATE, APPEND, and terminal snapshots into the current paragraph result. APPEND data is accumulated by result index, APPEND received before a typed UPDATE is held until its type is known, and the terminal `PARAGRAPH` snapshot becomes authoritative so late frames cannot duplicate or overwrite the final result.
The PR also fixes a server-side ordering ambiguity. APPEND events were buffered for up to 100 ms by `AppendOutputRunner`, while UPDATE events bypassed that queue and could overtake an earlier APPEND. A client
cannot distinguish that delayed APPEND from one genuinely produced after the UPDATE. APPEND and UPDATE now share the same queue, with UPDATE acting as an ordering boundary: preceding APPEND chunks are flushed
before the UPDATE, and subsequent APPEND chunks remain after it.
A focused, versioned capture records the callback-order evidence and observed WebSocket sequences with paragraph streaming enabled and disabled. Reducer tests replay the capture and cover delayed-frame permutations. The client remains defensive against late frames for compatibility with older servers.
The change is limited to the standard paragraph result-rendering path. Application output rendering is unchanged.
### What type of PR is it?
Bug Fix
### Todos
* [x] Declare the APPEND and UPDATE receive payloads and their type asymmetry
* [x] Consume streaming paragraph output in the Angular New UI
* [x] Preserve APPEND and UPDATE ordering on the server
* [x] Cover coalesced, reordered, and late output events
* [x] Record and replay enabled and disabled streaming captures
* [x] Add browser coverage for output accumulation while RUNNING
* [ ] Add the post-fix streaming scenario to the notebook parity registry after apache/zeppelin#5456 lands
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-6659
The reusable WebSocket fixture infrastructure tracked by ZEPPELIN-6665 and ZEPPELIN-6671 is not available yet, so this PR includes the focused capture and ordering replay required by ZEPPELIN-6659.
The notebook parity registry is being introduced by ZEPPELIN-6660 in apache/zeppelin#5456. After that PR lands, this change will register streaming output as a separate result scenario. The scenario will require the first chunk to be visible while the paragraph is RUNNING, later chunks to accumulate rather than replace earlier output, and the terminal result to contain every chunk exactly once.
### How should this be tested?
* Run the server ordering regression test:
```bash
./mvnw test -pl zeppelin-server -Dtest=AppendOutputRunnerTest
```
* Run the frontend reducer and SDK contract tests:
```bash
cd zeppelin-web-angular
npm run test:shell -- src/app/core/paragraph-base/paragraph-output-state.spec.ts projects/zeppelin-sdk/src/interfaces/message-data-type-map.interface.spec.ts
```
* Validate the WebSocket operation contract:
```bash
npm run check:websocket-contract
```
The browser test runs a shell paragraph that prints three chunks separated by 3 and 5 seconds. It verifies that `first` is visible while the paragraph is RUNNING, `second` is appended while the paragraph
remains RUNNING, and the FINISHED result contains `first`, `second`, and `third`.
The capture replay additionally verifies coalesced APPEND chunks, APPEND before a typed UPDATE, UPDATE overtaking a queued APPEND, APPEND after the terminal `PARAGRAPH`, and terminal fallback when `zeppelin.websocket.paragraph_status_progress.enable=false`.
### Questions:
* Does the license files need to update? No.
* Is there breaking changes for older versions? No.
* Does this needs documentation? No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6659] Render streaming interpreter output in Angular New UI [zeppelin]
Posted by "voidmatcha (via GitHub)" <gi...@apache.org>.
voidmatcha merged PR #5460:
URL: https://github.com/apache/zeppelin/pull/5460
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6659] Render streaming interpreter output in Angular New UI [zeppelin]
Posted by "voidmatcha (via GitHub)" <gi...@apache.org>.
voidmatcha commented on PR #5460:
URL: https://github.com/apache/zeppelin/pull/5460#issuecomment-5652726260
Merged into master (0c35c61f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6659] Render streaming interpreter output in Angular New UI [zeppelin]
Posted by "miinhho (via GitHub)" <gi...@apache.org>.
miinhho commented on PR #5460:
URL: https://github.com/apache/zeppelin/pull/5460#issuecomment-5645308447
Thanks for the detailed review and for sharing the follow-up branch.
I rebased this PR onto the latest master and incorporated the three commits from 'fix/pr5460-output-ordering`.
For personalized mode, `onOutputUpdated` now returns before mutating the shared paragraph output buffer. This prevents ownerless streaming output from being persisted by `checkpointOutput` and exposed through
another user's paragraph. I added a regression test covering the UPDATE → checkpoint → other-user paragraph path.
I also registered the streaming scenario as `NB-PARITY-022` and tagged the existing E2E test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6659] Render streaming interpreter output in Angular New UI [zeppelin]
Posted by "miinhho (via GitHub)" <gi...@apache.org>.
miinhho commented on PR #5460:
URL: https://github.com/apache/zeppelin/pull/5460#issuecomment-5647433977
For second issue, I would like to create a separate issue and address it independently. Changing the synchronization model requires broader review of the ordering, checkpoint, and concurrency guarantees, and simply
removing `synchronized` could reintroduce the APPEND/UPDATE ordering race fixed by this PR.
One idea I currently have is to replace the single global runner with a fixed number of `noteId`-keyed single-writer lanes:
- each lane would process its events through a single-thread scheduled executor
- events for the same note would always be submitted to the same lane and remain FIFO
- APPEND events would continue to be accumulated and periodically flushed
- UPDATE, UPDATE_ALL/clear, and checkpoint operations would be submitted as ordering boundaries and wait for their lane task to complete before the RPC returns
- notes assigned to different lanes could be processed concurrently, while a fixed number of lanes would avoid per-note executor lifecycle and cleanup issues.
This is only an initial idea. I plan to create a separate issue for the bottleneck and use it to review the ordering requirements and possible implementation approaches before preparing a follow-up PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ZEPPELIN-6659] Render streaming interpreter output in Angular New UI [zeppelin]
Posted by "miinhho (via GitHub)" <gi...@apache.org>.
miinhho commented on PR #5460:
URL: https://github.com/apache/zeppelin/pull/5460#issuecomment-5647418724
Thanks for pointing these out.
For first issue, I addressed both paths mentioned in the review.
On the server side, `NotebookServer.onOutputClear` now skips ownerless clear events in personalized mode before mutating or broadcasting any shared paragraph state. Added the regression coverage verifies the case:
- the existing shared result is preserved for users cloned after the clear event
- the shared output buffer is also preserved through a later checkpoint
- the ownerless clear event is neither broadcast to the note not multicast to existing users.
This is a defensive suppression rather than full owner-aware personalized streaming. Routing the clear to a particluar user's paragraph remains part of [ZEPPELIN-6704](https://issues.apache.org/jira/browse/ZEPPELIN-6704).
On the client side, a non-empty `PARAGRAPH_UPDATE_OUTPUT` is now treated as an authoritative replacement. Any pending APPEND chunks for that result index are discarded instead of being concatenated with the
replacement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org