You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "sandugood (via GitHub)" <gi...@apache.org> on 2026/06/07 12:18:19 UTC
[PR] feat(TUI): enable different plan rendering format for stages [datafusion-ballista]
sandugood opened a new pull request, #1828:
URL: https://github.com/apache/datafusion-ballista/pull/1828
# Which issue does this PR close?
Adresses #1807
# Rationale for this change
`Scheduler`'s REST API interface now supports different styles of plan's rendering for stages. Beforehand, in the Ballista's TUI we could only render the "default" looking plan.
# What changes are included in this PR?
Switching between different formatting styles for stage's plan.
# Are there any user-facing changes?
Yes.
Now we have three possible variants of stage's plan formatting:
1. Default (which is enabled by default when pressing `p` in the job's Stages section. Also we can switch back to it by pressing `d`
2. Tree-format - by pressing `t`
3. Metrics - by pressing `m`
## Here is the demonstration:
https://github.com/user-attachments/assets/50b9844a-65c9-4a9b-b6c9-ba1c7cfea81a
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(TUI): enable various plan rendering formats for stages [datafusion-ballista]
Posted by "sandugood (via GitHub)" <gi...@apache.org>.
sandugood commented on PR #1828:
URL: https://github.com/apache/datafusion-ballista/pull/1828#issuecomment-4653683142
Here is the demonstration of both:
1. TUI
https://github.com/user-attachments/assets/62befecd-335c-4f76-ada8-f531907f4a54
2. Web TUI
https://github.com/user-attachments/assets/b585ce2c-5554-454d-981e-482057b8e10c
cc @martin-g
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(TUI): enable various plan rendering formats for stages [datafusion-ballista]
Posted by "sandugood (via GitHub)" <gi...@apache.org>.
sandugood commented on PR #1828:
URL: https://github.com/apache/datafusion-ballista/pull/1828#issuecomment-4643199205
> When physica plan shown for a specific job can we press t it similar to show it as a tree ?
Yeah, going to add it to the job list plan as well. So users might switch physical plan rendering between "default" and "tree"
After comitting going to show a video snip as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(TUI): enable various plan rendering formats [datafusion-ballista]
Posted by "martin-g (via GitHub)" <gi...@apache.org>.
martin-g commented on PR #1828:
URL: https://github.com/apache/datafusion-ballista/pull/1828#issuecomment-4659629818
Thank you, @sandugood !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(TUI): enable various plan rendering formats for stages [datafusion-ballista]
Posted by "milenkovicm (via GitHub)" <gi...@apache.org>.
milenkovicm commented on PR #1828:
URL: https://github.com/apache/datafusion-ballista/pull/1828#issuecomment-4642905910
Can we change to tree on job plan as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(TUI): enable various plan rendering formats [datafusion-ballista]
Posted by "martin-g (via GitHub)" <gi...@apache.org>.
martin-g merged PR #1828:
URL: https://github.com/apache/datafusion-ballista/pull/1828
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(TUI): enable various plan rendering formats for stages [datafusion-ballista]
Posted by "sandugood (via GitHub)" <gi...@apache.org>.
sandugood commented on PR #1828:
URL: https://github.com/apache/datafusion-ballista/pull/1828#issuecomment-4642949842
Right now this is set via config.
Also, when pressing `p` in the main `Jobs` menu (with all of the jobs listed), the popup window already shows:
- stage plan
- physical plan
- logical plan
Not sure if we there is really room for another window (that would be switching between default stage plan and tree-style stage plan), but I'm opened to suggestions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(TUI): enable various plan rendering formats for stages [datafusion-ballista]
Posted by "martin-g (via GitHub)" <gi...@apache.org>.
martin-g commented on code in PR #1828:
URL: https://github.com/apache/datafusion-ballista/pull/1828#discussion_r3370013969
##########
ballista-cli/src/tui/app.rs:
##########
@@ -221,13 +221,47 @@ impl App {
_ => {}
}
} else if popup.is_plan_view() {
- match key.code {
- KeyCode::Esc => popup.set_no_details_view(),
- KeyCode::Up => popup.scroll_up(),
- KeyCode::Down => popup.scroll_down(),
- KeyCode::Left => popup.scroll_left(),
- KeyCode::Right => popup.scroll_right(),
- _ => {}
+ use crate::tui::domain::jobs::stages::StagePlanTab;
+
+ // Collect job_id + tab to fetch while popup is still borrowed,
+ // then drop the borrow before the async call.
+ let fetch = match key.code {
+ KeyCode::Char('d') => popup
Review Comment:
You will have to do the same for the Web TUI.
At the moment the key event handing is duplicated due to differences between Crossterm/Ratzilla event handling (async/sync) :-/
##########
ballista-cli/src/tui/app.rs:
##########
@@ -797,6 +831,11 @@ impl App {
UiData::JobStagesData(job_id, stages) => {
self.job_stages_popup = Some(JobStagesPopup::new(job_id, stages));
}
+ UiData::JobStagesPlanData(_job_id, tab, stages) => {
Review Comment:
I was going to suggest to not pass `job_id` if it won't be used but I think it might be better to check that it is the id of the currently edited job. To prevent a late Ajax response modifying something unrelated.
##########
ballista-cli/src/tui/domain/jobs/stages.rs:
##########
@@ -102,6 +122,36 @@ impl JobStagesPopup {
}
}
+ pub fn cache_plan_response(&mut self, fmt: StagePlanTab, resp: JobStagesResponse) {
+ match fmt {
+ StagePlanTab::Default => self.plan_cache.default = Some(resp.clone()),
+ StagePlanTab::Tree => self.plan_cache.tree = Some(resp.clone()),
+ StagePlanTab::Metrics => self.plan_cache.metrics = Some(resp.clone()),
+ }
+ // If we're currently on that tab, update the live stages too so the
+ // selection / scroll state is preserved.
+ let active_fmt = self.active_plan_format();
+ if active_fmt == Some(fmt) {
+ self.stages = resp;
+ }
+ }
Review Comment:
The cloning can be avoided if the tab is not the current active one:
```suggestion
pub fn cache_plan_response(&mut self, fmt: StagePlanTab, resp: JobStagesResponse) {
let active_fmt = self.active_plan_format();
if active_fmt.as_ref() == Some(&fmt) {
self.stages = resp.clone();
}
match fmt {
StagePlanTab::Default => self.plan_cache.default = Some(resp),
StagePlanTab::Tree => self.plan_cache.tree = Some(resp),
StagePlanTab::Metrics => self.plan_cache.metrics = Some(resp),
}
}
```
##########
ballista-cli/src/tui/domain/jobs/stages.rs:
##########
@@ -92,6 +108,10 @@ impl JobStagesPopup {
Self {
job_id,
scrollbar_state: ScrollbarState::new(stages.stages.len()),
+ plan_cache: PlanCache {
+ default: Some(stages.clone()),
Review Comment:
This is not always true.
There is an TUI config `job.stage.plan.tree` that may say that the `tree` rendering is the default.
We can drop the config now since it will be very easy to change the plan rendering type.
##########
ballista-cli/src/tui/ui/main/jobs/mod.rs:
##########
@@ -102,6 +103,40 @@ pub async fn load_job_stages_popup(app: &App, job_id: &str) -> TuiResult<()> {
.await
}
+/// Loading stage's plan to render the popup window
+#[cfg(not(feature = "web"))]
+pub async fn load_stage_plan(
+ app: &App,
+ job_id: &str,
+ tab: crate::tui::domain::jobs::stages::StagePlanTab,
+) -> TuiResult<()> {
+ use crate::tui::domain::jobs::stages::StagePlanTab;
+
+ let fmt = match tab {
+ StagePlanTab::Default => None,
Review Comment:
```suggestion
StagePlanTab::Default => Some(""),
```
or `Some("default")`
to bypass the TUI config setting `job.stage.plan.tree`
##########
ballista-cli/src/tui/domain/jobs/stages.rs:
##########
@@ -102,6 +122,36 @@ impl JobStagesPopup {
}
}
+ pub fn cache_plan_response(&mut self, fmt: StagePlanTab, resp: JobStagesResponse) {
+ match fmt {
+ StagePlanTab::Default => self.plan_cache.default = Some(resp.clone()),
+ StagePlanTab::Tree => self.plan_cache.tree = Some(resp.clone()),
+ StagePlanTab::Metrics => self.plan_cache.metrics = Some(resp.clone()),
+ }
+ // If we're currently on that tab, update the live stages too so the
+ // selection / scroll state is preserved.
+ let active_fmt = self.active_plan_format();
+ if active_fmt == Some(fmt) {
+ self.stages = resp;
+ }
+ }
+
+ #[allow(dead_code)]
+ pub fn cached_response(&self, tab: &StagePlanTab) -> Option<JobStagesResponse> {
+ match tab {
+ StagePlanTab::Default => self.plan_cache.default.clone(),
+ StagePlanTab::Tree => self.plan_cache.tree.clone(),
+ StagePlanTab::Metrics => self.plan_cache.metrics.clone(),
+ }
+ }
Review Comment:
Better return a reference
```suggestion
pub fn cached_response(&self, tab: &StagePlanTab) -> Option<&JobStagesResponse> {
match tab {
StagePlanTab::Default => self.plan_cache.default.as_ref(),
StagePlanTab::Tree => self.plan_cache.tree.as_ref(),
StagePlanTab::Metrics => self.plan_cache.metrics.as_ref(),
}
}
```
##########
ballista-cli/src/tui/ui/footer.rs:
##########
@@ -80,6 +80,12 @@ pub(super) fn render_footer(f: &mut Frame, area: Rect, app: &App) {
} else if app.is_job_stage_plan_popup_open() {
current_view_key_bindings
.push(Span::from("[↑↓] Scroll up/down, "));
Review Comment:
I see we missed to add the horizontal scrolling binding here.
Please add it too!
```suggestion
.push(Span::from("[↑↓←→] Scroll, "));
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(TUI): enable various plan rendering formats for stages [datafusion-ballista]
Posted by "sandugood (via GitHub)" <gi...@apache.org>.
sandugood commented on PR #1828:
URL: https://github.com/apache/datafusion-ballista/pull/1828#issuecomment-4644260023
Thanks for review @martin-g
Converting to draft and going to tackle it in subsequent days
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(TUI): enable various plan rendering formats for stages [datafusion-ballista]
Posted by "sandugood (via GitHub)" <gi...@apache.org>.
sandugood commented on code in PR #1828:
URL: https://github.com/apache/datafusion-ballista/pull/1828#discussion_r3376433840
##########
ballista-cli/src/tui/domain/jobs/stages.rs:
##########
@@ -92,6 +108,10 @@ impl JobStagesPopup {
Self {
job_id,
scrollbar_state: ScrollbarState::new(stages.stages.len()),
+ plan_cache: PlanCache {
+ default: Some(stages.clone()),
Review Comment:
Removed the config fully, because now user has the possibility to switch between different styles of physical plan formatting.
Note: I am using the `default` as a naming convention (like default-style rendering of physical plan) and because pressing `d` on the keyboard seemed like a good choice
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(TUI): enable various plan rendering formats for stages [datafusion-ballista]
Posted by "sandugood (via GitHub)" <gi...@apache.org>.
sandugood commented on PR #1828:
URL: https://github.com/apache/datafusion-ballista/pull/1828#issuecomment-4653700572
> The banner text does not render well for you for some reason.
By the way, have no idea why it renders gibberish on my Ubuntu :slightly_smiling_face:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(TUI): enable various plan rendering formats for stages [datafusion-ballista]
Posted by "milenkovicm (via GitHub)" <gi...@apache.org>.
milenkovicm commented on PR #1828:
URL: https://github.com/apache/datafusion-ballista/pull/1828#issuecomment-4643005246
When physica plan shown for a specific job can we press t it similar to show it as a tree ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(TUI): enable various plan rendering formats for stages [datafusion-ballista]
Posted by "sandugood (via GitHub)" <gi...@apache.org>.
sandugood commented on code in PR #1828:
URL: https://github.com/apache/datafusion-ballista/pull/1828#discussion_r3376414394
##########
ballista-cli/src/tui/app.rs:
##########
@@ -797,6 +831,11 @@ impl App {
UiData::JobStagesData(job_id, stages) => {
self.job_stages_popup = Some(JobStagesPopup::new(job_id, stages));
}
+ UiData::JobStagesPlanData(_job_id, tab, stages) => {
Review Comment:
You are right, it won't be used
Removed for now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(TUI): enable various plan rendering formats for stages [datafusion-ballista]
Posted by "martin-g (via GitHub)" <gi...@apache.org>.
martin-g commented on PR #1828:
URL: https://github.com/apache/datafusion-ballista/pull/1828#issuecomment-4643771740
The banner text does not render well for you for some reason/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org