You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Anonymous Coward (Code Review)" <ge...@cloudera.org> on 2023/01/12 12:20:51 UTC

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

18770832838@163.com has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19417


Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................

IMPALA-11823: Add more items to Impala web UI queries page

When operating and maintaining an Impala cluster or analyzing historical
query performance, it will be helpful if we can visually see the memory
consumed, the amount of data read, and other information of the query
from the historical query page of the web UI. The current historical
query page  does not display these information, so we should count this
information when the query is executed and display it on the web page,
as shown in the attached image.

Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
---
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
M be/src/scheduling/admission-control-client.h
M be/src/scheduling/admission-control-service.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/local-admission-control-client.cc
M be/src/scheduling/local-admission-control-client.h
M be/src/scheduling/remote-admission-control-client.cc
M be/src/scheduling/remote-admission-control-client.h
M be/src/scheduling/schedule-state.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/admission_control_service.proto
M www/queries.tmpl
M www/query_detail_tabs.tmpl
M www/query_plan.tmpl
M www/query_summary.tmpl
24 files changed, 425 insertions(+), 95 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/19417/1
-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 1
Gerrit-Owner: Anonymous Coward <18...@163.com>

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 11:

(3 comments)

> Patch Set 11:
> 
> Hi, Riza, that looks really good, and I've also looked into it a little, but I'm not very good at front-end development. Maybe we can create a new Jira to track this requirement and let other professionals complete it.

In that case, can we consider saving some space by merging two columns into one?  Currently, the queries table still too wide to the right in my 1792x1120 resolution screen. More detailed comments in queries.tmpl.

Also, in www/common-header.tmpl, we currently have this:

<div class='container-fluid' style='width:80%'>

I think we should raise the width there to 90% to use some empty space in left and right margin.

http://gerrit.cloudera.org:8080/#/c/19417/11/www/queries.tmpl
File www/queries.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/11/www/queries.tmpl@162
PS11, Line 162:     <td>{{duration}}</td>
              :     <td>{{queued_duration}}</td>
nit: Merge into one column and break an entry to two lines to save space?

Duration / Queued Duration


http://gerrit.cloudera.org:8080/#/c/19417/11/www/queries.tmpl@164
PS11, Line 164:     <td>{{mem_usage}}</td>
              :     <td>{{mem_est}}</td>
nit: Merge into one column and break an entry to two lines to save space?

Mem Usage / Mem Estimate


http://gerrit.cloudera.org:8080/#/c/19417/11/www/queries.tmpl@167
PS11, Line 167: <td>{{bytes_read}}</td>
              :     <td>{{bytes_sent}}</td>
nit: Merge into one column and break an entry to two lines to save space?

Bytes Read / Bytes Sent



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 11
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 03 May 2023 00:24:13 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 1:

(15 comments)

http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/scheduling/admission-control-client.h
File be/src/scheduling/admission-control-client.h:

http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/scheduling/admission-control-client.h@48
PS1, Line 48:       std::unique_ptr<QuerySchedulePB>* schedule_result, 
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/client-request-state.h
File be/src/service/client-request-state.h:

http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/client-request-state.h@355
PS1, Line 355:     int64_t wait_end_time_ms = wait_end_time_ms_ > 0LL ? 
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/client-request-state.h@357
PS1, Line 357:     return wait_end_time_ms - wait_start_time_ms_; 
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:

http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-http-handler.cc@440
PS1, Line 440:   const string& printed_queued_duration = PrettyPrinter::Print(record.wait_time_ms, 
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-http-handler.cc@445
PS1, Line 445:   const string& printed_mem_usage = PrettyPrinter::Print(record.total_peak_mem_usage, 
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-http-handler.cc@450
PS1, Line 450:   const string& printed_mem_est = PrettyPrinter::Print(record.cluster_mem_est, 
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-http-handler.cc@471
PS1, Line 471:   const string& printed_bytes_read = PrettyPrinter::Print(record.bytes_read, 
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-server.h
File be/src/service/impala-server.h:

http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-server.h@957
PS1, Line 957:     
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-server.h@1221
PS1, Line 1221:   typedef boost::unordered_map<TUniqueId, std::shared_ptr<QueryStateRecord>*> 
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-server.h@1225
PS1, Line 1225:   /// Sets the given query_record (and retried_query_record too if given) for the given 
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-server.h@1226
PS1, Line 1226:   /// query_id. Returns an error Status if the given query_id cannot be found in the 
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-server.h@1229
PS1, Line 1229:       const TUniqueId& query_id, std::shared_ptr<QueryStateRecord>& query_record, 
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-server.cc
File be/src/service/impala-server.cc:

http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-server.cc@754
PS1, Line 754:     // The record of the retried query should always be earlier in the query log compared 
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-server.cc@755
PS1, Line 755:     // to the original query. Since the query log is a FIFO queue, this means that if the 
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19417/1/be/src/service/impala-server.cc@961
PS1, Line 961:       is_query_missing = 
line has trailing whitespace



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 1
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 12 Jan 2023 12:21:40 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Kurt Deschler, Yifan Zhang, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19417

to look at the new patch set (#9).

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................

IMPALA-11823: Add more items to Impala web UI queries page

When operating and maintaining an Impala cluster or analyzing historical
query performance, it will be helpful if we show the memory consumed,
the amount of data read, and other information of the query from the
historical query page of the web UI. The current historical query page
does not display these information, so we should count this information
when the query is executed and display it on the web page.

This patch modifies the query list page (/queries) and query detail
pages (/query_plan, etc.).
On the list page, some metrics are added for each query record,
including queuing time, memory usage, memory estimation, bytes read, and
bytes sent. In addition, the Details column now shows the query ID and
the position is adjusted to make them at the top of the record for easy
clicking.
On the query detail page, a similar record table is added to display the
key information of the current query. In addition, a timeline display is
added to the summary page (which is exactly the same as the timeline in
the profile, just for quick viewing). For queries that are running, the
above information will be automatically refreshed (only for the plan and
summary tabs).
To make it clear what each metric means, tooltips are added to all list
headers.

This patch also fixes an issue regarding invalid access to record.
Specifically, the original ImpalaServer::GetQueryRecord function cannot
guarantee that the returned query_record is always valid. After
returning, this value is not protected by query_log_lock_ and may become
an invalid iterator due to deleted entries in query_log_index_. This
patch uses shared_ptr instead of the original unique_ptr to extend the
lifetime of query_record after GetQueryRecord returns.

Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
---
M be/src/runtime/coordinator.h
M be/src/scheduling/admission-control-client.h
M be/src/scheduling/admission-control-service.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/local-admission-control-client.cc
M be/src/scheduling/local-admission-control-client.h
M be/src/scheduling/remote-admission-control-client.cc
M be/src/scheduling/remote-admission-control-client.h
M be/src/scheduling/schedule-state.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/admission_control_service.proto
M www/queries.tmpl
M www/query_detail_tabs.tmpl
M www/query_plan.tmpl
M www/query_summary.tmpl
23 files changed, 594 insertions(+), 163 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/19417/9
-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 9
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Kurt Deschler (Code Review)" <ge...@cloudera.org>.
Kurt Deschler has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 11:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/runtime/coordinator.h
File be/src/runtime/coordinator.h:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/runtime/coordinator.h@216
PS11, Line 216:     /// Note that it is the sum of peaks, not the peak of the sum.
> > This doesn't seem like a concise enough metric to present unless it
I ws concerned about skew across different fragments. I guess it's good enough. Being more concise would be difficult.


http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/client-request-state.h
File be/src/service/client-request-state.h:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/client-request-state.h@361
PS11, Line 361:     // negative if a system clock reset happened after the query was initiated.
> > Maybe better to return 0 if duration is negative?
Done


http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc@538
PS11, Line 538:   value->AddMember("mem_usage", mem_usage, document->GetAllocator());
> If we proceed with the current computation, we shouldn't call this mem_usag
Maybe est_peak_mem?


http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-server.cc
File be/src/service/impala-server.cc:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-server.cc@2374
PS11, Line 2374:   cluster_mem_est = 0;
> Move these assignments to else branch below.
Please follow up on this.



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 11
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Tue, 25 Apr 2023 01:45:28 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 10:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12581/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 10
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 08 Mar 2023 09:58:41 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................

IMPALA-11823: Add more items to Impala web UI queries page

When operating and maintaining an Impala cluster or analyzing historical
query performance, it will be helpful if we show the memory consumed,
the amount of data read, and other information of the query from the
historical query page of the web UI. The current historical query page
does not display these information, so we should count this information
when the query is executed and display it on the web page.

This patch modifies the query list page (/queries) and query detail
pages (/query_plan, etc.).
On the list page, some metrics are added for each query record,
including queuing time, memory usage, memory estimation, bytes read, and
bytes sent. In addition, the Details column now shows the query ID and
the position is adjusted to make them at the top of the record for easy
clicking.
On the query detail page, a similar record table is added to display the
key information of the current query. In addition, a timeline display is
added to the summary page (which is exactly the same as the timeline in
the profile, just for quick viewing). For queries that are running, the
above information will be automatically refreshed (only for the plan and
summary tabs).
To make it clear what each metric means, tooltips are added to all list
headers.

Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Reviewed-on: http://gerrit.cloudera.org:8080/19417
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/runtime/coordinator.h
M be/src/scheduling/admission-control-client.h
M be/src/scheduling/admission-control-service.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/local-admission-control-client.cc
M be/src/scheduling/local-admission-control-client.h
M be/src/scheduling/remote-admission-control-client.cc
M be/src/scheduling/remote-admission-control-client.h
M be/src/scheduling/schedule-state.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/admission_control_service.proto
M www/common-header.tmpl
M www/queries.tmpl
M www/query_detail_tabs.tmpl
M www/query_plan.tmpl
M www/query_summary.tmpl
24 files changed, 550 insertions(+), 107 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 15
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 13: Code-Review+2

(1 comment)

The new UI looks pretty good to me. Thanks for this awesome work, Zihao!

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc@530
PS11, Line 530: printed_queued_duration
> > nit: renaming variables of "printed_*" to "*_str" might be better,
I see. It's ok to keep the current names.



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 13
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 23 May 2023 23:18:27 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 7:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19417/7/be/src/util/runtime-profile.h
File be/src/util/runtime-profile.h:

http://gerrit.cloudera.org:8080/#/c/19417/7/be/src/util/runtime-profile.h@735
PS7, Line 735:   
line has trailing whitespace



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 7
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Tue, 28 Feb 2023 08:13:06 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 8:

(6 comments)

The portal looks pretty well now! I'm still reviewing the changes of QueryStateRecord. Post some minor comments for the UI first.

http://gerrit.cloudera.org:8080/#/c/19417/8//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19417/8//COMMIT_MSG@20
PS8, Line 20: the Action and query ID columns are merged
nit: we don't have the "query ID" column before this patch. Maybe reword to "the Details column now shows the query ID"


http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.h
File be/src/service/impala-server.h:

http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.h@1223
PS8, Line 1223: The QueryStateRecord
              :   /// value is owned by 'query_log_' so the entry in this index must be removed when
              :   /// it is removed from 'query_log_'.
Should we update this?


http://gerrit.cloudera.org:8080/#/c/19417/6/www/queries.tmpl
File www/queries.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/6/www/queries.tmpl@32
PS6, Line 32:     <th title="Canc
> > This column just has the "Action" string. Can we merge the "Query
Ah, yeah, I mixed up the "Details" and "Action" columns. Thanks for applying the change!


http://gerrit.cloudera.org:8080/#/c/19417/8/www/queries.tmpl
File www/queries.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/8/www/queries.tmpl@36
PS8, Line 36: the time submitted to Impala
nit: "the time when the query was submitted"


http://gerrit.cloudera.org:8080/#/c/19417/8/www/queries.tmpl@41
PS8, Line 41: the completed number of scans / the total number of scans
nit: "the completed number of scan ranges / the total number of scan ranges"


http://gerrit.cloudera.org:8080/#/c/19417/8/www/queries.tmpl@95
PS8, Line 95:     <th title="The scan progress of the query, i.e. the completed number of scans / the total number of scans.">Scan Progress</th>
Can we somehow deduplicate the description strings, e.g. defined as javascript variables? Otherwise, we might miss some changes when modifying the description.



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 8
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Mon, 06 Mar 2023 02:35:28 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 9:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.cc
File be/src/service/impala-server.cc:

http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.cc@823
PS8, Line 823:       // If the original profile was accessible by the user, then the retried profile
             :       // must be accessible by the user as well.
             :       status = CheckProfileAccess(user, retried_query_record->effective_user,
             :           retried_query_record->user_has_profile_access);
             :       DCHECK(status.ok());
             :       RETURN_IF_ERROR(status);
> > Any reason we need to check this again?
Sorry, I mean we already invoke CheckProfileAccess() at line 818. Do we need to call it again here?



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 9
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 08 Mar 2023 03:14:35 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Kurt Deschler, Yifan Zhang, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19417

to look at the new patch set (#4).

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................

IMPALA-11823: Add more items to Impala web UI queries page

When operating and maintaining an Impala cluster or analyzing historical
query performance, it will be helpful if we show the memory consumed,
the amount of data read, and other information of the query from the
historical query page of the web UI. The current historical query page
does not display these information, so we should count this information
when the query is executed and display it on the web page.

Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
---
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
M be/src/scheduling/admission-control-client.h
M be/src/scheduling/admission-control-service.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/local-admission-control-client.cc
M be/src/scheduling/local-admission-control-client.h
M be/src/scheduling/remote-admission-control-client.cc
M be/src/scheduling/remote-admission-control-client.h
M be/src/scheduling/schedule-state.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/admission_control_service.proto
M www/queries.tmpl
M www/query_detail_tabs.tmpl
M www/query_plan.tmpl
M www/query_summary.tmpl
24 files changed, 471 insertions(+), 95 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/19417/4
-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 4
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 10:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h
File be/src/service/impala-server.h:

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h@1220
PS6, Line 1220: tion are a
> Done
Sorry to be back on this. But can we move this into a separate patch so we can backport it to older branches like 4.1.x and 4.2.x? We need a separate JIRA as well.

This patch is mainly for adding new features to the Web UI. It seems this change is not required by the current patch. But I might miss something. If this patch does depend on the change of shared_ptr, we can rebase the current patch on top of the separated patch and merge the fix first.



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 10
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Mon, 13 Mar 2023 02:38:40 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 8:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:

http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-http-handler.cc@921
PS8, Line 921: found
nit: "found" is ambiguous. Could you also rename this to "inflight"?


http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.h
File be/src/service/impala-server.h:

http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.h@1226
PS8, Line 1226: std::shared_ptr<QueryStateRecord>*
The value type is already a pointer. Can we use "std::shared_ptr<QueryStateRecord>" directly instead of "std::shared_ptr<QueryStateRecord>*"?


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h
File be/src/service/impala-server.h:

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h@1220
PS6, Line 1220: shared_ptr
> > Could you explain why we change this to shared_ptr? After this, the
Ah, right. This patch also fixes a concurrent issue. Please mention this in the commit message as well. Otherwise, it's confusing why we need this change for the new UI. Please also update the variable comments accordingly.


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h@1234
PS6, Line 1234: &
Let's use pointer instead of reference for output arguments.


http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.cc
File be/src/service/impala-server.cc:

http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.cc@823
PS8, Line 823:       // If the original profile was accessible by the user, then the retried profile
             :       // must be accessible by the user as well.
             :       status = CheckProfileAccess(user, retried_query_record->effective_user,
             :           retried_query_record->user_has_profile_access);
             :       DCHECK(status.ok());
             :       RETURN_IF_ERROR(status);
Any reason we need to check this again?



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 8
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Mon, 06 Mar 2023 03:34:13 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 12:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12945/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 12
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Fri, 05 May 2023 08:29:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 5:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12334/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 5
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 08 Feb 2023 07:42:23 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
18770832838@163.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 6:

(11 comments)

http://gerrit.cloudera.org:8080/#/c/19417/6//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19417/6//COMMIT_MSG@15
PS6, Line 15: 
> Please also mention the changes in the summary page and the query plan page
Done


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/runtime/coordinator.h
File be/src/runtime/coordinator.h:

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/runtime/coordinator.h@215
PS6, Line 215: peak ​memory
> nit: There is an invisable character <200b> before "memory". It's invisable
Done


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/scheduling/local-admission-control-client.cc
File be/src/scheduling/local-admission-control-client.cc:

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/scheduling/local-admission-control-client.cc@47
PS6, Line 47:         request.query_id, schedule_result, 0, nullptr,
> nit: could you add some comments about the constants? So it's more readable
Done


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-http-handler.cc@930
PS6, Line 930:       query_record = make_shared<ImpalaServer::QueryStateRecord>(*query_handle);
> Do we actually use this value in the current if-clause?

Yes, query_record is defined as a null pointer above, it is initialized here for in-flight queries, and it is initialized in the if (!found) branch below for completed queries. Then, no matter which way it is initialized, it will be used in the code below, such as in QueryStateToJson.


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h
File be/src/service/impala-server.h:

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h@961
PS6, Line 961: The estimated memory usage of this query on cluster.
> nit: "The cluster wide estimated memory usage of this query"
Done


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h@1220
PS6, Line 1220: shared_ptr
> Could you explain why we change this to shared_ptr? After this, the
 > QueryStateRecord value is not owned by 'query_log_'. Do we use it
 > somewhere that we need to extend its life cycle?

Because I think the original GetQueryRecord function may cause illegal access problems, although the function locks query_log_lock_ in the execution, the query_record it provides cannot guarantee to be valid, because it is out of the protection of query_log_lock_ after returning, if query_log_index_ just deletes the corresponding record at this time, then query_record will be an invalid iterator. In order to solve this problem, I considered replacing unique_ptr with shared_ptr to extend the life cycle of GetQueryRecord function after returning.


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h@1226
PS6, Line 1226: std::shared_ptr<QueryStateRecord>*
> Can we use raw pointer directly?

I think we can't use raw pointer here in order not to break the functionality of shared_ptr.


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/util/runtime-profile.cc
File be/src/util/runtime-profile.cc:

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/util/runtime-profile.cc@1810
PS6, Line 1810: void RuntimeProfile::GetTimeline(string& output) const {
> Codes in this method duplicate those in RuntimeProfile::PrettyPrintSubclass
Done


http://gerrit.cloudera.org:8080/#/c/19417/6/common/protobuf/admission_control_service.proto
File common/protobuf/admission_control_service.proto:

http://gerrit.cloudera.org:8080/#/c/19417/6/common/protobuf/admission_control_service.proto@155
PS6, Line 155:   /// The estimated memory usage of this query on cluster.
> nit: "The cluster wide estimated memory usage of this query"
Done


http://gerrit.cloudera.org:8080/#/c/19417/6/www/queries.tmpl
File www/queries.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/6/www/queries.tmpl@32
PS6, Line 32:     <th>Action</th>
> This column just has the "Action" string. Can we merge the "Query
 > ID" column to this? i.e. replace the "Action" string with the
 > actual query id.
 > 
 > A common scenario is searching the query id in this page and click
 > the "Action" link. It'd be nice if we can click the query id
 > directly.

I think it's a great idea, but you mean "Detail" instead of "Action", right? Because "Action" is to cancel or close the query, so I merged "Detail" and "Query ID", so that clicking on the query id can jump to the details page of the query.


http://gerrit.cloudera.org:8080/#/c/19417/6/www/queries.tmpl@43
PS6, Line 43:     <th>Bytes Sent</th>
> We'd better add some tooltips to explain the columns:
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 6
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Tue, 28 Feb 2023 08:38:26 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
18770832838@163.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 11:

(6 comments)

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc@538
PS11, Line 538:   value->AddMember("mem_usage", mem_usage, document->GetAllocator());
> Sorry approx_peak_mem will be better as est usually implies not
 > measured.

Thanks for your suggestion. However, I think mem_usage is appropriate enough. It is not an estimated or approximate value, but the sum of the actual memory usage peaks of each executor. This calculation method corresponds to the memory estimation value, and it can reflect the actual memory consumption of the query on the cluster, also can be compared with the memory estimation value to see if estimation is reasonable.


http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-server.cc
File be/src/service/impala-server.cc:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-server.cc@2374
PS11, Line 2374:   cluster_mem_est = 0;
> Please follow up on this.
Done


http://gerrit.cloudera.org:8080/#/c/19417/11/www/queries.tmpl
File www/queries.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/11/www/queries.tmpl@160
PS11, Line 160:     <td>{{start_time}}</td>
              :     <td>{{end_time}}</td>
> In my local machine, both start and end time shows 9 decimal points detail
Done


http://gerrit.cloudera.org:8080/#/c/19417/11/www/queries.tmpl@162
PS11, Line 162:     <td>{{duration}}</td>
              :     <td>{{queued_duration}}</td>
> nit: Merge into one column and break an entry to two lines to save space?
Done


http://gerrit.cloudera.org:8080/#/c/19417/11/www/queries.tmpl@164
PS11, Line 164:     <td>{{mem_usage}}</td>
              :     <td>{{mem_est}}</td>
> nit: Merge into one column and break an entry to two lines to save space?
Done


http://gerrit.cloudera.org:8080/#/c/19417/11/www/queries.tmpl@167
PS11, Line 167: <td>{{bytes_read}}</td>
              :     <td>{{bytes_sent}}</td>
> nit: Merge into one column and break an entry to two lines to save space?
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 11
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Fri, 05 May 2023 08:19:14 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 14: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 14
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Wed, 24 May 2023 04:46:44 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 9:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12566/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 9
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Tue, 07 Mar 2023 09:37:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
18770832838@163.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h
File be/src/service/impala-server.h:

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h@1220
PS6, Line 1220: shared_ptr
> Sorry to be back on this. But can we move this into a separate
 > patch so we can backport it to older branches like 4.1.x and 4.2.x?
 > We need a separate JIRA as well.
 > 
 > This patch is mainly for adding new features to the Web UI. It
 > seems this change is not required by the current patch. But I might
 > miss something. If this patch does depend on the change of
 > shared_ptr, we can rebase the current patch on top of the separated
 > patch and merge the fix first.

Okay, I created a new JIRA IMPALA-11998 to track this issue specifically. Afterwards, I will separate the portion of code that solves this issue and submit a patch for it. We can then handle this patch after that is merged, right?



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 6
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Tue, 14 Mar 2023 09:42:17 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 11:

Any reason not to serve the tables in /queries with datatables?
Would be cool if we can make it nested row, so the table is not too wide:
https://datatables.net/examples/api/row_details.html

/sessions page already use datatable.
Consider pushing Duration to Resource Pool (except maybe State) to the row details?
I think those columns will not be the immediate attention when looking at query page. Datatables still provide search box that can search content in row details, I think.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 11
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 19 Apr 2023 01:20:13 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 14: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 14
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 23 May 2023 23:19:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19417

to look at the new patch set (#2).

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................

IMPALA-11823: Add more items to Impala web UI queries page

When operating and maintaining an Impala cluster or analyzing historical
query performance, it will be helpful if we can visually see the memory
consumed, the amount of data read, and other information of the query
from the historical query page of the web UI. The current historical
query page  does not display these information, so we should count this
information when the query is executed and display it on the web page,
as shown in the attached image.

Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
---
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
M be/src/scheduling/admission-control-client.h
M be/src/scheduling/admission-control-service.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/local-admission-control-client.cc
M be/src/scheduling/local-admission-control-client.h
M be/src/scheduling/remote-admission-control-client.cc
M be/src/scheduling/remote-admission-control-client.h
M be/src/scheduling/schedule-state.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/admission_control_service.proto
M www/queries.tmpl
M www/query_detail_tabs.tmpl
M www/query_plan.tmpl
M www/query_summary.tmpl
24 files changed, 427 insertions(+), 96 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/19417/2
-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Kurt Deschler, Yifan Zhang, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19417

to look at the new patch set (#5).

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................

IMPALA-11823: Add more items to Impala web UI queries page

When operating and maintaining an Impala cluster or analyzing historical
query performance, it will be helpful if we show the memory consumed,
the amount of data read, and other information of the query from the
historical query page of the web UI. The current historical query page
does not display these information, so we should count this information
when the query is executed and display it on the web page.

Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
---
M be/src/runtime/coordinator.h
M be/src/scheduling/admission-control-client.h
M be/src/scheduling/admission-control-service.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/local-admission-control-client.cc
M be/src/scheduling/local-admission-control-client.h
M be/src/scheduling/remote-admission-control-client.cc
M be/src/scheduling/remote-admission-control-client.h
M be/src/scheduling/schedule-state.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/admission_control_service.proto
M www/queries.tmpl
M www/query_detail_tabs.tmpl
M www/query_plan.tmpl
M www/query_summary.tmpl
23 files changed, 474 insertions(+), 95 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/19417/5
-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 5
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 4:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/12322/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 4
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Tue, 07 Feb 2023 09:29:33 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 6:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12335/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 6
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 08 Feb 2023 08:24:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Kurt Deschler, Yifan Zhang, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19417

to look at the new patch set (#7).

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................

IMPALA-11823: Add more items to Impala web UI queries page

When operating and maintaining an Impala cluster or analyzing historical
query performance, it will be helpful if we show the memory consumed,
the amount of data read, and other information of the query from the
historical query page of the web UI. The current historical query page
does not display these information, so we should count this information
when the query is executed and display it on the web page.

This patch modifies the query list page (/queries) and query detail
pages (/query_plan, etc.).
On the list page, some metrics are added for each query record,
including queuing time, memory usage, memory estimation, bytes read, and
bytes sent. In addition, the Action and query ID columns are merged and
the position is adjusted to make them at the top of the record for easy
clicking.
On the query detail page, a similar record table is added to display the
key information of the current query. In addition, a timeline display is
added to the summary page (which is exactly the same as the timeline in
the profile, just for quick viewing). For queries that are running, the
above information will be automatically refreshed (only for the plan and
summary tabs).
To make it clear what each metric means, tooltips are added to all list
headers.

Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
---
M be/src/runtime/coordinator.h
M be/src/scheduling/admission-control-client.h
M be/src/scheduling/admission-control-service.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/local-admission-control-client.cc
M be/src/scheduling/local-admission-control-client.h
M be/src/scheduling/remote-admission-control-client.cc
M be/src/scheduling/remote-admission-control-client.h
M be/src/scheduling/schedule-state.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/admission_control_service.proto
M www/queries.tmpl
M www/query_detail_tabs.tmpl
M www/query_plan.tmpl
M www/query_summary.tmpl
23 files changed, 511 insertions(+), 157 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/19417/7
-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 7
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12161/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 13 Jan 2023 02:35:11 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 11:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12778/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 11
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 12 Apr 2023 02:21:27 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
18770832838@163.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 11:

Hi, Riza, that looks really good, and I've also looked into it a little, but I'm not very good at front-end development. Maybe we can create a new Jira to track this requirement and let other professionals complete it.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 11
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Mon, 24 Apr 2023 12:05:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 13: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 13
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 09 May 2023 01:18:24 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 13: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 13
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 23 May 2023 16:05:35 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 14:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9342/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 14
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 23 May 2023 23:19:07 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 7:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12478/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 7
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Tue, 28 Feb 2023 08:32:11 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 12:

(1 comment)

Thank you for accepting my comments.
The query table still wide to the right in my screen, but overall looks better now.

I just have one more nit.

http://gerrit.cloudera.org:8080/#/c/19417/12/www/queries.tmpl
File www/queries.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/12/www/queries.tmpl@147
PS12, Line 147: {{duration}}<br/>{{queued_duration}}
nit: For here, title row, and other similar column below: consider add '/' before <br/> tag.

{{duration}} /<br/>{{queued_duration}}

The slash separator adds clarity that there are 2 values in one cell, much like the current "Scan Progress" column.



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 12
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Fri, 05 May 2023 21:36:01 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Kurt Deschler, Riza Suminto, Yifan Zhang, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19417

to look at the new patch set (#13).

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................

IMPALA-11823: Add more items to Impala web UI queries page

When operating and maintaining an Impala cluster or analyzing historical
query performance, it will be helpful if we show the memory consumed,
the amount of data read, and other information of the query from the
historical query page of the web UI. The current historical query page
does not display these information, so we should count this information
when the query is executed and display it on the web page.

This patch modifies the query list page (/queries) and query detail
pages (/query_plan, etc.).
On the list page, some metrics are added for each query record,
including queuing time, memory usage, memory estimation, bytes read, and
bytes sent. In addition, the Details column now shows the query ID and
the position is adjusted to make them at the top of the record for easy
clicking.
On the query detail page, a similar record table is added to display the
key information of the current query. In addition, a timeline display is
added to the summary page (which is exactly the same as the timeline in
the profile, just for quick viewing). For queries that are running, the
above information will be automatically refreshed (only for the plan and
summary tabs).
To make it clear what each metric means, tooltips are added to all list
headers.

Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
---
M be/src/runtime/coordinator.h
M be/src/scheduling/admission-control-client.h
M be/src/scheduling/admission-control-service.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/local-admission-control-client.cc
M be/src/scheduling/local-admission-control-client.h
M be/src/scheduling/remote-admission-control-client.cc
M be/src/scheduling/remote-admission-control-client.h
M be/src/scheduling/schedule-state.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/admission_control_service.proto
M www/common-header.tmpl
M www/queries.tmpl
M www/query_detail_tabs.tmpl
M www/query_plan.tmpl
M www/query_summary.tmpl
24 files changed, 550 insertions(+), 107 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/19417/13
-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 13
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 13:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12958/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 13
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Sat, 06 May 2023 07:18:11 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Kurt Deschler, Riza Suminto, Yifan Zhang, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19417

to look at the new patch set (#12).

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................

IMPALA-11823: Add more items to Impala web UI queries page

When operating and maintaining an Impala cluster or analyzing historical
query performance, it will be helpful if we show the memory consumed,
the amount of data read, and other information of the query from the
historical query page of the web UI. The current historical query page
does not display these information, so we should count this information
when the query is executed and display it on the web page.

This patch modifies the query list page (/queries) and query detail
pages (/query_plan, etc.).
On the list page, some metrics are added for each query record,
including queuing time, memory usage, memory estimation, bytes read, and
bytes sent. In addition, the Details column now shows the query ID and
the position is adjusted to make them at the top of the record for easy
clicking.
On the query detail page, a similar record table is added to display the
key information of the current query. In addition, a timeline display is
added to the summary page (which is exactly the same as the timeline in
the profile, just for quick viewing). For queries that are running, the
above information will be automatically refreshed (only for the plan and
summary tabs).
To make it clear what each metric means, tooltips are added to all list
headers.

Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
---
M be/src/runtime/coordinator.h
M be/src/scheduling/admission-control-client.h
M be/src/scheduling/admission-control-service.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/local-admission-control-client.cc
M be/src/scheduling/local-admission-control-client.h
M be/src/scheduling/remote-admission-control-client.cc
M be/src/scheduling/remote-admission-control-client.h
M be/src/scheduling/schedule-state.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/admission_control_service.proto
M www/common-header.tmpl
M www/queries.tmpl
M www/query_detail_tabs.tmpl
M www/query_plan.tmpl
M www/query_summary.tmpl
24 files changed, 514 insertions(+), 107 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/19417/12
-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 12
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12308/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Fri, 03 Feb 2023 12:53:54 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Kurt Deschler (Code Review)" <ge...@cloudera.org>.
Kurt Deschler has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 11:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/runtime/coordinator.h
File be/src/runtime/coordinator.h:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/runtime/coordinator.h@216
PS11, Line 216:     /// Note that it is the sum of peaks, not the peak of the sum.
This doesn't seem like a concise enough metric to present unless it is from a very short interval or maybe per-fragment.


http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/client-request-state.h
File be/src/service/client-request-state.h:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/client-request-state.h@361
PS11, Line 361:     // negative if a system clock reset happened after the query was initiated.
Maybe better to return 0 if duration is negative?


http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc@538
PS11, Line 538:   value->AddMember("mem_usage", mem_usage, document->GetAllocator());
If we proceed with the current computation, we shouldn't call this mem_usage.


http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-server.cc
File be/src/service/impala-server.cc:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-server.cc@2374
PS11, Line 2374:   cluster_mem_est = 0;
Move these assignments to else branch below.



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 11
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 12 Apr 2023 14:22:07 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 11: Code-Review+1

(2 comments)

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc@530
PS11, Line 530: printed_queued_duration
nit: renaming variables of "printed_*" to "*_str" might be better, e.g. "printed_queued_duration" -> "queued_duration_str"


http://gerrit.cloudera.org:8080/#/c/19417/8/www/queries.tmpl
File www/queries.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/8/www/queries.tmpl@95
PS8, Line 95:     <th title="{{tips_scan_progress}}">Scan Progress</th>
> Done. As I couldn't find a solution using JavaScript (in fact, I don't know
Ack



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 11
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 12 Apr 2023 09:14:49 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
18770832838@163.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 11:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc@530
PS11, Line 530: printed_queued_duration
> nit: renaming variables of "printed_*" to "*_str" might be better,
 > e.g. "printed_queued_duration" -> "queued_duration_str"

Oh, I see that other fields in the context are using the naming convention of "printed_*", so I followed that. Do we need to modify all of them (including the existing ones) to "*_str"?



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 11
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 12 Apr 2023 12:15:02 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Kurt Deschler, Yifan Zhang, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19417

to look at the new patch set (#11).

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................

IMPALA-11823: Add more items to Impala web UI queries page

When operating and maintaining an Impala cluster or analyzing historical
query performance, it will be helpful if we show the memory consumed,
the amount of data read, and other information of the query from the
historical query page of the web UI. The current historical query page
does not display these information, so we should count this information
when the query is executed and display it on the web page.

This patch modifies the query list page (/queries) and query detail
pages (/query_plan, etc.).
On the list page, some metrics are added for each query record,
including queuing time, memory usage, memory estimation, bytes read, and
bytes sent. In addition, the Details column now shows the query ID and
the position is adjusted to make them at the top of the record for easy
clicking.
On the query detail page, a similar record table is added to display the
key information of the current query. In addition, a timeline display is
added to the summary page (which is exactly the same as the timeline in
the profile, just for quick viewing). For queries that are running, the
above information will be automatically refreshed (only for the plan and
summary tabs).
To make it clear what each metric means, tooltips are added to all list
headers.

This patch also fixes an issue regarding invalid access to record.
Specifically, the original ImpalaServer::GetQueryRecord function cannot
guarantee that the returned query_record is always valid. After
returning, this value is not protected by query_log_lock_ and may become
an invalid iterator due to deleted entries in query_log_index_. This
patch uses shared_ptr instead of the original unique_ptr to extend the
lifetime of query_record after GetQueryRecord returns.

Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
---
M be/src/runtime/coordinator.h
M be/src/scheduling/admission-control-client.h
M be/src/scheduling/admission-control-service.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/local-admission-control-client.cc
M be/src/scheduling/local-admission-control-client.h
M be/src/scheduling/remote-admission-control-client.cc
M be/src/scheduling/remote-admission-control-client.h
M be/src/scheduling/schedule-state.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/admission_control_service.proto
M www/queries.tmpl
M www/query_detail_tabs.tmpl
M www/query_plan.tmpl
M www/query_summary.tmpl
23 files changed, 522 insertions(+), 100 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/19417/11
-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 11
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Kurt Deschler, Yifan Zhang, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19417

to look at the new patch set (#8).

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................

IMPALA-11823: Add more items to Impala web UI queries page

When operating and maintaining an Impala cluster or analyzing historical
query performance, it will be helpful if we show the memory consumed,
the amount of data read, and other information of the query from the
historical query page of the web UI. The current historical query page
does not display these information, so we should count this information
when the query is executed and display it on the web page.

This patch modifies the query list page (/queries) and query detail
pages (/query_plan, etc.).
On the list page, some metrics are added for each query record,
including queuing time, memory usage, memory estimation, bytes read, and
bytes sent. In addition, the Action and query ID columns are merged and
the position is adjusted to make them at the top of the record for easy
clicking.
On the query detail page, a similar record table is added to display the
key information of the current query. In addition, a timeline display is
added to the summary page (which is exactly the same as the timeline in
the profile, just for quick viewing). For queries that are running, the
above information will be automatically refreshed (only for the plan and
summary tabs).
To make it clear what each metric means, tooltips are added to all list
headers.

Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
---
M be/src/runtime/coordinator.h
M be/src/scheduling/admission-control-client.h
M be/src/scheduling/admission-control-service.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/local-admission-control-client.cc
M be/src/scheduling/local-admission-control-client.h
M be/src/scheduling/remote-admission-control-client.cc
M be/src/scheduling/remote-admission-control-client.h
M be/src/scheduling/schedule-state.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/admission_control_service.proto
M www/queries.tmpl
M www/query_detail_tabs.tmpl
M www/query_plan.tmpl
M www/query_summary.tmpl
23 files changed, 511 insertions(+), 157 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/19417/8
-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 8
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Kurt Deschler (Code Review)" <ge...@cloudera.org>.
Kurt Deschler has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 11:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc@538
PS11, Line 538:   value->AddMember("mem_usage", mem_usage, document->GetAllocator());
> Maybe est_peak_mem?
Sorry approx_peak_mem will be better as est usually implies not measured.



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 11
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Thu, 27 Apr 2023 15:16:56 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 6:

(12 comments)

This is a good addition to Impala which will greately improve the supportability. I still need some time to review this due to the scope of this patch.

http://gerrit.cloudera.org:8080/#/c/19417/6//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19417/6//COMMIT_MSG@15
PS6, Line 15: 
Please also mention the changes in the summary page and the query plan page.


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/runtime/coordinator.h
File be/src/runtime/coordinator.h:

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/runtime/coordinator.h@215
PS6, Line 215: peak ​memory
nit: There is an invisable character <200b> before "memory". It's invisable in Gerrit. But when opening this file in vim, it shows

 /// Total peak <200b>memory usage for this query at all backend.


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/scheduling/local-admission-control-client.cc
File be/src/scheduling/local-admission-control-client.cc:

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/scheduling/local-admission-control-client.cc@47
PS6, Line 47:         request.query_id, schedule_result, 0, nullptr,
nit: could you add some comments about the constants? So it's more readable for reviewers:

    status = ExecEnv::GetInstance()->admission_controller()->WaitOnQueued(
        request.query_id, schedule_result, /*timeout_ms*/ 0, /*wait_timed_out*/ nullptr,
        wait_start_time_ms, wait_end_time_ms);


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-http-handler.cc@930
PS6, Line 930:       query_record = make_shared<ImpalaServer::QueryStateRecord>(*query_handle);
Do we actually use this value in the current if-clause?


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h
File be/src/service/impala-server.h:

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h@961
PS6, Line 961: The estimated memory usage of this query on cluster.
nit: "The cluster wide estimated memory usage of this query"


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h@1220
PS6, Line 1220: shared_ptr
Could you explain why we change this to shared_ptr? After this, the QueryStateRecord value is not owned by 'query_log_'. Do we use it somewhere that we need to extend its life cycle?


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h@1226
PS6, Line 1226: std::shared_ptr<QueryStateRecord>*
Can we use raw pointer directly?


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/util/runtime-profile.cc
File be/src/util/runtime-profile.cc:

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/util/runtime-profile.cc@1810
PS6, Line 1810: void RuntimeProfile::GetTimeline(string& output) const {
Codes in this method duplicate those in RuntimeProfile::PrettyPrintSubclassCounters():
https://github.com/apache/impala/blob/ff7b5db6002ccb047262cd7118e2e11ab09ef40a/be/src/util/runtime-profile.cc#L1506-L1535

Could you refactor them to remove duplicated codes?


http://gerrit.cloudera.org:8080/#/c/19417/6/common/protobuf/admission_control_service.proto
File common/protobuf/admission_control_service.proto:

http://gerrit.cloudera.org:8080/#/c/19417/6/common/protobuf/admission_control_service.proto@155
PS6, Line 155:   /// The estimated memory usage of this query on cluster.
nit: "The cluster wide estimated memory usage of this query"


http://gerrit.cloudera.org:8080/#/c/19417/2/www/queries.tmpl
File www/queries.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/2/www/queries.tmpl@31
PS2, Line 31:     <th>Details</th>
            :     <th>Action</th>
> > Not sure if moving "Details" and "Action" to the front will break
Ack


http://gerrit.cloudera.org:8080/#/c/19417/6/www/queries.tmpl
File www/queries.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/6/www/queries.tmpl@32
PS6, Line 32:     <th>Action</th>
This column just has the "Action" string. Can we merge the "Query ID" column to this? i.e. replace the "Action" string with the actual query id.

A common scenario is searching the query id in this page and click the "Action" link. It'd be nice if we can click the query id directly.


http://gerrit.cloudera.org:8080/#/c/19417/6/www/queries.tmpl@43
PS6, Line 43:     <th>Bytes Sent</th>
We'd better add some tooltips to explain the columns:
https://www.w3schools.com/css/css_tooltip.asp

For instance, for "Bytes Sent", it's total bytes sent by all fragment instances. Without the tooltip, users might misunderstand it as total bytes sent to the client.

If this requires lot of changes, we can add the tooltips in a follow-on JIRA.



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 6
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Mon, 20 Feb 2023 08:48:14 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 1:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/12154/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 1
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 12 Jan 2023 12:32:55 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 2:

(8 comments)

Thanks for contributing to Impala! The new pages look good to me. I still need some time to review the correctness of new metrics. Post some comments first.

http://gerrit.cloudera.org:8080/#/c/19417/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19417/2//COMMIT_MSG@10
PS2, Line 10: can visually see
nit: replace with "show"


http://gerrit.cloudera.org:8080/#/c/19417/2//COMMIT_MSG@13
PS2, Line 13:   
nit: remove one space here


http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/scheduling/admission-controller.cc
File be/src/scheduling/admission-controller.cc:

http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/scheduling/admission-controller.cc@1297
PS2, Line 1297: int64_t& wait_start_time_ms, int64_t& wait_end_time_ms,
These are output parameters. In our code style, we try to use pointers instead of references for output parameters.

Could you also reorder the parameters so input parameters are in front of output parameters? i.e.

 Status AdmissionController::WaitOnQueued(const UniqueIdPB& query_id,
    int64_t timeout_ms, unique_ptr<QuerySchedulePB>* schedule_result,
    int64_t* wait_start_time_ms, int64_t* wait_end_time_ms,
    bool* wait_timed_out)

Impala C++ Style Guide: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65868536


http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/scheduling/admission-controller.cc@1300
PS2, Line 1300: 
Should we initialize 'wait_start_time_ms' and 'wait_end_time_ms' at the begining? Not sure if their values will be undefined in some quick return paths.


http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/service/client-request-state.h
File be/src/service/client-request-state.h:

http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/service/client-request-state.h@657
PS2, Line 657: Are 
nit: remove "Are"


http://gerrit.cloudera.org:8080/#/c/19417/2/common/protobuf/admission_control_service.proto
File common/protobuf/admission_control_service.proto:

http://gerrit.cloudera.org:8080/#/c/19417/2/common/protobuf/admission_control_service.proto@193
PS2, Line 193: microseconds
milliseconds? 1s = 1000 milliseconds. 1 milliseconds = 1000 milliseconds


http://gerrit.cloudera.org:8080/#/c/19417/2/www/queries.tmpl
File www/queries.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/2/www/queries.tmpl@31
PS2, Line 31:     <th>Details</th>
            :     <th>Action</th>
Not sure if moving "Details" and "Action" to the front will break any apps. Is there any motivations for moving?


http://gerrit.cloudera.org:8080/#/c/19417/2/www/query_detail_tabs.tmpl
File www/query_detail_tabs.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/2/www/query_detail_tabs.tmpl@99
PS2, Line 99:   record.rows[1].cells[4].textContent = json['record_json']['duration'];
It's hard to maintain the cell indexes if we want to reorder some in the future. Can we define some constants and use them instead? E.g.

 HEADER_ROW = 0;
 DATA_ROW = 1;
 DURATION_INDEX = 4;
 QUEUED_DURATION_INDEX = 5;
 ...
 ROWS_FETCHED_INDEX = 11;

and use them like

 record.rows[DATA_ROW].cells[DURATION_INDEX].textContent = json['record_json']['duration'];

These constances can also be used in refresh_record().



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 02 Feb 2023 12:06:29 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
18770832838@163.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/runtime/coordinator.cc
File be/src/runtime/coordinator.cc:

http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/runtime/coordinator.cc@1429
PS2, Line 1429: UpdatePeakMemUsageAndBytesRead
> nit: Maybe we can use the existing Coordinator::ComputeQueryResourceUtiliza
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 08 Feb 2023 08:15:05 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
18770832838@163.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.cc
File be/src/service/impala-server.cc:

http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.cc@823
PS8, Line 823:       // If the original profile was accessible by the user, then the retried profile
             :       // must be accessible by the user as well.
             :       status = CheckProfileAccess(user, retried_query_record->effective_user,
             :           retried_query_record->user_has_profile_access);
             :       DCHECK(status.ok());
             :       RETURN_IF_ERROR(status);
> Sorry, I mean we already invoke CheckProfileAccess() at line 818.
 > Do we need to call it again here?

Oops, sorry about that. That was indeed a mistake. I forgot to delete some needless code, I will fix it later.



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 8
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 08 Mar 2023 09:36:13 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Kurt Deschler, Yifan Zhang, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19417

to look at the new patch set (#10).

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................

IMPALA-11823: Add more items to Impala web UI queries page

When operating and maintaining an Impala cluster or analyzing historical
query performance, it will be helpful if we show the memory consumed,
the amount of data read, and other information of the query from the
historical query page of the web UI. The current historical query page
does not display these information, so we should count this information
when the query is executed and display it on the web page.

This patch modifies the query list page (/queries) and query detail
pages (/query_plan, etc.).
On the list page, some metrics are added for each query record,
including queuing time, memory usage, memory estimation, bytes read, and
bytes sent. In addition, the Details column now shows the query ID and
the position is adjusted to make them at the top of the record for easy
clicking.
On the query detail page, a similar record table is added to display the
key information of the current query. In addition, a timeline display is
added to the summary page (which is exactly the same as the timeline in
the profile, just for quick viewing). For queries that are running, the
above information will be automatically refreshed (only for the plan and
summary tabs).
To make it clear what each metric means, tooltips are added to all list
headers.

This patch also fixes an issue regarding invalid access to record.
Specifically, the original ImpalaServer::GetQueryRecord function cannot
guarantee that the returned query_record is always valid. After
returning, this value is not protected by query_log_lock_ and may become
an invalid iterator due to deleted entries in query_log_index_. This
patch uses shared_ptr instead of the original unique_ptr to extend the
lifetime of query_record after GetQueryRecord returns.

Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
---
M be/src/runtime/coordinator.h
M be/src/scheduling/admission-control-client.h
M be/src/scheduling/admission-control-service.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/local-admission-control-client.cc
M be/src/scheduling/local-admission-control-client.h
M be/src/scheduling/remote-admission-control-client.cc
M be/src/scheduling/remote-admission-control-client.h
M be/src/scheduling/schedule-state.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/admission_control_service.proto
M www/queries.tmpl
M www/query_detail_tabs.tmpl
M www/query_plan.tmpl
M www/query_summary.tmpl
23 files changed, 590 insertions(+), 166 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/19417/10
-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 10
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
18770832838@163.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 8:

(10 comments)

Thank you again for reviewing the code!

http://gerrit.cloudera.org:8080/#/c/19417/8//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19417/8//COMMIT_MSG@20
PS8, Line 20: the Action and query ID columns are merged
> nit: we don't have the "query ID" column before this patch. Maybe reword to
Done


http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:

http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-http-handler.cc@921
PS8, Line 921: found
> nit: "found" is ambiguous. Could you also rename this to "inflight"?
Done


http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.h
File be/src/service/impala-server.h:

http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.h@1223
PS8, Line 1223: The QueryStateRecord
              :   /// value is owned by 'query_log_' so the entry in this index must be removed when
              :   /// it is removed from 'query_log_'.
> Should we update this?
Done


http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.h@1226
PS8, Line 1226: std::shared_ptr<QueryStateRecord>*
> The value type is already a pointer. Can we use "std::shared_ptr<QueryStateRecord>"
 > directly instead of "std::shared_ptr<QueryStateRecord>*"?

It can indeed be replaced with std::shared_ptr, but I cannot think of a reason to do so. In my opinion, all that is needed here is a pointer to the content of query_log_ (whatever it may be) since it is essentially an index of id to query_log_ content and does not need to hold the content itself. It is always updated together with query_log_. If replaced with std::shared_ptr, it may slightly increase memory consumption, so I have not made any changes for now. However, please do point out if I have overlooked anything.


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h
File be/src/service/impala-server.h:

http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h@1220
PS6, Line 1220: shared_ptr
> Ah, right. This patch also fixes a concurrent issue. Please mention this in
Done


http://gerrit.cloudera.org:8080/#/c/19417/6/be/src/service/impala-server.h@1234
PS6, Line 1234: &
> Let's use pointer instead of reference for output arguments.
Done


http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.cc
File be/src/service/impala-server.cc:

http://gerrit.cloudera.org:8080/#/c/19417/8/be/src/service/impala-server.cc@823
PS8, Line 823:       // If the original profile was accessible by the user, then the retried profile
             :       // must be accessible by the user as well.
             :       status = CheckProfileAccess(user, retried_query_record->effective_user,
             :           retried_query_record->user_has_profile_access);
             :       DCHECK(status.ok());
             :       RETURN_IF_ERROR(status);
> Any reason we need to check this again?

Although I didn't write this line of code, I think I can explain it. This check failure doesn't seem to cause serious problems, so in the release version we just need to return and report an error. However, this is indeed an unusual situation, and in the debug version we may need to increase its severity to detect potential issues.


http://gerrit.cloudera.org:8080/#/c/19417/8/www/queries.tmpl
File www/queries.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/8/www/queries.tmpl@36
PS8, Line 36: the time submitted to Impala
> nit: "the time when the query was submitted"
Done


http://gerrit.cloudera.org:8080/#/c/19417/8/www/queries.tmpl@41
PS8, Line 41: the completed number of scans / the total number of scans
> nit: "the completed number of scan ranges / the total number of scan ranges
Done


http://gerrit.cloudera.org:8080/#/c/19417/8/www/queries.tmpl@95
PS8, Line 95:     <th title="The scan progress of the query, i.e. the completed number of scans / the total number of scans.">Scan Progress</th>
> Can we somehow deduplicate the description strings, e.g. defined as javascr
Done. As I couldn't find a solution using JavaScript (in fact, I don't know JavaScript), I solved it using C++ and mustache, just like the "waiting to be closed" tooltip. If you have a better solution, please let me know.



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 8
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Tue, 07 Mar 2023 09:45:04 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 11:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19417/11/www/queries.tmpl
File www/queries.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/11/www/queries.tmpl@160
PS11, Line 160:     <td>{{start_time}}</td>
              :     <td>{{end_time}}</td>
In my local machine, both start and end time shows 9 decimal points detail

2023-05-02 15:20:12.761465000

IMHO, 3 decimal point should be enough. User can open query profile to get more precision or get the json format of the query page instead.

2023-05-02 15:20:12.761



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 11
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 03 May 2023 00:31:37 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
18770832838@163.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 12:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19417/12/www/queries.tmpl
File www/queries.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/12/www/queries.tmpl@147
PS12, Line 147: {{duration}}<br/>{{queued_duration}}
> nit: For here, title row, and other similar column below: consider
 > add '/' before <br/> tag.
 > 
 > {{duration}} /<br/>{{queued_duration}}
 > 
 > The slash separator adds clarity that there are 2 values in one
 > cell, much like the current "Scan Progress" column.

Thanks for your suggestion. I also think it's necessary to add some extra symbols to indicate that there are two values in one cell. After trying several options, I found that using a separator line may be more clear. Therefore, I used <hr/> to achieve this.



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 12
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Sat, 06 May 2023 07:09:30 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
18770832838@163.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 11:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/runtime/coordinator.h
File be/src/runtime/coordinator.h:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/runtime/coordinator.h@216
PS11, Line 216:     /// Note that it is the sum of peaks, not the peak of the sum.
> This doesn't seem like a concise enough metric to present unless it
 > is from a very short interval or maybe per-fragment.

Sorry, I didn't fully understand what you meant. Are you suggesting that it should be the peak value of the sum instead of the sum of the peak values? If so, let me explain the reason for setting it up this way. The intention behind setting this value (total_peak_mem_usage) is to quickly identify queries that consume a significant amount of memory on the page, such as large data aggregations or JOIN operations. This value is also used to compare against the estimated memory usage to determine whether the query plan or memory estimate is reasonable. Since the estimated memory is the sum of the maximum memory usage that every backends can potentially use, the actual usage value for comparison should also be the sum of peaks. Therefore, this variable records the total peak memory usage for the query plan across all backends, which serves as a metric for measuring memory usage of the query.


http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/client-request-state.h
File be/src/service/client-request-state.h:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/client-request-state.h@361
PS11, Line 361:     // negative if a system clock reset happened after the query was initiated.
> Maybe better to return 0 if duration is negative?

This follows the same treatment as duration, see: https://github.com/apache/impala/blob/3cea0e9dbbf842b8eb3a377f881c9a116b97eed5/be/src/service/impala-http-handler.cc#L303-L307


http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:

http://gerrit.cloudera.org:8080/#/c/19417/11/be/src/service/impala-http-handler.cc@538
PS11, Line 538:   value->AddMember("mem_usage", mem_usage, document->GetAllocator());
> If we proceed with the current computation, we shouldn't call this
 > mem_usage.

Could you please share your reasoning for why we shouldn't use "mem_usage" as the metric name? We have added tooltips on the web page to further explain the meaning of this metric. If you have any other suggestions or ideas, please let me know.



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 11
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Thu, 13 Apr 2023 06:19:34 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Kurt Deschler, Yifan Zhang, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19417

to look at the new patch set (#6).

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................

IMPALA-11823: Add more items to Impala web UI queries page

When operating and maintaining an Impala cluster or analyzing historical
query performance, it will be helpful if we show the memory consumed,
the amount of data read, and other information of the query from the
historical query page of the web UI. The current historical query page
does not display these information, so we should count this information
when the query is executed and display it on the web page.

Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
---
M be/src/runtime/coordinator.h
M be/src/scheduling/admission-control-client.h
M be/src/scheduling/admission-control-service.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/local-admission-control-client.cc
M be/src/scheduling/local-admission-control-client.h
M be/src/scheduling/remote-admission-control-client.cc
M be/src/scheduling/remote-admission-control-client.h
M be/src/scheduling/schedule-state.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/admission_control_service.proto
M www/queries.tmpl
M www/query_detail_tabs.tmpl
M www/query_plan.tmpl
M www/query_summary.tmpl
23 files changed, 474 insertions(+), 95 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/19417/6
-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 6
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
18770832838@163.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 2:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/scheduling/admission-controller.cc
File be/src/scheduling/admission-controller.cc:

http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/scheduling/admission-controller.cc@1297
PS2, Line 1297: int64_t& wait_start_time_ms, int64_t& wait_end_time_ms,
> These are output parameters. In our code style, we try to use pointers inst
Done


http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/scheduling/admission-controller.cc@1300
PS2, Line 1300: 
> Should we initialize 'wait_start_time_ms' and 'wait_end_time_ms' at the beg
Done


http://gerrit.cloudera.org:8080/#/c/19417/2/www/query_detail_tabs.tmpl
File www/query_detail_tabs.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/2/www/query_detail_tabs.tmpl@99
PS2, Line 99:   record.rows[1].cells[4].textContent = json['record_json']['duration'];
> It's hard to maintain the cell indexes if we want to reorder some in the fu
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Tue, 07 Feb 2023 09:47:58 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/runtime/coordinator.cc
File be/src/runtime/coordinator.cc:

http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/runtime/coordinator.cc@1429
PS2, Line 1429: UpdatePeakMemUsageAndBytesRead
nit: Maybe we can use the existing Coordinator::ComputeQueryResourceUtilization to get the total resource usage of a query. Besides, do you think the total bytes_sent metric is also worth adding to the web page?



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Thu, 02 Feb 2023 14:17:34 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
18770832838@163.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 2:

(6 comments)

http://gerrit.cloudera.org:8080/#/c/19417/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19417/2//COMMIT_MSG@10
PS2, Line 10: can visually see
> nit: replace with "show"
Done


http://gerrit.cloudera.org:8080/#/c/19417/2//COMMIT_MSG@13
PS2, Line 13:   
> nit: remove one space here
Done


http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/runtime/coordinator.cc
File be/src/runtime/coordinator.cc:

http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/runtime/coordinator.cc@1429
PS2, Line 1429: UpdatePeakMemUsageAndBytesRead
> nit: Maybe we can use the existing Coordinator::ComputeQueryResourceUtilization
 > to get the total resource usage of a query. Besides, do you think
 > the total bytes_sent metric is also worth adding to the web page?

Thanks for the advice! I will try to put bytes_sent also to the page, and try to get the total resource usage of a query from ComputeQueryResourceUtilization.


http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/service/client-request-state.h
File be/src/service/client-request-state.h:

http://gerrit.cloudera.org:8080/#/c/19417/2/be/src/service/client-request-state.h@657
PS2, Line 657: Are 
> nit: remove "Are"
Done


http://gerrit.cloudera.org:8080/#/c/19417/2/common/protobuf/admission_control_service.proto
File common/protobuf/admission_control_service.proto:

http://gerrit.cloudera.org:8080/#/c/19417/2/common/protobuf/admission_control_service.proto@193
PS2, Line 193: microseconds
> milliseconds? 1s = 1000 milliseconds. 1 milliseconds = 1000 milliseconds
Done


http://gerrit.cloudera.org:8080/#/c/19417/2/www/queries.tmpl
File www/queries.tmpl:

http://gerrit.cloudera.org:8080/#/c/19417/2/www/queries.tmpl@31
PS2, Line 31:     <th>Details</th>
            :     <th>Action</th>
> Not sure if moving "Details" and "Action" to the front will break
 > any apps. Is there any motivations for moving?

Thanks for the code review!
There are some reason, in practice, we find it inconvenient to put "Details" and "Action" at the end of the list, because we often need to drag the browser slider to the right to click the button in the case of a long record.



-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Fri, 03 Feb 2023 12:33:52 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Kurt Deschler, Yifan Zhang, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19417

to look at the new patch set (#3).

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................

IMPALA-11823: Add more items to Impala web UI queries page

When operating and maintaining an Impala cluster or analyzing historical
query performance, it will be helpful if we show the memory consumed,
the amount of data read, and other information of the query from the
historical query page of the web UI. The current historical query page
does not display these information, so we should count this information
when the query is executed and display it on the web page.

Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
---
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
M be/src/scheduling/admission-control-client.h
M be/src/scheduling/admission-control-service.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/local-admission-control-client.cc
M be/src/scheduling/local-admission-control-client.h
M be/src/scheduling/remote-admission-control-client.cc
M be/src/scheduling/remote-admission-control-client.h
M be/src/scheduling/schedule-state.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/admission_control_service.proto
M www/queries.tmpl
M www/query_detail_tabs.tmpl
M www/query_plan.tmpl
M www/query_summary.tmpl
24 files changed, 427 insertions(+), 96 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/19417/3
-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>

[Impala-ASF-CR] IMPALA-11823: Add more items to Impala web UI queries page

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19417 )

Change subject: IMPALA-11823: Add more items to Impala web UI queries page
......................................................................


Patch Set 8:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12479/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19c75461a6405025fa433ae84d2c94d013fcaacb
Gerrit-Change-Number: 19417
Gerrit-PatchSet: 8
Gerrit-Owner: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Anonymous Coward <18...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Tue, 28 Feb 2023 09:01:15 +0000
Gerrit-HasComments: No