You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org> on 2022/04/05 15:02:18 UTC

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18385


Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................

IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Before this patch the message was added to the profile in Open(),
which can be called multiple times in subplans.

Moved it to Close(), which is only called once in the lifetime
of a Node/Aggregator. This can be also useful if async_codegen=1,
as the codegen may not be finished yet during Open().

A drawback of this is that this info won't be visible when the
Node is still active, but I don't think that it is a very useful
info in a still running query.

Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
---
M be/src/exec/aggregator.cc
M be/src/exec/exec-node.cc
2 files changed, 7 insertions(+), 4 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/18385/1
-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 8:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8043/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 8
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Apr 2022 08:13:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 7: Code-Review+2

Thanks for handling this, Csaba!


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 7
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Apr 2022 08:03:56 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 1:

> Hi Csaba, thanks for picking this up!
 > We hit similar issue in the past at IMPALA-9851, where the fix is
 > to truncate the message to 128kb.
 > Should we defensively add similar truncation/check in
 > RuntimeProfileBase::AddInfoStringInternal?

Hi Riza! Capping messages to some sane limit makes sense, though I am not sure that it should go to the same commit to make this easy to backport.


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 06 Apr 2022 19:20:23 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 1: Code-Review+1

Thanks for fixing this quickly!

Can we add a test using the query in the JIRA description?


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 07 Apr 2022 02:01:42 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 1:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/10399/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 Apr 2022 18:03:46 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/10408/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 06 Apr 2022 19:38:26 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 6:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/10435/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 6
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Apr 2022 08:20:24 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 2:

(1 comment)

Added a regression test, which turned out to be a bit more complex than I thought, as I found no easy way to check that a regexp does NOT match any line in the profile.

http://gerrit.cloudera.org:8080/#/c/18385/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/18385/1//COMMIT_MSG@13
PS1, Line 13: 
            : 
I have realized that this is not true, we don't report codegen errors here in case of async codegen.



-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 11 Apr 2022 13:52:52 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18385/6/tests/common/test_result_verifier.py
File tests/common/test_result_verifier.py:

http://gerrit.cloudera.org:8080/#/c/18385/6/tests/common/test_result_verifier.py@647
PS6, Line 647: ,
flake8: E501 line too long (100 > 90 characters)



-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 6
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Apr 2022 08:00:54 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 5: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18385/5/tests/common/test_result_verifier.py
File tests/common/test_result_verifier.py:

http://gerrit.cloudera.org:8080/#/c/18385/5/tests/common/test_result_verifier.py@647
PS5, Line 647:       "runtime profile:" + '\n'.join(unexpected_matched_lines))
Can we print the actual profile like above?



-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 5
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Apr 2022 07:37:01 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 4:

Ps 4 fixed the test broken by changing the order of exec option in the profile.


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 12 Apr 2022 07:53:40 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 1: Code-Review+1

(1 comment)

> Hi Riza! Capping messages to some sane limit makes sense, though I am not sure that it should go to the same commit to make this easy to backport.

Thank you, Csaba. That make sense. Let's track that in a separate patch.

http://gerrit.cloudera.org:8080/#/c/18385/1/be/src/exec/exec-node.cc
File be/src/exec/exec-node.cc:

http://gerrit.cloudera.org:8080/#/c/18385/1/be/src/exec/exec-node.cc@331
PS1, Line 331: codegen_status_msgs_
We might want to cap the size of this array too next time, to avoid blowing up ExecOption string.



-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 06 Apr 2022 23:14:58 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 8: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 8
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Apr 2022 12:31:35 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 1:

Hi Csaba, thanks for picking this up!
We hit similar issue in the past at IMPALA-9851, where the fix is to truncate the message to 128kb.
Should we defensively add similar truncation/check in RuntimeProfileBase::AddInfoStringInternal?


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 Apr 2022 16:54:35 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Riza Suminto, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18385

to look at the new patch set (#2).

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................

IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Before this patch the message was added to the profile in Open(),
which can be called multiple times in subplans.

Moved it to Close(), which is only called once in the lifetime
of a Node/Aggregator.

A drawback of this is that this info won't be visible when the
Node is still active, but I don't think that it is a very useful
info in a still running query.

Also added a new feature to test_result_verifier.py:
Inside RUNTIME_PROFILE section row_regex can be negated with !,
so !row_regex [regex] means that regex is not matched by any line
in the profile.

Testing:
- added a regression test

Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
---
M be/src/exec/aggregator.cc
M be/src/exec/exec-node.cc
M testdata/workloads/functional-query/queries/QueryTest/nested-types-subplan-single-node.test
M tests/common/test_result_verifier.py
4 files changed, 40 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/18385/2
-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Riza Suminto, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18385

to look at the new patch set (#7).

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................

IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Before this patch the message was added to the profile in Open(),
which can be called multiple times in subplans.

Moved it to Close(), which is only called once in the lifetime
of a Node/Aggregator.

A drawback of this is that this info won't be visible when the
Node is still active, but I don't think that it is a very useful
info in a still running query.

Also added a new feature to test_result_verifier.py:
Inside RUNTIME_PROFILE section row_regex can be negated with !,
so !row_regex [regex] means that regex is not matched by any line
in the profile.

Testing:
- added a regression test

Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
---
M be/src/exec/aggregator.cc
M be/src/exec/exec-node.cc
M testdata/workloads/functional-query/queries/QueryTest/nested-types-subplan-single-node.test
M testdata/workloads/functional-query/queries/QueryTest/union-const-scalar-expr-codegen.test
M tests/common/test_result_verifier.py
5 files changed, 44 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/18385/7
-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 7
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 7:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/10436/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 7
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Apr 2022 08:22:49 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................

IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Before this patch the message was added to the profile in Open(),
which can be called multiple times in subplans.

Moved it to Close(), which is only called once in the lifetime
of a Node/Aggregator.

A drawback of this is that this info won't be visible when the
Node is still active, but I don't think that it is a very useful
info in a still running query.

Also added a new feature to test_result_verifier.py:
Inside RUNTIME_PROFILE section row_regex can be negated with !,
so !row_regex [regex] means that regex is not matched by any line
in the profile.

Testing:
- added a regression test

Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Reviewed-on: http://gerrit.cloudera.org:8080/18385
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/exec/aggregator.cc
M be/src/exec/exec-node.cc
M testdata/workloads/functional-query/queries/QueryTest/nested-types-subplan-single-node.test
M testdata/workloads/functional-query/queries/QueryTest/union-const-scalar-expr-codegen.test
M tests/common/test_result_verifier.py
5 files changed, 44 insertions(+), 9 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 9
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 5:

The 2 failed tests look unrelated (caused by JCM OOM)
https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5555/


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 5
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 12 Apr 2022 20:57:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 5: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8037/


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 5
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 12 Apr 2022 12:20:17 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8035/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 11 Apr 2022 13:53:56 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 3: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8035/


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 11 Apr 2022 18:19:36 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/10423/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 11 Apr 2022 14:09:33 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/10428/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 12 Apr 2022 08:12:22 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 8: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 8
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Apr 2022 08:13:27 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Riza Suminto, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18385

to look at the new patch set (#4).

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................

IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Before this patch the message was added to the profile in Open(),
which can be called multiple times in subplans.

Moved it to Close(), which is only called once in the lifetime
of a Node/Aggregator.

A drawback of this is that this info won't be visible when the
Node is still active, but I don't think that it is a very useful
info in a still running query.

Also added a new feature to test_result_verifier.py:
Inside RUNTIME_PROFILE section row_regex can be negated with !,
so !row_regex [regex] means that regex is not matched by any line
in the profile.

Testing:
- added a regression test

Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
---
M be/src/exec/aggregator.cc
M be/src/exec/exec-node.cc
M testdata/workloads/functional-query/queries/QueryTest/nested-types-subplan-single-node.test
M testdata/workloads/functional-query/queries/QueryTest/union-const-scalar-expr-codegen.test
M tests/common/test_result_verifier.py
5 files changed, 43 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/18385/4
-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18385 )

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8037/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 5
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 12 Apr 2022 07:54:44 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Riza Suminto, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18385

to look at the new patch set (#6).

Change subject: IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile
......................................................................

IMPALA-11200: Avoid redundant "Codegen enabled" messages in profile

Before this patch the message was added to the profile in Open(),
which can be called multiple times in subplans.

Moved it to Close(), which is only called once in the lifetime
of a Node/Aggregator.

A drawback of this is that this info won't be visible when the
Node is still active, but I don't think that it is a very useful
info in a still running query.

Also added a new feature to test_result_verifier.py:
Inside RUNTIME_PROFILE section row_regex can be negated with !,
so !row_regex [regex] means that regex is not matched by any line
in the profile.

Testing:
- added a regression test

Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
---
M be/src/exec/aggregator.cc
M be/src/exec/exec-node.cc
M testdata/workloads/functional-query/queries/QueryTest/nested-types-subplan-single-node.test
M testdata/workloads/functional-query/queries/QueryTest/union-const-scalar-expr-codegen.test
M tests/common/test_result_verifier.py
5 files changed, 43 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/18385/6
-- 
To view, visit http://gerrit.cloudera.org:8080/18385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iad2e31900ee6d29385cc8adc6bbf067d91f6450f
Gerrit-Change-Number: 18385
Gerrit-PatchSet: 6
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>