You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@impala.apache.org by "Anurag Mantripragada (Code Review)" <ge...@cloudera.org> on 2019/09/16 21:18:10 UTC

[Impala-ASF-CR] IMPALA-8761: Improve events processor configuration validation.

Anurag Mantripragada has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14240

Change subject: IMPALA-8761: Improve events processor configuration validation.
......................................................................

IMPALA-8761: Improve events processor configuration validation.

This patch aims to improve the validation of configuration keys from the HMS.

IMPALA-8559 introduced configuration validation for events processor configurations.
In the existing implementation, the events processor errors out as soon as it sees a
validation failure. If there are more than one configuration errors, the users may
have to restart HMS each time they fix an configuration error. This is bad user
experience. This change collects all the configuration issues and logs expected
values before erroring out. Users can now fix all issues in one go.

Testing:
Added testValidateConfigs() to assert if multiple incorrect values are detected at once.

Change-Id: I73480872ef93215d05c1fd922e64eb68a8a63a42
---
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java
2 files changed, 43 insertions(+), 3 deletions(-)

git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/14240/1
--
To view, visit http://gerrit.cloudera.org:8080/14240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I73480872ef93215d05c1fd922e64eb68a8a63a42
Gerrit-Change-Number: 14240
Gerrit-PatchSet: 1
Gerrit-Owner: Anurag Mantripragada <an...@cloudera.com>

[Impala-ASF-CR] IMPALA-8761: Improve events processor configuration validation.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14240 )

Change subject: IMPALA-8761: Improve events processor configuration validation.
......................................................................


Patch Set 3: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/14240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I73480872ef93215d05c1fd922e64eb68a8a63a42
Gerrit-Change-Number: 14240
Gerrit-PatchSet: 3
Gerrit-Owner: Anurag Mantripragada <an...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Sep 2019 01:49:05 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8761: Improve events processor configuration validation.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14240 )

Change subject: IMPALA-8761: Improve events processor configuration validation.
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/4569/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/14240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I73480872ef93215d05c1fd922e64eb68a8a63a42
Gerrit-Change-Number: 14240
Gerrit-PatchSet: 2
Gerrit-Owner: Anurag Mantripragada <an...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Mon, 16 Sep 2019 22:01:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8761: Improve events processor configuration validation.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14240 )

Change subject: IMPALA-8761: Improve events processor configuration validation.
......................................................................


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4960/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/14240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I73480872ef93215d05c1fd922e64eb68a8a63a42
Gerrit-Change-Number: 14240
Gerrit-PatchSet: 3
Gerrit-Owner: Anurag Mantripragada <an...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 18 Sep 2019 21:36:36 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8761: Improve events processor configuration validation.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14240 )

Change subject: IMPALA-8761: Improve events processor configuration validation.
......................................................................

IMPALA-8761: Improve events processor configuration validation.

This patch aims to improve the validation of configuration keys
from the HMS.

IMPALA-8559 introduced configuration validation for events processor
configurations. In the existing implementation, the events processor
errors out as soon as it sees a validation failure. If there are
more than one configuration errors, the users may have to restart
HMS each time they fix a configuration error. This is bad user
experience. This change collects all the configuration issues and
logs expected values before erroring out. Users can now fix all
issues in one go.

Testing:
Added testValidateConfigs() to assert if multiple incorrect values
are detected at once.

Change-Id: I73480872ef93215d05c1fd922e64eb68a8a63a42
Reviewed-on: http://gerrit.cloudera.org:8080/14240
Reviewed-by: Bharath Vissapragada <bh...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java
2 files changed, 43 insertions(+), 5 deletions(-)

Approvals:
Bharath Vissapragada: Looks good to me, approved
Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/14240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I73480872ef93215d05c1fd922e64eb68a8a63a42
Gerrit-Change-Number: 14240
Gerrit-PatchSet: 4
Gerrit-Owner: Anurag Mantripragada <an...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>

[Impala-ASF-CR] IMPALA-8761: Improve events processor configuration validation.

Posted by "Anurag Mantripragada (Code Review)" <ge...@cloudera.org>.

Anurag Mantripragada has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/14240 )

Change subject: IMPALA-8761: Improve events processor configuration validation.
......................................................................

IMPALA-8761: Improve events processor configuration validation.

This patch aims to improve the validation of configuration keys
from the HMS.

IMPALA-8559 introduced configuration validation for events processor
configurations. In the existing implementation, the events processor
errors out as soon as it sees a validation failure. If there are
more than one configuration errors, the users may have to restart
HMS each time they fix a configuration error. This is bad user
experience. This change collects all the configuration issues and
logs expected values before erroring out. Users can now fix all
issues in one go.

Testing:
Added testValidateConfigs() to assert if multiple incorrect values
are detected at once.

git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/14240/3
--
To view, visit http://gerrit.cloudera.org:8080/14240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I73480872ef93215d05c1fd922e64eb68a8a63a42
Gerrit-Change-Number: 14240
Gerrit-PatchSet: 3
Gerrit-Owner: Anurag Mantripragada <an...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>

[Impala-ASF-CR] IMPALA-8761: Improve events processor configuration validation.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14240 )

Change subject: IMPALA-8761: Improve events processor configuration validation.
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/4592/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/14240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I73480872ef93215d05c1fd922e64eb68a8a63a42
Gerrit-Change-Number: 14240
Gerrit-PatchSet: 3
Gerrit-Owner: Anurag Mantripragada <an...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 18 Sep 2019 22:08:21 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8761: Improve events processor configuration validation.

Posted by "Anurag Mantripragada (Code Review)" <ge...@cloudera.org>.

Anurag Mantripragada has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/14240 )

Change subject: IMPALA-8761: Improve events processor configuration validation.
......................................................................

IMPALA-8761: Improve events processor configuration validation.

This patch aims to improve the validation of configuration keys from the HMS.

IMPALA-8559 introduced configuration validation for events processor
configurations. In the existing implementation, the events processor errors
out as soon as it sees a validation failure. If there are more than one
configuration errors, the users may have to restart HMS each time they fix
a configuration error. This is bad user experience. This change collects all
the configuration issues and logs expected values before erroring out.
Users can now fix all issues in one go.

Testing:
Added testValidateConfigs() to assert if multiple incorrect values are
detected at once.

git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/14240/2
--
To view, visit http://gerrit.cloudera.org:8080/14240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I73480872ef93215d05c1fd922e64eb68a8a63a42
Gerrit-Change-Number: 14240
Gerrit-PatchSet: 2
Gerrit-Owner: Anurag Mantripragada <an...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>

[Impala-ASF-CR] IMPALA-8761: Improve events processor configuration validation.

Posted by "Bharath Vissapragada (Code Review)" <ge...@cloudera.org>.

Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/14240 )

Change subject: IMPALA-8761: Improve events processor configuration validation.
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/14240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I73480872ef93215d05c1fd922e64eb68a8a63a42
Gerrit-Change-Number: 14240
Gerrit-PatchSet: 3
Gerrit-Owner: Anurag Mantripragada <an...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 18 Sep 2019 21:36:26 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8761: Improve events processor configuration validation.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14240 )

Change subject: IMPALA-8761: Improve events processor configuration validation.
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/4568/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/14240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I73480872ef93215d05c1fd922e64eb68a8a63a42
Gerrit-Change-Number: 14240
Gerrit-PatchSet: 1
Gerrit-Owner: Anurag Mantripragada <an...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Mon, 16 Sep 2019 21:58:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8761: Improve events processor configuration validation.

Posted by "Bharath Vissapragada (Code Review)" <ge...@cloudera.org>.

Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/14240 )

Change subject: IMPALA-8761: Improve events processor configuration validation.
......................................................................


Patch Set 2: Code-Review+2

(4 comments)

http://gerrit.cloudera.org:8080/#/c/14240/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/14240/2//COMMIT_MSG@9
PS2, Line 9: This patch aims to improve the validation of configuration keys from the HMS.
nit: line wrap


http://gerrit.cloudera.org:8080/#/c/14240/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:

http://gerrit.cloudera.org:8080/#/c/14240/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@267
PS2, Line 267: failedValidationResults
nit: validationErrors?


http://gerrit.cloudera.org:8080/#/c/14240/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@273
PS2, Line 273:         if (!result.isValid()) {
nit: single line.


http://gerrit.cloudera.org:8080/#/c/14240/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@282
PS2, Line 282: failedValidationResults.size() > 0
nit: !validationErrors.isEmpty() (more concise)



-- 
To view, visit http://gerrit.cloudera.org:8080/14240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I73480872ef93215d05c1fd922e64eb68a8a63a42
Gerrit-Change-Number: 14240
Gerrit-PatchSet: 2
Gerrit-Owner: Anurag Mantripragada <an...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 18 Sep 2019 18:50:25 +0000
Gerrit-HasComments: Yes